The primary motivation for this research stems from the operational challenges hindering the seamless integration of Artificial Intelligence (AI) into clinical workflows. Despite the remarkable potential of Deep Learning (DL) models, their real-world application is frequently obstructed by "domain shift", a phenomenon where model performance degrades due to variations in ultrasound hardware, vendor-specific imaging protocols, and patient populations. In our previous research (1, 6), we proposed a multi-task learning framework based on Unsupervised Domain Adaptation (UDA) and demonstrated its superior capability in overcoming these technical hurdles, ensuring high-precision segmentation across diverse datasets. Nevertheless, even the most accurate algorithms may fail to gain clinical traction if they operate as "black-box" systems lacking an intuitive, interactive interface for physician oversight.
The main aim of this study is to present the development and clinical validation of EchoAI, a comprehensive web-based platform designed to bridge the gap between advanced AI algorithms and bedside diagnostics. Built upon our previously validated UDA engine, EchoAI automates the quantification of cardiac function and structure while prioritizing a "human-in-the-loop" philosophy. By providing a user-centered interface that allows for real-time visualization and editing of AI-generated masks, the platform fosters clinical trust and ensures that the final diagnostic report remains under the physician's expert control. This research highlights how a robust, vendor-agnostic web ecosystem can transform complex computational models into practical Clinical Decision Support Systems (CDSS) for modern cardiology departments.
Protocol
2.1 Study Design and Setting
This research was conducted as a cross-sectional observational study (7) at the Department of Cardiology, Guilan University of Medical Sciences (GUMS). The evaluation process involved a purposive sample of 12 domain experts to ensure the system was validated across different levels of clinical expertise. The study was stratified into two distinct groups: Group A, consisting of six board-certified cardiologists with over ten years of professional experience, and Group B, comprising six senior cardiology residents. This diverse group of participants provided a comprehensive basis for assessing the platform's reliability and usability in a real-world clinical environment.
2.2 System Architecture and AI Engine
The EchoAI platform is architected as a modular, three-tier web application (8) designed to ensure scalability, maintainability, and cross-platform compatibility. As illustrated in Figure 1, the high-level architecture organizes the system into three distinct vertical sections: the Presentation Layer (Client/Browser), the Logic Layer (Django Web Server), and the AI Inference Engine.
At the client side, a clean web-based interface displays echocardiography videos across standard views such as A4C, A2C, and PLAX. The data flow initiates with the upload of DICOM files from the client to the web server. The logic tier at the center utilizes the Model-View-Template (MVT) pattern via the Django framework, which strictly separates data access logic from user interaction (9). Heavy computational tasks, such as video processing by the AI engine, are handled asynchronously via Celery to prevent the user interface from freezing. The web server also interacts with a PostgreSQL database for structured metadata and S3 file storage for managing raw medical imagery.
The core of this system resides within the AI Inference Engine, which utilizes a specialized deep learning framework based on Unsupervised Domain Adaptation (UDA) (1, 6). This engine, previously developed to learn domain-invariant features, automatically classifies and processes multiple acoustic windows. To guarantee a fluid, near-real-time clinical experience, the PyTorch models were optimized using NVIDIA TensorRT, significantly reducing inference latency to approximately 42 ms per frame.
Finally, the processing results, including segmentation masks and quantitative metrics, are returned to the web server as JSON data. From there, they are sent back to the client for real-time physician verification and the generation of the final PDF clinical report.
Figure 1. Data-flow centered architecture of the EchoAI platform, highlighting the interaction between the Django web server and the optimized AI inference engine (Prepared by Authors, 2025).
2.3 Experimental Procedures and Human-in-the-Loop Interaction
Evaluation sessions were conducted individually in a controlled environment designed to simulate a standard echocardiography reading room. The procedure commenced with a ten-minute onboarding tutorial to familiarize participants with the multi-view analysis and the "human-in-the-loop" verification mechanism. Following this, each participant was tasked with performing a complete diagnostic workflow on a curated dataset of five anonymized patient cases.
As detailed in the Activity Infographic (Figure 2), the workflow is structured around a "Human-centric" conditional logic. Rather than operating as an autonomous black box, the system treats AI-generated outputs as preliminary suggestions that must pass through a mandatory physician validation gateway. This interaction allows the physician to remain in full command of the diagnostic process; if inaccuracies are detected, the system transitions into a "Manual Edit Mode," enabling the expert to adjust segmentation masks before finalizing the clinical report.
The technical fluidity and responsiveness of this interaction are achieved through the communication protocol visualized in the Sequence Infographic (Figure 3). By utilizing asynchronous AJAX requests, the system offloads heavy UDA computations to the backend inference engine without freezing the user interface. This architecture ensures a non-blocking experience with near real-time feedback (latency < 42ms), allowing the physician to review and verify masks seamlessly as part of their natural clinical routine.
Figure 2. EchoAI Decision Workflow and Human-in-the-Loop Integration (Prepared by Authors, 2025).
Figure 3. Asynchronous Real-time Communication Architecture (Prepared by Authors, 2025).
2.4 Statistical Analysis
The quantitative reliability and clinical usability of the EchoAI platform were assessed using a multi-faceted statistical approach to ensure both technical accuracy and user satisfaction. To determine the strength of association between the automated system's outputs and the expert manual ground truth, Pearson Correlation (r) was calculated for functional and structural metrics. To further evaluate the clinical interchangeability of the measurements and identify any potential systematic bias, a Bland-Altman analysis was performed, providing a rigorous assessment of the agreement between AI-generated data and human expertise.
To quantify the level of clinical trust and diagnostic reliability, Cohen’s Kappa coefficient (κ) was utilized to measure the inter-rater agreement for the binary classification of pathologies, such as identifying Normal versus Abnormal cardiac states. Subjective user experience and interface effectiveness were measured using the standardized Computer System Usability Questionnaire (CSUQ) (10) on a 7-point Likert scale, focusing on three core dimensions: System Quality, Information Quality, and Interface Quality. For all statistical evaluations, including correlation, agreement, and usability metrics, the level of statistical significance was strictly defined as P<0.001.
Results and Discussion
3.1 System Implementation and Computational Performance
The EchoAI platform was successfully deployed as a "zero-footprint" web application within the secure intranet of Guilan University of Medical Sciences (GUMS), achieving full operational stability without requiring client-side installation. A critical finding of this study was the effectiveness of NVIDIA TensorRT optimization, which reduced the average inference latency to 42 ms per frame. Consequently, the total processing time for a complete cardiac cycle, encompassing view classification, multi-view segmentation, and volumetric calculation, averaged 1.15 seconds. This sub-second latency was perceived as near-instantaneous by clinicians, facilitating a fluid diagnostic workflow that does not disrupt the standard clinical routine. By utilizing a vendor-agnostic architecture, EchoAI democratizes access to high-end quantification tools, particularly in resource-constrained settings where proprietary workstations are unavailable.
3.2 Clinical Validation and Diagnostic Accuracy
The integrated UDA engine demonstrated exceptional concordance with expert manual measurements across all functional and structural parameters. As illustrated in the Correlation Analysis (Figure 4A), the automated calculation of LVEF showed a robust linear relationship with the expert ground truth (r=0.95, P<0.001), with a coefficient of determination (R²) of 0.90. Furthermore, the Bland-Altman analysis (Figure 4B) confirmed high reliability with a negligible mean bias of -0.17%, indicating that the system does not systematically over- or underestimate cardiac function. The narrow limits of agreement (-8.77% to +8.42%) suggest that EchoAI is clinically interchangeable with manual reporting across a wide spectrum of cardiac states, from severe systolic dysfunction to hyperdynamic conditions. Structural metrics derived from the PLAX view, including IVS and PW thickness, also maintained sub-millimeter precision with Mean Absolute Errors (MAE) of 0.08 cm and 0.09 cm, respectively. More details are available in Table 1.
Table 1. Comparison of quantitative echocardiographic parameters derived from the automated EchoAI system versus expert manual measurements (Ground Truth) across the validation dataset.
| Clinical Parameter |
Manual Reference (Mean ± SD) |
Automated System (Mean ± SD) |
Mean Absolute Error (MAE) |
Pearson Correlation (r) |
P-value |
| Functional Metrics |
| LVEF (%) |
52.4 ± 12.1 |
51.8 ± 11.9 |
4.10% |
0.94 |
< 0.001 |
| LVEDV (ml) |
115.3 ± 38.2 |
112.5 ± 36.8 |
12.3 ml |
0.92 |
< 0.001 |
| LVESV (ml) |
58.6 ± 29.5 |
56.9 ± 28.1 |
8.7 ml |
0.93 |
< 0.001 |
| Structural Metrics |
| IVS Thickness (cm) |
1.08 ± 0.24 |
1.05 ± 0.22 |
0.08 cm |
0.88 |
< 0.001 |
| PW Thickness (cm) |
1.02 ± 0.19 |
0.99 ± 0.18 |
0.09 cm |
0.85 |
< 0.001 |
LVEF: Left Ventricular Ejection Fraction; LVEDV: Left Ventricular End-Diastolic Volume; LVESV: Left Ventricular End-Systolic Volume; IVS: Interventricular Septum; PW: Posterior Wall. Note: P-values indicate the statistical significance of the correlation coefficient.
Figure 4. EchoAI Decision Workflow and Human-in-the-Loop Integration (Prepared by Authors, 2025).
3.3 Usability Assessment and Clinical Trust
The subjective evaluation of the platform validated the efficacy of our User-Centered Design (UCD) approach. The overall system usability, measured via the CSUQ, achieved a global mean score of 6.18 ± 0.45, significantly exceeding the benchmark for high satisfaction in medical software. As shown in Table 2, "Interface Quality" received the highest ratings (6.42), corroborating the success of the "Medical Dark Mode" in minimizing visual fatigue. Interestingly, the stratified analysis revealed that trainees (Group B) reported higher "Information Quality" scores than experts (6.50 vs. 6.08), suggesting that the automated visual overlays serve as a critical cognitive aid for less experienced clinicians.
The reliability of EchoAI as a Clinical Decision Support System (CDSS) was further evidenced by a Cohen’s Kappa (κ) of 0.88, representing "almost perfect" agreement in pathology classification. Crucially, the "Human-in-the-loop" efficiency analysis (Figure 5) showed that cardiologists accepted 86% of the AI-generated segmentation masks without any modification. This high acceptance rate, combined with the ability to perform minor localized corrections in the remaining 14% of cases, effectively reduces the manual workload while fostering the professional trust necessary for routine clinical adoption. Although limited by a single-center design and 2D focus, EchoAI represents a paradigm shift from algorithm-centric to user-centric medical AI.
Table 2. Detailed CSUQ Usability Scores.
| CSUQ Sub-Scale |
Group A: Experts (Mean ± SD) |
Group B: Trainees (Mean ± SD) |
Overall (Mean ± SD) |
| System Quality (SysQual) |
6.02 ± 0.51 |
6.08 ± 0.48 |
6.05 ± 0.49 |
| Information Quality (InfoQual) |
6.08 ± 0.62 |
6.50 ± 0.41 |
6.29 ± 0.55 |
| Interface Quality (IntQual) |
6.38 ± 0.35 |
6.46 ± 0.38 |
6.42 ± 0.36 |
| Overall Satisfaction |
6.10 ± 0.55 |
6.26 ± 0.35 |
6.18 ± 0.45 |
Figure 5. EchoAI Decision Workflow and Human-in-the-Loop Integration (Prepared by Authors, 2025).
3.4 Limitations
Despite the promising technical and clinical outcomes, this study has several limitations that must be acknowledged. First, the clinical usability and workflow validation were conducted at a single tertiary care center. While the Unsupervised Domain Adaptation (UDA) engine successfully addressed vendor-specific domain shifts, the subjective acceptance of the CDSS may vary across different hospital workflows and healthcare institutions; thus, multi-center prospective trials are warranted. Second, the "human-in-the-loop" interaction was evaluated using a relatively small sample size, involving 12 clinicians and a curated set of 5 patient cases. Future studies must scale this evaluation to encompass a larger cohort of users and a broader, more complex range of pathologies in real-time clinical routines. Third, the current iteration of the EchoAI platform is restricted to standard 2D echocardiographic views (A4C, A2C, and PLAX). It does not yet incorporate 3D echocardiography or advanced speckle-tracking strain imaging, which are increasingly critical for comprehensive cardiac assessment. Finally, akin to all ultrasound-based AI models, the system's segmentation accuracy remains inherently dependent on the quality of the initial acoustic windows. In patients with poor echogenicity—such as those with severe obesity or chronic obstructive pulmonary disease (COPD)—the automated masks may exhibit reduced precision, thereby necessitating a higher rate of manual physician intervention.
Acknowledgment
The authors wish to express their profound gratitude to the Department of Cardiology and the Cardiovascular Diseases Research Center at Dr. Heshmat Hospital, Guilan University of Medical Sciences (GUMS), for their essential collaboration. We are especially thankful to the cardiologists and residents who volunteered their expertise and time to evaluate the platform. Their professional insights and constructive feedback were vital in optimizing the user interface and significantly refining the clinical workflow of the EchoAI system.
Ethical Considerations
The project was found to be in accordance with the ethical principles and national norms and standards for conducting Medical Research in Iran (Approval ID: IR.GUMS.REC.1404.472).
Conflicts of Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Financial Support and Sponsorship
This research received no specific grant from any funding agency in the public sector.
