Highlights
-
•
POCCUS performed by novice operators was feasible, rapid, and provided sufficient image quality for accurate interpretation in a high-volume outpatient setting.
-
•
A 2-tiered approach, with AI-ECG first followed by POCCUS for AI-ECG-positive patients, improved positive predictive value from 32% to 64% and overall accuracy from 67% to 88%, while maintaining high negative predictive value (93%).
-
•
This workflow offers a practical, scalable method for early detection of structural heart disease and opens avenues for future pragmatic, prospective screening studies.
ABSTRACT
Background
Early structural heart disease (SHD) detection is crucial for improving prognostic outcomes, but widely accessible screening methods are lacking. The advent of artificial intelligence-enabled electrocardiograms (AI-ECG) and point-of-care cardiac-ultrasonography (POCCUS) offers promising new approaches for patient screening. We explored the feasibility and potential of integrating these innovative technologies into a practical SHD screening framework.
Methods
Outpatients undergoing ECG at the Mayo Clinic ECG laboratory between November 2023 and February 2024 were pragmatically offered POCCUS, performed by novice operators and reviewed by expert echocardiographers. AI-ECG and POCCUS findings were integrated to assess SHD, including reduced left ventricular systolic function (ejection-fraction<50%), aortic stenosis, and increased left ventricular wall thickness suggestive of cardiac amyloidosis or hypertrophic cardiomyopathy. Operators were blinded to patients’ comorbidities and formal echocardiogram results.
Results
Of 486 patients (median-age 64 years; 49% women), 286 had available formal echocardiography, with 17.5% having SHD. AI-ECG had a 32% positive predictive value (PPV) and a 94% negative predictive value (NPV) to detect any SHD. Adding POCCUS increased the overall PPV to 64% with an NPV of 93%, with an increase in diagnostic accuracy from 67% to 88%. Notably, 89.5% (17/19) of the “false positives” by AI-ECG + POCCUS had less-than-moderate-SHD. Applying the AI-ECG + POCCUS screening workflow on the entire cohort resulted in a number-needed-to-screen of 8 to identify 1 patient requiring formal echocardiography.
Conclusions
The integration of AI-ECG and POCCUS holds promise as a potentially effective screening method for SHD, facilitating improved patient selection for formal echocardiography.
Graphical abstract
Detecting Structural Heart Disease with AI-ECG and POCCUS integration.
Background
Structural heart diseases (SHD), including left ventricular systolic dysfunction (LVSD), aortic stenosis (AS), cardiac amyloidosis (CA), and hypertrophic cardiomyopathy (HCM), pose significant risks of heart failure and mortality, often evading detection until advanced stages or fatal outcomes manifest. ,,, Advances in medicine have introduced treatments that alter the course of SHD, emphasizing the critical need for early diagnosis and proactive clinical and imaging follow-up to identify candidates for beneficial interventions. , Thus, standardized, easy-to-implement, and cost-effective screening measures for SHD are needed.
Artificial intelligence-enabled electrocardiogram (AI-ECG) algorithms have proven effective in detecting SHD in the general population, boasting high accuracy with areas under the curve ranging from 0.85 to 0.96 and an impressive negative predictive value (NPV) nearing 99%. ,,, However, their low positive predictive value (PPV; 11%-34%) limits their utility as standalone screening tools, potentially leading to frequent unnecessary and costly transthoracic echocardiograms (TTEs). This issue is compounded by the overuse of expensive diagnostics and the constrained capacity of echocardiography laboratories to manage screening TTEs for patients flagged by positive AI-ECG results, necessitating an innovative approach.
Recent advances in medical technology have facilitated the development of compact, high-quality portable ultrasound devices. The emergence of small, handheld ultrasound machines capable of acquiring high-resolution 2D echocardiography images has transformed point-of-care assessment. Portable, cost-effective, and increasingly deployed across various clinical settings, point-of-care cardiac-ultrasonography (POCCUS) enables straightforward qualitative evaluation of cardiac function and structure through simple 2-dimensional imaging.
Considering these developments, integrating POCCUS screening with current AI-ECG models presents a promising and underexplored opportunity to identify patients with SHD. Outpatient ECG labs, bustling with activity, are ideal settings where AI-driven ECG analyses are routinely conducted. This environment offers a practical, real-world context for combining ECG and POCCUS for screening purposes. The current study aims to assess the feasibility and predictive implications of integrating AI-ECG and POCCUS screening for SHD within a diverse outpatient population.
Methods
Study design
We conducted a proof-of-concept quality-of-care project at the ECG Laboratory of Mayo Clinic in Rochester, Minnesota, USA. This analysis aimed to assess the feasibility and diagnostic yield of a unique screening framework employing a 2-layer screening process integrating AI-ECG and POCCUS classifications.
To explore the feasibility of integrating POCCUS scanning into the ECG lab, the Mayo Clinic Echocardiography and ECG laboratories initiated an in-house real-world project (Supplementary Figure 1). Novice internal medicine residents’ operators with no echocardiography experience (FBA, RS), trained in-house to acquire POCCUS imaging, were present in the ECG lab between November 1, 2023, and February 28, 2024. In total, handheld POCCUS was available on 27 days across the study period, corresponding to approximately 6.5 days per month. Since the study was not designed to maximize patient recruitment, operators were instructed to obtain verbal patient consent before initiating scans. Patient enrollment ranged from 9 to 35 patients per scan day. A target enrollment of 500 patients was predefined to ensure adequate feasibility and precision of proportion estimates, and recruitment stopped once this goal was reached. Adult patients (≥18 years) undergoing ambulatory ECG on days when POCCUS operators were available were offered POCCUS after their ECG, regardless of AI-ECG results, using a pragmatic convenience sampling approach. Interested patients were briefed by the ECG technician and the POCCUS operator and provided oral consent before scanning. In summary, inclusion required 3 criteria: (1) patient availability and willingness to participate, (2) verbal consent, and (3) presence of a trained novice operator to perform the scan. POCCUS studies were uploaded and stored on an encrypted online patient-care platform after acquisition and subsequently reviewed by a trained cardiologist with 8 years of experience (GT) to assess the likelihood of structural heart disease (SHD). Any abnormal findings not previously documented were communicated to the patient’s primary provider. Patients with poor-quality POCCUS images were excluded. No further exclusion criteria were applied. The Mayo clinic institutional review board approved the project.
AI-ECG
The Mayo Clinic ECG Laboratory performs an average of 250 ECGs daily. All ECGs are acquired by a technician in the supine position at a sampling rate of 500 Hz using a GE-Marquette ECG machine (Marquette, WI). The data are stored using the MUSE data management system (GE Healthcare, Chicago, IL) for review and immediate clinical interpretation. AI-ECG data, generated through validated algorithms, becomes accessible upon confirmation of the ECG clinical interpretation report. This data provides probabilities and categorizations for aortic stenosis, decreased left ventricular ejection fraction (LVEF), HCM, and CA using specific thresholds: 40.6%, 25.6%, 11%, and 48.5%, respectively. ,,, These data are available for healthcare providers to review on a separate platform linked to the patient’s electronic medical record. Of note, the cut-off for LVSD in the AI-ECG algorithm used in the current study was validated for detecting LVEF <35%. However, its application in the general population also demonstrated an AUC of 0.880 for identifying LVEF <50%.
POCCUS
Novice POCCUS operators, first-year postgraduate physicians with no prior cardiac ultrasound training, participated in a structured, 2-day training program led by an expert advanced cardiac sonographer. The 2-day POCCUS training program followed a standardized curriculum designed for novice physician trainees. Day 1 included self-directed, video-based modules on cardiac anatomy, ultrasound principles, and handheld device operation. Day 2 consisted of hands-on scanning sessions supervised by an expert cardiac sonographer, during which trainees practiced acquiring parasternal, apical, and subcostal views on healthy volunteers (Supplementary Table 1). While implemented with physician trainees, this model is intended to be scalable to nonphysician personnel for broader adoption (Supplementary Table 1). POCCUS was conducted using a portable Lumify S4-1 handheld ultrasound device (Phillips Healthcare). Lumify S4-1, as most hand-held devices, doesn’t have quantitative Doppler. Although patients were imaged in the supine position due to the ECG lab setting, operators were instructed to request a left lateral decubitus position when feasible, particularly to optimize parasternal and apical views. POCCUS was obtained regardless of the AI-ECG result. Clips were acquired in 10-second loops from parasternal (long and short axes), apical (4 chambers), and subcostal acoustic windows. The studies were saved with patient-identifier numbers, uploaded, and stored on a dedicated, secured POCCUS review platform (Qpath, Telexy, Canada). Subsequently, a trained cardiologist reviewed the studies to assess the likely presence of SHD. Interpretation relied on visual assessment of the 2-dimensional POCCUS studies. Suspected LVSD was defined as LVEF less than 50%, possible AS was defined by significant calcification or restricted excursion of the aortic valve leaflets, and increased left ventricular wall thickness (ILVWT) was determined by offline assessment of LV walls. Patients with ILVWT were grouped under a single category, without formal distinction between HCM and CA; however, features such as asymmetrical septal thickening or cavity obliteration suggested HCM, whereas concentric thickening, myocardial speckling, and pericardial effusion raised suspicion for CA. The physician operators conducting POCCUS and the interpreting cardiologists were blinded to patients’ medical records and AI-ECG results.
Image interpretation and quality assessment
A 4-point scoring system was used to evaluate the image quality of 2D POCCUS. This scale was developed specifically for the purposes of this feasibility study, with criteria based on the visibility of key cardiac structures, appropriate optimization of imaging parameters (including orientation, gain, depth, and focus), and the overall interpretability of each scan. While novel to this study, the scoring system is conceptually similar to image quality grading frameworks used in standard transthoracic echocardiography and aligns with protocols previously described in the literature.
Although most patients were imaged in the supine position due to the spatial constraints of the ECG laboratory, novice operators were instructed to request a left lateral decubitus position whenever feasible, particularly to optimize parasternal and apical views. While apical windows were occasionally limited, parasternal views frequently provided sufficient visualization of the target anatomy to allow diagnostic screening for structural heart disease. This highlights the feasibility of handheld POCCUS for use in pragmatic screening settings. All POCCUS images were interpreted by a single cardiologist with expertise in echocardiography. To assess intraobserver reproducibility, a subset of 40 patient scans was randomly selected from the study cohort for each of the 3 diagnostic targets: (1) low EF, (2) AS, and (3) increased LV wall thickness suggestive of hypertrophic cardiomyopathy or cardiac amyloidosis. Care was taken to ensure that each subset included a balanced mix of positive and negative findings to represent the full spectrum of image interpretations encountered in the study. Each scan was re-evaluated in a blinded fashion several months after the initial interpretation by the same cardiologist with echocardiographic expertise. Diagnostic findings were recorded as binary outcomes (0 = negative, 1 = positive), focusing on 3 predefined screening targets: LVSD, AS, and increased LV wall thickness suggestive of HCM or cardiac amyloidosis. Intraobserver agreement was assessed using Cohen’s κ coefficient, which accounts for chance agreement.
The scoring system for POCCUS image quality is summarized in Supplementary Table 2, ranging from 1 (nondiagnostic) to 4 (good) based on visualization of key cardiac structures and overall interpretability.
Data collection
All baseline characteristics, encompassing demographics, medical comorbidities, cardiovascular history, and results of formal echocardiography studies, were extracted and validated through meticulous manual chart review. After enrollment and based on the availability of a formal echocardiogram, gold-standard for diagnostic comparison, in the electronic medical record, patients were retrospectively stratified.
Valvular heart disease was categorized as any degree (mild, moderate, or severe) of stenosis or regurgitation affecting any of the 4 cardiac valves. For the purposes of diagnostic performance analysis, only patients with AS meeting criteria for AI-ECG detection were included in the SHD-positive group, while other valvular abnormalities were excluded as they fall outside the scope of current AI-ECG algorithms.
Statistical analysis
The main aim of the current analysis was to examine the feasibility and diagnostic yield of incorporating AI-ECG and POCCUS to screen for SHD in a nonselective patient population. Since operators were blinded to the patient’s medical history and diagnostic workup, the study population comprised patients with and without recently available formal TTE evaluation.
The analysis was conducted in 2 stages. The first stage focused on evaluating diagnostic performance among patients with a previously available formal TTE, which served as the reference standard. The NPV and PPV of the AI-ECG alone and of the combined AI-ECG + POCCUS approach were assessed for detecting SHD (Supplementary Table 3). These metrics were calculated both overall and by SHD subtype, including LVSD, AS, ILVWT. Patients with prior AVR were excluded from the AS analysis but included in the LVSD and ILVWT analyses, consistent with the validation cohorts for these algorithms. Given the relatively high prevalence of cardiac amyloidosis among patients with AS and low EF, and the risk of underdiagnosing HCM in this population, we retained these patients in the screening workflow for non-AS outcomes. Additionally, no data currently supports that postsurgical changes impair AI-ECG precision, highlighting the added value of the POCCUS layer in these cases. The second stage involved a screening simulation conducted in the full study cohort, regardless of TTE availability. A 2-step model was applied: first, AI-ECG was used to screen all patients. Those flagged as “positive” were subsequently further classified using POCCUS. The goal was to simulate a stepwise screening algorithm that leverages the high NPV of AI-ECG while accounting for the potential logistical complexity and cost associated with widespread POCCUS use. Patients classified as “positive” by both methods were considered likely to have SHD and were identified as candidates for formal TTE referral. The NNS was calculated as the number of patients who would need to undergo the AI-ECG plus POCCUS workflow for 1 patient to be triaged to formal TTE, reflecting eligibility for referral rather than confirmed disease and depending on the characteristics of the screened population and disease prevalence.
Data are presented as means and standard deviations or median and interquartile range (IQR) for continuous variables, per the variable’s distribution, and as frequencies and percentages for categorical variables. Comparisons between groups were performed using chi-square tests for categorical variables and independent t -tests or Mann–Whitney U test for continuous variables, according to the variable’s distribution.
A 2-sided P -value of less than 0.05 was considered statistically significant for all analyses. Statistical analysis was performed using the JMP software version 17.0.0.
Results
During the days of available POCCUS imaging in the ECG lab, an average of 250 ± 32 ECGs were performed in the ECG lab every day. A daily average of 25 (9-37) POCCUS studies were performed, corresponding to 10% to 15% of all daily ECGs. The average duration of the POCCUS studies was 6 ± 2 minute from when the ECG was done until the POCCUS operator left the room.
A total of 500 patients were included in the study. A total of 14 patients were excluded because of unavailable POCCUS studies: eleven due to upload and storage failures and 3 due to uninterpretable imaging. This resulted in a final study population of 486 patients, 286 of which with a previous formal echocardiogram ( Figure 1 ). One-hundred and twenty-one (42%) of TTEs were done on the same day as the POCCUS and only 39 (13%) were done ≥ 1 year before and the remaining 45% were done within this intermediate time frame. Agreement was excellent across all diagnostic categories, with κ values of 1.00 for LVSD, 0.95 for AS, and 0.92 for increased LV wall thickness. These results demonstrate high reproducibility and support the reliability of focused POCCUS interpretation in a handheld screening context under real-world conditions.
Study’s flowchart.
Baseline characteristics
Patients had a median age of 64 years (IQR: 53.2-71.4) and 49% were female. Baseline characteristics of the study population across formal echocardiographic assessment availability status are provided in Table 1 . Patients with available TTEs had higher rates of hypertension, diabetes, atrial fibrillation, and clinical heart failure but lower rates of coronary artery disease and chronic kidney disease. A total of 28 patients had previous aortic valve replacement, all of whom had TTE evaluation. LVSD, HCM, and CA were diagnosed in 10%, 2%, and 2%, respectively. Of the patients with available formal TTE assessment, 50 (17.5%) had documented confirmed SHD. Among the 51 patients identified with valvular heart disease in Table 1 , several had conditions not currently detectable by AI-ECG algorithms, including MR, MS, TR, AR, or less-than-severe AS. These patients were not included in the SHD-positive group used for PPV/NPV calculations, which focused exclusively on AI-detectable conditions: LVSD, AS, HCM, and CA.
Table 1
Baseline characteristics of the study population across formal TTE status.
| Without TTE ( n = 200) | With TTE ( n = 286) | Total ( n = 486) | P -value | |
|---|---|---|---|---|
| Age (median, IQR) | 61.0 (49.7, 68.7) | 56.0 (57.0, 73.9) | 64.0 (53.2, 72) 71.4) | <.01 |
| Female (%) | 112 (56) | 127 (44) | 239 (49) | .01 |
| Hypertension (%) | 50 (25) | 106 (37) | 156 (32) | <.01 |
| Diabetes (%) | 17 (9) | 43 (15) | 60 (12) | .02 |
| Chronic Kidney Disease (%) | 14 (7) | 52 (18) | 66 (14) | <.01 |
| Coronary artery disease (%) | 36 (18) | 97 (34) | 133 (27) | <.01 |
| Heart failure (%) | 3 (1) | 75 (26) | 78 (16) | <.01 |
| Valvular heart disease (%) | NA | 51 (18) | 51 (10) | <.01 |
| AVR (%) | 0 (0) | 28 (10) | 28 (5) | <.01 |
| Atrial fibrillation (%) | 9 (5) | 94 (33) | 103 (21) | <.01 |
| HCM (%) | NA | 6 (2) | 6 (1) | <.01 |
| CA (%) | NA | 5 (2) | 5 (1) | <.01 |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
