Background
The first pediatric appropriate use criteria (AUC) address the use of initial transthoracic echocardiography in outpatients by all ordering providers. The aim of this study was to appraise the performance of the AUC across pediatric cardiologists, noncardiologist subspecialists, and primary care providers (PCPs). A further aim was to describe the variations in ordering patterns of different groups of practitioners, which could serve as the basis for targeted quality improvement activities.
Methods
Electronic health records for Seattle Children’s Hospital and its four regional sites were retrospectively reviewed for initial transthoracic echocardiographic studies performed on patients aged ≤18 years. A sample of 1,000 consecutive studies and a sample of 1,514 studies in which studies ordered by noncardiologists were enriched were reviewed. The ordering provider type, study indication, and findings (normal, incidental, or abnormal) were classified. Indications mapped to three categories: appropriate (A), may be appropriate (M), and rarely appropriate (R). If multiple indications were documented, the highest level of appropriateness was used.
Results
In the consecutive sample, pediatric cardiologists ordered 81%, noncardiologist subspecialists 13%, and PCPs 5% of the total studies. In the enriched sample, only 4% were unclassifiable by the AUC. Abnormal findings were identified in 23% of A, 13% of M, and 9% of R studies ( P = .03). Appropriateness varied among the three groups of providers ( P < .001). For pediatric cardiologists, 67% of studies were indication category A, 13% M, and 14% R. Noncardiologist subspecialists ordered the highest percentage of A studies (88%) and the lowest percentage of R studies (1%). PCPs had the highest percentage of R indications (18%), and 23% could not be fully classified, because of insufficient order information. Yield of abnormal findings was highest for subspecialists (23%), intermediate for cardiologists (19%), and lowest for PCPs (15%; P = .03).
Conclusions
The AUC performed well across all provider types, as measured by the low percentage of unclassifiable indications and the observed relationship between greater appropriateness and higher yield of abnormal findings. The three provider types differed in appropriateness rates and had distinct ordering patterns, which could form the basis for future targeted quality improvement efforts.
Highlights
- •
The AUC performed well across all provider types, as measured by the low percentage of U indications (4%) and the observed relationship between greater appropriateness and higher yield of abnormal findings.
- •
Pediatric cardiologists, noncardiologist subspecialists, and PCPs demonstrated distinct ordering patterns with varied appropriateness rates.
- •
Noncardiologist subspecialists achieved the highest percentage of A studies and the lowest percentage of R studies.
- •
The cardiologists’ practices were broadly consistent with major national trends.
- •
Incomplete documentation by the PCPs did not allow classification of almost a quarter of cases, highlighting the need for targeted quality improvement efforts.
Attention ASE Members:
The ASE has gone green! Visit www.aseuniversity.org to earn free continuing medical education credit through an online activity related to this article. Certificates are available for immediate access upon successful completion of the activity. Nonmembers will need to join the ASE to access this great member benefit!
Methods
Patient Population
We conducted a retrospective, existing-data study to describe TTE ordering practices and associated transthoracic echocardiographic findings. We queried our digital echocardiography system for all first-time transthoracic examinations performed on outpatients aged ≤18 years at Seattle Children’s Hospital (SCH) main campus and 18 outreach locations in central and western Washington and Alaska. Patients with known cardiac disease, as determined by review of electronic health records, were excluded. In addition, any patient known to have been previously evaluated by TTE, either as an inpatient or at an outside institution, was excluded.
Data Collection
Data collection occurred in reverse chronologic order, beginning in November 2015 through July 2015. One thousand consecutive qualifying studies were reviewed, including all ordering provider types. Then, to acquire sufficient numbers of studies ordered by noncardiologist subspecialists and PCPs for meaningful comparisons, we continued to accrue qualifying examinations ordered by each of these two provider types in reverse chronologic order. We included 300 examinations ordered by noncardiologist subspecialists dating back to January 2015. The subspecialist providers included physicians, as well as nurse practitioners and physician assistants employed by the various subspecialty divisions at SCH. Four hundred transthoracic examinations dating back to June 2011 and ordered by PCPs were included because of a high frequency of instances in which the available clinical documentation was insufficient to fully classify the indication. The PCP group consisted of pediatricians, family medicine physicians, nurse practitioners, and physician assistants from Washington and Alaska. Age, gender, date of study, race, insurance, clinic location, and ordering provider were obtained by querying the digital echocardiography database and electronic medical records. Electronic health records were reviewed to confirm and categorize the ordering provider and classify the indication for the study. Available electronic health records included clinic notes for TTE ordered by SCH providers and request forms submitted by PCPs. The TTE report was reviewed to classify the findings of the study.
All cases were reviewed for consistency of classification by one of the investigators (S.K.). In cases with discrepant or unclear classification, a supervising expert investigator (A.H.S.) determined the final classification.
TTE Classification Process
We classified three data elements for each transthoracic echocardiographic study ( Figure 1 ). First, we classified the ordering provider as a pediatric cardiologist, noncardiologist subspecialist, or PCP. Second, the study indication was assigned into one of the nine major category tables and one of the 113 individual indications within these tables as detailed by the AUC. We followed the assumptions and definitions provided in the AUC document. In particular, “innocent murmur” was defined as a grade I–II/VI systolic, ejection or vibratory type with no other abnormal auscultatory findings in a patient >8 weeks of age. If the indication could be mapped to a table but was not sufficiently described to assign to a specific indication, we characterized the study as “not otherwise specified” (NOS). For example, the indication “murmur” with no additional information was classified as murmur NOS. In contrast, a U indication was one that did not appear at all in the AUC, such as ejection click. The selected indication corresponded to a specific appropriateness score (1–9) and level of appropriateness category, A (7–9), M (4–6), or R (1–3), as defined by the AUC. If multiple indications were documented, the highest level of appropriateness was used. Finally, after the first two steps were completed, we classified the study findings according to the grading system outlined in a multicenter study by Sachdeva et al to ensure comparable results. As such, the findings were categorized as normal, incidental, or abnormal. Abnormal results were further subdivided on the basis of severity as minor, moderate, or severe, as detailed in Figure 2 . A diagnosis of left ventricular hypertrophy (LVH) was assigned if the report summary commented specifically on either (1) the presence of LVH or (2) left ventricular mass indexed to height in meters 2.7 greater than the 95th percentile for age and gender.
Statistical Analysis
Patient characteristics, provider type, and AUC data were summarized using counts and percentages for categorical data and means and SDs for continuous data. Associations with AUC were assessed using χ 2 tests, and univariate logistic regression was used to obtain odds ratios for abnormal results on TTE. Analyses were performed using Stata version 14.2 (StataCorp, College Station, TX).
Because the study dates for PCPs spanned the AUC release date, we investigated the effect of the publication by comparing ordering practices of PCPs between the two chronologic eras. All the transthoracic studies ordered by cardiologists and noncardiology subspecialists were performed after the AUC publication date. All study procedures were approved by the SCH institutional review board. Given the existing data study design, informed consent was not required.
Methods
Patient Population
We conducted a retrospective, existing-data study to describe TTE ordering practices and associated transthoracic echocardiographic findings. We queried our digital echocardiography system for all first-time transthoracic examinations performed on outpatients aged ≤18 years at Seattle Children’s Hospital (SCH) main campus and 18 outreach locations in central and western Washington and Alaska. Patients with known cardiac disease, as determined by review of electronic health records, were excluded. In addition, any patient known to have been previously evaluated by TTE, either as an inpatient or at an outside institution, was excluded.
Data Collection
Data collection occurred in reverse chronologic order, beginning in November 2015 through July 2015. One thousand consecutive qualifying studies were reviewed, including all ordering provider types. Then, to acquire sufficient numbers of studies ordered by noncardiologist subspecialists and PCPs for meaningful comparisons, we continued to accrue qualifying examinations ordered by each of these two provider types in reverse chronologic order. We included 300 examinations ordered by noncardiologist subspecialists dating back to January 2015. The subspecialist providers included physicians, as well as nurse practitioners and physician assistants employed by the various subspecialty divisions at SCH. Four hundred transthoracic examinations dating back to June 2011 and ordered by PCPs were included because of a high frequency of instances in which the available clinical documentation was insufficient to fully classify the indication. The PCP group consisted of pediatricians, family medicine physicians, nurse practitioners, and physician assistants from Washington and Alaska. Age, gender, date of study, race, insurance, clinic location, and ordering provider were obtained by querying the digital echocardiography database and electronic medical records. Electronic health records were reviewed to confirm and categorize the ordering provider and classify the indication for the study. Available electronic health records included clinic notes for TTE ordered by SCH providers and request forms submitted by PCPs. The TTE report was reviewed to classify the findings of the study.
All cases were reviewed for consistency of classification by one of the investigators (S.K.). In cases with discrepant or unclear classification, a supervising expert investigator (A.H.S.) determined the final classification.
TTE Classification Process
We classified three data elements for each transthoracic echocardiographic study ( Figure 1 ). First, we classified the ordering provider as a pediatric cardiologist, noncardiologist subspecialist, or PCP. Second, the study indication was assigned into one of the nine major category tables and one of the 113 individual indications within these tables as detailed by the AUC. We followed the assumptions and definitions provided in the AUC document. In particular, “innocent murmur” was defined as a grade I–II/VI systolic, ejection or vibratory type with no other abnormal auscultatory findings in a patient >8 weeks of age. If the indication could be mapped to a table but was not sufficiently described to assign to a specific indication, we characterized the study as “not otherwise specified” (NOS). For example, the indication “murmur” with no additional information was classified as murmur NOS. In contrast, a U indication was one that did not appear at all in the AUC, such as ejection click. The selected indication corresponded to a specific appropriateness score (1–9) and level of appropriateness category, A (7–9), M (4–6), or R (1–3), as defined by the AUC. If multiple indications were documented, the highest level of appropriateness was used. Finally, after the first two steps were completed, we classified the study findings according to the grading system outlined in a multicenter study by Sachdeva et al to ensure comparable results. As such, the findings were categorized as normal, incidental, or abnormal. Abnormal results were further subdivided on the basis of severity as minor, moderate, or severe, as detailed in Figure 2 . A diagnosis of left ventricular hypertrophy (LVH) was assigned if the report summary commented specifically on either (1) the presence of LVH or (2) left ventricular mass indexed to height in meters 2.7 greater than the 95th percentile for age and gender.
Statistical Analysis
Patient characteristics, provider type, and AUC data were summarized using counts and percentages for categorical data and means and SDs for continuous data. Associations with AUC were assessed using χ 2 tests, and univariate logistic regression was used to obtain odds ratios for abnormal results on TTE. Analyses were performed using Stata version 14.2 (StataCorp, College Station, TX).
Because the study dates for PCPs spanned the AUC release date, we investigated the effect of the publication by comparing ordering practices of PCPs between the two chronologic eras. All the transthoracic studies ordered by cardiologists and noncardiology subspecialists were performed after the AUC publication date. All study procedures were approved by the SCH institutional review board. Given the existing data study design, informed consent was not required.
Results
Study Population
One thousand consecutive qualifying transthoracic echocardiographic studies were accrued in reverse chronologic order from November to July 2015. In that consecutive sample, cardiologists ordered 81%, noncardiology subspecialists 13%, and PCPs 5% of the total studies. Sixty-one percent of the examinations had been performed at the main campus of SCH and its satellite outpatient locations in the metropolitan Seattle area, 15% in the south Puget Sound region of Washington, 13% in central Washington, and 10% in Alaska. In this cohort, 80% of examinations revealed normal or incidental findings, and 20% had abnormal findings.
Our final data set consisted of 814 studies ordered by pediatric cardiologists, 300 ordered by noncardiologist subspecialists, and 400 ordered by PCPs. The ordering providers encompassed 40 pediatric cardiologists, 100 noncardiologist subspecialists, and 256 PCPs. Detailed characteristics of our patient population are shown in Table 1 . Cardiologists ordered TTE at all different locations, while 97% of examinations ordered by noncardiologist subspecialists and 99% by PCPs were performed at the main SCH campus. Sixteen percent of patients were <1 year of age. Notably, the patients referred by PCPs were somewhat more likely to be Caucasian and to have private insurance.
| Total ( N = 1,514) | Ordering provider type | P ∗ | |||
|---|---|---|---|---|---|
| Pediatric cardiologists ( n = 814) | Noncardiologist subspecialists ( n = 300) | PCPs ( n = 400) | |||
| Gender | .32 | ||||
| Female | 618 (43%) | 312 (42%) | 140 (47%) | 166 (42%) | |
| Male | 825 (57%) | 431 (58%) | 160 (53%) | 234 (58%) | |
| Age (y) | <.001 | ||||
| <1 | 245 (16%) | 156 (19%) | 25 (8%) | 64 (16%) | |
| 1–5 | 287 (19%) | 132 (16%) | 47 (16%) | 108 (27%) | |
| 5–10 | 256 (17%) | 131 (16%) | 62 (21%) | 63 (16%) | |
| >10 | 725 (48%) | 395 (48%) | 165 (55%) | 165 (41%) | |
| Race | <.001 | ||||
| Caucasian | 829 (58%) | 410 (56%) | 169 (56%) | 250 (63%) | |
| African American | 101 (7%) | 51 (7%) | 28 (9%) | 22 (5.5%) | |
| Asian | 95 (6%) | 46 (6%) | 29 (10%) | 23 (6%) | |
| Native American | 47 (3%) | 39 (5%) | 4 (2%) | 4 (1%) | |
| Pacific Islander | 11 (1%) | 6 (1%) | 3 (1%) | 2 (0.5%) | |
| Other/refused | 353 (25%) | 187 (25%) | 67 (22%) | 99 (25%) | |
| Insurance | .001 | ||||
| Private | 770 (56%) | 375 (52%) | 155 (54%) | 240 (64%) | |
| Medicaid | 609 (44%) | 341 (48%) | 134 (46%) | 134 (36%) | |
| TTE location | <.001 | ||||
| SCH main campus | 925 (61%) | 238 (29%) | 290 (97%) | 397 (99%) | |
| SCH satellite clinics | 263 (17%) | 258 (32%) | 4 (1%) | 1 (0.2%) | |
| South Sound Group | 129 (9%) | 123 (14%) | 4 (1%) | 2 (0.5%) | |
| Central Washington | 113 (7%) | 113 (14%) | 0 (0%) | 0 (0%) | |
| Alaska | 84 (6%) | 82 (10%) | 2 (1%) | 0 (0%) | |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
