Background
A survey of pediatric cardiologists who participated in the Pediatric Appropriate Use of Echocardiography study was conducted to assess attitudes toward appropriate use criteria (AUC) and the relationship between perceptions of usefulness of a multifaceted educational intervention (EI) and the appropriateness of transthoracic echocardiography (TTE).
Methods
Self-reported helpfulness and impact of a four-component EI (feedback of personal appropriateness data before EI, lecture, self-assignment of AUC indications, and monthly feedback) was surveyed. Physicians’ perceptions were correlated with measured changes in appropriateness after EI by center.
Results
Responses were obtained from 54 of 89 physicians (61%; 52% general cardiologists, 24% imaging specialists), and most (72%) felt that AUC were helpful in health care cost reduction. More physicians with ≤10 years of experience self-reported ordering TTE less often because of AUC ( P = .04). Subspecialty did not influence TTE ordering practice. Centers whose physicians had higher rates of reading the document had higher appropriateness. A change in practice following EI was self-reported by 31 of 54 respondents (57%). All components of EI were felt to be helpful. Helpfulness and self-reported impact of each EI component tracked together ( r = 0.61; 95% CI, 0.16–0.85; P = .01) but varied among centers. Centers with higher perceived practice impact of EI overall had greater changes in measured appropriateness after EI.
Conclusions
AUC were perceived to be useful by a majority of pediatric cardiologists surveyed. Centers with a positive attitude toward AUC and higher engagement with EI had higher actual appropriateness of TTE orders. Improving physicians’ attitudes toward AUC and EI may improve outpatient TTE utilization.
Highlights
- •
A majority of the pediatric cardiologists who participated in the survey felt that AUC were helpful in healthcare cost reductions.
- •
Centers whose physicians had a higher rate of reading the AUC document had a higher appropriateness.
- •
A positive attitude toward AUC and higher engagement with educational intervention were associated with higher actual improvement in appropriateness of TTE orders.
In response to the rising costs of imaging-related procedures, appropriate use criteria (AUC) were introduced in adult cardiology in 2005. Implementation studies in adult cardiology were followed by the use of educational interventions (EIs) to improve the appropriate use of cardiac imaging. These EIs had variable impact, with the most significant improvement noted in studies using the audit and feedback mechanism for EI. The first pediatric AUC were published in 2014 to guide providers in the appropriateness of indications for transthoracic echocardiography (TTE) during initial outpatient evaluation. Anecdotally, similar to the initial response in adult cardiology, these were viewed by some with trepidation but by others as a useful tool.
Our group conducted the Pediatric Appropriate Use of Echocardiography (PAUSE) study to obtain baseline appropriateness data and study the effectiveness of a multifaceted EI. This study showed that a passive release of the AUC document did not have a significant impact on the appropriateness of TTE orders, while a multifaceted EI improved it, albeit with a variable impact on the participating centers. It is important to understand the factors that could potentially influence the engagement of physicians with EIs, as this will affect the appropriateness of TTE orders. We conducted a survey of the physicians who participated in the PAUSE study to learn about their attitudes toward the pediatric AUC document and the perceived impact of AUC on clinical practice. We further surveyed physicians’ perceptions of the usefulness and impact of EI and correlated them with the appropriateness of TTE following EI at each center.
Methods
Pediatric cardiologists from four centers that participated in the PAUSE study were surveyed in January 2016 on their attitudes toward AUC and the perceived impact of the EI that was conducted during the PAUSE study from July to October 2015. Participating centers included the Emory University School of Medicine and Children’s Healthcare of Atlanta, the Children’s Hospital at Montefiore, Nicklaus Children’s Hospital, and NewYork-Presbyterian, Morgan Stanley Children’s Hospital. The multiphased quality improvement project, when presented to the respective institutional review boards of the participating centers, did not qualify as a research project for all centers except for NewYork-Presbyterian, Morgan Stanley Children’s Hospital, where the study was approved by the institutional review board. Detailed methods of data collection for the pre-EI phase (January 1, 2015, to April 30, 2015) and the post-EI phase (July 1, 2015, to October 30, 2015) have been published previously. Briefly, the EI included four components: feedback by the site investigator (SI) to individual physicians regarding their individual appropriateness data before EI and how it compared with that of their own centers and the study as a whole, lecture on AUC, self-assignment of AUC indications by the provider, and monthly personal feedback via e-mail by the SI on appropriateness rate during the EI. During the pre-EI phase, the SI assigned the AUC indications after review of the clinic notes, but during the EI phase, the providers were required to assign AUC indications before ordering the TTE as a part of the EI. On the basis of the study indication, appropriateness ratings were assigned as appropriate, may be appropriate, or rarely appropriate.
Survey Questionnaire
The survey was built using online survey software (SurveyMonkey, Palo Alto, CA) and disseminated to the physicians by the SIs via e-mail. The survey was open for a period of 2 weeks, and a reminder to complete the survey was sent 1 week after the initial e-mail. Participants in the survey were asked to reveal their centers but not their own identities, to assure them of privacy and allow them to answer all questions truthfully. Centers were deidentified and labeled as A, B, C, and D for the purpose of analysis. The survey questionnaire is included as Supplemental Table 1 (available at www.onlinejase.com ). In brief, the survey questions included physician experience, based on number of years since graduation from cardiology fellowship (0–5, >5–10, >10–20, and >20 years) and the area of subspecialization (general cardiology, noninvasive imaging, interventional cardiology, electrophysiology, heart failure and transplantation, and others such as pulmonary hypertension and cardiac critical care). A five-point Likert-type scale (strongly agree, agree, neutral, disagree, and strongly disagree) was used to ask the respondents if they thought AUC were overall useful in improving patient care, improving patient care in their own daily practice, and reducing health care costs. Respondents were also asked if they had read the AUC document, completely or partially (only tables and flowcharts), before the pre-EI phase and if they made any changes to their practice on the basis of the AUC document. Respondents were then asked if the overall EI process was helpful, if they made any changes to their practice (substantial changes, small changes, and no changes), and in what way they changed their practice (order more or fewer transthoracic echocardiographic studies). They were then queried specifically about each of the four components of the EI using a perceived (self-reported) “helpfulness score” (very helpful, somewhat helpful, not helpful, can’t remember/not applicable). They were asked how each component influenced their practice. Respondents were further asked that if they made any change following EI and what was the most common clinical indication (murmur, chest pain, syncope, palpitations, or others) and the patient age group (<3 months, 3 months to 1 year, >1–5 years, >5–10 years, and >10 years) for which they made such a change.
Statistical Analysis
Statistical analyses were performed using SAS version 9.4 (SAS Institute, Cary, NC). Statistical significance was assessed at the .05 level. Descriptive statistics were calculated for all variables of interest, including counts and percentages, or means and SDs, as appropriate. Chi-square tests were used to assess the relationship between physician experience and self-reported changes in their practice. When expected cell counts were small (less than five), an exact test was used in place of the χ 2 test. To assess the impact of EI on change in practice, responses were dichotomized for statistical analyses. Responses of substantial or small change were recoded as 1 (to indicate a change was made), and no change was recoded as 0. This dichotomization was done for each of the four components of the EI. These impact scores were tabulated at both the center level and overall to yield the average effects of EI. EI helpfulness scores were recoded in an ordinal fashion (4 = very helpful, 3 = somewhat helpful, 2 = not helpful, 1 = can’t remember/not applicable) to calculate the correlation between perceived impact of EI and EI helpfulness scores. Similarly, a correlation was used to quantify the association between center appropriateness rate and physician AUC document reading rate. Because of the ordinal nature of the data, Spearman’s rank-order correlation coefficient with associated 95% CI is reported.
Results
The overall survey response rate was 61% (54 of 89), with 36% (5 of 14) at center A, 74% (28 of 38) at center B, 33% (5 of 15) at center C, and 73% (16 of 22) at center D. Characteristics of the 54 respondents are presented in Table 1 . The majority (35 of 54 [65%]) had ≤10 years of experience since graduation from fellowship. Overall, there were no significant differences in physician experience among the centers. Nearly half (28 of 54 [52%]) were general cardiologists, and 13 of 54 (24%) were noninvasive imaging specialists. Nearly all general cardiologists (26 of 28 [92%]) and all imaging specialists (13 of 13 [100%]) reported reading the AUC document completely or partially, compared with other specialists (8 of 13 [61%]; P = .02). There was no significant difference in the reported reading of the AUC document on the basis of physician experience.
| Characteristic | n (%) |
|---|---|
| Center | |
| A | 5 (9.3) |
| B | 28 (51.9) |
| C | 5 (9.3) |
| D | 16 (29.6) |
| Number of years since graduation from fellowship | |
| 0–5 | 18 (33.3) |
| >5–10 | 17 (31.5) |
| >10–20 | 9 (16.7) |
| >20 | 10 (18.5) |
| Primary area of practice | |
| General pediatric cardiology | 28 (51.9) |
| Electrophysiology | 3 (5.6) |
| Heart failure and transplantation | 3 (5.6) |
| Interventional cardiology | 4 (7.4) |
| Noninvasive imaging | 13 (24.1) |
| Other | 3 (5.6) |
| Familiarity with the AUC document | |
| Completely read | 20 (37) |
| Partially read | 27 (50) |
| Not read | 5 (9.3) |
| Cannot remember | 2 (3.7) |
Most respondents felt that the AUC were useful in all domains. The strongest agreement was for health care cost reduction, followed by improved general patient care ( Table 2 ). There were no differences in the physicians’ general attitudes toward the AUC between those who had partially versus completely read the AUC document. Physicians’ experience and areas of practice did not influence their general attitudes toward AUC ( P = .1). Thirty-three physicians (61%) self-reported making changes in practice following publication of AUC, though 29 reported only small changes, with four reporting substantial changes. More physicians with ≤10 years of experience (20 of 26 [77%]) self-reported changing their practice by ordering TTE less often after reading the AUC document compared with those with >10 years of experience (13 of 26 [50%]; P = .04). Area of practice did not affect TTE ordering practice ( P = .10). Table 3 shows center-based comparison of the actual appropriateness rate during the pre-EI phase, rate of reading the AUC document, and self-reported change in practice of the respondents. Centers whose physicians had higher rates of reading the AUC document (partially or completely) also had higher rates of appropriateness. The center with the lowest rate of reading the document (center C) had the lowest rate of appropriateness and the lowest rate of self-reported likelihood of practice change. This center also had the lowest response rate for the survey.
| Participant response | Improving patient care in general ( n = 53) | Improving care in your practice ( N = 54) | Reducing health care costs ( N = 54) |
|---|---|---|---|
| Strongly agree | 3 (5.7%) | 4 (7.4%) | 10 (18.5%) |
| Agree | 32 (60.4%) | 25 (46.3%) | 29 (53.7%) |
| Neutral | 14 (26.4%) | 18 (33.3%) | 14 (25.9%) |
| Disagree | 4 (7.5%) | 6 (11.1) | 1 (1.9%) |
| Strongly disagree | 0 (0.0%) | 1 (1.9%) | 0 (0.0%) |
| Center | n | Percent appropriate TTE (%) | Percent rarely appropriate TTE (%) | Rate of reading AUC document | Rate of perceived practice change |
|---|---|---|---|---|---|
| A | 5 | 77.4 | 6.8 | 5/5 (100.0%) | 3/4 (75.0%) |
| B | 28 | 77.4 | 4.9 | 26/28 (92.9%) | 19/28 (67.9%) |
| C | 5 | 51.6 | 34.8 | 2/5 (40.0%) | 2/4 (50.0%) |
| D | 16 | 63.2 | 10.5 | 14/16 (87.5%) | 9/16 (56.3%) |
A change in practice following EI was self-reported by 31 of 54 respondents (57%). A center-based comparison of the helpfulness score for each component of the EI is shown in Figure 1 . All components of EI were felt to be helpful, and monthly feedback had the highest helpfulness score. The mean helpfulness score for feedback of personal data before EI was 3.0 ± 1.0, for staff lecture was 2.9 ± 1.1, for self-assignment of AUC indication was 2.9 ± 0.9, and for monthly feedback was 3.2 ± 0.6. Helpfulness score and perceived impact of each EI components tracked together ( r = 0.61; 95% CI, 0.16–0.85; P = .01) but varied among centers ( Figure 2 ). Table 4 shows the self-reported impact of EI and actual rates of practice change following EI. Physicians at center D found each component to be least helpful and had the lowest perceived impact. Overall, the EIs were effective, increasing TTE for indications rated appropriate by 3.1% and reducing those rated rarely appropriate by 1.7%. Centers with higher self-reported practice impact of EI overall had greater changes in measured appropriateness after EI. Center C had the highest perceived impact of EI components and the highest increase in appropriate indications and decline in rarely appropriate indications. Center D had the lowest perceived overall impact and increased rate of rarely appropriate indications after EI.
| Center (physicians) | Overall EI: perceived practice impact | Mean of EI components | Percent appropriate TTE (%) | Percent rarely appropriate TTE (%) | ||||
|---|---|---|---|---|---|---|---|---|
| Pre-EI | Post-EI | Change | Pre-EI | Post-EI | Change | |||
| A (5) | 3/4 (75.0%) | (50.0%) | 77.4 | 84.2 | +6.8 | 6.8 | 4.5 | −2.3 |
| B (28) | 19/28 (67.9%) | (56.3%) | 77.4 | 81.2 | +3.8 | 4.9 | 2.4 | −2.5 |
| C (5) | 3/5 (60.0%) | (61.7%) | 51.6 | 64.3 | +12.7 | 34.8 | 18.5 | −16.3 |
| D (16) | 6/15 (40.0%) | (16.1%) | 63.2 | 67.3 | +4.1 | 10.5 | 19.3 | +8.8 |
| Overall (54) | 31/52 (59.6%) | (45.3%) | 73.4 | 76.5 | +3.1 | 9.2 | 7.5 | −1.7 |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
