Background
Although the use of facilitated reporting to generate echocardiographic reports has many advantages over traditional dictation and transcription, it is not perfect, because physicians can still select finding codes that are in conflict with each other.
Methods
The investigators developed a tool allowing the creation of combinations of finding codes that should not be (contradictory) or are typically not found (inconsistent) together and then analyzed a historical collection of finalized echocardiographic reports to determine the frequency with which reporting discrepancies were present.
Results
A total of 96,772 reports were evaluated over an 11-year period. The frequencies of contradictory findings in final reports were 4.0%, 3.6%, and 7.1% for transthoracic echocardiography, transesophageal echocardiography, and stress echocardiography, respectively. Nearly a quarter (24.1%) of finalized transthoracic reports and 10.1% of transesophageal reports had findings that were inconsistent with each other.
Conclusions
This study demonstrates that facilitated reporting of echocardiographic studies, using a discrete set of finding codes, allows the generation of rules that can be used to identify discrepancies and alert readers to the need to correct or review their interpretations.
Facilitated reporting (FR) of echocardiographic studies, in which a physician selects from a menu of statements (“finding codes”) that describe study findings to generate a report, has proven valuable. A finding code is typically a short text statement, such as “normal left ventricular [LV] systolic function,” which when selected from a drop-down menu triggers a more complete sentence, such as “The LV systolic performance is within normal limits,” to appear in the printed report. The final report is generated sentence by sentence by selecting all the appropriate finding code statements. Finding codes themselves are a discrete set of phrases that are standard to an institution or practice group. The resulting report is stored electronically in a medical information system. This report generation technique provides for consistency among readers, improved report organization, enhanced data mining, and quicker turnaround of finalized reports. Although avoiding the discrepancies inherent in the dictation and transcription process, FR is not perfect, because physicians can still select finding codes that are in conflict with each other.
Discrepancies generated during FR may represent the presence of contradictory reporting elements, such as “mild mitral regurgitation” and “moderate mitral regurgitation” present in the same report. More subtle discrepancy detection reflects a more advanced relationship between the elements of a report and can be used to point out potential inconsistencies in the report. For example, although “severe mitral regurgitation” and “normal size left atrium” may coexist in some clinical situations, in the vast majority of cases, the presence of both in the same report is likely to be erroneous.
Although there is agreement on what should be included in an echocardiographic report, there is no consensus on the set of finding codes used to generate reports among different institutions. This lack of standardization makes it difficult to develop a set of standard “rules” to detect report inconsistencies. We have developed a tool that allows us to interactively create rules that formalize the relationships among distinct finding codes that are contradictory and should not appear together in a medical report, along with a rules engine that evaluates the rules against a report. In a similar fashion, this tool can be used to create combinations of finding codes that are typically not found together, alerting users to inconsistencies and perhaps the need to review their interpretations.
Having created this set of rules, we sought to analyze a historical collection of finalized echocardiographic reports and determine the frequency with which reporting discrepancies were present.
Methods
A prototype tool was developed that allowed users to easily generate rules consisting of finding code combinations that are to be flagged for appearing in a report together. The tool allows users to view all finding codes in the reporting application that are available to an interpreting physician and select any pair of finding codes to represent a rule violation or “conflict.” Because the group of finding codes is different for transthoracic echocardiographic (TTE), transesophageal echocardiographic (TEE), and stress echocardiographic report generation, different rule sets were made for all 3 of these modalities.
For both TEE and TTE reports, two different rule sets were generated. The first consisted of finding code combinations that were considered contradictory and should never appear together in the same report. These rule sets were defined as “contradictory.” Examples of this type of conflict are “normal LV size” appearing with “moderate LV dilatation” and “no left atrial appendage thrombus” appearing in the same report as “left atrial appendage thrombus was present.” A second set of rules for TTE and TEE reporting consisted of finding codes that, although possible, are physiologically unusual to occur together. These rule sets were defined as “inconsistent.” Examples of this type of conflict are “severe tricuspid regurgitation” with “normal size right atrium” and “severe LV diastolic dysfunction” with “normal size left atrium.” For stress echocardiographic reporting, only contradictory finding code rules were generated.
Once created, the rules were then applied retrospectively to a database of clinically generated reports (Xcelera; Philips Medical Systems, Andover, MA) from the adult echocardiography laboratory at the University of Chicago. During the retrospective time period evaluated, no dictation was used to generate reports, although the interpreter could type free text in open-comment boxes. Because the free-text statements were not structured finding codes, they were not evaluated for contradictions or inconsistencies. The incidence of each discrepancy type for each type of report template (TTE, TEE, and stress echocardiographic reports) was computed.
Results
For TTE reporting, 2608 rules (2116 contradictory and 492 inconsistent) were created from 752 finding codes. For TEE reporting, 1530 rules (1190 contradictory and 340 inconsistent) were generated from 635 finding codes. Stress echocardiographic reporting had 1280 rules complied from 205 distinct finding codes. A total of 96,772 reports were evaluated over an 11-year period, from May 1998 to June 2009.
Of 79,542 adult TTE reports reviewed, 4.0% contained findings that were mutually exclusive, indicating that contradictions were included in the final reports ( Table 1 ). Of the studies with contradictions, most (85%) included single contradictory report finding pairs. Of 6988 TEE reports, 3.6% contained ≥1 contradictory rule violation. After evaluating 10,242 stress echocardiographic reports, 7.1% of the reports were found to have pairs of findings that both could not be true. Stress reports were more likely than TTE or TEE reports to have contradictory findings and more likely to have >1 contradiction per report ( P < .01).
| Variable | TTE reports | TEE reports | Stress echocardiographic reports |
|---|---|---|---|
| Number of studies examined | 79,542 | 6988 | 10,242 |
| % of reports with ≥1 discrepancy | 4.0% | 3.6% | 7.1% ∗ |
| Median number of discrepancies per report | 1 | 1 | 1 |
| % of reports with 1 discrepancy | 85% | 93% | 53% ∗ |
| % of reports with >1 discrepancies | 15% | 7% | 47% ∗ |
Of the 2116 contradictory TTE rules, 1570 (74%) were never violated during the 11-year retrospective period. Of the 546 contradictory rules that were violated at least once, no single rule accounted for >4.6% of all violations. However, there were rules that were far more frequently violated than others. The top decile of most frequent rule violations accounted for 59% of all violations, and the top quartile represented 79% of the total number of reported mistakes. The most common contradictory TTE discrepancies in finalized reports for each structure of the heart are shown in Table 2 .
| No LV regional wall motion abnormalities were noted | Regional wall motion abnormalities cannot be excluded |
| RV performance is normal | The right ventricular performance is mildly reduced |
| The LA is normal sized | Left atrial volume when indexed for body size is mildly increased |
| The MV is structurally normal | The mitral valve chordae are thickened and/or calcified |
| No significant TV disease | There is mild-moderate tricuspid regurgitation |
| No significant AV disease | There is mild aortic regurgitation |
| There is no significant PV disease | There is moderate pulmonic valvular regurgitation |
| The aortic root is normal in size | There is mild aortic root dilatation |
| The echocardiographic findings demonstrate no significant hemodynamic compromise from the pericardial effusion | Diastolic compression of the right atrium is present |
Of the 1190 contradictory TEE rules, 924 (78%) were never violated during the retrospective period. Of the 266 contradictory rules that were violated at least once, the most commonly violated rule was “mild atheromatous disease of the ascending aorta or aortic arch” appearing with “moderate atheromatous disease of the ascending aorta or aortic arch.”
Of the 1280 contradictory stress echocardiographic rules, 970 (76%) were never violated during the retrospective period. Of the 310 contradictory stress rules that were violated at least once, the most commonly violated rule was “dobutamine stress echocardiogram was performed” appearing with “blood pressure response to exercise was normal.”
When the report database was examined, nearly a quarter (24.1%) of finalized TTE reports had findings that were inconsistent with each other ( Table 3 ). The frequency of inconsistencies was higher than that of contradictions, as was the number of studies with >1 discrepancy present (39% inconsistent vs 15% contradictory, P < .01). Of the 492 inconsistent TTE rules, most were violated at least once (75%). The most commonly found inconsistent finding code pairs are shown in Table 4 . The rate at which TEE reports had inconsistent elements was 10.1%, which is statistically less frequent than TTE reports. The majority of TEE reports had a single inconsistency (84%). Of the 340 inconsistent TEE rules, 32% were violated at least once.
| Variable | TTE reports | TEE reports |
|---|---|---|
| Number of studies examined | 79,542 | 6988 |
| % of reports with ≥1 discrepancy | 24% | 10.2% ∗ |
| Median number of discrepancies per report | 1 | 1 |
| % of reports with 1 discrepancy | 61% | 84% |
| % of reports with >1 discrepancies | 39% | 16% |
Stay updated, free articles. Join our Telegram channel
Full access? Get Clinical Tree
