Study Objective

The objective of this study was to determine whether implementing a strict definition of negative findings on TTE would improve the performance characteristics of TTE sufficiently to exclude vegetation.

Patient Population Derivation

The patient population was derived through a retrospective analysis of the Duke Echocardiography Lab Database (DELD). The setup of the DELD has been previously described. Briefly, the DELD is a prospectively maintained digital archive of all echocardiograms obtained at Duke University Hospital and satellite clinics since 1995. This is linked to a corresponding searchable reporting database with clinical information derived from the electronic health record.

Patient Population

The DELD was searched for adult patients who underwent TTE followed by TEE within 7 days between January 1, 2007, and February 28, 2014. Patients who underwent the studies for any of the following indications were included: history of fever, bacteremia, and IE evaluation. The majority of transthoracic studies (>90%) were performed using the Philips iE33 platform (Philips Medical Systems, Andover, MA), which was fully integrated at our institution by January 1, 2007. The remaining studies were performed using the GE Vivid 7 or E9 platform (GE Healthcare, Little Chalfont, UK). Studies were performed using a 2.5-MHz phased-array probe.

The transthoracic echocardiographic imaging protocol for IE includes multiple zoomed-in views of each valve, two-dimensional sweeps of valves, the use of nontraditional windows to probe abnormal findings, and the use of fundamental frequencies to enhance spatial resolution beyond that of harmonic imaging alone. Patients with prior valve repair or replacement, complex congenital heart disease, prior heart transplantation, or left ventricular assist devices were excluded because of the higher incidence of IE and increased difficulty of imaging in these groups using TTE alone. TEE was performed using the Philips iE33 machine and an omniplane 3.5-MHz phased-array probe using fundamental frequencies.

Definition of Vegetation

A vegetation was defined as an independently mobile or oscillating echogenic target or mass in a heart chamber or valve on an echocardiographic study done for one of the indications listed under “Patient Population.” To identify studies showing vegetations, a string search was performed in the DELD on the “final interpretation” free-text field and valve comments fields for “vegetation,” “target,” “structure,” “material,” “strand,” “density,” and “mass.” Misspellings were accounted for by using only the first few letters of each word to search, followed by a manual review of the results. Studies with mobile targets seen on a valve, without more likely alternative etiologies (e.g., senile degeneration, benign neoplasm, annular calcification), were considered suggestive of vegetation. The results of follow-up TEE were considered the gold standard for presence of vegetation.

Strict Criteria for Negative Results on TTE

Two separate analyses were performed to evaluate the diagnostic characteristics of TTE in the evaluation of IE. In the first analysis, the standard definition of positive or negative results on TTE was used, based solely on the presence or absence of vegetation. For the second analysis, a strict definition of negative findings on TTE was developed, and studies not meeting these criteria were considered to have positive or indeterminate findings. The criteria for strict negative results on TTE are shown in Figure 1 .

Study cohort derivation and strict negative criteria. This figure outlines the derivation of the study cohort and shows the list of strict negative criteria. LVAD, Left ventricular assist device. ^∗ Normal anatomy defined as tricuspid aortic, pulmonic, and tricuspid valves, mitral valve without mitral annular calcification, no mitral valve prolapse, no atrial septal defect or ventricular septal defect, repaired or unrepaired.

The strict negative criteria were designed to identify transthoracic echocardiograms that provide strong evidence against vegetation. Valvular regurgitation is characteristic of IE and has been associated with increased morbidity and mortality in patients diagnosed with IE. Thus, studies with mild or greater regurgitation of any valve were labeled indeterminate. Because serial echocardiographic data may not be available for all patients presenting with suspected IE, the stability of regurgitant lesions was not included in the criteria.

Findings that suggest increased risk for IE were also exclusions in the strict negative criteria. These include valvular stenosis or sclerosis, intracardiac foreign bodies, and anatomic abnormalities such as ventricular septal defects and structural valvular disease, which have all been shown to increase the risk for IE. Large or complex pericardial effusions were also excluded, because these have been associated with both increased rates of IE and worse outcomes when IE is present.

Finally, all studies were required to have adequate technical quality to detect all components of the strict negative criteria, with moderate or better ultrasound quality defined as adequate visualization of anatomic structures, chamber morphology, endocardial borders, and cardiac function from the standard acoustic windows without the need for contrast. Thus, studies without adequate image quality to detect the presence of valvular regurgitation, valvular sclerosis, or the presence of intracardiac hardware would be considered “poor” and not meeting the strict negative criteria. Ultrasound quality is graded at the time of clinical read for all echocardiographic studies performed in our laboratory. This grading, along with a search of all comments in the clinical report for “poor,” “limited,” or “inadequate” visualization of cardiac structures, was used to determine ultrasound quality for each echocardiogram.

Baseline Characteristics and Outcomes Data

Duke Enterprise Data Unified Content Explorer is a Web-based query tool used to extract data generated as a by-product of the care of patients in the Duke University Health System. The cohort of patients generated from the initial query of the DELD was uploaded to Duke Enterprise Data Unified Content Explorer, and clinical data, including prior diagnoses, blood culture results, and outcomes data, were extracted. The available outcomes data applicable to this study were mortality and surgical intervention.

Statistical Analysis

The sensitivity (true positive/condition positive), specificity (true negative/condition negative), NPV (true negative/test outcome negative), and positive predictive value (true positive/test outcome positive) were determined for both the standard and strict negative approaches using the standard 2 × 2 table approach. Condition positive for both approaches was defined by the presence of vegetation on TEE. In the standard approach, test positive was defined by presence of vegetation on TTE, whereas in the strict negative approach, test positive was defined by TTE failing to meet all of the strict negative criteria (positive or indeterminate). Therefore, in the strict negative criteria approach, true positives were transthoracic studies with positive or indeterminate results that subsequently had positive findings on TEE, whereas true negatives were transthoracic studies that met the strict negative criteria and subsequently had negative findings on TEE ( Figure 2 B). Next, negative and positive likelihood ratios (false-negative rate/true-negative rate and true-positive rate/false-positive rate, respectively) and the resulting relationships for each approach between pretest and posttest odds using the Bayes theorem were determined. A test indication curve was then generated to compare the effect of the negative likelihood ratio on posttest probability using the two transthoracic echocardiographic interpretation approaches. Differences in baseline characteristics between the strict negative subgroup and the positive or indeterminate subgroup were determined by using the Fisher exact test or unpaired Student’s t test. Statistical calculations were performed using GraphPad (GraphPad Software, San Diego, CA), and P values < .05 were considered to indicate statistical significance. Results are presented as mean ± SD or values with 95% CIs.

(A) Flow diagram showing the results of using the standard approach. (B) Flow diagram showing the results of using the strict negative approach.

Ethics

This study was performed using only information gathered as part of the routine clinical evaluation of patients at our institution and was carried out under approval of the Duke University School of Medicine Institutional Review Board.