2 out of 9 computer programs performed as well as cardiologists in the interpretation of electrocardiograms
ACP J Club. 1992 May-June;116:86. doi:10.7326/ACPJC-1992-116-3-086
Willems JL, Abreu-Lima C, Arnaud P, et al. The diagnostic performance of computer programs for the interpretation of electrocardiograms. N Engl J Med. 1991 Dec 19; 325:1767-73.
To compare the accuracy of computer programs (CPs) and cardiologists in the diagnosis of ventricular hypertrophy and myocardial infarction from electrocardiograms (ECGs).
Interpretations of ECGs made by CPs and by cardiologists were compared with independent clinical diagnoses.
5 European centers.
ECGs were obtained from 1220 white, adult patients. Their diagnoses, based on echocardiography or cardiac catheterization, were ventricular hypertrophy (n = 291), or myocardial infarction (n = 516). 31 patients had both hypertrophy and infarction. Patients with primary cardiomyopathy were excluded. Controls (n = 382) were patients with no evidence of hypertrophy or infarction based on history, physical examination, and, in 96 patients, cardiac catheterization.
Description of tests and diagnostic standard
Standard ECG leads and the orthogonal X, Y, and Z leads were recorded simultaneously. ECGs showing major intraventricular conduction defects and those of poor quality were excluded. The diagnosis with the highest level of certainty made by each of 9 CPs was included in the study analysis. 8 cardiologists, blinded to clinical data except age and sex, also interpreted the ECGs. The standard for diagnosis was the consensus of 3 additional cardiologists from clinical evidence without reference to ECGs.
In comparison with cardiologists, CPs classified fewer ECGs correctly (CP median 70%, range 62% to 77% vs cardiologists' median 76%, range 73% to 81%, P < 0.001). However, 2 of the CPs were as accurate overall as the cardiologists (Hannover 76% and Leuven 77%). The median sensitivity of CPs was lower than that of cardiologists' ECG interpretations (P < 0.02) for left ventricular hypertrophy (57% vs 64%), right ventricular hypertrophy (32% vs 47%), anterior myocardial infarction (77% vs 85%), and inferior myocardial infarction (59% vs 72%). The percent of controls correctly classified by the CPs (median 91%, range 86% to 97%) was also lower than that of the cardiologists (median 96%, range 93% to 98%). The average CP and the average cardiologist both disagreed with the clinical classification in 27% of cases.
2 out of 9 computer programs performed as well as cardiologists in the interpretation of electrocardiograms for identifying 7 major cardiac disorders.
Source of funding: European Commission Medical and Public Health Research Programmes.
Address for article reprint: Dr. J.L. Willems, University Hospital Gasthuisberg, 49 Herestraat, 3000 Leuven, Belgium.
This study by Willems and colleagues is as important for its methods as it is for its results (1). The study's methods represent an important benchmark for evaluating the performance of clinically oriented computer programs. The study involved assessment of the computer programs by a team not associated with their development, independent cardiologic interpretation of the same ECGs, and independent clinical diagnosis of ventricular hypertrophy and myocardial infarction.
The results, comparing the ECG interpretations of 8 cardiologists and 9 computer programs, indicate that, for the moment, highly skilled cardiologists still outperform computer programs in the diagnosis of ventricular hypertrophy and myocardial infarction. But the computer systems are not far behind. These results are noteworthy because several of the computer programs included in the study are widely marketed in the United States and Canada. The study did not address important areas of ECG interpretation such as the detection of acute ischemic changes or the diagnosis of arrhythmias, in part because diagnostic standards independent of ECG were not available. Thus, generalization of the results to all areas of ECG interpretation should not be attempted.
Computer programs should, in general, be used only as tools to support clinicians' judgment (2), just as the stethoscope augments a doctor's hearing but does not replace what goes on between the ears. The study shows that the best computer-assisted ECG interpretation can now be of value to the average clinician (at least for certain categories of ECG diagnoses), especially when a cardiologist is not readily available. Further progress in computerized ECG interpretation is also quite likely, and the best machines now may be obsolete within a few years. Thus, readers should watch for future evaluations as well.
Randy A. Miller, MD
University of PittsburghPittsburgh, Pennsylvania, USA
Randy A. Miller, MD
University of Pittsburgh
Pittsburgh, Pennsylvania, USA