Current issues of ACP Journal Club are published in Annals of Internal Medicine


Diagnosis

An artificial neural network was more accurate for identifying acute myocardial infarction

ACP J Club. 1996 May-June;124:73. doi:10.7326/ACPJC-1996-124-3-073


Source Citation

Baxt WG, Skora J. Prospective validation of artificial neural network trained to identify acute myocardial infarction. Lancet. 1996 Jan 6;347:12-5.


Abstract

Objective

To determine the accuracy of an artificial neural network (ANN) for identifying acute myocardial infarction (MI) in adults with chest pain presenting to an emergency department (ED).

Design

A blinded comparison of the diagnostic accuracy of residents and attending staff with that of an ANN for the diagnosis of acute MI using patient history and physical data.

Setting

A university medical center ED in California, USA.

Patients

1070 adults (73% men, mean age in men 51.5 y and in women 57.3 y) who presented to the ED with anterior chest pain over a 19-month period.

Description of Tests and Diagnostic Standard

Medical residents and attending staff collected information on presenting illness (9 categories and age as a continuous variable), history (4 categories), physical examination (2 categories), and electrocardiogram (5 categories). Chest pain was categorized (noncardiac, angina, unstable angina, or acute MI). The ANN used complete data from 350 previous patients to "learn" how to diagnose acute MI. In 10% of patients, ED factual data were corrected using patient charts before using the ANN. Cardiac enzyme data were used only for the diagnostic standard. The diagnostic standard was ascertained from patient records and telephone calls to discharged participants using standard definitions of acute MI.

Main Outcome Measures

Sensitivity, specificity, and likelihood ratios for positive and negative diagnoses of acute MI made by physicians and the ANN.

Main Results

818 adults had noncardiac chest pain, 102 had angina, 75 had unstable angina, and 75 had acute MI. The sensitivity, specificity, and likelihood ratios of a positive and negative test for physicians were 73% (95% CI 63% to 83%), 81% (CI 79% to 84%), and {3.9 and 0.34}* and for the ANN were 96% (CI 91% to 100%), 96% (CI 95% to 98%), and {24 and 0.04}*, respectively. All patients who were misclassified by the ANN were also misclassified by the physicians.

Conclusion

Using data collected by physicians (variables on presenting illness, history, physical examination, and electrocardiogram), a computer-based neural network program was more accurate than were physicians at diagnosing acute myocardial infarction in adults presenting to the emergency department with anterior chest pain.

Source of funding: Not stated.

For article reprint: Dr. W.G. Baxt, Department of Emergency Medicine, Hospital of the University of Pennsylvania, 3400 Spruce Street, Philadelphia, PA 19104-4283, USA. FAX 215-662-3953.

*Numbers calculated from data in article.


Commentary

The results from the study by Baxt and Skora closely resemble results published 5 years ago from the same ED (1) and from another in the United Kingdom (2). The methods are sound except that the description of the referral mechanisms and use of enzymes to establish the diagnosis of acute MI are unclear, many patients attended late in the evolution of their acute MI (56% showed electrocardiographic changes), and some data were corrected before input to the ANN.

Advantages of nonlinear analytical tools include their ability to uncover high-order relations and their greater accuracy than conventional methods. In this study by Baxt and Skora, an ANN trained on patients with a prevalence of acute MI of 30% retained its accuracy despite a change in prevalence to 8%. Disadvantages of ANNs are that high-quality training and validation data are still required, clinical users cannot understand them (3), and ANNs are unable to quantify their certainty. Prognostic models and ANNs should give atypical patients a mid-range probability or output weight to indicate uncertainty. Most patients in this study, however, were given either very high or low output weights, suggesting that either atypical cases were excluded or the ANN failed to identify them. Current research focuses on merging conventional and nonlinear techniques to reduce these problems (3).

Clinically useful diagnostic or prognostic tools must be plausible, accurate, transferable, and effective (4). Before recommending this ANN for general use, its accuracy in different EDs and the responses of physicians to its output should be measured. It could also be compared with such aids to acute MI diagnosis as new markers of myocardial damage and chemometric techniques. Turning from diagnosis to management of chest pain, a randomized controlled trial measuring the effects of the ANN on thrombolysis, coronary-care-unit bed utilization, and patient outcomes seems appropriate.

Jeremy C. Wyatt, MD
Imperial Cancer Research Fund LaboratoriesLondon, England, UK


References

1. Baxt WG. Use of an artificial neural network for the diagnosis of myocardial infarction. Ann Intern Med. 1991;115:843-8.

2. Kennedy RL, Harrison RF, Marshall SJ, Hardisty CA. Analysis of clinical and electrocardiographic data from patients with acute chest pain using a neurocomputer. Q J Med. 1991;80:788-9.

3. Wyatt JC. Nervous about artificial neural networks? Lancet. 1995;346:1175-7.

4. Wyatt JC, Altman DG. Prognostic models: clinically useful, or quickly forgotten? BMJ. 1995;311:1539-41.