Readers' guide for articles evaluating diagnostic tests: what ACP Journal Club does for you and what you must do yourself
ACP J Club. 1991 Sept-Oct;115:A16. doi:10.7326/ACPJC-1991-115-2-A16
Related Content in the Archives
• Letter: Criteria for the evaluation of diagnostic tests
Articles about diagnostic tests are abstracted in ACP Journal Club only if they meet specified criteria. As a prerequisite, tests must be of interest to a broad range of internists. Thus, readers may need to look elsewhere for studies of diagnostic tests for uncommon disorders.
Subsequent criteria address the believability and applicability of the results of the study. First, the evaluation must include clearly defined comparison groups, at least one of which is free of the disorder of interest. Readers may want to take this one step further, however, and insist that studies include consecutive patients or randomly selected patients for whom the diagnosis is in doubt. Any diagnostic test looks good if obviously normal subjects are compared with those who obviously have the disease in question. In most cases we do not need sophisticated testing to differentiate the normal population from the sick. Rather, we are interested in examining patients who are suspected but not known to have the disease of interest and in differentiating those who do from those who do not. If the patients enrolled in the study do not represent this “diagnostic dilemma” group, the test may perform well in the study, but we should not believe that it will perform well in clinical practice.
The second “believability” criterion insists on an independent, “blind” comparison of the test with the performance of a “gold” standard. Readers should assure themselves that the “gold” standard really does measure a clinically important state. For example, for gastric ulcer, an invasive test, such as gastroscopy, is often used as the gold standard rather than symptoms of dyspepsia alone. The gold standard result should not be available to those interpreting the test. Also, if the gold standard requires subjective interpretation (as would be the case even for gastroscopy), the interpreter should not know the test result. Blinding the interpreters of the test to the gold standard and vice versa minimizes the risk of bias.
If these 2 criteria are met, the study can be used as a basis for performance of the test in clinical practice. Readers must do some additional work, however, to apply the test properly to their patients. Most tests merely indicate an increase or decrease in the probability of disease. To apply imperfect tests appropriately, readers must estimate the probability of disease before the test is done (“pretest probability”), then revise this probability according to the test result.
The clinician's estimation of pretest probability is based on the patient's history, physical examination and initial testing, and the clinician's own experience with this type of problem. Thus, it is usually not possible to “look it up.” Although forming accurate estimations from examination and experience may sound like a daunting task, it is what we implicitly do; we just do not usually make the estimates explicit. The pretest probability is the basis for incorporating the test result. Readers can use the pretest probability from the ACP Journal Club abstract as a guide, especially if the patients were randomly selected from a defined group or a consecutive series and the clinical setting was similar to the reader's. Even then, the findings from the patient must be taken into account.
Clinicians will find that the best way to use a diagnostic test is if they know the extent to which a particular test result increases or decreases the probability that the patient has the target condition. This information is captured in the “likelihood ratios” associated with each range of test results. You will find that for most diagnostic tests reported in the ACP Journal Club, we will provide you with the likelihood ratios. Then, using a simple nomogram (1), you can go from the pre-test likelihood and use the likelihood ratio to find the post-test likelihood. One of the papers in our series about using the medical literature to solve patient problems gives a detailed explanation of this process (2).
Gordon H. Guyatt, MD, MSc
2. Jaeschke R, Guyatt GH, Sackett DL for the Evidence-based Medicine Working Group. A users' guide to the medical literature. III - How to use an article about a diagnostic test. Part B. What are the results and will they help me in caring for my patient. JAMA 1994;271:703-7.
Figure: Nomogram for interpreting diagnostic test results
Reproduced with permission from Fagan TJ. Nomogram for Baye's theorem. N Engl J Med. 1975;293:257.