Current issues of ACP Journal Club are published in Annals of Internal Medicine


Systematic reviews of diagnostic test evaluations: What's behind the scenes?


ACP J Club. 2004 Nov-Dec;141:A14. doi:10.7326/ACPJC-2004-141-3-A14

To the Editor

In the interesting editorial by Pai and colleagues (1), an important factor not mentioned is that the inter- and intraobserver variability of the gold standard must be known before one can assess any new test. No new test can be expected to match the gold standard more closely than the inter- and intraobserver variability for that gold standard. This issue has been addressed for the measurement of carotid stenosis (2).

Failure to appreciate this limitation will result in a new test not matching the gold standard, even when the new test is as good, and sometimes better than, the gold standard. Of course, any new test will also have its own inter- and intraobserver variability.

Peter R. Humphrey, MD
Walton Centre for Neurology and Neurosurgery
Liverpool, England, UK


1. Pai M, McCulloch M, Enanoria W, Colford JM Jr.Systematic reviews of diagnostic test evaluations: what's behind the scenes [Editorial]. ACP J Club. 2004 Jul-Aug;141:A11-3.

2. Young GR, Humphrey PR, Nixon TE, Smith ET. Variability in measurement of extracranial internal carotid artery stenosis as displayed by both digital subtraction and magnetic resonance angiography: an assessment of three caliper techniques and visual impression of stenosis. Stroke. 1996;27:467-73. [PubMed ID: 8610315]