Readers' guides for therapy: Was the assignment of patients to treatments randomized?
ACP J Club. 1991 May-June;114:A12. doi:10.7326/ACPJC-1991-114-3-A12
ACP Journal Club's criteria for reviewing and selecting articles on prevention or treatment include an insistence on “random allocation of participants to comparison groups” (seePurpose and Procedure). The application of this guide leads us to restrict the entries in our Therapeutics section to randomized clinical trials. Like other decisions with ultimate clinical consequences, this one produces both good and harm, and the weighing of the former against the latter provides a useful topic for this editorial.
The good follows from the ability of random allocation to limit our risks for both the false-positive conclusion that a treatment is efficacious when it is not, and the false-negative conclusion that a treatment is not efficacious when it is. This happy state is achieved by balancing, through random allocation, the treatment and control groups for characteristics that influence their prognosis in important ways (for example, hypertension worsens the prognosis of patients with symptomatic carotid stenosis), yet are extraneous to the question posed (in this case: “Would carotid endarterectomy reduce the risk for subsequent stroke and death in symptomatic patients, regardless of their baseline blood pressures?”).
But why the insistence on randomization, given that there are six other, and often easier, ways to overcome this problem of “confounding”? These include
• exclusion (throwing out all hypertensives);
• stratified sampling (selectively including and excluding individual patients until operated and nonoperated patients included identical proportions of hypertensives);
• individual matching (pairing every operated patient with a nonoperated patient of similar blood pressure);
• stratified analysis (generating four separate rates of subsequent stroke and death for operated and nonoperated, hypertensive and normotensive patients);
• standardization (applying these four separate rates of subsequent stroke and death to a single “standard” population containing an arbitrary proportion of hypertensives, thereby reducing the four rates to just two, one for operated and one for nonoperated patients); and
• multivariate modeling (applying powerful statistical models in which one can “adjust” not only for hypertension, but for as many other confounders as the investigators can imagine and their study budget can afford to measure).
The reason for the insistence on randomization is simple but mighty: All the other six methods of overcoming confounders require that we know all important confounders before the study begins; conclusions reached through their application are only as reliable as current understandings of all important elements of the disease under attack.
Randomization does not require such hubris; it balances treatment groups for unknown as well as known confounders. Thus, although the credibility of the results of the six varieties of subexperiments fails with the next material contribution to our understanding of the disease in question, the credibility of the randomized experiment endures.
As forewarned in the opening paragraph of this editorial, there is a harm that accompanies the good of restricting therapeutic articles to randomized trials: Efficacious therapies that have not yet been validated in randomized trials will go unreported in ACP Journal Club. In limiting the risk for the false-positive dissemination of useless and harmful treatments, our gains in specificity are accompanied by losses in sensitivity, for our policy occasionally will impose (until the execution and reporting of the relevant randomized trial) a false negative silence on treatments destined to be shown to be efficacious. If we (or anybody else!) had the prescience required to report just those incompletely tested treatments that were foreordained to do more good than harm, we could add another department to ACP Journal Club (perhaps titled: “Trust Us”!) and shorten the period of false-negative silence. Barring that eventuality, the current restrictive policy will produce more good than harm in the management of all our patients.
David L. Sackett, MD, MSc
This is the first in a series of editorials that will describe the reasons behind the criteria that we have chosen for selection of articles for abstraction in ACP Journal Club.
R. Brian Haynes, MD, PhD
Hamilton, Ontario, Canada