Current issues of ACP Journal Club are published in Annals of Internal Medicine

Resource Corner

Statistical Evidence in Medical Trials: What Do the Data Really Tell Us?


ACP J Club. 2007 Jan-Feb;146:A10. doi:10.7326/ACPJC-2007-146-1-A10

After reading the first paragraph of Statistical Evidence in Medical Trials: What Do the Data Really Tell Us? I was immediately drawn in, mesmerized, and entertained right through to the end of the book. That says a lot for a book about statistics! It reads better than some of my bedside novels, but is not at all fictional. The book begins with a statistical joke—a perfect introduction for both the serious statistician and the more lighthearted clinician—that sets the tone for the remainder of the book. Many chapters are prefaced with a comical pictorial or word sketch that hints at the content to follow, while making the reader hungry enough to read on. That the injection of humor was possible in a statistics book is in itself quite enlightening. This point alone makes the book a worthwhile read, but the worth does not stop there.

The primary purpose of this book is to provide an understanding of how medical literature should be interpreted, in spite of its limitations. The concepts are useful for beginners to experts. In the words of the author, “this book is for consumers, not producers, of medical research.”

The book grew from the experience of the author, a statistician, who has provided training and expertise in interpreting medical literature in the hospital setting. The major thesis is that “you should worry more about how the data were collected rather than how it was analyzed.”. Fittingly, then, the author refrains from using numbers and formulas throughout the book. Concepts are explained with words and pictures so effectively that the absence of numbers goes unnoticed. As a result, one comes away with an understanding of the very essence—the conceptual core-of what is important for interpreting clinical trials. This is much more than can be said for many books about statistics in medicine.

In fewer than 200 pages and 7 chapters, the book covers a sizable range of topics necessary for discerning the literature. Chapter 1—Apples or Oranges? Selection of the Control Group, discusses the risk for unfair (apples-to-oranges) comparisons and how to detect them across a variety of study designs. Chapter 2—Who Was Left out? Exclusions, Refusals, and Drop-outs, discusses the implications of selective recruitment, purposeful exclusions of troublemakers, and the vexing problem of incomplete follow-up. Chapter 3—Mountain or Molehill? The Clinical Importance of the Results, discusses how to detect whether trial results can be considered worthy enough to change your practice, or whether they are trivial. Chapter 4—What Do the Other Witnesses say? Corroborating Evidence, addresses how studies should be interpreted in the context of whether other evidence (or its absence) corroborates or detracts from its message. Chapter 5—Do the Pieces Fit Together? Systematic Overviews and Meta-Analyses, discusses ways to properly assess the totality of the evidence when > 1 study exists. Chapter 6—What Do All These Numbers Mean? explains such concepts as confidence interval, odds ratio, number needed to treat, and correlation, without ever resorting to statistical jargon or complex formulae. Chapter 7-Where is the evidence? Searching for Information, offers a simple strategy for finding clinical trials of highest quality in response to well-built clinical questions.

Each subtopic presented is accompanied by a short explanation and ≥ 1 example from the literature. Excerpts from open-access journal articles are used as examples so that readers can access the full-text freely if desired. The author provides a good balance of point–counterpoint discussions for areas that remain controversial (e.g., blinding is important, but can be overrated). At the end of each chapter, key points are summarized for easy reference. The author's Web site provides further examples and opportunity for more advanced learning.

The reader should be forewarned about 2 detractions in the book. First, there are a number of flaws in the wordsmithing and grammar. Second, some of the medical terminology or clinical explanations are less than perfect, which is not entirely surprising given that Simon himself admits up front that he is not a clinician. However, I found that both of these limitations were easy to overlook, and other benefits far outweighed any detractions.

Clearly this book is not just another statistics book. It borders on the side of being revolutionary—a statistics book without numbers! While this might be considered near sacrilege in the world of pure statistics, for the purposes of inciting balanced, practical, evidence-based clinical decision making, it is nearly a 5-star resource. The tasteful humor injected throughout the text is just the perfect spoonful of sugar to make the medicine go down.

Janet Martin, PharmD
London Health Sciences Centre
London, Ontario, Canada


Clinical Usefulness: 5 of 5 stars

Methods/Quality of Information: 4 of 5 stars

Statistical Evidence in Medical Trials: What Do the Data Really Tell Us? is available online at for US $44.50.