Current issues of ACP Journal Club are published in Annals of Internal Medicine


Beyond ACP Journal Club: how to harness MEDLINE for review articles

ACP J Club. 1996 May-June;124:A12. doi:10.7326/ACPJC-1996-124-3-A12

With this editorial, we continue the series on “how to harness MEDLINE” for clinical problems (1-4). Previous editorials in this series have discussed the use of MEDLINE in finding methodologically sound primary studies on the therapy 1), diagnosis (2), etiology (3), and prognosis (4) of disease. We now turn to the secondary literature of review articles, including overviews and meta-analyses. Individual studies are seldom definitive, particularly if they are small or do not provide a clear consensus. But reviews, if done systematically (5) with a cumulative summary of the evidence, can provide clearer direction for practitioners. Not all reviews are of similar quality. Just as for the primary studies of therapy, diagnosis, etiology, and prognosis, many reviews on a topic can exist, but not all of them warrant making changes in clinical practice.

As with the previous editorials, we begin with a clinical scenario. You are a woman of 40-something years of age, a general internist, and you are having lunch in the cafeteria with some residents. One resident asks you about your approach to hormone replacement therapies (HRTs) for postmenopausal women. You automatically start to tell her what you prescribe in your practice, but she interrupts you to ask whether you are going to take HRT yourself. You suddenly realize that you have not made a personal decision. You are paged to the clinic and do not have time to answer her question.

That evening you have a few minutes and recall the question. The topic is complex. HRT certainly reduces troublesome menopausal symptoms and improves quality of life. The decision to use hormones, however, requires consideration of family history, risk factors, and personal preferences, along with both the protective and potentially harmful aspects of long-term HRT. You decide to search the literature for articles that adequately address this risk-benefit trade-off. Starting with the current MEDLINE file, searched with OVID CD-ROM (1992 through January 1996), you find 1853 articles that are indexed with the medical subject heading (MeSH) “estrogen replacement therapy.” HRT is the major emphasis of 1304 of these articles. The retrieval includes 216 clinical trials, 173 cohort studies, 62 case-control studies, and 273 review articles. The remaining 580 papers are editorials, letters, case reports, animal studies, and others. With this multitude of studies on the topic, you decide that perhaps a systematic review that summarizes the evidence would be desirable. You look at a few of the review article titles but soon grow discouraged with looking for a good summary on HRT because of the large number of citations retrieved.

Your next approach is Scientific American Medicine (SAM) (6). The index for the 3-volume print version indicates that HRT is discussed fully on 5 of the pages that are within the 18-page chapter on the ovary that is part of the endocrine section. It is a useful summary but has not been updated since mid-1993. In addition, you are daunted by the fact that various aspects of HRT are discussed throughout the other 3 volumes of SAM (e.g., endometrial cancer risk is described in 6 other chapters). You also feel that, although risks and benefits are discussed, the quality of evidence you are looking for is not there.

You then remember your recent subscription to ACP Journal Club on Disk, with the cumulated contents of ACP Journal Club from 1991 to Nov/Dec 1995 (call American College of Physicians [ACP] at 800-523-1546 for information about disk subscriptions). You search the index using the MeSH term “estrogen replacement therapy” and retrieve 4 articles. (If you searched using a textword approach with the words “estrogen” and “postmenopausal,” your retrieval would include an additional 5 articles.) An article titled “Estrogen and progestin for postmenopausal women: a meta-analysis” by Grady and colleagues (7) appears to have the information you are seeking. It is a systematic review that includes odds ratios and relative risks for breast and endometrial cancer, coronary heart disease, hip fractures, and data on differences in mortality rates. In addition, the citation at the end of the commentary is to an ACP clinical practice guideline (8) titled “Guidelines for counseling postmenopausal women about preventive hormone therapy.”

Traditional review articles are summaries of literature and experience written by someone with expertise in the content area. Some of these review articles are valuable contributions to medical knowledge, but many are not because they do not include all the relevant evidence or because they inextricably mix the opinions and biases of the authors with the evidence. Systematic reviews and meta-analyses are often a vast improvement over traditional review articles. Standards for judging the quality of review articles (5) include a clear objective; a methods section that describes how studies were sought, selected, appraised, and analyzed; and a results section that provides the individual and combined results of the studies. Meta-analyses form a special subclass of systematic review articles in which the results of individual studies are combined, using a common outcome such as survival, and then summarized statistically across studies (5).

Both the Grady and colleagues' meta-analysis and the ensuing ACP clinical guideline are on target in your quest for a good, evidence-based summary for HRT decision making, but they are a few years old. You wonder if any more recent meta-analyses or systematic review articles on HRT exist. Going back into MEDLINE using the following set of terms, you retrieve 20 articles:

estrogen replacement therapy (as a major concept)


meta-analysis (pt) or [review (pt) and all MEDLINE (textword)]*



By the look of their titles, 2 of the 13 meta-analyses appear to be broad overviews on all aspects of HRT (9, 10), whereas the others cover more specific aspects of HRT (12 on breast, colorectal, or endometrial cancer; 3 on cardiovascular disease; and 3 on the use of various HRT combinations). The systematic review by McKoen (9) is a good summary of the evidence and one that you could also use in your discussion with patients.

The National Library of Medicine also produces a complementary database to MEDLINE called HSTAR (Health Services and Technology Assessment Research). It includes citations to studies and reports that evaluate health care services and technology or that can be used to evaluate them. Subject areas include the relations among needs, demand, supply, use, organization, cost, financing, and outcome of health services; quality assessment and assurance; health technology assessment; practice guidelines; and planning for health services. This database covers material from 1985 and includes citations from MEDLINE and other databases, along with listings of conference papers and abstracts. Running the meta-analysis search strategy in HSTAR retrieves 23 citations. The additional 5 citations are 4 that are in the MEDLINE files before 1992 and 1 citation to a published abstract. MEDLINE does not include citations that are published only in abstract form.

The Table includes search strategies for systematic review articles and meta-analyses that can be used in either the MEDLINE or HSTAR databases. These strategies have not been tested against a “gold standard,” for example, hand searches of the literature. Better search strategies may emerge when such studies are done. In the meantime, we would appreciate any feedback on these approaches. Also, please let us know of other searching or retrieval topics you would like covered. Our ideas include how to search for clinical practice guidelines, tips on very comprehensive literature searching (in contrast to the focused clinical searching we have described), and guides to health care information on the Internet.

Table. Searching Strategies for Review Articles and Meta-analyses

Best simple search:
meta-analysis (pt) or [review (pt) and MEDLINE (textword)]
To maximize retrieval:
meta-analysis (pt) or overview (textword) or meta-analysis (textword) or metaanalysis (textword) or [review (pt) and MEDLINE (textword)]

K. Ann McKibbon
Cindy J. Walker-Dilks
Nancy L. Wilczynski
R. Brian Haynes

*The rationale for using the terms review (pt) and MEDLINE (textword) is to retrieve review articles that describe their data collection methods that typically include a MEDLINE search, which is an indicator of good-quality, evidence-based reviews (5).


1. McKibbon KA, Walker CJ.Beyond ACP Journal Club: how to harness MEDLINE for therapy problems. ACP J Club. 1994 Jul-Aug;121:A10-2.

2. McKibbon KA, Walker CJ.Beyond ACP Journal Club: how to harness MEDLINE for diagnostic problems. ACP J Club. 1994 Sep-Oct;121:A10-2.

3. Walker-Dilks CJ, McKibbon KA, Haynes RB.Beyond ACP Journal Club: how to harness MEDLINE for etiology problems. ACP J Club. 1994 Nov-Dec;121:A10-1.

4. McKibbon KA, Walker-Dilks C, Haynes RB, Wilczynski N.Beyond ACP Journal Club: how to harness MEDLINE for prognosis problems. ACP J Club. 1995 Jul-Aug;123:A12-4.

5. Oxman AD, Cook DL, Guyatt GH, for the Evidence-Based Medicine Working Group. Users' guides to the medical literature. VI. How to use an overview. JAMA. 1994;272:1367-71.

6. Dale DC, Federmann DD, eds. Scientific American Medicine. New York: Scientific American Inc.; 1978-1996.

7. Review: Long-term estrogen therapy increases the risk for breast and endometrial cancer but decreases the risk for coronary heart disease and hip fracture in postmenopausal women in primary prevention trials. ACP J Club 1993;118:65.

8. American College of Physicians. Guidelines for counseling postmenopausal women about preventive hormone therapy. Ann Intern Med. 1992;117:1038-41.

9. McKeon VA. Hormone replacement therapy: evaluating the risks and benefits. J Obstet Gynecol Neonatal Nurs. 1994; 23:647-57.

10. Session DR, Kelly AC, Jewelewicz R. Current concepts in estrogen replacement therapy in menopause. Fert Steril. 1993; 59:277-84.