Current issues of ACP Journal Club are published in Annals of Internal Medicine


Identifying systematic reviews: key resources


ACP J Club. 2000 May-June;132:A11. doi:10.7326/ACPJC-2000-132-3-A11

How effective are antibiotics for treating patients with sore throat? The first choice of resource for the practitioner to answer a clinical effectiveness question such as this one should be a systematic review. By systematically searching, assessing, and summarizing the research evidence by using rigorous methods, the best systematic reviews present a time-saving distillation of the current state of knowledge. The number of systematic reviews has been growing and continues to grow rapidly. This growth has fortunately been accompanied by efforts to identify, collect, and present reviews in easy-to-access collections. This editorial presents a strategy for searching for systematic reviews of effectiveness by using a selection of the best of these collections.

The Cochrane Library

The first stage, and often the only necessary stage, in finding an up-to-date systematic review of the effects of health care interventions is to search the Cochrane Library (1). The Cochrane Database of Systematic Reviews (CDSR) in the Cochrane Library contains a unique, cumulative collection of full-text systematic reviews that are valuable not only for their rigorous methods but also for their regular updates, which occur as new research evidence is published and in response to valid criticisms. If searchers are unable to find an up-to-date Cochrane review in the CDSR on their topic of interest, the Cochrane Library has summaries of other quality-assessed reviews (including reviews of diagnostic tests) in the Database of Abstracts of Reviews of Effectiveness (DARE) and abstracts of technology assessments in the Health Technology Assessment (HTA) database. An initial search of the Cochrane Library can save much time and effort in searching other resources. Nonsubscribers to the full Cochrane Library can search abstracts of Cochrane reviews together with DARE and HTA database records free on the NHS Centre for Reviews and Dissemination web site (2).

The most effective approach to searching the Cochrane Library is to use both text words and Medical Subject Headings (MeSH) to ensure that relevant records are not missed. This approach is required because many records are not MeSH indexed. The Cochrane Library offers the facility to look up MeSH thesaurus terms and to build searches that address structured questions (patient, intervention, comparison, outcome [PICO]) (3). Given the size of the full-text records in the Cochrane Library, it is advisable to use the proximity operators (NEAR and NEXT) to ensure that search terms in the retrieved records are related contextually. For example, the following search would find reviews of the effectiveness of antibiotics for sore throat:

#1 antibiotics

#2 antibiotics*:me

#3 (sore next throat*)

#4 pharyngitis:me

#5 (#1 or #2) and (#3 or #4)

In the March 2000 issue this strategy finds a Cochrane systematic review of 22 trials on the topic and indicates that antibiotics have only modest absolute benefits (4). The example shows some useful search features: “*” is a truncation symbol used to retrieve words beginning with “throat,” and “*:me” explodes the MeSH term “antibiotics” to retrieve all of its narrower terms (e.g., “bacteriocins”).

ACP Journal Club and Evidence-Based Medicine

If no relevant Cochrane reviews or DARE abstracts are found, the next best step is to use other publications that assess and comment on high-quality published reviews, although they are not updated like Cochrane reviews. ACP Journal Club and Evidence-Based Medicine are 2 such publications, now available as The print versions of these journals have indexes to aid searching.

Clinical Evidence

Clinical Evidence (5) is a recently launched publication that summarizes the results of reviews and other high-quality evidence to answer specific clinical questions. The book is arranged by disease areas and has an index. The index yields 1 entry for sore throat and describes the results of the Cochrane review found by the Cochrane Library search. However, the version of the review cited in Clinical Evidence, Issue 2 has since been updated in the Cochrane Library, and although the findings have not radically changed, the fact that the most recent version of a Cochrane review will always be in the Cochrane Library reinforces the view that the Cochrane Library should be the first place to search.

U.S. reviews and technology assessments

Internet users have access to an excellent range of full-text evidence reports and evidence-based guidelines from U.S. research agencies at the Health Services/Technology Assessment Text (HSTAT) Web site (6). This site offers a simple, “fill-in-the-form” search option and searches can be restricted to groups of publications.

Indexes to systematic reviews

Although systematic reviews are at the forefront of high-quality evidence and are indisputably valuable, the fact that they are published in a range of print and electronic media can render them difficult to locate. Unfortunately, no single service currently indexes all of the publications listed above. A time lag also exists between publication of reviews and their appearance in any index. Web-based services are available, however, that can aid in the retrieval of systematic reviews.

Turning Research into Practice (TRIP) (7) is an Internet search service that covers a wide range of U.K. and U.S. reviews and evidence-based guidelines. The search interface is unsophisticated and offers limited search refinement. Although it only searches the titles of reviews, it is up-to-date and reasonably comprehensive. It is best to search using a variety of synonyms (e.g., search “sore throat” as well as “pharyngitis”).

ScHARR-Lock’s Guide to the Evidence (8) is intended to complement TRIP and lists references to reviews and guidelines. Optimal searching with this tool involves choosing the appropriate MeSH terms (“pharyngitis” and “antibiotics” in this case) and then proceeding to the relevant parts of the alphabetical listing that is retrieved.

Bibliographic databases

When full-text databases or collections of detailed summaries (such as the Cochrane Library or are not available or do not identify reviews on a topic of interest, the user must step back from the distilled literature and search large bibliographic data-bases. Such databases as MEDLINE, Cumulative Index to Nursing and Allied Health Literature (CINAHL), and EMBASE/Excerpta Medica can be searched for the bibliographic citation and abstracts of reviews, although further steps are then required to assess and obtain the full texts of the reviews. The Ovid Evidence-based Medicine Reviews database offers a unique combination of Cochrane reviews linked to MEDLINE records (9).

Several predesigned systematic review search filters are available for MEDLINE and CINAHL to which the searcher just needs to add specific subject terms. Search filters should ideally be derived by analyzing a gold standard of relevant records and testing the resulting search on another collection of relevant records. The following filter for MEDLINE (Ovid version) has been so derived (10) and has both mid-range sensitivity (55%) and precision (71%).

#1 systematic adj review$.tw.

#2 data adj

#3 published adj studies.ab.

#4 data adj extraction.ab.

#5 meta-analysis/

#6 meta-analysis.ti.




#10 animal/

#11 human/

#12 10 not (10 and 11)

#13 (your subject terms)

#14 13 not (7 or 8 or 9 or 12)

#15 or/1-6

#16 14 and 15

Filters have been produced with different levels of sensitivity and precision according to the requirements of the user in terms of the volume of records retrieved (11). Searchers should be aware that filters can retrieve reviews of varying quality. Most search filters simply focus on themes, such as diagnosis, or on methods, such as reviews, and cannot screen for quality. Quality assessment remains the task of the reader.

Ongoing reviews

Many reviews are in progress, and checking for these may be prudent if an answer is not required immediately. The Cochrane Library contains protocols of Cochrane reviews and technology assessments in progress. The U.K. National Research Register (12) contains records of ongoing reviews in the United Kingdom, and there are Web sites that record ongoing research in the United States (13).

Information services

Finally, if readers cannot quickly find a recent systematic review on the topic of interest, they should contact an information specialist or librarian who has expertise in searching the literature. He or she can help the reader develop the search strategy and can suggest other resources. In some countries, such specialist information services as the NHS Centre for Reviews and Dissemination in the United Kingdom have been funded to provide access to information about systematic reviews.

Looking forward

The increasing quantity of published research syntheses and the impetus to become more evidence based prompts the observation that practice would be best served if the complete knowledge base was gathered together in 1 place. The exciting prospect of this happening during the next few years exists in the United Kingdom with the development of the National Electronic Library for Health (14) and, in particular, its “knowledge floor,” which will provide a single access route to reviews and quality-assessed summaries of reviews.

Julie Glanville, MSc
NHS Centre for Reviews and DisseminationUniversity of York
York, England, UK

Carol Lefebvre, MSc
U.K. Cochrane CentreNHS Research and Development Programme
Oxford, England, UK


1. The Cochrane Library [quarterly CD-ROM and database online]. Oxford: Update Software; 1999.

2. Database of Abstracts of Reviews of Effectiveness (DARE).

3. Booth A, O’Rourke AJ. Evidence-Based Medicine. 1999 Sep-Oct:4;133-6.

4. Del Mar CB, Glasziou PP, Spinks AB. Cochrane Review, latest version 16 Sep 1999. In: The Cochrane Library. Oxford: Update Software.

5. Clinical Evidence. Issue 2. London: BMJ Publishing; 1999.

6. National Library of Medicine’s Health Services/Technology Assessment Text (HSTAT).

7. Turning Research into Practice database.

8. ScHARR-Lock’s Guide to the Evidence.

9. Ovid Evidence-based Medicine Reviews.

10. Boynton J, Glanville J, McDaid D, Lefebvre C. Journal of Information Science. 1998;24:137-57.

11. Search filter collections. and

12. National Research Register (NRR).

13. U.S. ongoing research Web sites. and

14. National Electronic Library for Health (NeLH).