Of studies, syntheses, synopses, and systems: the “4S” evolution of services for finding current best evidencePDF
ACP J Club. 2001 Mar-Apr;134:A11. doi:10.7326/ACPJC-2001-134-2-A11
Practical resources to support evidence-based health care decisions are rapidly evolving. New and better services are being created through the combined forces of increasing numbers of clinically important studies, increasingly robust evidence synthesis and synopsis services, and better information technology and systems. The need for these resources is spurred by demands for higher quality at lower cost from health services, but the impact of better information resources is being blunted by noisy pretenders promising “the earth” but yielding just the dirt. Providers and consumers of evidence-based health care can help themselves to best current evidence by recognizing and using the most evolved information services for the topics that concern them.
The Figure provides a “4S” hierarchical structure, with original “studies” at the base, “syntheses” (systematic reviews) of evidence just above the base, “synopses” of studies and syntheses next up, and the most evolved evidence-based information “systems” at the top. Information seekers should begin looking at the highest-level resource available for the problem that prompted their search.
A perfect evidence-based clinical information system would integrate and concisely summarize all relevant and important research evidence about a clinical problem and would automatically link, through an electronic medical record, a specific patient's circumstances to the relevant information. The user would then consult the system—in fact, be reminded by the system—whenever the patient's medical record was reviewed. The information contained in the system would be based on an explicit review process for finding and evaluating new evidence as it is published and then reliably updated whenever important new research evidence became available. The clinician and patient could therefore always have the benefit of the current best evidence. The system would not tell decision makers what to do. Those judgments would need to integrate the system's evidence with the patient's circumstances and wishes (1). Rather, the system would ensure that the cumulative research evidence concerning the patient's problem was immediately at hand. Further, the user's first point of interaction would be a short synopsis to maximize speed of use, but there would be links to syntheses and then to original studies so that the user could delve as deeply as needed to verify the accuracy, currency, and details of the synopsis.
Current systems don't reach this level of perfection as yet, but production models exist for parts of such systems. Electronic medical record systems with computerized decision support rules have been shown in randomized trials to improve the process and sometimes the outcome (2) of care. However, these cover a limited range of clinical problems, are not necessarily based on current best evidence, and are mainly “homebuilt” and thus are not easily acquired in most practice settings.
Given that we have some way to go before current best evidence is integrated into electronic medical records, some excellent, but less-developed systems are now readily available. For example, some electronic textbooks integrate evidence-based information about specific clinical problems and provide regular updating. UpToDate (on CD-ROM and the Internet www.uptodate.com]) is one of the leading examples of an evidence-based textbook for general internal medicine at present. However, it is not explicit about the processes it uses to ensure that all relevant evidence is reviewed, assessed, and included, and it isn't integrated into electronic medical records. Scientific American Medicine (www.samed.com) provides similar referencing and updating and is now connected with the aggregated information services of WebMD (webmd.com). Clinical Evidence (www.evidence.org) is a new contender with an explicit review process; it integrates evidence about prevention and therapy for a broad and rapidly expanding array of clinical problems in all medical disciplines and provides a model for the 4S approach to building information systems firmly based on underpinning studies, syntheses, and synopses. Clinical Evidence is also available on Ovid (www.ovid.com) as a separate title, with integration into Ovid's Evidence-Based Medicine Reviews (EBMR) service promised for 2001.
Although none of these systems are integrated with electronic medical records, they can be run through the same computers that run electronic medical records so that one need not go to a remote location to find them. Unfortunately, connecting the right information to a specific patient's problems requires that clinicians understand evidence-based care principles and that they apply some effort and skill in using the resources. Fortunately, these emerging information systems reduce these burdens considerably.
When no evidence-based information system exists for a clinical problem, then synopses of individual studies and reviews are the next best source. What busy practitioner has time to use evidence-based resources if the evidence is presented in its original form or even as detailed systematic reviews? While these detailed articles and reviews are essential building blocks, they are often indigestible if consumed on the run. The perfect synopsis would provide exactly enough information to support a clinical action. The declarative titles for each abstract that appears in ACP Journal Club and Evidence-Based Medicine represent an attempt at this, for example, “Review: Antibiotics do not lead to general improvement in upper respiratory tract infections.” In some circumstances, this can be enough information to allow the decision maker either to proceed, assuming familiarity with the nature of the intervention and its alternatives, or to look further for the details, which, for an ideal synopsis, are immediately at hand. The full abstract for this item is in Evidence-Based Medicine, with an abstract and commentary on one page. The synopsis in the Table contains the essential information on the treatment effects (or lack thereof, in this case) in a format that could be adopted to wireless palmtop Internet devices.
Table. A prototype for evidence synopsis for hand-held computers*
|Question||Study groups||Outcome (n = 5 studies)||Follow-up||Weighted EER||Weighted CER||RBR (95% CI)||NNT|
|In patients with acute upper respiratory tract infections, what is the efficacy and safety of antibiotics (compared with placebo) in curing infection and improving nasopharyngeal symptoms?||Experimental: antibiotics (tetracycline, penicillin, ampicillin, amoxicillin, erythromycin, and cotrimoxazole) Control: placebo||General Improvement||1 to 7 d||51.2% (-5 to 10)||52.5%||2%||NS|
|Outcome||Weighted EER||Weighted CER||RRI (CI)||NNH|
|Adverse effects||9.7%||3.6% (-25 to 340)||82%||NS|
*Conclusion: In patients with acute upper respiratory infection, antibiotics are no
more beneficial in terms of general improvement than is placebo and are associated
with a nonsignificant increase in adverse effects.
†NS = not statistically significant; RBR = relative benefit reduction. Other abbreviations defined in Glossary; RBR, RRI, NNT, NNH, and CI calculated from data in article.
Based on: Review: Antibiotics do not lead to general improvement in upper respiratory tract infections. Evidence-Based Medicine. 1999 Jul-Aug;4:121. Abstract of: Arroll B, Kenealy T. Antibiotics versus placebo in the common cold. In: The Cochrane Database Syst Rev. 1998;CD000247 (latest version 8 Apr 1998).
If more detail is needed or no synopsis is at hand, then databases of systematic reviews (syntheses) are available, notably the Cochrane Library, which is on CD-ROM, the Internet (www.cochranelibrary.com/), Aries Knowledge Finder (KF) (www.kfinder.com), and Ovid's EBMR service. These summaries are based on rigorous searches for evidence, explicit scientific reviews of the studies uncovered in the search, and systematic assembly of the evidence to provide as clear a signal about the effects of a health care intervention as the evidence will allow. Unfortunately, the Cochrane reviews do not extend to clinical topics other than preventive or therapeutic interventions.
Stimulated by the success of the Cochrane Collaboration, the number of systematic reviews in the medical literature has grown tremendously in the past few years; if the Cochrane Library doesn't have a review on the topic you are interested in, it is worthwhile to look in MEDLINE. Better still, Ovid EBMR and Aries KF provide one-stop shopping for both Cochrane and non-Cochrane systematic reviews. For the example of antibiotics for upper respiratory infections, a search on Ovid's Best Evidence, Cochrane, and MEDLINE databases retrieves 17 items, including a Best Evidence synopsis and 4 relevant Cochrane reviews. Limiting the search to “EBM Reviews” (a check box below the search window in Ovid) cuts the retrieved items down to the latter 5. Cochrane reviews are also now indexed in MEDLINE: “cold and antibiotics and Cochrane Review” brings up 3 items, including Cochrane reviews on antibiotics for acute bronchitis and for the common cold.
If every other S fails (i.e., no system, synopses, or syntheses), then it's time to look for original studies. These can be retrieved on the Web in several ways. Especially if you don't know which database is best suited to your question, search engines tuned for health care content can assemble access across a number of Web-based services. At least one of these search engines is attentive to issues of quality of evidence, namely, SUMSearch (sumsearch.uthscsa.edu/searchform45.htm). Nevertheless, the user must appraise the items identified by such a search to determine which fall within the schema presented here. Many will not, especially when convenience of access is favored over quality. There are also at least 2 levels of evidence-based databases to search directly, specialized and general. If the topic falls within the areas of internal medicine and primary care, then Best Evidence (now acpjc.org) provides a specialized, evidence-based service because the abstractd articles have been appraised for scientific merit and clinical relevance. If the search is for a treatment, then the Cochrane Library includes the Cochrane Controlled Trials Register. SilverPlatter and other bibliographic database companies have specialized versions of MEDLINE. MEDLINE itself is freely available (www.ncbi.nlm.nih.gov/PubMed/), and the Clinical Queries screen provides detailed search strategies that home in on clinical content for diagnosis, prognosis, therapy, and etiology.
If none of these services provides a satisfying result, it is time to go to the main search screen in MEDLINE's PubMed and try there. If you still have no luck and the topic is, say, a new treatment (one your patients have asked about but you don't yet know about), then try Google (www.google.com). It is incredibly fast and can get you to a product monograph in a few milliseconds. At least you will find what the manufacturer of the treatment claims it can do along with detailed information on adverse effects, contraindications, and prescribing. The Google home page allows you to add a Google search window to your Web browser's tool bar. Unless you are a very slow typist, this is the fastest way to get to almost any service on the Internet, including all the ones named in this article that are Web accessible.
It's worth emphasizing that almost all the resources just reviewed are available on the Internet. The added value of accessing these services on the Web is considerable, including links to full-text journal articles, patient information, and complementary texts.
Is it time to change how you seek best evidence?
Compare the 4S approach with how you usually seek evidence-based information. Is it time to revise your tactics? If, for example, it surprises you that MEDLINE is so low on the 4S list of resources for finding current best evidence, then this communication will have served a purpose: Resources for finding evidence have evolved in the past few years, and searches can be a lot quicker and more satisfying for answering clinical questions if the features of your quest match those of one of the evolved services. This is in no way a knock against MEDLINE, which continues to serve as a premier access route to the studies and reviews that form the foundation for all the other more specialized databases reviewed above. Big rewards can be gained from becoming familiar with these new resources and using them whenever the right clinical question presents itself.
R. Brian Haynes, MD, PhD
Hamilton, Ontario, Canada
1. Haynes RB, Sackett DL, Gray JR, Cook DL, Guyatt GH. Transferring evidence from research into practice: 1. The role of clinical care research evidence in clinical decisions [Editorial]. ACP J Club. 1996 Nov-Dec;125:A14-6.
Conflict of interest statement: Brian Haynes has direct or indirect connections with many of the evidence-based resources used as examples above, including ACP Journal Club (editor), Evidence-Based Medicine (co-editor), Best Evidence (editor), the Cochrane Library (reviewer and former board member and Cochrane Centre director), Scientific American Medicine (associate editor), Clinical Evidence (advisory board), and PubMed Clinical Queries (developer). These resources are used to illustrate the concepts in the paper; there are other, and perhaps better, examples.