Of studies, syntheses, synopses, summaries, and systems: the “5S” evolution of information services for evidence-based health care decisionsPDF
ACP J Club. 2006 Nov-Dec;145:A8. doi:10.7326/ACPJC-2006-145-3-A08
Success in delivering evidence-based health care relies heavily on the ready availability of current best evidence about diagnosis, treatment, and prevention options for health disorders, ideally tailored to the characteristics and context of the individual patient or population and the resources of the provider. While existing information resources fall short of perfection, the past decade has seen considerable progress, and an attractive array of services is now available for many health care decisions. Providers and consumers of evidence-based health care can help themselves to the best current evidence by recognizing the most “evolved” information services in the topic areas of concern to them.
A “4S” model for the organization of evidence-based information services, proposed several years ago (1), begins with original studies at the foundation; syntheses (that is, systematic reviews, such as Cochrane Reviews) at the next level up; then synopses (very brief descriptions of original articles and reviews, such as those that appear in the evidence-based journals); and the most evolved services, systems (such as computerized decision support systems that link individual patient characteristics to pertinent evidence) at the top.
George Box, an industrial statistician, once pointed out that “All models are wrong, some are useful” (2), and so it is with the 4S model. Conceptually, this model has been useful for both describing and guiding the development of evidence-based information services, and it has also been wrong in oversimplifying the relationship of these services to original studies. In this editorial we add a layer to the model, namely, clinical topic summaries of evidence about all pertinent management options for a health condition, such as those included in Clinical Evidence and PIER. A second purpose of the editorial is to explore how the layers are relevant to clinical decisions in ways that may not be apparent in the model.
The 5S model
The augmented “5S” model is shown in the Figure. This model's additional layer, summaries, resides between synopses (succinct descriptions of an individual study or a systematic review) and systems (decision support services that match information from individual patients with the best evidence from research that applies). Summaries integrate best available evidence from the lower layers (drawing on syntheses [i.e., systematic reviews] as much as possible) to provide a full range of evidence concerning management options for a given health problem (e.g., acute coronary syndromes [ACS]). The lower layers, synopses, syntheses, and original studies, most often examine only one aspect of management (e.g., a specific drug or drug class for ACS, such as angiotensin-converting enzyme inhibitors), leaving decision makers to do their own integration and, for original studies, their own critical appraisal of the evidence. If a current topic summary exists, it would summarize the relevant synopses, syntheses, or studies about several aspects of a health condition. Thus, a current summary“trumps” an individual synopsis, synthesis, or study or a collection of these.
The only more compiled source would be a system, such as an electronic medical record, in which an individual patient's characteristics were automatically linked to the current best evidence that matched the patient's specific circumstances, with caregivers being reminded or notified of key aspects of management (3). Such computerized decision support systems are currently few and far between, and those in existence often fall short of ensuring that the evidence supporting the system is the best available and kept up to date. Summaries, however, can easily be made universally available (e.g., via the Internet) and it is more feasible to keep them up to date, and provide at least pasive support by automatically linking them to individual patient problems in electronic medical records.
Users of evidence reports at any level of the 5S pyramid need to be aware of the underlying methods of assembly and assure themselves that these methods are sound. At each level, the standards for evidence generation, retrieval, selection, and analysis should be explicit and at the highest evidence standard possible. For example, systems based on guidelines for patient care should be explicit about the source of the guidelines, and the guidelines should be based on systematic reviews of the pertinent evidence to date. Summaries should include details of the retrieval process used to find best evidence, the appraisal process for rating the quality of evidence should be explicit and auditable, key references should be provided for all care recommendations, the date of most recent updating should be stated, and updating should be done frequently enough to assure that important new evidence has not been neglected.
Services that provide synopses should have defined procedures for retrieving and appraising original and review articles and should report evidence quantitatively. For example, synopses of treatment studies and syntheses of such studies should include control and intervention group event rates, relative risk reductions, and numbers needed to treat, with 95% confidence intervals.
Where to look for current best evidence
How can this model guide decision makers to find the evidence they need, with speed and confidence? Begin the search for evidence to guide clinical decisions at the highest possible level in the 5S pyramid of evidence. If you have a computerized decision support system integrated into your electronic medical record system that reliably links your patient's characteristics with current evidence-based guidelines for care, you don't need to look any further. If you do not work in such an environment (or if you do but the system does not provide support for your patient's problems), then the next place to look is for integrated evidence in a summary service, such as Clinical Evidence (www.clinicalevidence.com/ceweb/conditions/index.jsp) or PIER (pier.acponline.org/index.html?jhp). If the topic isn't covered there, look for a synopsis in one of the evidence-based journals suited to your practice, such as ACP Journal Club (www.acpjc.org), Evidence-Based Medicine (ebm.bmjjournals.com/), and Evidence-Based Nursing (ebn.bmjjournals.com/). If you have no success there, look for a systematic review in BMJUpdates+ (bmjupdates.com), the Cochrane Library (www3.interscience.wiley.com/cgi-bin/mrwhome/106568753/HOME?CRETRY=1&SRETRY=0/), or PubMed Clinical Queries (www.ncbi.nlm.nih.gov/entrez/query/static/clinical.shtml). Failing that, you could look up original studies via BMJUpdates+ first, then Clinical Queries if needed.
If you are not familiar with which evidence-based resource is best for a particular clinical problem—or you know that the resources you usually use don't cover the problem at hand—then “federated search engines,” such as TRIP (www.tripdatabase.com) and SUMsearch (sumsearch.uthscsa.edu/), provide means to search many resources, with the retrieval being organized according to the source of evidence. But if you use such services, you will need to keep your critical appraisal filters on alert: The quality of the retrieval depends on the source, and many sources do not provide critical appraisal of evidence.
Some limitations of the resources and the model
We return to George Box's dictum about models being imperfect. First, the higher you go up the pyramid, the scarcer the resources become. Thus, there are a vanishing number of computerized decision support systems that reliably link patients' individual characteristics to current best evidence, and even the best of these covers only a tiny range of medical problems, often just a single condition. Second, the number of evidence-based summary publications is growing, but the number of disease conditions covered is probably just a few hundred. Indeed, developers are finding that the cost of maintaining even 200 to 300 conditions is very high. The costs, of course, must be passed along to the consumer, a situation that is made problematic by the plethora of cheap imitations (it seems that the label “evidence-based” is being applied to anything that contains a reference to the medical literature, no matter how old or unsystematic).
Third, processing information takes time, and synoptic services typically provide their commentaries months after publication of the original article or synthesis. The Cochrane Collaboration estimates that it will be many years before existing original treatment reports are summarized (3), and reviews of diagnosis, prognosis, and etiology are scarce. Further, even when the same evidence is being examined, syntheses may disagree with one another, a problem that can sometimes be resolved by considering the original studies to see if one or more of them more closely match the clinical problem at hand.
The original literature certainly covers a much broader range of clinical problems. However, even if the decision maker goes to the trouble to root out all the original studies on a given problem, there is no guarantee that a satisfactory study has been done, or that the patient will not have unique characteristics, making extrapolation of existing evidence less certain.
Is it time to change the way you seek best evidence?
Compare the 5S approach with how you usually seek evidence-based information. Is it time to revise your tactics? If, for example, it surprises you that PubMed is so low on the 5S list of resources for finding current best evidence, then this communication will have served a purpose: Resources for finding evidence have evolved in the past few years, and searches can be a lot quicker and more satisfying for answering clinical questions if the features of your quest match those of one of the more advanced services. This is in no way a knock against PubMed, which continues to provide a premier access route to the studies and reviews that form the foundation for all of the other more specialized databases reviewed here. Big rewards can be gained from becoming familiar with these new resources, beginning at the top of the pyramid, and using them whenever the right clinical question presents itself.
Acknowledgments: Thank you to Scott Richardson, Donna Ciliska, and Paul Glasziou for their comments and suggestions.
Conflict of interest statement: Brian Haynes has direct or indirect connections with many of the evidence-based resources used as examples above, including ACP Journal Club (editor), Evidence-Based Medicine (co-editor), Evidence-Base Nursing (coordinating editor), BMJUpdates+ (editor), The Cochrane Library (reviewer and former board member and Cochrane Centre director), Clinical Evidence (advisory board), PIER (evidence supplier) and PubMed Clinical Queries (developer). These resources are used to illustrate the concepts in the paper; there are other, perhaps better, examples.
R. Brian Haynes, MD, PhD
Hamilton, Ontario, Canada
1. Haynes RB. Of studies, syntheses, synopses, and systems: the “4S” evolution of services for finding current best evidence. ACP J Club. 2001 Mar-Apr;134(2):A11-3. [PubMed ID: 11280129]
3. Mallett S, Clarke M. How many Cochrane reviews are needed to cover existing evidence on the effects of health care interventions? ACP J Club. 2003 Jul-Aug;139(1):A11. [PubMed ID: 12841730]