Current issues of ACP Journal Club are published in Annals of Internal Medicine


Editorial

Transferring evidence from research into practice: 3. Developing evidence-based clinical policy

ACP J Club. 1997 Mar-Apr;126:A14. doi:10.7326/ACPJC-1997-126-2-A14



In previous editorials in this series (1, 2), we described a path that leads from health care research evidence to evidence-based health care. The steps include getting the evidence straight, developing evidence-based clinical policy, and then applying the policy.

A policy is a statement of what should be done. Policies come in many forms. Health policies, for example, determine the funding of health care, and public health policies cover such issues as immunization and screening. Clinical policies, the focus of this essay, state what should be done in clinical practice.

Clinical policies may be created by organizations and committees, but as clinicians, we also have our own personal policies for handling specific clinical situations (3). Personal policies are essential for action but are often problematic, particularly if they do not include evidence beyond our own clinical experiences. Formal policies are similarly problematic if they do not incorporate the best evidence. Evidence-based medicine provides ways to find and incorporate research evidence in clinical policies, both formal and personal.

Why do we need clinical policies?

Clinicians often make substantially different management decisions for similar clinical situations. This variation in practice often occurs in geographically close communities and is not consistently explained by differences in patient characteristics or preferences (4, 5). More important, variation in management often includes practices that are inconsistent with good evidence about optimal care.

Inadequacy of individual clinical practice is a problem. In response, professional bodies, businesses, and government agencies are developing formal clinical practice guidelines, practice parameters, clinical standards, consensus statements, expert recommendations, decision analyses, care paths, and disease management programs. As these groups develop clinical policies, they face a spectrum of challenges, from keeping up with the evolution of medical knowledge at the input end to handling often perplexing differences in the circumstances of care delivery at the output end.

Requirements for evidence-based clinical policy

The development of evidence-based clinical policy involves several stages, including defining the clinical problem, finding and appraising evidence, and converting the best evidence into clinical policy. Defining the clinical problem to be addressed is crucial, and a precise definition can convert a hopeless global problem into a number of soluble discreet ones. For example, generating clinical policies for evaluating and managing everyone with high blood pressure is much more difficult than generating guidelines for specific, well-defined subgroups within that large population. In the process of considering these subgroups, policy developers can set aside circumstances in which they (parallel with policy developers) lack the requisite evidence, such as the use of certain drugs for hypertension during pregnancy and the management of persons with several conditions.

We have discussed the second step, finding and appraising the evidence, in the previous articles in this series (1, 2). We described how recent important advances in evidence retrieval, summarization, and valuation have greatly enhanced the assembly of evidence for guideline and policy development. Development of sound evidence-based policies also requires us to recognize that not all evidence is created equal. The Canadian Task Force on Periodic Health Examination (6) pioneered the process of grading the strength of evidence, and their approach has since been adopted and adapted by many other groups. Recently, Guyatt and colleagues (7) have proposed a modification for grading evidence for health interventions, giving priority to systematic reviews of evidence and the size of effect that is judged clinically important (Table). Recommendations should cite the evidence on which they are based and summarize the strength of that evidence.


Table. Grades of Recommendations for a Specified Level of Baseline Risk*

A1 RCTs with homogeneous results, with all CIs on one side of NNT
A2 RCTs with homogeneous results, with CIs overlapping threshold NNT
B1 RCTs with heterogeneous results, with all CIs on one side of threshold NNT
B2 RCTs with heterogeneous results, with CIs overlapping threshold NNT
C1 Non-RCTs and observational studies, with all CIs on one side of threshold NNT
C2 Non-RCTs and observational studies, with CIs overlapping threshold NNT

*CIs = confidence intervals; NNT = number needed to treat; RCTs = randomized controlled trials. Modified from reference 7.

Clinical policies must also consider evidence about the burden of disease, barriers to implementing policy (and what can be done to overcome them), safety, acceptability, and cost-effectiveness (8). Even if the evidence is complete, it usually does not directly lead to treatment decisions. Clinical decisions usually involve tradeoffs, and these tradeoffs must take individual and societal values into account. Acknowledging that values are part of any clinical policy emphasizes the desirability of ensuring that all persons who will be covered by the policy have had some say in its creation, either directly or through representatives.

Whenever possible, we should make use of formally developed policies that have been through all of these steps and that fit our own clinical setting. In deciding whether a particular policy is worthy, we can look to detailed guidelines for evaluating the policy-development process (9, 10).

Scope of clinical policies

How much of our clinical practice is amenable to the sort of policy-making we have described? It is possible to formulate clinical policies in all situations for which there is adequate evidence to allow extrapolations from evidence to real-life clinical situations: for example, “all patients who enter the hospital within 12 hours of onset of suspected myocardial infarction should be assessed for thrombolytic therapy within 30 minutes of arrival.” Fortunately, a high proportion of inpatient care in medicine (11), surgery (McCulloch P. Personal communication), and psychiatry (12) can be supported by sound evidence from research.

Further, advances in the generation and summarization of evidence (13) and in standards for developing and reporting clinical practice guidelines (14, 15) have enhanced the prospects for sound and timely evidence-based guidelines. That these “guidelines for guidelines” can be followed in part or in whole has already been shown (16). Controlled trials of implementing guidelines have shown benefits for the process and outcomes of care (17).

Limitations of clinical policies

Although advances in guideline development and application are encouraging, we must still face some limitations of clinical policies. Because of the interplay of evidence, circumstances, and patient preferences, there is no such thing as “one policy fits all.” For example, in some settings, immediate percutaneous coronary angioplasty may be better than streptokinase for acute myocardial infarction (18), but immediate angioplasty requires resources that are not available in most places. Even in a setting with an expert staff and angiography suite that can be put on standby for emergent cases, only a subset of patients who are clinically stable and seen early in the course of acute myocardial infarction are likely to be candidates for angioplasty. Thus, a credible clinical policy cannot recommend immediate angioplasty for all patients with acute myocardial infarction no matter how sophisticated the setting, let alone in all settings. Further, clinical acumen and judgment must always be applied to tailor the policy to the individual patient.

Another limiting problem is the resistance of clinical practice to change. As McDonald has pointed out (3), we treat most patients according to personal, informal clinical policies based on our knowledge of medical practice and our own clinical experience. These “rules of thumb” or “heuristics” differ from practitioner to practitioner and account for much of the considerable variation in clinical practice. McDonald suggests that these informal clinical policies constitute formidable barriers to evidence-based health care. Primum non nocere (first do no harm) is an example of such a policy. Attributed to Hippocrates and intoned by graduating medical students to the present time, this advice can appropriately be applied to treatments of unknown efficacy but is clearly unworkable for efficacious treatments that have short-term adverse effects (including lethal ones) but that do more good than harm in the long run. Obvious examples include virtually all efficacious invasive procedures, but many medical interventions have similar features, including cancer chemotherapy and treatments for chronic disorders.

The inadequacies and inconsistencies of personal clinical practices have spurred professional bodies, businesses, and government agencies to become increasingly involved. Unfortunately, the time and effort required to develop valid and useful clinical policies have been found to be huge, and at best they can cover a modest fraction of practice. Moreover, these policies can be “hijacked” by politics (19) and, even when they are evidence-based, can quickly become outdated as new evidence emerges.

Formal policy is also limited in its ability to take different subgroups of patients into account. For example, the effect that variations in a patient's expected event rate have on numbers needed to treat and the probability that a patient will benefit from a treatment mean that some subgroups of patients are more likely to be harmed than helped despite solid evidence of efficacy for the “average patient” (20).

At present, the formation and application of clinical policy is too often arbitrary and authoritarian, at both the organizational and personal clinical level. It is our view that formal clinical policy development must involve “front-line” practitioners and that policy cannot be effectively or safely applied without the active collaboration of practitioners who understand the basis for the policy as well as the circumstances and wishes of the patient. On the other hand, practitioners cannot continue to be so informal and idiosyncratic in dealing with individual clinical problems. Fortunately, emerging evidence shows that involving local practitioners in policy development, coupled with responsive local systems, can succeed where the creation of national guidelines fails (21).

J.A. Muir Gray
R. Brian Haynes
David L. Sackett
Deborah J. Cook
Gordon H. Guyatt


References

1. Haynes RB, Sackett DL, Muir Gray JA, Cook DJ, Guyatt GH.Transferring evidence from research into practice: 1. The role of clinical care research evidence in clinical decisions. ACP J Club. 1996 Nov-Dec;125:A14-A16.

2. Haynes RB, Sackett DL, Muir Gray JA, Cook DJ, Guyatt GH.Transferring evidence from research into practice: 2. Getting the evidence straight. ACP J Club. 1997 Jan-Feb;126:A14-A16.

3. McDonald CJ. Medical heuristics: the silent adjudicators of clinical practice. Ann Intern Med. 1996;124:56-62.

4. Iscoe NA, Goel V, Wu K, et al. Variation in breast cancer surgery in Ontario. Can Med Assoc J. 1994;150:345-52.

5. Health Services Research Group. Small-area variations: what are they and what do they mean? Can Med Assoc J. 1992;146: 467-70.

6. Canadian Task Force on Periodic Health Examination. The periodic health examination: 1. Introduction. Can Med Assoc J. 1979;121:1193-254.

7. Guyatt GH, Sackett DL, Sinclair JC, et al., for the Evidence-Based Medicine Working Group. Users guides to the medical literature. IX. A method for grading health care recommendations. JAMA. 1995;274:1880-4.

8. Muir Gray JA. Evidence-Based Healthcare. London: Churchill Livingstone; 1997.

9. Hayward RS, Wilson MC, Tunis SR, Bass EB, Guyatt GH, for the Evidence-Based Medicine Working Group. Users guides to the medical literature. VIII. How to use clinical practice guidelines. A. Are the recommendations valid? JAMA. 1995; 274:570-4.

10. Hayward RS, Wilson MC, Tunis SR, Bass EB, Guyatt GH, for the Evidence-Based Medicine Working Group. Users guides to the medical literature. VIII. How to use clinical practice guidelines. B. What are the recommendations and will they help you in caring for your patients? JAMA. 1995;274:1630-2.

11. Ellis J, Mulligan I, Rowe J, Sackett DL, on behalf of the A-Team, Nuffield Department of Clinical Medicine. Inpatient general medicine is evidence based. Lancet. 1995;346:407-10.

12. Geddes JR, Game D, Jenkins NE, et al. In-patient psychiatric treatment is evidence-based. Proceedings of the Royal College of Psychiatrists Winter Meeting, Stratford, UK. 23-25 January, 1996.

13. Chalmers I, Dickersin K, Chalmers TC. Getting to grips with Archie Cochrane's agenda [Editorial]. BMJ. 1992;305:786-8.

14. Consensus development at the NIH: improving the program. Institute of Medicine. Washington, DC: National Academy Press; 1990.

15. Carter A. Background to the “guidelines for guidelines” series. Can Med Assoc J. 1993;148:383.

16. Canadian Guide to Clinical Preventive Health Care. Canadian Task Force on the Periodic Health Examination. Ottawa: Ministry of Supplies and Services; 1994.

17. Grimshaw JM, Russell IT. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993;342:1317-22.

18. de Boer MJ, Hoorntje JC, Ottervanger JP, et al. Immediate coronary angioplasty versus intravenous streptokinase in acute myocardial infarction: left ventricular ejection fraction, hospital mortality and reinfarction. J Am Coll Cardiol. 1994; 23:1004-8.

19. Skrabenek P. Nonsensus consensus [Editorial]. Lancet. 1990;336:120.

20. Guyatt GH, Jaeschke RZ, Cook DJ.Applying the findings of clinical trials to individual patient. ACP J Club. 1995 Mar-Apr;122:A12-A13.

21. Implementing Clinical Practice Guidelines. York, UK: NHS Centre for Reviews and Dissemination; 1994.