Current issues of ACP Journal Club are published in Annals of Internal Medicine


Resource allocation: beyond evidence-based medicine and cost-effectiveness analysis

ACP J Club. 1997 Nov-Dec;127:A16. doi:10.7326/ACPJC-1997-127-3-A16

You are sitting at your desk reading the latest issue of ACP Journal Club. The telephone rings. It's the CEO of your hospital (or managed care organization), who says, “The cardiac program has run up a $1 million unbudgeted expense in angioplasty stents in the past fiscal year. We can't afford it. Could you look into it and see whether these gizmos are worth it?”

Is this a question of evidence-based medicine (EBM)? Is it a question of cost-effectiveness analysis (CEA)? The answer to both questions is “yes,” but more fundamentally, it is a question of resource allocation. This editorial will argue that EBM and CEA are necessary but not the only perspectives from which to approach resource allocation.

What is resource allocation?

Resource allocation can be defined as the distribution of resources among competing programs or persons (1). It occurs simultaneously at the macro (provincial or state government), meso (hospital or managed care organization), and micro (bedside) level. It starts from the assumption that resources are scarce and that resource allocation decisions are inevitable. The key question is how to make these decisions fairly. (I will return to the problem of measuring the construct of fair resource allocation decision making later.)

Resource allocation occurs not just in situations of shrinking resources but also when budgets are stable or increasing. For instance, if your hospital (or managed care organization) received an additional $10 million, it would still be faced with the resource allocation dilemma of how to spend it. Unfortunately, one method of allocating resources is to deny citizens access to basic health care services; in this editorial, I will focus on resource allocation among persons who are entitled to some of the resources.

Although resource allocation decisions are unavoidable, they can be mitigated through 3 general strategies: 1) Don't do things that don't work (this is where EBM can have an impact), 2) don't do things that do work but that people don't want (this is where shared decision-making initiatives have their impact), and 3) don't do things inefficiently (this is where restructuring, improved business and management practices, and negotiating prices with suppliers have their impact).

Many disciplines provide theoretical approaches to resource allocation problems. For instance, philosophy approaches resource allocation from the perspective of theories of distributive justice: law, from the perspective of constitutional and human rights of nondiscrimination and (in Canada) legislated rights to “medically necessary” health care in the Canada Health Act; political science, from the perspective of democratic participation in decision making; clinical epidemiology, from the perspective of EBM; and economics, from the perspective of CEA. There is, however, no comprehensive, interdisciplinary theory of resource allocation that bridges and integrates these different perspectives.


EBM is “the conscientious and judicious use of current best evidence from clinical care research in the management of individual patients” (2, 3). The primary contribution of EBM to resource allocation decision making is an elaboration of the concept of effectiveness, which has 3 elements: benefits, harms, and level of evidence. Evidence-based practitioners (and their patients) are accustomed to balancing benefits and harms (e.g., balancing stroke reduction against bleeding in considering whether to recommend warfarin for a patient with nonrheumatic atrial fibrillation) and quantity- and quality-of-life benefits (e.g., trading off survival for sexual function in the choice of radical prostatectomy compared with radiotherapy in the treatment of clinically localized prostate cancer). Moreover, there are established measures of levels of evidence on which conclusions on effectiveness can be based (4).

CEA is “a method for evaluating the health outcomes and resource costs of health interventions [whose] central function is to show the relative value of alternative interventions for improving health” (5). The primary contribution of CEA to resource allocation decision making is the elaboration of the concept of efficiency. CEAs have been performed for many clinical decisions (6), and guidelines for the interpretation of such analyses have been published (7). The term cost-effectiveness analysis is used both in the limited sense of maximizing increments in effectiveness (e.g., as measured in quality-adjusted life years, or QALYs) per increment in costs (e.g., measured in dollars) and in the general sense of maximizing any objective set by health planners per increment in costs (8). I am using the phrase in the specific sense here, as it is commonly used in the medical literature, but will later discuss the question of whether the more general use of cost-effectiveness analysis can address the problems with which it is identified.

In practice, EBM and CEA are not as separate as they may seem. CEA can be viewed as a method that just provides another type of evidence—this time including costs and clinical outcomes—that EBM clinicians apply to patients (Cook D. Personal communication).

Daniels (9) has outlined 4 unsolved rationing problems that serve to highlight the limitations of EBM and CEA for resource allocation decision making.

1. The fair chances-best outcomes problem: How much should we favor producing the best outcome with our limited resources? For example, 2 patients need a heart transplant. One will live for 20 years and the other for 40 years. Which patient should receive the transplant? Or, 2 treatments are available for similar groups. One treatment restores patients to a higher level of functioning. Which group should receive the treatment? Some argue that claims of patients with different outcomes should be equal, some contend that resource allocation should be proportional to the differences in outcome, and some claim that the resource should be allocated to patients likely to have the best outcomes (10-12). Empirical evidence shows that persons are not always willing to trade equity for best outcomes (13-15). CEA does not generally account for equity concerns.

2. The aggregation problem: When should we allow the aggregation of modest benefits to more people to outweigh more substantial benefits to fewer people? An example of this problem arose in Oregon when tooth capping was prioritized over appendectomy. A philosopher, Kamm, has formulated the Principle of Irrelevant Utility—that no number of cured sore throats outweighs saving a life (16). CEA treats these “irrelevant utilities” as though they were relevant and thus values population health over the health of individuals (17).

3. The priorities problem: How much priority should we give to treating the sickest or most disabled patients? This problem has also been dubbed the “Coby Howard factor” after the 7-year-old boy who died of acute lymphoblastic leukemia while his mother was raising funds for a bone marrow transplant for which he was ineligible under Medicaid. CEA does not account for the “rule of rescue”—the perceived duty to save endangered lives if possible (18).

4. The democracy problem: When must we rely on fair democratic process as the only way to determine what constitutes a fair rationing policy? An example is the prioritization of vasectomy above hip replacement on the basis of community values. Different groups in society have different interests and values, and political processes are important in resource allocation decision making (19-21). CEA has strategies to account for this pluralism of values, but the best approach is uncertain.

To these 4 problems, I would add a 5th—the evidence problem: When resources are limited, should we fund a program where there is high-quality evidence of a small benefit or one where there is lower-quality evidence of a large benefit? An example is whether to fund angioplasty stents for patients with “favorable” coronary artery lesions and optimal results after angioplasty (high-quality evidence of a small clinical benefit) or for patients with “unfavorable” coronary artery lesions (lower-quality evidence of a large benefit). Although placing confidence limits around cost-effectiveness ratios accounts for the play of chance, it does not account for the potential bias that is reduced through the use of better study designs (as captured by levels of evidence). How CEA should consider the strength of evidence is largely unexplored.

Beyond EBM and CEA

EBM and CEA represent the values of effectiveness and efficiency—key values in resource allocation decision making (22). However, these are not the only relevant values. As noted, equity, public compared with individual health, the rule of rescue, pluralism, and strength of evidence are also important values in resource allocation decision making. Can these values be redefined in terms of CEA? Probably (23, 24). Is redefining these other values in such terms the best way to account for them in resource allocation decision making? I don't think so. That seems to me like the drunk looking for his dropped keys under the lamp post rather than where he dropped them because the light is better.

A better resource allocation meth-od would incorporate, but also go beyond, EBM and CEA. It would account for all the values identified by the 5 unsolved rationing problems (efficiency, equity, public vs individual health, rescue, pluralism, evidence) as well as other values not yet identified. For instance, what are we to make of the male oncologist arguing for more funding for prostate cancer treatment against the female oncologist arguing for more funding for breast cancer treatment? Scholars in health policy have begun to develop resource allocation “methodologies” that go beyond EBM and CEA in the context of considering what services should be publicly funded (25-28).

Let's return to the telephone call from the CEO about angioplasty stents. This is an example of a meso-level resource allocation problem related to a new and emerging technology. (Giacomini ([29] has emphasized the importance of considering how problems are framed, or “assembled,” and not just how the decisions are made.) To address this resource allocation problem, we would need to address the issue of efficiency, as well as the rule of rescue, and develop a process to ensure that other values (such as equity, sex, population health, and evidence as well as values yet unidentified) are adequately considered. Experience with the Advisory Group on Angioplasty Stents of The Toronto Hospital (TTH) and the Expert Panel on Coronary Artery Stenting of the Cardiac Care Network of Ontario (CCN) has highlighted for me the importance of having principles that go beyond efficiency for resource allocation decision making 30, 31).

For instance, one could argue that a proposed service should be funded if it satisfies the principle of efficiency (or the principle of need) and the principle of democratic process.

1. The principle of efficiency: interventions that have been proved to have important clinical benefit with high-quality evidence (randomized controlled trials) and have been shown to be economically feasible using CEA whenever possible should be considered for implementation. Interventions that have been evaluated with less-rigorous methods (cohort studies, case series, or expert opinion) or have a sound pathophysiologic basis and are associated with negligible costs (e.g., seat belts) should also be considered for implementation. For example, the CCN Expert Panel recommended funding for angioplasty stents for patients with “favorable” coronary artery lesions and suboptimal results after angioplasty (larger clinical benefit) but not for patients with “favorable” lesions who achieved optimal results with angioplasty alone (smaller clinical benefit).

2. The principle of need: less rigorous (cohort studies, case series, or expert opinion) or pathophysiologic evidence of benefit plus extreme need, defined as an impending risk for death or severe harm that could be reduced by the treatment, in the absence of other treatment options. It may become necessary to limit this category if costs become unreasonable. So, for instance, the TTH Advisory Group recommended funding for patients with cardiogenic shock and the CCN Expert Panel recommended funding for patients with vessels that threatened their survival.

3. The principle of democratic process: endorsement of resource allocation decisions based on efficiency and need by consensus, most stakeholder communities, or interested groups. Particular attention should be paid to the values inherent in such terms as important clinical benefit, economic attractiveness, and negligible or reasonable costs and to concerns of equity (including sex) and public compared with individual health. So, for instance, the TTH Advisory Group presented their evolving recommendations to the hospital's clinical ethics committee, community advisory committee, and board for comments and endorsement.

Will these principles lead to good or better resource allocation decisions? The first step toward answering this critical question would be to define a “good” resource allocation decision, a pivotal concept that itself is far from clear.

Once a resource allocation decision has been made, important issues in implementation of the decision remain. These issues are beyond the scope of this editorial, but I will mention some of them. 1) Public-private mix: If an effective treatment is not funded publicly, should it be available for private purchase? 2) Consent and disclosure: Should physicians inform patients who might choose an unfunded treatment about its availability? 3) The physician-patient relationship: Is the appropriate role of the physician to serve only the patient's interests without regard to resource constraints, or should the physician advocate for patients within the limits of fairly developed societal guidelines on resource use?

The bottom line

EBM and CEA are extremely useful tools to incorporate into methodology for resource allocation, but they are not sufficient. “Cost-effectiveness estimates should not be used in a mechanistic fashion; at best they provide a useful aid for decision making” (32). As the Panel on Cost-Effectiveness in Health and Medicine noted, “… CEA can be used as an aid to decision makers who must weigh the information it provides in the context of … other values” (33).

The pressing research challenge is to develop and evaluate an interdisciplinary methodology for resource allocation decision making that incorporates but goes beyond EBM and CEA; integrates the various theoretical approaches to resource allocation of philosophy, law, political science, economics, and clinical epidemiology; proves useful to resource allocation decision makers; and is perceived as fair by the communities whose resources are at stake.

Peter A. Singer, MD, MPH
University of Toronto
Toronto, Ontario, Canada

Acknowledgments: The author thanks Deborah Cook, Allan Detsky, Ed Etchells, Murray Krahn, Don Redelmeier, James Wright, and 3 anonymous reviewers for providing helpful comments on this manuscript.


1. McKneally MF, Dickens BM, Meslin EM, Singer PA. CMAJ. 1997;157:163-7.

2. Haynes RB, Sackett DL, Gray JM, Cook DJ, Guyatt GH.Transferring evidence from research into practice: 1. The role of clinical care research evidence in clinical decisions. ACP J Club. 1996 Nov-Dec;125:A14-6.

3. Evidence-Based Medicine Working Group. JAMA. 1992;268:2420-5.

4. Cook DJ, Guyatt GH, Laupacis A, Sackett DL. Chest. 1992;102(Suppl 4): 305S-11S.

5. Russell LB, Gold MR, Siegel JE, Daniels N, Weinstein MC. JAMA. 1996;276:1172-7.

6. Tengs TO, Adams ME, Pliskin JS, et al. Risk Anal. 1995;15:369-90.

7. Laupacis A, Feeny D, Detsky AS, Tugwell PX. Can Med Assoc J. 1992;146: 473-81.

8. Detsky AS, Naglie IG. Ann Intern Med. 1990;113:147-54.

9. Daniels N. Hastings Cent Rep. 1994;24:27-9.

10. Taurek J. Philosophy and Public Affairs. 1977;6:293-316.

11. Broome J. Hastings Cent Rep. 1994;24:36-9.

12. Rakowski E. Hastings Cent Rep. 1994; 24:33-6.

13. Nord E. J Med Ethics. 1993;19:37-42.

14. Nord E, Richardson J, Street A, Kuhse H, Singer P. Soc Sci Med. 1995;41:1429-37.

15. Ubel PA, DeKay ML, Baron J, Asch DA. N Engl J Med. 1996;334:1174-7.

16. Kamm FM. Hastings Cent Rep. 1994, 24: 29-32.

17. Naylor CD.Cost-effectiveness analysis: Are the outputs worth the inputs?. ACP J Club. 1996 Jan-Feb;124:A12-4.

18. Hadorn DC. JAMA. 1991;265:2218-25.

19. Bailey MA. Hastings Cent Rep. 1994;24: 39-42.

20. Deber RB. Int J Technol Assess Health Care. 1992;8:131-7.

21. Wiktorowicz M, Deber R. Health Policy. 1997;40:1-24.

22. Eddy DM. JAMA. 1996;275:650-7.

23. Deber RB, Goel V. Med Decis Making. 1990;10:181-94.

24. Eddy DM. JAMA. 1991;266:2135-41.

25. Kitzhaber JA. BMJ. 1993; 307:373-7.

26. Government Committee on Choices in Health Care. Choices in Health Care. Rijswik, Netherlands: Ministry of Welfare, Health, and Cultural Affairs; 1992.

27. Deber R, Narine L, Baranek P, et al. The Public/Private Mix in Health Care. Ottawa: National Forum on Health, 1997; [In press].

28. Canadian Medical Association. Core and Comprehensive Health Care Services: A Framework for Decision Making. Ottawa: Canadian Medical Association; 1994.

29. Giacomini M. Centre for Health Economics and Policy Analysis Working Paper 97-1. Hamilton, Ontario: McMaster University; March 1997.

30. Etchells E, Singer PA, Lazzam C, et al. Toronto: University of Toronto Joint Centre for Bioethics; 1996.

31. Almond D, Chisholm R, Cohen E, et al. CCN Expert Panel on Intracoronary Stents: Final Recommendations. Toronto: Cardiac Care Network of Ontario; 1997. (Full text available through the CCN website,

32. Mason J, Drummond M, Torrance G. BMJ. 1993;306:570-2.

33. Russell LB, Gold MR, Siegel JE, Daniels N, Weinstein MC. JAMA. 1996;276:1172-7.