Professional Development
Health Technology Assessment in Canada: Problems and Opportunities
Murray Krahn, MD
About the Author
Murray Krahn is chair of pharmacoeconomics and director of the Health Economics and Technology Collaborative, University of Toronto, Toronto, Ontario. He was the 2007 recipient of the CSIM Senior Investigator Award. Correspondence may be addressed to murray.krahn@theta.utoronto.ca.
Consider Avastin, a new drug for metastatic colorectal cancer. Avastin costs $37,500 for a 10-month course of treatment and results in an average gain in survival of 10 months. Should this drug be funded by provincial formularies, or not?
Decisions about new drugs and technology are complicated. On the one hand, clinicians and policy makers want to offer patients state-of-the-art care, including new and promising drugs, devices, and health technologies. On the other hand, costs are often exorbitant, and the evidence about effectiveness is often incomplete at the time of decision. Evidence about impacts on quality of life and function are also often incomplete or missing. Some new technologies, for example, reproductive technologies, raise questions about social and ethical values. A new technology may not be adopted if there are practical or logistical barriers (e.g., no physician billing codes). Conversely, technology may introduce destabilizing effects on the health system (e.g., the effects of widespread use of angioplasty on the profession of cardiovascular surgery).
Health technology assessment (HTA) is the field that has arisen to meet the challenge posed by new drugs and technology.1–3 As you might expect, given the litany of issues raised above, it is an interdisciplinary field that requires consideration of factors beyond those usually entertained by even evidence-oriented physicians. These include cost, quality of life, ethics, social values, and health system integrity.
Battista and Hodge state that technology assessment has three features in addition to its interdisciplinary orientation. It is oriented toward policy, as opposed to the care of individual patients. It involves synthesizing data, often of disparate sorts. And, finally, an integral part of the process is disseminating and communicating results, or knowledge translation.4
The technology assessment enterprise has been flourishing in recent years. There are an increasing number of academic societies (e.g., Health Technology Assessment International [HTAi] and the International Society for Pharmacoeconomics and Outcomes Research [ISPOR]) and journals (e.g., International Journal of Technology Assessment in Health Care, Pharmacoeconomics,5–7 and Medical Decision Making) devoted to this enterprise. Citations associated with MeSH headings including decision analysis, economic evaluation, health technology assessment, cost, and cost benefit rose from fewer than 500 per year in 1980–1985 to approximately 10,000 per year in 1995.
In Canada, there are a growing number of health technology agencies, including the national agency, the Canadian Agency for Drugs and Technology in Health (CADTH), in Ottawa, and provincial and regional agencies, concentrated in Alberta, Ontario, and Quebec. The International Organization for Health Technology Assessment (INAHTA), includes 47 member agencies in 23 countries (http://www.inahta.org).
Although HTA is increasing in importance and influence, not least in Canada, it faces a number of challenges: (1) the dominance of evidence-based medicine (EBM), (2) the use of low-quality and biased cost-effectiveness evaluations, (3) a weak evidence base for “context sensitive” evidence, and (4) a lack of capacity at the producer and receptor levels.
The Dominance of EBM
EBM has been characterized as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients.”8 Because of the high regard in which EBM is held, not least by general internists, who are among its leading expositors, the idea that “too much” EBM might pose a problem may strike some as counterintuitive. While we do want new technologies and drugs to be supported by high-quality evidence about effectiveness, this is not enough. Certain types of empirical data that are relevant from a broader social perspective, such as costs, individual and social preferences for health outcomes, and ethical and social values, are not very well integrated into the EBM scientific paradigm.9–11 And so we must look further.
Economic evaluation, or cost-effectiveness analysis (CEA) as it is colloquially known, is a method for comparing “alternative courses of action in terms of both their costs and consequences.”12 This is a powerful approach to integrating information about both how well technologies work and data describing how patients feel about the outcomes of care (patient preferences or utilities), the costs of new technology, and costs saved by new technology.
Both EBM and CEA offer guides to decision making. They offer criteria for good decisions about technology. We should support new drugs and technologies if there is good quality evidence of effectiveness, and if they are cost effective.
But how are we to determine and weight these factors? Who will be evaluating the evidence and making the decision? It is one thing to have criteria, but the process by which decisions are made also needs to be laid out. Above all, the process should be fair. Accountability for reasonableness13–15 is a set of bioethical principles that describes the characteristics of fair decisions:
1. Publicity. The reasons behind a particular decision should be publicly available.
2. Relevance. There must be reasons that are relevant and adequate (although they do not need to be particular types of reasons.
3. Appeals. There must be some type of appeals mechanism after decisions are made.
4. Enforcement. A fair decision is one that can be enforced.
Both cost-effectiveness and accountability for reasonableness are gaining in prominence and influence. Cost effectiveness is part of new drug evaluations for CADTH and various provincial agencies. The consonance of technology with social values is increasingly a part of the discourse around new technology. As of 2008, however, it is still evidence of effectiveness and the broader culture of EBM that dominate discussions and reports about new technology.
The Use of Low-Quality and Biased Cost-Effectiveness Evaluations
The suboptimal implementation of CEA and, in particular, the use of low-quality and biased cost-effectiveness evaluations present a challenge. At present, most cost-effectiveness evaluations that are submitted to provincial formulary committees and the national Common Drug Review are performed by industry. There are two problems with this. Firstly, the quality of submissions varies. If the market for a drug is small, companies may be reluctant to perform or support a good-quality study and may be unresponsive to requests for better analyses. Secondly, a CEA performed by industry may be systematically biased.16 The National Institute of Health and Clinical Excellence in the United Kingdom takes this problem seriously by funding seven academic centres to produce credible, independent evaluations of new technology (http://www.nice.org.uk/). No such system is available in Canada.
Weak Evidence Base for Context Sensitive Evidence
The evidence base for what has been termed “context sensitive” evidence is weak. This kind of evidence includes patients’ valuations of health outcomes and the cost of health care provision – empirical data that are highly relevant to decisions about new drugs and technologies but that are not considered to be “evidence.”17 Studies generating this type of evidence are typically small and less rigorously performed than those evaluating the effectiveness of treatment. As a result, we generally have much less confidence in our estimates for these outcomes. Technology assessments as a whole become less scientifically credible when important components are of low quality.
Lack of Capacity at Producer and Receptor Levels
Capacity is a problem both at the producer and the receptor levels. There are relatively few decision makers within provincial and federal ministries of health and Canadian health care institutions with training in health outcomes research, health economics, or other disciplines related to technology assessment. Technology assessments tend to be complex and may not be fully understood by those without specialized knowledge in the area. High-quality assessments lack influence if they are presented in a way that readers cannot understand. On the producer side, there is also a very significant unmet demand from industry and government for well-trained researchers and practitioners of HTA. Graduate students in this area often receive job offers prior to the completion of their training.
Future Directions and Possibilities
However, with these problems come opportunities. Economic evaluation as a discipline is growing in credibility, consistency, rigour, and influence. An increasing number of jurisdictions in Canada are using the language of fairness and accountability for reasonableness. CADTH is establishing partnerships with credible academic institutions to provide unbiased technology assessments. The University of Alberta in Calgary and McMaster University in Hamilton, Ontario, offer graduate level training in HTA programs. The ULYSSES program, a collaboration of three Canadian universities (Universities of Ottawa, Montreal, and Toronto) and two European universities (Catholic University of Rome and University of Barcelona), offers an international experience and master’s level training in HTA (http://www.ulyssesprogram.net/).
There are opportunities for general internists in this area. General internists are obliged to think broadly when evaluating clinical problems. It may be this inclination that explains why many are also leaders in HTA and related areas. If you are a trainee or practitioner with a philosophical penchant who likes to think about the social consequences of new drugs and technology, this might be an area where your broad clinical training and experience can allow you to flourish. Take some courses, go to a meeting or two (Society for Medical Decision Making, ISPOR, HTAi), or join a provincial or national formulary or drug evaluation committee (e.g., the Common Drug Review). Join a technology assessment committee at your institution or at the health region or provincial level. There’s a definite need and real opportunity for educated, thoughtful clinicians to help improve decision making around new drugs and technology, to help shape decisions that are consonant with the best scientific evidence and the values of Canadians.
References
1. Hailey DM. Health technology assessment in Canada: diversity and evolution. Med J Aust 2007;187:286–8.
2. McDaid D. Co-ordinating health technology assessment in Canada: a European perspective. Health Policy 2003;63:205–13.
3. McGregor M. What decision-makers want and what they have been getting. Value Health 2006;9:181–5.
4. Battista RN, Hodge MJ. The evolving paradigm of health technology assessment: reflections for the millennium. CMAJ 1999;160:1464–7.
5. Detsky AS. Guidelines for economic analysis of pharmaceutical products: a draft document for Ontario and Canada. Pharmacoeconomics 1993;3:354–61.
6. PausJenssen AM, Singer PA, Detsky AS. Ontario’s formulary committee: how recommendations are made. Pharmacoeconomics 2003;21:285–94.
7. Ess SM, Schneeweiss S, Szucs TD. European healthcare policies for controlling drug expenditure. Pharmacoeconomics 2003;21:89–103.
8. Sackett DL, Rosenberg WM, Gray JA, et al. Evidence based medicine: what it is and what it isn’t. BMJ 1996;312(7023):71–2.
9. Chong C, Chen I, Naglie G, Krahn M. Do clinical practice guidelines incorporate evidence on patient preferences? Med Decis Making 2007;27(4):E63–4.
10. Schunemann HJ, Fretheim A, Oxman AD. Improving the use of research evidence in guideline development: 10. Integrating values and consumer involvement. Health Res Policy Syst 2006;4:22.
11. Taylor J, Wilkinson D, Blue IA, Dollard JT. Evidence-based rural general practice: barriers and solutions in South Australia. Rural Remote Health 2002;2(1):116.
12. Drummond M, O’Brien, Stoddart G, Torrance G. Methods for the economic evaluation of health care programmes. Oxford University Press: Oxford, New York, Toronto; 1997.
13. Gibson JL, Martin DK, Singer PA. Priority setting in hospitals: fairness, inclusiveness, and the problem of institutional power differences. Soc Sci Med 2005;61:2355–62. Epub 2005 Jun 2359.
14. Daniels N. Accountability for reasonableness. BMJ 2000;321(7272):1300–1.
15. Daniels N. Decisions about access to health care and accountability for reasonableness. J Urban Health 1999;76:176–91.
16. Bell CM, Urbach DR, Ray JG, et al. Bias in published cost effectiveness studies: systematic review. BMJ 2006;332(7543):699–703.
17. Lomas J, Culyer A, McCutcheon L, et al. Conceptualizing and combining evidence for health system guidance. Ottawa: Canadian Health Services Research Foundation; 2005.
Article citation: Krahn M. Health Technology Assessment in Canada: Problems and Opportunities. Can J Gen Intern Med 2008;3((2):69-71
|
