Perspectives in GIM
Industry-Initiated Drug Trials Are Far Less Credible to Canadian Internists than Investigator-Initiated Trials
David L. Sackett, MD
About the Author
David Sackett was the founding chair of the Department of Clinical Epidemiology and Biostatistics at McMaster University, Hamilton, Ontario. A quintessential internist, he is retired from clinical practice and runs methods workshops for new clinician-researchers at the Trout Research an Education Centre in Ontario.
Many randomized trials of promising drugs are initiated, designed, executed, and analyzed by independent investigators at arm’s length from the drug industry. The crucial element of such trials is that all patient data bearing on the efficacy of the drug are sent to the independent investigator, not to the manufacturer (which receives only safety data until the trial’s conclusion). In some investigator-initiated trials receiving industry support, the manufacturer might be represented on the steering committee but is kept blind to the emerging results on efficacy. Thus, the manufacturer is excluded from the trial’s unblinded trial monitoring committee that examines the emerging efficacy data and decides when to unblind the principal investigators and help them decide whether to continue or conclude the trial. Moreover, the independent investigator retains total academic freedom in interpreting, presenting, and publishing the trial results and conclusions. Finally, this independence can be achieved and maintained when the drug and its corresponding placebo are donated by its manufacturer, and even when partial or complete funding for the trial comes from industry.
Increasingly, however, randomized drug trials are initiated, conducted, executed, and analyzed by the drug’s manufacturer. Industry (or its hired contract research organization) recruits the clinical collaborators, usually paying them for each patient they enrol (sometimes with bonuses for achieving a target number). All field data on efficacy as well as safety are sent to the manufacturer, which carries out all the analyses in house. Trial results are interpreted by the manufacturer, sometimes with advice from external paid consultants. Finally, the manufacturer controls the interpretation, presentation, and publication of the trial results, even when they appear under the names of outsiders.
Three recent developments are threatening the credibility of industry-initiated trials. First, in an unprecedented simultaneous move, the editors of 13 leading medical journals declared it was necessary to impose much stricter requirements on authors to disclose their ties to industry and to include “the role of the sponsor” in their trial reports.1
Second, the “bounties” paid to clinicians for putting their patients into trials, under increasing attention from the lay media,2 have led professional organizations to strengthen or re-emphasize their codes of ethics. For example, the code of the Canadian Medical Association declares that the payment for putting one’s patients in a clinical trial must not constitute a financial inducement.3 It restricts the ethical limit to just “replace lost income” for putting a patient into the trial rather than devoting that same time to seeing, and billing for, other patients. Moreover, the size and circumstances of the payment must be approved by the relevant ethics committee. Finally, the patient must be told that the doctor is being paid to put them into the trial.
Third, the “fairness” of publications coming from industry-funded trials has come under fire. For example, a group of Danish investigators led by Bodil Als-Nielsen examined 370 publications from both industry-funded and otherwise-funded trials of the same drugs for the same conditions.4 As expected, the treatment effects observed in industry-funded and otherwise-funded trials were similar in magnitude. However, the industry-funded trials were five times as likely to recommend the treatment they studied as were the otherwise-funded trials that demonstrated the same effects of the same treatments for the same conditions.
My literature search yielded only one formal study that examined the credibility of industry-initiated and investigator-initiated trials. In it, a UK team led by Samena Chaudhry invented two sets of fictitious authors for an authentic report of a “positive” trial of a drug that relieved the pain of herpes zoster.5 In one version, the authors declared they were “employees of a fictitious company and potentially held stock options” in the company. In the other, the authors declared they “were from an ambulatory care centre and had no competing interest.” Two random samples (150 each) of British Medical Association (BMA) members received both of these reports and were asked to rate their believability, interest, importance, and relevance (response rate = 59%). Not only was the identical report judged “less believable” when it was authored by industry employees, it was judged “less interesting,” “less important,” and “less relevant.”
Both before and after this BMA study, I surveyed Canadian internists for their opinions on the relative credibility of an identical, fictitious trial when it was initiated and executed by university-based, independent investigators or by industry, and I report the results here.
Method
Two surveys were carried out to answer the question, among Canadian internists, is their impression of the credibility of a “positive” randomized trial affected by whether it is conducted (i.e., designed, carried out, analyzed, interpreted, and written up) by independent university-based investigators or by the drug industry? Both surveys employed the same fictitious trial scenario:
Suppose that two randomized trials of different promising drugs asked the same question: “Among patients with unstable angina pectoris, will the addition of drug X [or Y] to current best medical therapy reduce the risk of nonfatal myocardial infarction or death?” In both, the promising drug was reported to both statistically and clinically significantly improve the outcomes of patients with unstable angina pectoris: the NNT (number needed to be treated) to prevent one more myocardial infarction or death was 31. Both reports had passed formal peer review and had appeared in the same major medical journal. However, one report resulted from an industry-initiated trial in which all patient data were sent to the drug’s manufacturer, which performed all the interim and final analyses and saw unblinded data through membership on the trial’s Data Safety and Monitoring Board. The second, otherwise-identical report resulted from a university-based, investigator-initiated trial in which all data went to the university, which performed all the interim and final analyses and excluded employees or stockholders of the drug’s manufacturer from the Data Safety and Monitoring Board.
Internists were asked to rate the “credibility” of each trial on a six-point Likert scale from “the trial is not at all credible” to “the trial is of extremely high credibility.”
The first survey was carried out in 2001 among 132 volunteer internists attending “grand rounds” and other educational sessions at hospitals in Toronto, Hamilton, and London, Ontario, and Edmonton, Alberta. All respondents received both scenarios and responded to each.
I was concerned that two potential sources of bias may have distorted the results of this 2001 survey. First, it was carried out among volunteers, who might have participated because they already held negative views of industry-initiated trials: volunteer bias. Second, the juxtaposition of both trials in a single questionnaire may have exaggerated real differences in credibility: juxtaposition bias.
Accordingly, and to overcome the potential volunteer bias, a second (2004) survey was carried out on a random sample of 150 internists who were members of the nationwide Canadian Society of Internal Medicine. They were stratified by age (above and below their median age) and sex. To avoid the potential juxtaposition bias, each stratum was randomized to three groups of equal size: one received just the industry-initiated trail, the second received just the investigator-initiated trial, and the third received both trials (the latter replicating the 2001 survey). The 2004 survey was submitted to and approved by the Grey Bruce Health Services Ethics Committee. When colleagues were reported dead or too ill to respond, they were replaced by a random member of their stratum.
For ease in data collection, and to minimize study costs, colleagues in the 2004 survey were asked to write the number corresponding to their credibility judgment in a circle or box on the back of a personal cheque that, when cashed or deposited, would find its way through the banking system back to me. In a substudy, the cheque amounts were randomized to $5 or $10. Nonrespondents were sent up to two follow-up letters containing self-addressed postcards.
The data from both surveys were collated by hand and entered into contingency tables that were eyeballed, graphed, and analyzed by contingency chi-squared. Thereafter, all documents containing personal information were shredded.
Results
The 2001 survey results from 150 volunteers are summarized in Figure 1. Fully 90% of these internists found the industry-initiated trial to be less credible than the university investigator–initiated trial. The industry-initiated trial was 10 times as likely as the university investigator–initiated trial to be rated “not at all credible,” of “only low credibility,” or of “questionable credibility” (61% vs. 6.0%; p < .001). Moreover, the university investigator–initiated trial was 10 times as likely as the industry-initiated trial to be rated “of high credibility” or “of extremely high credibility” (86% vs. 8.6%; p < .001).
Figure 1. 2001 survey results.
Responses to the 2004 survey of a random sample of 150 CSIM members were obtained from 117 members (response rate = 78%); these responses are presented in Figure 2. No apparent differences in either response rates or responses were found in the different age, sex, or cheque-size strata.
Figure 2. 2004 survey results.
Responses in this 2004 survey were virtually identical among the subsets of internists randomized to receive just one or both scenarios. This result suggested the absence of any important juxtaposition bias (juxtaposing examples of “good” and “bad” trials) in the volunteer 2001 survey. In view of this finding, the responses of the three subsets in the 2004 survey were combined in subsequent analyses.
Internists’ opinions in the random sample survey of 2004 tended to be even more negative than those measured in the volunteer survey of 2001. The proportion of industry-initiated trials judged “not at all credible” or “of low credibility” rose from 60% in 2001 to 80% in 2004. This result suggests that the 2001 findings were probably not the result of volunteer bias (in which internists with a low opinion of industry-initiated research would be more likely to take part than internists with a high opinion of industry-initiated research).
In the random sample 2004 survey, the industry-initiated trial was 20 times as likely as the university investigator–initiated trial to be rated “not at all credible,” of “only low credibility,” or of “questionable credibility” (81% vs. 4%; p < .001). Moreover, the university investigator–initiated trial was 18 times as likely as the industry-initiated trial to be judged “of high credibility” or “of extremely high credibility” (87% vs. 5%; p < .001).
Discussion
Drug trials initiated and carried out by the drug industry are of low, and probably worsening, credibility in the eyes of Canadian internists. This finding is all the more remarkable when it is contrasted with measures of their methodological quality. As shown by Bodil Als-Nielsen and colleagues, “for-profit” trials are more likely to report concealed allocation and double-blinding than “non-profit” trials.4
On the other hand, public accusations that the drug industry suppresses unfavourable trial results have become commonplace (e.g., the current controversy over Vioxx). In addition, websites6 and blogs7 devoted to criticizing the drug industry are rapidly expanding. Moreover, there may be a growing recognition of Dr. Als-Neilson et al.’s finding that the same drug is five times as likely to be recommended in a “for-profit” trial report as in a “non-profit” trial report.
What might industry do to reverse this trend? One obvious move (but clearly self-serving to the lead author and his colleagues) would be for industry to canvass university-based trialists for independent investigators interested in their new drugs, and then fund totally independent trials of the drugs. In this context, it is my view that the credibility problem would not be resolved by greater reliance on for-profit “contract research organizations.” These private enterprises face difficult conflicts of interest when their future “sales” are affected by their current delivery of “successful” trials.
On the other hand, given the growing pressure on drug manufacturers to abandon free lunches, starter packs, pens, and other inducements, they could simply abandon the top-down physician-directed approach to increasing sales. Rather, industry could completely bypass increasingly skeptical physicians and concentrate all its efforts on the direct-to-consumer advertising that increases sales from the bottom up.8
Finally, it has been 3 years since the most recent survey of Canadian internists reported here. I hope that this brief report stimulates other CSIM members to repeat and expand surveys of the credibility of industry-initiated versus independent university-initiated trials.
Acknowledgements
Dr. Mahesh K. Raju, then president of the Canadian Society of Internal Medicine, kindly provided a mailing list of CSIM members. Barbara Sackett assisted in the design, dissemination, coding, and analysis of the survey questionnaires. Portions of this work have appeared in abstracts and oral presentations of the Canadian Society of Internal Medicine and the Society for Clinical Trials.
Conflicts of Interest
Dave Sackett has been wined, dined, supported, transported, and paid to speak by countless pharmaceutical firms for over 40 years, beginning with two research fellowships and interest-free loans that allowed him to stay to finish medical school. Dozens of his randomized trials have been supported in part (but never in whole) by pharmaceutical firms, who never received or analyzed primary data and never had veto power over any reports, presentations, or publications of the results. He has twice worked as a paid consultant to advise pharmaceutical firms whether their products caused lethal side effects; on both occasions he told them “yes.” He has testified as an unpaid expert witness for a stroke victim who successfully sued a manufacturer of oral contraceptives, and as a paid expert in preparing individual suits against manufacturers of female hormones and nonsteroidal anti-inflammatory drugs and in a class-action suit against a manufacturer of prosthetic heart valves. He was paid by a pharmaceutical firm to develop “levels of evidence” for determining the causation of adverse drug reactions. His wife inherited and sold stock in a pharmaceutical company. While head of a division of medicine, he enforced the banning of drug-detail personnel from clinical teaching units (despite the threat of withdrawal of drug industry funding for resident research projects). He received the Pharmaceutical Manufacturers’ Association of Canada Medal of Honour (and cash) for Contributions to Medical Science in Canada for the decade 1984–1994. His most recent award (the 2005 Baxter International Foundation Prize for Health Services Research) was sponsored by the Baxter International Foundation. The initial version of this statement was filed in 2002 at http://bmj.bmjjournals.com/cgi/content/full/324/7336/539/DC1.
References
1. Davidoff F, DeAngelis D, Drazen JM, et al. Sponsorship, authorship and accountability. Can Med Assoc J 2001;165:786–8.
2. Eichenwald K, Kolata G. Drug trials hide conflicts for doctors. The New York Times May 16, 1999.
3. Canadian Medical Association. CMA Policy. Physicians and the pharmaceutical industry (update 2001). Ottawa: Author; 2001.
4. Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials. A reflection of treatment effect or adverse events? JAMA 2003;290:921–8.
5. Chaudhry S, Schroter S, Smith R, Morris J. Does declaration of competing interests affect readers’ perceptions? A randomised trial. BMJ 2002;325:1391–2.
6. No Free Lunch. Home page. Author, 2007; http://www.nofreelunch.org/. Accessed December 18, 2007.
7. Health Care Renewal. Home page. Author, 2007; http://hcrenewal.blogspot.com/. Accessed December 18, 2007.
8. CNN Money.com. Business and financial news. Cable News Network; http://money.cnn.com/news/newsfeeds/articles/newstex/IBD-0001-21941278.htm.
Sackett DL. Industry-initiated drug trials are far less credible to Canadian internists than investigator-initiated trials. Can J Gen Intern Med 2008;3(1):29–32.
|
