Through blogs and comments, patients and experts explore what it takes to find good health care and make the most of it.
Guest Blog: The Confusing Morass of Medical Evidence
Ken Covinsky | July 28, 2011
Practitioners of evidence-based medicine use published evidence from the medical literature to guide them as they try to provide the best care for each patient. But sometimes the medical literature just feels like a big morass.
The difficulty applying the medical literature to patient care is illustrated by two studies published in the past few months in the very best medical journals. The studies give precisely opposite answers to an important clinical question.
The studies focus on the management of COPD (sometimes called emphysema or chronic bronchitis). COPD, which causes very distressing symptoms such as shortness of breath, is very common, especially in Geriatric patients. It is one of the most common causes of hospitalization, and costs Medicare billions of dollars. One of the studies, published in the NEJM, was recently reviewed on GeriPal. The other study was published in the Annals of Internal Medicine this month.
Patients with COPD are often treated with long acting inhaler medicines to control symptoms, and hopefully prevent exacerbations that lead to hospitalization. However, there are two types of long acting medicines that are available: Beta agonists (such as salmererol) and anticholinergics (such as tiotropium). It is has been a guess as to which of these medicines are better, and there has been little to guide clinicians as to which medicine to use.
Well, after these two studies, it is still a guess. It is worth briefly reviewing each of these studies, since they provide a useful allegory about the limitations of evidence-based medicine. One could teach a entire class on evidence based medicine or epidemiology focused on these two papers. And if you have a good understanding of the strengths and limitations of both studies, you can have a feeling of confidence in your skill critically evaluating the medical literature.
The more recent study published in the Annals of Internal Medicine leveraged an administrative database in Ontario that had information on nearly all patients in the Province over the age of 65 with diagnoses of COPD. They used billing codes from patient encounters to identify patients with COPD. Then they determined whether these patients were first prescribed a Beta agonist or an anticholinergic. Since Ontario provided these patient prescription coverage, use of these drugs was recorded in the database. The study then compared rates of death and hospitalization in elders prescribed Beta agonists and anticholinergics. The study suggested patients given B-agonists did better. Patients given anticholinergics had 14% higher risk of death and a 13% higher risk of hospitalization for COPD than patients given Beta Agonists.
But, this study does not prove B-agonists are better. This study was not randomized. It compared patients who happened to be given a B-agonist against those who happened to be given an anticholinergic. It is possible patients given anticholinergics were sicker than patients given Beta agonists and that higher levels of illness severity, not drug choice explain outcome differences. The authors did a lot of analyses to address this possibility that seem to suggest that differences in illness severity do not explain their results. However, with this type of study design, there is simply no way to know for certain are due to the drug choice rather than other patient characteristics.
The other study, published in NEJM was recently discussed in detail on GeriPal, so we will only discuss it briefly here. This study was a randomized trial comparing a B-agonist and anticholinergic. It showed that patients who were treated with the anticholinergic did better'they were less likely to be hospitalized for COPD.
Those who know guidelines for medical evidence might instinctively say 'duh'what's the problem?? Randomized studies are the gold standard, so I am going to believe the NEJM randomized study. Bring out the anticholinergics.' After all, randomization (when successful) assures the groups are virtually identical except for the choice of drug. As a result, outcome differences can be attributed to the drug.
Unfortunately, it is not so simple. As we discussed, the subjects in the NEJM study do not look like most patients with COPD. They were much younger than the typical patient. (Average age of 62). Also, there was an absurd set of enrollment restrictions. Most patients with COPD would probably not qualify for the study. Even though the vast majority of patients with COPD have several other medical problems, the study excluded patients with many co-existing illnesses. So, while the ideal study design of randomization was used, the nature of the patients in the study markedly limit the ability to generalize the findings to real world COPD patients. (In contrast, the patients in the Annals study closely resemble real world patients with COPD).
So, what should one do? If you have a patient with COPD, should you use a long acting Beta-Agonist or an anticholinergic?
My first answer would be that I have no idea.
But of course, we have real patients to treat, and we have to make a decision. So my vote would be for the B-agonist, especially for the older patients I see. I slightly favor the results of the Annals study because the patients were much more like the patients I see with COPD and outcome of mortality examined in this study is more important than the hospitalization outcome assessed in the NEJM study. The stronger study design of the NEJM study does not make up for focus on patients that bear limited resemble to the typical patient.
But, in the end the choice of drug remains an educated guess. As is often the case, the application of evidence based medicine to a common clinical question yields as much confusion as clarification.
More Blog Posts by Ken Covinsky
Ken Covinsky, M.D., MPH, is a Professor of Medicine in the Division of Geriatrics at University of California, San Francisco. Dr. Covinsky leads the division’s research training and mentoring programs, conducts his own research program funded by the National Institute on Aging, and cares for patients in the San Francisco VAMC Geriatrics Clinic and inpatient medical service. He contributes regularly to GeriPal, a geriatrics and palliative care blog, where this post was originally published. You can follow him on Twitter @geri_doc.
Comments on this post
Please note: CFAH reserves the right to moderate all comments posted to the Prepared Patient Blog. Any inappropriate postings will be removed.
July 28, 2011 at 3:58 PM
Good to know sometimes there are more questions than answers for the providers too. As patients, some of us are trying to do our part to control costs by asking if there are alternative, less costly treatments or generic drug options. We expect complete, full answers, keeping in mind the evidence is conflicted at times. http://whatstherealcost.org/video.php?post=five-questions
|Carolyn Thomas says|
August 2, 2011 at 10:48 PM
Thanks Susan for including that link to this terrific short video. So simple, yet so profound. And thank you Dr. Ken for such clear examples of how confusing evidence-based medicine can be.
There is yet another confusing issue not mentioned here about the study reported in the NEJM. The study's conclusion ("tiotropium is more effective than salmeterol in preventing exacerbations") sounds reasonably convincing - until you read the fine print disclosing that the study was funded by drug companies Boehringer-Ingelheim and Pfizer.
And then you remember that Tiotropium bromide capsules for inhalation are made by Boehringer-Ingelheim and Pfizer under the trade name Spiriva.
Add Your Comment
|Evidence Is Only One Data Point in Our Treatment Decisions
Jessie Gruman | November 13, 2013
|Accuracy of Health News: Pressure on Journalists, Consequences for Us
Jessie Gruman | October 2, 2013
|Robotic Surgery Roundup: Take Me Out to the Ballgame and Much More
Gary Schwitzer | August 16, 2013
|Whatever Happened to Underuse of Medical Services?
Trudy Lieberman | April 2, 2013
For a Less Biased Study, Try Randomization
“White-Noise” Therapy Alone Not Enough to Curb Tinnitus