Archive for the ‘Marya Zilberberg’ Category

Guest Blog: The Beautiful Uncertainty of Science

Marya Zilberberg
Wednesday, February 2nd, 2011

Marya Zilberberg, MD, MPH, is a physician health services researcher with a specific interest in healthcare-associated complications and a broad interest in the state of our healthcare system. She is the Founder and President of EviMed Research Group, LLC, a consultancy specializing in epidemiology, health services and outcomes research. She is also a professor of Epidemiology at the University of Massachusetts, Amherst. Dr. Zilberberg is a member of multiple journal editorial boards and is frequently invited to speak about evidence-based medicine, research methods and healthcare-associated complications. She blogs about healthcare at Healthcare, etc.

I am so tired of this all-or-nothing discussion about science! On the one hand there is a chorus singing praises to science and calling people who are skeptical of certain ideas unscientific idiots. On the other, with equal penchant for eminence-based thinking, are the masses convinced of conspiracies and nefarious motives of science and its perpetrators. And neither will stop and listen to the other side’s objections, and neither will stop the name-calling. So, is it any wonder we are not getting any closer to the common ground? And if you are not a believer in the common ground, let me say that we are only getting farther away from the truth, if such a thing exists, by retreating further into our cognitive corners. These corners are comfortable places, with our comrades-in-arms sharing our, shall we say, passionate opinions. Yet this is not the way to get to a better understanding.

Because I spend so much time contemplating our larger understanding of science, the title “Are We Hard-Wired to Doubt Science” proved to be a really inflammatory way to suck me into thinking about everything I am interested in integrating: scientific method, science literacy and communication and brain science. The author, on the heels of doing a story on the opposition to smart meters in California, was led to try to understand why we are so quick to reject science:

But some very intelligent people I interviewed had little use for the existing (if sparse) science. How, in a rational society, does one understand those who reject science, a common touchstone of what is real and verifiable?

The absence of scientific evidence doesn’t dissuade those who believe childhood vaccines are linked to autism, or those who believe their headaches, dizziness and other symptoms are caused by cellphones and smart meters. And the presence of large amounts of scientific evidence doesn’t convince those who reject the idea that human activities are disrupting the climate.

She goes on to think about the different ways of perceiving risk, and how our brains play tricks on us by perpetuating our many cognitive biases. In essence, new data are unable to sway our opinion because of rescue bias, or our drive to preserve what we think we know to be true and to reject what our intuition tells us is false. If we follow this argument to its logical conclusion, it means that we just need to throw our hands up in the air and accept the status quo, whatever it is.

I happen to think that the author missed an opportunity to educate her readers about why we need to come to a better understanding and how to get there. The public (and even some of my fellow scientists) needs to understand what science is and, even more importantly, what it is not.

First, science is not dogma. Karl Popper had a very simple litmus test for scientific thinking: He asked how you would go about disproving a particular idea. If you think that the idea is above being disproved, then you are engaging in dogma and not science. The essence of scientific method is developing an hypothesis from either a systematically observed pattern or from a theoretical model. The hypothesis is necessarily formulated as the null, making the assumption of no association the departure point for proving the contrary. So, to “prove” that the association is present you need to rule out any other potential explanation for what may appear to be an association. For example, if thunder were always followed by rain, it might be easy to engage in the “post hoc ergo propter hoc” fallacy and conclude that thunder caused rain. But before this could become a scientific theory, you would have to show that there was no other explanation that would disprove this association.

So, the second point is that science is driven by postulating and then disproving the null hypotheses. By definition, an hypothesis can only be disproved if we 1). the association exists, and 2). the constellation of phenomena is not explained by something else. And here is the third and critical point, the point that produces equal parts frustration and inspiration to learn more: That “something else” as the explanation of a certain association is by definition informed only by what we know today. It is this very quality of knowledge production, the constancy of the pursuit, that lends the only certain property to science, the property of uncertainty. And our brains have a hard time holding and living with this uncertainty.

The tension between uncertainty and the need to make public policy has taken on a political life of its own. What started out as a modest storm of subversion of science by politics in the tobacco debate, has now escalated into a cyclone of everyday leveraging of the scientific uncertainties for political and economic gains. After all, how can we balance the accounting between the theoretical models predicting climate doom in the future and the robust current-day economic gains produced by the very pollution that feeds these models? How can we even conceive that our food production system, yielding more abundant and cheaper food than ever before, is driving the epidemic of obesity and the catastrophe of antimicrobial resistance? And because we are talking about science, and because, as that populist philosopher Yogi Berra famously quipped, “Predictions are hard, especially about the future,” the uncertainty of our estimates overshadows the probability of their correctness. Yet by the time the future becomes present, we will be faced with potentially insurmountable challenges of a new world.

I have heard some scientists express reluctance about “coming clean” to the public about just how uncertain our knowledge is. Nonsense! What we need under the circumstances is greater transparency, public literacy and engagement. Science is not something that happens in the bastions of higher education or behind the thick walls of corporations. Science is all around and within us. And if you believe in God, you have to believe that God is a scientist, a tinkerer, always looking for a more elegant solution. The language of science that may seem daunting and obfuscatory. Yet do not be afraid — patterns of a language are easy to decipher with some willingness and a dictionary. Our brains are attuned to the most beautiful explanations of the universe. Science is what provides them.

Self-determination is predicated upon knowledge and understanding. Abdicating our ability to understand the scientific method leaves us subject to political demagoguery. Don’t be a puppet. We are all born scientists. Embrace your curiosity, tune out the noise of those at the margins who are not willing to engage in a sensible dialogue, leave them to their schoolyard brawling. And likewise, leave the politicians, corporate interests, and, alas, many a journalist, and start learning the basics of scientific philosophy and thought. Allow the uncertainty of knowledge excite and delight you. You will not be disappointed.

Guest Blog: Why Medical Testing Is Never a Simple Decision

Marya Zilberberg
Monday, December 20th, 2010

Marya Zilberberg, MD, MPH, is a physician health services researcher with a specific interest in healthcare-associated complications and a broad interest in the state of our healthcare system. She is the Founder and President of EviMed Research Group, LLC, a consultancy specializing in epidemiology, health services and outcomes research. She is also a professor of Epidemiology at the University of Massachusetts, Amherst. Dr. Zilberberg is a member of multiple journal editorial boards and is frequently invited to speak about evidence-based medicine, research methods and healthcare-associated complications. She blogs about healthcare at Healthcare, etc.


A couple of days ago, Archives of Internal Medicine published a case report online. Now, it is rather unusual for a high impact journal to publish even a case series, let alone a case report. Yet this was done in the vein of highlighting their theme of “less is more” in medicine. This motif was announced by Rita Redberg many months ago, when she solicited papers to shed light on the potential harms that we perpetrate in healthcare with errors of commission.

The case in question is one of a middle-aged woman presenting to the emergency room with vague symptoms of chest pain. Although from reading the paper it becomes clear that the pain is highly unlikely to represent heart disease, the doctors caring for the patient elect to do a non-invasive CT angiography test, just to “reassure” the patient, as the authors put it. Well, lo’ and behold, the test comes back positive, the woman goes for an invasive cardiac catheterization, where, though no disease is found, she suffers a very rare but devastating tear of one of the arteries in her heart. As you can imagine, she gets very ill, requires a bypass surgery and ultimately an urgent heart transplant. Yup, from healthy to a heart transplant patient in just a few weeks. Nice, huh?

The case illustrates the pitfalls of getting a seemingly innocuous test for what appears to be a humanistic reason – patient reassurance. Yet, look at the tsunami of harm that followed this one decision. But what is done is done. The big question is, can cases like this be prevented in the future? And if so, how? I will submit to you that Bayesian approaches to testing can and should reduce such complications. Here is how.

First, what is Bayesian thinking? Bayesian thinking, formalized mathematically through Bayes theorem, refers to taking the probability of disease being there into account when interpreting subsequent test results. What does this mean? Well, let us take the much embattled example of mammography and put some numbers to the probabilities. Let us assume that an otherwise healthy woman between 40 and 50 years of age has a 1% chance of developing breast cancer (that is 1 out of every 100 such women, or 100 out of 10,000 undergoing screening). Now, let’s say that a screening mammogram is able to pick up 80% of all cancers that are actually there (true positives), meaning that 20% go unnoticed by this technology. So, among the 100 women with actual breast cancer of the 10,000 women screened, 80 will be diagnosed as having cancer, while 20 will be missed. OK so far? Let’s go on.

Let us also assume that, in a certain fraction of the screenings, mammography will merely imagine that a cancer is present, when in fact there is no cancer. Let us say that this happens about 10% of the time. So, going back to the 10,000 women we are screening, of 9,900 who do NOT have cancer (remember that only 100 can have a true cancer), 10%, or 990 individuals, will still be diagnosed as having cancer. So, tallying up all of the positive mammograms, we are now faced with 1,070 women diagnosed with breast cancer. But of course, of these women only 80 actually have the cancer, so what’s the deal? Well, we have arrived at the very important idea of the value of a positive test: this roughly tells us how sure we should be that a positive test actually means that the disease is present. It is a simple ratio of the real positives (true positives, in our case the 80 women with true cancer) and all of the positives obtained with the test (in our case 1,070). This is called positive predictive value of a test, and in our mammography example for women between ages of 40 and 50 it turns out to be 7.5%. So, what this means is that over 90% of the positive mammograms in this population will turn out to be false positives.

Now, let us look at the flip side of this equation, or the value of a negative test. Of the 8,930 negative mammograms, only 20 will be false negatives (remember that in our case mammography will only pick up 80 out of 100 true cancers). This means that the other 8,910 negative results are true negatives, making the value of a negative test, or negative predictive value, 8,910/8,930 = 99.8%, or just fantastic! So, if the test is negative, we can be pretty darn sure that there is no cancer. However, if the test is positive, while cancer is present in 80 women, 900 others will undergo unnecessary further testing. And for every subsequent test a similar calculus applies, since all tests are fallible.

Let’s do one more maneuver. Let’s say that now we have a population of 10,000 women who have a 10% chance of having breast cancer (as is the case with an older population). The sensitivity and specificity of mammography do not change, yet the positive and negative predictive values do. So, among these 10,000 women, 1,000 are expected to have cancer, of which 800 will be picked up on mammography. Among the 9,000 without cancer, a mammogram will “find” a cancer in 900. So, the total positive mammograms add up to 1,700, of which nearly 50% are true positives (800/1,700 = 47.1%). Interestingly, the negative predictive value does not change a whole lot (8,100/[8,100 + 200]) = 97.6%, or still quite acceptably high). So, while among younger women at a lower risk for breast cancer, a positive mammogram indicates the presence of disease in only 8% of the cases, for older women it is about 50% correct.

These two examples illustrate how exquisitely sensitive an interpretation of any test result is to the pre-test probability that a patient has the disease. Applying this to the woman in the case report in the Archives, some back-of-the-napkin calculations based on the numbers in the report suggest that, while a negative CT angiogram would indeed have been reassuring, a positive one would only create confusion, as it, in fact, did.

To be sure, if we had a perfect test, or one that picked up disease 100% of the time when it was present and did not mislabel people without the disease as having it, we would not need to apply this type of Bayesian accounting. However, to the best of my knowledge, no such test exists in today’s clinical practice. Therefore, engaging in explicit calculations of what results can be expected in a particular patient from a particular test before ordering such a test can save a lot of headaches, and perhaps even lives. In fact, I do hope that the developers of our new electronic medical environments are giving this serious thought, as these simple algorithms should be built into all decision support systems. Bayes theorem is an idea whose time has surely come.