I recently read that the CDC has recommended that all people be tested for HIV as part of a routine check up with one’s doctor. You can read about it here. At first I though this was a great idea, that may have important implications for reducing a preventable and incurable disease. But then I read this book, Calculated Risks, by the psychologist Gert Gigerenzer, who is associated with the Max Planck Institute for Human Development in
Gigerenzer discusses the fact that people are particularly bad at reasoning about probabilities, particularly in respect to understanding base rates. Now, this has long been known since the pioneering studies by Daniel Kanneman and Amos Tversky in the 1970s, but Gigerenzer has followed up on these ideas in new domains.
I recently posed the following scenario to about ten friends:
Imagine you go to your doctor, who informs you that you should take an HIV test as a routine part of your checkup. You are a heterosexual who has been in a few relationships over the past five years, you have never used IV drugs, and you have no other risk factors for HIV. The doctor informs you that the probability of someone like you actually having HIV are quite low, something in the order of 1 in 10,000. He also tells you that the test is 99.99 percent accurate in providing a negative test when the person is actually negative. A week later, you receive a call from the doctor informing you that the test came back positive, indicating that you are infected with HIV.
What would you do? Please circle all that apply.
- Commit suicide
- Ask to be tested again
- Inform your current sexual partner
- Do nothing – forget about it.
Given the facts as stated above, what do you believe is the probability that given a positive test, that you actually have acquired HIV?
- 0%
- 10%
- 30%
- 50%
- 70%
- 90%
- 99%
- 99.99%
We’ll get to the results of this informal study in a minute. But first, what actually is the probability? The vast majority of people would likely say 99.99%, because the test is 99.99% accurate. However, this is to forget the fact that the actual prevalence rate of the disease is extremely low – 1 in 10,000 or .001 percent of the population in question, is actually HIV positive. Why does this matter?
Well, it matters because of something known as Bayes Theorem. Bayes Theorem is a statistical solution to what is know as the inverse probability problem. The idea is this: The probability of event A conditional on event B is different than the probability of event B conditional on event A. So, the probability of a person being HIV positive given a positive HIV test is different from the probability of having a positive HIV test given the probability of being HIV positive. Bayes Theorum is a mathematical expression of the relationship among these events.
Think of it this way. The HIV test is highly accurate. 99.99 percent of the time, it will give the right answer: HIV + given HIV +, or HIV – given HIV -. But .01 percent of the time, it will give the wrong answer: HIV + given HIV – or HIV – given HIV +.
Gigerenzer argues that one should think of it in terms of frequencies rather than probabilities. He poses the problem this way: 10,000 people go in to have the test. 1 person will test positive and actually be positive. But 1 person (.01 % of 10.000 = 1) will test positive and actually be negative. And 9,998 will test negative and actually be negative. Hence, if you are from a population in which the incidence rate is extremely low, the probability that the test is inaccurate is quite high. In this particular case, it is .50. You have a 50/50 chance of actually being positive if you have a positive result and are from the population in question.
The problem is that even a test with a high degree of accuracy, if used many many times, will once in a while create a false positives and false negatives just by chance. Blood samples may get the wrong identification number, or samples could get polluted, for instance. As much as they might not like to admit it, doctors are not infallible, and neither are the technicians who actually analyze the samples. I’ve made mistakes in analyzing my own data, as my reviewers often, and often unkindly, point out.
But people don’t often think in Bayesian terms, and it is likely that the only time you encounter Bayes Theorem is in that boring college statistics class (like mine), in which it will quickly go in one ear and out the other. (I can hear my students thinking – “Inverse probability – whatever”). But here is the scary thing, from page 121 of Calculated Risks: “At a conference on AIDS held in 1987, former senator Lawton Chiles of Florida reported that of 22 blood donors in Florida who were notified that they had tested HIV-positive with ELIZA [a common test for HIV], 7 had committed suicide…The medical text documenting this tragedy many years later informed the reader that even the results of both AIDS tests (Eliza and Western blot) are positive, the chances are only 50-50 that the individual is infected.”
This blew my mind! What percentage of those 7 people could have been spared their fate by understanding Bayes Theorem and inverse probability? The thing that scares me the most is the question of whether medical professionals who inform their patients of their HIV status actually know how to calculate a Bayesian probability problem themselves, or are trained in this fallacy of the test so that they can accurately inform their patients. To find this out, I informally asked a variety of friends who hold MDs the same question that I asked my friends – the hypothetical senario. None of them gave the correct answer on the spot (then again, I was asking them at parties so they didn't have calculators on hand). However, several of them answered the question partially correctly – one mentioned that you have to take the overall incidence rate into question. But several gave the completely wrong answer – 99.99%! One mentioned, “I remember in medical school that there is some formula that one can use to figure this out, but I suck at statistics and have no idea how to calculate it.”
So this has gotten me thinking: Should we as a society simply mandate generalized HIV testing? Should we not at least consider the fact that people – even doctors, even professors, even statistics professors – are notoriously bad at calculating probabilities, and specifically Bayesian inverse probability problems? How can we better inform patients about their actual probabilities of having diseases?
And how vast is the problem? What about other diagnostic tests? What about in other domains in which people must make judgments?
Returning to my friends, what were their answers?
Well, on the second part, 7 said 99.99%, 1 said 99%, 1 said 90% and only one said 50%. On the first part, 2 said they would commit suicide. All but 2 said that they would get retested, and only three said they would tell their current sexual partner (I would lament about this for a while, but the fact is that this evidence isn’t fair as I didn’t acquire information as to whether any of them actually have a current sexual partner, but this is an entirely different issue that wouldn’t fit on the margin of this page.)
So let's pretend that my informal sample was a real sample (big leap of faith considering the various raffish and seedy characters I consider my friends) and say that 20% of people would commit suicide if they received a positive HIV test. Imagine we test all 300,000,000 people in the U.S. That's a lot of needles! Just by chance alone, 3,000,000 would test positive and actually be negative. Of those, 600,000 (20% of 3,000,000) would kill themselves. In 2002, 16,000 people in the U.S. perished due to the AIDS epidemic. We could in a single year increase the death rate due to AIDS by some 375.00%. Simply by having universal testing. And the really not funny thing is that 1,500,000 of these people are not even infected with HIV, but were victims of our reliance and confidence in the medical establishment.If the CDC goes through with this plan, I hope they have better statistics professors than I teaching health care providers how to talk Bayesian with their patients.
