Prediction and postdiction

I share in the horror over the massacre at Virginia Tech. Part of the shock for all of us in academia is that we tend to think of school as the diving board of life: a place where lives begin, not end. I see my own students and colleagues in the images of both the victims as well as the perpetrator. And that both saddens and terrifies me.

However, I find it upsetting that people are criticizing Virginia Tech for ignoring warning signs. I find that response represents a severe and all-too-common error in thinking about direct versus inverse probability.

The basic premise rests in the fact that the following statements are not identical:

1. Given person A wears a trenchcoat, is quiet, and writes plays involving murder, there is a high probability that he will go on a shooting spree.
2. Given person A went on a shooting spree, there is a high probability that he wore a trenchcoat, was quiet, and wrote plays involving murder.

The first statement is one that is a matter of direct probability. You multiply the proportion of people who wear trenchcoats, who are quiet, and who write plays about murder, and you get a number which represents the proportion of the population that are going to go on shooting sprees. However, there is a little problem: a lot of people wear trenchcoats, are quiet, and write plays about murder. Round them up, and you might find a group that includes a significant proportion of the undergraduate population. However, a significant proportion of the undergraduate population will not end up going on a shooting spree. That concerns the error inherent in mistaking direct and inverse probability.

A 17th century minister named Thomas Bayes noted that probability of event A conditional on B is generally different than the probability of B conditional on A. Bayes came up with a solution that links them mathematically by a simple equation called Bayes Theorem. In short, the theorem explains that the relationship between direct and inverse probability is a function of the base rate, or frequency, of the two events. Psychologists have long known that people ignore base rates, resulting in serious errors in reasoning that can be fatal.

For example, there are many cases of heterosexual, non-IV drug users who killed themselves because they received news of a positive HIV test. Their doctors probably informed them that the HIV test has an accuracy of 99.99 percent. However, that does not mean that they can be 99.99% sure that they have HIV. Why? Because of the base rate: tens of thousands of people take the HIV test, and HIV prevalence is extremely low (about 1 in 10,0000) among that particular population (heterosexual non-IV drug users). Every once in a while, given the fact that tens of thousands of tests are run, a negative sample will test positive (the 0.01 percent inaccuracy part). So the actual probability of being HIV positive given a positive HIV test (again, if you are a heterosexual non-IV drug user) is actually about 50% (I calculated this using Bayes Theorem). But most doctors I've interviewed never even heard of Thomas Bayes or his theorem, and make the same mistake their patients do in assuming that direct and inverse probability are the same.

In a similar way, it is easy to say, after the fact, the postdiction that someone should have known, that someone should have done something, that the university was irresponsible for not having seen the warning signs. But the problem with these statements is that they ignore the base rate of the frequency of these warning signs. A lot of college students show warning signs, and many students show warning signs that are considerably worse than those exhibited by Mr. Cho. And unless one were to come up with a better diagnostic tool than trenchcoats or murderous plays - like a crystal ball - I don't believe it is possible to predict who is just an awkward student (of which there are many) and who is going to be a serial killer. I hope we someday can tell them apart, but knowing a bit about human nature - and the way we make errors in our reasoning about probability - makes me doubt it.

Experiments in Punditry

I recently appeared as a talking head on an hour-long cable news show concerning a bill introduced in California that would ban spanking. I was asked if I would make some points as to the limitations of the bill, which I agreed to, so long as I could clarify that I was anti-spanking and that there are many alternatives to corporal punishment that are more effective and less harmful.

But then what problem do I have with a bill banning spanking? Well, basically I believe that an ounce of prevention is worth a pound of cure. If we as a society really care about children, we should teach children how to be good parents – by being good parents ourselves, and by developing programs that teach pregnant women and their partners and families about the challenges of raising children, and how to deal with all the stresses inherent in being a parent. To intervene AFTER the spank has occurred, as this law does is pointless – for whatever harm is inherent in spanking (if any) is already done, when the entire harm could have been avoided in the first place if we only took the steps of educating people before the harm occurs. Also, there are many people who are unfit to be parents, and they need to be discouraged from fostering children until they are ready to be assume what amounts to one of the most difficult social roles imaginable.

We could go on for hours about semantic issues related to the meaning of a spank versus abuse, ontological issues related to what is discipline and punishment, teleological issues related to what is the aim of good or bad parenting, epistemological issues related to how we learn to practice parenthood, and so on. That is an issue for another blog.

My main concern has to do with the use and abuse of research to put forth an agenda and the nature of scientific objectivity. The other guests with whom I was discussing these issues, while I respected their thoughts and viewpoints, all had some agenda. For instance, one was a member of the Family Research Council, a Christian conservative think tank. The other owned a company that runs programs that teaches parenting skills. What struck me was the extent to which these individuals were emotionally invested in the issue. I never really thought much about spanking, because there seems to be larger issues at hand – pardon the pun – such as child physical and sexual abuse. But I respect that in certain cultural contexts, spanking is a normative practice, and even if in some cultures it may not be prescriptive, to apply a universal standard would be to ignore the fact that parental discipline occupies a complex cultural space in which various factors – cultural, social, educational, economic, sociological, and so on – intersect. The inner workings of the family is not a sphere in which a “one size fits all” policy seems appropriate.

I have always taken issue with extremes. “Anything goes” and “Nothing goes” seem to be fruitless approaches towards thinking through most problems. But I believe that the vast majority of people, the vast majority of the time, shy away from thinking through an issue but rather feel through it. Steven Corbert was onto something when he popularized the term truthiness. Most people’s convictions are based upon how they feel, not how they think, and often what we feel we should do, and what we really should do, are in stark contrast. Too many people trust their guts, and unless you are a gourmand, you probably should question what your stomach tells you.

So it was kind of fun to be on television. They had a make-up lady do my face and hair - it was the first time I had to wear makeup since that time in graduate school when I went to a party in drag. There were lights and cameras and satellite hook-ups and all sorts of interesting things. I went into this with one thought in my mind: “Imagine the hypocrisy of a state that practices the ultimate form of corporal punishment – the death penalty – trying to outlaw spanking.” It didn’t set well with the liberal side of me. But neither does spanking children. I do regret one thing that I said…that one alternative to spanking is distracting children – that children have short attention spans and that if they want to touch a hot stove, to distract them with something else, like the television.”

I want to officially say here that I think that spanking is far better than forcing a child to watch the garbage that counts as television these days. I had a nightmare last night of parents following the sage advice of Professor Experimentaholic and putting their children in front of Bill O’Reilly's No Spin Zone. Now there should be a law against THAT. I'd be the first in line to vote for that.

At another level, I guess I am being a bit hypocritical because I say this after having appeared on television myself. Oh well.

I don’t think I would ever make a very good pundit. Pundits tell you what to think, not how to think. Pundits tell you what is right from wrong, not how to determine for oneself what is right and wrong. Any bit of knowledge or wisdom that I could tell you in 30 seconds or less is probably not knowledge or wisdom worth having.

But what I didn’t get to talk about is the nature of the research on corporal punishment, the vast majority of which is fundamentally flawed. The main flaw comes in the form of mistaking correlation for causation. The vast majority of these studies do something like this: Get a group of parents and ask them whether they spank their child, or how frequently, or how hard. Then they measure behavioral problems in the two groups of children, do some multiple regression with dummy variables, and find that children of parents who spank them exhibit more behavioral problems. Then the wise researcher then goes and says that spanking causes behavioral problems. The problem with this approach should be apparent immediately to anyone since Aristotle’s Metaphysics. With this quasi-experimental design, you can not say that the spanking causes behavioral problems because it is equally possible that children with behavioral problems are more likely to get spanked. To do the study right, you need to take a group of new parents, and divide them into two groups – one who spank their children, and one that is prohibited from spanking, and measure the outcome. However, quite fortunately, I doubt there is an institutional board out there that would be willing to allow researchers to tell a group of women who wouldn’t spank their children to do so, and with vigor!

So we are left with some pretty questionable causal links. (There are a number of other problems I won’t go into, such as the fact that parents who spank their kids also are more likely to abuse them, which is not to say that spanking is a gateway practice into abuse, which is again another causal misinterpretation). And I don’t mind studies with poor causal links: they lead you down the path to more careful empirical investigations. My only fear is that given this fact, people – people with Ph.D.s and official-sounding titles at important-sounding institutions who really should know better than to pull this kind of nonsense - seem willing to make sweeping interpretations and inferences based on this paucity of available data. And the only rationale I can come to is that they were convinced of interpretations of the data long before they ever looked at any data of any kind. It doesn’t matter what the data says when you approach it with the zealotry of a pundit. You’ll find what you want in it – it will be like a Rorschach test. And you will always find what you want in your data when you have a vested financial or spiritual interest in the outcome. It’s not even worth citing a study, or citing research, or even pretending that you are even doing science at all.

I caution the reader, as I caution my research methodology class, that warning bells should start ringing in your head when any talking head says, “Research shows that…” or “Studies have determined that…” or variants thereof. What research? Which studies? Are there studies that show the opposite effect? Who did them? Who funded them? How were they collected? Who were the participants? Who were the interviewers? What scales were used? The devil is in the details, and I don’t want to start basing social policy or law or putting people in jail or funding some guy’s company that teaches parenting skills based upon poorly constructed and controlled quasi-experiments. Or worse still, evaluations of former participants of such programs. Or the fact that the president of this company has several lovely children who he never laid a hand upon and who have grown up to become well-adjusted productive members of society. This isn’t evidence of anything. But to parade this as evidence seems to me to be about as dubious as the kind of claims made by alternative resistance exercise machines, diet pills that will make you lose 10 pounds a week, and dead Nigerian kings whose heirs have to remove $10,000,000 from the country.

If I had my thirty seconds of sage advice, it would be this: I think one needs to have an open mind, but not so open that your brains fall out. Likewise, it is useful to have convictions, but you shouldn’t let those convictions lead you blindly down the road to conclusions. And it is easy to be a talking head, but a head not attached to a solid body of empirical data and carefully designed research should have its microphone turned off.

Notes from a fellow experimentaholic

One of the world's most famous experimentaholics was the British polymath Sir Francis Galton. Galton had a fetish for measuring, and he measured everything, from people's heights to their attractiveness to their intelligence to their fingertips. It was galton who first came up with the notion of regression to the mean, a common occurance such that people who do well on a test tend to do worse the second time one takes a test, and those who do really bad improve on the second. This has to do with the fact that people who do very well are smart, plus had a little luck. Test them again, and they may not be so lucky. Galton came up with the correlation coefficient and other handy statistical tools. He also was the father of eugenics, something perhaps not to be so proud about.

However, in leafing through issues of the journal Nature from a hundred odd years ago, I found this curious little solution to a common problem: the fact that one can have one's cake and eat it too, but not having the appetite to finish the whole thing. The problem is that those cut parts that lay exposed to air get stale. A common problem. Well, apparently, a "F.G." devised the solution described in the following Letters to Nature section. On further investigation, I discovered that our curious cake consumer F.G. is none other than Francis Galton himself!



This goes to show - it was easier back in the day to get something published in Nature. If only that were the case today!

Principles of indifference

It’s happened to you, I’m sure, as it has happened to me. You go about your life, working on your computer, editing files, processing data. You don’t get up in the morning thinking that today will be the day. The day that you fire up your computer, with all the hopes of getting that paper written, and instead are faced with the Blue Screen of Death. A message blinks “Can not find hard drive” or something equally vague and frightening.

It happened to me one winter evening in Ann Arbor, Michigan, where I was a post-doc. I hadn’t backed up any of my files for about six months. I restarted my Mac after it wouldn’t connect to the internet and . . . it never started again. Blank screen. I took it to dealers, tech support…nothing. Someone offered $5,000 to try and salvage the data off the hard drive, but I figured it wasn’t worth it. I could start from scratch, having learned an important lesson. But I lost hundreds of hours of time rewriting papers, reanalyzing data, and piecing together all that I had lost. Fortunately, some of the stuff was recoverable – colleagues who had drafts of manuscripts emailed me the files. But a lot was lost forever.

I ran into our friendly computer tech support director Harold the other day. It is because of Harold’s relentless campaign on backing up data that inspired me to burn all my work onto a CD every week. But it makes me wonder about the psychology involved in backing up one’s data. Even after my experience, I still don’t always back up my data. Some Fridays I just feel like going home – “I’ll do it tomorrow.” And I suppose one of these tomorrow’s I will fire up the machine and get that screen of death.

And who will I call? Harold. And often, what can Harold do? Nothing. This is not because Harold isn’t a great IT person – it is because Harold isn’t Zeus and can not pull a dues ex machina, coming down to earth and intervening with the ones and zeros that constitute the information on the hard drive.

I recently wrote a post about LaPlace’s 1822 Rule of Succession. Gigerenzer talks about a related notion to the Rule of Succession known as the Principle of Indifference. It basically has to do with what probability you assign to the possibility of an event given no prior knowledge. Recall my discussion of HIV infection – you can apply Bayes Theorem to situations in which you know the base rate – or frequency of – an prior probability, such as the proportion of people of a certain population that is infected with HIV. However, when you don’t know this prior probability, you are faced with a situation of indifference in which you assign equal probabilities to the likelihood of an event.

Electronics tend to follow an inverse power law in terms of their life-span. That is, take a hundred iPods. Once they leave the factory, 100 of them work. A month later, 99 work (i.e., you may have dropped it in the toilet by accident). A year later 80 of them work, and so on, until only 1 is still working some years down the line. And this is okay, because electronics are replaced with newer gadgets. But it is the probability of failure I am concerned with here: All electronic devices will someday fail. But why are we so confident that on any given time that you shut down your machine without backing up your data, that tomorrow when you start it, that the machine will in fact work rather than result in the blue screen of death.

I think the answer has to do with the law of succession, the same problem faced by Adam and Eve in the Garden of Eden. When you first buy your computer and turn it on, it is as if your computer gives you a white marble which you place in a box. Already in the box is a Blue Marble of Death, because you don’t know whether or not it will ever turn on again as it could be a lemon. Every time you start your computer and it works, your computer gives you another white marble from its box which you place in yours. Eventually your box will contain a lot of white marbles, representing your confidence in your computer.

But your computer doesn’t see it this way. Your computer is doing something different. It starts off with a lot of white marbles in its box, and one blue marble of death. And every time you turn it on, it selects a white marble from the box, and doesn’t replace it – it gives it to you, improving your confidence in it. So it may begin with a thousand white marbles and one blue marble of death. And the next time you turn it on, it selects a marble from its box. With high probability it is a white marble. But with every successive selection from this box, the probability increases and increases that the blue marble will be chosen until that day when suddenly, and unexpectedly, you turn on the computer, with a very high confidence in it working, while simultaneously, the computer selects the blue marble.

The result? You’re screwed.

The following figure provides a depiction of the expected probability that you have that your computer will start given a certain number of prior starts (by the rule of succession), the expected probability that your computer will select the blue ball of death randomly without replacement on any given start up. And then we encounter the real world. This hypothetical computer selected the white ball of hope all the way up to reboot 77. Then, on reboot 78, the computer selected the Blue Ball of Death, and CRASH! There went your work (unless you backed it up).

Do we reason this way? People are bad as assessing the probability and success and failure, which is why do many people gamble despite the odds. Hope springs eternal in the Garden of Eden, in Atlantic City, and in your office. But even with all the hope in the world, there comes a day when your luck runs out. It could happen on the 78^th time, on the 788^th time, or the 7888^th time. But it WILL happen. And you can either lose three hours of your life, or three years.

Your choice.

Hope springs eternal in the Garden of Eden

I was speaking with a friend yesterday about relationships. The conversation we were having was about the fact that just because your last relationship was bad and ended bitterly, how does that influence how you think about future relationships? Not all relationships end in misery, right? ("Just mine" you might be thinking...)

I suspect that the solution to this problem can be found,as always, in an unexpected place: probability theory. There is an appropriately-named statistical problem discussed by the French mathematician Pierre Simon LaPlace (1749-1827) called the “First Night in Paradise." Basically it goes like this:

It is the night after Adam and Eve’s first day in paradise. Together, they watch the sun rise and illuminate the marvelous trees, flowers, and birds. At some point, the air gets cooler, and the sun sank below the horizon. Will it stay dark forever? Adam and Eve wonder, What is the probability that the sun would rise again tomorrow?

Whether Adam and Eve were truly discussing this statistical conundrum, as opposed to addressing less cerebral concerns of a non-probabilistic nature, is besides the point. What is the probability that the sun will rise again?

The classic answer is that if Adam and Eve had never seen the sun rising, they would assign equal probabilities to both outcomes. Gigerenzer, in his book Calculated Risks, likens it to placing a white marble (the sun will rise again) and a black marble (the sun will not) into a bag, and picking one randomly. However, they did witness the sun rise once that morning when they woke up, so they place another white marble in the bag, which makes the probability of the sun rising the next day .66.

(Interestingly, as an aside, one might ask the question, "What was Adam's subjective assessment of the probability that Eve would still be there the next morning?" After all, she did simply just appear there, and who knows? God might just be teasing him and yank her out of the garden.)

As we, know, Adam and Eve woke from that first night in paradise to another sunny day in paradise. But what is the probability now, given two sunrises, that the sun would rise once again the next day? The answer is simple. They add another white marble to the bag, making the probability go up to .75. And the sun rises again and another marble goes in the bag, and so on. This is known as the rule of succession. Introduced by LaPlace, it has a simple formula:

(n+1)/(n+2)

If you are like me, and are thirty odd years old, the subjective probability that I assign to the sun rising tomorrow is quite high. Specifically it is

(365*30 + 1) / (365*30 + 2) = 10951 / 10952 = .99990869247626004. Pretty good odds. I am, in fact, relatively convinced that the sun also rises tomorrow.

But what about relationships? Does the law of succession apply here? What is the probability, given a certain number of previous relationships, that you believe that your next one is headed to splitsville? It is just like adding white balls representing sucess and black balls representing failure to a jar and selecting one.

I asked ten friends and colleagues today how many serious relationships they had been in over the course of their lives. The median value of that number was 7 – your typical friend of mine has been in 7 serious relationships (which I defined as lasting over 6 months).

What is the average person’s subjective estimate of the probability of the next relationship ending given the rule of succession? Quite simple: (7 + 1) / (7 + 2) = 8 / 9 = .888.

So is it the case that with a greater number of failed relationships, does one use the law of succession to determine whether one’s next relationship is going to fail? Maybe. This would be an interesting study to conduct – the correlation between the number of failed relationships one has had with one’s pessimism about the probability that one’s next relationship is also going to fail. Of course there are confounds up the wazoo, but it would be a fun and simple study to do on a larger scale than my ten friends.

But there is hope here. While you might suspect that your next relationship will fail with a probability of .88, that means that you also have a probability of .12 that the next one will work out. And I like that 12 percent – I might like to call it a “Hope Factor.” Sure, the hope factor diminishes after every successive failure, but by definition, this means that one can never lose hope entirely – the law of succession may asymptote very near zero, but it never quite reaches it, due to the fact that the denominator is always one greater than the numerator. Below is a figure of the relationship between the hope and despair factors by the number of failed relationships. You can see how the region of despair increases with every sucessive failed relationship, but that region of hope never quite goes away.

As they say: Hope springs eternal! And perhaps this is why it was the last evil remaining in Pandora's Box.

The possible and the probable

I spent the last few days at the National Science Foundation reviewing grants. Truth be told - you know you're nerdy when you walk into a restaurant after having forgotten to remove one's NSF temporary ID badge reading "Hello, my name is experimentaholic." and are reminded of this fact by an attractive yet not-so-nerdy restaurant hostess. Reviewing grant applications, in and of itself, in not very much fun, because I simply don't like being the one who decides who gets funded and who doesn't. I think everyone should get funded, myself included! But this is not the state of affairs - only some get funded, others get the apology letter. And I've gotten my share of those thin ones.

Which brings me to the topic of the probable and the possible. What does probability mean? For instance, these people who wrote grants submit them, and there is a certain probability that they will get funded, which is based on how much money NSF has to dish out. They also have a certain expectation of the probability that their grant will get funded, which may differ from that which is actually possible given the availability of funds.

But what does it mean to expect an event with a certain probability? Like dying on a plane crash. The probability of this event is quite low - something on the order of one in eight million. So we get on the plane, and most of the time, get off the plane. But when you get on that one plane that explodes, and you ARE that one in eight million, the probability is not quite 1 but certainly close to it. But what does uncertainty mean, at least at the level of a psychological construct?

I know that there is a lot of evidence on this - and most of it points to the fact that people are particularly bad at assessing probabilities and changing their behaviors based upon these assessments. People drove after 9/11 out of fear of flying, and this increase of traffic and the resulting accidents that ensued killed more people than had died on the planes of 9/11. True! I buy a lottery ticket when the powerball is in the hundreds of millions, realizing full well that the probability of actually winning is the same as correctly guessing a random a number between zero and 120,526,770.

I am at the moment in a coffee shop, it is a Friday night, and two students next to me are discussing Spinoza, and specifically free will. This makes me wonder about our free will in relation to our assessment - and flawed assessments - of probabilities. I have always dreaded flying, even knowing that the probability of dying is the same probability of guessing a number correctly between zero and eight million. But what if I am just lucky (or unlucky) that day? Everyone who has ever died in a plane crash has gotten on that plane imagining that the probability of a crash was one in eight million. Which is fine, if you're a member of that elite group of eight million. But if you're not, not.

One way to think about it perhaps is actuarially. When you get on a plane, you have to think of the degree to which taking this trip will actually reduce your life. A related situation is when you buy a lottery ticket. A lottery ticket is really worth what you paid for it, plus whatever the probability of winning the ticket multiplied by the reward. Imagine the senario of the powerball. The ticket costs $1. The jackpot is 100 million But the real value of the ticket is $1 + $100,000,000 X 1/120,526,770. Which is $1.82. Of course, afterwards, any given ticket is either worth $0 or $100,000,000...but in the seconds before the balls drop, it is worth $1.82. What about one's life? The probability of dying in a plane crash is 1 in 8 million. As a thirty year old, I expect (or I should say hope) to live another 30 years at least. More would be better. 30 years comes out to 946,080,000 seconds. But by getting on that plane I am reducing my life by 946,080,000 seconds X 1/8000000 or 118.26 seconds. Of course, after the plane lands or crashes, that amount changes to 0 seconds or 946,080,000 seconds. Is it worth it? Does this way of looking at the problem solve the problem, or just create new ones?

Which brings me back to free will. Does the fact that we live in uncertain world make it such that our free will is compromised? I don't know. How does our inability to reason about probabilities affect our decisions? Probably in some domains, probably not in others. Some people are notoriously bad at making comparative judgments of probabilities and possibilities - those who drove rather than fly after 9/11, who are now in the grave or scattered to the winds because their dread risk overcame their rational capacities.

In this sense, statistical probability and mental probabilities must be related, but they are not homologies. Keyes wrote something about this, on personal probability, but I confess I haven't read it, yet. Perhaps I'll read it someday. It's both possible and probable.

I'll have some love with that microscopic $9.95 piece of salmon

A student of mine once did an experiment using this web site called "Craig's List Missed Connections." Basically, it is a site in which people post notes to the universe that say things like, "I saw you walking down the street and you glanced at me. You were wearing a red hat and a green scarf...coffee?" And so if you think you are this person you can email this individual back and hopefully begin a life of love and lust together. That, or realize you have absolutely nothing to discuss over coffee.

Recently I was reading this site (perhaps perversely to see if someone posted something about that guy in the coffee shop cursing SPSS under his breath when the p value for his most recent experiment is greater than .05.) One thing I’ve noticed in passing through these listings is how there are many posts about missed connections at Whole Foods - the local hip foodstore in my neighborhood.

But this has gotten me into wondering…what is it about Whole Foods that inspires people into longing? Is it something about the air of an organic, expensive grocery store that makes people lust after strangers? Is it that only well-to-do people who can afford well-to-do people’s clothing and well-to-do people’s makeup and well-to-do people’s hair styling can afford shopping at the place? Or is it a unique place in this world - a world in which we communicate so often over the internet that we no longer know how to approach a stranger and say hello, but have to go back to our internet world and post a little "I think I saw you looking at me, but I am not sure and I'm oh so insecure!

(Here I would like to add that such people should read the excellent paper by Clark and Hatfield (1989) in which they had experimenters go around Florida State University's campus and randomly ask strangers if they would go out on a date with them that night. Something like 60 percent said yes! Could be something unique to Floridians, though.)

I wanted first to determine if there was an effect at all. I did a quick search and found that in the past 45 days on craigslist there were none less than 37 postings of missed connections at whole foods. But right next door to whole foods is a regular old supermarket known as Superfresh. How many quick glances and subtle smiles have been exchanged there? Only 8. And this includes all the Superfreshes in the city.

I figured I would try different cities. Let’s try Ann Arbor, Michigan, which I know has a Whole Foods because I used to shop there. They also have Kroeger’s, the Ann Arbor equivalent to Superfresh. 1 hit for whole foods, none for Kroeger. I tried Chicago…5 for whole foods, 1 for Dominick’s (there are dozens of Dominicks out there)

What is the probability that this could be due to chance? I'll stick to the Philadelphia one because it has the most data. Imagine that it were purely due to chance that sightings would occur at Whole Foods and at Super Fresh. Then the expected probability would be .50 - a fifty-fifty chance. What we observe is 8/37 or a 0.21 percent sightings at Superfresh. What are the odds that this is due to chance? Luckily, we have the good old binomial test to tell us the odds. The actual probability that this could be due to chance is 0.000007687045. That's a very small odds! There is something fishy going on, and I don't just mean the salmon I ate for dinner. That I bought at Whole Foods.

I suggest you experimentaholics and statistiphiles try this in your own cities. And if I see a post for that guy talking about Chi square at the produce section, I’ll know I’ve finally been missed.