Finding the missing piece

Summer is time for love, and I offer you an analysis of the probability of meeting your soul mate. In his Symposium, Plato writes that people were originally both male and female. To prevent them from acquiring the power of the Gods, Zeus cut them in half. Life then became for humans a search for the other half that was lost, in order for them to once again attempt to ascend Mount Olympus.

It is a pretty story, and one that resonates with many a New Age follower’s conception of love. Even if it doesn’t quite jive with reality. But why not?

The reason once again has to do with probability. There is a well known problem in the statistics literature called the Birthday Paradox. Basically, the paradox questions, “How many people do you need to put into a room to get a 50% chance that two of the people have the same birthday?” The answer to this is 23. Therein lies the paradox – this seems like a very small number. Most people intuitively think that one needs to have 365/2 or 183 people in the room. But that intuition is wrong. The true formula is as follows:

N = .5 + SQRT((.25 + 2 * 365 * ln(2))

I would like to generalize this problem to the question of finding one’s soul mate. Specifically, the question I address is how many people would you have to include in a room for there to be a 50% chance that any two people in the room were once joined at the hip before Zeus took his shears and sliced them apart? Now, this assumes that the people in the room were randomly selected from the entire world population, which is approximately six billion people. The answer to this is simple, and follows the formula above:

N = .5 + SQRT((.25 + 2 * 6,000,000,000 * ln(2)) = 91,202

Wow! It would take a room of 91,202 people for there to be a 50% chance that any two people in that room were actually soul mates. One would have to have a pretty big room, perhaps as large as this room, which fits over 100,000 people.

Imagine a speed dating service in which you have five minutes to talk with every other person in this room. For those two lucky halves to actually have a chance to meet, the speed dating round would have to last for 91,202*5 minutes, which comes out to 316.67 days. All for only two of those 91,202 people to meet. This says nothing of the 91,200 people who leave Michigan Stadium brokenhearted or satisficing with someone who is not their other half.

But that doesn’t help you, dear reader, who doesn’t care about any two people encountering their soul mate, but rather about the probability that YOU will meet your soul mate. Well, the problem is that to calculate this, one would require a supercomputer that can handle a hell of a lot more decimal places than my humble Dell laptop. Needless to say, you would need a room about as big as this place to have a 50% chance of meeting your other half.

What does this tell you? Well, quite simply, you have about as much of a chance of meeting your other half as you do winning the powerball lottery. Either that, or the search for love isn’t a search for that one soul mate that Zeus cut from your side, but that any of a number of possible other halves fit approximately well enough that they can serve as soul mates. I tend to believe this is the case, because I know many a person who has claimed to have met their soul mate, and I know the probability of this actually occurring has about a snowball’s chance in hell. And whether or not I believe in soul mates matters very little. I do believe in probability!

But I tend to be hopeful. And I am a romantic who enjoys Greek mythology. I know my other half is wandering around out there. Of this I am certain beyond any statistical doubt. Whoever she is, I know she would appreciate my calculation of the infinitesimally small likelihood that we would ever meet.

Experimentaholic discovers a new force in the universe!

Leg day. There is nothing worse than leg day.

So ever since my semester ended, I have been spending more time at my gym. I generally don’t have much time for working out during the semester, but summer means time to try and avoid the ravages of time by exercise. Once you turn 25 apparently you lose a pound of muscle and gain a pound of fat per year. Lord knows I’m fat enough already to know that I don’t need another pound of fat come January, so there I go. But the other day I was really annoyed about going, since it was leg day.

Leg day is the day everyone hates, the day when you throw several hundred pounds on the barbell and do squats, or hold dumb bells and do lunges. Arm day is easy. Leg day just blows. But one thing I have noticed recently is that most men skip the leg exercises. They have skinny legs but massive chests and arms, which is obviously because they only do chest and arm exercises. But women don’t seem to skip those leg exercises – there are generally many women doing the ass and thigh machines. With one exception.

Over a month ago, when I started to go to the gym more regularly, I ran into this woman named Kelly. Kelly is stunningly beautiful. Tall, dark hair, gorgeous eyes, she could be mistaken for an actress. (She is actually a fashion model and aspiring actress…I know this only because we once got into a conversation because we were both wearing the same university tee shirt.) Kelly is one of those women who does her leg exercises quite regularly. And suddenly one day I noticed something rather perplexing about the nature of the universe. First, I noticed Kelly was doing her leg exercises. But then I noticed that there were several men, also doing leg exercises. I thought, “Wait a minute – that section is usually empty!” The next day…same thing. The following day, yet again.

Then, I had an epiphany. Maybe there is a new physical force in the universe, akin to gravity, that emanates solely from Kelly. Let’s call it the Kelly Field. It is a physical force that somehow attracts men to go to the leg section of the gym and work away those chicken-legs. I felt like Newton, but instead of an apple that struck me, it was a dumb bell.

I began to wonder how I could test whether Kelly was the source of a new force field that should be further investigated by more empirical studies, or whether this effect was merely due to chance or coincidence. So I designed a little experiment to test the hypothetical existence of the Kelly Field. Whenever I saw Kelly doing leg exercises, I counted the number of men and the number of women who would also be doing leg exercises. But after about two seconds of thinking, I realized this is not a very good experimental design: One needs a control condition. Maybe it is the case that when anyone – not just Kelly - goes into the leg section, that it attracts men to that section of the gym. So I had to have an alternative hypothesis…the existence of a Experimentaholic Field. If it were the case that anyone going to the leg section would attract men, then when I go to the leg section I would find just as many men working their legs as when Kelly was working out in that section. So I simply began to count the number of men in the leg section when Kelly was working out there, and the number of men when I was working out there. I eliminated a possible confound by never doing leg exercises when Kelly was doing leg exercises. Unlike other men, I was somehow unaffected by this force field, or at least could resist it at times.

At the end of three weeks, I had tallied the following totals over the course of 10 observations of Kelly and Experimentaholic. Below is a chart with the actual counts

Kelly: Men = 20, Women = 14

Experimentaholic: Men = 3, Women = 18

As can be seen in the contingency table above, there were far more men doing leg exercises in the presence of Kelly than in the presence of Sean. But this could be due to chance, right? How can one tell?

Well, the British statistician Karl Pearson saved the day when he came up with the formula for the Chi-Square test. Chi square allows you to determine whether there are associations among binary variables are likely to be due to chance. I ran the test, and found that the Chi Square test showed that the probability of such a large discrepancy in the number of men versus women working on their legs in the presence of Kelly versus Sean being due to chance is extremely low…like one in ten thousand. So the effect is real: there actually is a Kelly Field.

I wonder about the mechanism behind the Kelly Field. Does it work like gravity, and that its effect is a function of the squared distance between Kelly and men? Or is it something more like magnetism? Will we someday be able to throw out the theory of the Kelly Field and replace it with a Quantum Kelly Relativity Theory? Or a String Theory of Kelly? Only further investigations into the Kelly Field will provide us with these answers.

Currently I am working on my manuscript for the journal Science about this.

Actually, I’ll get to it once I get back from the gym.

Hope springs eternal in the Garden of Eden

I was speaking with a friend yesterday about relationships. The conversation we were having was about the fact that just because your last relationship was bad and ended bitterly, how does that influence how you think about future relationships? Not all relationships end in misery, right? ("Just mine" you might be thinking...)

I suspect that the solution to this problem can be found,as always, in an unexpected place: probability theory. There is an appropriately-named statistical problem discussed by the French mathematician Pierre Simon LaPlace (1749-1827) called the “First Night in Paradise." Basically it goes like this:

It is the night after Adam and Eve’s first day in paradise. Together, they watch the sun rise and illuminate the marvelous trees, flowers, and birds. At some point, the air gets cooler, and the sun sank below the horizon. Will it stay dark forever? Adam and Eve wonder, What is the probability that the sun would rise again tomorrow?

Whether Adam and Eve were truly discussing this statistical conundrum, as opposed to addressing less cerebral concerns of a non-probabilistic nature, is besides the point. What is the probability that the sun will rise again?

The classic answer is that if Adam and Eve had never seen the sun rising, they would assign equal probabilities to both outcomes. Gigerenzer, in his book Calculated Risks, likens it to placing a white marble (the sun will rise again) and a black marble (the sun will not) into a bag, and picking one randomly. However, they did witness the sun rise once that morning when they woke up, so they place another white marble in the bag, which makes the probability of the sun rising the next day .66.

(Interestingly, as an aside, one might ask the question, "What was Adam's subjective assessment of the probability that Eve would still be there the next morning?" After all, she did simply just appear there, and who knows? God might just be teasing him and yank her out of the garden.)

As we, know, Adam and Eve woke from that first night in paradise to another sunny day in paradise. But what is the probability now, given two sunrises, that the sun would rise once again the next day? The answer is simple. They add another white marble to the bag, making the probability go up to .75. And the sun rises again and another marble goes in the bag, and so on. This is known as the rule of succession. Introduced by LaPlace, it has a simple formula:

(n+1)/(n+2)

If you are like me, and are thirty odd years old, the subjective probability that I assign to the sun rising tomorrow is quite high. Specifically it is

(365*30 + 1) / (365*30 + 2) = 10951 / 10952 = .99990869247626004. Pretty good odds. I am, in fact, relatively convinced that the sun also rises tomorrow.

But what about relationships? Does the law of succession apply here? What is the probability, given a certain number of previous relationships, that you believe that your next one is headed to splitsville? It is just like adding white balls representing sucess and black balls representing failure to a jar and selecting one.

I asked ten friends and colleagues today how many serious relationships they had been in over the course of their lives. The median value of that number was 7 – your typical friend of mine has been in 7 serious relationships (which I defined as lasting over 6 months).

What is the average person’s subjective estimate of the probability of the next relationship ending given the rule of succession? Quite simple: (7 + 1) / (7 + 2) = 8 / 9 = .888.

So is it the case that with a greater number of failed relationships, does one use the law of succession to determine whether one’s next relationship is going to fail? Maybe. This would be an interesting study to conduct – the correlation between the number of failed relationships one has had with one’s pessimism about the probability that one’s next relationship is also going to fail. Of course there are confounds up the wazoo, but it would be a fun and simple study to do on a larger scale than my ten friends.

But there is hope here. While you might suspect that your next relationship will fail with a probability of .88, that means that you also have a probability of .12 that the next one will work out. And I like that 12 percent – I might like to call it a “Hope Factor.” Sure, the hope factor diminishes after every successive failure, but by definition, this means that one can never lose hope entirely – the law of succession may asymptote very near zero, but it never quite reaches it, due to the fact that the denominator is always one greater than the numerator. Below is a figure of the relationship between the hope and despair factors by the number of failed relationships. You can see how the region of despair increases with every sucessive failed relationship, but that region of hope never quite goes away.

As they say: Hope springs eternal! And perhaps this is why it was the last evil remaining in Pandora's Box.

I'll have some love with that microscopic $9.95 piece of salmon

A student of mine once did an experiment using this web site called "Craig's List Missed Connections." Basically, it is a site in which people post notes to the universe that say things like, "I saw you walking down the street and you glanced at me. You were wearing a red hat and a green scarf...coffee?" And so if you think you are this person you can email this individual back and hopefully begin a life of love and lust together. That, or realize you have absolutely nothing to discuss over coffee.

Recently I was reading this site (perhaps perversely to see if someone posted something about that guy in the coffee shop cursing SPSS under his breath when the p value for his most recent experiment is greater than .05.) One thing I’ve noticed in passing through these listings is how there are many posts about missed connections at Whole Foods - the local hip foodstore in my neighborhood.

But this has gotten me into wondering…what is it about Whole Foods that inspires people into longing? Is it something about the air of an organic, expensive grocery store that makes people lust after strangers? Is it that only well-to-do people who can afford well-to-do people’s clothing and well-to-do people’s makeup and well-to-do people’s hair styling can afford shopping at the place? Or is it a unique place in this world - a world in which we communicate so often over the internet that we no longer know how to approach a stranger and say hello, but have to go back to our internet world and post a little "I think I saw you looking at me, but I am not sure and I'm oh so insecure!

(Here I would like to add that such people should read the excellent paper by Clark and Hatfield (1989) in which they had experimenters go around Florida State University's campus and randomly ask strangers if they would go out on a date with them that night. Something like 60 percent said yes! Could be something unique to Floridians, though.)

I wanted first to determine if there was an effect at all. I did a quick search and found that in the past 45 days on craigslist there were none less than 37 postings of missed connections at whole foods. But right next door to whole foods is a regular old supermarket known as Superfresh. How many quick glances and subtle smiles have been exchanged there? Only 8. And this includes all the Superfreshes in the city.

I figured I would try different cities. Let’s try Ann Arbor, Michigan, which I know has a Whole Foods because I used to shop there. They also have Kroeger’s, the Ann Arbor equivalent to Superfresh. 1 hit for whole foods, none for Kroeger. I tried Chicago…5 for whole foods, 1 for Dominick’s (there are dozens of Dominicks out there)

What is the probability that this could be due to chance? I'll stick to the Philadelphia one because it has the most data. Imagine that it were purely due to chance that sightings would occur at Whole Foods and at Super Fresh. Then the expected probability would be .50 - a fifty-fifty chance. What we observe is 8/37 or a 0.21 percent sightings at Superfresh. What are the odds that this is due to chance? Luckily, we have the good old binomial test to tell us the odds. The actual probability that this could be due to chance is 0.000007687045. That's a very small odds! There is something fishy going on, and I don't just mean the salmon I ate for dinner. That I bought at Whole Foods.

I suggest you experimentaholics and statistiphiles try this in your own cities. And if I see a post for that guy talking about Chi square at the produce section, I’ll know I’ve finally been missed.