When Sally Clark’s 11-week-old baby died of mysterious causes, doctors attributed the death to sudden infant death syndrome, or SIDS. Sally gave birth to a second child and that child unbelievably also died as an infant. First responders couldn’t help but notice the odd coincidence. What are the chances that the same mother would lose two children to SIDS? They suspected foul play and charged Sally with murdering both children by suffocation.
An expert witness at her trial said the chances of two children in the same family dying of SIDS was 1 in 73 million. That was enough for the jury to reject the SIDS hypothesis. Sally went to jail.
The human mind, it turns out, is particularly lame when it comes to conditional probabilities. A conditional probability is the probability of one thing happening given another thing has already happened.
We know the chances of a family with two kids having one boy and one girl is about 50%. There are four possible outcomes, two boys, two girls, and two ways to get to one of each: a boy first then a girl and a girl first then a boy. Since each of the four outcomes have roughly the same probability of happening and two of the four outcomes have one girl and one boy, we know that about 50% of two-child families have one of each.
Then, we find out that a two-child family has at least one girl. What does that do to the probabilities? That is when the question turns into a conditional probability: what are the chances that a two-child family has one boy and one girl, given that one of the children is a girl? Our intuition tells us about 50%; if one child is a girl the other child has a 50/50 chance of being a boy.
Our intuition, in this case, is wrong. Going back to our four outcomes, the new information allows us to eliminate one of the four outcomes: we know the family does not have two boys. Each of the other three outcomes is possible. Since two of the three outcomes have a boy and a girl, given the new information we know the chances of one boy and one girl increase to about 67%. If we were trying to identify one-boy-and-one-girl families, our hit rate would increase from 50% to 67% if we targeted families with at least one girl.
The key piece of data that our intuition glosses over with conditional probabilities is what statisticians call the reference class. The reference class is the total population that you are plucking your sample from. In the two-child problem, the reference class before we find out about the family having one girl is all two-child families. All four outcomes are possible in that reference class. Once we find out about the one girl, the reference class shrinks. Only three out of the four possible outcomes are possible. The shrinking reference class increases the probabilities from 50% to 67%.
The expert witness at Sally Clark’s trial put Sally in the wrong reference class. The one-in-seventy-three-million stat he quoted was most likely accurate, but it wasn’t relevant. That stat describes the frequency of two SIDS deaths in the same family from the population at large, which is the wrong reference class for Sally. Knowing that Sally had two infants die in her family puts her in a much smaller reference class that consists of just the families with two infant deaths.
Within that reference class, the frequency of two SIDS deaths is quite high. Within that reference class, in fact, the chances of two SIDS deaths are about five times higher than the chances of the two infants being suffocated by their mother. Based on a revised and more accurate assessment of the statistical evidence, Sally’s conviction was overturned, but only after she spent three years in jail.
Here’s another example. I test positive for cancer. The doctor tells me the test is 90% accurate. My intuition tells me I have cancer. My intuition again ignores the reference class.
Let’s assume 1,000 people (1%) in a population of 100,000 have this form of cancer. Since the test is 90% accurate, 900 (90%) of the 1,000 people with the cancer test positive (true positives) and 100 (10%) test negative (false negatives). Likewise, 89,100 (90%) of the 99,000 people that do not have the cancer would test negative (true negatives) and 9,900 would test positive (false positives).
My reference class after the positive test consists of all those people with positive tests: 900 true positives and 9,900 false positives. Of 10,800 people that tested positive, just 900 (8%) have the cancer.
Only 8% of the people that tested positive have cancer, despite a test that is 90% accurate. Our intuition tells us that my chance of having cancer is equal to the accuracy of the test, but that ignores the reference class which in this case consists of lots of false positives.
Ignoring the reference class, it turns out, plays tricks on investors as well. Investors ignore the reference class and jump to wrong conclusions all the time.
Most active investors try to be somewhat selective. They have a mix of indicators they use to select stocks. A “1-in-100” investment strategy would target one stock that meets their investment criteria out of every 100 stocks in the population at large. If their investment process is 90% accurate, some might think nine out of ten stocks they pick would be winners. Nothing could be further from the truth. Applying an investment process that is 90% accurate to a one-in-a-hundred population of investments increases the hit rate from 1% to 8%. False positives overwhelm the few winners.
We believe that breeds overconfidence. Most investors totally ignore the false positives in their reference class.
There are two ways for investors to increase the yield from their investment process: improve the process or shrink the reference class. Most investors ignore the second.
A great analogy comes from the world of fishing. A fishing guide’s skill at catching fish means nothing if that guide takes clients to a lake with no fish. A good fishing guide doesn’t take clients to just any random lake; a good guide narrows the reference class by taking his clients to lakes with high concentrations of fish. Narrowing the reference class in this way can have a bigger effect on yield than honing his fishing skills.
The same is true with investing. By shrinking our reference class to areas that we believe have high concentrations of great investments, we can boost our hit rate of finding great investments. By way of example, if the total investment universe has 1 in 100 great investments, the hit rate for a 90%-accurate investment process is 8%. But if we concentrate on an area within the investment universe with 10 great investments out of 100, the hit rate for the same 90%-accurate investment process increases to 50%. The reference class, in other words, has a huge effect on the hit rate of finding great investments.
How do we narrow the reference class? By geography, by industry, by market cap, by economic regime, by anything we can. We have a very broad investment mandate at KP7. Within that mandate, we focus our efforts on areas that we believe have high concentrations of great investments. We quite simply fish in lakes with high concentrations of fish.
