Confirmation Bias Part 2: Examples & Avoidance

Our previous post explained the concept of “confirmation bias,” which is the tendency to pursue and embrace information that matches our existing beliefs.

Here are some general examples of how confirmation bias can creep into our day-to-day thinking, and three proven ways to avoid the pitfall:

Decision-Driven Data
As previously noted, the inclination to look for supportive data can easily lead us to serious mistakes. Social scientists report that analyses of investments we favor inexorably take on a rosier look than investments we are doubtful about.

Many small choices go into collecting and crunching data and analyzing opportunity and risk and presenting results. Absent a conscientious effort to avoid confirmation bias, small choices — all valid on their own — tend
to be made to support our initial opinion. We think we are making data-driven decisions, but we are really collecting decision-driven data.

For example, author Daniel Kahneman once described a study of high-performing schools to determine if size played a role in quality of educational outcomes. The data indicated that the top quartile in educational performance contained a disproportionate number of small schools, supporting the hypothesis that small schools provided better quality education. This led to some expensive policy decisions that produced no educational benefit. It turned out that small schools are disproportionately represented in the worst performing quartile as well, due to the statistical tendency of larger populations to “regress to the mean” or basically become more “average” and thus to be under-represented in the top and bottom quartile.

First Impressions
Confirmation bias also plays an important role in the inordinate impact of first impressions. A first impression provides a very tiny and possibly serendipitous sample of a candidate’s qualities and qualifications. Yet,
people who believe this is a very intelligent candidate before the interview tend to notice more signs of high intelligence.

Here are three things we can do to protect our decision-making process from conformation bias and potential distortion:

  1. Recognize the bias and remind yourself to look for it in your decisions and analyses. Remind yourself that the authors of everything you read (including this article) are making a point that is supported by the data they present, but is not necessarily by data they do not present — and in fact may not even have seen if they did not look hard enough for contrary data. Remind yourself that the talented and well-meaning people providing you with analysis and recommendations are also subject to confirmation bias. Ask for contrary data.
  2. Ask “what else could it be?” Think creatively about alternative explanations and alternative solutions. Explore the whole feasible set, if possible.
  3. Encourage the expression of contrary views and ideas. “If you value the differences in people, the differences will produce value.” Aggressively seek out and try to understand contrarian views. For many people, the first impulse is to refute contrarian views and argue our own. But the best decisions are likely to be made by those who “seek first to understand rather than be understood.”