Confirmation Bias Part 2: Examples & Avoidance

Our previous post explained the concept of “confirmation bias,” which is the tendency to pursue and embrace information that matches our existing beliefs.

Here are some general examples of how confirmation bias can creep into our day-to-day thinking, and three proven ways to avoid the pitfall:

Decision-Driven Data
As previously noted, the inclination to look for supportive data can easily lead us to serious mistakes. Social scientists report that analyses of investments we favor inexorably take on a rosier look than investments we are doubtful about.

Many small choices go into collecting and crunching data and analyzing opportunity and risk and presenting results. Absent a conscientious effort to avoid confirmation bias, small choices — all valid on their own — tend
to be made to support our initial opinion. We think we are making data-driven decisions, but we are really collecting decision-driven data.

For example, author Daniel Kahneman once described a study of high-performing schools to determine if size played a role in quality of educational outcomes. The data indicated that the top quartile in educational performance contained a disproportionate number of small schools, supporting the hypothesis that small schools provided better quality education. This led to some expensive policy decisions that produced no educational benefit. It turned out that small schools are disproportionately represented in the worst performing quartile as well, due to the statistical tendency of larger populations to “regress to the mean” or basically become more “average” and thus to be under-represented in the top and bottom quartile.

First Impressions
Confirmation bias also plays an important role in the inordinate impact of first impressions. A first impression provides a very tiny and possibly serendipitous sample of a candidate’s qualities and qualifications. Yet,
people who believe this is a very intelligent candidate before the interview tend to notice more signs of high intelligence.

Here are three things we can do to protect our decision-making process from conformation bias and potential distortion:

  1. Recognize the bias and remind yourself to look for it in your decisions and analyses. Remind yourself that the authors of everything you read (including this article) are making a point that is supported by the data they present, but is not necessarily by data they do not present — and in fact may not even have seen if they did not look hard enough for contrary data. Remind yourself that the talented and well-meaning people providing you with analysis and recommendations are also subject to confirmation bias. Ask for contrary data.
  2. Ask “what else could it be?” Think creatively about alternative explanations and alternative solutions. Explore the whole feasible set, if possible.
  3. Encourage the expression of contrary views and ideas. “If you value the differences in people, the differences will produce value.” Aggressively seek out and try to understand contrarian views. For many people, the first impulse is to refute contrarian views and argue our own. But the best decisions are likely to be made by those who “seek first to understand rather than be understood.”

Confirmation Bias – Has it Happened to You?

CONFIRMATION BIAS AT WORK

It has happened to most of us. Has it happened to you?

That is, has there been a time when data supported a decision you knew to be the right one, but for some reason or reasons you did not get the outcome you expected?

Perhaps you find an exciting investment opportunity like the winners you have spotted before, but it yields mediocre or poor results. Or despite your experience and successful track record when judging candidates, a person you just “knew” would be a good fit turns out to be a bad hire.

With experience can come wisdom… but also confirmation bias.

Confirmation bias is the tendency to pursue and embrace information that matches our existing beliefs. We tend to seek out and enjoy people who write or say exactly what we think. We gravitate toward these sources not for information but for confirmation.

Researcher and writer Thomas Gilovich posits the “most likely reason for the excessive influence of confirmatory information is that it is easier to deal with cognitively.” It’s easier to think what we think!

Yet confirmation bias in business can be especially hazardous and costly to highly-experienced and successful individuals. These minds are adept at spotting patterns, learning from experience, scanning the horizon and connecting the dots. If that describes your talents, take a look at this classic puzzle nicely presented by the “The Upshot.”

If you attempted the puzzle, how did you do?

For those who opted out, in this puzzle participants are given a numerical pattern and are asked to determine the underlying rule. The pattern is quite simple, and participants can test their theories as often as they like before specifying the rule. Yet 77% of participants fail to identify the rule because as soon as they find a pattern that supports their theory they conclude it is the correct rule.

In other words, 77% of participants succumb to confirmation bias.

This is a common occurrence in business. When trying to solve problems or make decisions we overwhelmingly look for patterns that support our theories rather than looking for data that would clue us in that we have missed the mark. And with each piece of data that does not refute our theory, we become more confident in our belief.

This exercise shows how people tend to work at proving their theories right, instead of robustly testing the theories to prove them wrong. Once we have seen enough supporting evidence to confirm we are right, it is far more natural for us to fully embrace our premise or idea.

For instance, maybe we are tasked with determining why a certain work process is not being done well. Is the work done less well by inexperienced employees, or when the machine is overdue for maintenance, or when the materials have a certain characteristic?

We could test all three of these ideas with data. But our natural confirmation bias makes us far more likely to look for evidence that the idea we favor is correct than to look for ways it may be mistaken. So, we start testing the idea we think is most likely and as soon as we find enough evidence to support it, we risk diving into the solution and excluding the other possibilities; and we could very well be headed down a path of action that is sub-optimum for our organization.

In our next post we’ll take a closer look at examples of confirmation bias in the workplace and steps that can be taken to avoid it.

Conventional Wisdom & Utilization

As you are most likely aware, “utilization” is a measure of the actual number of units produced divided by the number possible when machines and people work at full capacity.

Conventional wisdom says that the best way to maximize profits is to encourage every department within an organization to achieve 100% utilization. Like so much of conventional wisdom, this has a ring of truth to it; and it has the added beauty of simplicity. We can evaluate and reward each department independently of one another, and if everyone is given incentives to get as close as possible to 100% utilization, then the company will surely be maximally profitable.

But this premise will fail us in the real world… a world riddled with variation.

For example, let’s say a company has three operations:
• Glass Blowing
• Filament Insertion
• Cap & Wrap

Utilization of the 3 departments is 50% in Glass Blowing, 100% in Filament Insertion, and 80% in Cap & Wrap. So where do you focus your improvement efforts? The natural conclusion is that you would focus on increasing utilization in Glass Blowing: either by increasing production (which would simply increase the inventory of bulbs waiting for insertion) or by decreasing capacity.

But if you look at the throughput of the process as a whole, you see that Filament Insertion is the bottleneck. At 100% utilization, they are unable to produce enough to keep the next operation, Cap & Wrap, fully utilized. Furthermore, Glass Blowing, despite the lousy utilization numbers, is already piling up inventories of bulbs waiting for filaments. The utilization numbers suggest that Filament Insertion is the last area needing improvement, but to improve the process flow, it must be the first area to improve.

If the world were perfectly predictable, we could reduce the capacity in Glass Blowing and Cap & Wrap to exactly match Filament Insertion to achieve 100% utilization. But if we did so in ‘Murphy’s world,’ any variation in glass blowing production — such as machine downtime, absenteeism, yield deterioration, material availability or quality issues — will not only impact Glass Blowing utilization numbers, but the bottleneck — Filament Insertion —will also be idle! Production opportunity lost at the bottleneck is lost forever. Instead of trying to optimize individual operations, identify the bottleneck and make sure there is enough capacity in the feeder operations to ensure that any disruptions do not impact the utilization of the bottleneck capacity. Instead of aiming to maximize utilization at each operation, as conventional wisdom would have us do, we must find and eliminate waste at the ‘bottleneck’ or ‘rate-limiting’ step in order to increase profitability.

Is Bigger Better?

Is Bigger Better?

Conventional wisdom holds that bigger is better, and there are numerous factors that support this perspective. Consider that when you’re an organization or a business and you’re doing better, then you’re going to get bigger — so, therefore, bigger is better!

And certainly issues such as bulk buying discounts or other economies of scale favor size.

But, the tumultuous forces of an uncertain world might just favor agility. Consider that while a cost/benefit analysis will typically reveal that unit cost is much lower for a single large-capacity piece of equipment than for several smaller machines, rarely does that analysis incorporate the impact of uncertainty and variation on total cost; and, in reality, variation in either market demand or supply (such as machine downtime), or the relative inflexibility of a single large capacity machine can drive inefficiencies that greatly offset the lower unit cost that was calculated when a purchasing decision was made.

For example, a commercial bakery could purchase one large capacity mixer that could produce 100,000 loaves for far less cost per loaf than two smaller mixers. The large mixer produces large batch sizes; that’s how it gets its great efficiencies. But if the market is looking for variety, none of which is ordered in bulk, the large mixer results in the worst of both worlds: you either produce large batch sizes and have a lot of scrap if the demand does not materialize in time, or you waste the purchased capacity by preparing batch sizes more closely tied to current demand for the product variety. Either way, you can never really produce enough variety for the market, because the equipment produces only one variety at a time.

Capacity to produce must be as flexible as the market is variable and dynamic. Often this runs directly counter to economies of scale. Optimizing the machine-cost-per-unit can sub-optimize the profitability of the process as a whole.

So, ultimately, to effectively answer the “is bigger better” question, we must go well beyond conventional wisdom or what we assume to be true, and instead consider a wider range of variables. Possibly this well-known quote, sometimes attributed to Mark Twain but also to Josh Billings, sums-it-up best: “It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so.”

Does Your Organization Have a Strategic Internal Communication Plan?

Missing Link in Communication?

In a previous post we identified five ways to enhance the success of Continuous Improvement (CI) within an organization, with “communication” being one of the keys.

Consider that, even if a team applies the CI methodology to great success but no one hears about it, the goal of making CI a cultural way of doing business will not catch on.

However, facilitating consistent and open internal communication is one of the many things in life that might be simple, but not necessarily easy.

For example, Bruce Bolger, Co-Founder of the International Center for Enterprise Engagement, shared an interesting observation recently when he said, “Most organizations put far more effort into communicating with customers than with employees.”

We’ve found Mr. Bolger’s comments to be accurate. In many cases, customer communication is the higher priority, thus making it easy to put internal communications on the back burner. In other instances, the “silo” approach to operations tends to result in haphazard internal communication.

To gain the best results from its CI as well as its Engagement effort, an organization must connect these initiatives, along with internal communications, to a strategic and systematic approach.

The Pathway to Engagement

The path leading to a culture of engagement is linked with productivity, performance and job satisfaction. It follows a clear objective of engaging people around the one thing they all have in common—and the one thing that can bring about increased profitability and a sustainable competitive edge—the work.

As we all know, traditional employee engagement efforts have primarily failed to yield tangible results. They have also failed the sustainability test. As is the case with any improvement or change initiative, an ad-hoc approach involving little or no planning or structure, and lacking defined, measurable objectives, is prone to failure. This approach might be called “engagement for engagement’s sake.”

In contrast, a more focused approach of improving both the work and the workplace in a measurable way can result in high-levels of productivity, profitability and engagement!

As explained by Robin Gee, Coca-Cola’s Director of Employee Engagement, “We engage employees in aggressive efforts to eliminate waste and reinvest those savings in ways that are visible and meaningful to the employees.”

This perspective differs from traditional attempts at employee engagement in two critically-important ways:

  • A strong focus on productivity and continuous improvement as catalysts to engagement
  • A strong focus on measurement and return on investment

Of course this perspective is not necessarily new. For example, in 2012 ISO 10018 was introduced, which provides guidance on engaging people in an organization’s quality management system, and on enhancing their involvement and competence within it. The standard is applicable to any organization, regardless of size, type, or activity.

You might also note that ISO 10018 standards provide considerable leeway on how an organization specifically goes about its attainment. The emphasis placed on each requirement depends on an organization’s specific brand, culture, people, situation and goals. If you’d like to determine how close your organization is to achieving ISO 10018 certification, Engagement Strategies Media has created a chart that outlines the pathway. You can access the chart here.

5 Ways to Enhance CI Success

Our previous post summarized three of the most common reasons why CI efforts fail. Today’s focus is on how to avoid those pitfalls and increase the likelihood of success.

Generally speaking, in order to ensure on-going success, an organization must make sure that its measurement systems, rewards, recognition, and communication systems support CI. But more than that, one must make sure that management behavior itself supports CI.

Our Partners in Improvement groups identified the following five best practices for making an enterprise-level CI effort more successful:

  1. Top Management Support: Senior-level leadership must visibly support CI efforts. It’s best if management meets with the teams and individuals regularly for the specific purpose of seeing how the improvement project is going and what can he or she do to support the effort and speed progress.
  2. Team Training: During our Partners discussions it was agreed that nearly everyone in the company needs some basic training. But team leaders need to be very well trained, so that they can ensure that the team follows the methodology, asks the right questions, gathers the right data, and stays on trac. It was also noted that team leaders should be very carefully chosen.
  3. Diligent Upfront Work: Project planning, even before the launch, as critical to success. This involves defining the right charter, problem statement, scope, time frame, and team.
  4. Once an enterprise-level CI plan is launched, the first principle is that nothing succeeds like success. Starting out with carefully selected projects staffed with highly qualified people is a good way to promote that success. Giving the earlier projects careful guidance and support (as referenced in bullet #1 above) is another best practice that increases the likelihood of some early wins. Making “speed to success” a priority should also be part of the plan.
  5. Communication is the next most important thing. If a team applies the CI methodology to great success but no one hears about it, the goal of making CI a cultural way of doing business will not catch on. In other words, “advertising” is important! Intranet, newsletters, presentations, story boards, discussions at staff meetings and formal recognition programs are all ways to communicate success and make sure that everyone learns from successful experiences.

3 Reasons Continuous Improvement Efforts Fail

Why Projects Fail…

During one of our Partners In Improvement forums it was noted that in approximately 80% of the cases organizations embark on a path of Continuous Improvement, they abandon the effort prematurely.

The reason? No results.

The Partners went on to the discuss “why” so many CI efforts fail to succeed, and agreed that the following three causes are among the most common:

  1. Lack of buy-in from both managers and participants derails many improvement efforts. Management support is required to free up the resources to work on improvement, without which meetings tend to get pushed out and progress slows. The slower the effort moves, the more likely it becomes that priorities will change, or new opportunities or problems arise that decrease available resources further. When projects fail to produce good results, buy-in deteriorates rapidly. Unless serious intervention counters this adverse reinforcing loop, subsequent efforts become less and less likely to succeed.
  2. Lack of data when defining a project is another common reason for failure. Without data the waste is not adequately quantified, thus increasing the likelihood of working on the wrong things and the likelihood that priorities will shift before the project is complete — leading to no results and subsequent lack of buy-in.
  3. Along similar lines, poor decisions about scope can cause stalls and frustration during implementation and can ultimately result in failure to achieve goals. If the project tackles too much at once, progress will be slow; and if the team substitutes opinions for facts/data about the problem and possible solutions in an effort to accelerate pace, they are likely to make a number of wrong turns — once again slowing progress and bringing the effort to an unsuccessful conclusion.

Fortunately there are some straightforward ways to avoid these three common pitfalls, which we will summarize in our next post.

All About Run Charts

Run Charts are simple line graphs of data plotted over time. They are used to better-understand the performance of a process, as they help people distinguish between random variation and special causes, or to track information and predict trends or patterns.

A run chart can also reveal whether a process is stable by looking for a consistent central tendency, variation and randomness of pattern.

One of the most common CI tools, a run chart is easy to interpret and does not require tedious calculations or special software to produce.

Sample Run Chart

How to create a run chart:

    1. Identify the question that the run chart will answer and obtain data that will answer the question over a specified period of time. For example, if you were looking at how long it takes to complete a task, you will make note of the time taken (in minutes) to complete it over a specified period of time.
    2. Gather data, generally collect at least 10 data points to detect meaningful patterns.
    3. Create a graph with vertical line (y axis) and a horizontal line (x axis).
    4. On the vertical line (y axis), draw the scale related to the variable you are measuring. In our example, this would include the complete range of observations measuring time-to-completion
    5. On the horizontal line (x axis), draw the time or sequence scale.
    6. Plot the data, calculate the median and include into the graph.
    7. Interpret the chart. Four simple rules can be used to distinguish between random and non-random variations:
      1. Shift – 6 consecutive points above or below the median
      2. Trend – 5+ consecutive points going up or down
      3. Too many/too few runs – too few or too many crossings of the median line
      4. Astronomical data point – a data point that is clearly different from all others (often a judgement call)

All About Histograms

First introduced by Karl Pearson, an English mathematician and bio-statistician credited with establishing the discipline of mathematical statistics, a histogram is a graph figure which is used to display past data. It differs from the more well-recognized bar graph because a bar graph relates two variables, but a histogram relates only one (i.e., “earnings per month” in our example.

More specifically, histograms represent the distribution of numerical data, providing an estimate of the probability distribution of the continuous variable. Data within a histogram is displayed in “bins” and each bin has the same width. The example above uses $25 as its bin width and shows how many people earned between $800 and $825 per month, $825 and $850 per month, and so on. In other words, the “frequency” of each.

Histograms often provide new insights into the dynamics of process performance by indicating the number of times (frequency) each outcome occurred. Note that the mode of this frequency distribution is between $900 and $925, which occurs some 150 times.

To make a histogram, follow the following simple steps:\

  1. On the vertical axis, place frequencies. Label this axis “Frequency” covering the total span of gathered data points. In the example above, the span ranges from 0 to 200.
  2. On the horizontal axis, place the lower value of each interval measured. In the example above the first “bin” represents earnings between $800 and $825 per month, followed by a bin representing earnings between $825 and $850 per month, and so on.
  3. Draw a bar extending from the lower value of each interval to the lower value of the next interval on the horizontal axis, and reaching up to the associated frequency measurement.

Challenges and best practices associated with continuous improvement