What should you conclude when you host a new event, try a new tactic, or field test a new product and the initial results are extremely good? Is this evidence of a paradigmatic shift of such proportions that the established way of doing things will forever change?
But even if it does, the “off the charts” nature of the event that was experienced may skew your ability to see what is coming next: a regression to the mean.
The concept is fairly straightforward and simply states that extreme results will tend—on average—to be followed by (“regression”) less extreme results (“the mean”). Imagine the launch of a new product, into which the company has sunk a lot of money. Great care is taken in all aspects of design and development. On launch, the initial results are simply mind-blowing. Never before in the history of the company (or industry) have there ever been results like these!
All the excitement generated by a wildly successful outcome is both understandable and potentially dangerous, if it is not recognized for what it is. The exceptional result may well indicate a new opportunity and “the wave of the future” for the company. But good leaders will not be distracted by the emotion of the highly desirable outcome or confused into thinking that all or most outcomes will be similarly wonderful.
The next one will be more average.
This is a statement that is true over time, simply due to the inherent statistical probabilities of the possible outcomes. So it is important to avoid attaching “a causal interpretation to the inevitable fluctuations of a random process” (Kahneman, 2013). Your brilliant strategy, amazing engineering, and incredible marketing may have delivered a stunning result, but even so, you should expect that you merely achieved an outcome on the right of the normal distribution—not necessarily a new normal (see the chart below).
The excitement of a highly favorable outcome early on may predispose you to overestimate your prediction of what this means for you (or your company, the industry, etc.) As Kahneman explains:
…it is natural for System 1 [the emotionally-driven part of our minds] to generate overconfident judgments, because confidence, as we have seen, is determined by the coherence of the best story you can tell from the evidence at hand. Be warned: your intuitions will deliver predictions that are too extreme and you will be inclined to put far too much faith in them.
The solution? Larger sample sizes and brutal assessment of reality are good places to start.
For more on this:
- Kahneman, D. (2013). Thinking, Fast and Slow. New York: Farrar, Straus and Giroux.