Managing Overconfidence
Russo & Schoemaker
A test of confidence
How should managers deal with the often unreliable opinions they receive? The answer lies in
recognizing that most people’s beliefs are distorted by deep-seated overconfidence. Once we
understand its nature and causes, we can better devise plans for controlling it. The first step is to
document and measure the problem’s severity.
Meta knowledge
Meta knowledge, an appreciation of what we do know and what we do not know. Normally we
define knowledge as consisting of all the facts, concepts, relationships, theories, and so on that
we have accumulated over time. Meta knowledge concerns a higher level of expertise:
understanding the nature, scope, and limits of our basic, or primary knowledge. Meta knowledge
includes the uncertainty of our estimates and predictions, and the ambiguity inherent in our press
and world views.
We draw on our meta knowledge when we conclude that we have enough information and are
ready to make a decision now. If we think we are ready to decide when we are not, we may make
costly mistakes. Only when we appreciate the limits of our primary knowledge can we sensibly
ask for more or better information.
Few people are well calibrated, that is, few people can accurately asses their uncertainty. If a
question falls outside your area of expertise, should you be excused if your confidence interval
misses it? No, whether you know a lot or a little about a subject, you are still responsible for
knowing how much you don’t know.
Job relevance tends to reduce overconfidence, would such a pattern be confirmed by a more
systematic study? Moreover, is the reduction in overconfidence only partial, or would questions
very specific to peoples jobs drastically reduce overconfidence?
Better primary knowledge is generally associated with better (though still imperfect) meta
knowledge. That is, experts know better what they don’t know, and this fact is one key to effective
solutions, as we discuss next.
Developing good meta knowledge
Once the existence of overconfidence is acknowledged, two elements are essential: feedback and
accountability. Feedback that is accurate, timely and precise tells us by how much our estimates
missed the mark. Accountability forces us to confront that feedback, recalibrate our perceptions
about primary knowledge, and temper our opinions accordingly. Experience is inevitable, learning
is not.
Overconfidence persists in spite of experience because we often fail to learn form experience in
order to learn, we need feedback about the accuracy of our opinions and doubts. We also need
the motivation to translate this information into better meta knowledge.
We believe that timely feedback accountability can gradually reduce the bias toward
overconfidence in almost all professions. Being well calibrated is a teachable, learnable skill.
No single cause or prototypical situation can be consistency connected with overconfidence.
There are three classes of causes: cognitive, physiological and motivational.
Cognitive causes of overconfidence
Availability: A major reason for overconfidence in predictions is that people have difficulty in
imagining all the ways that people have difficulty in imagining all the way that events can unfold.
The availability bias, whats out of sight is often out of mind. We fail to envision important
pathways in the complex net of future events, we become unduly confident about predictions
based on the fewer pathways we actually do consider.
,Anchoring: A tendency to anchor on one value or idea and not adjust away form it sufficiently.
The confirmation bias: When making predictions or forecasts, we often lean toward one
perspective, and the natural tendency is to seek support for our initial view rather than to look for
disconfirming evidence. Unfortunately the more complex and uncertain a decision, the easier is to
find one-sides support. Realistic confidence requires seeking disconfirming, as well as confirming,
evidence.
Hindsight: Hindsight makes us believe that the world is more predictable than it really is. What
happened often seems more likely afterward that it did beforehand, since we fail to appreciate the
full uncertainty that existed overtime. (illusion of omniscience)
Cognitive remedies to overconfidence
Accelerated feedback: In contrast to learning form experience, which tends to be slow and
expensive, good feedback will reduce overconfident cheaply. Ty to improve your thinking by
bringing to mind relevant considerations that might easily be overlooked.
Counter-argumentation: Think of rasons why your initial beliefs might be wrong, or ask others to
offer counterarguments.
Paths to trouble: If we are overconfident in predicting success because we cannot see the paths
to potential trouble, fault trees may help. A fault tree is a hierarchical diagram designed to help
identify all the paths to some specific fault or problem. (The mor causes generated, the smaller is
the error of assuming that all relevant causes are already listed)
Paths to future: If deeper thinking is called for, beyond the listening reasons, explicit scenario
analysis may be useful. Scenarios focus on their conjunction. It tells in vivid detail how the future
might unfold.
Awareness alone: Although these techniques are valuable, we happily acknowledge that, for
many managers, awareness alone may be all that is needed. Good managers often devise their
own solution to the problems of overconfidence.
General versus specific awareness
Although general awareness of a bias is invaluable, it does not guarantee that the bias will be
spotted in every instance.
Psychological causes of overconfidence
Because overconfidence is a distortion of judgement, it is often thought of as a purely mental
phenomenon. However at times it has biochemical causes. (Drugs & Euphoria feeling)
Overconfidence in groups
Group judgments can be better than individual ones, precisely because in groups people are
forced to recognize that other see the work differently than they do. This often sparks a realization
that perhaps their own views are held with unjustifiable conviction. At other times, groups may
bolster the majority opinion to even more extreme levels.
Motivational factors in overconfidence
One legitimate cause of overconfidence is our believe in our abilities. Many of these people are
distorting reality yet their optimism has motivational value.
Deciding and doing
Damage can be avoided if managers distinguish between deciding and doing. Deciding requires
realism but in implementing the decision, the motivational benefits of overconfidence frequently
outweigh its dangers. Be aware of when you are functioning as a decider and when you are a
doer, motivator or implementor. When you are deciding, be realistic, both about how much you
know and how much you don’t know. When you are implementing, indulge in overconfidence
when, and if it is valuable to your performance of that of others.
, Judgment under uncertainty: Heuristics and biases
Tversky and Kahneman
Most important decisions are based on beliefs concerning the likelihood of uncertain events,
usually expressed in statements such as i think that… chances are… it is unlikely that… People
rely on a limited number of heuristics principles by which they reduce the complex tasks of
assessing likelihoods and predicting values to simpler judgmental operations.
Three features of overestimated images:
- People are not generally aware of the rules that govern their impressions
- People cannot deliberately control their perceptual impressions
- It is possible to learn to recognize the situations in which impressions are likely to be biased,
and to deliberately make approbate corrections.
Representativeness
Representativeness heuristic, in which probabilities are evaluated by the degree to which a is
representative of b, i.e. by the degree of similarity between them. If the outcome is not
representative of the generating process, probability is judged to be low.
Insensitivity to prior probability of outcomes
One of the factors that have a major effect on probability but has no effect on representativeness
is the prior probability. If people evaluate probability by representatives, therefore, prior
probabilities will be neglected. People respond differently when given no evidence and when
given worthless evidence. When no specific evidence is given prior probabilities are properly
utilized, when worthless evidence is given prior probability are ignored.
Insensitivity to sample size
To evaluate the probability of obtaining a particular result in a sample drawn form a specified
population, people typically apply the representativeness heuristic. That is, they assess the
likelihood of a sample result by the similarity of this result to the corresponding parameter.
Misconceptions of chance
People expect that a sequence of events generated by a random process will represent the
essential characteristics of that process even when the sequence is short. A locally representative
sequence, however deviates systematically form chance expectation, it contains too many
alternations and too few runs.
Law of small numbers, according to which even small samples are highly representative of
the populations from which they are drawn. As a consequence the researches put too much faith
in the results of small sample, and grossly overestimated the replicability of such results. This bias
has pernicious consequences for the conduct of research, it leads to over interpretations of
findings and the choice of inadequate sample sizes.
Insensitivity to predictive accuracy
If people predict solely in terms of the favorableness of the description, their predictions will be
incentive to the reliability of the evidence and to the expected accuracy of the prediction. If
expelled accuracy is minimal, the same predictions should be made in all cases.
Several studies of numerical predictions have demonstrated that intuitive predictions do
not conform to this rule, and that subjects show little or no regard for considerations of expected
accuracy.
The illusion of validity
As we have seen, people often predict by selecting the outcome that is most representative of the
input. The confidence they have in their prediction depends primarily on the degree of
representativeness attained in the prediction with little or no regard for the factors that limit
predictive accuracy. People tend to have greater confidence in prediction based on redundant
input variables than in predictions based on uncorrelated variables.
Misconceptions of regressions
Russo & Schoemaker
A test of confidence
How should managers deal with the often unreliable opinions they receive? The answer lies in
recognizing that most people’s beliefs are distorted by deep-seated overconfidence. Once we
understand its nature and causes, we can better devise plans for controlling it. The first step is to
document and measure the problem’s severity.
Meta knowledge
Meta knowledge, an appreciation of what we do know and what we do not know. Normally we
define knowledge as consisting of all the facts, concepts, relationships, theories, and so on that
we have accumulated over time. Meta knowledge concerns a higher level of expertise:
understanding the nature, scope, and limits of our basic, or primary knowledge. Meta knowledge
includes the uncertainty of our estimates and predictions, and the ambiguity inherent in our press
and world views.
We draw on our meta knowledge when we conclude that we have enough information and are
ready to make a decision now. If we think we are ready to decide when we are not, we may make
costly mistakes. Only when we appreciate the limits of our primary knowledge can we sensibly
ask for more or better information.
Few people are well calibrated, that is, few people can accurately asses their uncertainty. If a
question falls outside your area of expertise, should you be excused if your confidence interval
misses it? No, whether you know a lot or a little about a subject, you are still responsible for
knowing how much you don’t know.
Job relevance tends to reduce overconfidence, would such a pattern be confirmed by a more
systematic study? Moreover, is the reduction in overconfidence only partial, or would questions
very specific to peoples jobs drastically reduce overconfidence?
Better primary knowledge is generally associated with better (though still imperfect) meta
knowledge. That is, experts know better what they don’t know, and this fact is one key to effective
solutions, as we discuss next.
Developing good meta knowledge
Once the existence of overconfidence is acknowledged, two elements are essential: feedback and
accountability. Feedback that is accurate, timely and precise tells us by how much our estimates
missed the mark. Accountability forces us to confront that feedback, recalibrate our perceptions
about primary knowledge, and temper our opinions accordingly. Experience is inevitable, learning
is not.
Overconfidence persists in spite of experience because we often fail to learn form experience in
order to learn, we need feedback about the accuracy of our opinions and doubts. We also need
the motivation to translate this information into better meta knowledge.
We believe that timely feedback accountability can gradually reduce the bias toward
overconfidence in almost all professions. Being well calibrated is a teachable, learnable skill.
No single cause or prototypical situation can be consistency connected with overconfidence.
There are three classes of causes: cognitive, physiological and motivational.
Cognitive causes of overconfidence
Availability: A major reason for overconfidence in predictions is that people have difficulty in
imagining all the ways that people have difficulty in imagining all the way that events can unfold.
The availability bias, whats out of sight is often out of mind. We fail to envision important
pathways in the complex net of future events, we become unduly confident about predictions
based on the fewer pathways we actually do consider.
,Anchoring: A tendency to anchor on one value or idea and not adjust away form it sufficiently.
The confirmation bias: When making predictions or forecasts, we often lean toward one
perspective, and the natural tendency is to seek support for our initial view rather than to look for
disconfirming evidence. Unfortunately the more complex and uncertain a decision, the easier is to
find one-sides support. Realistic confidence requires seeking disconfirming, as well as confirming,
evidence.
Hindsight: Hindsight makes us believe that the world is more predictable than it really is. What
happened often seems more likely afterward that it did beforehand, since we fail to appreciate the
full uncertainty that existed overtime. (illusion of omniscience)
Cognitive remedies to overconfidence
Accelerated feedback: In contrast to learning form experience, which tends to be slow and
expensive, good feedback will reduce overconfident cheaply. Ty to improve your thinking by
bringing to mind relevant considerations that might easily be overlooked.
Counter-argumentation: Think of rasons why your initial beliefs might be wrong, or ask others to
offer counterarguments.
Paths to trouble: If we are overconfident in predicting success because we cannot see the paths
to potential trouble, fault trees may help. A fault tree is a hierarchical diagram designed to help
identify all the paths to some specific fault or problem. (The mor causes generated, the smaller is
the error of assuming that all relevant causes are already listed)
Paths to future: If deeper thinking is called for, beyond the listening reasons, explicit scenario
analysis may be useful. Scenarios focus on their conjunction. It tells in vivid detail how the future
might unfold.
Awareness alone: Although these techniques are valuable, we happily acknowledge that, for
many managers, awareness alone may be all that is needed. Good managers often devise their
own solution to the problems of overconfidence.
General versus specific awareness
Although general awareness of a bias is invaluable, it does not guarantee that the bias will be
spotted in every instance.
Psychological causes of overconfidence
Because overconfidence is a distortion of judgement, it is often thought of as a purely mental
phenomenon. However at times it has biochemical causes. (Drugs & Euphoria feeling)
Overconfidence in groups
Group judgments can be better than individual ones, precisely because in groups people are
forced to recognize that other see the work differently than they do. This often sparks a realization
that perhaps their own views are held with unjustifiable conviction. At other times, groups may
bolster the majority opinion to even more extreme levels.
Motivational factors in overconfidence
One legitimate cause of overconfidence is our believe in our abilities. Many of these people are
distorting reality yet their optimism has motivational value.
Deciding and doing
Damage can be avoided if managers distinguish between deciding and doing. Deciding requires
realism but in implementing the decision, the motivational benefits of overconfidence frequently
outweigh its dangers. Be aware of when you are functioning as a decider and when you are a
doer, motivator or implementor. When you are deciding, be realistic, both about how much you
know and how much you don’t know. When you are implementing, indulge in overconfidence
when, and if it is valuable to your performance of that of others.
, Judgment under uncertainty: Heuristics and biases
Tversky and Kahneman
Most important decisions are based on beliefs concerning the likelihood of uncertain events,
usually expressed in statements such as i think that… chances are… it is unlikely that… People
rely on a limited number of heuristics principles by which they reduce the complex tasks of
assessing likelihoods and predicting values to simpler judgmental operations.
Three features of overestimated images:
- People are not generally aware of the rules that govern their impressions
- People cannot deliberately control their perceptual impressions
- It is possible to learn to recognize the situations in which impressions are likely to be biased,
and to deliberately make approbate corrections.
Representativeness
Representativeness heuristic, in which probabilities are evaluated by the degree to which a is
representative of b, i.e. by the degree of similarity between them. If the outcome is not
representative of the generating process, probability is judged to be low.
Insensitivity to prior probability of outcomes
One of the factors that have a major effect on probability but has no effect on representativeness
is the prior probability. If people evaluate probability by representatives, therefore, prior
probabilities will be neglected. People respond differently when given no evidence and when
given worthless evidence. When no specific evidence is given prior probabilities are properly
utilized, when worthless evidence is given prior probability are ignored.
Insensitivity to sample size
To evaluate the probability of obtaining a particular result in a sample drawn form a specified
population, people typically apply the representativeness heuristic. That is, they assess the
likelihood of a sample result by the similarity of this result to the corresponding parameter.
Misconceptions of chance
People expect that a sequence of events generated by a random process will represent the
essential characteristics of that process even when the sequence is short. A locally representative
sequence, however deviates systematically form chance expectation, it contains too many
alternations and too few runs.
Law of small numbers, according to which even small samples are highly representative of
the populations from which they are drawn. As a consequence the researches put too much faith
in the results of small sample, and grossly overestimated the replicability of such results. This bias
has pernicious consequences for the conduct of research, it leads to over interpretations of
findings and the choice of inadequate sample sizes.
Insensitivity to predictive accuracy
If people predict solely in terms of the favorableness of the description, their predictions will be
incentive to the reliability of the evidence and to the expected accuracy of the prediction. If
expelled accuracy is minimal, the same predictions should be made in all cases.
Several studies of numerical predictions have demonstrated that intuitive predictions do
not conform to this rule, and that subjects show little or no regard for considerations of expected
accuracy.
The illusion of validity
As we have seen, people often predict by selecting the outcome that is most representative of the
input. The confidence they have in their prediction depends primarily on the degree of
representativeness attained in the prediction with little or no regard for the factors that limit
predictive accuracy. People tend to have greater confidence in prediction based on redundant
input variables than in predictions based on uncorrelated variables.
Misconceptions of regressions
