Streetlights and shadows
Searching for the keys to adapti ve decision making; Gary Klein
Adaptive decision making
Claim 1: teaching people procedures helps them perform tasks more
skillfully
Procedures are made so that workers can perform this task even without years of practice.
Procedures help us to evaluate performance and are also relevant for health care because procedures
can improve performance and safety.
Procedures that trainee pilots were being taught didn’t match how skilled pilots performed the same
tasks. By the time people become proficient, they are seeing situations instead of calculating
procedures. Experts rely on their immediate intuitive responses.
The Dreyfus model of expertise emphasizes intuition and tacit knowledge that can’t be captured in
rules and procedures. People might need some rules in order to get started but they have to move
beyond rules in order to achieve mastery.
No matter how comprehensive procedures, people will probably run into something unexpected.
One way to ensure that a set of procedures is sufficient, is to take every eventuality into account.
However, this strategy can result in the opposite problem: procedural guides that are over-sufficient
and sometimes incomprehensible.
Procedures are often out of date because work practices keep evolving. Because they keep evolving,
guides are rarely complete. Over time, some procedures became obsolete or even counterproductive.
The people doing the job learned workarounds, they used their experience to adapt. The procedures
lagged behind the actual way people did their work. Up-to-date procedures had to be interpreted and
carried out by workers using their judgment and experience and obsolete procedures created even
more headaches.
Procedures can lead to mindlessness and complacency When we become passive, we don’t try to
improve our skills.
Procedures can erode expertise. People taught to understand the system develop richer mental
models than people taught to follow procedures.
Procedures can mislead us. The biggest worry is that following procedures can lead us in the wrong
direction and that we won’t notice because the reliance on procedures had made us so complacent.
In summary, we can see that procedures are insufficient, can get in the way, can interfere with
developing and applying expertise and can erode over time. Procedures work best in well-ordered
situations in which we don’t have to worry about changing conditions and we don’t have to take
context into account to figure out how to apply the procedures.
Unintended consequences:
We would establish adequate performance as the new ideal. It creates an endless cycle of updating
and we would live with ineffective designs. We discourage people from using their judgment. We
could give early retirement to senior workers, their tacit knowledge wouldn’t be useful and they are
the ones least likely to refer to procedures when they perform their jobs. We wouldn’t worry about
all the nuances of context. We would generate massive volumes of procedures because it is too
expensive to go back and cull procedures we don’t need anymore. We would save money by retaining
,our current ways of doing the work. We would issue procedures as a way to change behavior even
though there may be simpler and more effective strategies.
Replacement of state 1; in complex situations, people will need judgment skills to follow procedures
effectively and to go beyond them when necessary.
Like all tools, procedures have strengths and weaknesses, this are strengths:
- Training tools, help novices get started in learning a task
- They are memory aids, help workers overcome memory slips
- They can safeguard against interruptions, checklist helps to carry out all steps
- They reduce workload, easier to attend to critical aspects
- They are a way to compile experience and historical information. Useful when there is a lot of
turnover and few workers ever develop much skill. Less-experienced workers can do a
reasonable acceptable job.
- They can help teams coordinate by imposing consistency. Can predict one another’s next
moves.
Procedural guides really can’t explain the tacit knowledge that people acquire over decades of
experience.
Teaching procedures
Another way to teach procedures: set up scenarios for various kinds of challenges and let the new
workers go through the scenarios. If the procedures make sense, then workers should get to see what
happened when they depart from optimal procedures. People can appreciate why the procedures
were put into place and can also gain a sense of the limitations of the procedures.
Seeing the invisible
Explicit knowledge is easy to write down, easy to communicate, easy to teach and easy to learn.
Tacit knowledge is being able to do things without being able to explain how, doesn’t boil down to
facts and rules, cannot be learned from a textbook.
We compare the situation against our experiences and judge whether the situation feels safe or
whether it matcher earlier situations in which we can cut it too close and got honked at. This is called
pattern matching.
Tacit knowledge plays a prominent part in our ability to cope with complex conditions.
We depend on unconscious processes to carry out tasks. That’s how our experience gets translated
into our actions, therefore its stays hidden under the surface of our lives.
Explicit knowledge
1. declarative information
2. routines and procedures
Tacit knowledge
3. perceptual skills
4. workarounds
5. pattern matching
6. judging typicality
7. mental models
Perceptual skills
With experience we learn where to look as well as how to make discriminations and recognize
connections. We learn how to direct our attention.
, Skilled performance depends on the way we look and listen. It depends on what we can notice and
on what kinds of discriminations we can make.
Adapting procedures.
The challenge lies in judging whether the criteria have been met. Knowing how to violate the
procedures is a type of tacit knowledge. Skilled performance depends on the kinds of perceptual
discriminations we can make and on how we interpret, modify and replace the standard procedures
when they don’t work.
Pattern matching
Every type of expert we have studied has built up a repertoire of patterns to quickly make sense of
what is happening. They are the basis of intuitions The patterns let us judge what category of
situation we are facing. Store detectives attempt to put people into two categories: shoppers and
shoplifters. They are studying the pattern of activity, and asking themselves whether this is something
a legitimate shopper would be doing of whether it seems like the ruse of the shoplifter.
Expertise depends on perceptual discriminations, the ability to go beyond standard procedures and
pattern matching.
Typicality and anomaly
We draw on dozens and hundreds of experiences to sense when something seems familiar, or to pick
up anomalies. The interplay between noticing typical cases and anomalous ones is a type of tacit
knowledge found in many fields.
Mental models
Mental models are the stories we construct to understand how things work. They mirror the events
or system they are modeling ,but they capture only a limited aspect of those events or that system.
We form our mental models from the way we understand causes. Except when having a conversation
about procedures, we don’t how to learn from one another about tacit knowledge. Because tacit
knowledge is tacit, we struggle to describe it.
Because we know more than we can tell, trainers gravitate toward describing the explicit parts of the
work.
In short, when we try to improve performance we usually emphasize explicit knowledge more than
tacit knowledge. In giving feedback, we tend to focus on specific procedures. These tendencies aren’t
problematic in well-ordered situations. However, they are inadequate for improving performance in
complex situations. There are techniques, such as Cognitive Task Analysis, that are specifically
designed to capture tacit knowledge. But these require training and practice.
Claim 2: decision biases distort our thinking
Three types of heuristics, strategies we commonly use in thinking.
1. The anchoring-and-adjustment heuristic
When we have to make an estimate and we don’t know the answer, one strategy we use is to find a
plausible answer and then adjust it up or down. The initial estimate is the anchor. People usually
don’t adjust the anchor very much.
2. The framing heuristics
The way a question is presented can determine the answer. As in the example the frame we use will
affect which types of data we notice. We can influence people’s judgments just by the way we frame
the problem.
Searching for the keys to adapti ve decision making; Gary Klein
Adaptive decision making
Claim 1: teaching people procedures helps them perform tasks more
skillfully
Procedures are made so that workers can perform this task even without years of practice.
Procedures help us to evaluate performance and are also relevant for health care because procedures
can improve performance and safety.
Procedures that trainee pilots were being taught didn’t match how skilled pilots performed the same
tasks. By the time people become proficient, they are seeing situations instead of calculating
procedures. Experts rely on their immediate intuitive responses.
The Dreyfus model of expertise emphasizes intuition and tacit knowledge that can’t be captured in
rules and procedures. People might need some rules in order to get started but they have to move
beyond rules in order to achieve mastery.
No matter how comprehensive procedures, people will probably run into something unexpected.
One way to ensure that a set of procedures is sufficient, is to take every eventuality into account.
However, this strategy can result in the opposite problem: procedural guides that are over-sufficient
and sometimes incomprehensible.
Procedures are often out of date because work practices keep evolving. Because they keep evolving,
guides are rarely complete. Over time, some procedures became obsolete or even counterproductive.
The people doing the job learned workarounds, they used their experience to adapt. The procedures
lagged behind the actual way people did their work. Up-to-date procedures had to be interpreted and
carried out by workers using their judgment and experience and obsolete procedures created even
more headaches.
Procedures can lead to mindlessness and complacency When we become passive, we don’t try to
improve our skills.
Procedures can erode expertise. People taught to understand the system develop richer mental
models than people taught to follow procedures.
Procedures can mislead us. The biggest worry is that following procedures can lead us in the wrong
direction and that we won’t notice because the reliance on procedures had made us so complacent.
In summary, we can see that procedures are insufficient, can get in the way, can interfere with
developing and applying expertise and can erode over time. Procedures work best in well-ordered
situations in which we don’t have to worry about changing conditions and we don’t have to take
context into account to figure out how to apply the procedures.
Unintended consequences:
We would establish adequate performance as the new ideal. It creates an endless cycle of updating
and we would live with ineffective designs. We discourage people from using their judgment. We
could give early retirement to senior workers, their tacit knowledge wouldn’t be useful and they are
the ones least likely to refer to procedures when they perform their jobs. We wouldn’t worry about
all the nuances of context. We would generate massive volumes of procedures because it is too
expensive to go back and cull procedures we don’t need anymore. We would save money by retaining
,our current ways of doing the work. We would issue procedures as a way to change behavior even
though there may be simpler and more effective strategies.
Replacement of state 1; in complex situations, people will need judgment skills to follow procedures
effectively and to go beyond them when necessary.
Like all tools, procedures have strengths and weaknesses, this are strengths:
- Training tools, help novices get started in learning a task
- They are memory aids, help workers overcome memory slips
- They can safeguard against interruptions, checklist helps to carry out all steps
- They reduce workload, easier to attend to critical aspects
- They are a way to compile experience and historical information. Useful when there is a lot of
turnover and few workers ever develop much skill. Less-experienced workers can do a
reasonable acceptable job.
- They can help teams coordinate by imposing consistency. Can predict one another’s next
moves.
Procedural guides really can’t explain the tacit knowledge that people acquire over decades of
experience.
Teaching procedures
Another way to teach procedures: set up scenarios for various kinds of challenges and let the new
workers go through the scenarios. If the procedures make sense, then workers should get to see what
happened when they depart from optimal procedures. People can appreciate why the procedures
were put into place and can also gain a sense of the limitations of the procedures.
Seeing the invisible
Explicit knowledge is easy to write down, easy to communicate, easy to teach and easy to learn.
Tacit knowledge is being able to do things without being able to explain how, doesn’t boil down to
facts and rules, cannot be learned from a textbook.
We compare the situation against our experiences and judge whether the situation feels safe or
whether it matcher earlier situations in which we can cut it too close and got honked at. This is called
pattern matching.
Tacit knowledge plays a prominent part in our ability to cope with complex conditions.
We depend on unconscious processes to carry out tasks. That’s how our experience gets translated
into our actions, therefore its stays hidden under the surface of our lives.
Explicit knowledge
1. declarative information
2. routines and procedures
Tacit knowledge
3. perceptual skills
4. workarounds
5. pattern matching
6. judging typicality
7. mental models
Perceptual skills
With experience we learn where to look as well as how to make discriminations and recognize
connections. We learn how to direct our attention.
, Skilled performance depends on the way we look and listen. It depends on what we can notice and
on what kinds of discriminations we can make.
Adapting procedures.
The challenge lies in judging whether the criteria have been met. Knowing how to violate the
procedures is a type of tacit knowledge. Skilled performance depends on the kinds of perceptual
discriminations we can make and on how we interpret, modify and replace the standard procedures
when they don’t work.
Pattern matching
Every type of expert we have studied has built up a repertoire of patterns to quickly make sense of
what is happening. They are the basis of intuitions The patterns let us judge what category of
situation we are facing. Store detectives attempt to put people into two categories: shoppers and
shoplifters. They are studying the pattern of activity, and asking themselves whether this is something
a legitimate shopper would be doing of whether it seems like the ruse of the shoplifter.
Expertise depends on perceptual discriminations, the ability to go beyond standard procedures and
pattern matching.
Typicality and anomaly
We draw on dozens and hundreds of experiences to sense when something seems familiar, or to pick
up anomalies. The interplay between noticing typical cases and anomalous ones is a type of tacit
knowledge found in many fields.
Mental models
Mental models are the stories we construct to understand how things work. They mirror the events
or system they are modeling ,but they capture only a limited aspect of those events or that system.
We form our mental models from the way we understand causes. Except when having a conversation
about procedures, we don’t how to learn from one another about tacit knowledge. Because tacit
knowledge is tacit, we struggle to describe it.
Because we know more than we can tell, trainers gravitate toward describing the explicit parts of the
work.
In short, when we try to improve performance we usually emphasize explicit knowledge more than
tacit knowledge. In giving feedback, we tend to focus on specific procedures. These tendencies aren’t
problematic in well-ordered situations. However, they are inadequate for improving performance in
complex situations. There are techniques, such as Cognitive Task Analysis, that are specifically
designed to capture tacit knowledge. But these require training and practice.
Claim 2: decision biases distort our thinking
Three types of heuristics, strategies we commonly use in thinking.
1. The anchoring-and-adjustment heuristic
When we have to make an estimate and we don’t know the answer, one strategy we use is to find a
plausible answer and then adjust it up or down. The initial estimate is the anchor. People usually
don’t adjust the anchor very much.
2. The framing heuristics
The way a question is presented can determine the answer. As in the example the frame we use will
affect which types of data we notice. We can influence people’s judgments just by the way we frame
the problem.
