NOTES – EVIDENCE BASED MANAGEMENT
Course setup
Four themes:
1. What is Evidence Based Management
2. Organizational Decision Making
3. Data Driven Decision Making
4. Implementing Decisions
LECTURE 1 - What is Evidence Based Management
Readings for week 1
Barends, E., Briner, R., & Rousseau, D. M. (2015). Evidence based practice: The basic
principles. Center for Evidence Based Management. Booklet at CEBMa. org retrieved at
October 8, 2019.
Why do evidence-based practice?
We always use evidence as a base for our decisions - but that’s not the same as
adopting an evidence-based approach
All practitioners always use evidence in their decisions on evidence but
Pay limited attention to quality and relevance of evidence
Use limited sources and types of evidence
Are easily pushed off track when trying to make better-informed decisions
Decisions about are important problems/opportunities and most likely solutions
should be based on the best available evidence
Evidence means any relevant information/data
– Scientific findings
– Organizational/context data
– Professional expertise
– Stakeholder concerns/perceptions
Where did the idea of evidence-based practice come from?
– 1990 Medicine
– 1998 Education
– 1998 Probation service
– 1999 Housing policy
– 1999 Social care
– 1999 Regeneration policy and practice
– 2000 Nursing
– 2000 Criminal justice
– 2005 Management?
Evidence-based practice
, It is the
– conscientious (effort, trying hard)
– explicit (clarity, writing down, communicating) and
– judicious (making judgement of reliability, considering trustworthiness,
believability)
– use of evidence
– from multiple sources to
– increase the likelihood of a favourable outcome by
– taking a structured/stepped approach
About the process
Not about certainties but probabilities (X is more likely than Y or doing nothing)
Used first to identify problem or opportunity and if one identified...
…Then used to identify possible solution or intervention
What is Evidence-Based Practice? (Barends, E., Briner, R., & Rousseau, D. M., 2015)
The basic idea of evidence-based practice is that good-quality decisions should be based on
a combination of critical thinking and the best available evidence. It is an approach to
decision-making and day-to-day work practice that helps practitioners to critically evaluate
the extent to which they can trust the evidence they have at hand. It also helps practitioners
to identify, find and evaluate additional evidence relevant to their decisions.
Many managers pay little attention to the quality of that evidence → results in bad decisions
based on unfounded beliefs. The bottom line is bad decisions, poor outcomes, and limited
understanding of why things go wrong.
Evidence-based practice is about making decisions through the conscientious, explicit and
judicious use of the best available evidence from multiple sources by ... (six steps that
Evidence Based Management proposes) (Barends, E., Briner, R., & Rousseau, D. M., 2015)
1. Asking: translating a practical issue or problem into an answerable question
2. Acquiring: systematically searching for and retrieving the evidence
, 3. Appraising: critically judging the trustworthiness and relevance of the evidence
4. Aggregating: weighing and pulling together the evidence
5. Applying: incorporating the evidence into the decision-making process
6. Assessing: evaluating the outcome of the decision taken
... to increase the likelihood of a favorable outcome.
These steps are needed to judiciously search and apply evidence in order to make decisions
How to do evidence based management
An example: Employee engagement
What is it?
Very popular management/human resources idea
Used by many medium-large sized organizations and seen as driver employee
performance
A bit like commitment, a bit like satisfaction, who knows?
(next slides - bunch of books abt employee engagement)
Example of employee engagement (EE)
Suppose you and your organization and senior management team believe that low
EE is somehow a problem in your organization
And because it is, you should measure or look at your EE scores and attempt to
increase ‘low’ scores somehow
How would you (a) approach this problem and (b) identify a solution from an
evidence-based practice approach?
Element 1: Practitioners’ professional expertise
Identifying the problem
– Have we seen EE problems before? What happened?
– Based on our experience, is the level of EE a problem?
– What do we believe about causes and consequences of low EE?
Identifying solution (only if EE is a problem)
– Have we seen EE interventions before? What happened?
– What do we believe about EE interventions?
, – Based on our experience, is the level of EE here a problem? What are costs &
benefits of intervening?
How relevant and applicable and trustworthy is this expertise?
Element 2: Organizational data
Identifying the problem
– What is the EE level?
– Are our measures of EE valid and reliable?
– Do data show that low EE is causing problems?
Identifying solution (only if EE is a problem)
– What attempts to enhance EE are currently in place and are they working?
– Are there relationships between EE and other data? E.g., employee type? Shift?
– Does cost/benefit indicate if increasing EE is worth it?
How relevant and applicable and trustworthy are our organizational data?
Element 3: Scientific literature
Identifying the problem
– What are the average rates of EE in my sector and location – is the level
here‘low’?
– What does systematically reviewed scientific evidence suggest are the
problemswith low EE?
Identifying solution (only if EE is a problem)
– What does research evidence from systematic reviews of scientific evidence
suggest are major causes of low EE?
– What does research evidence from systematic reviews of scientific evidence tell
us about interventions to increase EE?
How relevant and applicable and trustworthy are the scientific findings?
Element 4: Stakeholders values and concerns
Identifying the problem
– How do employees feel about and view the EE ‘problem’?
– Do they see negative consequences?
– What do managers think about the problem?
– Do customers or clients or service users have a view?
Identifying solution (only if EE is a problem)
– How do employees feel about and view the solutions?
– What do managers think about the solutions?
– What alternative explanations and proposed solutions do others have?
How relevant and applicable and trustworthy is evidence about stakeholder concerns?
What sources of evidence should be considered? (Barends, E., Briner, R., & Rousseau, D.
M., 2015)
Course setup
Four themes:
1. What is Evidence Based Management
2. Organizational Decision Making
3. Data Driven Decision Making
4. Implementing Decisions
LECTURE 1 - What is Evidence Based Management
Readings for week 1
Barends, E., Briner, R., & Rousseau, D. M. (2015). Evidence based practice: The basic
principles. Center for Evidence Based Management. Booklet at CEBMa. org retrieved at
October 8, 2019.
Why do evidence-based practice?
We always use evidence as a base for our decisions - but that’s not the same as
adopting an evidence-based approach
All practitioners always use evidence in their decisions on evidence but
Pay limited attention to quality and relevance of evidence
Use limited sources and types of evidence
Are easily pushed off track when trying to make better-informed decisions
Decisions about are important problems/opportunities and most likely solutions
should be based on the best available evidence
Evidence means any relevant information/data
– Scientific findings
– Organizational/context data
– Professional expertise
– Stakeholder concerns/perceptions
Where did the idea of evidence-based practice come from?
– 1990 Medicine
– 1998 Education
– 1998 Probation service
– 1999 Housing policy
– 1999 Social care
– 1999 Regeneration policy and practice
– 2000 Nursing
– 2000 Criminal justice
– 2005 Management?
Evidence-based practice
, It is the
– conscientious (effort, trying hard)
– explicit (clarity, writing down, communicating) and
– judicious (making judgement of reliability, considering trustworthiness,
believability)
– use of evidence
– from multiple sources to
– increase the likelihood of a favourable outcome by
– taking a structured/stepped approach
About the process
Not about certainties but probabilities (X is more likely than Y or doing nothing)
Used first to identify problem or opportunity and if one identified...
…Then used to identify possible solution or intervention
What is Evidence-Based Practice? (Barends, E., Briner, R., & Rousseau, D. M., 2015)
The basic idea of evidence-based practice is that good-quality decisions should be based on
a combination of critical thinking and the best available evidence. It is an approach to
decision-making and day-to-day work practice that helps practitioners to critically evaluate
the extent to which they can trust the evidence they have at hand. It also helps practitioners
to identify, find and evaluate additional evidence relevant to their decisions.
Many managers pay little attention to the quality of that evidence → results in bad decisions
based on unfounded beliefs. The bottom line is bad decisions, poor outcomes, and limited
understanding of why things go wrong.
Evidence-based practice is about making decisions through the conscientious, explicit and
judicious use of the best available evidence from multiple sources by ... (six steps that
Evidence Based Management proposes) (Barends, E., Briner, R., & Rousseau, D. M., 2015)
1. Asking: translating a practical issue or problem into an answerable question
2. Acquiring: systematically searching for and retrieving the evidence
, 3. Appraising: critically judging the trustworthiness and relevance of the evidence
4. Aggregating: weighing and pulling together the evidence
5. Applying: incorporating the evidence into the decision-making process
6. Assessing: evaluating the outcome of the decision taken
... to increase the likelihood of a favorable outcome.
These steps are needed to judiciously search and apply evidence in order to make decisions
How to do evidence based management
An example: Employee engagement
What is it?
Very popular management/human resources idea
Used by many medium-large sized organizations and seen as driver employee
performance
A bit like commitment, a bit like satisfaction, who knows?
(next slides - bunch of books abt employee engagement)
Example of employee engagement (EE)
Suppose you and your organization and senior management team believe that low
EE is somehow a problem in your organization
And because it is, you should measure or look at your EE scores and attempt to
increase ‘low’ scores somehow
How would you (a) approach this problem and (b) identify a solution from an
evidence-based practice approach?
Element 1: Practitioners’ professional expertise
Identifying the problem
– Have we seen EE problems before? What happened?
– Based on our experience, is the level of EE a problem?
– What do we believe about causes and consequences of low EE?
Identifying solution (only if EE is a problem)
– Have we seen EE interventions before? What happened?
– What do we believe about EE interventions?
, – Based on our experience, is the level of EE here a problem? What are costs &
benefits of intervening?
How relevant and applicable and trustworthy is this expertise?
Element 2: Organizational data
Identifying the problem
– What is the EE level?
– Are our measures of EE valid and reliable?
– Do data show that low EE is causing problems?
Identifying solution (only if EE is a problem)
– What attempts to enhance EE are currently in place and are they working?
– Are there relationships between EE and other data? E.g., employee type? Shift?
– Does cost/benefit indicate if increasing EE is worth it?
How relevant and applicable and trustworthy are our organizational data?
Element 3: Scientific literature
Identifying the problem
– What are the average rates of EE in my sector and location – is the level
here‘low’?
– What does systematically reviewed scientific evidence suggest are the
problemswith low EE?
Identifying solution (only if EE is a problem)
– What does research evidence from systematic reviews of scientific evidence
suggest are major causes of low EE?
– What does research evidence from systematic reviews of scientific evidence tell
us about interventions to increase EE?
How relevant and applicable and trustworthy are the scientific findings?
Element 4: Stakeholders values and concerns
Identifying the problem
– How do employees feel about and view the EE ‘problem’?
– Do they see negative consequences?
– What do managers think about the problem?
– Do customers or clients or service users have a view?
Identifying solution (only if EE is a problem)
– How do employees feel about and view the solutions?
– What do managers think about the solutions?
– What alternative explanations and proposed solutions do others have?
How relevant and applicable and trustworthy is evidence about stakeholder concerns?
What sources of evidence should be considered? (Barends, E., Briner, R., & Rousseau, D.
M., 2015)
