1
Week 1: What is evidence-based management
WHY DO WE DO EVIDENCE-BASED MANAGEMENT?
We always use evidence as a base for our decisions – but that’s not the same as adopting an
evidence-based approach
Helps practitioners to critically evaluate the extent to which they can trust the evidence they have at
hand. It also helps practitioners to identify, find and evaluate additional evidence relevant to their
decisions. All practitioners always use evidence in their decisions on evidence but:
• Pay limited attention to quality and relevance of evidence
• Use limited sources and types of evidence
• Are easily pushed off trach when trying to make better-informed decisions
Decisions about are important problems/opportunities and most likely solutions should be based on
the best available evidence
Evidence-based practices believes that decisions should be based on a combination of critical
thinking and the best available evidence.
Evidence means any relevant information/data
• Scientific findings
• Organizational/context data
• Professional expertise
• Stakeholder concerns/perceptions
WHERE DID IT COME FROM?
1990 Medicine
1998 Education
1998 Probation service
1999 Housing policy
1999 Social care
1999 Regeneration policy and practice
2000 Nursing
2000 Criminal justice
2005 Management?
EVIDENCE-BASED PRACTICE
It is the conscientious (effort, trying hard) explicit (clarity) judicious (making judgment of reliability)
use of evidence from multiple sources by…
1. Asking: translating a practical issue or problem into an answerable question
2. Acquiring: systematically searching for and retrieving the evidence
3. Appraising: critically judging the trustworthiness and relevance of the evidence
4. Aggregating: weighing and pulling together the evidence
5. Applying: incorporating the evidence into the decision-making process
6. Assessing: evaluating the outcome of the decision taken
… to increase the likelihood of favorable outcome taking a structured/stepped approach.
It is about the process, not about certainties but probabilities
HOW TO DO EVIDENCE BASED MANAGEMENT
Example: Employee engagement
• Very popular management/hr idea
• Used by many medium-large sized organization and seen as driver of employee performance
, 2
Example of employee engagement (EE)
• Suppose you and your organization and senior management team believe that low EE is
somehow a problem in your organization
• And because it is, you should measure or look at your EE scores and attempt to increase
‘low’ scores somehow
• How would you (a) approach this problem and (b) identify a solution from an evidence-based
practice approach?
FOUR SOURCES
1. Scientific literature: empirical studies
2. Organization: internal data
3. Stakeholders: values and concerns
4. Practitioners: professional expertise
The scientific literature = scientific research published in academic journals.
• volume has escalated hugely over the past few decades
• much relevant research from outside the management discipline
• new research is always being produced, which often changes our understanding
• essential to know how to search for studies and to be able to judge how trustworthy and
relevant they are.
The organization
• can be ‘hard’ numbers (e.g. staff turnover rates ) or ‘soft’ elements (perceptions)
• essential to identifying problems that require managers’ attention
• essential to determining likely causes, plausible solutions and what is needed to implement
these solutions
Practitioners = professional experience and judgment of managers, consultants, business leaders and
other practitioners
• accumulated over time through reflection on the outcomes of similar actions taken in similar
situations also referred to as ‘tacit’ knowledge
• differs from intuition and personal opinion because it reflects the specialized knowledge
acquired by repeated experience and practice of specialized activities
• can be vital for determining whether a management issue really does require attention, if
the available organizational data are trustworthy, whether research findings apply in a
particular situation or how likely a proposed solution is to work in a particular context.
Stakeholders = stakeholder values and concerns reflect what stakeholders believe to be important,
which in turn affects how they tend to react to the possible consequences of the organization’s
decisions.
• organizations that serve or respond to different stakeholders can reach very different
decisions on the basis of the same evidence
• important for ethical reasons
• provides a frame of reference from which to analyze evidence from other sources
• provides important information to predict the outcome
Element 1: Practitioners’ professional expertise
Identifying the problem:
• Have we seen EE problems before? What happened?
• Based on our experience, is the level of EE a problem?
• What do we believe about causes and consequences of low EE?
, 3
Identifying solution (only if EE is a problem):
• Have we seen EE interventions before? What happened?
• What do we believe about EE interventions?
• Based on our experience, is the level of EE here a problem? What are costs & benefits of
intervening?
How relevant, applicable, and trustworthy is this expertise?
Element 2: Organizational data
Identifying the problem:
• What is the EE level?
• Are our measures of EE valid and reliable?
• Do data show that low EE is causing problems?
Identifying solution (only if EE is a problem):
• What attempts to enhance EE are currently in place and are they working?
• Are there relationships between EE and other data? E.g., employee type? Shift?
• Does cost/benefit indicate if increasing EE is worth it?
How relevant and applicable and trustworthy are our organizational data?
Element 3: Scientific literature
Identifying the problem
• What are the average rates of EE in my sector and location – is the level here ‘low’?
• What does systematically review scientific evidence suggest are the problems with low EE?
Identifying solution (only if EE is a problem)
• What does research evidence from systematic reviews of scientific evidence suggest are
major causes of low EE?
• What does research evidence from systematic reviews of scientific evidence tell us about
interventions to increase EE?
How relevant and applicable and trustworthy are the scientific findings?
Element 4: Stakeholders values and concerns
Identifying the problem
• How do employees feel about and view the EE ‘problem’?
• Do they see negative consequences?
• What do managers think about the problem?
• Do customers or clients or service users have a view?
Identifying solution (only if EE is a problem)
• How do employees feel about and view the solutions?
• What do managers think about the solutions?
• What alternative explanations and proposed solutions do others have?
How relevant and applicable and trustworthy is evidence about stakeholder concerns?
Week 1: What is evidence-based management
WHY DO WE DO EVIDENCE-BASED MANAGEMENT?
We always use evidence as a base for our decisions – but that’s not the same as adopting an
evidence-based approach
Helps practitioners to critically evaluate the extent to which they can trust the evidence they have at
hand. It also helps practitioners to identify, find and evaluate additional evidence relevant to their
decisions. All practitioners always use evidence in their decisions on evidence but:
• Pay limited attention to quality and relevance of evidence
• Use limited sources and types of evidence
• Are easily pushed off trach when trying to make better-informed decisions
Decisions about are important problems/opportunities and most likely solutions should be based on
the best available evidence
Evidence-based practices believes that decisions should be based on a combination of critical
thinking and the best available evidence.
Evidence means any relevant information/data
• Scientific findings
• Organizational/context data
• Professional expertise
• Stakeholder concerns/perceptions
WHERE DID IT COME FROM?
1990 Medicine
1998 Education
1998 Probation service
1999 Housing policy
1999 Social care
1999 Regeneration policy and practice
2000 Nursing
2000 Criminal justice
2005 Management?
EVIDENCE-BASED PRACTICE
It is the conscientious (effort, trying hard) explicit (clarity) judicious (making judgment of reliability)
use of evidence from multiple sources by…
1. Asking: translating a practical issue or problem into an answerable question
2. Acquiring: systematically searching for and retrieving the evidence
3. Appraising: critically judging the trustworthiness and relevance of the evidence
4. Aggregating: weighing and pulling together the evidence
5. Applying: incorporating the evidence into the decision-making process
6. Assessing: evaluating the outcome of the decision taken
… to increase the likelihood of favorable outcome taking a structured/stepped approach.
It is about the process, not about certainties but probabilities
HOW TO DO EVIDENCE BASED MANAGEMENT
Example: Employee engagement
• Very popular management/hr idea
• Used by many medium-large sized organization and seen as driver of employee performance
, 2
Example of employee engagement (EE)
• Suppose you and your organization and senior management team believe that low EE is
somehow a problem in your organization
• And because it is, you should measure or look at your EE scores and attempt to increase
‘low’ scores somehow
• How would you (a) approach this problem and (b) identify a solution from an evidence-based
practice approach?
FOUR SOURCES
1. Scientific literature: empirical studies
2. Organization: internal data
3. Stakeholders: values and concerns
4. Practitioners: professional expertise
The scientific literature = scientific research published in academic journals.
• volume has escalated hugely over the past few decades
• much relevant research from outside the management discipline
• new research is always being produced, which often changes our understanding
• essential to know how to search for studies and to be able to judge how trustworthy and
relevant they are.
The organization
• can be ‘hard’ numbers (e.g. staff turnover rates ) or ‘soft’ elements (perceptions)
• essential to identifying problems that require managers’ attention
• essential to determining likely causes, plausible solutions and what is needed to implement
these solutions
Practitioners = professional experience and judgment of managers, consultants, business leaders and
other practitioners
• accumulated over time through reflection on the outcomes of similar actions taken in similar
situations also referred to as ‘tacit’ knowledge
• differs from intuition and personal opinion because it reflects the specialized knowledge
acquired by repeated experience and practice of specialized activities
• can be vital for determining whether a management issue really does require attention, if
the available organizational data are trustworthy, whether research findings apply in a
particular situation or how likely a proposed solution is to work in a particular context.
Stakeholders = stakeholder values and concerns reflect what stakeholders believe to be important,
which in turn affects how they tend to react to the possible consequences of the organization’s
decisions.
• organizations that serve or respond to different stakeholders can reach very different
decisions on the basis of the same evidence
• important for ethical reasons
• provides a frame of reference from which to analyze evidence from other sources
• provides important information to predict the outcome
Element 1: Practitioners’ professional expertise
Identifying the problem:
• Have we seen EE problems before? What happened?
• Based on our experience, is the level of EE a problem?
• What do we believe about causes and consequences of low EE?
, 3
Identifying solution (only if EE is a problem):
• Have we seen EE interventions before? What happened?
• What do we believe about EE interventions?
• Based on our experience, is the level of EE here a problem? What are costs & benefits of
intervening?
How relevant, applicable, and trustworthy is this expertise?
Element 2: Organizational data
Identifying the problem:
• What is the EE level?
• Are our measures of EE valid and reliable?
• Do data show that low EE is causing problems?
Identifying solution (only if EE is a problem):
• What attempts to enhance EE are currently in place and are they working?
• Are there relationships between EE and other data? E.g., employee type? Shift?
• Does cost/benefit indicate if increasing EE is worth it?
How relevant and applicable and trustworthy are our organizational data?
Element 3: Scientific literature
Identifying the problem
• What are the average rates of EE in my sector and location – is the level here ‘low’?
• What does systematically review scientific evidence suggest are the problems with low EE?
Identifying solution (only if EE is a problem)
• What does research evidence from systematic reviews of scientific evidence suggest are
major causes of low EE?
• What does research evidence from systematic reviews of scientific evidence tell us about
interventions to increase EE?
How relevant and applicable and trustworthy are the scientific findings?
Element 4: Stakeholders values and concerns
Identifying the problem
• How do employees feel about and view the EE ‘problem’?
• Do they see negative consequences?
• What do managers think about the problem?
• Do customers or clients or service users have a view?
Identifying solution (only if EE is a problem)
• How do employees feel about and view the solutions?
• What do managers think about the solutions?
• What alternative explanations and proposed solutions do others have?
How relevant and applicable and trustworthy is evidence about stakeholder concerns?
