Onderzoeksmethoden P&O
Samenvatting
Academiejaar 2019-2020
Prof. Greet Van Hoye
EP
1
,Inhoud
Introduction to research HRM & OB evidence-based ........................................................................................... 6
Objectives .......................................................................................................................................................... 6
Final competences............................................................................................................................................. 6
Approach ........................................................................................................................................................... 6
Evaluation .......................................................................................................................................................... 7
Application HR & OB research ........................................................................................................................... 7
Evidence-based management (EBM) ................................................................................................................ 8
History ............................................................................................................................................................... 8
History evidence-based practice ....................................................................................................................... 8
Threats/opportunities EBM? ............................................................................................................................. 9
Evidence-based practice.................................................................................................................................... 9
Evidence-based decision-making ...................................................................................................................... 9
Scientific evidence ........................................................................................................................................... 10
Organizational evidence .................................................................................................................................. 10
Experiential evidence ...................................................................................................................................... 10
Stakeholder evidence ...................................................................................................................................... 10
In his own words … .......................................................................................................................................... 10
Organizational evidence .................................................................................................................................. 11
Experiential evidence ...................................................................................................................................... 11
Stakeholder evidence ...................................................................................................................................... 11
Exercise............................................................................................................................................................ 11
But … best available evidence ......................................................................................................................... 12
Importance of critical appraisal....................................................................................................................... 12
Examples critical appraisal .............................................................................................................................. 12
Best available evidence ................................................................................................................................... 13
What is the scientific evidence for evidence-based management? ............................................................... 13
Common misconceptions ................................................................................................................................ 14
Becoming an evidence-based HR practitioner ................................................................................................ 14
Evidence-based practice.................................................................................................................................. 14
Searching for scientific evidence ..................................................................................................................... 15
Scientific evidence: Practitioner access .......................................................................................................... 15
Research topics and research questions in HRM and OB ................................................................................... 16
Evidence-based practice.................................................................................................................................. 16
2
, Master’s dissertation: topic choice ................................................................................................................. 16
Topic choice..................................................................................................................................................... 17
Bridging the S-p gap in topic choice ................................................................................................................ 18
Determining your research question .............................................................................................................. 18
Consensus shifting ........................................................................................................................................... 19
Consensus shifting: example ........................................................................................................................... 19
Consensus creation ......................................................................................................................................... 19
Consensus creation: example.......................................................................................................................... 20
Reformulate in consensus terms..................................................................................................................... 20
The introduction: setting the hook ................................................................................................................. 21
The introduction .............................................................................................................................................. 21
Who cares? .................................................................................................................................................. 21
What do we know, what don’t we know, and so what? ............................................................................. 22
What will we learn?..................................................................................................................................... 22
The introduction: Process ............................................................................................................................... 22
Grounding hypotheses .................................................................................................................................... 23
Grounding hypotheses: example .................................................................................................................... 23
Formulating hypotheses.................................................................................................................................. 24
Introduction carless & Hetherington (2011) ................................................................................................... 24
Design and critical evaluation of research in HRM & OB .................................................................................... 26
Evidence-based practice.................................................................................................................................. 26
Critical appraisal of evidence .......................................................................................................................... 26
What is the best available evidence? .............................................................................................................. 26
Internal validity ............................................................................................................................................... 27
Methodological pitfalls.................................................................................................................................... 27
Correlation does not equal causation! ............................................................................................................ 28
Reverse causation ........................................................................................................................................... 28
Causality .......................................................................................................................................................... 28
Methodological pitfalls.................................................................................................................................... 28
Applied to hrm and ob .................................................................................................................................... 29
Construct validity ............................................................................................................................................ 30
Evidence for Construct validity ....................................................................................................................... 30
Statistical conclusion validity .......................................................................................................................... 31
External validity ............................................................................................................................................... 31
3
, Examples ......................................................................................................................................................... 31
Common method bias ..................................................................................................................................... 32
Procedural remedies ....................................................................................................................................... 33
Statistical remedies ......................................................................................................................................... 33
CRAFTING THE METHOD section: 3C’s ............................................................................................................ 34
Amj major flaws research design .................................................................................................................... 34
Design & method ng & burke (2010) .............................................................................................................. 35
Quantitative research in HRM & OB ................................................................................................................... 37
Quantitative research...................................................................................................................................... 37
Quantitative research questions ..................................................................................................................... 37
Experiments..................................................................................................................................................... 37
Surveys ............................................................................................................................................................ 38
Administration survey ..................................................................................................................................... 39
Designing a survey........................................................................................................................................... 39
Developing a measure ..................................................................................................................................... 40
Writing items ................................................................................................................................................... 40
Administration survey ..................................................................................................................................... 41
Increasing response rate ................................................................................................................................. 41
Moderation and mediation ............................................................................................................................. 41
Moderation ..................................................................................................................................................... 42
Mediation ........................................................................................................................................................ 42
Moderator or mediator ................................................................................................................................... 43
Testing moderation ......................................................................................................................................... 43
Testing mediation............................................................................................................................................ 44
Testing Moderation and mediation ................................................................................................................ 45
Carless & Hetherington (2011) ........................................................................................................................ 45
Qualitative research in HRM & OB ...................................................................................................................... 47
Research questions ......................................................................................................................................... 47
Data collection and processing ....................................................................................................................... 48
Reporting ......................................................................................................................................................... 49
Similarities ....................................................................................................................................................... 49
Qualitative research: Don’ts (Pratt, AMJ, 2009) ............................................................................................. 49
Qualitative research: Do’s (Pratt, AMJ, 2009) ................................................................................................. 50
......................................................................................................................................................................... 51
4
, The interview................................................................................................................................................... 51
Type of interviews ........................................................................................................................................... 52
Interview Guide ............................................................................................................................................... 52
Example Interview Guide ................................................................................................................................ 52
Taking the interview ........................................................................................................................................ 53
Attention! ........................................................................................................................................................ 53
Analysis interviews .......................................................................................................................................... 54
Supporting software ........................................................................................................................................ 55
Article qualitative study (Bouwhuis et al., 2018) ............................................................................................ 55
......................................................................................................................................................................... 56
Reporting research in HRM & OB ........................................................................................................................ 57
Results – Quantitative research ...................................................................................................................... 57
Results: exercises ............................................................................................................................................ 58
Discussion ........................................................................................................................................................ 60
Discussion: common mistakes (Geletkanycz & Tepper, 2012)........................................................................ 61
Discussion: research ........................................................................................................................................ 62
Remember? ..................................................................................................................................................... 63
Limitations: which type of validity? ................................................................................................................ 64
Future research: which type of validity? ......................................................................................................... 64
Integrative practice seminar ............................................................................................................................... 65
Final competences........................................................................................................................................... 65
Evaluation ........................................................................................................................................................ 65
Questions ........................................................................................................................................................ 66
Examenvragen januari 2018 Onderzoeksmethoden – P&O............................................................................ 70
5
,Research Methods in Human
Resource Management and
Organizational Behavior
Introduction to research HRM & OB evidence-based
Objectives
To acquire advanced knowledge, understanding, and skills on
social-scientific research methods
Domain of HRM and OB
Principles of evidence-based HRM
To be able to judge the design and results of HRM & OB research with respect to quality and
usefulness
For master’s dissertation and work field
Final competences
Understand the need for and the value of evidence-based HRM
Critically evaluate the quality of research in HRM & OB
Critically evaluate the usefulness of research in HRM & OB
Translate problems and challenges in the work field of HRM & OB into focused research questions
Address problems and challenges in the work field of HRM & OB on the basis of scientific research
Approach
Focus on understanding and application
Requires preparation, active participation, and exercise
All class questions/exercises = Example exam questions
Integrative practice seminar = Practice exam
Slides on Ufora before class
If preparation is needed, this needs to be done before unlocking the slides
6
,Evaluation
Periodic evaluation (100%)
Written exam with open questions
Slides + Reader on Ufora
▪ Reader provides more background and explanation to the slides
Focus on understanding and application
So NOT: “Give the definition of validity”
BUT: “Evaluate the validity of this research design” or “Propose a valid design to answer this
research question”
Periodic evaluation (100%)
Active participation is the best preparation
Most questions will be about a (new) scientific HRM & OB article
You can bring a dictionary English/native language (no notes!)
Example questions = all class questions/exercises
▪ Tip: Practice on other articles
No specific questions on SPSS
▪ But be able to interpret results tables from articles (e.g., descriptives, correlations,
regression, mediation, moderation)
Application HR & OB research
Large scientific knowledge on the effectiveness of many HR & OB practices (supported by multiple
studies or meta-analyses)
But is this known and applied in the field?
Large survey on knowledge HR & OB practices
US: N = 959 HR managers (Rynes et al., 2002, HRM)
NL: N = 626 HR professionals (Sanders et al., 2008, IJHRM)
7
, Large Scientist-Practitioner gap
Results HR & OB research are not enough known and applied in the work field
Better research knowledge was predicted by
Job/educational level
Professional certification
Positive attitude to usefulness academic research
Academic reading
Better research knowledge would allow HR professionals to make better and more successful
decisions
Need for evidence-based management
Evidence-based management (EBM)
“The systematic, evidence-informed practice of management, incorporating scientific knowledge in
the content and process of making decisions”
“Knowing how to obtain and use scientific evidence and reliable business knowledge helps
practitioners respond effectively to the uncertainty they face everyday in organizations”
History
“there is a large research-user gap”
“practitioners do not read academic journals”
“the findings of research into what is an effective intervention are not being translated into actual
practice”
“academics not practitioners are driving the research agenda”
“practice is being driven more by fads and fashions than research”
History evidence-based practice
1991: medicine
1998: education
2000: nursing
2005-2006: management
8
,Threats/opportunities EBM?
- Information overload
- Overestimate individual experience (N=1)
- Inaccurate information/beliefs
- Preference for quick solutions
- Hypes and trends in management
- Bias and errors in decision-making
Eg., attribution bias, confirmation bias
+ Call for more accountability
+ Evolution to knowledge management (eg., CIO)
+ Higher accessibility of information (eg., Internet, MIS)
+ Need for more rational and objective decision-making
Notes: A lot info and not much time. Also look at other evidence then just your personal experience…
Intelligence is not evidence based.
Attribution bias: if it is wrong, it’s not your fault. You search some other but when it is right it’s because of
you.
More accountability. Collecting more info.
Evidence-based practice
is about making decisions through the conscientious, explicit and thoughtful use of the best available
evidence from multiple sources by
Asking: translating a practical issue or problem into an answerable question? Become aware
of the decisions you make
Acquiring: systematically searching for and retrieving the evidence
Appraising: critically judging the trustworthiness and relevance of the evidence
Aggregating: weighing and pulling together the evidence
Applying: incorporating the evidence into the decision-making process
Assessing: evaluating the outcome of the decision taken, Use it for imput for later decisions
to increase the likelihood of a favorable outcome.
Evidence-based decision-making
Notes: More and more info is available in the organization.
Not intuition, but based on experience, try to rationalize how
can I help the stakeholder and how the will be effected.
9
, Scientific evidence
Chesley Burnett "Sully" Sullenberger III (Danville (Californië), 23
januari 1951) is een Amerikaans piloot in de burgerluchtvaart,
veiligheidsexpert en ongevallenanalist. Hij werd vooral bekend als
de gezagvoerder die de noodlanding van US Airways-vlucht 1549 op
de rivier de Hudson in New York uitvoerde op 15 januari 2009.
Chesley Sullenberger, USAIR pilot, has been a visiting scholar at UC Berkeley’s Collaborative for
Catastrophic Risk Management since 2007
Does research on how to make decisions to maintain safety despite technological complexity and crisis
conditions
Organizational evidence
Has written and analyzed aviation accident reports for over 20 years
Experiential evidence
Formal Education to Prime His Skills: Sully is a graduate of the U.S. Air Force Academy and holds
masters degrees from both Purdue University in Industrial Psychology and the University of Northern
Colorado in Public Administration
Used Decision Aids to Support Good Decision: As Sully considered what decision to make that day, he
had his copilot review and follow all checklists on board relevant to crash landings
Stakeholder evidence
The last person to leave the plane, Chesley Sullenberger twice walked the plane’s aisle to check all
passengers were off
Sully’s last act onboard was to grab the passenger list. Used on-shore to verify rescue of all
passengers and crew
In his own words …
“One way of looking at this might be that for 42 years, I've been making small, regular deposits in this
bank of experience, education and training.
And on January 15, 2009 the balance was sufficient so that I could make a very large withdrawal.”
10
Samenvatting
Academiejaar 2019-2020
Prof. Greet Van Hoye
EP
1
,Inhoud
Introduction to research HRM & OB evidence-based ........................................................................................... 6
Objectives .......................................................................................................................................................... 6
Final competences............................................................................................................................................. 6
Approach ........................................................................................................................................................... 6
Evaluation .......................................................................................................................................................... 7
Application HR & OB research ........................................................................................................................... 7
Evidence-based management (EBM) ................................................................................................................ 8
History ............................................................................................................................................................... 8
History evidence-based practice ....................................................................................................................... 8
Threats/opportunities EBM? ............................................................................................................................. 9
Evidence-based practice.................................................................................................................................... 9
Evidence-based decision-making ...................................................................................................................... 9
Scientific evidence ........................................................................................................................................... 10
Organizational evidence .................................................................................................................................. 10
Experiential evidence ...................................................................................................................................... 10
Stakeholder evidence ...................................................................................................................................... 10
In his own words … .......................................................................................................................................... 10
Organizational evidence .................................................................................................................................. 11
Experiential evidence ...................................................................................................................................... 11
Stakeholder evidence ...................................................................................................................................... 11
Exercise............................................................................................................................................................ 11
But … best available evidence ......................................................................................................................... 12
Importance of critical appraisal....................................................................................................................... 12
Examples critical appraisal .............................................................................................................................. 12
Best available evidence ................................................................................................................................... 13
What is the scientific evidence for evidence-based management? ............................................................... 13
Common misconceptions ................................................................................................................................ 14
Becoming an evidence-based HR practitioner ................................................................................................ 14
Evidence-based practice.................................................................................................................................. 14
Searching for scientific evidence ..................................................................................................................... 15
Scientific evidence: Practitioner access .......................................................................................................... 15
Research topics and research questions in HRM and OB ................................................................................... 16
Evidence-based practice.................................................................................................................................. 16
2
, Master’s dissertation: topic choice ................................................................................................................. 16
Topic choice..................................................................................................................................................... 17
Bridging the S-p gap in topic choice ................................................................................................................ 18
Determining your research question .............................................................................................................. 18
Consensus shifting ........................................................................................................................................... 19
Consensus shifting: example ........................................................................................................................... 19
Consensus creation ......................................................................................................................................... 19
Consensus creation: example.......................................................................................................................... 20
Reformulate in consensus terms..................................................................................................................... 20
The introduction: setting the hook ................................................................................................................. 21
The introduction .............................................................................................................................................. 21
Who cares? .................................................................................................................................................. 21
What do we know, what don’t we know, and so what? ............................................................................. 22
What will we learn?..................................................................................................................................... 22
The introduction: Process ............................................................................................................................... 22
Grounding hypotheses .................................................................................................................................... 23
Grounding hypotheses: example .................................................................................................................... 23
Formulating hypotheses.................................................................................................................................. 24
Introduction carless & Hetherington (2011) ................................................................................................... 24
Design and critical evaluation of research in HRM & OB .................................................................................... 26
Evidence-based practice.................................................................................................................................. 26
Critical appraisal of evidence .......................................................................................................................... 26
What is the best available evidence? .............................................................................................................. 26
Internal validity ............................................................................................................................................... 27
Methodological pitfalls.................................................................................................................................... 27
Correlation does not equal causation! ............................................................................................................ 28
Reverse causation ........................................................................................................................................... 28
Causality .......................................................................................................................................................... 28
Methodological pitfalls.................................................................................................................................... 28
Applied to hrm and ob .................................................................................................................................... 29
Construct validity ............................................................................................................................................ 30
Evidence for Construct validity ....................................................................................................................... 30
Statistical conclusion validity .......................................................................................................................... 31
External validity ............................................................................................................................................... 31
3
, Examples ......................................................................................................................................................... 31
Common method bias ..................................................................................................................................... 32
Procedural remedies ....................................................................................................................................... 33
Statistical remedies ......................................................................................................................................... 33
CRAFTING THE METHOD section: 3C’s ............................................................................................................ 34
Amj major flaws research design .................................................................................................................... 34
Design & method ng & burke (2010) .............................................................................................................. 35
Quantitative research in HRM & OB ................................................................................................................... 37
Quantitative research...................................................................................................................................... 37
Quantitative research questions ..................................................................................................................... 37
Experiments..................................................................................................................................................... 37
Surveys ............................................................................................................................................................ 38
Administration survey ..................................................................................................................................... 39
Designing a survey........................................................................................................................................... 39
Developing a measure ..................................................................................................................................... 40
Writing items ................................................................................................................................................... 40
Administration survey ..................................................................................................................................... 41
Increasing response rate ................................................................................................................................. 41
Moderation and mediation ............................................................................................................................. 41
Moderation ..................................................................................................................................................... 42
Mediation ........................................................................................................................................................ 42
Moderator or mediator ................................................................................................................................... 43
Testing moderation ......................................................................................................................................... 43
Testing mediation............................................................................................................................................ 44
Testing Moderation and mediation ................................................................................................................ 45
Carless & Hetherington (2011) ........................................................................................................................ 45
Qualitative research in HRM & OB ...................................................................................................................... 47
Research questions ......................................................................................................................................... 47
Data collection and processing ....................................................................................................................... 48
Reporting ......................................................................................................................................................... 49
Similarities ....................................................................................................................................................... 49
Qualitative research: Don’ts (Pratt, AMJ, 2009) ............................................................................................. 49
Qualitative research: Do’s (Pratt, AMJ, 2009) ................................................................................................. 50
......................................................................................................................................................................... 51
4
, The interview................................................................................................................................................... 51
Type of interviews ........................................................................................................................................... 52
Interview Guide ............................................................................................................................................... 52
Example Interview Guide ................................................................................................................................ 52
Taking the interview ........................................................................................................................................ 53
Attention! ........................................................................................................................................................ 53
Analysis interviews .......................................................................................................................................... 54
Supporting software ........................................................................................................................................ 55
Article qualitative study (Bouwhuis et al., 2018) ............................................................................................ 55
......................................................................................................................................................................... 56
Reporting research in HRM & OB ........................................................................................................................ 57
Results – Quantitative research ...................................................................................................................... 57
Results: exercises ............................................................................................................................................ 58
Discussion ........................................................................................................................................................ 60
Discussion: common mistakes (Geletkanycz & Tepper, 2012)........................................................................ 61
Discussion: research ........................................................................................................................................ 62
Remember? ..................................................................................................................................................... 63
Limitations: which type of validity? ................................................................................................................ 64
Future research: which type of validity? ......................................................................................................... 64
Integrative practice seminar ............................................................................................................................... 65
Final competences........................................................................................................................................... 65
Evaluation ........................................................................................................................................................ 65
Questions ........................................................................................................................................................ 66
Examenvragen januari 2018 Onderzoeksmethoden – P&O............................................................................ 70
5
,Research Methods in Human
Resource Management and
Organizational Behavior
Introduction to research HRM & OB evidence-based
Objectives
To acquire advanced knowledge, understanding, and skills on
social-scientific research methods
Domain of HRM and OB
Principles of evidence-based HRM
To be able to judge the design and results of HRM & OB research with respect to quality and
usefulness
For master’s dissertation and work field
Final competences
Understand the need for and the value of evidence-based HRM
Critically evaluate the quality of research in HRM & OB
Critically evaluate the usefulness of research in HRM & OB
Translate problems and challenges in the work field of HRM & OB into focused research questions
Address problems and challenges in the work field of HRM & OB on the basis of scientific research
Approach
Focus on understanding and application
Requires preparation, active participation, and exercise
All class questions/exercises = Example exam questions
Integrative practice seminar = Practice exam
Slides on Ufora before class
If preparation is needed, this needs to be done before unlocking the slides
6
,Evaluation
Periodic evaluation (100%)
Written exam with open questions
Slides + Reader on Ufora
▪ Reader provides more background and explanation to the slides
Focus on understanding and application
So NOT: “Give the definition of validity”
BUT: “Evaluate the validity of this research design” or “Propose a valid design to answer this
research question”
Periodic evaluation (100%)
Active participation is the best preparation
Most questions will be about a (new) scientific HRM & OB article
You can bring a dictionary English/native language (no notes!)
Example questions = all class questions/exercises
▪ Tip: Practice on other articles
No specific questions on SPSS
▪ But be able to interpret results tables from articles (e.g., descriptives, correlations,
regression, mediation, moderation)
Application HR & OB research
Large scientific knowledge on the effectiveness of many HR & OB practices (supported by multiple
studies or meta-analyses)
But is this known and applied in the field?
Large survey on knowledge HR & OB practices
US: N = 959 HR managers (Rynes et al., 2002, HRM)
NL: N = 626 HR professionals (Sanders et al., 2008, IJHRM)
7
, Large Scientist-Practitioner gap
Results HR & OB research are not enough known and applied in the work field
Better research knowledge was predicted by
Job/educational level
Professional certification
Positive attitude to usefulness academic research
Academic reading
Better research knowledge would allow HR professionals to make better and more successful
decisions
Need for evidence-based management
Evidence-based management (EBM)
“The systematic, evidence-informed practice of management, incorporating scientific knowledge in
the content and process of making decisions”
“Knowing how to obtain and use scientific evidence and reliable business knowledge helps
practitioners respond effectively to the uncertainty they face everyday in organizations”
History
“there is a large research-user gap”
“practitioners do not read academic journals”
“the findings of research into what is an effective intervention are not being translated into actual
practice”
“academics not practitioners are driving the research agenda”
“practice is being driven more by fads and fashions than research”
History evidence-based practice
1991: medicine
1998: education
2000: nursing
2005-2006: management
8
,Threats/opportunities EBM?
- Information overload
- Overestimate individual experience (N=1)
- Inaccurate information/beliefs
- Preference for quick solutions
- Hypes and trends in management
- Bias and errors in decision-making
Eg., attribution bias, confirmation bias
+ Call for more accountability
+ Evolution to knowledge management (eg., CIO)
+ Higher accessibility of information (eg., Internet, MIS)
+ Need for more rational and objective decision-making
Notes: A lot info and not much time. Also look at other evidence then just your personal experience…
Intelligence is not evidence based.
Attribution bias: if it is wrong, it’s not your fault. You search some other but when it is right it’s because of
you.
More accountability. Collecting more info.
Evidence-based practice
is about making decisions through the conscientious, explicit and thoughtful use of the best available
evidence from multiple sources by
Asking: translating a practical issue or problem into an answerable question? Become aware
of the decisions you make
Acquiring: systematically searching for and retrieving the evidence
Appraising: critically judging the trustworthiness and relevance of the evidence
Aggregating: weighing and pulling together the evidence
Applying: incorporating the evidence into the decision-making process
Assessing: evaluating the outcome of the decision taken, Use it for imput for later decisions
to increase the likelihood of a favorable outcome.
Evidence-based decision-making
Notes: More and more info is available in the organization.
Not intuition, but based on experience, try to rationalize how
can I help the stakeholder and how the will be effected.
9
, Scientific evidence
Chesley Burnett "Sully" Sullenberger III (Danville (Californië), 23
januari 1951) is een Amerikaans piloot in de burgerluchtvaart,
veiligheidsexpert en ongevallenanalist. Hij werd vooral bekend als
de gezagvoerder die de noodlanding van US Airways-vlucht 1549 op
de rivier de Hudson in New York uitvoerde op 15 januari 2009.
Chesley Sullenberger, USAIR pilot, has been a visiting scholar at UC Berkeley’s Collaborative for
Catastrophic Risk Management since 2007
Does research on how to make decisions to maintain safety despite technological complexity and crisis
conditions
Organizational evidence
Has written and analyzed aviation accident reports for over 20 years
Experiential evidence
Formal Education to Prime His Skills: Sully is a graduate of the U.S. Air Force Academy and holds
masters degrees from both Purdue University in Industrial Psychology and the University of Northern
Colorado in Public Administration
Used Decision Aids to Support Good Decision: As Sully considered what decision to make that day, he
had his copilot review and follow all checklists on board relevant to crash landings
Stakeholder evidence
The last person to leave the plane, Chesley Sullenberger twice walked the plane’s aisle to check all
passengers were off
Sully’s last act onboard was to grab the passenger list. Used on-shore to verify rescue of all
passengers and crew
In his own words …
“One way of looking at this might be that for 42 years, I've been making small, regular deposits in this
bank of experience, education and training.
And on January 15, 2009 the balance was sufficient so that I could make a very large withdrawal.”
10
