Summary Onderzoeksmethoden P&O

Onderzoeksmethoden P&O
Samenvatting




Academiejaar 2019-2020
Prof. Greet Van Hoye


EP




1

,Inhoud
Introduction to research HRM & OB evidence-based ........................................................................................... 6
Objectives .......................................................................................................................................................... 6
Final competences............................................................................................................................................. 6
Approach ........................................................................................................................................................... 6
Evaluation .......................................................................................................................................................... 7
Application HR & OB research ........................................................................................................................... 7
Evidence-based management (EBM) ................................................................................................................ 8
History ............................................................................................................................................................... 8
History evidence-based practice ....................................................................................................................... 8
Threats/opportunities EBM? ............................................................................................................................. 9
Evidence-based practice.................................................................................................................................... 9
Evidence-based decision-making ...................................................................................................................... 9
Scientific evidence ........................................................................................................................................... 10
Organizational evidence .................................................................................................................................. 10
Experiential evidence ...................................................................................................................................... 10
Stakeholder evidence ...................................................................................................................................... 10
In his own words … .......................................................................................................................................... 10
Organizational evidence .................................................................................................................................. 11
Experiential evidence ...................................................................................................................................... 11
Stakeholder evidence ...................................................................................................................................... 11
Exercise............................................................................................................................................................ 11
But … best available evidence ......................................................................................................................... 12
Importance of critical appraisal....................................................................................................................... 12
Examples critical appraisal .............................................................................................................................. 12
Best available evidence ................................................................................................................................... 13
What is the scientific evidence for evidence-based management? ............................................................... 13
Common misconceptions ................................................................................................................................ 14
Becoming an evidence-based HR practitioner ................................................................................................ 14
Evidence-based practice.................................................................................................................................. 14
Searching for scientific evidence ..................................................................................................................... 15
Scientific evidence: Practitioner access .......................................................................................................... 15
Research topics and research questions in HRM and OB ................................................................................... 16
Evidence-based practice.................................................................................................................................. 16

2

, Master’s dissertation: topic choice ................................................................................................................. 16
Topic choice..................................................................................................................................................... 17
Bridging the S-p gap in topic choice ................................................................................................................ 18
Determining your research question .............................................................................................................. 18
Consensus shifting ........................................................................................................................................... 19
Consensus shifting: example ........................................................................................................................... 19
Consensus creation ......................................................................................................................................... 19
Consensus creation: example.......................................................................................................................... 20
Reformulate in consensus terms..................................................................................................................... 20
The introduction: setting the hook ................................................................................................................. 21
The introduction .............................................................................................................................................. 21
Who cares? .................................................................................................................................................. 21
What do we know, what don’t we know, and so what? ............................................................................. 22
What will we learn?..................................................................................................................................... 22
The introduction: Process ............................................................................................................................... 22
Grounding hypotheses .................................................................................................................................... 23
Grounding hypotheses: example .................................................................................................................... 23
Formulating hypotheses.................................................................................................................................. 24
Introduction carless & Hetherington (2011) ................................................................................................... 24
Design and critical evaluation of research in HRM & OB .................................................................................... 26
Evidence-based practice.................................................................................................................................. 26
Critical appraisal of evidence .......................................................................................................................... 26
What is the best available evidence? .............................................................................................................. 26
Internal validity ............................................................................................................................................... 27
Methodological pitfalls.................................................................................................................................... 27
Correlation does not equal causation! ............................................................................................................ 28
Reverse causation ........................................................................................................................................... 28
Causality .......................................................................................................................................................... 28
Methodological pitfalls.................................................................................................................................... 28
Applied to hrm and ob .................................................................................................................................... 29
Construct validity ............................................................................................................................................ 30
Evidence for Construct validity ....................................................................................................................... 30
Statistical conclusion validity .......................................................................................................................... 31
External validity ............................................................................................................................................... 31

3

, Examples ......................................................................................................................................................... 31
Common method bias ..................................................................................................................................... 32
Procedural remedies ....................................................................................................................................... 33
Statistical remedies ......................................................................................................................................... 33
CRAFTING THE METHOD section: 3C’s ............................................................................................................ 34
Amj major flaws research design .................................................................................................................... 34
Design & method ng & burke (2010) .............................................................................................................. 35
Quantitative research in HRM & OB ................................................................................................................... 37
Quantitative research...................................................................................................................................... 37
Quantitative research questions ..................................................................................................................... 37
Experiments..................................................................................................................................................... 37
Surveys ............................................................................................................................................................ 38
Administration survey ..................................................................................................................................... 39
Designing a survey........................................................................................................................................... 39
Developing a measure ..................................................................................................................................... 40
Writing items ................................................................................................................................................... 40
Administration survey ..................................................................................................................................... 41
Increasing response rate ................................................................................................................................. 41
Moderation and mediation ............................................................................................................................. 41
Moderation ..................................................................................................................................................... 42
Mediation ........................................................................................................................................................ 42
Moderator or mediator ................................................................................................................................... 43
Testing moderation ......................................................................................................................................... 43
Testing mediation............................................................................................................................................ 44
Testing Moderation and mediation ................................................................................................................ 45
Carless & Hetherington (2011) ........................................................................................................................ 45
Qualitative research in HRM & OB ...................................................................................................................... 47
Research questions ......................................................................................................................................... 47
Data collection and processing ....................................................................................................................... 48
Reporting ......................................................................................................................................................... 49
Similarities ....................................................................................................................................................... 49
Qualitative research: Don’ts (Pratt, AMJ, 2009) ............................................................................................. 49
Qualitative research: Do’s (Pratt, AMJ, 2009) ................................................................................................. 50
......................................................................................................................................................................... 51

4

, The interview................................................................................................................................................... 51
Type of interviews ........................................................................................................................................... 52
Interview Guide ............................................................................................................................................... 52
Example Interview Guide ................................................................................................................................ 52
Taking the interview ........................................................................................................................................ 53
Attention! ........................................................................................................................................................ 53
Analysis interviews .......................................................................................................................................... 54
Supporting software ........................................................................................................................................ 55
Article qualitative study (Bouwhuis et al., 2018) ............................................................................................ 55
......................................................................................................................................................................... 56
Reporting research in HRM & OB ........................................................................................................................ 57
Results – Quantitative research ...................................................................................................................... 57
Results: exercises ............................................................................................................................................ 58
Discussion ........................................................................................................................................................ 60
Discussion: common mistakes (Geletkanycz & Tepper, 2012)........................................................................ 61
Discussion: research ........................................................................................................................................ 62
Remember? ..................................................................................................................................................... 63
Limitations: which type of validity? ................................................................................................................ 64
Future research: which type of validity? ......................................................................................................... 64
Integrative practice seminar ............................................................................................................................... 65
Final competences........................................................................................................................................... 65
Evaluation ........................................................................................................................................................ 65
Questions ........................................................................................................................................................ 66
Examenvragen januari 2018 Onderzoeksmethoden – P&O............................................................................ 70




5

,Research Methods in Human
Resource Management and
Organizational Behavior
Introduction to research HRM & OB evidence-based

Objectives

 To acquire advanced knowledge, understanding, and skills on
 social-scientific research methods
 Domain of HRM and OB
 Principles of evidence-based HRM
 To be able to judge the design and results of HRM & OB research with respect to quality and
usefulness
 For master’s dissertation and work field



Final competences

 Understand the need for and the value of evidence-based HRM
 Critically evaluate the quality of research in HRM & OB
 Critically evaluate the usefulness of research in HRM & OB
 Translate problems and challenges in the work field of HRM & OB into focused research questions
 Address problems and challenges in the work field of HRM & OB on the basis of scientific research

Approach

 Focus on understanding and application
 Requires preparation, active participation, and exercise
 All class questions/exercises = Example exam questions
 Integrative practice seminar = Practice exam
 Slides on Ufora before class
 If preparation is needed, this needs to be done before unlocking the slides

6

,Evaluation

 Periodic evaluation (100%)
 Written exam with open questions
 Slides + Reader on Ufora
▪ Reader provides more background and explanation to the slides
 Focus on understanding and application
 So NOT: “Give the definition of validity”
 BUT: “Evaluate the validity of this research design” or “Propose a valid design to answer this
research question”
 Periodic evaluation (100%)
 Active participation is the best preparation
 Most questions will be about a (new) scientific HRM & OB article
 You can bring a dictionary English/native language (no notes!)
 Example questions = all class questions/exercises
▪ Tip: Practice on other articles
 No specific questions on SPSS
▪ But be able to interpret results tables from articles (e.g., descriptives, correlations,
regression, mediation, moderation)

Application HR & OB research

 Large scientific knowledge on the effectiveness of many HR & OB practices (supported by multiple
studies or meta-analyses)
 But is this known and applied in the field?
 Large survey on knowledge HR & OB practices
 US: N = 959 HR managers (Rynes et al., 2002, HRM)
 NL: N = 626 HR professionals (Sanders et al., 2008, IJHRM)




7

, Large Scientist-Practitioner gap
 Results HR & OB research are not enough known and applied in the work field
 Better research knowledge was predicted by
 Job/educational level
 Professional certification
 Positive attitude to usefulness academic research
 Academic reading
 Better research knowledge would allow HR professionals to make better and more successful
decisions
 Need for evidence-based management

Evidence-based management (EBM)

“The systematic, evidence-informed practice of management, incorporating scientific knowledge in
the content and process of making decisions”
“Knowing how to obtain and use scientific evidence and reliable business knowledge helps
practitioners respond effectively to the uncertainty they face everyday in organizations”


History

 “there is a large research-user gap”
 “practitioners do not read academic journals”
 “the findings of research into what is an effective intervention are not being translated into actual
practice”
 “academics not practitioners are driving the research agenda”
 “practice is being driven more by fads and fashions than research”


History evidence-based practice

1991: medicine
1998: education
2000: nursing
2005-2006: management




8

,Threats/opportunities EBM?

- Information overload
- Overestimate individual experience (N=1)
- Inaccurate information/beliefs
- Preference for quick solutions
- Hypes and trends in management
- Bias and errors in decision-making
Eg., attribution bias, confirmation bias
+ Call for more accountability
+ Evolution to knowledge management (eg., CIO)
+ Higher accessibility of information (eg., Internet, MIS)
+ Need for more rational and objective decision-making

Notes: A lot info and not much time. Also look at other evidence then just your personal experience…
Intelligence is not evidence based.
Attribution bias: if it is wrong, it’s not your fault. You search some other but when it is right it’s because of
you.
More accountability. Collecting more info.

Evidence-based practice

 is about making decisions through the conscientious, explicit and thoughtful use of the best available
evidence from multiple sources by
 Asking: translating a practical issue or problem into an answerable question? Become aware
of the decisions you make
 Acquiring: systematically searching for and retrieving the evidence
 Appraising: critically judging the trustworthiness and relevance of the evidence
 Aggregating: weighing and pulling together the evidence
 Applying: incorporating the evidence into the decision-making process
 Assessing: evaluating the outcome of the decision taken, Use it for imput for later decisions


to increase the likelihood of a favorable outcome.


Evidence-based decision-making

Notes: More and more info is available in the organization.
Not intuition, but based on experience, try to rationalize how
can I help the stakeholder and how the will be effected.




9

, Scientific evidence

Chesley Burnett "Sully" Sullenberger III (Danville (Californië), 23
januari 1951) is een Amerikaans piloot in de burgerluchtvaart,
veiligheidsexpert en ongevallenanalist. Hij werd vooral bekend als
de gezagvoerder die de noodlanding van US Airways-vlucht 1549 op
de rivier de Hudson in New York uitvoerde op 15 januari 2009.




 Chesley Sullenberger, USAIR pilot, has been a visiting scholar at UC Berkeley’s Collaborative for
Catastrophic Risk Management since 2007
 Does research on how to make decisions to maintain safety despite technological complexity and crisis
conditions

Organizational evidence

 Has written and analyzed aviation accident reports for over 20 years


Experiential evidence

 Formal Education to Prime His Skills: Sully is a graduate of the U.S. Air Force Academy and holds
masters degrees from both Purdue University in Industrial Psychology and the University of Northern
Colorado in Public Administration
 Used Decision Aids to Support Good Decision: As Sully considered what decision to make that day, he
had his copilot review and follow all checklists on board relevant to crash landings


Stakeholder evidence

 The last person to leave the plane, Chesley Sullenberger twice walked the plane’s aisle to check all
passengers were off
 Sully’s last act onboard was to grab the passenger list. Used on-shore to verify rescue of all
passengers and crew


In his own words …

“One way of looking at this might be that for 42 years, I've been making small, regular deposits in this
bank of experience, education and training.
And on January 15, 2009 the balance was sufficient so that I could make a very large withdrawal.”




10