LECTURE 1 - CHAPTER 7 & CHAPTER 5A COURSE BOOK: THE RESEARCH PROCESS
5 steps in scientific research:
1. Problem
2. Research question: fact oriented, information-gathering question meaning question that can be
answered using scientific methods
a. Derive the sub-research questions
3. Literature study/search, expert insight: key factors are identified
4. Build a conceptual model to describe which relations you want to test in the research design.
It clearly defines the boundaries of a study, it depicts what you will focus on, but excludes all
other topics at the same time
a. Concept Y: dependent variable that we want to change i.e. response
b. Concept X1: independent variable - factors that influence concept Y
c. Concept X2: independent variable - factors that influence concept Y
i. There is a wide range concept Xn to choose from
ii. We expect to be a casual relation between concept Y and X1 and X2
d. Take control variables into account, to make sure variables such as age, gender, and
education level do not influence the central concept
5. Propositions/hypotheses: research questions formulated in a clearly directional way
a. Arrows in the model represent hypothesis - expectation of how (+/-) and in which
direction the IV influences the DV
2 additional steps specific to survey research:
6. Indicators
From conceptual model to definitions which allow you to measure something
- Conceptual definition is usually the theoretical one but it is not the one which is the most suited
- Scientific definition, often based on scientific literature
Performance: the performance of an athlete that participates at least once a month in a particular event
From conceptual definition to operational definition
- Operational definition is a more applied/practical/concrete version of the conceptual definition
- Look at particular factors within situation that you want to study
- Specification of a conceptual definition in a particular context to make the concept
measurable
Performance: number of points scored in an official game or the average improvement per month
measured in seconds
From definitions to indicators
- Link definitions to indicators that describe all main aspects of operational definitions and which
need to be measured to be certain that you are measuring the concept you are interested in
- If a combination of indicators covers all aspects of a concept, it can be said to be high in content
validity, but construct validity is also important that is, do the indicators fit with existing theory
about the concept?
Indicators of training: (1) number of hours a week that was spend in the gym, (2) number of hours spent
on endurance training
, 7. Questions
From indicators to questions
- Indicators are linked to questions to make sure you are covering the entire topic
- Indicators are aspects of the concept being studied
Once all this information is gathered → start the sampling and gathering data process
Sampling: how do you select your participants?
Gather data: sending out the surveys, reminders…
Once you’ve collected the data → SPSS and analyzing results (read SPSS guide)
- With the data you combine the answers to the questions of each indicator to have a sum
variable of each concept (although you can’t always make sum variables)
- Use statistical analysis to test hypothesis; relationship between X and Y variable
From results to conclusions
- Depending on the results of statistical analysis, hypothesis will be supported or rejected
Experiences, opinions, and behaviors of individual respondents → survey
Experiences, opinions, and behaviors of others → measured at the group level, aggregated data so not
suitable for interviews with individual employees as they result in unreliable data, these cannot be
compared against individual data
CHAPTER 7: PRIMARY DATA COLLECTION WITH SURVEYS
CHARACTERISTICS OF DATA COLLECTION
Approaches to acquiring data can be classified along three dimensions:
1. First dimensions distinguishes between whether researchers collect data specifically for a
current study; primary data or whether they make use of already existing data; secondary data
2. Second dimension describes what kind of data are collected: quantitative or qualitative
3. Third dimension refers to the method of data collection:
a. Communication approach: researcher asks questions either personally or indirectly
through surveys
b. Observation approach: researchers observe certain phenomena or events and record
what they see
i. Qualitative studies use participant observation
ii. Quantitative studies refer to structured observation
, LECTURE 2: CHAPTER 14 - VALIDITY, RELIABILITY, AND DATA TYPES
MEASUREMENT ERROR - potential sources of error:
● Mesurer (researcher)
● Participant
● Instrument
The measurement error is the difference between the observed value and the true value
Observed value = true value + measurement error
Measurement error can be:
1. Systematic error: always in the same direction - worse
2. Random error: both directions, equals each other out (+-)
VALIDITY - “Do I measure what I want to measure?”
● Various categorizations of types of validity - two main broad ones:
○ Internal validity: ability of a research instrument to measure what you want to measure
○ External validity: extent to which findings can meaningfully be generalized to other
settings, situations and/or persons
Types of internal validity: (measuring one is usually not enough)
● Content validity: extent to which the measuring instrument provides adequate coverage of the
concept you study
○ Disadvantage: requires clear idea about what constitutes your concept
● Face validity: “at first sight” a good translation of the construct
○ Weak evidence on its own; simply “looks about right”
○ Better to select groups of experts to give their opinion
● Criterion-related validity
○ Predictive validity: construct A (now) causes construct B (future)
■ Indicator A is associated with indicator B (=criterion)
■ Possible to predict future (before B is measured)
■ Measured at different moments: 2019 assessment test and 2021 job
performance
■ You need to have clear theoretical prediction of which factors should be linked
■
○ Concurrent validity: construct A (now) causes construct B (now)
■ Indicator A is associated with indicator B (=criterion)
5 steps in scientific research:
1. Problem
2. Research question: fact oriented, information-gathering question meaning question that can be
answered using scientific methods
a. Derive the sub-research questions
3. Literature study/search, expert insight: key factors are identified
4. Build a conceptual model to describe which relations you want to test in the research design.
It clearly defines the boundaries of a study, it depicts what you will focus on, but excludes all
other topics at the same time
a. Concept Y: dependent variable that we want to change i.e. response
b. Concept X1: independent variable - factors that influence concept Y
c. Concept X2: independent variable - factors that influence concept Y
i. There is a wide range concept Xn to choose from
ii. We expect to be a casual relation between concept Y and X1 and X2
d. Take control variables into account, to make sure variables such as age, gender, and
education level do not influence the central concept
5. Propositions/hypotheses: research questions formulated in a clearly directional way
a. Arrows in the model represent hypothesis - expectation of how (+/-) and in which
direction the IV influences the DV
2 additional steps specific to survey research:
6. Indicators
From conceptual model to definitions which allow you to measure something
- Conceptual definition is usually the theoretical one but it is not the one which is the most suited
- Scientific definition, often based on scientific literature
Performance: the performance of an athlete that participates at least once a month in a particular event
From conceptual definition to operational definition
- Operational definition is a more applied/practical/concrete version of the conceptual definition
- Look at particular factors within situation that you want to study
- Specification of a conceptual definition in a particular context to make the concept
measurable
Performance: number of points scored in an official game or the average improvement per month
measured in seconds
From definitions to indicators
- Link definitions to indicators that describe all main aspects of operational definitions and which
need to be measured to be certain that you are measuring the concept you are interested in
- If a combination of indicators covers all aspects of a concept, it can be said to be high in content
validity, but construct validity is also important that is, do the indicators fit with existing theory
about the concept?
Indicators of training: (1) number of hours a week that was spend in the gym, (2) number of hours spent
on endurance training
, 7. Questions
From indicators to questions
- Indicators are linked to questions to make sure you are covering the entire topic
- Indicators are aspects of the concept being studied
Once all this information is gathered → start the sampling and gathering data process
Sampling: how do you select your participants?
Gather data: sending out the surveys, reminders…
Once you’ve collected the data → SPSS and analyzing results (read SPSS guide)
- With the data you combine the answers to the questions of each indicator to have a sum
variable of each concept (although you can’t always make sum variables)
- Use statistical analysis to test hypothesis; relationship between X and Y variable
From results to conclusions
- Depending on the results of statistical analysis, hypothesis will be supported or rejected
Experiences, opinions, and behaviors of individual respondents → survey
Experiences, opinions, and behaviors of others → measured at the group level, aggregated data so not
suitable for interviews with individual employees as they result in unreliable data, these cannot be
compared against individual data
CHAPTER 7: PRIMARY DATA COLLECTION WITH SURVEYS
CHARACTERISTICS OF DATA COLLECTION
Approaches to acquiring data can be classified along three dimensions:
1. First dimensions distinguishes between whether researchers collect data specifically for a
current study; primary data or whether they make use of already existing data; secondary data
2. Second dimension describes what kind of data are collected: quantitative or qualitative
3. Third dimension refers to the method of data collection:
a. Communication approach: researcher asks questions either personally or indirectly
through surveys
b. Observation approach: researchers observe certain phenomena or events and record
what they see
i. Qualitative studies use participant observation
ii. Quantitative studies refer to structured observation
, LECTURE 2: CHAPTER 14 - VALIDITY, RELIABILITY, AND DATA TYPES
MEASUREMENT ERROR - potential sources of error:
● Mesurer (researcher)
● Participant
● Instrument
The measurement error is the difference between the observed value and the true value
Observed value = true value + measurement error
Measurement error can be:
1. Systematic error: always in the same direction - worse
2. Random error: both directions, equals each other out (+-)
VALIDITY - “Do I measure what I want to measure?”
● Various categorizations of types of validity - two main broad ones:
○ Internal validity: ability of a research instrument to measure what you want to measure
○ External validity: extent to which findings can meaningfully be generalized to other
settings, situations and/or persons
Types of internal validity: (measuring one is usually not enough)
● Content validity: extent to which the measuring instrument provides adequate coverage of the
concept you study
○ Disadvantage: requires clear idea about what constitutes your concept
● Face validity: “at first sight” a good translation of the construct
○ Weak evidence on its own; simply “looks about right”
○ Better to select groups of experts to give their opinion
● Criterion-related validity
○ Predictive validity: construct A (now) causes construct B (future)
■ Indicator A is associated with indicator B (=criterion)
■ Possible to predict future (before B is measured)
■ Measured at different moments: 2019 assessment test and 2021 job
performance
■ You need to have clear theoretical prediction of which factors should be linked
■
○ Concurrent validity: construct A (now) causes construct B (now)
■ Indicator A is associated with indicator B (=criterion)
