Qualitative research methods 4
- Richie & Lewis H7 (focusgroups)
- Gelissen H1 (observation and observer)
Aantekening + sheets: controlling and experiment design
2 ways of controlling results:
1. By statistical control: Measure confounding variables, then include as control variables in
statistical analysis (but: no measurement of confounding variables means no control!)
This way is not used by qualitative research.
Confounding variables: variables that may cause spuriousness.
2. By research design: Make systematic comparisons between groups that are as identical as
possible, except for the characteristic that presumably has an effect.
Systematic comparison is very important for qualitative research. No comparison, no
conclusion.
Dimensions of research designs
1. Number of individuals or groups to be included
2. Number of measurement moments: one time or more times?
3. How are groups constructed: make groups or take existing groups?
4. Intentional manipulation by researcher, natural occurrence, or existing variation in data?
Researchers manipulate (intervention): e.g. give one group medicine, others not.
Different types of experimental design:
1. Experimental design: the best way to find causal inference/effects
- Also called: classical experimental design, randomized experiment, standard design, and true
experiment.
- Laboratory experiment, small setting (one way)
- Field experiment, natural setting (other way)
How does the experimental design work? (Question oral exam):
1. Randomly split group of people in two (or more) groups: experimental and control group.
2. Perform pre-test: measure score of dependent variable. Y
3. Experimental group(s) is manipulated (get intervention), control group is not manipulated. X
4. Perform post-test: measure score of dependent variable. Y
Why randomly (1): groups that we make are identical on all aspects. Due this, there are no
systematically differences between groups. So we can compare them.
Effects in experimental design:
If well executed, the following confounding factors are controlled. Otherwise these threats can cause
the internal (causal) validity.
- History: something happened to the groups between the pre-test and the post-test in
addition of the intervention of the researcher. This influences the dependent variable.
- Testing: the pre-test influences the score of the post-test. If it occurs, it occurs by both but
the groups are randomized so the outcome is still average. Other solution is waiting a long
time between tests so they will forget the previous answers.
- Instrumentation: use different instrumentation by the tests.
- Mortality/attrition: people drop out.
- Maturation: people get older and tired, values and opinions can change.
1
, - Selection bias: if randomization didn’t work correctly, groups cannot be compared. This leads
to systematic differences.
All of the above is under control through randomization.
2. Quasi-experimental design: Almost identical to true experiment, but without random assignment.
- So we can expect systematically differences between the groups.
- The researcher does manipulate the independent variable
- Risk: occurring of selection bias effect because of no randomizing.
Example: no use of random groups but premaster students and regular. The fact is that premaster
have more motivation so they score an 8 without a handsome teacher. The regular students are
thought by a handsome teacher and score a 9. There are a lot differences which have influence.
3. Non-experimental design: score lowest on causal inferences.
- No manipulating of the independent variable
- No randomizing
Cross-sectional designs: measurement on one time so you can’t see any development. E.g. one-time
interview. But you can ask about past, today, and future and then you get a small impression of this.
Possible to extend with retrospective elements
Longitudinal designs, repeated cross-sectional design, and trend design: order a random group, so
you can say something about the group, not about the individual.
Cohort design: age group.
Panel design: take random group at first time and follow this people for the rest of the research. Data
collection of the same person on different points in time. High chance of mortality effect.
Case study designs (special case: natural experiment): study few group very intensely.
All the above designs are highly relevant for qualitative research.
Random sampling is not the same as random assignment/randomization (oral question):
- Random sampling is the process of randomly selecting samples from a population of
potential participants.
- Random assignment is the process of randomly allocating individuals to the conditions of an
experiment. Chance to come in a control group or experimental group. With the result that
there are no systematic differences.
After choice of research design, choice of sampling strategy: many different procedures
2 methods for quantitative research:
Probability sampling: generalize from sample to population.
- To do that we need a sample frame: list of all units of analysis in population.
- Every unit of analysis has a known probability of being selected so: probability sample.
Simple random sample: based on chance, most basic strategy in quantitative research.
- Every unit has the same known probability of becoming selected.
- Sample fraction n/N, with n=sample size and N=population size.
- List all units and randomly sample, e.g. 30%
Sample: A sample frame is 3-4 times larger as the effective sample that we are looking for. So you
secure attrition/mortality.
Household screen: small survey that you do in order to determine who is in the selection.
2
- Richie & Lewis H7 (focusgroups)
- Gelissen H1 (observation and observer)
Aantekening + sheets: controlling and experiment design
2 ways of controlling results:
1. By statistical control: Measure confounding variables, then include as control variables in
statistical analysis (but: no measurement of confounding variables means no control!)
This way is not used by qualitative research.
Confounding variables: variables that may cause spuriousness.
2. By research design: Make systematic comparisons between groups that are as identical as
possible, except for the characteristic that presumably has an effect.
Systematic comparison is very important for qualitative research. No comparison, no
conclusion.
Dimensions of research designs
1. Number of individuals or groups to be included
2. Number of measurement moments: one time or more times?
3. How are groups constructed: make groups or take existing groups?
4. Intentional manipulation by researcher, natural occurrence, or existing variation in data?
Researchers manipulate (intervention): e.g. give one group medicine, others not.
Different types of experimental design:
1. Experimental design: the best way to find causal inference/effects
- Also called: classical experimental design, randomized experiment, standard design, and true
experiment.
- Laboratory experiment, small setting (one way)
- Field experiment, natural setting (other way)
How does the experimental design work? (Question oral exam):
1. Randomly split group of people in two (or more) groups: experimental and control group.
2. Perform pre-test: measure score of dependent variable. Y
3. Experimental group(s) is manipulated (get intervention), control group is not manipulated. X
4. Perform post-test: measure score of dependent variable. Y
Why randomly (1): groups that we make are identical on all aspects. Due this, there are no
systematically differences between groups. So we can compare them.
Effects in experimental design:
If well executed, the following confounding factors are controlled. Otherwise these threats can cause
the internal (causal) validity.
- History: something happened to the groups between the pre-test and the post-test in
addition of the intervention of the researcher. This influences the dependent variable.
- Testing: the pre-test influences the score of the post-test. If it occurs, it occurs by both but
the groups are randomized so the outcome is still average. Other solution is waiting a long
time between tests so they will forget the previous answers.
- Instrumentation: use different instrumentation by the tests.
- Mortality/attrition: people drop out.
- Maturation: people get older and tired, values and opinions can change.
1
, - Selection bias: if randomization didn’t work correctly, groups cannot be compared. This leads
to systematic differences.
All of the above is under control through randomization.
2. Quasi-experimental design: Almost identical to true experiment, but without random assignment.
- So we can expect systematically differences between the groups.
- The researcher does manipulate the independent variable
- Risk: occurring of selection bias effect because of no randomizing.
Example: no use of random groups but premaster students and regular. The fact is that premaster
have more motivation so they score an 8 without a handsome teacher. The regular students are
thought by a handsome teacher and score a 9. There are a lot differences which have influence.
3. Non-experimental design: score lowest on causal inferences.
- No manipulating of the independent variable
- No randomizing
Cross-sectional designs: measurement on one time so you can’t see any development. E.g. one-time
interview. But you can ask about past, today, and future and then you get a small impression of this.
Possible to extend with retrospective elements
Longitudinal designs, repeated cross-sectional design, and trend design: order a random group, so
you can say something about the group, not about the individual.
Cohort design: age group.
Panel design: take random group at first time and follow this people for the rest of the research. Data
collection of the same person on different points in time. High chance of mortality effect.
Case study designs (special case: natural experiment): study few group very intensely.
All the above designs are highly relevant for qualitative research.
Random sampling is not the same as random assignment/randomization (oral question):
- Random sampling is the process of randomly selecting samples from a population of
potential participants.
- Random assignment is the process of randomly allocating individuals to the conditions of an
experiment. Chance to come in a control group or experimental group. With the result that
there are no systematic differences.
After choice of research design, choice of sampling strategy: many different procedures
2 methods for quantitative research:
Probability sampling: generalize from sample to population.
- To do that we need a sample frame: list of all units of analysis in population.
- Every unit of analysis has a known probability of being selected so: probability sample.
Simple random sample: based on chance, most basic strategy in quantitative research.
- Every unit has the same known probability of becoming selected.
- Sample fraction n/N, with n=sample size and N=population size.
- List all units and randomly sample, e.g. 30%
Sample: A sample frame is 3-4 times larger as the effective sample that we are looking for. So you
secure attrition/mortality.
Household screen: small survey that you do in order to determine who is in the selection.
2
