1
Samenvatting boek ‘Real World Research’ Methodologie 3
Chapter 6 Fixed Designs
Introduction: zowel kwantitatief als kwalitatief. Fixed designs are theory driven. We must already
have a substantial amount of conceptual understanding about a phenomenon before it is worthwhile
following the risky strategy of investing precious time and resources in such designs. You should also
know what kind of results you are going to get, and how you will analyse them, before you collect the
data. Even with a preceding exploratory phase, fixed designs should always be piloted. You carry out
a mini-version of the study before committing yourself to the big one. In the ‘true’ experiment, two
or more groups are set up, with random allocation of people to the groups. The experimenter then
actively manipulates the situation so that different groups get different treatments. Single-case
design, focuses on individuals rather than groups and effectively seeks to use persons as their own
control, with their being subjected to different experimentally manipulated conditions at different
times. Quasi-experiments lack the random allocation to different conditions found in true
experiments. Non-experimental fixed designs do not involve active manipulation of the situation by
the researcher.
General features of fixed designs: fixed designs are usually concerned with aggregates: with group
properties and with general tendencies. In traditional experiments, results are reported in terms of
group averages rather than what individuals have done. Because of this, there is a danger of the
ecological fallacy – that is, of assuming that inferences can be made about individuals from such
aggregate data. The relative weakness of fixed designs is that they cannot capture the subtleties and
complexities of individual human behaviour. For that you need flexible designs. The advantage of
fixed designs is in being able to transcend individual differences and identify patterns and processes
which can be linked to social structures and group, or organizational, features. Researchers typically
remain at a greater physical and emotional distance from the study than those using flexible designs.
In experimental research the experimenter effect is well known. It is now widely acknowledged that
the beliefs, values and expectations of the researcher can influence the research process. There are
often long periods of preparation and design preliminaries before data collection and a substantial
period of analysis after data collection.
Establishing trustworthiness in fixed design research: validity refers to the accuracy of a result.
Generalizability refers to the extent to which the findings of the research are more generally
applicable, for example in other contexts, situations or times, or to persons other than those directly
involved.
Validity: is there a real, direct link between the two things? Reliability: this is the stability or
consistency with which we measure something. Unless a measure is reliable, it cannot be valid. While
reliability is necessary, it is not sufficient. Unreliability may have various causes, including:
- Participant error: the pupil’s performance might fluctuate widely from occasion to occasion
on a more or less random basis.
- Participant bias
- Observer error
- Observer bias: causes problems in interpretation.
- Types of validity: does it measures what you think it measures? construct validity Face
validity and Predictive criterion validity. Any one way of measuring or gathering data is likely
to have its shortcomings, which suggests the use of multiple methods of data collection. One
could use all three of the approaches to assessing educational achievement discussed above
Samenvatting boek ‘Real World Research’ Methodologie 3
Chapter 6 Fixed Designs
Introduction: zowel kwantitatief als kwalitatief. Fixed designs are theory driven. We must already
have a substantial amount of conceptual understanding about a phenomenon before it is worthwhile
following the risky strategy of investing precious time and resources in such designs. You should also
know what kind of results you are going to get, and how you will analyse them, before you collect the
data. Even with a preceding exploratory phase, fixed designs should always be piloted. You carry out
a mini-version of the study before committing yourself to the big one. In the ‘true’ experiment, two
or more groups are set up, with random allocation of people to the groups. The experimenter then
actively manipulates the situation so that different groups get different treatments. Single-case
design, focuses on individuals rather than groups and effectively seeks to use persons as their own
control, with their being subjected to different experimentally manipulated conditions at different
times. Quasi-experiments lack the random allocation to different conditions found in true
experiments. Non-experimental fixed designs do not involve active manipulation of the situation by
the researcher.
General features of fixed designs: fixed designs are usually concerned with aggregates: with group
properties and with general tendencies. In traditional experiments, results are reported in terms of
group averages rather than what individuals have done. Because of this, there is a danger of the
ecological fallacy – that is, of assuming that inferences can be made about individuals from such
aggregate data. The relative weakness of fixed designs is that they cannot capture the subtleties and
complexities of individual human behaviour. For that you need flexible designs. The advantage of
fixed designs is in being able to transcend individual differences and identify patterns and processes
which can be linked to social structures and group, or organizational, features. Researchers typically
remain at a greater physical and emotional distance from the study than those using flexible designs.
In experimental research the experimenter effect is well known. It is now widely acknowledged that
the beliefs, values and expectations of the researcher can influence the research process. There are
often long periods of preparation and design preliminaries before data collection and a substantial
period of analysis after data collection.
Establishing trustworthiness in fixed design research: validity refers to the accuracy of a result.
Generalizability refers to the extent to which the findings of the research are more generally
applicable, for example in other contexts, situations or times, or to persons other than those directly
involved.
Validity: is there a real, direct link between the two things? Reliability: this is the stability or
consistency with which we measure something. Unless a measure is reliable, it cannot be valid. While
reliability is necessary, it is not sufficient. Unreliability may have various causes, including:
- Participant error: the pupil’s performance might fluctuate widely from occasion to occasion
on a more or less random basis.
- Participant bias
- Observer error
- Observer bias: causes problems in interpretation.
- Types of validity: does it measures what you think it measures? construct validity Face
validity and Predictive criterion validity. Any one way of measuring or gathering data is likely
to have its shortcomings, which suggests the use of multiple methods of data collection. One
could use all three of the approaches to assessing educational achievement discussed above
