Non-experimental vs Quasi-Experimental designs
Non-experimental designs: make little or no attempt to control threats to
internal validity (e.g., differential research design, post-test only non-
equivalent control group design, pretest-post-test design)
Quasi-experimental designs: attempt to limit threats to internal validity (e.g.,
pretest-posttest non-equivalent control group design, time-series design)
How can we be sure that our interventions/policy change has brought about the
changes we see in our participants?
Sometimes we start with a non-experimental design where we have one group of
participants. We measure their DV levels (e.g., well-being) before the intervention or
program (e.g., well-being intervention) and measure the DV again after the
intervention.
This is a simple pre-post design that falls under a non-experimental research
strategy. So, we may start with this design and modify it by adding a comparison
group - or a set of participants who will not participate in our intervention and
measure these participants simultaneously as our intervention group. By simply
adding this comparison group, we will be reducing some of the threats to internal
validity.
We have between-subjects designs and within-subjects designs.
Between-subjects design: participants are assigned to different groups or conditions, such as
Treatment A and Treatment B. Treatment A can be an experimental group and Treatment B
a control group.
Within-subjects design: the same participants are exposed to conditions or treatments (e.g.,
Treatment A and Treatment B).
NON-EQUIVALENT CONTROL GROUPS (PART 1)
We randomly assign participants to either an intervention/treatment group or a control
group in experimental designs.
,By doing a random assignment, we are trying to match participants or equalise our groups.
We know that in non-experimental or quasi-experimental designs, we do not randomly
assign participants to different conditions. That means that our two groups are not exactly
the same (i.e., individual differences).
Non-equivalent comparison groups play a vital role in our quasi-experimental designs. They
may help us reduce some of the threats to internal validity.
Example:
• Study Title: Teaching Emotional Intelligence: An Academic Course for Hospital
Teachers
• Emotional Intelligence: EI has been defined by Goleman (1998, p. 317) as: ‘‘the
capacity for recognising our own feelings and those of others, for motivating
ourselves, and for managing emotions well in ourselves and in our relationships’’.
Before the intervention 10 weeks after the intervention
In this study, they did a pre-test: Teachers were asked to take the Emotional Intelligence
Questionnaire before the intervention
The same participants were asked to take the EIQuestionnaire after the intervention
(second observation)
O1- Initial observation
X = intervention/treatment
O2 = second observation after the training programme
This is an example of a non-experimental pre-post design where the same participants
undergo the same treatment or intervention (X) and are tested before and after the
intervention.
Results
The results of the study were quite positive. They saw an increase in total EQ and all the
dimensions of Emotional Intelligence (i.e., appraisal and expression of emotion/ regulation
of emotion/utilisation of emotion).
They were quite positive that after the intervention, teachers could identify and express
their emotions positively; control their emotions, and use their emotions in positive ways.
, All these changes from pre-intervention to post-intervention were significant, even though
the difference was not too massive.
What is the problem with this design in this specific study? What are some of the possible
threats to internal validity?
Threats to internal validity
• History - environmental events unrelated to the intervention that may affect the
outcomes
Pre-post designs are vulnerable to threats to internal validity.
The differences we observed in this study between the pre-test and the post-test could be
explained by many factors, such as history.
Other threats: environmental events unrelated to the intervention, such as education policy
change or any other events that may affect the outcome in this study.
How can we try to eliminate this threat?
Answer: One way is by including a non-equivalent control/comparison group
Non-Equivalent Comparison Groups
In Hen’s (2020) study, we can include another group of teachers working in a different
hospital who will not be exposed to our Emotional Intelligence intervention. By having a
non-equivalent comparison group, we can be able to rule out the effect of history.
This graph shows that both the intervention and the comparison group are improving over
time. So, this idea that the comparison group or the non-equivalent control or comparison
group will also have whatever is affecting the intervention group and that is also affecting
the dependent variable.
Non-experimental designs: make little or no attempt to control threats to
internal validity (e.g., differential research design, post-test only non-
equivalent control group design, pretest-post-test design)
Quasi-experimental designs: attempt to limit threats to internal validity (e.g.,
pretest-posttest non-equivalent control group design, time-series design)
How can we be sure that our interventions/policy change has brought about the
changes we see in our participants?
Sometimes we start with a non-experimental design where we have one group of
participants. We measure their DV levels (e.g., well-being) before the intervention or
program (e.g., well-being intervention) and measure the DV again after the
intervention.
This is a simple pre-post design that falls under a non-experimental research
strategy. So, we may start with this design and modify it by adding a comparison
group - or a set of participants who will not participate in our intervention and
measure these participants simultaneously as our intervention group. By simply
adding this comparison group, we will be reducing some of the threats to internal
validity.
We have between-subjects designs and within-subjects designs.
Between-subjects design: participants are assigned to different groups or conditions, such as
Treatment A and Treatment B. Treatment A can be an experimental group and Treatment B
a control group.
Within-subjects design: the same participants are exposed to conditions or treatments (e.g.,
Treatment A and Treatment B).
NON-EQUIVALENT CONTROL GROUPS (PART 1)
We randomly assign participants to either an intervention/treatment group or a control
group in experimental designs.
,By doing a random assignment, we are trying to match participants or equalise our groups.
We know that in non-experimental or quasi-experimental designs, we do not randomly
assign participants to different conditions. That means that our two groups are not exactly
the same (i.e., individual differences).
Non-equivalent comparison groups play a vital role in our quasi-experimental designs. They
may help us reduce some of the threats to internal validity.
Example:
• Study Title: Teaching Emotional Intelligence: An Academic Course for Hospital
Teachers
• Emotional Intelligence: EI has been defined by Goleman (1998, p. 317) as: ‘‘the
capacity for recognising our own feelings and those of others, for motivating
ourselves, and for managing emotions well in ourselves and in our relationships’’.
Before the intervention 10 weeks after the intervention
In this study, they did a pre-test: Teachers were asked to take the Emotional Intelligence
Questionnaire before the intervention
The same participants were asked to take the EIQuestionnaire after the intervention
(second observation)
O1- Initial observation
X = intervention/treatment
O2 = second observation after the training programme
This is an example of a non-experimental pre-post design where the same participants
undergo the same treatment or intervention (X) and are tested before and after the
intervention.
Results
The results of the study were quite positive. They saw an increase in total EQ and all the
dimensions of Emotional Intelligence (i.e., appraisal and expression of emotion/ regulation
of emotion/utilisation of emotion).
They were quite positive that after the intervention, teachers could identify and express
their emotions positively; control their emotions, and use their emotions in positive ways.
, All these changes from pre-intervention to post-intervention were significant, even though
the difference was not too massive.
What is the problem with this design in this specific study? What are some of the possible
threats to internal validity?
Threats to internal validity
• History - environmental events unrelated to the intervention that may affect the
outcomes
Pre-post designs are vulnerable to threats to internal validity.
The differences we observed in this study between the pre-test and the post-test could be
explained by many factors, such as history.
Other threats: environmental events unrelated to the intervention, such as education policy
change or any other events that may affect the outcome in this study.
How can we try to eliminate this threat?
Answer: One way is by including a non-equivalent control/comparison group
Non-Equivalent Comparison Groups
In Hen’s (2020) study, we can include another group of teachers working in a different
hospital who will not be exposed to our Emotional Intelligence intervention. By having a
non-equivalent comparison group, we can be able to rule out the effect of history.
This graph shows that both the intervention and the comparison group are improving over
time. So, this idea that the comparison group or the non-equivalent control or comparison
group will also have whatever is affecting the intervention group and that is also affecting
the dependent variable.
