Research methods and techniques 2
HOC 1: Introduction to scientific reasoning
Overview lecture:
1. Psychology is a way of thinking (chapter 1)
2. Sources of information (chapter 2)
1. Psychology is a way of thinking
Producing versus consuming research
When you work as a psychologist, you are a consumer of
science to help a patient, to look up a therapy,… —>
requires you to think critically about research so you can
use it in an appropriate way
When you run scientific experiments yourself, you are a
producer of research. This could happen when you do a
PHD
• Examples to illustrate the importance of the research consumer role:
o Facilitated communication treatment
➢ for example when someone can’t talk, a health care worker could assist
this person and through a keyboard, this person could select letters by
hand and form words
➢ It might be a good idea to check the reliability of this method before
actually using it on patients —> it actually seems that this method of
communication treatment is not very great, because different people see
different things and communicate different things (which results in the
patient not being able to really express his feelings/thoughts) —> it’s
more the health care workers who influence the communication of the
person
➢ ➔ something that might look good at first glance is not always as great as
it looked when you do research
o Scared-straight approach
➢ This is a method where they try getting people back on the right path by
scaring them ➔ for example, there are shows where young adults are
being sent to prison and treated terribly
➢ After a few days, the young people are so scared that they promise to be a
better person
➢ The effectiveness of this method does not seem to be good when
compared to other kids who were treated with another method. The
group that went through the scared-straight approach, was more likely to
be arrested, in criminal environments,… ➔ didn’t work
1
, o Mindfulness and academic performance
➢ Children who received mindfulness training had better academic
performances than the control group.
➢ It seems that using mindfulness in school results in better academic
performances, so it might be a good idea to integrate mindfulness
• Critical mindset is essential!!!
• Not all published research is correct or robust
A good example of this is the study that was done by ‘Bem’ and led to what is now
called ‘the Replication crisis in psychology’ = he did 9 experiments where he
‘proved’ that there is such a thing as ‘premonition’ (≈ predicting the future)
o Example of 1 experiment: Students were given a list of words (words were shown
one by one), after a certain point he stopped showing the words and the students
had to write down as many words as possible (which they remembered) →
recollection task
o Second part of the experiment was that the students could see some of the
words again (subset) + had to copy them again by writing them down —> strange,
normally it’s the other way around in an experiment. He did this to achieve
training effects!
o ➔ The conclusion was that the words people most likely remembered were the
words from that subset that they were training after writing them down. It’s like
people were already predicting which words they would see next and they
remembered them better!
o How to explain this? —> led to researchers questioning their way of researching
o Problems! ➔ Bem only reported the results of 9 experiments, but he actually ran
hundreds. He only reported experiments where he proved that people could
predict it! He did hundreds of experiments were this was not proven, but he
didn’t report these results ➔ very important to know = shows you have to be
critical!
o Also led to replication crisis = most studies that were done before, could not be
replicated because most experiments were lucky shots.
How scientists work
• Science is based on empiricism
• Scientists test theories
• Scientists work on fundamental and applied problems
• Science is continuously evolving
• Scientists publish their findings in scientific journals
• Scientists communicate with the general public via journalists
Empiricism
• We use an empirical method in scientific research and we use data
• Empirical method is based on data obtained through:
o Our senses (sight, hearing, touch)
o Instruments that assist our senses (thermometer, questionnaires, timer)
• Empiricists aim to do research in a systematic, rigorous, and replicable manner
2
,• Empiricism is not based on own experiences, intuition, or authority
Scientists test theories
We use an empirical approach to test theories. To do so, we use the
empirical research cycle as shown on the left. We start with a theory. From
this theory, we form a few research questions which we will try to resolve in
an experiment. Before we do that, we produce a research design and some
hypotheses. After this, we obtain data and we can accept/change the
theory
Experiment Harry Harlow: there were a few contradictory
theories about the bond between mother and child:
1) This bond exists because the mom feeds the baby
2) This bond exists because the mom provides comfort,
warmth and safety for the baby
➔ They made 2 surrogate moms, one with a hard ‘body’, but with
food and one who was soft and warm, but didn’t have ability to
feed
This experiment led to the ‘contact comfort theory’ —> the soft, warm and comforting
mom was more important for the baby than the feeding mother
• Characteristics of good theories
o Supported by data
o Falsifiable —> must be constructed in such a way that you could create a study
that could disprove a theory. If it is not falsifiable, it is pseudoscience
o Parsimonious (“Occam’s razor”) = if we have 2 explanations for a phenomenon
and they are both qualitative, the easiest explanation will be the one we use
• Examples of non-falsifiable theories:
o Facilitated communication treatment believers
o See book “De ongelovige Thomas heeft een punt”
• Theories are evaluated based on all available evidence
o A theory can never be “proven” (we always say we ‘support’ a theory), but it can
be falsified (when the theory is not correct)
➢ For example = ‘all swans are white’ —> if you go check this in a few lakes
and you only find white swans, you can ‘support’ the theory. This doesn’t
mean that there are ONLY white swans in the whole wide world (because
you did not check them all), so you cannot ‘prove’ the theory. Once you
see 1 black swan, the theory is immediately falsified!
o Replication is crucial!
3
, Scientists form a community
• Merton’s scientific norms:
1. Universalism = everyone, regardless of their background, should be able to do
science —> it’s not who you are that matters, but how you do science!
2. Communality —> you should share the results you became with the scientific
community
3. Disinterestedness —> scientists should pursue research not because they are
interested in money, fame,… but it should be done because people want to find
out something
4. Organized skepticism —> we should have a system that is in place to check
each others work, so that we can evaluate it and see to what extent we can
accept claims
Fundamental VS applied research
• There is a difference between the type of research
1) Fundamental research is being done because we want to advance theoretical
insights, learn more about a phenomenon,… —> more theoretical
2) Applied research means that we are not really interested in advancing theories,
but we want to use this theory to apply in practical situations —> we will be
testing in a specific setting whether a specific theory works and can be used
Both kinds of research are being used, depending on the domain of science
Overall, the general focus in psychology is more on fundamental research
Continuously evolving
• Theories are continuously tested, modified, and falsified
• Research triggers follow-up research (example: impact of color on approach-and
avoidance-motivation in context (Meier et al., 2012))
o Example = experiment: ‘how does color affect motivation’
➢ People were divided into 2 groups. The people in both groups got a
booklet with anagrams. Both booklets were the same qua exercises
1) one group had a booklet with a red cover
2) the other group had a booklet with a blue cover
➢ They tested how many exercises were solved by both groups and in what
time → Group with red cover = slower, Group with blue cover = faster
➢ ➔ lea to the thought that color could have an effect on motivation. Red
would lead people to be more careful (because it is associated with
danger), try to avoid mistakes which results in being slower ➔ red
triggers avoidance motivation (slower). Blue triggers approach
motivation (faster)
4
HOC 1: Introduction to scientific reasoning
Overview lecture:
1. Psychology is a way of thinking (chapter 1)
2. Sources of information (chapter 2)
1. Psychology is a way of thinking
Producing versus consuming research
When you work as a psychologist, you are a consumer of
science to help a patient, to look up a therapy,… —>
requires you to think critically about research so you can
use it in an appropriate way
When you run scientific experiments yourself, you are a
producer of research. This could happen when you do a
PHD
• Examples to illustrate the importance of the research consumer role:
o Facilitated communication treatment
➢ for example when someone can’t talk, a health care worker could assist
this person and through a keyboard, this person could select letters by
hand and form words
➢ It might be a good idea to check the reliability of this method before
actually using it on patients —> it actually seems that this method of
communication treatment is not very great, because different people see
different things and communicate different things (which results in the
patient not being able to really express his feelings/thoughts) —> it’s
more the health care workers who influence the communication of the
person
➢ ➔ something that might look good at first glance is not always as great as
it looked when you do research
o Scared-straight approach
➢ This is a method where they try getting people back on the right path by
scaring them ➔ for example, there are shows where young adults are
being sent to prison and treated terribly
➢ After a few days, the young people are so scared that they promise to be a
better person
➢ The effectiveness of this method does not seem to be good when
compared to other kids who were treated with another method. The
group that went through the scared-straight approach, was more likely to
be arrested, in criminal environments,… ➔ didn’t work
1
, o Mindfulness and academic performance
➢ Children who received mindfulness training had better academic
performances than the control group.
➢ It seems that using mindfulness in school results in better academic
performances, so it might be a good idea to integrate mindfulness
• Critical mindset is essential!!!
• Not all published research is correct or robust
A good example of this is the study that was done by ‘Bem’ and led to what is now
called ‘the Replication crisis in psychology’ = he did 9 experiments where he
‘proved’ that there is such a thing as ‘premonition’ (≈ predicting the future)
o Example of 1 experiment: Students were given a list of words (words were shown
one by one), after a certain point he stopped showing the words and the students
had to write down as many words as possible (which they remembered) →
recollection task
o Second part of the experiment was that the students could see some of the
words again (subset) + had to copy them again by writing them down —> strange,
normally it’s the other way around in an experiment. He did this to achieve
training effects!
o ➔ The conclusion was that the words people most likely remembered were the
words from that subset that they were training after writing them down. It’s like
people were already predicting which words they would see next and they
remembered them better!
o How to explain this? —> led to researchers questioning their way of researching
o Problems! ➔ Bem only reported the results of 9 experiments, but he actually ran
hundreds. He only reported experiments where he proved that people could
predict it! He did hundreds of experiments were this was not proven, but he
didn’t report these results ➔ very important to know = shows you have to be
critical!
o Also led to replication crisis = most studies that were done before, could not be
replicated because most experiments were lucky shots.
How scientists work
• Science is based on empiricism
• Scientists test theories
• Scientists work on fundamental and applied problems
• Science is continuously evolving
• Scientists publish their findings in scientific journals
• Scientists communicate with the general public via journalists
Empiricism
• We use an empirical method in scientific research and we use data
• Empirical method is based on data obtained through:
o Our senses (sight, hearing, touch)
o Instruments that assist our senses (thermometer, questionnaires, timer)
• Empiricists aim to do research in a systematic, rigorous, and replicable manner
2
,• Empiricism is not based on own experiences, intuition, or authority
Scientists test theories
We use an empirical approach to test theories. To do so, we use the
empirical research cycle as shown on the left. We start with a theory. From
this theory, we form a few research questions which we will try to resolve in
an experiment. Before we do that, we produce a research design and some
hypotheses. After this, we obtain data and we can accept/change the
theory
Experiment Harry Harlow: there were a few contradictory
theories about the bond between mother and child:
1) This bond exists because the mom feeds the baby
2) This bond exists because the mom provides comfort,
warmth and safety for the baby
➔ They made 2 surrogate moms, one with a hard ‘body’, but with
food and one who was soft and warm, but didn’t have ability to
feed
This experiment led to the ‘contact comfort theory’ —> the soft, warm and comforting
mom was more important for the baby than the feeding mother
• Characteristics of good theories
o Supported by data
o Falsifiable —> must be constructed in such a way that you could create a study
that could disprove a theory. If it is not falsifiable, it is pseudoscience
o Parsimonious (“Occam’s razor”) = if we have 2 explanations for a phenomenon
and they are both qualitative, the easiest explanation will be the one we use
• Examples of non-falsifiable theories:
o Facilitated communication treatment believers
o See book “De ongelovige Thomas heeft een punt”
• Theories are evaluated based on all available evidence
o A theory can never be “proven” (we always say we ‘support’ a theory), but it can
be falsified (when the theory is not correct)
➢ For example = ‘all swans are white’ —> if you go check this in a few lakes
and you only find white swans, you can ‘support’ the theory. This doesn’t
mean that there are ONLY white swans in the whole wide world (because
you did not check them all), so you cannot ‘prove’ the theory. Once you
see 1 black swan, the theory is immediately falsified!
o Replication is crucial!
3
, Scientists form a community
• Merton’s scientific norms:
1. Universalism = everyone, regardless of their background, should be able to do
science —> it’s not who you are that matters, but how you do science!
2. Communality —> you should share the results you became with the scientific
community
3. Disinterestedness —> scientists should pursue research not because they are
interested in money, fame,… but it should be done because people want to find
out something
4. Organized skepticism —> we should have a system that is in place to check
each others work, so that we can evaluate it and see to what extent we can
accept claims
Fundamental VS applied research
• There is a difference between the type of research
1) Fundamental research is being done because we want to advance theoretical
insights, learn more about a phenomenon,… —> more theoretical
2) Applied research means that we are not really interested in advancing theories,
but we want to use this theory to apply in practical situations —> we will be
testing in a specific setting whether a specific theory works and can be used
Both kinds of research are being used, depending on the domain of science
Overall, the general focus in psychology is more on fundamental research
Continuously evolving
• Theories are continuously tested, modified, and falsified
• Research triggers follow-up research (example: impact of color on approach-and
avoidance-motivation in context (Meier et al., 2012))
o Example = experiment: ‘how does color affect motivation’
➢ People were divided into 2 groups. The people in both groups got a
booklet with anagrams. Both booklets were the same qua exercises
1) one group had a booklet with a red cover
2) the other group had a booklet with a blue cover
➢ They tested how many exercises were solved by both groups and in what
time → Group with red cover = slower, Group with blue cover = faster
➢ ➔ lea to the thought that color could have an effect on motivation. Red
would lead people to be more careful (because it is associated with
danger), try to avoid mistakes which results in being slower ➔ red
triggers avoidance motivation (slower). Blue triggers approach
motivation (faster)
4
