Research Methods II
1. Introduction to scientific reasoning
Producing versus consuming research
- Critical mindset is essential
o Some people only show a few of their trials used
You cannot immediately accept what is being published
- Not all published research is correct or robust
o Changing measures, adding variables, so it supports the hypothesis
- Replication crisis in psychology
o Not all studies can be replicated because of the way research is
conducted
H OW S C I E N T I S T S W O R K
- Science is based on empiricism
- Scientists test theories
- Scientists work on fundamental and applied problems
- Science is continuously evolving
- Scientists publish their findings in scientific journals
- Scientists communicate with the general public via journalists
EMPIRICISM
- Empirical method is based on data obtained through:
o Our senses (sight, hearing, touch)
o Instruments that assist our senses (thermometer, questionnaires,
timer)
We collect data
- Empiricists aim to do research in a systematic (collecting data in the same
way for every participant), rigorous (being transparent, what troubles
came during the study, how to exactly replicate it), and replicable manner
- Empiricism is not based on own experiences, intuition, or authority figures
o It will solve issues that other things cant (objectivism)
1
,Scientists test theories
Have a certain belief they want to test à they need a specific hypothesis (what they
expect to see) à collect data à does it support the hypothesis or not? à yes = proof
for theory; no = change theory/abandon it or my study design is wrong
Theory-data cycle only applies to confirmatory or deductive approaches !!!!
Note: There is a difference between exploratory
(inductive) and confirmatory (deductive) research!
Example: cupboard theory (attachement bc the person
feeds you, makes sure u survive) vs contact comfort
theory (attachement by the person who gives you love and
affection)
- Monkey with bowl, metallic wire, no comfort à
cupboard theory
- Monkey with cloth but no food à contact comfort theory
o The monkey stayed mostly w the cloth monkey; contact comfort
theory is right, so cupboard theory is wrong
CHARACTERISTICS OF GOOD THEORIES
o Supported by data
o Falsifiable
o Parsimonious (“Occam’s razor”); we go with the simplest
explanation
- Examples of non-falsifiable theories:
o Facilitated communication treatment believers
o See book “De ongelovige Thomas heeft een punt”
- Theories are evaluated based on all available evidence
o A theory can never be “proven”, but it can be falsified
One falsification is enough to disapprove a theory and
abandon it
Replication is crucial to come to a conclusion
SCIENTISTS FORM A COMMUNITY
- Merton’s scientific norms:
o 1) Universalism: You can do research regardless of your
background; ethnicity, religion, sex
2
, o 2) Communality: study finds should be made public to everyone, it
shouldn’t be privatized
o 3) Disinterestedness: scientist should do research to improve or
expand understanding of something, not to become rich or famous
o 4) Organized skepticism: scientists have a critical approach to each
others studies, then it will be published for example
Fundamental versus applied research
1) Basic research: simply to expand our knowledge about a topic, to
improve a theory, how something works
a. Needed before we move onto applied research
2) Translational research: not only trying to improve theoretical knowledge
but also wanting to develop a practical application, in a lab setting
3) Applied research: practical application in a real world setting
Continuously evolving
- Theories are continuously tested, modified, and falsified
- Research triggers follow-up research (example: impact of color on
approach- and avoidance-motivation in context (Meier et al., 2012))
o Red is associated with danger (traffic signs, avoiding things; less
motivation)
In romantic context it causes more motivation
o Blue is used for approaching (traffic signs; more motivation)
Follow-up research showed its also based on context
PUBLISHING
- Manuscripts are submitted to scientific journals (e.g., Nature, Science,
Psychological Bulletin)
- Peer-review process before publishing it in journals
o Parties involved include editor, reviewers, authors
o Reject, revise and resubmit, accept
o Because papers and research is important in an academic context
and can even determine someones career
- Its harder to get your study published compared to before
3
, COMMUNICATION
A lot of info gets lost in translation, a lot of articles reframe or rewrite research
findings to make it more attractive (the media)
Sources of information
- Researchers:
o Use a comparison group
o Control for third variables
o Try to evaluate information without bias
- Research is better than:
o Experience
o Intuition
o Authority arguments
R E S E A R C H V E RS U S E X PE R I E N C E
- Experience has no comparison group
- Experience is confounded by third variables
o Rage room: feel better after destroying place.. or feel better
because youre active?
- Bottom line: research is better than own experiences
- Dr. Benjamin Rush – Bloodletting; ‘removing bad blood to make u feel
better’
- Treatment was never compared to comparison group; so we cant see how
many recover without vs with; cant draw a real conclusion
o Only with a comparison group can you compare the effectiveness of
something
- Experience is confounded
- We do not control our own experience for multiple, simultaneously acting
factors
that influence our feelings and behaviors
- Researchers:
o use a control group
o control for confounders
o attempt to evaluate information without bias
o someone gave u terrible feedback; so that you are angry à then
asked to go in a room to wait
one group sits on a chair and waits before second phase
one group had a punching bag and should punch it while
waiting
one group had a punching bag with face of steve and should
punch it
4
1. Introduction to scientific reasoning
Producing versus consuming research
- Critical mindset is essential
o Some people only show a few of their trials used
You cannot immediately accept what is being published
- Not all published research is correct or robust
o Changing measures, adding variables, so it supports the hypothesis
- Replication crisis in psychology
o Not all studies can be replicated because of the way research is
conducted
H OW S C I E N T I S T S W O R K
- Science is based on empiricism
- Scientists test theories
- Scientists work on fundamental and applied problems
- Science is continuously evolving
- Scientists publish their findings in scientific journals
- Scientists communicate with the general public via journalists
EMPIRICISM
- Empirical method is based on data obtained through:
o Our senses (sight, hearing, touch)
o Instruments that assist our senses (thermometer, questionnaires,
timer)
We collect data
- Empiricists aim to do research in a systematic (collecting data in the same
way for every participant), rigorous (being transparent, what troubles
came during the study, how to exactly replicate it), and replicable manner
- Empiricism is not based on own experiences, intuition, or authority figures
o It will solve issues that other things cant (objectivism)
1
,Scientists test theories
Have a certain belief they want to test à they need a specific hypothesis (what they
expect to see) à collect data à does it support the hypothesis or not? à yes = proof
for theory; no = change theory/abandon it or my study design is wrong
Theory-data cycle only applies to confirmatory or deductive approaches !!!!
Note: There is a difference between exploratory
(inductive) and confirmatory (deductive) research!
Example: cupboard theory (attachement bc the person
feeds you, makes sure u survive) vs contact comfort
theory (attachement by the person who gives you love and
affection)
- Monkey with bowl, metallic wire, no comfort à
cupboard theory
- Monkey with cloth but no food à contact comfort theory
o The monkey stayed mostly w the cloth monkey; contact comfort
theory is right, so cupboard theory is wrong
CHARACTERISTICS OF GOOD THEORIES
o Supported by data
o Falsifiable
o Parsimonious (“Occam’s razor”); we go with the simplest
explanation
- Examples of non-falsifiable theories:
o Facilitated communication treatment believers
o See book “De ongelovige Thomas heeft een punt”
- Theories are evaluated based on all available evidence
o A theory can never be “proven”, but it can be falsified
One falsification is enough to disapprove a theory and
abandon it
Replication is crucial to come to a conclusion
SCIENTISTS FORM A COMMUNITY
- Merton’s scientific norms:
o 1) Universalism: You can do research regardless of your
background; ethnicity, religion, sex
2
, o 2) Communality: study finds should be made public to everyone, it
shouldn’t be privatized
o 3) Disinterestedness: scientist should do research to improve or
expand understanding of something, not to become rich or famous
o 4) Organized skepticism: scientists have a critical approach to each
others studies, then it will be published for example
Fundamental versus applied research
1) Basic research: simply to expand our knowledge about a topic, to
improve a theory, how something works
a. Needed before we move onto applied research
2) Translational research: not only trying to improve theoretical knowledge
but also wanting to develop a practical application, in a lab setting
3) Applied research: practical application in a real world setting
Continuously evolving
- Theories are continuously tested, modified, and falsified
- Research triggers follow-up research (example: impact of color on
approach- and avoidance-motivation in context (Meier et al., 2012))
o Red is associated with danger (traffic signs, avoiding things; less
motivation)
In romantic context it causes more motivation
o Blue is used for approaching (traffic signs; more motivation)
Follow-up research showed its also based on context
PUBLISHING
- Manuscripts are submitted to scientific journals (e.g., Nature, Science,
Psychological Bulletin)
- Peer-review process before publishing it in journals
o Parties involved include editor, reviewers, authors
o Reject, revise and resubmit, accept
o Because papers and research is important in an academic context
and can even determine someones career
- Its harder to get your study published compared to before
3
, COMMUNICATION
A lot of info gets lost in translation, a lot of articles reframe or rewrite research
findings to make it more attractive (the media)
Sources of information
- Researchers:
o Use a comparison group
o Control for third variables
o Try to evaluate information without bias
- Research is better than:
o Experience
o Intuition
o Authority arguments
R E S E A R C H V E RS U S E X PE R I E N C E
- Experience has no comparison group
- Experience is confounded by third variables
o Rage room: feel better after destroying place.. or feel better
because youre active?
- Bottom line: research is better than own experiences
- Dr. Benjamin Rush – Bloodletting; ‘removing bad blood to make u feel
better’
- Treatment was never compared to comparison group; so we cant see how
many recover without vs with; cant draw a real conclusion
o Only with a comparison group can you compare the effectiveness of
something
- Experience is confounded
- We do not control our own experience for multiple, simultaneously acting
factors
that influence our feelings and behaviors
- Researchers:
o use a control group
o control for confounders
o attempt to evaluate information without bias
o someone gave u terrible feedback; so that you are angry à then
asked to go in a room to wait
one group sits on a chair and waits before second phase
one group had a punching bag and should punch it while
waiting
one group had a punching bag with face of steve and should
punch it
4
