Week 1
Today’s program
MIR framework
Mixed methods
Replicability
Data analysis plan
Research Methodology
The scientific study of processes, methods and instruments used in research.
Rationale
Science aims at knowledge production
Not all questions are RQ
Good Conduct in Research requires best possible tools and procedures
Bad research hurts
Participants and populations
Funders
Society
MIR framework
Concept: gender
Attribute: physical sex
Instrument selection: what is your sex? (male/female)
Variable: sex
Different determination of attributes of concept → different instrument selection/design →
different variables
Study designs
A study design is the framework for data collection
The reference period of the research: past, present, longitudinal or not
, The level of control that is needed by the researcher: observational, quasi-experiment,
experiment
The number of data collection waves
One, two, more…
In the same measurement units or not
MIR framework and Mixed Methods Research
Mixed Methods Research has three components
Qualitative
Quantitative
Synthesis
Qualitative and quantitative part may be seen as modules in MIR framework
Synthesis between quali and quanti can take on different forms depending on
order/simultaneousness and aim of the modules
Fetters - summary
Why do we use the MIR framework?
Tool for conducting any type of scientific research
Flexible
In this course it ties it all together (‘red thread’)
Help expand your toolbox with specific techniques: always in the context of MIR
Transparency & good research conduct
Lead to replicability!
Data preservation
Old data can be useful
Assumes and relies on data integrity
New techniques to gain new insights into the same research question based on old data (also
a form of replication)
Theoretical & technical advances to answer new research questions
Replicability
Given: study design, instruments, data collection
Execution of data analysis
Transparency
Someone else can retrace all your steps
Necessary for replication!
Direct replication
Repeat what others did on newly collected data
Do conclusions still hold up?
Researcher triangulation in qualitative research
, Data analysis plan - what is
A data analysis plan is a protocol that describes
how the collected data will be handled
processed
and analysed to answer the research question
Data management plan describes full life cycle of data
how the original data will be stored
where and for how long
(un)lock, who has access, procedures on access, on deletion
Data analysis plan - why bother
Necessary for ALL data analyses
To check whether data to be collected indeed allows us to answer research questions
(internal validity!)
To communicate
To retrieve
To enhance replicability
Knowing where to start once data is in
To handle human condition (think cognitive biases)
To handle latter temptations (like cherry-picking and P-hacking)
Data analysis plan - contains
Reference to possible ethical issues
Data processing - e.g. how you’ll calculate new vars, code.
How completely missing and partial missing data is dealt with
Data exploration
When qualitative: coding technique for data reduction, coding-units
Data analysis
Which RQ’s are answered
Which associations
Modeling details
When qualitative: intercoder reliability
Reporting strategy
Key terms
Research design The design of a study defines the study type (descriptive, correlational, semi-
experimental, experimental, review, meta-analytic) and sub-type (e.g.,
descriptive-longitudinal case study), research problem, hypotheses,
independent and dependent variables, experimental design, and, if
applicable, data collection methods and a statistical analysis plan. A research
design is a framework that has been created to find answers to research
questions.
Operationalization Operationalization is the process of strictly defining variables into
measurable factors.
Internal validity Internal validity is the extent to which you can be confident that a cause-
and-effect relationship established in a study cannot be explained by other
factors.
Measurement Measurement validity is the extent to which a measurement tool measures
validity what it's supposed to measure.
External validity External validity is the extent to which you can generalize the findings of a
study to other situations, people, settings and measures. In other words, can
you apply the findings of your study to a broader context?
Reliability A measure is said to have a high reliability if it produces similar results
Today’s program
MIR framework
Mixed methods
Replicability
Data analysis plan
Research Methodology
The scientific study of processes, methods and instruments used in research.
Rationale
Science aims at knowledge production
Not all questions are RQ
Good Conduct in Research requires best possible tools and procedures
Bad research hurts
Participants and populations
Funders
Society
MIR framework
Concept: gender
Attribute: physical sex
Instrument selection: what is your sex? (male/female)
Variable: sex
Different determination of attributes of concept → different instrument selection/design →
different variables
Study designs
A study design is the framework for data collection
The reference period of the research: past, present, longitudinal or not
, The level of control that is needed by the researcher: observational, quasi-experiment,
experiment
The number of data collection waves
One, two, more…
In the same measurement units or not
MIR framework and Mixed Methods Research
Mixed Methods Research has three components
Qualitative
Quantitative
Synthesis
Qualitative and quantitative part may be seen as modules in MIR framework
Synthesis between quali and quanti can take on different forms depending on
order/simultaneousness and aim of the modules
Fetters - summary
Why do we use the MIR framework?
Tool for conducting any type of scientific research
Flexible
In this course it ties it all together (‘red thread’)
Help expand your toolbox with specific techniques: always in the context of MIR
Transparency & good research conduct
Lead to replicability!
Data preservation
Old data can be useful
Assumes and relies on data integrity
New techniques to gain new insights into the same research question based on old data (also
a form of replication)
Theoretical & technical advances to answer new research questions
Replicability
Given: study design, instruments, data collection
Execution of data analysis
Transparency
Someone else can retrace all your steps
Necessary for replication!
Direct replication
Repeat what others did on newly collected data
Do conclusions still hold up?
Researcher triangulation in qualitative research
, Data analysis plan - what is
A data analysis plan is a protocol that describes
how the collected data will be handled
processed
and analysed to answer the research question
Data management plan describes full life cycle of data
how the original data will be stored
where and for how long
(un)lock, who has access, procedures on access, on deletion
Data analysis plan - why bother
Necessary for ALL data analyses
To check whether data to be collected indeed allows us to answer research questions
(internal validity!)
To communicate
To retrieve
To enhance replicability
Knowing where to start once data is in
To handle human condition (think cognitive biases)
To handle latter temptations (like cherry-picking and P-hacking)
Data analysis plan - contains
Reference to possible ethical issues
Data processing - e.g. how you’ll calculate new vars, code.
How completely missing and partial missing data is dealt with
Data exploration
When qualitative: coding technique for data reduction, coding-units
Data analysis
Which RQ’s are answered
Which associations
Modeling details
When qualitative: intercoder reliability
Reporting strategy
Key terms
Research design The design of a study defines the study type (descriptive, correlational, semi-
experimental, experimental, review, meta-analytic) and sub-type (e.g.,
descriptive-longitudinal case study), research problem, hypotheses,
independent and dependent variables, experimental design, and, if
applicable, data collection methods and a statistical analysis plan. A research
design is a framework that has been created to find answers to research
questions.
Operationalization Operationalization is the process of strictly defining variables into
measurable factors.
Internal validity Internal validity is the extent to which you can be confident that a cause-
and-effect relationship established in a study cannot be explained by other
factors.
Measurement Measurement validity is the extent to which a measurement tool measures
validity what it's supposed to measure.
External validity External validity is the extent to which you can generalize the findings of a
study to other situations, people, settings and measures. In other words, can
you apply the findings of your study to a broader context?
Reliability A measure is said to have a high reliability if it produces similar results
