Introduction to Research Methods
Why study statistics?
● Statistics: a set of tools for describing, organizing, and interpreting information
○ Descriptive: describing data
■ What was the average mark in the exam?
○ Inferential statistics: testing the hypothesis
■ Was there a meaningful difference between marks?
Study Design
● Gets us from theoretical question to a useful answer
○ Theoretical construct (something you want to measure)
○ Operationalization: process by which we try to derive a measure from a theoretical
construct
○ Measure (method or tool you use to gather data)
○ Data: one or more variables that can be analyzed
● Predictor (independent variable)
● Outcome (dependent method)
● Levels of measurement
○ Categorical (nominal): categories with no logical order
■ Ex: gender, voting, behavior
○ Ordinal: categories have an order but distance isn’t meaningful
■ Ex: highest level of qualification
○ Interval: numbers but with no absolute zero
■ Ex: IQ score
○ Ratio: numbers with meaningful zero point
■ reaction time, number of siblings
● Types of Data
○ Continuous: can take any value of range
■ Ex: height, speed
○ Discrete: can only take certain value
■ ex: Number of people
,Types of Study:
● Case study: detailed study of a specific person
○ Can’t use statistics and cannot generalize
● Quasi-experimental: allows independent and dependent variable to vary naturally
● Experimental: manipulation of independent variable, then allows dependent variable to vary
naturally
Within vs Between experiments
● Within: each person takes part in each condition
● Between: each person only takes part in one condition
, Reliability and Validity
Study question: can we improve language outcomes for toddlers with language delay?
● Children with language difficulties have persistent language and education problems —> long
term effects: mental health and employment difficulties
○ Early intervention aims to prevent these long-term negative consequences
Measure language ability
1. Identify toddles with language delay
2. Provide pre-treatment baseline of language ability
1. Continuous data: all the words that the toddler can understand and/or speak
3. Measure change in language overtime
Reliability: consistency of your measurement; precision and repeatability
● Test-retest: consistency over time
● Inter-rater: if someone else repeats the same measurements, will they produce the same answer?
● Parallel forms: consistency across theoretically-equivalent measurements
● Internal consistency: consistency within different parts of a similar function
Validity: does what it says
● Internal: extent at which you are able to draw the correct conclusions about causal relationships
between variables internal to the study
● External: generalizability of findings
● Construct: does assessment measure what you want it to measure (theoretical construct)
● Face validity: do others think the test measures what it’s supposed to be measure?
● Ecological: study should closely approximate the real world scenario that is being investigated
Threats to Validity:
● Confound: additional, often unmeasured variable that is related to both the predictors and the
outcome
○ Threat to internal validity
● Artefact: finding only holds in a special situation that you happened to test in study
○ Threat to generalization or external validity
Why study statistics?
● Statistics: a set of tools for describing, organizing, and interpreting information
○ Descriptive: describing data
■ What was the average mark in the exam?
○ Inferential statistics: testing the hypothesis
■ Was there a meaningful difference between marks?
Study Design
● Gets us from theoretical question to a useful answer
○ Theoretical construct (something you want to measure)
○ Operationalization: process by which we try to derive a measure from a theoretical
construct
○ Measure (method or tool you use to gather data)
○ Data: one or more variables that can be analyzed
● Predictor (independent variable)
● Outcome (dependent method)
● Levels of measurement
○ Categorical (nominal): categories with no logical order
■ Ex: gender, voting, behavior
○ Ordinal: categories have an order but distance isn’t meaningful
■ Ex: highest level of qualification
○ Interval: numbers but with no absolute zero
■ Ex: IQ score
○ Ratio: numbers with meaningful zero point
■ reaction time, number of siblings
● Types of Data
○ Continuous: can take any value of range
■ Ex: height, speed
○ Discrete: can only take certain value
■ ex: Number of people
,Types of Study:
● Case study: detailed study of a specific person
○ Can’t use statistics and cannot generalize
● Quasi-experimental: allows independent and dependent variable to vary naturally
● Experimental: manipulation of independent variable, then allows dependent variable to vary
naturally
Within vs Between experiments
● Within: each person takes part in each condition
● Between: each person only takes part in one condition
, Reliability and Validity
Study question: can we improve language outcomes for toddlers with language delay?
● Children with language difficulties have persistent language and education problems —> long
term effects: mental health and employment difficulties
○ Early intervention aims to prevent these long-term negative consequences
Measure language ability
1. Identify toddles with language delay
2. Provide pre-treatment baseline of language ability
1. Continuous data: all the words that the toddler can understand and/or speak
3. Measure change in language overtime
Reliability: consistency of your measurement; precision and repeatability
● Test-retest: consistency over time
● Inter-rater: if someone else repeats the same measurements, will they produce the same answer?
● Parallel forms: consistency across theoretically-equivalent measurements
● Internal consistency: consistency within different parts of a similar function
Validity: does what it says
● Internal: extent at which you are able to draw the correct conclusions about causal relationships
between variables internal to the study
● External: generalizability of findings
● Construct: does assessment measure what you want it to measure (theoretical construct)
● Face validity: do others think the test measures what it’s supposed to be measure?
● Ecological: study should closely approximate the real world scenario that is being investigated
Threats to Validity:
● Confound: additional, often unmeasured variable that is related to both the predictors and the
outcome
○ Threat to internal validity
● Artefact: finding only holds in a special situation that you happened to test in study
○ Threat to generalization or external validity
