Research Methods in Clinical Neuropsychology
Learning outcomes
After the course, students have the knowledge and understanding of the following research
designs and research standards:
• Basic study designs, such as cross sectional studies, case control studies, and cohort
studies
• Randomized Controlled Trials (e.g. following CONSORT statement)
• Single case research designs: Design, application, and analysis
• Ethical considerations and research ethics in clinical neuropsychology
After the course, students understand important principles in the collection and analysis of
clinical data, including:
• Pitfalls in the analysis of quasi-experimental clinical studies
• Evaluating treatment efficacy: Consequences of study type and expected effect size
• Clinical decision making using diagnostic tests: Development, application and
interpretation of diagnostic tests
• Reevaluation of patients with neuropsychological impairments: Measurement of change
in clinical neuropsychological practice and research
• Selection of intervention methods: Understanding meta-analyses (e.g. following PRISMA
guidelines)
__
*Notes for this summary:
- Formulas are not a necessary part of the examination, they are purely provided for
better understanding
- For each week, literature and lecture summaries are provided in English
- This summary is meant as an addition to the materials required for the exam. You
will understand this summary best if you have attended to the material at least once.
This summary does not provide an over-abundance of additional explanations. It
focuses purely on the main concepts and will help you to more easily look up certain
terms or present a quick overview of what an article was about
,Week 1 – Designing and implementing basic study
designs
Lecture notes
Group designs
- Population sample
o Cross-sectional: one time point comparing patients to control
o Case-control: for each patient match a control who is similar
o Prospective longitudinal study: follow over time
o Retrospective longitudinal study: ask about patient’s history
- External validity: do the intended variables really measure the phenomenon of
interest, is the intended sample representative of the population
- Internal validity: is your study going as planned (are measures correct)
- Selection criteria: inclusion & exclusion
- Sampling
o Nonprobability: take whoever you can (convenience sampling, snowball
sampling)
o Probability: selective (simple, systematic, stratified, cluster sampling)
- Measurement
o Scaling: categoric/continuous variables
Continuous contain more information, have higher power, more
flexible and can scale back
o Precision = reliability: degree to which individuals retain relative position
within distribution of scores from one testing session to another (compared
to others)
Test-retest reliability
o Accuracy = validity: measure what you intend to
o Sensitivity: correctly identify patients with a disease (= true positives)
Reduce random error: standardizes measurement, trained staff,
automated instruments, blinding, repeat measurement
o Computerized vs. pencil-and-paper test
Psychometric obstacles: do computerized tests measure the same?
Technical obstacles: speed in tech development hampers work on
psychometrics
Theoretical obstacles: less body of knowledge
Strategic obstacles: rapidly growing amount of tests, incomparability
of results
Computerized adaptive testing: selecting next item based on previous
performance
Nominal response model: more variables, types of errors measured
Test linking: direct comparison of tests and integration into individual
report
Person fit statistics: relative to overall performance
Web-based (mobile/wearables): applied setting
, Pitfalls in significance testing
- Post-hoc testing: creating a new hypothesis based on your data and confirming
yourself
o Separate hypothesis-driven and exploratory part (needs replication)
- Cherry-picking: didn’t specify where you expect differences and choosing significant
ones later
- Nonspecific hypothesis: much higher chance to find something (a-error)
- Multiple testing: one hypothesis should have one test
- Correction for multiple testing: type-I error (higher chance to find significance) –
Bonferroni inflates type-II error (not reporting an effect though there is one)
Articles
Hulley, Newman, & Cummings (2013). Choosing the Study Subjects: Specification, Sampling,
and Recruitment (Chapter 3).
- Sample of study subjects should accurately represent the population
generalizability
o Population = a complete set of people with specified characteristics
o Sample = a subset of the population
- Selection criteria define the population to be studies
o Inclusion criteria = main characteristics of the target population relevant to
the research question
o Exclusion criteria = characteristics that might interfere with the research and
will not be studied
- Convenience sample = easily accessible people who meet entry criteria
o Consecutive sample = consecutively selecting subjects who meet entry
criteria to minimize selection bias
- Probability sampling = random process that guarantees each unit of the population
has a specified chance of being included in the sample (gold standard for
generalizability)
o Simple random sample = listing all people in the population and selecting a
subset at random
o Systematic sample = simple random sample + selecting sample by a
preordained periodic process (susceptible to errors caused by natural
periodicities in the population)
o Stratified random sample = dividing the population into subgroups according
to certain characteristics and taking a random sample from each of these
‘strata’ (draws disproportionately from subgroups less common in the
population that are of special interest)
o Cluster sample = a random sample of natural clusters of individuals (more
homogenous for variables of interest)
- Response rate = proportion of subjects selected for the study who consent to be
enrolled
o Nonresponse bias = may influence conclusions and generalizability
Learning outcomes
After the course, students have the knowledge and understanding of the following research
designs and research standards:
• Basic study designs, such as cross sectional studies, case control studies, and cohort
studies
• Randomized Controlled Trials (e.g. following CONSORT statement)
• Single case research designs: Design, application, and analysis
• Ethical considerations and research ethics in clinical neuropsychology
After the course, students understand important principles in the collection and analysis of
clinical data, including:
• Pitfalls in the analysis of quasi-experimental clinical studies
• Evaluating treatment efficacy: Consequences of study type and expected effect size
• Clinical decision making using diagnostic tests: Development, application and
interpretation of diagnostic tests
• Reevaluation of patients with neuropsychological impairments: Measurement of change
in clinical neuropsychological practice and research
• Selection of intervention methods: Understanding meta-analyses (e.g. following PRISMA
guidelines)
__
*Notes for this summary:
- Formulas are not a necessary part of the examination, they are purely provided for
better understanding
- For each week, literature and lecture summaries are provided in English
- This summary is meant as an addition to the materials required for the exam. You
will understand this summary best if you have attended to the material at least once.
This summary does not provide an over-abundance of additional explanations. It
focuses purely on the main concepts and will help you to more easily look up certain
terms or present a quick overview of what an article was about
,Week 1 – Designing and implementing basic study
designs
Lecture notes
Group designs
- Population sample
o Cross-sectional: one time point comparing patients to control
o Case-control: for each patient match a control who is similar
o Prospective longitudinal study: follow over time
o Retrospective longitudinal study: ask about patient’s history
- External validity: do the intended variables really measure the phenomenon of
interest, is the intended sample representative of the population
- Internal validity: is your study going as planned (are measures correct)
- Selection criteria: inclusion & exclusion
- Sampling
o Nonprobability: take whoever you can (convenience sampling, snowball
sampling)
o Probability: selective (simple, systematic, stratified, cluster sampling)
- Measurement
o Scaling: categoric/continuous variables
Continuous contain more information, have higher power, more
flexible and can scale back
o Precision = reliability: degree to which individuals retain relative position
within distribution of scores from one testing session to another (compared
to others)
Test-retest reliability
o Accuracy = validity: measure what you intend to
o Sensitivity: correctly identify patients with a disease (= true positives)
Reduce random error: standardizes measurement, trained staff,
automated instruments, blinding, repeat measurement
o Computerized vs. pencil-and-paper test
Psychometric obstacles: do computerized tests measure the same?
Technical obstacles: speed in tech development hampers work on
psychometrics
Theoretical obstacles: less body of knowledge
Strategic obstacles: rapidly growing amount of tests, incomparability
of results
Computerized adaptive testing: selecting next item based on previous
performance
Nominal response model: more variables, types of errors measured
Test linking: direct comparison of tests and integration into individual
report
Person fit statistics: relative to overall performance
Web-based (mobile/wearables): applied setting
, Pitfalls in significance testing
- Post-hoc testing: creating a new hypothesis based on your data and confirming
yourself
o Separate hypothesis-driven and exploratory part (needs replication)
- Cherry-picking: didn’t specify where you expect differences and choosing significant
ones later
- Nonspecific hypothesis: much higher chance to find something (a-error)
- Multiple testing: one hypothesis should have one test
- Correction for multiple testing: type-I error (higher chance to find significance) –
Bonferroni inflates type-II error (not reporting an effect though there is one)
Articles
Hulley, Newman, & Cummings (2013). Choosing the Study Subjects: Specification, Sampling,
and Recruitment (Chapter 3).
- Sample of study subjects should accurately represent the population
generalizability
o Population = a complete set of people with specified characteristics
o Sample = a subset of the population
- Selection criteria define the population to be studies
o Inclusion criteria = main characteristics of the target population relevant to
the research question
o Exclusion criteria = characteristics that might interfere with the research and
will not be studied
- Convenience sample = easily accessible people who meet entry criteria
o Consecutive sample = consecutively selecting subjects who meet entry
criteria to minimize selection bias
- Probability sampling = random process that guarantees each unit of the population
has a specified chance of being included in the sample (gold standard for
generalizability)
o Simple random sample = listing all people in the population and selecting a
subset at random
o Systematic sample = simple random sample + selecting sample by a
preordained periodic process (susceptible to errors caused by natural
periodicities in the population)
o Stratified random sample = dividing the population into subgroups according
to certain characteristics and taking a random sample from each of these
‘strata’ (draws disproportionately from subgroups less common in the
population that are of special interest)
o Cluster sample = a random sample of natural clusters of individuals (more
homogenous for variables of interest)
- Response rate = proportion of subjects selected for the study who consent to be
enrolled
o Nonresponse bias = may influence conclusions and generalizability
