HOORCOLLEGES RESEARCH METHODS
HC 1: BASIC STUDY DESIGNS
2 advantages of effect sizes over significance tests:
1. Significance depends on sample size, ES not
2. ES helps to compare significant effects with each other
Questions
- What means underpowered? Given few patients with dementia are
really impaired in prospective memory, it is unlikely that you will reveal
this effect in your study. If there’s an effect, your sample is not big
enough to show it, you miss true effect (power = likelihood of finding a
true effect).
- You evaluate a new treatment for cognitive impairment after acquired
brain damage. Name two different active control group designs (when
control group also receives a treatment) that you could implement in your
design:
1. Dose control: give same substance but lower dose
2. Dismantling design: performed with very complex treatments,
they do the same but take out 1 so-called active ingredient see
what exactly this ingredient causes.
- A new test is developed for the diagnosis of MCI. You perform a study to
validate this test in clinical practice. Which statistical values would you
report to describe the validity of such a test?
• Overall diagnostic accuracy (e.g. AUC)
• Sensitivity towards disease, specificity towards no disease
- Name two research designs that you can apply to explore the
effectiveness of a treatment on one patient (single case studies)?
• ABAB design
• Multiple baseline design
- What is the Reliable Change Index?
,• Indicates the change in a measure from one time to another (e.g.
before and after treatment), taking into consideration the variability of
scores at the first time, as well as the test-retest reliability.
- How does a Delphi methodology contribute to reaching consensus?
• A Delphi methodology includes a group of experts on a topic (the expert
panel) and facilitates collective decision making by a structured and
anonymous way of communication.
- Cross sectional: compare 2 groups at 1 time point.
- Case-control: for each person you match a control person (same
age, education).
Selection, recruitment, measurement
, - Internal validity = when I assess my people then I will do certain
mistakes (confounding variable, did not understand instructions). So
my actual subjects and measurements may deviate from the
intended sample and variables. Important in experiments.
- Ultimate goal for us is the external validity!
Group designs
Selection and recruitment
Establishing selection criteria:
Inclusion criteria:
▪ Demographic characteristics
▪ Clinical characteristics
▪ Geographic characteristics
▪ Temporal characteristics
Exclusion criteria
▪ Risk of being lost at follow up
▪ Inability to provide good data
▪ At risk for possible adverse effects
▪ Non-representative for population
Sampling
Nonprobability samples:
▪ Convenience samples
▪ Snowball sampling
Probability samples:
▪ Simple random samples
, ▪ Systematic random sample
▪ Stratified random sample
▪ Cluster sample
Uitleg probability samples:
1. Simple Random Sample
Elke persoon of eenheid in de populatie heeft een gelijke kans om
geselecteerd te worden.
Voorbeeld: Namen in een hoed stoppen en willekeurig trekken.
2. Systematic Random Sample
Selecteer elke n-de persoon of eenheid uit een gesorteerde lijst. De
startpunt wordt willekeurig gekozen.
Voorbeeld: In een lijst van 100 mensen selecteer je elke 10e persoon.
3. Stratified Random Sample
De populatie wordt opgedeeld in subgroepen (strata) op basis van
gedeelde kenmerken. Uit elke subgroep wordt een willekeurige steekproef
genomen.
Voorbeeld: Een populatie opdelen in mannen en vrouwen en vervolgens
willekeurig een aantal mensen uit beide groepen kiezen.
4. Cluster Sample
De populatie wordt in clusters (groepen) verdeeld. Een paar clusters
worden willekeurig gekozen, en vervolgens wordt iedereen in die clusters
onderzocht.
Voorbeeld: Scholen in een stad worden als clusters beschouwd. Je
selecteert willekeurig 5 scholen en onderzoekt alle studenten daar.
Deze methoden worden vaak gekozen afhankelijk van de aard van de
populatie en het onderzoek.
In reality we rarely have probability samples cause of different reasons.
So in most cases it’s a convenience sample that we use.
HC 1: BASIC STUDY DESIGNS
2 advantages of effect sizes over significance tests:
1. Significance depends on sample size, ES not
2. ES helps to compare significant effects with each other
Questions
- What means underpowered? Given few patients with dementia are
really impaired in prospective memory, it is unlikely that you will reveal
this effect in your study. If there’s an effect, your sample is not big
enough to show it, you miss true effect (power = likelihood of finding a
true effect).
- You evaluate a new treatment for cognitive impairment after acquired
brain damage. Name two different active control group designs (when
control group also receives a treatment) that you could implement in your
design:
1. Dose control: give same substance but lower dose
2. Dismantling design: performed with very complex treatments,
they do the same but take out 1 so-called active ingredient see
what exactly this ingredient causes.
- A new test is developed for the diagnosis of MCI. You perform a study to
validate this test in clinical practice. Which statistical values would you
report to describe the validity of such a test?
• Overall diagnostic accuracy (e.g. AUC)
• Sensitivity towards disease, specificity towards no disease
- Name two research designs that you can apply to explore the
effectiveness of a treatment on one patient (single case studies)?
• ABAB design
• Multiple baseline design
- What is the Reliable Change Index?
,• Indicates the change in a measure from one time to another (e.g.
before and after treatment), taking into consideration the variability of
scores at the first time, as well as the test-retest reliability.
- How does a Delphi methodology contribute to reaching consensus?
• A Delphi methodology includes a group of experts on a topic (the expert
panel) and facilitates collective decision making by a structured and
anonymous way of communication.
- Cross sectional: compare 2 groups at 1 time point.
- Case-control: for each person you match a control person (same
age, education).
Selection, recruitment, measurement
, - Internal validity = when I assess my people then I will do certain
mistakes (confounding variable, did not understand instructions). So
my actual subjects and measurements may deviate from the
intended sample and variables. Important in experiments.
- Ultimate goal for us is the external validity!
Group designs
Selection and recruitment
Establishing selection criteria:
Inclusion criteria:
▪ Demographic characteristics
▪ Clinical characteristics
▪ Geographic characteristics
▪ Temporal characteristics
Exclusion criteria
▪ Risk of being lost at follow up
▪ Inability to provide good data
▪ At risk for possible adverse effects
▪ Non-representative for population
Sampling
Nonprobability samples:
▪ Convenience samples
▪ Snowball sampling
Probability samples:
▪ Simple random samples
, ▪ Systematic random sample
▪ Stratified random sample
▪ Cluster sample
Uitleg probability samples:
1. Simple Random Sample
Elke persoon of eenheid in de populatie heeft een gelijke kans om
geselecteerd te worden.
Voorbeeld: Namen in een hoed stoppen en willekeurig trekken.
2. Systematic Random Sample
Selecteer elke n-de persoon of eenheid uit een gesorteerde lijst. De
startpunt wordt willekeurig gekozen.
Voorbeeld: In een lijst van 100 mensen selecteer je elke 10e persoon.
3. Stratified Random Sample
De populatie wordt opgedeeld in subgroepen (strata) op basis van
gedeelde kenmerken. Uit elke subgroep wordt een willekeurige steekproef
genomen.
Voorbeeld: Een populatie opdelen in mannen en vrouwen en vervolgens
willekeurig een aantal mensen uit beide groepen kiezen.
4. Cluster Sample
De populatie wordt in clusters (groepen) verdeeld. Een paar clusters
worden willekeurig gekozen, en vervolgens wordt iedereen in die clusters
onderzocht.
Voorbeeld: Scholen in een stad worden als clusters beschouwd. Je
selecteert willekeurig 5 scholen en onderzoekt alle studenten daar.
Deze methoden worden vaak gekozen afhankelijk van de aard van de
populatie en het onderzoek.
In reality we rarely have probability samples cause of different reasons.
So in most cases it’s a convenience sample that we use.
