College 1: Evidence based treatment & cognitive disorders
Randomized controlled trials (RCT) are prospective studies that measure the
effectiveness of a new intervention or treatment. There are a few steps to take when
performing a RCT:
1. Selection of sample from a population
2. Assessment at baseline
3. Random assignment of intervention and control (blinded)
4. Following the cohort over time
5. Assessment of outcome
6. Assessment at follow up
It is important to be aware of what the control condition is doing. If possible the random
assignment should be blinded, this means that neither the examiner nor the participant
knows in which condition the participants are.
When the treatment is successful you need to ask a few questions:
1. How was success measured?
When looking at the neuropsychological tests, you can ask yourself in which cognitive
domain? Single tests of multivariate analysis (e.g. compound score)? Types of variables
(accuracy, speed, variability of performance)? When looking at the behavioral
observations, you can ask yourself by whom? In which situation? Or was it measured
with self- and informant reports?
It is also important to look at indications of functioning in daily life like job outcomes or
drug problems. It must be transferable to real life activities, for example with speed of
processing the training program is very much like the test condition. But you train the
test material, so becoming better in the test and not the actual speed of processing in
real life.
, 2. When was success measured?
Assessment at post-treatment or at follow up? What happened between
post-treatment assessment and follow up? Who are the people who drop out between
the outcome assessment and the follow-up assessment? The ones who remain are more
likely to benefit from it. If you have a follow-up assessment with fewer people, there is
greater success because they didn’t drop out.
Follow-up and adherence(=aanhankelijkheid) to protocol:
1. Choose participants who likely adhere to protocol
2. Make the intervention simple
3. Make study visits convenient and enjoyable
4. Make study measurements painless, useful, and interesting
5. Encourage participants to continue in the trial
6. Find participants who are lost to follow-up
3. Which data were considered?
Intention-to-treat would mean we take everybody. So an analysis of every participant
according to the randomized group assigned. Per-protocol means we only take those
who followed the way it was designed. So analysis of only participants who adhered to
the protocol. Mostly per-protocol the effect is larger. These results are however biased,
because people who are more at the treatments, will be more motivated.
Intention-to-treat also considers people who were not treated according to protocol
(because of drop out, missed treatment sessions, non-compliance, etc.). It likely
provides a more conservative estimation of treatment effects (i.e. tendency to
underestimate the full effect of a treatment).
4. Which analysis was taken?
Pre-assessment, post-assessment or follow-up assessment. They are statistically
interesting. Analysis of variance (ANOVA) looks at the main effect time, the main effect
group and the interaction time x group.
5. Is it clinically useful?
Statistical significance is not automatically clinical significance. Significance is an effect
larger than chance, it is better than doing nothing. It is not necessarily meaningful. The
goal is to maximize the functioning, not necessarily to solve the whole problem. The
mean score is less meaningful.
6. What is the comparison level?
, Advantage Disadvantage
No treatment
Simple and cheap; controls for time, Ethical issues; may lead to drop out; may
testing, regression to mean lead to independent treatment seeking
Waitlist
Guarantees treatment; some control for Ethical issues related to delaying
expectancy effects treatment; not indicated for long-term
follow-up
Placebo
Good control for expectancy effects; Ethical issues of irrelevant treatment; not
good control for nonspecific treatment double-blind in experience-based studies
effects; allows assessment of adverse
effects
NOTE: With placebo the question is if the new treatment is better than what is usually
done.
Usual care
Most acceptable to patients and Requires large N to achieve adequate
treaters; flexible to replace or power; usual care highly variable within
superimpose usual care and between institutions
Devised usual care
Guarantees minimum treatment; usually Requires large N to achieve adequate
acceptable to participants power; may not be acceptable to
participants if too minimal (too much
reduced)
NOTE: With devised usual care it is active treatment comparisons. So you look at the
time or the amount of treatments that are necessary. Everybody gets the treatment but
in slightly different ways. So the question is what made the difference. Or you want to
find a new one that is just as good as the old one, but you want a new one because of
money or etc.
, Dose control
More ethical than no treatment; Dose variation not always possible; limited
Examination of dose-response relationship hypothesis testing; requires large N to
achieve adequate power
Dismantling design= investigates therapies that have multiple components with the
goal of trying to identify those features of the therapy that are either the active
mechanisms of change or identify the degree to which specific components add to
the magnitude of change attributable to other components.
Acceptable to participants; precise Requires a priori knowledge of active
examination of active ingredients; for ingredients; requires large N to achieve
treatment with theoretical justification adequate power
Equivalence trial= compare a new treatment for a disease or condition with an existing
treatment, usually the standard one.
Acceptable to participants May compromise internal validity;
requires large N to achieve adequate
power; requires significant resources
Social robots in dementia care
Robot seal Paro. These pets give feedback, increasing awareness in the patient. It is a
treatment of anxiety and distressed behavior in dementia. It also requires attention and
care that distracts from other concerns. As a control condition you can just give them
anything, so a plant or something that needs them to take care of.
PICO framework is mnemonic in evidence-based practice to frame and answer any
clinical question (can also be used more universally for scientific questions of any
discipline). It stands for Patient, problem or population, Intervention, Comparison or
control condition and Outcome.
Why study processing speed?
Randomized controlled trials (RCT) are prospective studies that measure the
effectiveness of a new intervention or treatment. There are a few steps to take when
performing a RCT:
1. Selection of sample from a population
2. Assessment at baseline
3. Random assignment of intervention and control (blinded)
4. Following the cohort over time
5. Assessment of outcome
6. Assessment at follow up
It is important to be aware of what the control condition is doing. If possible the random
assignment should be blinded, this means that neither the examiner nor the participant
knows in which condition the participants are.
When the treatment is successful you need to ask a few questions:
1. How was success measured?
When looking at the neuropsychological tests, you can ask yourself in which cognitive
domain? Single tests of multivariate analysis (e.g. compound score)? Types of variables
(accuracy, speed, variability of performance)? When looking at the behavioral
observations, you can ask yourself by whom? In which situation? Or was it measured
with self- and informant reports?
It is also important to look at indications of functioning in daily life like job outcomes or
drug problems. It must be transferable to real life activities, for example with speed of
processing the training program is very much like the test condition. But you train the
test material, so becoming better in the test and not the actual speed of processing in
real life.
, 2. When was success measured?
Assessment at post-treatment or at follow up? What happened between
post-treatment assessment and follow up? Who are the people who drop out between
the outcome assessment and the follow-up assessment? The ones who remain are more
likely to benefit from it. If you have a follow-up assessment with fewer people, there is
greater success because they didn’t drop out.
Follow-up and adherence(=aanhankelijkheid) to protocol:
1. Choose participants who likely adhere to protocol
2. Make the intervention simple
3. Make study visits convenient and enjoyable
4. Make study measurements painless, useful, and interesting
5. Encourage participants to continue in the trial
6. Find participants who are lost to follow-up
3. Which data were considered?
Intention-to-treat would mean we take everybody. So an analysis of every participant
according to the randomized group assigned. Per-protocol means we only take those
who followed the way it was designed. So analysis of only participants who adhered to
the protocol. Mostly per-protocol the effect is larger. These results are however biased,
because people who are more at the treatments, will be more motivated.
Intention-to-treat also considers people who were not treated according to protocol
(because of drop out, missed treatment sessions, non-compliance, etc.). It likely
provides a more conservative estimation of treatment effects (i.e. tendency to
underestimate the full effect of a treatment).
4. Which analysis was taken?
Pre-assessment, post-assessment or follow-up assessment. They are statistically
interesting. Analysis of variance (ANOVA) looks at the main effect time, the main effect
group and the interaction time x group.
5. Is it clinically useful?
Statistical significance is not automatically clinical significance. Significance is an effect
larger than chance, it is better than doing nothing. It is not necessarily meaningful. The
goal is to maximize the functioning, not necessarily to solve the whole problem. The
mean score is less meaningful.
6. What is the comparison level?
, Advantage Disadvantage
No treatment
Simple and cheap; controls for time, Ethical issues; may lead to drop out; may
testing, regression to mean lead to independent treatment seeking
Waitlist
Guarantees treatment; some control for Ethical issues related to delaying
expectancy effects treatment; not indicated for long-term
follow-up
Placebo
Good control for expectancy effects; Ethical issues of irrelevant treatment; not
good control for nonspecific treatment double-blind in experience-based studies
effects; allows assessment of adverse
effects
NOTE: With placebo the question is if the new treatment is better than what is usually
done.
Usual care
Most acceptable to patients and Requires large N to achieve adequate
treaters; flexible to replace or power; usual care highly variable within
superimpose usual care and between institutions
Devised usual care
Guarantees minimum treatment; usually Requires large N to achieve adequate
acceptable to participants power; may not be acceptable to
participants if too minimal (too much
reduced)
NOTE: With devised usual care it is active treatment comparisons. So you look at the
time or the amount of treatments that are necessary. Everybody gets the treatment but
in slightly different ways. So the question is what made the difference. Or you want to
find a new one that is just as good as the old one, but you want a new one because of
money or etc.
, Dose control
More ethical than no treatment; Dose variation not always possible; limited
Examination of dose-response relationship hypothesis testing; requires large N to
achieve adequate power
Dismantling design= investigates therapies that have multiple components with the
goal of trying to identify those features of the therapy that are either the active
mechanisms of change or identify the degree to which specific components add to
the magnitude of change attributable to other components.
Acceptable to participants; precise Requires a priori knowledge of active
examination of active ingredients; for ingredients; requires large N to achieve
treatment with theoretical justification adequate power
Equivalence trial= compare a new treatment for a disease or condition with an existing
treatment, usually the standard one.
Acceptable to participants May compromise internal validity;
requires large N to achieve adequate
power; requires significant resources
Social robots in dementia care
Robot seal Paro. These pets give feedback, increasing awareness in the patient. It is a
treatment of anxiety and distressed behavior in dementia. It also requires attention and
care that distracts from other concerns. As a control condition you can just give them
anything, so a plant or something that needs them to take care of.
PICO framework is mnemonic in evidence-based practice to frame and answer any
clinical question (can also be used more universally for scientific questions of any
discipline). It stands for Patient, problem or population, Intervention, Comparison or
control condition and Outcome.
Why study processing speed?
