Evidence-based interventions (PSMKB-1)
Week 1: Introduction and finding evidence: sources
1. Schulz, K.F., Altman, D.G., & Moher, D. (2010). CONSORT 2010
Statement: Updated guidelines for reporting parallel group
randomized trials. Journal of Clinical Epidemiology, 63, 834-840.
doi.org/10.1186/1745-6215-11-32
Aim of CONSORT statement: improve reporting of randomized controlled trials. The figure and table
provide guidance for reporting all randomized controlled trials.
The CONSORT 2010 Statement addresses the minimum criteria. With CONSORT 2010, they
intentionally declined to produce a rigid structure for the reporting of randomized trials.
2. Silverman, W., Pina, A., &
Viswesvaran, C. (2008).
Evidence-based
psychosocial treatments for phobic and anxiety disorders in children
and adolescents. Journal of Clinical Child and Adolescent Psychology,
37, 105-130. doi.org/10.1080/15374410701817907
The article reviews psychosocial treatments for phobic and anxiety disorders in youth.
6 types of treatment studies according to Nathan and Gorman’s criteria (2002);
1. Type 1 studies; most rigorous. Involve randomized, prospective clinical trial methodology.
Involve comparisons groups with random assignment, blind assessment, clear presentation
of the study’s inclusion and exclusion criteria, state-of-the art diagnostic methods, adequate
sample size to offer statistical power and clearly described statistical methods.
2. Type 2 studies; clinical trials in which the intervention is tested but at least one aspect of the
Type 1 study requirement is missing.
, 3. Type 3 studies; methodologically limited. Open trials aimed at obtaining pilot data and are
thereby subject to observer bias, but they still indicate whether the treatment would be
worth pursuing using a more rigorous design.
4. Type 4 studies; reviews with secondary data analyses such as meta-analyses.
5. Type 5 studies; reviews that do not include secondary data analyses.
6. Type 6 studies; case studies, essays and opinion papers.
Treatment classifications (according to Chambless and Hollon, 1998);
- Well-established treatments=
There must be at least two good group-design experiments, conducted in at least two
independence research settings and by independent investigatory teams, demonstrating
efficacy by showing the treatment to be: a) statistically significantly superior to pill or
psychological placebo or to another treatment OR b) equivalent (or not significantly
different) to an already established treatment in experiments with statistical power
being sufficient to detect moderate differences.
AND;
Treatment manuals or logical equivalent were used for the treatment
Conducted with a population, treated for specified problems, for whom inclusion
criteria have been delineated in a reliable, valid manner
Reliable and valid outcome assessment measures, at minimum tapping the problems
targeted for change were used
Appropriate data analyses
- Probably efficacious treatments=
There must be at least two good experiments showing the treatment is superior
(statistically significantly so) to a wait-list control group OR one or more good
experiments meeting the well-established treatment criteria with the one exception of
having been conducted in at least two independent research settings and by
independent investigatory teams
- Possibly efficacious treatments= at least one ‘good’ study showing the treatment to be
efficacious in the absence of conflicting evidence
- Experimental treatments= treatment not yet tested in trials meeting task force criteria for
methodology
Summary of studies;
- Most of the studies were RCT’s and were methodologically robust or fairly rigorous
- Most studies were Type 1 or Type 2
- No treatment was well-established
- Probably efficacious: Individual Cognitive Behavior Therapy, Group Cognitive Behavior
Therapy (GCBT), GCBT with Parents, GCBT for social phobia (SOP), and Social Effectiveness
Training for children with SOP
- The other treatments were either possibly efficacious or experimental
- Individual child and adolescent treatments did not result in better outcomes than group
treatments for either anxiety reduction or reduction of other symptoms
- Cognitive behavioral treatments, in individual or group formats, with and without parent
involvement, lead to positive treatment response in children and adolescents with phobic
and anxiety disorders
Unresolves issues:
, - Establishing well-established treatments
- Moving beyond waitlist control conditions; further research should move beyond waitlists
and examines participants’ utilization of specific treatment strategies
- Improving measurement; the lack of significant differences could be due in part to
insensitivity of the existing measures in detecting the specific skills that are being targeted in
treatment programs. Another measurement problem relates to the assumption of metric
equivalence (=whether the items on a scale have the same meaning across groups).
- Enhancing statistical power
- Handling treatment non-completers, missing data and outliers; outcome literature has paid
little/no attention to issues relating to the handling of missing data and outliers. These
factors can substantially alter studies’ findings and thus the conclusions.
- Dismantling strategies; moderator and mediator analyses
- Increasing attention to developmental issues
- Increasing minority representation; investigators need to carefully reconsider how they
frame the problem of excessive fear and anxiety to minority groups even as early as the
study’s recruitment phase. Once minorities are enrolled in the study, it is incumbent on
investigators to adapt and modify existing CBT programs in ways that are more sensitively
attuned to the cultural context of the minority group.
- Integrating pharmacological treatment of anxiety disorders
3. Menzies, D. (2011). Systematic reviews and meta-analyses. The
International Journal of Tuberculosis and Lung Disease, 5, 582 – 593.
doi.org/10.5588/ijtld.10.0719
The article describes the principles for conducting a systematic review and methods of summarizing
information (including meta-analysis).
PICO format Population, Intervention, Comparison, Outcomes.
Meta-analysis Process of pooling estimates of effect from different studies into one
summary estimate.
Systematic reviews are preferred over narrative reviews because they represent a formal,
reproducible and potentially less biased approach to summarizing published information.
Simplified guide to understanding systematic reviews:
1. Formulate the study question; should be developed using the PICO format (Table 1).
2. Study selection criteria (Table 2)
Cohort studies advantages: broader spectrum of patients who might have been excluded
from randomized trials findings from cohort studies are more applicable to real
practice. However, cohort studies are also subject to selection bias.
Study selection criteria should follow the same PICO format, i.e., selecting studies that
included the population identified by the study questions, as well as the interventions,
comparisons, and outcomes specified by the study question.
3. Search strategy (Table 3)
Key words should be broad enough to include almost all relevant studies on the topic
Include as many languages as possible
Publication bias: when an intervention is found to be effective, they are more likely to be
submitted by authors and accepted by journals for publication. Studies that do not find a
, significant association are less likely to be published. Including unpublished studies may
balance the publication bias effect.
4. Selection of studies: from titles to full text (Table 4)
First review of the titles alone
Second review all abstracts
Lastly review the full text
Double-blind study Neither the patients nor the treating team should be aware of the
intervention.
Approaches to minimizing problems of heterogeneity;
- Ensure that the pooled estimates are made with random effects models
- Perform meta-regression in which pooled estimates of effect are adjusted for covariates that
are different between studies and thus may contribute to differences in crude effect
estimates between studies
- Perform stratified analyses in which pooled estimates of effect are performed within sub-
groups defined by values of a covariate that has an important effect on estimates of effect
Week 1: Introduction and finding evidence: sources
1. Schulz, K.F., Altman, D.G., & Moher, D. (2010). CONSORT 2010
Statement: Updated guidelines for reporting parallel group
randomized trials. Journal of Clinical Epidemiology, 63, 834-840.
doi.org/10.1186/1745-6215-11-32
Aim of CONSORT statement: improve reporting of randomized controlled trials. The figure and table
provide guidance for reporting all randomized controlled trials.
The CONSORT 2010 Statement addresses the minimum criteria. With CONSORT 2010, they
intentionally declined to produce a rigid structure for the reporting of randomized trials.
2. Silverman, W., Pina, A., &
Viswesvaran, C. (2008).
Evidence-based
psychosocial treatments for phobic and anxiety disorders in children
and adolescents. Journal of Clinical Child and Adolescent Psychology,
37, 105-130. doi.org/10.1080/15374410701817907
The article reviews psychosocial treatments for phobic and anxiety disorders in youth.
6 types of treatment studies according to Nathan and Gorman’s criteria (2002);
1. Type 1 studies; most rigorous. Involve randomized, prospective clinical trial methodology.
Involve comparisons groups with random assignment, blind assessment, clear presentation
of the study’s inclusion and exclusion criteria, state-of-the art diagnostic methods, adequate
sample size to offer statistical power and clearly described statistical methods.
2. Type 2 studies; clinical trials in which the intervention is tested but at least one aspect of the
Type 1 study requirement is missing.
, 3. Type 3 studies; methodologically limited. Open trials aimed at obtaining pilot data and are
thereby subject to observer bias, but they still indicate whether the treatment would be
worth pursuing using a more rigorous design.
4. Type 4 studies; reviews with secondary data analyses such as meta-analyses.
5. Type 5 studies; reviews that do not include secondary data analyses.
6. Type 6 studies; case studies, essays and opinion papers.
Treatment classifications (according to Chambless and Hollon, 1998);
- Well-established treatments=
There must be at least two good group-design experiments, conducted in at least two
independence research settings and by independent investigatory teams, demonstrating
efficacy by showing the treatment to be: a) statistically significantly superior to pill or
psychological placebo or to another treatment OR b) equivalent (or not significantly
different) to an already established treatment in experiments with statistical power
being sufficient to detect moderate differences.
AND;
Treatment manuals or logical equivalent were used for the treatment
Conducted with a population, treated for specified problems, for whom inclusion
criteria have been delineated in a reliable, valid manner
Reliable and valid outcome assessment measures, at minimum tapping the problems
targeted for change were used
Appropriate data analyses
- Probably efficacious treatments=
There must be at least two good experiments showing the treatment is superior
(statistically significantly so) to a wait-list control group OR one or more good
experiments meeting the well-established treatment criteria with the one exception of
having been conducted in at least two independent research settings and by
independent investigatory teams
- Possibly efficacious treatments= at least one ‘good’ study showing the treatment to be
efficacious in the absence of conflicting evidence
- Experimental treatments= treatment not yet tested in trials meeting task force criteria for
methodology
Summary of studies;
- Most of the studies were RCT’s and were methodologically robust or fairly rigorous
- Most studies were Type 1 or Type 2
- No treatment was well-established
- Probably efficacious: Individual Cognitive Behavior Therapy, Group Cognitive Behavior
Therapy (GCBT), GCBT with Parents, GCBT for social phobia (SOP), and Social Effectiveness
Training for children with SOP
- The other treatments were either possibly efficacious or experimental
- Individual child and adolescent treatments did not result in better outcomes than group
treatments for either anxiety reduction or reduction of other symptoms
- Cognitive behavioral treatments, in individual or group formats, with and without parent
involvement, lead to positive treatment response in children and adolescents with phobic
and anxiety disorders
Unresolves issues:
, - Establishing well-established treatments
- Moving beyond waitlist control conditions; further research should move beyond waitlists
and examines participants’ utilization of specific treatment strategies
- Improving measurement; the lack of significant differences could be due in part to
insensitivity of the existing measures in detecting the specific skills that are being targeted in
treatment programs. Another measurement problem relates to the assumption of metric
equivalence (=whether the items on a scale have the same meaning across groups).
- Enhancing statistical power
- Handling treatment non-completers, missing data and outliers; outcome literature has paid
little/no attention to issues relating to the handling of missing data and outliers. These
factors can substantially alter studies’ findings and thus the conclusions.
- Dismantling strategies; moderator and mediator analyses
- Increasing attention to developmental issues
- Increasing minority representation; investigators need to carefully reconsider how they
frame the problem of excessive fear and anxiety to minority groups even as early as the
study’s recruitment phase. Once minorities are enrolled in the study, it is incumbent on
investigators to adapt and modify existing CBT programs in ways that are more sensitively
attuned to the cultural context of the minority group.
- Integrating pharmacological treatment of anxiety disorders
3. Menzies, D. (2011). Systematic reviews and meta-analyses. The
International Journal of Tuberculosis and Lung Disease, 5, 582 – 593.
doi.org/10.5588/ijtld.10.0719
The article describes the principles for conducting a systematic review and methods of summarizing
information (including meta-analysis).
PICO format Population, Intervention, Comparison, Outcomes.
Meta-analysis Process of pooling estimates of effect from different studies into one
summary estimate.
Systematic reviews are preferred over narrative reviews because they represent a formal,
reproducible and potentially less biased approach to summarizing published information.
Simplified guide to understanding systematic reviews:
1. Formulate the study question; should be developed using the PICO format (Table 1).
2. Study selection criteria (Table 2)
Cohort studies advantages: broader spectrum of patients who might have been excluded
from randomized trials findings from cohort studies are more applicable to real
practice. However, cohort studies are also subject to selection bias.
Study selection criteria should follow the same PICO format, i.e., selecting studies that
included the population identified by the study questions, as well as the interventions,
comparisons, and outcomes specified by the study question.
3. Search strategy (Table 3)
Key words should be broad enough to include almost all relevant studies on the topic
Include as many languages as possible
Publication bias: when an intervention is found to be effective, they are more likely to be
submitted by authors and accepted by journals for publication. Studies that do not find a
, significant association are less likely to be published. Including unpublished studies may
balance the publication bias effect.
4. Selection of studies: from titles to full text (Table 4)
First review of the titles alone
Second review all abstracts
Lastly review the full text
Double-blind study Neither the patients nor the treating team should be aware of the
intervention.
Approaches to minimizing problems of heterogeneity;
- Ensure that the pooled estimates are made with random effects models
- Perform meta-regression in which pooled estimates of effect are adjusted for covariates that
are different between studies and thus may contribute to differences in crude effect
estimates between studies
- Perform stratified analyses in which pooled estimates of effect are performed within sub-
groups defined by values of a covariate that has an important effect on estimates of effect
