Module 1.1
Hoorcollege 1: meta-analysis
By the end of this week, you should be able to:
1. Clearly define a research question
2. Describe how to apply a systematic approach to finding articles for a research
question
3. Describe the different types of meta-analyses
4. Describe between-study heterogeneity parameters
What is a meta-analysis: an analysis of analysis. All data in a MA are the results of earlier studies.
Systematic reviews: finding all related evidence in a transparent way with clearly defined criteria, and
describing this evidence.
Meta-analysis: start off in the same way as a systematic review, but then we aim to combine,
summarise and interpret all evidence relating to our research question in a quantitative way.
A systematic review is a little bit more descriptive and in a MA we want to be able to quantify all the
evidence.
What we usually look at in a MA are effect sizes. Effect size → metric to quantify both direction and
magnitude of the relationship between two variables.
For both (MA and systematic review) it starts of with a clear defined research question. This can be
done with PICO. The PICO principle assists you in organizing and focusing your question into a
searchable query. So it also helps in the MA to formulate search strategy for studies to include and to
help define inclusion and exclusion criteria.
The systematic approach (including clear reasons why studies were included or excluded) in a meta-
analysis is important so it is reproducible and transparent.
, To ensure all relevant information is included in reporting, most use the PRISMA checklist (PRISMA:
Preferred Reporting Items for Systematic Reviews and Meta-analyses).
To minimize the articles that will be used in a meta-analyses→ this is often done by at least two
researchers independently (or a subsample by a second researcher). So both need to come to the
same articles that will be used.
Hedge’s G is an effect size and it is similar to Cohen’s D. and It devolves the means and SD of your
intervention group and your control group. However Hedge’s G takes account for the fact that your
sample sizes for the control group and the intervention group might not be the same. And in this
case for Hedge’s G the lower the value (so more negative), the larger the reduction or the less
depressive symptoms in the exercise intervention compared to the controlled intervention. So the
more negative in Hedge’s G the more effective the exercise intervention was. (the research question
was if exercise intervention can be used to reduce symptoms of depression).
Fixed-effect meta-analysis
Assumption: there is one true underlying effect in all the studies. So all the studies are trying to
estimate this one true effect.
This means that any variation across studies must be due to sampling error, because when there was
no sampling error they should all report the same effect sizes of this one true underlying effect.
the larger the sample (n), the lower the standard error or sampling error. Standard error is an
measure of the sampling error. You can also see this in the formula.
So this means that studies with a lower standard error must reported an effect that is closer to the
true effect then studies with a relative large standard error, because then there is more sampling
error. And therefore when we want to come up with an overall effect we need to pool the effect
sizes. We want studies that have a better position, so studies with a lower standard error have more
weight then studies where the positions are less good.
To determine the weight of a study it Is 1 divided by the standard error squared for that study. So
when it is 1 divided by the standard error squared. You can see when it is smaller your value would
be larger then when the standard error is larger.
The sum of weighted effect size of each study we add this up and we divided it by the sum of all the
individual weight. And it is divided by the sum of all the individual weights, because other wise your
pooled effect would be keep going up when you add studies.
Note: there is an alternative method for binary effect size data, but most effect sizes are continuous.
So you will see that most often the method of inverse-variance weighting will be used.
Hoorcollege 1: meta-analysis
By the end of this week, you should be able to:
1. Clearly define a research question
2. Describe how to apply a systematic approach to finding articles for a research
question
3. Describe the different types of meta-analyses
4. Describe between-study heterogeneity parameters
What is a meta-analysis: an analysis of analysis. All data in a MA are the results of earlier studies.
Systematic reviews: finding all related evidence in a transparent way with clearly defined criteria, and
describing this evidence.
Meta-analysis: start off in the same way as a systematic review, but then we aim to combine,
summarise and interpret all evidence relating to our research question in a quantitative way.
A systematic review is a little bit more descriptive and in a MA we want to be able to quantify all the
evidence.
What we usually look at in a MA are effect sizes. Effect size → metric to quantify both direction and
magnitude of the relationship between two variables.
For both (MA and systematic review) it starts of with a clear defined research question. This can be
done with PICO. The PICO principle assists you in organizing and focusing your question into a
searchable query. So it also helps in the MA to formulate search strategy for studies to include and to
help define inclusion and exclusion criteria.
The systematic approach (including clear reasons why studies were included or excluded) in a meta-
analysis is important so it is reproducible and transparent.
, To ensure all relevant information is included in reporting, most use the PRISMA checklist (PRISMA:
Preferred Reporting Items for Systematic Reviews and Meta-analyses).
To minimize the articles that will be used in a meta-analyses→ this is often done by at least two
researchers independently (or a subsample by a second researcher). So both need to come to the
same articles that will be used.
Hedge’s G is an effect size and it is similar to Cohen’s D. and It devolves the means and SD of your
intervention group and your control group. However Hedge’s G takes account for the fact that your
sample sizes for the control group and the intervention group might not be the same. And in this
case for Hedge’s G the lower the value (so more negative), the larger the reduction or the less
depressive symptoms in the exercise intervention compared to the controlled intervention. So the
more negative in Hedge’s G the more effective the exercise intervention was. (the research question
was if exercise intervention can be used to reduce symptoms of depression).
Fixed-effect meta-analysis
Assumption: there is one true underlying effect in all the studies. So all the studies are trying to
estimate this one true effect.
This means that any variation across studies must be due to sampling error, because when there was
no sampling error they should all report the same effect sizes of this one true underlying effect.
the larger the sample (n), the lower the standard error or sampling error. Standard error is an
measure of the sampling error. You can also see this in the formula.
So this means that studies with a lower standard error must reported an effect that is closer to the
true effect then studies with a relative large standard error, because then there is more sampling
error. And therefore when we want to come up with an overall effect we need to pool the effect
sizes. We want studies that have a better position, so studies with a lower standard error have more
weight then studies where the positions are less good.
To determine the weight of a study it Is 1 divided by the standard error squared for that study. So
when it is 1 divided by the standard error squared. You can see when it is smaller your value would
be larger then when the standard error is larger.
The sum of weighted effect size of each study we add this up and we divided it by the sum of all the
individual weight. And it is divided by the sum of all the individual weights, because other wise your
pooled effect would be keep going up when you add studies.
Note: there is an alternative method for binary effect size data, but most effect sizes are continuous.
So you will see that most often the method of inverse-variance weighting will be used.
