Week 2 module 1: meta-analysis
Difference between fixed MA and random Ma is the way the studies are weighted. They are pooled
the same way. The difference is that in the random we add Tau-squared.
Between study heterogeinity refers to the range of true effect sizes
Learning objectives:
- Run and interpret meta-analysis
- Determine the extent of publication bias
- Run and interpret subgroup analysis
- Create and interpret forest plots
The research question that is used in the example is: whether musicians perform better in memory
tasks then non musicians. They had three different kind op memories: working, short-term or long-
term. And three different kinds of stimuli type: tonal, verbal and visuospatial. They did a MA
separate for each type of memory. In the table K= the number of studies that is used.
Meta-analysis in Jasp look at the slides:
- Ad the button for meta-analysis at the +
- Setting up your MA data: usually it is already set-up. Each bit of data is manually entert from
the studies that had been included. And somebody go throw the studies to extract the
relevant information. The most important thing u need in your data set is the effect size.
Also we have study labels, wich we will use for the forest plot. And you need to think about
potential moderators (i.e. subgroup analyses). Sometimes you have to compute the effect
size that you want.
- U can select in wich one you are interesting in at the top of the variable click vinkje voor
degene die je wilt en kruisje voor degene die je niet wilt
- Meta-analyses menu: classical MA: effect size is hedges G, effect size standard error is SE,
method you can see the different types of MA. In drop down menu of statistics, select
estimetes and confidence interval.
Tables of the FIXED MA
Table 1: the top line indicates wether there is something significant in the second table. In this case
because we aren’’t looking at subgroup analyses it is just the intercept.
Jasp also gives us the between study heterogeinity in the second row in table one. So even though
we selected fixed effect MA it still gives us this test of between study heterogeinity.
Table 2: the intercept indicates our effect there, the estimate there.
Also gives confidence intervals for pooled effect on the right hand side of table 2.
, Tables of random MA
Are the same as fixed, but has one table more.
At first line of table one you can see something significantly different from 0 in table 2. Again
estimate intercept is the pooled effect for the random MA (table 2).
The Q is reported in table 1 not significant amound of between study heterogeinity.
Table 3 other between study heterogeinity parameters. (we look at I -squared). You also can see
the confidence intervals, because they selected this in the drop down menu. So we can see in the
table the confidence interval for I-squared also have a big range. However the Q was not significant
different and the I is moderate.
Forest plots
Most common figures to represent the data. There are important thing to know about the forest
plot:
- Each row refers to a study in the MA
- The Square is the observed effect in the study
- Information on the right about the weights, effect sizes, confidence interval etc. varies
between the forest plots.
- The vertical line in the middle represents the value of when there is not an effect. So when
there is no difference between the groups. So vertical line indicates the value for when there
is no effect.
- The lines around the squares are the confidence intervals. When they overlap zero it is not
an good thing, because the confidence intervals are then not significant, because they got a 0
between.
- The pooled effect is almost always noted with the diamond and the edges of the diamond
refers to the confidence interval. Dus hoe breeder the diamond is the bigger is the
confidence interval
- The squares also denotes the relative weight of the study by the size they have. So how
smaller the square how less the weight is.
Jasp does not always give you the weight of the studies in the forest plot, but you can get an idea
about that with the confidence intervals.
Publication bias
The MA is only as good as the data it is based on. If data is biased the results will be biased and the
pooled effect will be over or underestimated.
Some attempt to minimize publication bias by purposively looking in pre-print repositories
(artiekelen die nog niet gepubliceerd zijn) and theses but these are not peer-reviewed, and may
have other issues.
Publication biases is one of many reporting biases that can effect MA (language bias (only use studies
that they can read), citation bias (artiekelen that support the hypotheses are more likely to cited in
Difference between fixed MA and random Ma is the way the studies are weighted. They are pooled
the same way. The difference is that in the random we add Tau-squared.
Between study heterogeinity refers to the range of true effect sizes
Learning objectives:
- Run and interpret meta-analysis
- Determine the extent of publication bias
- Run and interpret subgroup analysis
- Create and interpret forest plots
The research question that is used in the example is: whether musicians perform better in memory
tasks then non musicians. They had three different kind op memories: working, short-term or long-
term. And three different kinds of stimuli type: tonal, verbal and visuospatial. They did a MA
separate for each type of memory. In the table K= the number of studies that is used.
Meta-analysis in Jasp look at the slides:
- Ad the button for meta-analysis at the +
- Setting up your MA data: usually it is already set-up. Each bit of data is manually entert from
the studies that had been included. And somebody go throw the studies to extract the
relevant information. The most important thing u need in your data set is the effect size.
Also we have study labels, wich we will use for the forest plot. And you need to think about
potential moderators (i.e. subgroup analyses). Sometimes you have to compute the effect
size that you want.
- U can select in wich one you are interesting in at the top of the variable click vinkje voor
degene die je wilt en kruisje voor degene die je niet wilt
- Meta-analyses menu: classical MA: effect size is hedges G, effect size standard error is SE,
method you can see the different types of MA. In drop down menu of statistics, select
estimetes and confidence interval.
Tables of the FIXED MA
Table 1: the top line indicates wether there is something significant in the second table. In this case
because we aren’’t looking at subgroup analyses it is just the intercept.
Jasp also gives us the between study heterogeinity in the second row in table one. So even though
we selected fixed effect MA it still gives us this test of between study heterogeinity.
Table 2: the intercept indicates our effect there, the estimate there.
Also gives confidence intervals for pooled effect on the right hand side of table 2.
, Tables of random MA
Are the same as fixed, but has one table more.
At first line of table one you can see something significantly different from 0 in table 2. Again
estimate intercept is the pooled effect for the random MA (table 2).
The Q is reported in table 1 not significant amound of between study heterogeinity.
Table 3 other between study heterogeinity parameters. (we look at I -squared). You also can see
the confidence intervals, because they selected this in the drop down menu. So we can see in the
table the confidence interval for I-squared also have a big range. However the Q was not significant
different and the I is moderate.
Forest plots
Most common figures to represent the data. There are important thing to know about the forest
plot:
- Each row refers to a study in the MA
- The Square is the observed effect in the study
- Information on the right about the weights, effect sizes, confidence interval etc. varies
between the forest plots.
- The vertical line in the middle represents the value of when there is not an effect. So when
there is no difference between the groups. So vertical line indicates the value for when there
is no effect.
- The lines around the squares are the confidence intervals. When they overlap zero it is not
an good thing, because the confidence intervals are then not significant, because they got a 0
between.
- The pooled effect is almost always noted with the diamond and the edges of the diamond
refers to the confidence interval. Dus hoe breeder the diamond is the bigger is the
confidence interval
- The squares also denotes the relative weight of the study by the size they have. So how
smaller the square how less the weight is.
Jasp does not always give you the weight of the studies in the forest plot, but you can get an idea
about that with the confidence intervals.
Publication bias
The MA is only as good as the data it is based on. If data is biased the results will be biased and the
pooled effect will be over or underestimated.
Some attempt to minimize publication bias by purposively looking in pre-print repositories
(artiekelen die nog niet gepubliceerd zijn) and theses but these are not peer-reviewed, and may
have other issues.
Publication biases is one of many reporting biases that can effect MA (language bias (only use studies
that they can read), citation bias (artiekelen that support the hypotheses are more likely to cited in
