LITERATURE REVIEW MPA What is literature review?
LECTURE 2
It is a critical assessment of published literature on a particular
Eduardo Urias topic.
It is not a list like an annotated bibliography in which a summary
of each source is listed one by one
A literature review requires you to think critically
> “Proper thinking is about forming an argument or a critical analysis that you
can back up with evidence and reinforce with appropriate examples” (Swain,
2009: 12)
> Strengths and weaknesses of existing research
> Why and how you think their ideas or theories might be improved.
Swain, H. (2009). How to be a student No. 61: The art of learning how to think, The Guardian: Education,
10 March 2009. Available at https://www.theguardian.com/education/2009/mar/10/students-higher-
education
0
difference with quantitative/qualitative research is you don’t collect primary data
A critical assessment Critical Approach but your data is from previous studies
It is a critical assessment of published literature on a particular
topic.
> Synthesis is a key aspect!
> Classification, comparison and/or evaluation of the current
beyond summarizing = like a puzzle to make own point/contribution
knowledge
substantive findings, issues and themes = explain outcome of
Populations, interventions, outcomes synthesis
theoretical contributions
methodological contributions
Open questions (avenues for further investigation) = they questions not answered yet
Contradictions Review must compare and contrast the existing views
> Holistic perspective on the topic
> Combining knowledge and understanding what has been written
> More than just an annotated bibliography
, Why write a review? Why write a review?
There must be a need for the review Two general kinds of topics
> Mature topics
Literature review must contribute to the existing knowledge base Large corpus
> Creating a new dimension or fresh perspectives, e.g. Need for a review, critique and potential reconceptualization
Research agenda Ambiguities, contradictions, and gaps in the knowledge base
What works e.g. findings based on different methods
Taxonomy > New/Emerging topics
Alternative model / conceptual framework promotes a new way of thinking about a topic Small corpus
Initial or preliminary conceptualisation (i.e. new model, framework)
> Keep in mind: it is not just a summary! Synthesis of the literature to date
often students make the mistake to search for a research gap, but since a literature review is based on current literature
it’s counterintuitive that a gap can be filled. However with a literature review research gaps can be found.
Types of literature reviews
• more narrow focus with a precise research question and
objective
• search aims to include all relevant studies
• often broad aim to describe a field and no structural method
• predetermined inclusion and exclusion criteria
• selecting studies often based on leads in literature = purposive
• quality assessment done by following protocols or checklists
selecting and decided what to include by the reviewer
• analysis and synthesis are often done in tabular form and short
• quality assessment also not following a structured method, and
summary answers
when performed based on the reviewers opinion
• methodological report is presented for transparency and
• analysis often discursive and no methodological report is written
reproductivity
there is a variety of reviews
• diversion of terminology often leads to
confusion and misused terms
• SALSA framework can be used to
determine specific type of review
know for exam 4 key types + key
characteristics + strengths and
weaknesses
Source: Aveyard, H. (2014). Doing a literature review in health and social care: A practical
guide. McGraw-Hill Education (UK).
,Traditional versus systematic review Systematic Review - Key phases
= part of inclusion
and exclusion criteria
Source: Jesson, J., Matheson, L., & Lacey, F. M. (2011). Doing your literature review: Source: Jesson, J., Matheson, L., & Lacey, F. M. (2011). Doing your literature review:
Traditional and systematic techniques. Sage. Traditional and systematic techniques. Sage.
Many people mistakenly use the terms systematic analysis and
meta-analysis interchangeably, but a systematic analysis (or Using the distinction between narrative/traditional and systematic reviews implies
Hierarch of Evidence systematic review) refers to the structured process of collecting
and critically evaluating all relevant studies on a topic, whereas Types of review the message that systematic reviews are of higher quality. Where the distinction
between configuring and aggregating focuses more on the different aims and not
a quality statement can be made based on this.
a meta-analysis is a mathematical or statistical analysis that
combines comparable data from those studies. Therefore,
researchers must first determine whether the extracted data are
similar and feasible enough to perform a meta-analysis.
so homogenous data needed for meta-analysis
the rate of development of new approaches to reviewing is
too fast and the overlap of approaches too great for that to
be helpful. (Gough et al., 2012) —> they critique the use of the many different names
Key dimensions on which reviews differ:
they suggest using a more simple distinction in a
continuum between configuring reviews to aggregating
reviews —> more flexible
• configuring = interpret and understand, new topics
• aggregating = collecting empirical data to describe
and test predefined concepts - adding to make
empirical statements. —> about seeking evidence
to inform decision-making
Gough, D., Thomas, J., & Oliver, S. (2012).
Clarifying differences between review designs
and methods. Systematic reviews, 1(1), 28.
, “Systematic Review” x “Systematic Approach” Types of literature reviews
Type of review Short description
Systematic review: reviewers follow a strict protocol
Narrative review Reports the author’s findings in a condensed format that typically summarizes the contents of each
article
however, often we want to focus on using a systematic approach in order to write the review:
Explicit and rigorous methods to identify, critically appraise, and Critical review Demonstrate extensive literature research and critically evaluate the quality. Includes degree of
analysis and conceptual innovation. Typically results in hypothesis or model
synthesize relevant studies in order to answer a predefined Integrative review Broadest type of research review methods (includes experimental and non-experimental research)
May combine data from theoretical as well as empirical data
question. Mapping review Map+categorize existing literature to plan further reviews and/or research by identifying gaps
> Comprehensive searching strategy
Mixed methods Combination of review approaches for example combining quantitative with qualitative research
> develop inclusion and exclusion criteria
> Critical appraisal to ensure that only high quality papers are included. Overview Generic term: summary of the literature that attempts to survey data and describe its characteristics
> the findings of all the papers are then pulled together and combined using a Scoping review Preliminary assessment of potential size and scope of available research literature. Aims to identify
nature and extent of research evidence (usually including ongoing research)
systematic approach.
State-of-the art review Tend to address more current matters in contrast to other combined retrospective and current
approaches. May offer new perspectives on issue or point out area for further research
Systematic search and Combines strengths of critical review with a comprehensive search process. Typically addresses broad
review questions to produce ‘best evidence synthesis’
Meta-analysis Technique that statistically combines the results of quantitative studies to provide a more precise
Critical appraisal is different from critical assessment in that critical appraisal involves systematically effect of the results
evaluating the quality, validity, and reliability of research evidence, whereas critical assessment focuses
more broadly on identifying and discussing the strengths and weaknesses of a study or argument + Source: Grant, Maria J., and Andrew Booth. "A typology of reviews: an analysis of 14 review types and associated
contrast and compare to make sure the result section is not just a summary. methodologies." Health Information & Libraries Journal 26.2 (2009): 91-108.
11
Vrije Universiteit Amsterdam
Types of literature reviews
LITERATURE REVIEW
Type of review Search type Appraisal MPA 2020-2021
Narrative review Broad, often non-documented No, but some data on study design is
needed Evelien de Hoop
Critical review Seeks to identify the most significant items in the field No, evaluates by contribution
Integrative review Comprehensive search to identify maximum primary sources, using 2 or Reports coded by quality, but not Lecture 3: Review Question & Exploratory Search
more strategies. Purposive sampling may be combined with excluded
comprehensive search if appropriate
Mapping review Completeness of searching determined by time/scope constraints No
Mixed methods Requires either very sensitive search to retrieve all studies or separately Generic appraisal tool or checklists
conceived quantitative and qualitative strategies
Overview May or may not include comprehensive searching (depends whether Depends on how systematic methods
systematic overview or not) are
Scoping review Completeness of searching determined by time/scope constraints. May No
include research in progress
State-of-the art Aims for comprehensive searching of current literature No
review
Systematic search Aims for exhaustive, comprehensive searching Yes
and review
Meta-analysis Aims for exhaustive, comprehensive searching. May use funnel plot to May use to determine inclusion
assess completeness 0
Source: Grant, Maria J., and Andrew Booth. "A typology of reviews: an analysis of 14 review types
and associated methodologies." Health Information & Libraries Journal 26.2 (2009): 91-108.
Vrije Universiteit Amsterdam
LECTURE 2
It is a critical assessment of published literature on a particular
Eduardo Urias topic.
It is not a list like an annotated bibliography in which a summary
of each source is listed one by one
A literature review requires you to think critically
> “Proper thinking is about forming an argument or a critical analysis that you
can back up with evidence and reinforce with appropriate examples” (Swain,
2009: 12)
> Strengths and weaknesses of existing research
> Why and how you think their ideas or theories might be improved.
Swain, H. (2009). How to be a student No. 61: The art of learning how to think, The Guardian: Education,
10 March 2009. Available at https://www.theguardian.com/education/2009/mar/10/students-higher-
education
0
difference with quantitative/qualitative research is you don’t collect primary data
A critical assessment Critical Approach but your data is from previous studies
It is a critical assessment of published literature on a particular
topic.
> Synthesis is a key aspect!
> Classification, comparison and/or evaluation of the current
beyond summarizing = like a puzzle to make own point/contribution
knowledge
substantive findings, issues and themes = explain outcome of
Populations, interventions, outcomes synthesis
theoretical contributions
methodological contributions
Open questions (avenues for further investigation) = they questions not answered yet
Contradictions Review must compare and contrast the existing views
> Holistic perspective on the topic
> Combining knowledge and understanding what has been written
> More than just an annotated bibliography
, Why write a review? Why write a review?
There must be a need for the review Two general kinds of topics
> Mature topics
Literature review must contribute to the existing knowledge base Large corpus
> Creating a new dimension or fresh perspectives, e.g. Need for a review, critique and potential reconceptualization
Research agenda Ambiguities, contradictions, and gaps in the knowledge base
What works e.g. findings based on different methods
Taxonomy > New/Emerging topics
Alternative model / conceptual framework promotes a new way of thinking about a topic Small corpus
Initial or preliminary conceptualisation (i.e. new model, framework)
> Keep in mind: it is not just a summary! Synthesis of the literature to date
often students make the mistake to search for a research gap, but since a literature review is based on current literature
it’s counterintuitive that a gap can be filled. However with a literature review research gaps can be found.
Types of literature reviews
• more narrow focus with a precise research question and
objective
• search aims to include all relevant studies
• often broad aim to describe a field and no structural method
• predetermined inclusion and exclusion criteria
• selecting studies often based on leads in literature = purposive
• quality assessment done by following protocols or checklists
selecting and decided what to include by the reviewer
• analysis and synthesis are often done in tabular form and short
• quality assessment also not following a structured method, and
summary answers
when performed based on the reviewers opinion
• methodological report is presented for transparency and
• analysis often discursive and no methodological report is written
reproductivity
there is a variety of reviews
• diversion of terminology often leads to
confusion and misused terms
• SALSA framework can be used to
determine specific type of review
know for exam 4 key types + key
characteristics + strengths and
weaknesses
Source: Aveyard, H. (2014). Doing a literature review in health and social care: A practical
guide. McGraw-Hill Education (UK).
,Traditional versus systematic review Systematic Review - Key phases
= part of inclusion
and exclusion criteria
Source: Jesson, J., Matheson, L., & Lacey, F. M. (2011). Doing your literature review: Source: Jesson, J., Matheson, L., & Lacey, F. M. (2011). Doing your literature review:
Traditional and systematic techniques. Sage. Traditional and systematic techniques. Sage.
Many people mistakenly use the terms systematic analysis and
meta-analysis interchangeably, but a systematic analysis (or Using the distinction between narrative/traditional and systematic reviews implies
Hierarch of Evidence systematic review) refers to the structured process of collecting
and critically evaluating all relevant studies on a topic, whereas Types of review the message that systematic reviews are of higher quality. Where the distinction
between configuring and aggregating focuses more on the different aims and not
a quality statement can be made based on this.
a meta-analysis is a mathematical or statistical analysis that
combines comparable data from those studies. Therefore,
researchers must first determine whether the extracted data are
similar and feasible enough to perform a meta-analysis.
so homogenous data needed for meta-analysis
the rate of development of new approaches to reviewing is
too fast and the overlap of approaches too great for that to
be helpful. (Gough et al., 2012) —> they critique the use of the many different names
Key dimensions on which reviews differ:
they suggest using a more simple distinction in a
continuum between configuring reviews to aggregating
reviews —> more flexible
• configuring = interpret and understand, new topics
• aggregating = collecting empirical data to describe
and test predefined concepts - adding to make
empirical statements. —> about seeking evidence
to inform decision-making
Gough, D., Thomas, J., & Oliver, S. (2012).
Clarifying differences between review designs
and methods. Systematic reviews, 1(1), 28.
, “Systematic Review” x “Systematic Approach” Types of literature reviews
Type of review Short description
Systematic review: reviewers follow a strict protocol
Narrative review Reports the author’s findings in a condensed format that typically summarizes the contents of each
article
however, often we want to focus on using a systematic approach in order to write the review:
Explicit and rigorous methods to identify, critically appraise, and Critical review Demonstrate extensive literature research and critically evaluate the quality. Includes degree of
analysis and conceptual innovation. Typically results in hypothesis or model
synthesize relevant studies in order to answer a predefined Integrative review Broadest type of research review methods (includes experimental and non-experimental research)
May combine data from theoretical as well as empirical data
question. Mapping review Map+categorize existing literature to plan further reviews and/or research by identifying gaps
> Comprehensive searching strategy
Mixed methods Combination of review approaches for example combining quantitative with qualitative research
> develop inclusion and exclusion criteria
> Critical appraisal to ensure that only high quality papers are included. Overview Generic term: summary of the literature that attempts to survey data and describe its characteristics
> the findings of all the papers are then pulled together and combined using a Scoping review Preliminary assessment of potential size and scope of available research literature. Aims to identify
nature and extent of research evidence (usually including ongoing research)
systematic approach.
State-of-the art review Tend to address more current matters in contrast to other combined retrospective and current
approaches. May offer new perspectives on issue or point out area for further research
Systematic search and Combines strengths of critical review with a comprehensive search process. Typically addresses broad
review questions to produce ‘best evidence synthesis’
Meta-analysis Technique that statistically combines the results of quantitative studies to provide a more precise
Critical appraisal is different from critical assessment in that critical appraisal involves systematically effect of the results
evaluating the quality, validity, and reliability of research evidence, whereas critical assessment focuses
more broadly on identifying and discussing the strengths and weaknesses of a study or argument + Source: Grant, Maria J., and Andrew Booth. "A typology of reviews: an analysis of 14 review types and associated
contrast and compare to make sure the result section is not just a summary. methodologies." Health Information & Libraries Journal 26.2 (2009): 91-108.
11
Vrije Universiteit Amsterdam
Types of literature reviews
LITERATURE REVIEW
Type of review Search type Appraisal MPA 2020-2021
Narrative review Broad, often non-documented No, but some data on study design is
needed Evelien de Hoop
Critical review Seeks to identify the most significant items in the field No, evaluates by contribution
Integrative review Comprehensive search to identify maximum primary sources, using 2 or Reports coded by quality, but not Lecture 3: Review Question & Exploratory Search
more strategies. Purposive sampling may be combined with excluded
comprehensive search if appropriate
Mapping review Completeness of searching determined by time/scope constraints No
Mixed methods Requires either very sensitive search to retrieve all studies or separately Generic appraisal tool or checklists
conceived quantitative and qualitative strategies
Overview May or may not include comprehensive searching (depends whether Depends on how systematic methods
systematic overview or not) are
Scoping review Completeness of searching determined by time/scope constraints. May No
include research in progress
State-of-the art Aims for comprehensive searching of current literature No
review
Systematic search Aims for exhaustive, comprehensive searching Yes
and review
Meta-analysis Aims for exhaustive, comprehensive searching. May use funnel plot to May use to determine inclusion
assess completeness 0
Source: Grant, Maria J., and Andrew Booth. "A typology of reviews: an analysis of 14 review types
and associated methodologies." Health Information & Libraries Journal 26.2 (2009): 91-108.
Vrije Universiteit Amsterdam
