Beleid- en evaluatieonderzoek
Valérie Pattyn: Policy Evaluation in Europe
INTRODUCTION
The call to anchor policies in evidence can be considered the result of at least three
interlocking tendencies:
1. The increased interest in the economy, efficiency and effectiveness of public policies
2. Policy-making has become increasingly complex
3. The desire for enhanced social responsiveness and societal support by involving
citizens in the policy process
WAVES OF POLICY EVALUATION IN EUROPE
The first wave of policy evaluation in Europe was inspired by the US, where evaluations
emerged in the 1960s-1970s as part of social programs like the Great Society. This wave
primarily impacted Northwestern European countries, such as the UK, Germany and
Scandinavia, which had mixed economies that required government intervention for market
regulation. Empirical data collection for policy planning and the use of randomized
experiments characterized this period, with governments aiming to remain neutral in
evaluations. However, countries like the Netherlands and Switzerland were less involved,
relying instead on consensus-based decision-making.
The second wave, beginning in the 1990s and driven by New Public Management (NPM)
principles, emphasized government efficiency and accountability, introducing tools like
benchmarking and performance measurement. This shift was pushed by both the US/UK and
the EU, especially as new EU member states in Southern, Central and Eastern Europe were
required to develop evaluation capacities for regional ang agricultural funds. In this second
wave, the focus of evaluation often centered on output measures, especially in fields funded
by the EU, contrasting with the more varied evaluation practices in first-wave countries.
PRESENT-DAY EVALUATION CULTURE AND CAPACITY IN EUROPE: THREE CASE STUDIES
Evaluation culture, defined as shared values on evaluation importance and evaluation
capacity, the resources for conducting evaluations are challenging to measure due to varied
interpretations. Today, the UK’s evaluation culture is robust, supported by organizations like
the UK Evaluation Society and procedural standards like the Green and Magenta books,
although political agendas can influence evaluations. The National Audit Office further
supports oversight by producing numerous evaluation reports and helping Parliament assess
policies’ value for money. However, the UK faced challenges, such as inconsistent cost-
effectiveness evaluations across government departments.
The Flanders case study highlights how the region embraced policy evaluation in alignment
with New Public Management reforms and European Union influences, particularly through the
2006 Beter Bestuurlijk Beleid (BBB) reform. BBB assigned departments and agencies specific
roles in policy evaluation, promoting capacity-building in areas previously lacking in
evaluation activity. This shift led to the establishment of the Flemish Evaluation Platform in
2007, which gathered substantial interest, indicating a growing evaluation culture.
Evaluation practices in Flanders include institutionalized requirements, such as ex ante
assessments and a regulatory impact assessment framework. However, this faces criticism for
inconsistencies, especially for evaluations conducted after policy decisions. Despite
embedding evaluation in the executive branch, parliamentary support remains weak, with
limited structural engagement.
Overall, Flanders has advanced significantly in evaluation capacity and culture, with strong
adherence in sectors like education, labor and environment. Yet, evaluation is often driven by
legal obligations rather than intrinsic policy learning, indicating reliance on externa motivators
like NPM and the EU.
, The EU case study reflects an evolving evaluation system over four periods, shifting from
sporadic evaluation in the 1980s to a structured system centered on accountability. Reforms
in the 2000s introduced an impact assessment system, prioritizing ex ante evaluation. The
Smart Regulation program in 2007 emphasized evidence-based policymaking, but tensions
remain between impact assessment and ex post evaluation. The European Court of Auditors
now incorporates performance audits, blending evaluation and audit functions. Evaluations
culture in the EU remains uneven, with spending departments leading and legislative
evaluations gaining prominence. Despite advancements, challenges persists in evaluating
complex, multi-state policies.
TRENDS, POSSIBILITES AND CHALLENGES
Current trends in evaluation research reveal three key areas of focus:
1. There’s a shift towards evaluating broader policies beyond individual projects or
programs, exemplified by the EU’s move to include regulatory evaluations. This has
broadened academic inquiry into such evaluations
2. Stakeholder involvement in evaluation has gained traction, with research increasingly
examining its impact on evaluation use and potential biases. This includes specific
studies on European contexts
3. There’s growing interest in alternative forms of evaluation use, beyond direct policy
influence, such as how evaluations benefit organizational functioning or provide long-
term insights. The term ‘’evaluation influence’’ has also started to replace ‘’evaluation
use’’
Future challenges for evaluation research in Europe include the need for large-scale,
comparative studies on how evaluation is institutionalized across different countries and how
the EU impacts evaluation practices. Systematic research on stakeholder involvement remains
a priority, particularly regarding its effects on evaluation outcomes and stakeholders’ views on
methodology. Additionally, there is a need for longitudinal research on the legacy of New
Public Management and EU influence, particularly in how they impact learning-focused
evaluations and evaluation quality across member states.
Lut Mergaert: Ex Ante and Ex Post Evaluations
INTRODUCTION
Key to an efficient, effective and coherent policy cycle is that the ex ante and ex post
components of evaluation are linked. The research involves analyzing key evaluation
documents from EU Research and Technological Development Framework Programmes (FPs),
which follow a cyclical, iterative policy-making process. This examination reveals several
issues:
1. Mismatches between evaluations
2. Non-linear policy learning
3. Disparities among evaluation types
4. Significant institutional barriers
Ultimately, the article concludes that the link between ex ante and ex post evaluations in EU
research policy is weak, with limited coherence and impact on gender equality. It suggests
reforms to enhance this linkage, emphasizing the need for better integration in policy learning
and evaluation practices across the EU.
LINKING EX ANTE AND EX POST EVALUATIONS AS PART OF A COHERENT POLICY CYCLE
Current EU scholarship tends to treat ex ante and ex post evaluations as separate fields,
focusing on methods for each in isolation. This separation challenges coherent policy learning,
Valérie Pattyn: Policy Evaluation in Europe
INTRODUCTION
The call to anchor policies in evidence can be considered the result of at least three
interlocking tendencies:
1. The increased interest in the economy, efficiency and effectiveness of public policies
2. Policy-making has become increasingly complex
3. The desire for enhanced social responsiveness and societal support by involving
citizens in the policy process
WAVES OF POLICY EVALUATION IN EUROPE
The first wave of policy evaluation in Europe was inspired by the US, where evaluations
emerged in the 1960s-1970s as part of social programs like the Great Society. This wave
primarily impacted Northwestern European countries, such as the UK, Germany and
Scandinavia, which had mixed economies that required government intervention for market
regulation. Empirical data collection for policy planning and the use of randomized
experiments characterized this period, with governments aiming to remain neutral in
evaluations. However, countries like the Netherlands and Switzerland were less involved,
relying instead on consensus-based decision-making.
The second wave, beginning in the 1990s and driven by New Public Management (NPM)
principles, emphasized government efficiency and accountability, introducing tools like
benchmarking and performance measurement. This shift was pushed by both the US/UK and
the EU, especially as new EU member states in Southern, Central and Eastern Europe were
required to develop evaluation capacities for regional ang agricultural funds. In this second
wave, the focus of evaluation often centered on output measures, especially in fields funded
by the EU, contrasting with the more varied evaluation practices in first-wave countries.
PRESENT-DAY EVALUATION CULTURE AND CAPACITY IN EUROPE: THREE CASE STUDIES
Evaluation culture, defined as shared values on evaluation importance and evaluation
capacity, the resources for conducting evaluations are challenging to measure due to varied
interpretations. Today, the UK’s evaluation culture is robust, supported by organizations like
the UK Evaluation Society and procedural standards like the Green and Magenta books,
although political agendas can influence evaluations. The National Audit Office further
supports oversight by producing numerous evaluation reports and helping Parliament assess
policies’ value for money. However, the UK faced challenges, such as inconsistent cost-
effectiveness evaluations across government departments.
The Flanders case study highlights how the region embraced policy evaluation in alignment
with New Public Management reforms and European Union influences, particularly through the
2006 Beter Bestuurlijk Beleid (BBB) reform. BBB assigned departments and agencies specific
roles in policy evaluation, promoting capacity-building in areas previously lacking in
evaluation activity. This shift led to the establishment of the Flemish Evaluation Platform in
2007, which gathered substantial interest, indicating a growing evaluation culture.
Evaluation practices in Flanders include institutionalized requirements, such as ex ante
assessments and a regulatory impact assessment framework. However, this faces criticism for
inconsistencies, especially for evaluations conducted after policy decisions. Despite
embedding evaluation in the executive branch, parliamentary support remains weak, with
limited structural engagement.
Overall, Flanders has advanced significantly in evaluation capacity and culture, with strong
adherence in sectors like education, labor and environment. Yet, evaluation is often driven by
legal obligations rather than intrinsic policy learning, indicating reliance on externa motivators
like NPM and the EU.
, The EU case study reflects an evolving evaluation system over four periods, shifting from
sporadic evaluation in the 1980s to a structured system centered on accountability. Reforms
in the 2000s introduced an impact assessment system, prioritizing ex ante evaluation. The
Smart Regulation program in 2007 emphasized evidence-based policymaking, but tensions
remain between impact assessment and ex post evaluation. The European Court of Auditors
now incorporates performance audits, blending evaluation and audit functions. Evaluations
culture in the EU remains uneven, with spending departments leading and legislative
evaluations gaining prominence. Despite advancements, challenges persists in evaluating
complex, multi-state policies.
TRENDS, POSSIBILITES AND CHALLENGES
Current trends in evaluation research reveal three key areas of focus:
1. There’s a shift towards evaluating broader policies beyond individual projects or
programs, exemplified by the EU’s move to include regulatory evaluations. This has
broadened academic inquiry into such evaluations
2. Stakeholder involvement in evaluation has gained traction, with research increasingly
examining its impact on evaluation use and potential biases. This includes specific
studies on European contexts
3. There’s growing interest in alternative forms of evaluation use, beyond direct policy
influence, such as how evaluations benefit organizational functioning or provide long-
term insights. The term ‘’evaluation influence’’ has also started to replace ‘’evaluation
use’’
Future challenges for evaluation research in Europe include the need for large-scale,
comparative studies on how evaluation is institutionalized across different countries and how
the EU impacts evaluation practices. Systematic research on stakeholder involvement remains
a priority, particularly regarding its effects on evaluation outcomes and stakeholders’ views on
methodology. Additionally, there is a need for longitudinal research on the legacy of New
Public Management and EU influence, particularly in how they impact learning-focused
evaluations and evaluation quality across member states.
Lut Mergaert: Ex Ante and Ex Post Evaluations
INTRODUCTION
Key to an efficient, effective and coherent policy cycle is that the ex ante and ex post
components of evaluation are linked. The research involves analyzing key evaluation
documents from EU Research and Technological Development Framework Programmes (FPs),
which follow a cyclical, iterative policy-making process. This examination reveals several
issues:
1. Mismatches between evaluations
2. Non-linear policy learning
3. Disparities among evaluation types
4. Significant institutional barriers
Ultimately, the article concludes that the link between ex ante and ex post evaluations in EU
research policy is weak, with limited coherence and impact on gender equality. It suggests
reforms to enhance this linkage, emphasizing the need for better integration in policy learning
and evaluation practices across the EU.
LINKING EX ANTE AND EX POST EVALUATIONS AS PART OF A COHERENT POLICY CYCLE
Current EU scholarship tends to treat ex ante and ex post evaluations as separate fields,
focusing on methods for each in isolation. This separation challenges coherent policy learning,
