Lecture 1
Educational effectiveness
EER: general questions
- What makes a “good” school?
o Better talk about effective/ ineffective/ less effective
- How do we make more schools “good”?
o How can we improve/ make schools more effective
- Are there differences between schools in student outcomes?
o About what students learn in school
- How large are the differences between schools?
o How important is it to choose a school
- Why do students have more learning gains in school A
than in school B?
- How can we measure the effectiveness of schools?
Effectiveness of teachers?
o Good is perception → but we like facts – we want to measure it objectively
o Effectiveness of teachers
- Is an effective school effective for all subjects? For all students?
o Is it easy to say that a school is effective
o Maybe for some types of student/ for some courses
Objectives (goals to achieve in this class)
- Learning outcomes
o the student understands the history, the theoretical backgrounds and the main
concepts of educational effectiveness research (EER).
o the student can distinguish the different levels in EER
o the student has insight into the methods in EER
o the student can interpret the results of EER
o the student understands meaningful clues for educational improvement
o the student can make critical comments regarding EER
- Educational goals
o the student shows a critical attitude towards the role of educational effectiveness
research in educational policy and educational practice.
o the students shows interest in lifelong learning and keeping up-to-date with recent
discussions in educational research.
o the student is interested in improving education for all students.
,Introductory text
Reynolds, D., Sammons, P., De Fraine, B., Van Damme, J., Townsend, T., Teddlie, C., &
Stringfield, S. (2014). Educational effectiveness research (EER): a state-of-the-art review.
School Effectiveness and School Improvement, 25, 197-230.
History of EER
- ’60-’70: Coleman report: Report says that the school you go to does not matter – it matters
what family you are born in
→ but ofcourse teachers,… didn’t want to hear that schools don’t matter → studies in the
UA: schools do matter! Although they only matter for a very small part
- ’80: Field grew → new methods – improvements → use of multilevel models (very common)
we will learn how to understand (not write)
- ’90: Black box was opened: until then we were interested in output and input but not in what
happened. More interested in what happens in schools – in how schools teach students and
how they do it. Finding process variables that matter.
- ’00: Class and teacher matter way more than school. → talking about educational effective
research.
- Methodological advances: effectiveness researchers are interested in statistics,… models
help to see the complexity of education
- Mixed methods: combine quantitative studies with qualitative studies
Educational effectiveness (Scheerens, 2004)
- Association between
o Effectiveness enhancing conditions of schooling
o Output (effectiveness criterion)
CIPO Model: break open the black box
See what works
and what didn’t
Output is always
the dependent
variable
= one of the main models
, - Output:
o Always interested in students (not in teachers)
o Students matter – their output/ learning is the criterium
- Process:
o Somewhat interested in teachers because they are on the process level → teacher
will never be an output variable but a process variable → what happens in schools
(policy, how they teach, communicate with students)
- Input
o Put in the system to improve it
o May be differences in schools that schools cannot always be taken responsible for
▪ Vb. Students home language may be taken into account to take a fair
compresence between school
- Context
o Vb. Poor neighbourhood vs. rich
o Vb. Country
Effectiveness criteria
1. School results / test results
2. School career (grade repeating, drop-out, …)
3. Metacognitive skills / self-regulated learning
4. Student well-being
5. Motivation / engagement
6. Selfesteem / academic self-concept
7. Values / attitudes
o It’s not all about test results
8. Social skills
o Complex to study – valuable but it’s hard to study
9. Behaviour (truancy, …)
10. Long term results (succes in labour market, …)
11. Learning gains
o About improving over time
12. Equity
o Outcomes of one student can’t show equity
o About gaps between students (gender, language at home, socioeconomic status,…)
o Have the gap as small as possible
13. …
➔ There are a lot of different effectiveness criteria
➔ There is no ultimate criteria that works all the time – you have to combine
Theoretical models
- Carroll (1963): Model of school learning
- Edmonds (1979) : five factor model
- Scheerens (1990): CIPO-model
- Creemers (1994): comprehensive model of educational effectiveness
- Scheerens (2015): Multilevel model of education
- Creemers & Kyriakides (2006): Dynamic model of educational effectiveness
- …
, ➔ Most research is data driven – some brought theory that give insights in what might
matter
Five factor model
1) strong educational leadership
2) emphasis on basic skills acquisition
3) safe and orderly learning climate
4) high expectations of pupils’ achievement
5) frequent monitoring of pupils’ progress
(Edmonds, 1979)
➔ interesting story
➔ Edmonds write an article that wasn’t data driven but of what he thinks that you should
do
➔ in the end he talks about those 5 variables (also a 6th one: financially)
Always also check the old courses cause their might be something left out
Carroll’s model of school learning
time = central variable
Model implicates that time is
important
Achievement is a work of two
types of time
o Required time (some students need more time to study – not fair)
o Active time (the time really devoted to studying)
=> Main idea: if active time is large and the required time is small, then learning is at
his max and achievement will be good -> main question: how can we reduce
required time and increase active time?
o Quality of instruction: teacher has an impact on it
o To improve achievement we can improve the quality of instruction
o Time available to the task: time management and classmangment + amount of time
pupils want to spent (interest,…)
▪ Start on time (teachers need to structure)
▪ A lot of 5 minutes can make a big difference
o Teachers can work on the available time (vb. With homework) and quality of
instructions
Educational effectiveness
EER: general questions
- What makes a “good” school?
o Better talk about effective/ ineffective/ less effective
- How do we make more schools “good”?
o How can we improve/ make schools more effective
- Are there differences between schools in student outcomes?
o About what students learn in school
- How large are the differences between schools?
o How important is it to choose a school
- Why do students have more learning gains in school A
than in school B?
- How can we measure the effectiveness of schools?
Effectiveness of teachers?
o Good is perception → but we like facts – we want to measure it objectively
o Effectiveness of teachers
- Is an effective school effective for all subjects? For all students?
o Is it easy to say that a school is effective
o Maybe for some types of student/ for some courses
Objectives (goals to achieve in this class)
- Learning outcomes
o the student understands the history, the theoretical backgrounds and the main
concepts of educational effectiveness research (EER).
o the student can distinguish the different levels in EER
o the student has insight into the methods in EER
o the student can interpret the results of EER
o the student understands meaningful clues for educational improvement
o the student can make critical comments regarding EER
- Educational goals
o the student shows a critical attitude towards the role of educational effectiveness
research in educational policy and educational practice.
o the students shows interest in lifelong learning and keeping up-to-date with recent
discussions in educational research.
o the student is interested in improving education for all students.
,Introductory text
Reynolds, D., Sammons, P., De Fraine, B., Van Damme, J., Townsend, T., Teddlie, C., &
Stringfield, S. (2014). Educational effectiveness research (EER): a state-of-the-art review.
School Effectiveness and School Improvement, 25, 197-230.
History of EER
- ’60-’70: Coleman report: Report says that the school you go to does not matter – it matters
what family you are born in
→ but ofcourse teachers,… didn’t want to hear that schools don’t matter → studies in the
UA: schools do matter! Although they only matter for a very small part
- ’80: Field grew → new methods – improvements → use of multilevel models (very common)
we will learn how to understand (not write)
- ’90: Black box was opened: until then we were interested in output and input but not in what
happened. More interested in what happens in schools – in how schools teach students and
how they do it. Finding process variables that matter.
- ’00: Class and teacher matter way more than school. → talking about educational effective
research.
- Methodological advances: effectiveness researchers are interested in statistics,… models
help to see the complexity of education
- Mixed methods: combine quantitative studies with qualitative studies
Educational effectiveness (Scheerens, 2004)
- Association between
o Effectiveness enhancing conditions of schooling
o Output (effectiveness criterion)
CIPO Model: break open the black box
See what works
and what didn’t
Output is always
the dependent
variable
= one of the main models
, - Output:
o Always interested in students (not in teachers)
o Students matter – their output/ learning is the criterium
- Process:
o Somewhat interested in teachers because they are on the process level → teacher
will never be an output variable but a process variable → what happens in schools
(policy, how they teach, communicate with students)
- Input
o Put in the system to improve it
o May be differences in schools that schools cannot always be taken responsible for
▪ Vb. Students home language may be taken into account to take a fair
compresence between school
- Context
o Vb. Poor neighbourhood vs. rich
o Vb. Country
Effectiveness criteria
1. School results / test results
2. School career (grade repeating, drop-out, …)
3. Metacognitive skills / self-regulated learning
4. Student well-being
5. Motivation / engagement
6. Selfesteem / academic self-concept
7. Values / attitudes
o It’s not all about test results
8. Social skills
o Complex to study – valuable but it’s hard to study
9. Behaviour (truancy, …)
10. Long term results (succes in labour market, …)
11. Learning gains
o About improving over time
12. Equity
o Outcomes of one student can’t show equity
o About gaps between students (gender, language at home, socioeconomic status,…)
o Have the gap as small as possible
13. …
➔ There are a lot of different effectiveness criteria
➔ There is no ultimate criteria that works all the time – you have to combine
Theoretical models
- Carroll (1963): Model of school learning
- Edmonds (1979) : five factor model
- Scheerens (1990): CIPO-model
- Creemers (1994): comprehensive model of educational effectiveness
- Scheerens (2015): Multilevel model of education
- Creemers & Kyriakides (2006): Dynamic model of educational effectiveness
- …
, ➔ Most research is data driven – some brought theory that give insights in what might
matter
Five factor model
1) strong educational leadership
2) emphasis on basic skills acquisition
3) safe and orderly learning climate
4) high expectations of pupils’ achievement
5) frequent monitoring of pupils’ progress
(Edmonds, 1979)
➔ interesting story
➔ Edmonds write an article that wasn’t data driven but of what he thinks that you should
do
➔ in the end he talks about those 5 variables (also a 6th one: financially)
Always also check the old courses cause their might be something left out
Carroll’s model of school learning
time = central variable
Model implicates that time is
important
Achievement is a work of two
types of time
o Required time (some students need more time to study – not fair)
o Active time (the time really devoted to studying)
=> Main idea: if active time is large and the required time is small, then learning is at
his max and achievement will be good -> main question: how can we reduce
required time and increase active time?
o Quality of instruction: teacher has an impact on it
o To improve achievement we can improve the quality of instruction
o Time available to the task: time management and classmangment + amount of time
pupils want to spent (interest,…)
▪ Start on time (teachers need to structure)
▪ A lot of 5 minutes can make a big difference
o Teachers can work on the available time (vb. With homework) and quality of
instructions
