Educational effectiveness
Introduction to EER
General questions/context
General context
What makes a “good” school?
Good education
● going the right things?
○ like educational philosophy: are we teaching the right subjects?
○ just answering to questions
● Doing things right?
○ thinking about are we doing things right
○ question of educational quality, part of what good education is
Educational quality
● goal oriented
○ is the policy for… the right one of reaching the right goals
● Doing things right?
○ just one thing/aspect of educational effectiveness
Educational effectiveness
● school/classroom process characteristics (most important characteristics 1)
○ effectiveness enhancing conditions
■ e.g. instruction of practice (e.g. focus on reading)
■ focus on school/classroom and not on SES, gender,..
● we can not change these characteristics
● we need to focus on characteristics we can change
● student outcomes (most important characteristics 2)
○ effectiveness criterion
■ comparison between schools based on the results
● e.g. drop-out rates, exam results, how many students in a
school employ for a higher education
○ student outcomes
■ we are not interested in the teacher
1
,General questions
● What makes a “good” school?
● how do we make more schools “good”? (educational improvement)
○ part of effectiveness, close to each other
○ use results of educational effectiveness studies
● Are there differences between schools in student outcomes?
○ or between teachers,...
○ is the school still effective is the results are good, but the wellbeing of the
students is not or if the drop-out rage is high
● How large are the differences between schools?
● Why do students have more learning gains in school A than in school B?
○ For example, copy-paste a practice from one school to another. Differences in
for example native language of the students make a difference in learning
gains
● How can we measure the effectiveness of schools? Effectiveness of teachers?
○ what should we do that a school is more effective is more effective than
another
○ see value added modeling (next class)
● Is an effective school effective for all subjects? For all students?
○ If we have found something that works for students, does it work for all
students? or does it work better for females than males,... and is it than more
effective for one subject to another
General course objectives
● learning outcomes
○ the student understands the history, the theoretical backgrounds and the
main concepts of educational effectiveness research (EER)
○ the student can distinguish the different levels in EER
○ the student has insight into the methods in EER
○ the student can interpret the results of EER
○ the student understands meaningful clues for educational improvement
○ the student can maken critical comments regarding EER
● educational goals
○ the student shows a critical attitude towards the role of educational
effectiveness research in educational policy and educational practice
○ the student shows interest in lifelong learning and keeping up-to-date with
recent discussions in educational research
○ the student is interested in improving education for all students
2
, Introduction
Reynolds, D., Sammons, P., De Fraine, B., Van Damme, J., Townsend, T., Teddlie, C., &
Stringfield, S. (2014). Educational effectiveness research (EER): a state-of-the-art review.
School Effectiveness and School Improvement, 25, 197-230.
History of EER
‘60-’70:
● response to Coleman et al. (1966) and Jencks et al.
(1972)
● school matters!
○ Coleman say that schools don’t matter,
because they can compensate for society
○ Look at the table and discuss why he said it
■ 2 models:
● first model: integrated 4
student characteristics
○ positive-negative
correlation with results
● second model:
■ R²: percentage that can be explained
by the model
● model 1: how much can be explained by the characteristics
○ R² = 0.21
● model 2: much more characteristics, also the school
characteristics
○ R² = 0.22
■ not much more than model 1, so don’t make a
huge difference
■ school characteristics don’t help you to explain
a difference in results
● Why are they wrong?
○ focussing on the wrong kind of characteristics: (see general context)
■ we need to focus on process characteristics (policy, curriculum,
expectation, instructional quality,...)
■ these are not process characteristics
○ we only have one variance
■ not a multi-level-model (next class)
■ normally we can split than in differences between schools and
differences between students within a school
● if we can see these differences, we can say something about
educational effectiveness
3
, ‘80:
● multilevel models/ scientific properties/differential effects
○ if we find a relationship between achievement and a certain practice in a
school, is this relationship the same for all students or not? or is the effect of
instructional quality on the math achievement different for boys than it is for
girl for example
‘90:
● open the black box
● input/output → input/process/output
○ input is about characteristics of teacher/teacher/school we can not change
■ big/small school, age, language,...
● importance of class/teacher level
○ teachers are important!
○ before that: school effectivenes, → ‘90: educational effectiveness
■ because a lot of multilevel studies indicated that the differences in
academic achievements of student, a bigger proportion of those
differences was explained by what the teacher was doing comparing
to what the schools was doing
> ‘00:
● large-scale longitudinal studies (ESLC, LOSO, SiBO,...)
○ give more information than studies that administer a test at the end of a
certain period
● international comparative research (PISA, TIMSS, PIRLS,...)
● dynamic nature of education
○ if you have factors at the school level, those have an influence on factors on
the class level
● methodological advances (multilevel ML, SEM,...)
○ to measure the dynamics
○ Structure equation modeling (SEM):
■ investigating indirect effects:
● Math (dependent)
● Teacher style: problem solving
● beliefs of the teacher
● investigate the direct relationship between these math and
problem solving style or direct relationship between beliefs and
math
● but indirect (SEM): relationship between beliefs and math that
is mediated by problem solving
4
Introduction to EER
General questions/context
General context
What makes a “good” school?
Good education
● going the right things?
○ like educational philosophy: are we teaching the right subjects?
○ just answering to questions
● Doing things right?
○ thinking about are we doing things right
○ question of educational quality, part of what good education is
Educational quality
● goal oriented
○ is the policy for… the right one of reaching the right goals
● Doing things right?
○ just one thing/aspect of educational effectiveness
Educational effectiveness
● school/classroom process characteristics (most important characteristics 1)
○ effectiveness enhancing conditions
■ e.g. instruction of practice (e.g. focus on reading)
■ focus on school/classroom and not on SES, gender,..
● we can not change these characteristics
● we need to focus on characteristics we can change
● student outcomes (most important characteristics 2)
○ effectiveness criterion
■ comparison between schools based on the results
● e.g. drop-out rates, exam results, how many students in a
school employ for a higher education
○ student outcomes
■ we are not interested in the teacher
1
,General questions
● What makes a “good” school?
● how do we make more schools “good”? (educational improvement)
○ part of effectiveness, close to each other
○ use results of educational effectiveness studies
● Are there differences between schools in student outcomes?
○ or between teachers,...
○ is the school still effective is the results are good, but the wellbeing of the
students is not or if the drop-out rage is high
● How large are the differences between schools?
● Why do students have more learning gains in school A than in school B?
○ For example, copy-paste a practice from one school to another. Differences in
for example native language of the students make a difference in learning
gains
● How can we measure the effectiveness of schools? Effectiveness of teachers?
○ what should we do that a school is more effective is more effective than
another
○ see value added modeling (next class)
● Is an effective school effective for all subjects? For all students?
○ If we have found something that works for students, does it work for all
students? or does it work better for females than males,... and is it than more
effective for one subject to another
General course objectives
● learning outcomes
○ the student understands the history, the theoretical backgrounds and the
main concepts of educational effectiveness research (EER)
○ the student can distinguish the different levels in EER
○ the student has insight into the methods in EER
○ the student can interpret the results of EER
○ the student understands meaningful clues for educational improvement
○ the student can maken critical comments regarding EER
● educational goals
○ the student shows a critical attitude towards the role of educational
effectiveness research in educational policy and educational practice
○ the student shows interest in lifelong learning and keeping up-to-date with
recent discussions in educational research
○ the student is interested in improving education for all students
2
, Introduction
Reynolds, D., Sammons, P., De Fraine, B., Van Damme, J., Townsend, T., Teddlie, C., &
Stringfield, S. (2014). Educational effectiveness research (EER): a state-of-the-art review.
School Effectiveness and School Improvement, 25, 197-230.
History of EER
‘60-’70:
● response to Coleman et al. (1966) and Jencks et al.
(1972)
● school matters!
○ Coleman say that schools don’t matter,
because they can compensate for society
○ Look at the table and discuss why he said it
■ 2 models:
● first model: integrated 4
student characteristics
○ positive-negative
correlation with results
● second model:
■ R²: percentage that can be explained
by the model
● model 1: how much can be explained by the characteristics
○ R² = 0.21
● model 2: much more characteristics, also the school
characteristics
○ R² = 0.22
■ not much more than model 1, so don’t make a
huge difference
■ school characteristics don’t help you to explain
a difference in results
● Why are they wrong?
○ focussing on the wrong kind of characteristics: (see general context)
■ we need to focus on process characteristics (policy, curriculum,
expectation, instructional quality,...)
■ these are not process characteristics
○ we only have one variance
■ not a multi-level-model (next class)
■ normally we can split than in differences between schools and
differences between students within a school
● if we can see these differences, we can say something about
educational effectiveness
3
, ‘80:
● multilevel models/ scientific properties/differential effects
○ if we find a relationship between achievement and a certain practice in a
school, is this relationship the same for all students or not? or is the effect of
instructional quality on the math achievement different for boys than it is for
girl for example
‘90:
● open the black box
● input/output → input/process/output
○ input is about characteristics of teacher/teacher/school we can not change
■ big/small school, age, language,...
● importance of class/teacher level
○ teachers are important!
○ before that: school effectivenes, → ‘90: educational effectiveness
■ because a lot of multilevel studies indicated that the differences in
academic achievements of student, a bigger proportion of those
differences was explained by what the teacher was doing comparing
to what the schools was doing
> ‘00:
● large-scale longitudinal studies (ESLC, LOSO, SiBO,...)
○ give more information than studies that administer a test at the end of a
certain period
● international comparative research (PISA, TIMSS, PIRLS,...)
● dynamic nature of education
○ if you have factors at the school level, those have an influence on factors on
the class level
● methodological advances (multilevel ML, SEM,...)
○ to measure the dynamics
○ Structure equation modeling (SEM):
■ investigating indirect effects:
● Math (dependent)
● Teacher style: problem solving
● beliefs of the teacher
● investigate the direct relationship between these math and
problem solving style or direct relationship between beliefs and
math
● but indirect (SEM): relationship between beliefs and math that
is mediated by problem solving
4
