BOOK
McKenney, S., & Reeves, T. C. (2019). Conducting Educational Design
Research (2e ed.).
CH 1. About educational design research
- Educational design research: a genre of research in which the iterative development of
solutions to practical and complex educational problems also provides the context for
empirical investigation, which yields theoretical understanding that can inform the work of
others.
- Ecologically valid study: researchers apply methods (observations/interviews) in the real-
life situations that is under investigation.
- External validity of the study (ability of a study’s results to be generalized) grows when
conducted under real world conditions.
Research is often classified as basic or applied:
• Basic research: using scientific methods to explore, describe and explain phenomena
shapes quest for fundamental understanding. Goal: developing theory.
o Characterized as an empirical cycle (De Groot).
o Five phases: observation, induction, deduction, testing and evaluation.
• Applied research: application of scientific methods to predict and control
phenomena. Goal: solving a real world problem through intervention.
o Characterized as a regulative cycle (Strien).
o Five phases: problem identification, diagnosis, planning, action and
evaluation.
Donald Stokes (1997): goals of science and their
relationship to application for use in book:
Pasteur’s quadrant: Basic science and technological
innovation.
Pasteur’s Quadrant:
• Bohr: pure basic research (his work had no
practical application of the knowledge)
• Edison: pure applied research (no publishing
his findings or contributing to a scientific understanding).
• Pasteur: solving real-world problems and finding knowledge.
Why need for understanding to guide design of educational practices?
1. Provide grounding that can inform initial or following design decisions.
2. Planning for short-term testing that allows designs to respond accordingly is needed
to increase the chances of complex interventions to succeed.
3. Inserting research into the design process can contribute greatly to the professional
development of educational designers.
1
,Characterizing educational design research
Theoretically oriented: using existing theory to frame inquiry. Scientific understanding is
used to frame not only research, but also shape the design of a solution to a real problem.
Interventionist: educational design research engages in the creative activity of developing
solutions. Examples: educational programs, products, processes, policies.
Collaborative: educational
design research is conducted in
collaboration with practice. It
requires collaboration among a
range of actors connected to the
problem at hand.
Responsively grounded: the
products are shaped by
participant expertise, literature
and field-testing.
Iterative: the insights and
interventions evolve over time
through multiple iterations of
investing, development, testing
and refinement. Process display
of a design study à
Theoretical understanding in
design research:
a) Underpins the design of an intervention
b) Frames the scientific inquiry
c) Advanced by findings generated through empirical testing of the intervention.
Theoretical contributions:
• Abstracted from empirical findings, useful for others outside the research setting.
• Design principles: educational design research can produce theoretical insights of a
prescriptive nature.
Practical contributions:
• Development of solutions to problems of practice.
• Inputs into educational environments are fine-tuned through empirical testing.
• Interventions are implemented in authentic settings with the goal of solving real
problems.
• Substantial elements of the intervention to be co-constructed by researchers and
practitioners working closely together, that is feasible and effective given the
situation at hand.
• Possibilities: strategies for developing preschooler oral language skills, para-teacher
professional development in Indian slums etc.
Generalization
à Allows insights from one situation to advance understanding not only of that specific
instance but of other similar instances.
• Can refer to reasoning from detailed facts to general principles (induction) (logic).
• Can refer to transferring the response learned to one stimulus to a similar stimulus
(psychology).
2
, • Being able to transfer theoretical insights and/or practical interventions to other
settings (educational design research).
Generalization can be improved by replication.
Forms of generalization (Firestone, 1993) help in findings ways to ascertain which lessons
learned in one context might be of value in a particular (different) setting.
• Analytic generalization: a process in which the “investigator is striving to generalize a
particular set of results to a broader theory”.
• Case-to-case generalization: refers to the transfer of ideas that takes place when a
person in one setting considers adopting an intervention for use in another setting.
o Design researchers incorporate ideas from other interventions into their own
designs.
o Arguments of Kennedy (1979) being compared to the notion of precedent
(used by judges in court):
§ Material facts
§ Appropriateness
§ The reason for the decision made in the original case
§ Grounds for the original decision.
Research for, on, and through interventions.
• Difference: whether or not the implementation process is considered and the role of
the designed intervention itself during empirical investigation.
• Research for: focuses on the problem, the context or the stakeholders.
o Seeks understanding about the initial situation à intervention is not known
yet.
• Research on: foregrounds the design of the intervention and uses information about
how the intervention works, with whom, under what conditions and with what
results to refine its characteristics.
o Generate knowledge about characteristics and functions of particular
intervention types.
• Research through: if the outcomes are foregrounded.
o Understanding the responses from the intervention.
Types of research goals
• Descriptive: focus on portraying how education works by describing phenomena
related to teaching, learning, and performance, typically using survey methods
and/or literature review.
• Interpretive: also explain the meaning or implications of phenomena related to
teaching, learning, performance, assessment, social interaction, leadership and other
educational factors.
• Predictive: focus on testing hypotheses related to theories of teaching, learning,
performance, assessment, social interaction, instructional design etc.
• Development: aim to produce interventions that speak to human teaching, learning
and performance challenges.
• Action: seek to improve professional practice, and sometimes in collaboration with
others.
Four foundational competencies that are required to enact each role:
• Empathy: to explore (un) shared goals or becoming exposed to the incentives,
motives and rewarded structures in different settings.
• Orchestration: helps attend to research framing, data collection, solution design etc.
3
, • Flexibility: supports optimization of the human and material resources available in
ways that remain aligned with instructional goals.
• Social competence: included robust repertoire of interaction strategies to be selected
as needed.
Difference between educational design research and educational design:
à Educational design research strives to make a theoretical contribution whereas
educational design does not. Educational design may use theoretical, empirical evidence to
create educational interventions; it does not contribute to understanding a particular
phenomenon through study of one or more instances of it.
CH 2. Concepts and examples
Theory: are explanations of real-world phenomena proved by scientific evidence. They
provide models or schemes for understanding the nature and causes of certain phenomena.
Theories describe or explain things that happen and can be used to predict or steer the
future occurrences. Theories are powerful and important mental tools because they help us
make sense of the world, and can be used to inform our manipulations of it.
Reasoning
• Deduction: process of deriving a conclusion that logically and necessarily follows
from a set of premises. Argument: learners stay engaged when given personalized
feedback; that learner is being given personalized feedback; that learner will stay
engaged.
• Induction: deriving a reliable generalization from multiple observations. Inductive
argument could be: each of these learners stayed engages; each of them was given
personalized feedback; personalized feedback contributes to engagement.
• Abduction: generates hypotheses about relationships between observable
phenomena. Argument: learner was given personalized feedback; learner stayed
engages; relationship may exist between feedback and engagement.
Types of theoretical research:
- Descriptive: describe a real-world phenomenon
- Explanatory: explain why or how certain phenomena exist
- Predictive: include using descriptions of phenomena and explanations of how and why
things work to predict effects.
- Prescriptive/normative: understandings of certain phenomena, how they work, and cause-
effect relationships are used to recommend certain activities that will yield certain effects.
Theories categorized by level
• Local theory: much of this work contributes theoretical understanding that is closely
tied to the specifics of the investigation, yielding local theory. Local theory is
produced when limited manifestations of a certain phenomenon are studied. It can
be used to describe, explain or predict, but can also be predictive.
• Middle-range theory: as interventions begin to mature, educational design research
may strive to develop middle-range theory. In educational design research middle-
range theories are developed when the enactment and effects of multiple
manifestations of an intervention are studied in several settings.
• High-level theory: high-level theory synthesizes middle-range theories and may be, in
part, derived from a wide range of manifestations of a certain intervention type
across many different settings. It is based on paradigms, sets of assumptions and
4
McKenney, S., & Reeves, T. C. (2019). Conducting Educational Design
Research (2e ed.).
CH 1. About educational design research
- Educational design research: a genre of research in which the iterative development of
solutions to practical and complex educational problems also provides the context for
empirical investigation, which yields theoretical understanding that can inform the work of
others.
- Ecologically valid study: researchers apply methods (observations/interviews) in the real-
life situations that is under investigation.
- External validity of the study (ability of a study’s results to be generalized) grows when
conducted under real world conditions.
Research is often classified as basic or applied:
• Basic research: using scientific methods to explore, describe and explain phenomena
shapes quest for fundamental understanding. Goal: developing theory.
o Characterized as an empirical cycle (De Groot).
o Five phases: observation, induction, deduction, testing and evaluation.
• Applied research: application of scientific methods to predict and control
phenomena. Goal: solving a real world problem through intervention.
o Characterized as a regulative cycle (Strien).
o Five phases: problem identification, diagnosis, planning, action and
evaluation.
Donald Stokes (1997): goals of science and their
relationship to application for use in book:
Pasteur’s quadrant: Basic science and technological
innovation.
Pasteur’s Quadrant:
• Bohr: pure basic research (his work had no
practical application of the knowledge)
• Edison: pure applied research (no publishing
his findings or contributing to a scientific understanding).
• Pasteur: solving real-world problems and finding knowledge.
Why need for understanding to guide design of educational practices?
1. Provide grounding that can inform initial or following design decisions.
2. Planning for short-term testing that allows designs to respond accordingly is needed
to increase the chances of complex interventions to succeed.
3. Inserting research into the design process can contribute greatly to the professional
development of educational designers.
1
,Characterizing educational design research
Theoretically oriented: using existing theory to frame inquiry. Scientific understanding is
used to frame not only research, but also shape the design of a solution to a real problem.
Interventionist: educational design research engages in the creative activity of developing
solutions. Examples: educational programs, products, processes, policies.
Collaborative: educational
design research is conducted in
collaboration with practice. It
requires collaboration among a
range of actors connected to the
problem at hand.
Responsively grounded: the
products are shaped by
participant expertise, literature
and field-testing.
Iterative: the insights and
interventions evolve over time
through multiple iterations of
investing, development, testing
and refinement. Process display
of a design study à
Theoretical understanding in
design research:
a) Underpins the design of an intervention
b) Frames the scientific inquiry
c) Advanced by findings generated through empirical testing of the intervention.
Theoretical contributions:
• Abstracted from empirical findings, useful for others outside the research setting.
• Design principles: educational design research can produce theoretical insights of a
prescriptive nature.
Practical contributions:
• Development of solutions to problems of practice.
• Inputs into educational environments are fine-tuned through empirical testing.
• Interventions are implemented in authentic settings with the goal of solving real
problems.
• Substantial elements of the intervention to be co-constructed by researchers and
practitioners working closely together, that is feasible and effective given the
situation at hand.
• Possibilities: strategies for developing preschooler oral language skills, para-teacher
professional development in Indian slums etc.
Generalization
à Allows insights from one situation to advance understanding not only of that specific
instance but of other similar instances.
• Can refer to reasoning from detailed facts to general principles (induction) (logic).
• Can refer to transferring the response learned to one stimulus to a similar stimulus
(psychology).
2
, • Being able to transfer theoretical insights and/or practical interventions to other
settings (educational design research).
Generalization can be improved by replication.
Forms of generalization (Firestone, 1993) help in findings ways to ascertain which lessons
learned in one context might be of value in a particular (different) setting.
• Analytic generalization: a process in which the “investigator is striving to generalize a
particular set of results to a broader theory”.
• Case-to-case generalization: refers to the transfer of ideas that takes place when a
person in one setting considers adopting an intervention for use in another setting.
o Design researchers incorporate ideas from other interventions into their own
designs.
o Arguments of Kennedy (1979) being compared to the notion of precedent
(used by judges in court):
§ Material facts
§ Appropriateness
§ The reason for the decision made in the original case
§ Grounds for the original decision.
Research for, on, and through interventions.
• Difference: whether or not the implementation process is considered and the role of
the designed intervention itself during empirical investigation.
• Research for: focuses on the problem, the context or the stakeholders.
o Seeks understanding about the initial situation à intervention is not known
yet.
• Research on: foregrounds the design of the intervention and uses information about
how the intervention works, with whom, under what conditions and with what
results to refine its characteristics.
o Generate knowledge about characteristics and functions of particular
intervention types.
• Research through: if the outcomes are foregrounded.
o Understanding the responses from the intervention.
Types of research goals
• Descriptive: focus on portraying how education works by describing phenomena
related to teaching, learning, and performance, typically using survey methods
and/or literature review.
• Interpretive: also explain the meaning or implications of phenomena related to
teaching, learning, performance, assessment, social interaction, leadership and other
educational factors.
• Predictive: focus on testing hypotheses related to theories of teaching, learning,
performance, assessment, social interaction, instructional design etc.
• Development: aim to produce interventions that speak to human teaching, learning
and performance challenges.
• Action: seek to improve professional practice, and sometimes in collaboration with
others.
Four foundational competencies that are required to enact each role:
• Empathy: to explore (un) shared goals or becoming exposed to the incentives,
motives and rewarded structures in different settings.
• Orchestration: helps attend to research framing, data collection, solution design etc.
3
, • Flexibility: supports optimization of the human and material resources available in
ways that remain aligned with instructional goals.
• Social competence: included robust repertoire of interaction strategies to be selected
as needed.
Difference between educational design research and educational design:
à Educational design research strives to make a theoretical contribution whereas
educational design does not. Educational design may use theoretical, empirical evidence to
create educational interventions; it does not contribute to understanding a particular
phenomenon through study of one or more instances of it.
CH 2. Concepts and examples
Theory: are explanations of real-world phenomena proved by scientific evidence. They
provide models or schemes for understanding the nature and causes of certain phenomena.
Theories describe or explain things that happen and can be used to predict or steer the
future occurrences. Theories are powerful and important mental tools because they help us
make sense of the world, and can be used to inform our manipulations of it.
Reasoning
• Deduction: process of deriving a conclusion that logically and necessarily follows
from a set of premises. Argument: learners stay engaged when given personalized
feedback; that learner is being given personalized feedback; that learner will stay
engaged.
• Induction: deriving a reliable generalization from multiple observations. Inductive
argument could be: each of these learners stayed engages; each of them was given
personalized feedback; personalized feedback contributes to engagement.
• Abduction: generates hypotheses about relationships between observable
phenomena. Argument: learner was given personalized feedback; learner stayed
engages; relationship may exist between feedback and engagement.
Types of theoretical research:
- Descriptive: describe a real-world phenomenon
- Explanatory: explain why or how certain phenomena exist
- Predictive: include using descriptions of phenomena and explanations of how and why
things work to predict effects.
- Prescriptive/normative: understandings of certain phenomena, how they work, and cause-
effect relationships are used to recommend certain activities that will yield certain effects.
Theories categorized by level
• Local theory: much of this work contributes theoretical understanding that is closely
tied to the specifics of the investigation, yielding local theory. Local theory is
produced when limited manifestations of a certain phenomenon are studied. It can
be used to describe, explain or predict, but can also be predictive.
• Middle-range theory: as interventions begin to mature, educational design research
may strive to develop middle-range theory. In educational design research middle-
range theories are developed when the enactment and effects of multiple
manifestations of an intervention are studied in several settings.
• High-level theory: high-level theory synthesizes middle-range theories and may be, in
part, derived from a wide range of manifestations of a certain intervention type
across many different settings. It is based on paradigms, sets of assumptions and
4
