Turkish Online Journal of Distance Education-TOJDE April 2017 ISSN 1302-6488 Volume: 18 Number: 2 Article 7
ANALYZING THE EFFICACY OF THE TESTING EFFECT USING
KAHOOTTM ON STUDENT PERFORMANCE
Dr. Darren H. IWAMOTO
Chaminade University of Honolulu
Honolulu, HI USA
Dr. Jace HARGIS
University of California, San Diego
La Jolla, CA USA
Erik Jon TAITANO
Chaminade University of Honolulu
Honolulu, HI USA
Ky VUONG
Chaminade University of Honolulu
Honolulu, HI USA
ABSTRACT
Lower than expected high-stakes examination scores were being observed in a first-year
general psychology class. This research sought an alternate approach that would assist
students in preparing for high-stakes examinations. The purpose of this study was to measure
the effectiveness of an alternate teaching approach based on the testing effect to address low
high-stakes examination scores. This was accomplished through the introduction of an online
quizzing application that utilized a game show-like user interface called Kahoot™. The results
showed a significant difference in high-stakes examination scores for students who utilized
Kahoot™ versus students who did not. It can be suggested that pedagogical tools like
Kahoot™ have the potential to enhance and improve high-stakes examination scores at the
college and university level. Students that used Kahoot™ felt positive about their
experience. The results of this study also suggest that creating a fun and engaging
environment also supports improved academic performance.
Keywords: Educational technology, self-regulation, formative assessment, gamification,
testing effect.
INTRODUCTION
As a professor in higher education for a number of years now, I have seen a common perplexing
pattern of student behavior. Students who reportedly study diligently and are engaged in class
are not always the ones that excel in summative course examinations. Two thoughts
immediately come to mind. Either I am being deceived by the perceived practices of my
students or how my students study and prepare for the examinations needs to improve. I
decided to focus on the latter and my inquiry led me to a memory retrieval practice called the
testing effect. Will the practice of the testing effect in a higher education setting influence
high-stakes examination scores? Finding an answer to that question was the motivation
behind this study.
80
, The objective of this pilot quasi-experimental action research study was to measure the
effectiveness of an alternate teaching approach based on the theory of the testing effect to
address low high-stakes examination scores. The sample groups are from an undergraduate
level psychology course in a four-year university located in the south Pacific. The intervention
used to implement the testing effect utilized an online quizzing application that utilized a game
show-like user interface called Kahoot™. This research assessed the effectiveness of this
approach by comparing high-stakes examination scores and also by surveying the sample
groups for their feedback on their experiences through this study.
LITERATURE REVIEW
The testing effect is a robust and reliable phenomenon that demonstrates that taking an initial
test improves performance on subsequent tests (Chan & McDermott, 2007). The testing effect
(or retrieval practice effect) has traditionally been measured in the laboratory setting,
however, evidence of the testing effect has recently been shown in real-world settings
(Agarwal, Bain, & Chamberlain, 2012). Low-stake quizzes have shown improved summative
exam scores over the course of a semester relative to not being quizzed (Roediger, Agarwal,
McDaniel, & McDermott, 2011a). Roediger et al. (2011a) found that sixth grade social studies
students performed better on material they had been quizzed on throughout the semester,
performed better than students who were tasked to re-read material, and also performed
better when tasked to self-quiz themselves outside of the classroom. The time-delay between
exposure to material, and chapter and final exams, were typically one to two months, which
was much longer than previous studies conducted in a laboratory setting. This showed
evidence of the long-term benefits from retrieval practice learning. The testing effect was also
demonstrated in eighth grade science students who underwent a longer time-delay of five to
eight months between the teacher’s lessons and chapter and final exams (McDaniel, Agarwal,
Huesler, McDermott, & Roediger, 2011).
McDaniel, Anderson, Derbish, and Morisette (2007) also demonstrated the testing effect
among college students. Students took weekly quizzes followed by unit tests and a final
cumulative exam. Quizzes were both short answer and multiple-choice format with the unit
tests and exam in a multiple-choice format. Improved performance was seen with quizzing
versus additional reading.
The testing effect has also been shown in novel situations (McDaniel, Thomas, Agarwal,
McDermott, & Roediger, 2011). Experiments conducted in seventh and eighth grade science
classrooms utilized different quiz and exam items as opposed to previously reported
experiments where questions were the same with random assignment of multiple-choice
items. Performance on quizzed material was significantly greater than non-quizzed material
as the previous studies demonstrated. Significant testing effect was also demonstrated
whether quizzes and exams were in the same or different formats.
McDaniel et al. (2011) discovered that the testing effect was seen when students had items of
an applied nature on a quiz and items of a conceptual nature on an exam, but not vice-
versa. The testing effect stands to dispel the misconception that test and retest is a form of
rote memorization. The test and retest cycle, which improves encoded information recall,
shows that active processing of information is required for the testing effect phenomenon to
occur (Yigit, Kiyici, & Cetinkaya, 2014).
Roediger and Butler (2011b) have identified five points regarding the testing effect. First,
testing produces better retention relative to passive studying methods (e.g., re-reading
material). Second, testing repeatedly is more beneficial than taking a single test. Third, the
81
ANALYZING THE EFFICACY OF THE TESTING EFFECT USING
KAHOOTTM ON STUDENT PERFORMANCE
Dr. Darren H. IWAMOTO
Chaminade University of Honolulu
Honolulu, HI USA
Dr. Jace HARGIS
University of California, San Diego
La Jolla, CA USA
Erik Jon TAITANO
Chaminade University of Honolulu
Honolulu, HI USA
Ky VUONG
Chaminade University of Honolulu
Honolulu, HI USA
ABSTRACT
Lower than expected high-stakes examination scores were being observed in a first-year
general psychology class. This research sought an alternate approach that would assist
students in preparing for high-stakes examinations. The purpose of this study was to measure
the effectiveness of an alternate teaching approach based on the testing effect to address low
high-stakes examination scores. This was accomplished through the introduction of an online
quizzing application that utilized a game show-like user interface called Kahoot™. The results
showed a significant difference in high-stakes examination scores for students who utilized
Kahoot™ versus students who did not. It can be suggested that pedagogical tools like
Kahoot™ have the potential to enhance and improve high-stakes examination scores at the
college and university level. Students that used Kahoot™ felt positive about their
experience. The results of this study also suggest that creating a fun and engaging
environment also supports improved academic performance.
Keywords: Educational technology, self-regulation, formative assessment, gamification,
testing effect.
INTRODUCTION
As a professor in higher education for a number of years now, I have seen a common perplexing
pattern of student behavior. Students who reportedly study diligently and are engaged in class
are not always the ones that excel in summative course examinations. Two thoughts
immediately come to mind. Either I am being deceived by the perceived practices of my
students or how my students study and prepare for the examinations needs to improve. I
decided to focus on the latter and my inquiry led me to a memory retrieval practice called the
testing effect. Will the practice of the testing effect in a higher education setting influence
high-stakes examination scores? Finding an answer to that question was the motivation
behind this study.
80
, The objective of this pilot quasi-experimental action research study was to measure the
effectiveness of an alternate teaching approach based on the theory of the testing effect to
address low high-stakes examination scores. The sample groups are from an undergraduate
level psychology course in a four-year university located in the south Pacific. The intervention
used to implement the testing effect utilized an online quizzing application that utilized a game
show-like user interface called Kahoot™. This research assessed the effectiveness of this
approach by comparing high-stakes examination scores and also by surveying the sample
groups for their feedback on their experiences through this study.
LITERATURE REVIEW
The testing effect is a robust and reliable phenomenon that demonstrates that taking an initial
test improves performance on subsequent tests (Chan & McDermott, 2007). The testing effect
(or retrieval practice effect) has traditionally been measured in the laboratory setting,
however, evidence of the testing effect has recently been shown in real-world settings
(Agarwal, Bain, & Chamberlain, 2012). Low-stake quizzes have shown improved summative
exam scores over the course of a semester relative to not being quizzed (Roediger, Agarwal,
McDaniel, & McDermott, 2011a). Roediger et al. (2011a) found that sixth grade social studies
students performed better on material they had been quizzed on throughout the semester,
performed better than students who were tasked to re-read material, and also performed
better when tasked to self-quiz themselves outside of the classroom. The time-delay between
exposure to material, and chapter and final exams, were typically one to two months, which
was much longer than previous studies conducted in a laboratory setting. This showed
evidence of the long-term benefits from retrieval practice learning. The testing effect was also
demonstrated in eighth grade science students who underwent a longer time-delay of five to
eight months between the teacher’s lessons and chapter and final exams (McDaniel, Agarwal,
Huesler, McDermott, & Roediger, 2011).
McDaniel, Anderson, Derbish, and Morisette (2007) also demonstrated the testing effect
among college students. Students took weekly quizzes followed by unit tests and a final
cumulative exam. Quizzes were both short answer and multiple-choice format with the unit
tests and exam in a multiple-choice format. Improved performance was seen with quizzing
versus additional reading.
The testing effect has also been shown in novel situations (McDaniel, Thomas, Agarwal,
McDermott, & Roediger, 2011). Experiments conducted in seventh and eighth grade science
classrooms utilized different quiz and exam items as opposed to previously reported
experiments where questions were the same with random assignment of multiple-choice
items. Performance on quizzed material was significantly greater than non-quizzed material
as the previous studies demonstrated. Significant testing effect was also demonstrated
whether quizzes and exams were in the same or different formats.
McDaniel et al. (2011) discovered that the testing effect was seen when students had items of
an applied nature on a quiz and items of a conceptual nature on an exam, but not vice-
versa. The testing effect stands to dispel the misconception that test and retest is a form of
rote memorization. The test and retest cycle, which improves encoded information recall,
shows that active processing of information is required for the testing effect phenomenon to
occur (Yigit, Kiyici, & Cetinkaya, 2014).
Roediger and Butler (2011b) have identified five points regarding the testing effect. First,
testing produces better retention relative to passive studying methods (e.g., re-reading
material). Second, testing repeatedly is more beneficial than taking a single test. Third, the
81
