MaChine SCoring of
Student Essays
Truth and ConsequenCes
edited by
PatriCia Freitag EriCsson
RiChard Haswell
, CONTENTS
Introduction 1
Patricia Freitag Ericsson and Richard H. Haswell
1 Interested Complicities: The Dialectic of Computer-Assisted
Writing Assessment 8
Ken S. McAllister and Edward M. White
2 The Meaning of Meaning: Is a Paragraph More than an
Equation? 28
Patricia Freitag Ericsson
3 Can’t Touch This: Reflections on the Servitude of Computers as
Readers 38
Chris M. Anson
4 Automatons and Automated Scoring: Drudges, Black Boxes, and
Dei Ex Machina 57
Richard H. Haswell
5 Taking a Spin on the Intelligent Essay Assessor 79
Tim McGee
6 ACCUPLACER’s Essay-Scoring Technology: When Reliability Does
Not Equal Validity 93
Edmund Jones
7 WritePlacer Plus in Place: An Exploratory Case Study 114
Anne Herrington and Charles Moran
8 E-Write as a Means for Placement into Three Composition
Courses: A Pilot Study 130
Richard N. Matzen Jr. and Colleen Sorensen
9 Computerized Writing Assessment: Community College Faculty
Find Reasons to Say “Not Yet” 138
William W. Ziegler
10 Piloting the COMPASS E-Write Software at Jackson State
Community College 147
, Teri T. Maddox
11 The Role of the Writing Coordinator in a Culture of Placement by
ACCUPLACER 154
Gail S. Corso
12 Always Already: Automated Essay Scoring and Grammar-Checkers
in College Writing Courses 166
Carl Whithaus
13 Automated Essay Grading in the Sociology Classroom: Finding
Common Ground 177
Edward Brent and Martha Townsend
14 Automated Writing Instruction: Computer-Assisted or Computer-
Driven Pedagogies? 199
Beth Ann Rothermel
15 Why Less Is Not More: What We Lose by Letting a Computer
Score Writing Samples 211
William Condon
16 More Work for Teacher? Possible Futures of Teaching Writing in
the Age of Computerized Writing Assessment 221
Bob Broad
17 A Bibliography of Machine Scoring of Student Writing,
1962–2005 234
Richard H. Haswell
Glossary 244
Notes 246
References 251
Index 262
, INTRO DUCT ION
Patricia Freitag Ericsson and Richard H. Haswell
We’re in the fifth year of the twenty-first century and the Parliament
of India, several universities in Italy, and four Catholic churches in
Monterrey, Mexico, all have bought cell-phone jammers. Meanwhile in
the State of Texas, USA, the State Board of Education has decided that
students who fail the essay-writing part of the state’s college entrance
examination can retake it either with ACT’s COMPASS tests using e-
Write or with the College Board’s ACCUPLACER tests using WritePlacer
Plus. Though dispersed geographically, these events have one thing in
common. They illustrate how new technology can sneak in the back
door and establish itself while those at the front gates, nominally in
charge, are not much noticing. All of a sudden cell phones are disturb-
ing legislative sessions and church services and allowing students to
cheat on examinations in new ways. All of a sudden students can pass
entrance examination essays in ways never allowed before, with their
essays scored by machines running commercial software programs. How
did this technology happen so fast?
And where were educators when it happened? We will leave the MPs
in India and the deacons in Mexico to account for themselves, but as for
automated essay scoring in the State of Texas, college educators can only
throw up their hands. The decisions on e-Write and WritePlacer Plus were
made by state government officials and industry lobbyists with no input
from writing experts or administrators in higher education. The Texas
step toward machine grading may not be typical so far. But in the near
future there will be plenty of like steps taken everywhere in academe.
The analysis and scoring of student essays by computer—the history,
the mechanisms, the theory, and the educational consequences—is the
topic of this collection of essays. It is an understatement to say that the
topic is rapidly growing in importance at all levels of the educational
enterprise, and that the perspective on it has been, up to this point,
dominated almost exclusively by the commercial purveyors of the prod-
uct. Other than the notable exceptions of articles by Dennis Baron in
the Chronicle of Higher Education (1998), Anne Herrington and Charles
Moran in College English (2001), Julie Cheville in English Journal (2004),
and Michael Williamson in the Journal of Writing Assessment (2004) and
Student Essays
Truth and ConsequenCes
edited by
PatriCia Freitag EriCsson
RiChard Haswell
, CONTENTS
Introduction 1
Patricia Freitag Ericsson and Richard H. Haswell
1 Interested Complicities: The Dialectic of Computer-Assisted
Writing Assessment 8
Ken S. McAllister and Edward M. White
2 The Meaning of Meaning: Is a Paragraph More than an
Equation? 28
Patricia Freitag Ericsson
3 Can’t Touch This: Reflections on the Servitude of Computers as
Readers 38
Chris M. Anson
4 Automatons and Automated Scoring: Drudges, Black Boxes, and
Dei Ex Machina 57
Richard H. Haswell
5 Taking a Spin on the Intelligent Essay Assessor 79
Tim McGee
6 ACCUPLACER’s Essay-Scoring Technology: When Reliability Does
Not Equal Validity 93
Edmund Jones
7 WritePlacer Plus in Place: An Exploratory Case Study 114
Anne Herrington and Charles Moran
8 E-Write as a Means for Placement into Three Composition
Courses: A Pilot Study 130
Richard N. Matzen Jr. and Colleen Sorensen
9 Computerized Writing Assessment: Community College Faculty
Find Reasons to Say “Not Yet” 138
William W. Ziegler
10 Piloting the COMPASS E-Write Software at Jackson State
Community College 147
, Teri T. Maddox
11 The Role of the Writing Coordinator in a Culture of Placement by
ACCUPLACER 154
Gail S. Corso
12 Always Already: Automated Essay Scoring and Grammar-Checkers
in College Writing Courses 166
Carl Whithaus
13 Automated Essay Grading in the Sociology Classroom: Finding
Common Ground 177
Edward Brent and Martha Townsend
14 Automated Writing Instruction: Computer-Assisted or Computer-
Driven Pedagogies? 199
Beth Ann Rothermel
15 Why Less Is Not More: What We Lose by Letting a Computer
Score Writing Samples 211
William Condon
16 More Work for Teacher? Possible Futures of Teaching Writing in
the Age of Computerized Writing Assessment 221
Bob Broad
17 A Bibliography of Machine Scoring of Student Writing,
1962–2005 234
Richard H. Haswell
Glossary 244
Notes 246
References 251
Index 262
, INTRO DUCT ION
Patricia Freitag Ericsson and Richard H. Haswell
We’re in the fifth year of the twenty-first century and the Parliament
of India, several universities in Italy, and four Catholic churches in
Monterrey, Mexico, all have bought cell-phone jammers. Meanwhile in
the State of Texas, USA, the State Board of Education has decided that
students who fail the essay-writing part of the state’s college entrance
examination can retake it either with ACT’s COMPASS tests using e-
Write or with the College Board’s ACCUPLACER tests using WritePlacer
Plus. Though dispersed geographically, these events have one thing in
common. They illustrate how new technology can sneak in the back
door and establish itself while those at the front gates, nominally in
charge, are not much noticing. All of a sudden cell phones are disturb-
ing legislative sessions and church services and allowing students to
cheat on examinations in new ways. All of a sudden students can pass
entrance examination essays in ways never allowed before, with their
essays scored by machines running commercial software programs. How
did this technology happen so fast?
And where were educators when it happened? We will leave the MPs
in India and the deacons in Mexico to account for themselves, but as for
automated essay scoring in the State of Texas, college educators can only
throw up their hands. The decisions on e-Write and WritePlacer Plus were
made by state government officials and industry lobbyists with no input
from writing experts or administrators in higher education. The Texas
step toward machine grading may not be typical so far. But in the near
future there will be plenty of like steps taken everywhere in academe.
The analysis and scoring of student essays by computer—the history,
the mechanisms, the theory, and the educational consequences—is the
topic of this collection of essays. It is an understatement to say that the
topic is rapidly growing in importance at all levels of the educational
enterprise, and that the perspective on it has been, up to this point,
dominated almost exclusively by the commercial purveyors of the prod-
uct. Other than the notable exceptions of articles by Dennis Baron in
the Chronicle of Higher Education (1998), Anne Herrington and Charles
Moran in College English (2001), Julie Cheville in English Journal (2004),
and Michael Williamson in the Journal of Writing Assessment (2004) and
