Computer Scoring of Essays
Online Information and Commentary

Evaluation of 2002 Stanford-9 Writing Assessment
results of experiments on the computer-scoring routine used on 9th-grade essays

Lawrence Rudner and Phill Gagne, "An Overview of Three Approaches to Scoring Written Essays by Computer," University of Maryland, College Park. (Awesome bibliography and links!)

Types of Computer-Scoring Routines:

Project Essay Grade (PEG)

Intelligent Essay Assessor (IEA)

E-rater

Bayesian Essay Test Scoring sYstem (BETSY)

Donald E. Powers, Jill Burstein (developer of E-Rater!), et. al., "Stumping E-Rater: Challenging the Validity of Automated Essay Scoring," GRE Research, GRE Board Professional Report No. 98-08bP, ETS Research Report 01-03, March 2001.

William H. Honan, "High Tech Comes to the Classroom: Machines That Grade Essays," New York Times, January 27, 1999.

"Computer to Grade GMAT Essays," FairTest Examiner, Winter 1999. (initial response to ETS move)

Bill Walsh, "Computer Scoring of Essays a Bad Idea," Media Literacy Review, University of Oregon, Eugene, College of Education. 

From McGill University, a response to and expansion on Walsh.

Vantage Learning  -- purveyors of Intellimetric, a popular essay-scoring program

Vantage Learning site with essay evaluation demo. No explanation, but potential for experimentation and empirical analysis.

Essaytech.com  -- includes a number of links to articles on weaknesses of computer scoring of essays (since this test-prep company emphasizes that, unlike its competitors, EssayTech submits all practice essays to human evaluators).

Holt Rinehart and Winston online Essay Scoring (includes free demo!):

American Board for Certification of Teacher Excellence to use Vantage Learning's computer scoring program to certify teachers!

Spin straight from a GMAT/ETS honcho, Fred McHale, in Spring, 1999:

Vantage Learning Intellimetric program used in writing classes in Columbia, SC, high school.

Hit Counter