Hostname: page-component-8448b6f56d-tj2md Total loading time: 0 Render date: 2024-04-17T23:23:49.584Z Has data issue: false hasContentIssue false

Evaluating State-of-the-Art Treebank-style Parsers for Coh-Metrix and Other Learning Technology Environments

Published online by Cambridge University Press:  22 May 2006

CHRISTIAN F. HEMPELMANN
Affiliation:
Institute for Intelligent Systems, Departments of Computer Science and Psychology, The University of Memphis, Memphis, TN 38120, USA e-mail: chmplmnn@memphis.edu, vrus@memphis.edu, a-graesser@memphis.edu, dsmcnamr@memphis.edu
VASILE RUS
Affiliation:
Institute for Intelligent Systems, Departments of Computer Science and Psychology, The University of Memphis, Memphis, TN 38120, USA e-mail: chmplmnn@memphis.edu, vrus@memphis.edu, a-graesser@memphis.edu, dsmcnamr@memphis.edu
ARTHUR C. GRAESSER
Affiliation:
Institute for Intelligent Systems, Departments of Computer Science and Psychology, The University of Memphis, Memphis, TN 38120, USA e-mail: chmplmnn@memphis.edu, vrus@memphis.edu, a-graesser@memphis.edu, dsmcnamr@memphis.edu
DANIELLE S. MCNAMARA
Affiliation:
Institute for Intelligent Systems, Departments of Computer Science and Psychology, The University of Memphis, Memphis, TN 38120, USA e-mail: chmplmnn@memphis.edu, vrus@memphis.edu, a-graesser@memphis.edu, dsmcnamr@memphis.edu

Abstract

This paper evaluates four of the most commonly used, freely available, state-of-the-art parsers on a standard benchmark as well as with respect to a set of data relevant for measuring text cohesion, as one example of a learning technology application that requires fast and accurate syntactic parsing. We outline advantages and disadvantages of existing technologies and make recommendations. Our performance report uses traditional measures based on a gold standard as well as novel dimensions for parsing evaluation. To our knowledge, this is the first attempt to evaluate parsers across genres and grade levels for the implementation in learning technology using both gold standard and directed evaluation methods.

Type
Papers
Copyright
2006 Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)