Publication
Parser Evaluation over Local and Non-Local Deep Dependencies in Large Corpora
Emily Bender; Daniel Flickinger; Stephan Oepen; Yi Zhang
In: Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing (EMNLP-2011), located at Empirical Methods, July 27-29, Edinburgh, United Kingdom, Association for Computational Linguistics, 2011.
Abstract
In order to obtain a fine-grained evaluation of
parser accuracy over naturally occurring text,
we study 100 examples each of ten reasonably frequent linguistic phenomena, randomly
selected from a parsed version of the English Wikipedia. We construct a corresponding set of gold-standard target dependencies
for these 1000 sentences, operationalize mappings to these targets from seven state-of-theart parsers, and evaluate the parsers against
this data to measure their level of success in
identifying these dependencies.