Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F10%3A10078024" target="_blank" >RIV/00216208:11320/10:10078024 - isvavai.cz</a>
Result on the web
—
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
Ways of Evaluation of the Annotators in Building the Prague Czech-English Dependency Treebank
Original language description
The paper presents several ways to measure and evaluate the annotation and annotators, proposed and used during the building of the Czech part of the Prague Czech-English Dependency Treebank. At first, the basic principles of the treebank annotation project are introduced (division to three layers: morphological, analytical and tectogrammatical). The main part of the paper describes in detail one of the important phases of the annotation process: three ways of evaluation of the annotators - inter-annotator agreement, error rate and performance. The measuring of the inter-annotator agreement is complicated by the fact that the data contain added and deleted nodes, making the alignment between annotations non-trivial. The error rate is measured by a setof automatic checking procedures that guard the validity of some invariants in the data. The performance of the annotators is measured by a booking web application. All three measures are later compared and related to each other.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
AI - Linguistics
OECD FORD branch
—
Result continuities
Project
Result was created during the realization of more than one project. More information in the Projects tab.
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)
Others
Publication year
2010
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 7th International Conference on Language Resources and Evaluation (LREC 2010)
ISBN
2-9517408-6-7
ISSN
—
e-ISSN
—
Number of pages
4
Pages from-to
—
Publisher name
European Language Resources Association
Place of publication
Valletta, Malta
Event location
Valletta, Malta
Event date
May 17, 2010
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—