Quality and Efficiency of Manual Annotation: Pre-annotation Bias
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F22%3A10457044" target="_blank" >RIV/00216208:11320/22:10457044 - isvavai.cz</a>
Result on the web
<a href="https://aclanthology.org/2022.lrec-1.312" target="_blank" >https://aclanthology.org/2022.lrec-1.312</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
Quality and Efficiency of Manual Annotation: Pre-annotation Bias
Original language description
This paper presents an analysis of annotation using an automatic pre-annotation for a mid-level annotation complexity task - dependency syntax annotation. It compares the annotation efforts made by annotators using a pre-annotated version (with a high-accuracy parser) and those made by fully manual annotation. The aim of the experiment is to judge the final annotation quality when pre-annotation is used. In addition, it evaluates the effect of automatic linguistically-based (rule-formulated) checks and another annotation on the same data available to the annotators, and their influence on annotation quality and efficiency. The experiment confirmed that the pre-annotation is an efficient tool for faster manual syntactic annotation which increases the consistency of the resulting annotation without reducing its quality.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
Result was created during the realization of more than one project. More information in the Projects tab.
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)
Others
Publication year
2022
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 13th Conference on Language Resources and Evaluation (LREC 2022)
ISBN
979-10-95546-72-6
ISSN
—
e-ISSN
—
Number of pages
10
Pages from-to
2909-2918
Publisher name
European Language Resources Association
Place of publication
Marseille, France
Event location
Marseille, France
Event date
Jun 20, 2022
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—