Annotators-in-the-loop: Testing a Novel Annotation Procedure on Italian Case Law
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3ALHVZXBXB" target="_blank" >RIV/00216208:11320/25:LHVZXBXB - isvavai.cz</a>
Result on the web
<a href="https://aclanthology.org/2023.law-1.12" target="_blank" >https://aclanthology.org/2023.law-1.12</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.18653/v1/2023.law-1.12" target="_blank" >10.18653/v1/2023.law-1.12</a>
Alternative languages
Result language
angličtina
Original language name
Annotators-in-the-loop: Testing a Novel Annotation Procedure on Italian Case Law
Original language description
The availability of annotated legal corpora is crucial for a number of tasks, such as legal search, legal information retrieval, and predictive justice. Annotation is mostly assumed to be a straightforward task: as long as the annotation scheme is well defined and the guidelines are clear, annotators are expected to agree on the labels. This is not always the case, especially in legal annotation, which can be extremely difficult even for expert annotators. We propose a legal annotation procedure that takes into account annotator certainty and improves it through negotiation. We also collect annotator feedback and show that our approach contributes to a positive annotation environment. Our work invites reflection on often neglected ethical concerns regarding legal annotation.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
—
Others
Publication year
2023
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII)
ISBN
978-1-959429-83-8
ISSN
—
e-ISSN
—
Number of pages
11
Pages from-to
118-128
Publisher name
Association for Computational Linguistics
Place of publication
—
Event location
Toronto, Canada
Event date
Jan 1, 2025
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—