Annotators-in-the-loop: Testing a Novel Annotation Procedure on Italian Case Law

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3ALHVZXBXB" target="_blank" >RIV/00216208:11320/25:LHVZXBXB - isvavai.cz</a>

Výsledek na webu

<a href="https://aclanthology.org/2023.law-1.12" target="_blank" >https://aclanthology.org/2023.law-1.12</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.18653/v1/2023.law-1.12" target="_blank" >10.18653/v1/2023.law-1.12</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Annotators-in-the-loop: Testing a Novel Annotation Procedure on Italian Case Law

Popis výsledku v původním jazyce

The availability of annotated legal corpora is crucial for a number of tasks, such as legal search, legal information retrieval, and predictive justice. Annotation is mostly assumed to be a straightforward task: as long as the annotation scheme is well defined and the guidelines are clear, annotators are expected to agree on the labels. This is not always the case, especially in legal annotation, which can be extremely difficult even for expert annotators. We propose a legal annotation procedure that takes into account annotator certainty and improves it through negotiation. We also collect annotator feedback and show that our approach contributes to a positive annotation environment. Our work invites reflection on often neglected ethical concerns regarding legal annotation.

Název v anglickém jazyce

Annotators-in-the-loop: Testing a Novel Annotation Procedure on Italian Case Law

Popis výsledku anglicky

The availability of annotated legal corpora is crucial for a number of tasks, such as legal search, legal information retrieval, and predictive justice. Annotation is mostly assumed to be a straightforward task: as long as the annotation scheme is well defined and the guidelines are clear, annotators are expected to agree on the labels. This is not always the case, especially in legal annotation, which can be extremely difficult even for expert annotators. We propose a legal annotation procedure that takes into account annotator certainty and improves it through negotiation. We also collect annotator feedback and show that our approach contributes to a positive annotation environment. Our work invites reflection on often neglected ethical concerns regarding legal annotation.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

—

Rok uplatnění

2023

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceedings of the 17th Linguistic Annotation Workshop (LAW-XVII)

ISBN

978-1-959429-83-8

ISSN

—

e-ISSN

—

Počet stran výsledku

11

Strana od-do

118-128

Název nakladatele

Association for Computational Linguistics

Místo vydání

—

Místo konání akce

Toronto, Canada

Datum konání akce

1. 1. 2025

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—