U-DepPLLaMA: Universal Dependency Parsing via Auto-regressive Large Language Models

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3ABFCHW7L6" target="_blank" >RIV/00216208:11320/25:BFCHW7L6 - isvavai.cz</a>

Výsledek na webu

<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85206211493&doi=10.4000%2f125nm&partnerID=40&md5=d90ffea5d474323458bc2bc032c3e7a0" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85206211493&doi=10.4000%2f125nm&partnerID=40&md5=d90ffea5d474323458bc2bc032c3e7a0</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.4000/125nm" target="_blank" >10.4000/125nm</a>

Jazyk výsledku

angličtina

Název v původním jazyce

U-DepPLLaMA: Universal Dependency Parsing via Auto-regressive Large Language Models

Popis výsledku v původním jazyce

This paper investigates the rapidly advancing domain of Large Language Models (LLMs) and their growing potential in various fields. A central focus is the exploration of LLMs, e.g., LLaMA, as powerful tools for modeling and representing linguistic information, especially in the realm of syntax. We aim to evaluate the ability of these models to encode syntactic information, especially when explicitly supplied, through fine-tuning processes. Traditionally, Dependency Parsing has relied on specific techniques and dedicated architectures. Our research shifts this approach, conceptualizing it as a sequence-to-sequence task where Language Models interpret and transform syntax into bracketed structures that reflect dependency graphs. We introduce U-DepPLLaMA (Universal Dependency Parsing via auto-regressive LLMs based on LLaMA), a novel architecture optimized for multilingual, end-to-end Dependency Parsing. Our experimental evaluation, across 50 datasets in 26 languages from the Universal Dependency Treebank, shows that LLMs can be effectively trained for dependency parsing without the need for task-specific architectures. The results are on par with current state-of-the-art methods and demonstrate resilience across varying sentence complexities and lengths. © 2024 Associazione Italiana di Linguistica Computazionale.

Název v anglickém jazyce

U-DepPLLaMA: Universal Dependency Parsing via Auto-regressive Large Language Models

Popis výsledku anglicky

This paper investigates the rapidly advancing domain of Large Language Models (LLMs) and their growing potential in various fields. A central focus is the exploration of LLMs, e.g., LLaMA, as powerful tools for modeling and representing linguistic information, especially in the realm of syntax. We aim to evaluate the ability of these models to encode syntactic information, especially when explicitly supplied, through fine-tuning processes. Traditionally, Dependency Parsing has relied on specific techniques and dedicated architectures. Our research shifts this approach, conceptualizing it as a sequence-to-sequence task where Language Models interpret and transform syntax into bracketed structures that reflect dependency graphs. We introduce U-DepPLLaMA (Universal Dependency Parsing via auto-regressive LLMs based on LLaMA), a novel architecture optimized for multilingual, end-to-end Dependency Parsing. Our experimental evaluation, across 50 datasets in 26 languages from the Universal Dependency Treebank, shows that LLMs can be effectively trained for dependency parsing without the need for task-specific architectures. The results are on par with current state-of-the-art methods and demonstrate resilience across varying sentence complexities and lengths. © 2024 Associazione Italiana di Linguistica Computazionale.

Druh

J_SC - Článek v periodiku v databázi SCOPUS

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

—

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Italian Journal of Computational Linguistics

ISSN

2499-4553

e-ISSN

—

Svazek periodika

10

Číslo periodika v rámci svazku

1

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

18

Strana od-do

21-38

Kód UT WoS článku

—

EID výsledku v databázi Scopus

2-s2.0-85206211493