Vector Norms as an Approximation of Syntactic Complexity

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F23%3A5C5XGL2I" target="_blank" >RIV/00216208:11320/23:5C5XGL2I - isvavai.cz</a>

Výsledek na webu

<a href="https://aclanthology.org/2023.resourceful-1.15/" target="_blank" >https://aclanthology.org/2023.resourceful-1.15/</a>

DOI - Digital Object Identifier

—

Jazyk výsledku

angličtina

Název v původním jazyce

Vector Norms as an Approximation of Syntactic Complexity

Popis výsledku v původním jazyce

"Internal representations in transformer models can encode useful linguistic knowledge about syntax. Such knowledge could help optimise the data annotation process. However, identifying and extracting such representations from big language models is challenging. In this paper we evaluate two multilingual transformers for the presence of knowledge about the syntactic complexity of sentences and examine different vector norms. We provide a fine-grained evaluation of different norms in different layers and for different languages. Our results suggest that no single part in the models would be the primary source for the knowledge of syntactic complexity. But some norms show a higher degree of sensitivity to syntactic complexity, depending on the language and model used."

Název v anglickém jazyce

Vector Norms as an Approximation of Syntactic Complexity

Popis výsledku anglicky

"Internal representations in transformer models can encode useful linguistic knowledge about syntax. Such knowledge could help optimise the data annotation process. However, identifying and extracting such representations from big language models is challenging. In this paper we evaluate two multilingual transformers for the presence of knowledge about the syntactic complexity of sentences and examine different vector norms. We provide a fine-grained evaluation of different norms in different layers and for different languages. Our results suggest that no single part in the models would be the primary source for the knowledge of syntactic complexity. But some norms show a higher degree of sensitivity to syntactic complexity, depending on the language and model used."

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

—

Rok uplatnění

2023

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

"Proceedings of the Second Workshop on Resources and Representations for Under-Resourced Languages and Domains"

ISBN

978-1-959429-73-9

ISSN

—

e-ISSN

—

Počet stran výsledku

11

Strana od-do

121-131

Název nakladatele

""

Místo vydání

Tórshavn, the Faroe Islands

Místo konání akce

Tórshavn, the Faroe Islands

Datum konání akce

1. 1. 2023

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—