Vector Norms as an Approximation of Syntactic Complexity
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F23%3A5C5XGL2I" target="_blank" >RIV/00216208:11320/23:5C5XGL2I - isvavai.cz</a>
Result on the web
<a href="https://aclanthology.org/2023.resourceful-1.15/" target="_blank" >https://aclanthology.org/2023.resourceful-1.15/</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
Vector Norms as an Approximation of Syntactic Complexity
Original language description
"Internal representations in transformer models can encode useful linguistic knowledge about syntax. Such knowledge could help optimise the data annotation process. However, identifying and extracting such representations from big language models is challenging. In this paper we evaluate two multilingual transformers for the presence of knowledge about the syntactic complexity of sentences and examine different vector norms. We provide a fine-grained evaluation of different norms in different layers and for different languages. Our results suggest that no single part in the models would be the primary source for the knowledge of syntactic complexity. But some norms show a higher degree of sensitivity to syntactic complexity, depending on the language and model used."
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
—
Others
Publication year
2023
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
"Proceedings of the Second Workshop on Resources and Representations for Under-Resourced Languages and Domains"
ISBN
978-1-959429-73-9
ISSN
—
e-ISSN
—
Number of pages
11
Pages from-to
121-131
Publisher name
""
Place of publication
Tórshavn, the Faroe Islands
Event location
Tórshavn, the Faroe Islands
Event date
Jan 1, 2023
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—