What do BERT Word Embeddings Learn about the French Language?

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3AJ5GETLXX" target="_blank" >RIV/00216208:11320/25:J5GETLXX - isvavai.cz</a>

Výsledek na webu

<a href="https://aclanthology.org/2024.clib-1.2" target="_blank" >https://aclanthology.org/2024.clib-1.2</a>

DOI - Digital Object Identifier

—

Jazyk výsledku

angličtina

Název v původním jazyce

What do BERT Word Embeddings Learn about the French Language?

Popis výsledku v původním jazyce

Pre-trained word embeddings (for example, BERT-like) have been successfully used in a variety of downstream tasks. However, do all embeddings, obtained from the models of the same architecture, encode information in the same way? Does the size of the model correlate to the quality of the information encoding? In this paper, we will attempt to dissect the dimensions of several BERT-like models that were trained on the French language to find where grammatical information (gender, plurality, part of speech) and semantic features might be encoded. In addition to this, we propose a framework for comparing the quality of encoding in different models.

Název v anglickém jazyce

What do BERT Word Embeddings Learn about the French Language?

Popis výsledku anglicky

Pre-trained word embeddings (for example, BERT-like) have been successfully used in a variety of downstream tasks. However, do all embeddings, obtained from the models of the same architecture, encode information in the same way? Does the size of the model correlate to the quality of the information encoding? In this paper, we will attempt to dissect the dimensions of several BERT-like models that were trained on the French language to find where grammatical information (gender, plurality, part of speech) and semantic features might be encoded. In addition to this, we propose a framework for comparing the quality of encoding in different models.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

—

Rok uplatnění

2024

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceedings of the Sixth International Conference on Computational Linguistics in Bulgaria (CLIB 2024)

ISBN

—

ISSN

2367-5578

e-ISSN

—

Počet stran výsledku

19

Strana od-do

14-32

Název nakladatele

Department of Computational Linguistics, Institute for Bulgarian Language, Bulgarian Academy of Sciences

Místo vydání

—

Místo konání akce

Sofia, Bulgaria

Datum konání akce

1. 1. 2025

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—