What do BERT Word Embeddings Learn about the French Language?
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3AJ5GETLXX" target="_blank" >RIV/00216208:11320/25:J5GETLXX - isvavai.cz</a>
Result on the web
<a href="https://aclanthology.org/2024.clib-1.2" target="_blank" >https://aclanthology.org/2024.clib-1.2</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
What do BERT Word Embeddings Learn about the French Language?
Original language description
Pre-trained word embeddings (for example, BERT-like) have been successfully used in a variety of downstream tasks. However, do all embeddings, obtained from the models of the same architecture, encode information in the same way? Does the size of the model correlate to the quality of the information encoding? In this paper, we will attempt to dissect the dimensions of several BERT-like models that were trained on the French language to find where grammatical information (gender, plurality, part of speech) and semantic features might be encoded. In addition to this, we propose a framework for comparing the quality of encoding in different models.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
—
Others
Publication year
2024
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the Sixth International Conference on Computational Linguistics in Bulgaria (CLIB 2024)
ISBN
—
ISSN
2367-5578
e-ISSN
—
Number of pages
19
Pages from-to
14-32
Publisher name
Department of Computational Linguistics, Institute for Bulgarian Language, Bulgarian Academy of Sciences
Place of publication
—
Event location
Sofia, Bulgaria
Event date
Jan 1, 2025
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—