A Primer on Pretrained Multilingual Language Models
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F26%3A5YJ38PUC" target="_blank" >RIV/00216208:11320/26:5YJ38PUC - isvavai.cz</a>
Result on the web
<a href="http://dx.doi.org/10.1145/3727339" target="_blank" >http://dx.doi.org/10.1145/3727339</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1145/3727339" target="_blank" >10.1145/3727339</a>
Alternative languages
Result language
angličtina
Original language name
A Primer on Pretrained Multilingual Language Models
Original language description
Multilingual Language Models (MLLMs) such as mBERT, XLM, XLM-R, and the like, have emerged as a viable option for bringing the power of pretraining to a large number of languages. Given their success in zero-shot transfer learning, there has emerged a large body of work in (i) building bigger MLLMs covering a large number of languages, (ii) creating exhaustive benchmarks covering a wider variety of tasks and languages for evaluating MLLMs, (iii) analysing the performance of MLLMs on monolingual, zero-shot cross-lingual and bilingual tasks, (iv) understanding the universal language patterns (if any) learnt by MLLMs, and (v) augmenting the (often) limited capacity of MLLMs to improve their performance on seen or even unseen languages. In this survey, we review the existing literature covering the above broad areas of research pertaining to MLLMs. Based on our survey, we recommend some promising directions of future research. © 2025 Copyright held by the owner/author(s).
Czech name
—
Czech description
—
Classification
Type
J<sub>SC</sub> - Article in a specialist periodical, which is included in the SCOPUS database
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
—
Others
Publication year
2025
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Name of the periodical
ACM Computing Surveys
ISSN
0360-0300
e-ISSN
—
Volume of the periodical
57
Issue of the periodical within the volume
9
Country of publishing house
US - UNITED STATES
Number of pages
23
Pages from-to
232
UT code for WoS article
—
EID of the result in the Scopus database
2-s2.0-105009263926