A Primer on Pretrained Multilingual Language Models

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F26%3A5YJ38PUC" target="_blank" >RIV/00216208:11320/26:5YJ38PUC - isvavai.cz</a>

Výsledek na webu

<a href="http://dx.doi.org/10.1145/3727339" target="_blank" >http://dx.doi.org/10.1145/3727339</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1145/3727339" target="_blank" >10.1145/3727339</a>

Jazyk výsledku

angličtina

Název v původním jazyce

A Primer on Pretrained Multilingual Language Models

Popis výsledku v původním jazyce

Multilingual Language Models (MLLMs) such as mBERT, XLM, XLM-R, and the like, have emerged as a viable option for bringing the power of pretraining to a large number of languages. Given their success in zero-shot transfer learning, there has emerged a large body of work in (i) building bigger MLLMs covering a large number of languages, (ii) creating exhaustive benchmarks covering a wider variety of tasks and languages for evaluating MLLMs, (iii) analysing the performance of MLLMs on monolingual, zero-shot cross-lingual and bilingual tasks, (iv) understanding the universal language patterns (if any) learnt by MLLMs, and (v) augmenting the (often) limited capacity of MLLMs to improve their performance on seen or even unseen languages. In this survey, we review the existing literature covering the above broad areas of research pertaining to MLLMs. Based on our survey, we recommend some promising directions of future research. © 2025 Copyright held by the owner/author(s).

Název v anglickém jazyce

A Primer on Pretrained Multilingual Language Models

Popis výsledku anglicky

Multilingual Language Models (MLLMs) such as mBERT, XLM, XLM-R, and the like, have emerged as a viable option for bringing the power of pretraining to a large number of languages. Given their success in zero-shot transfer learning, there has emerged a large body of work in (i) building bigger MLLMs covering a large number of languages, (ii) creating exhaustive benchmarks covering a wider variety of tasks and languages for evaluating MLLMs, (iii) analysing the performance of MLLMs on monolingual, zero-shot cross-lingual and bilingual tasks, (iv) understanding the universal language patterns (if any) learnt by MLLMs, and (v) augmenting the (often) limited capacity of MLLMs to improve their performance on seen or even unseen languages. In this survey, we review the existing literature covering the above broad areas of research pertaining to MLLMs. Based on our survey, we recommend some promising directions of future research. © 2025 Copyright held by the owner/author(s).

Druh

J_SC - Článek v periodiku v databázi SCOPUS

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

—

Rok uplatnění

2025

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

ACM Computing Surveys

ISSN

0360-0300

e-ISSN

—

Svazek periodika

57

Číslo periodika v rámci svazku

9

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

23

Strana od-do

232

Kód UT WoS článku

—

EID výsledku v databázi Scopus

2-s2.0-105009263926