DEMUX: Data-efficient Multilingual Learning

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3A8CLD4YKS" target="_blank" >RIV/00216208:11320/25:8CLD4YKS - isvavai.cz</a>

Result on the web

<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85200029020&partnerID=40&md5=b6a4c04f2d55b99b851622a9b69c58d1" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85200029020&partnerID=40&md5=b6a4c04f2d55b99b851622a9b69c58d1</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

DEMUX: Data-efficient Multilingual Learning

Original language description

Pre-trained multilingual models have enabled deployment of NLP technologies for multiple languages. However, optimally fine-tuning these models under an annotation budget, such that performance on desired target languages is jointly maximized, still remains an open question. In this paper, we introduce DEMUX, a framework that prescribes the exact data-points to label from vast amounts of unlabelled multilingual data, having unknown degrees of overlap with the target set. Unlike most prior works, our end-to-end framework is language-agnostic, accounts for model representations, and supports multilingual target configurations. Our active learning strategies rely upon distance and uncertainty measures to select task-specific neighbors that are most informative to label, given a model. DEMUX outperforms strong baselines in 84% of the test cases, in the zero-shot setting of disjoint source and target language sets (including multilingual target pools), across three models and four tasks. Notably, in low-budget settings (5-100 examples), we observe gains of up to 8-11 F1 points. Our code is released here. © 2024 Association for Computational Linguistics.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proc. Conf. North American Chapter Assoc. Comput. Linguist.: Hum. Lang. Technol., NAACL

ISBN

979-889176114-8

ISSN

—

e-ISSN

—

Number of pages

14

Pages from-to

7416-7429

Publisher name

Association for Computational Linguistics (ACL)

Place of publication

—

Event location

Mexico City

Event date

Jan 1, 2025

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—