DR-MIM: Zero-shot cross-lingual transfer via disentangled representation and mutual information maximization

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F26%3A5LH7K56Y" target="_blank" >RIV/00216208:11320/26:5LH7K56Y - isvavai.cz</a>

Result on the web

<a href="http://dx.doi.org/10.1016/j.ipm.2025.104389" target="_blank" >http://dx.doi.org/10.1016/j.ipm.2025.104389</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1016/j.ipm.2025.104389" target="_blank" >10.1016/j.ipm.2025.104389</a>

Result language

angličtina

Original language name

DR-MIM: Zero-shot cross-lingual transfer via disentangled representation and mutual information maximization

Original language description

Multilingual models have made significant progress in cross-lingual transferability through large-scale pretraining. However, the generated global representations are often mixed with language-specific noise, limiting their effectiveness in low-resource language scenarios. This paper explores how to more efficiently utilize the representations learned by multilingual pretraining models by separating language-invariant features from language-specific ones. To this end, we propose a novel cross-lingual transfer framework, DR-MIM, which explicitly decouples universal and language-specific features, reduces noise interference, and improves model stability and accuracy. Additionally, we introduce a mutual information maximization mechanism to strengthen the correlation between universal features and model outputs, further optimizing the quality of semantic representations. We conducted a systematic evaluation of this method on three cross-lingual natural language understanding benchmark datasets. On the TyDiQA dataset, DR-MIM improved the F1 score by 1.7% and the EM score by 4.5% over the best baseline. To further validate the model's generalization capability, we introduced two new tasks: paraphrase identification and natural language inference, and designed both within-language and cross-language analysis experiments. All experiments collectively covered 22 languages. Further ablation studies, generalization analysis, and visualization results all confirm the effectiveness and adaptability of our approach. © 2025

Czech name

—

Czech description

—

Type

J_SC - Article in a specialist periodical, which is included in the SCOPUS database

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2026

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical

Information Processing and Management

ISSN

0306-4573

e-ISSN

—

Volume of the periodical

63

Issue of the periodical within the volume

2

Country of publishing house

US - UNITED STATES

Number of pages

14

Pages from-to

104389

UT code for WoS article

—

EID of the result in the Scopus database

2-s2.0-105015354888