Are Multilingual Neural Machine Translation Models Better at Capturing Linguistic Features?

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F20%3A10424330" target="_blank" >RIV/00216208:11320/20:10424330 - isvavai.cz</a>

Výsledek na webu

<a href="https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=Ar.-9Myndf" target="_blank" >https://verso.is.cuni.cz/pub/verso.fpl?fname=obd_publikace_handle&handle=Ar.-9Myndf</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.14712/00326585.009" target="_blank" >10.14712/00326585.009</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Are Multilingual Neural Machine Translation Models Better at Capturing Linguistic Features?

Popis výsledku v původním jazyce

We investigate the effect of training NMT models on multiple target languages. We hypothesize that the integration of multiple languages and the increase of linguistic diversity will lead to a stronger representation of syntactic and semantic features captured by the model. We test our hypothesis on two different NMT architectures: The widely-used Transformer architecture and the Attention Bridge architecture. We train models on Europarl data and quantify the level of syntactic and semantic information discovered by the models using three different methods: SentEval linguistic probing tasks, an analysis of the attention structures regarding the inherent phrase and dependency information and a structural probe on contextualized word representations. Our results show evidence that with growing number of target languages the Attention Bridge model increasingly picks up certain linguistic properties including some syntactic and semantic aspects of the sentence whereas Transformer models are largely unaffe

Název v anglickém jazyce

Are Multilingual Neural Machine Translation Models Better at Capturing Linguistic Features?

Popis výsledku anglicky

We investigate the effect of training NMT models on multiple target languages. We hypothesize that the integration of multiple languages and the increase of linguistic diversity will lead to a stronger representation of syntactic and semantic features captured by the model. We test our hypothesis on two different NMT architectures: The widely-used Transformer architecture and the Attention Bridge architecture. We train models on Europarl data and quantify the level of syntactic and semantic information discovered by the models using three different methods: SentEval linguistic probing tasks, an analysis of the attention structures regarding the inherent phrase and dependency information and a structural probe on contextualized word representations. Our results show evidence that with growing number of target languages the Attention Bridge model increasingly picks up certain linguistic properties including some syntactic and semantic aspects of the sentence whereas Transformer models are largely unaffe

Druh

J_ost - Ostatní články v recenzovaných periodicích

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/GA18-02196S" target="_blank" >GA18-02196S: Reprezentace lingvistické struktury v neuronových sítích</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2020

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

The Prague Bulletin of Mathematical Linguistics

ISSN

0032-6585

e-ISSN

—

Svazek periodika

115

Číslo periodika v rámci svazku

1

Stát vydavatele periodika

CZ - Česká republika

Počet stran výsledku

20

Strana od-do

143-162

Kód UT WoS článku

—

EID výsledku v databázi Scopus

—