Deep Architectures for Neural Machine Translation

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F17%3A10372162" target="_blank" >RIV/00216208:11320/17:10372162 - isvavai.cz</a>

Result on the web

—

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Deep Architectures for Neural Machine Translation

Original language description

It has been shown that increasing model depth improves the quality of neural machine translation. However, different architectural variants to increase model depth have been proposed, and so far, there has been no thorough comparative study. In this work, we describe and evaluate several existing approaches to introduce depth in neural machine translation. Additionally, we explore novel architectural variants, including deep transition RNNs, and we vary how attention is used in the deep decoder. We introduce a novel &quot;BiDeep&quot; RNN architecture that combines deep transition RNNs and stacked RNNs. Our evaluation is carried out on the English to German WMT news translation dataset, using a single-GPU machine for both training and inference. We find that several of our proposed architectures improve upon existing approaches in terms of speed and translation quality. We obtain best improvements with a BiDeep RNN of combined depth 8, obtaining an average improvement of 1.5 BLEU over a strong shallow baseline.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

R - Projekt Ramcoveho programu EK

Publication year

2017

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the Second Conference on Machine Translation, Volume 1: Research Papers

ISBN

978-1-945626-96-8

ISSN

—

e-ISSN

neuvedeno

Number of pages

9

Pages from-to

99-107

Publisher name

Association for Computational Linguistics

Place of publication

Stroudsburg, PA, USA

Event location

København, Denmark

Event date

Sep 7, 2017

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—