Genre Transfer in NMT: Creating Synthetic Spoken Parallel Sentences using Written Parallel Data

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F23%3A10475925" target="_blank" >RIV/00216208:11320/23:10475925 - isvavai.cz</a>

Result on the web

<a href="https://www.lcs2.in/ICON-2022/index.html" target="_blank" >https://www.lcs2.in/ICON-2022/index.html</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Genre Transfer in NMT: Creating Synthetic Spoken Parallel Sentences using Written Parallel Data

Original language description

Text style transfer (TST) aims to control attributes in a given text without changing the content. The matter gets complicated when the boundary separating two styles gets blurred. We can notice similar difficulties in the case of parallel datasets in spoken and written genres. Genuine spoken features like filler words and repetitions in the existing spoken genre parallel datasets are often cleaned during transcription and translation, making the texts closer to written datasets. This poses several problems for spoken genre-specific tasks like simultaneous speech translation. This paper seeks to address the challenge of improving spoken language translations. We start by creating a genre classifier for individual sentences and then try two approaches for data augmentation using written examples: (1) a novel method that involves assembling and disassembling spoken and written neural machine translation (NMT) models, and (2) a rule-based method to inject spoken features. Though the observed results for (1) are not promising, we get some interesting insights into the solution. The model proposed in (1) fine-tuned on the synthesized data from (2) produces naturally looking spoken translations for writtenRIGHTWARDS ARROWspoken genre transfer in En-Hi translation systems. We use this system to produce a second-stage En-Hi synthetic corpus, which however lacks appropriate alignments of explicit spoken features across the languages. For the final evaluation, we fine-tune Hi-En spoken translation systems on the synthesized parallel corpora. We observe that the parallel corpus synthesized using our rule-based method produces the best results.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/GX19-26934X" target="_blank" >GX19-26934X: Neural Representations in Multi-modal and Multi-lingual Modeling</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2023

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

19th International Conference on Natural Language Processing

ISBN

978-1-959429-38-8

ISSN

—

e-ISSN

—

Number of pages

10

Pages from-to

224-233

Publisher name

Association for Computational Linguistics

Place of publication

Stroudsburg, PA, USA

Event location

Delhi, India

Event date

Dec 15, 2022

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—