Diacritics Restoration Using Neural Networks

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F18%3A10390197" target="_blank" >RIV/00216208:11320/18:10390197 - isvavai.cz</a>

Result on the web

<a href="http://www.lrec-conf.org/proceedings/lrec2018/summaries/573.html" target="_blank" >http://www.lrec-conf.org/proceedings/lrec2018/summaries/573.html</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Diacritics Restoration Using Neural Networks

Original language description

In this paper, we describe a novel combination of a character-level recurrent neural-network based model and a language model applied to diacritics restoration. In many cases in the past and still at present, people often replace characters with diacritics with their ASCII counterparts. Despite the fact that the resulting text is usually easy to understand for humans, it is much harder for further computational processing. This paper opens with a discussion of applicability of restoration of diacritics in selected languages. Next, we present a neural network-based approach to diacritics generation. The core component of our model is a bidirectional recurrent neural network operating at a character level. We evaluate the model on two existing datasets consisting of four European languages. When combined with a language model, our model reduces the error of current best systems by 20% to 64%. Finally, we propose a pipeline for obtaining consistent diacritics restoration datasets for twelve languages and

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

Result was created during the realization of more than one project. More information in the Projects tab.

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2018

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the 11th International Conference on Language Resources and Evaluation (LREC 2018)

ISBN

979-10-95546-00-9

ISSN

—

e-ISSN

neuvedeno

Number of pages

10

Pages from-to

1-10

Publisher name

European Language Resources Association

Place of publication

Paris, France

Event location

Miyazaki, Japan

Event date

May 7, 2018

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—