I-vectors in language modeling: An efficient way of domain adaptation for feed-forward models
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F18%3APU130794" target="_blank" >RIV/00216305:26230/18:PU130794 - isvavai.cz</a>
Result on the web
<a href="https://www.isca-speech.org/archive/Interspeech_2018/abstracts/1070.html" target="_blank" >https://www.isca-speech.org/archive/Interspeech_2018/abstracts/1070.html</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.21437/Interspeech.2018-1070" target="_blank" >10.21437/Interspeech.2018-1070</a>
Alternative languages
Result language
angličtina
Original language name
I-vectors in language modeling: An efficient way of domain adaptation for feed-forward models
Original language description
We show an effective way of adding context information to shallow neural language models. We propose to use Subspace Multinomial Model (SMM) for context modeling and we add the extracted i-vectors in a computationally efficient way. By adding this information, we shrink the gap between shallow feed-forward network and an LSTM from 65 to 31 points of perplexity on the Wikitext-2 corpus (in the case of neural 5-gram model). Furthermore, we show that SMM i-vectors are suitable for domain adaptation and a very small amount of adaptation data (e.g. endmost 5% of a Wikipedia article) brings a substantial improvement. Our proposed changes are compatible with most optimization techniques used for shallow feedforward LMs.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
Result was created during the realization of more than one project. More information in the Projects tab.
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)
Others
Publication year
2018
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of Interspeech 2018
ISBN
—
ISSN
1990-9772
e-ISSN
—
Number of pages
5
Pages from-to
3383-3387
Publisher name
International Speech Communication Association
Place of publication
Hyderabad
Event location
Hyderabad, India
Event date
Sep 2, 2018
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
000465363900706