Examining Structure of Word Embeddings with PCA

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F19%3A10405584" target="_blank" >RIV/00216208:11320/19:10405584 - isvavai.cz</a>

Výsledek na webu

<a href="http://dx.doi.org/10.1007/978-3-030-27947-9_18" target="_blank" >http://dx.doi.org/10.1007/978-3-030-27947-9_18</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-030-27947-9_18" target="_blank" >10.1007/978-3-030-27947-9_18</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Examining Structure of Word Embeddings with PCA

Popis výsledku v původním jazyce

In this paper we compare structure of Czech word embeddings for English-Czech neural machine translation (NMT), word2vec and sentiment analysis. We show that although it is possible to successfully predict part of speech (POS) tags from word embeddings of word2vec and various translation models, not all of the embedding spaces show the same structure. The information about POS is present in word2vec embeddings, but the high degree of organization by POS in the NMT decoder suggests that this information is more important for machine translation and therefore the NMT model represents it in more direct way. Our method is based on correlation of principal component analysis (PCA) dimensions with categorical linguistic data. We also show that further examining histograms of classes along the principal component is important to understand the structure of representation of information in embeddings.

Název v anglickém jazyce

Examining Structure of Word Embeddings with PCA

Popis výsledku anglicky

In this paper we compare structure of Czech word embeddings for English-Czech neural machine translation (NMT), word2vec and sentiment analysis. We show that although it is possible to successfully predict part of speech (POS) tags from word embeddings of word2vec and various translation models, not all of the embedding spaces show the same structure. The information about POS is present in word2vec embeddings, but the high degree of organization by POS in the NMT decoder suggests that this information is more important for machine translation and therefore the NMT model represents it in more direct way. Our method is based on correlation of principal component analysis (PCA) dimensions with categorical linguistic data. We also show that further examining histograms of classes along the principal component is important to understand the structure of representation of information in embeddings.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/GA18-02196S" target="_blank" >GA18-02196S: Reprezentace lingvistické struktury v neuronových sítích</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2019

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceedings of the 22nd International Conference on Text, Speech and Dialogue - TSD 2019

ISBN

978-3-030-27946-2

ISSN

0302-9743

e-ISSN

—

Počet stran výsledku

13

Strana od-do

211-223

Název nakladatele

Springer International Publishing

Místo vydání

Cham

Místo konání akce

Ljubljana, Slovenia

Datum konání akce

11. 9. 2019

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—