Examining Structure of Word Embeddings with PCA
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F19%3A10405584" target="_blank" >RIV/00216208:11320/19:10405584 - isvavai.cz</a>
Result on the web
<a href="http://dx.doi.org/10.1007/978-3-030-27947-9_18" target="_blank" >http://dx.doi.org/10.1007/978-3-030-27947-9_18</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/978-3-030-27947-9_18" target="_blank" >10.1007/978-3-030-27947-9_18</a>
Alternative languages
Result language
angličtina
Original language name
Examining Structure of Word Embeddings with PCA
Original language description
In this paper we compare structure of Czech word embeddings for English-Czech neural machine translation (NMT), word2vec and sentiment analysis. We show that although it is possible to successfully predict part of speech (POS) tags from word embeddings of word2vec and various translation models, not all of the embedding spaces show the same structure. The information about POS is present in word2vec embeddings, but the high degree of organization by POS in the NMT decoder suggests that this information is more important for machine translation and therefore the NMT model represents it in more direct way. Our method is based on correlation of principal component analysis (PCA) dimensions with categorical linguistic data. We also show that further examining histograms of classes along the principal component is important to understand the structure of representation of information in embeddings.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
<a href="/en/project/GA18-02196S" target="_blank" >GA18-02196S: Linguistic Structure Representation in Neural Networks</a><br>
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)
Others
Publication year
2019
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 22nd International Conference on Text, Speech and Dialogue - TSD 2019
ISBN
978-3-030-27946-2
ISSN
0302-9743
e-ISSN
—
Number of pages
13
Pages from-to
211-223
Publisher name
Springer International Publishing
Place of publication
Cham
Event location
Ljubljana, Slovenia
Event date
Sep 11, 2019
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—