Improving Word meaning representations using Wikipedia categories

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F18%3A43955049" target="_blank" >RIV/49777513:23520/18:43955049 - isvavai.cz</a>

Výsledek na webu

<a href="http://hdl.handle.net/11025/34807" target="_blank" >http://hdl.handle.net/11025/34807</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.14311/NNW.2018.28.029" target="_blank" >10.14311/NNW.2018.28.029</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Improving Word meaning representations using Wikipedia categories

Popis výsledku v původním jazyce

In this paper we extend Skip-Gram and Continuous Bag-of-Words Distributional word representations models via global context information. We use a corpus extracted from Wikipedia, where articles are organized in a hierarchy of categories. These categories provide useful topical information about each article. We present the four new approaches, how to enrich word meaning representation with such information. We experiment with the English Wikipedia and evaluate our models on standard word similarity and word analogy datasets. Proposed models significantly outperform other word representation methods when similar size training data of similar size is used and provide similar performance compared with methods trained on much larger datasets. Our new approach shows, that increasing the amount of unlabelled data does not necessarily increase the performance of word embeddings as much as introducing the global or sub-word information, especially when training time is taken into the consideration.

Název v anglickém jazyce

Improving Word meaning representations using Wikipedia categories

Popis výsledku anglicky

In this paper we extend Skip-Gram and Continuous Bag-of-Words Distributional word representations models via global context information. We use a corpus extracted from Wikipedia, where articles are organized in a hierarchy of categories. These categories provide useful topical information about each article. We present the four new approaches, how to enrich word meaning representation with such information. We experiment with the English Wikipedia and evaluate our models on standard word similarity and word analogy datasets. Proposed models significantly outperform other word representation methods when similar size training data of similar size is used and provide similar performance compared with methods trained on much larger datasets. Our new approach shows, that increasing the amount of unlabelled data does not necessarily increase the performance of word embeddings as much as introducing the global or sub-word information, especially when training time is taken into the consideration.

Druh

J_SC - Článek v periodiku v databázi SCOPUS

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2018

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Neural Network World

ISSN

1210-0552

e-ISSN

—

Svazek periodika

28

Číslo periodika v rámci svazku

6

Stát vydavatele periodika

CZ - Česká republika

Počet stran výsledku

12

Strana od-do

523-534

Kód UT WoS článku

—

EID výsledku v databázi Scopus

2-s2.0-85061489302