Learning Document Embeddings Along With Their Uncertainties

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F20%3APU138655" target="_blank" >RIV/00216305:26230/20:PU138655 - isvavai.cz</a>

Výsledek na webu

<a href="https://ieeexplore.ieee.org/document/9149686" target="_blank" >https://ieeexplore.ieee.org/document/9149686</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/TASLP.2020.3012062" target="_blank" >10.1109/TASLP.2020.3012062</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Learning Document Embeddings Along With Their Uncertainties

Popis výsledku v původním jazyce

Majority of the text modeling techniques yield only point-estimates of document embeddings and lack in capturing the uncertainty of the estimates. These uncertainties give a notion of how well the embeddings represent a document. We present Bayesian subspace multinomial model (Bayesian SMM), a generative log-linear model that learns to represent documents in the form of Gaussian distributions, thereby encoding the uncertainty in its covariance. Additionally, in the proposed Bayesian SMM, we address a commonly encountered problem of intractability that appears during variational inference in mixed-logit models. We also present a generative Gaussian linear classifier for topic identification that exploits the uncertainty in document embeddings. Our intrinsic evaluation using perplexity measure shows that the proposed Bayesian SMM fits the unseen test data better as compared to the state-of-the-art neural variational document model on (Fisher) speech and (20Newsgroups) text corpora. Our topic identification experiments showthat the proposed systems are robust to over-fitting on unseen test data. The topic ID results show that the proposedmodel outperforms state-of-the-art unsupervised topic models and achieve comparable results to the state-of-the-art fully supervised discriminative models.

Název v anglickém jazyce

Learning Document Embeddings Along With Their Uncertainties

Popis výsledku anglicky

Majority of the text modeling techniques yield only point-estimates of document embeddings and lack in capturing the uncertainty of the estimates. These uncertainties give a notion of how well the embeddings represent a document. We present Bayesian subspace multinomial model (Bayesian SMM), a generative log-linear model that learns to represent documents in the form of Gaussian distributions, thereby encoding the uncertainty in its covariance. Additionally, in the proposed Bayesian SMM, we address a commonly encountered problem of intractability that appears during variational inference in mixed-logit models. We also present a generative Gaussian linear classifier for topic identification that exploits the uncertainty in document embeddings. Our intrinsic evaluation using perplexity measure shows that the proposed Bayesian SMM fits the unseen test data better as compared to the state-of-the-art neural variational document model on (Fisher) speech and (20Newsgroups) text corpora. Our topic identification experiments showthat the proposed systems are robust to over-fitting on unseen test data. The topic ID results show that the proposedmodel outperforms state-of-the-art unsupervised topic models and achieve comparable results to the state-of-the-art fully supervised discriminative models.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/LQ1602" target="_blank" >LQ1602: IT4Innovations excellence in science</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2020

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

IEEE/ACM TRANSACTIONS ON AUDIO, SPEECH AND LANGUAGE PROCESSING

ISSN

2329-9290

e-ISSN

2329-9304

Svazek periodika

2020

Číslo periodika v rámci svazku

28

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

14

Strana od-do

2319-2332

Kód UT WoS článku

000562410300004

EID výsledku v databázi Scopus

2-s2.0-85090796297