Semantic Spaces for Improving language Modeling

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F14%3A43918497" target="_blank" >RIV/49777513:23520/14:43918497 - isvavai.cz</a>

Výsledek na webu

<a href="http://dx.doi.org/10.1016/j.csl.2013.05.001" target="_blank" >http://dx.doi.org/10.1016/j.csl.2013.05.001</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1016/j.csl.2013.05.001" target="_blank" >10.1016/j.csl.2013.05.001</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Semantic Spaces for Improving language Modeling

Popis výsledku v původním jazyce

Language models are crucial for many tasks in NLP (Natural Language Processing) and n-grams are the best way to build them. Huge effort is being invested in improving n-gram language models. By introducing external information (morphology, syntax, partitioning into documents, etc.) into the models a significant improvement can be achieved. The models can however be improved with no external information and smoothing is an excellent example of such an improvement. In this article we show another way of improving the models that also requires no external information. We examine patterns that can be found in large corpora by building semantic spaces (HAL, COALS, BEAGLE and others described in this article). These semantic spaces have never been tested inlanguage modeling before. Our method uses semantic spaces and clustering to build classes for a class-based language model. The class-based model is then coupled with a standard n-gram model to create a very effective language model. Our

Název v anglickém jazyce

Semantic Spaces for Improving language Modeling

Popis výsledku anglicky

Language models are crucial for many tasks in NLP (Natural Language Processing) and n-grams are the best way to build them. Huge effort is being invested in improving n-gram language models. By introducing external information (morphology, syntax, partitioning into documents, etc.) into the models a significant improvement can be achieved. The models can however be improved with no external information and smoothing is an excellent example of such an improvement. In this article we show another way of improving the models that also requires no external information. We examine patterns that can be found in large corpora by building semantic spaces (HAL, COALS, BEAGLE and others described in this article). These semantic spaces have never been tested inlanguage modeling before. Our method uses semantic spaces and clustering to build classes for a class-based language model. The class-based model is then coupled with a standard n-gram model to create a very effective language model. Our

Druh

J_x - Nezařazeno - Článek v odborném periodiku (Jimp, Jsc a Jost)

CEP obor

JD - Využití počítačů, robotika a její aplikace

OECD FORD obor

—

Projekt

<a href="/cs/project/ED1.1.00%2F02.0090" target="_blank" >ED1.1.00/02.0090: NTIS - Nové technologie pro informační společnost</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2014

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Computer Speech and Language

ISSN

0885-2308

e-ISSN

—

Svazek periodika

28

Číslo periodika v rámci svazku

1

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

18

Strana od-do

192-209

Kód UT WoS článku

—

EID výsledku v databázi Scopus

—