Segmentation from 97% to 100%: Is It Time for Some Linguistics?

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216224%3A14330%2F12%3A00062085" target="_blank" >RIV/00216224:14330/12:00062085 - isvavai.cz</a>

Výsledek na webu

<a href="http://www.fi.muni.cz/usr/sojka/papers/sojka-raslan2012.pdf" target="_blank" >http://www.fi.muni.cz/usr/sojka/papers/sojka-raslan2012.pdf</a>

DOI - Digital Object Identifier

—

Jazyk výsledku

angličtina

Název v původním jazyce

Segmentation from 97% to 100%: Is It Time for Some Linguistics?

Popis výsledku v původním jazyce

Many tasks in natural language processing (NLP) require emph{segmentation} algorithms: segmentation of paragraph into sentences, segmentation of sentences into words is needed in languages like Chinese or Thai, segmentation of words into syllables (emph{hyphenation}) or into morphological parts (e.g. getting word stem for indexing), and many other tasks (e.g. tagging) could be formulated as segmentation problems. We evaluate methodology of using emph{competing patterns} for these tasks and decide on the complexity of creation of space-optimal (minimal) patterns that completely (100,%) implement the segmentation task. We formally define this task and prove that it is in the class of emph{non-polynomial} optimization problems. However, finding space-efficient competing patterns for real NLP tasks is feasible and gives efficient scalable solutions of segmentation task: segmentation is done in emph{constant} time with respect to the size of segmented dictionary.

Název v anglickém jazyce

Segmentation from 97% to 100%: Is It Time for Some Linguistics?

Popis výsledku anglicky

Many tasks in natural language processing (NLP) require emph{segmentation} algorithms: segmentation of paragraph into sentences, segmentation of sentences into words is needed in languages like Chinese or Thai, segmentation of words into syllables (emph{hyphenation}) or into morphological parts (e.g. getting word stem for indexing), and many other tasks (e.g. tagging) could be formulated as segmentation problems. We evaluate methodology of using emph{competing patterns} for these tasks and decide on the complexity of creation of space-optimal (minimal) patterns that completely (100,%) implement the segmentation task. We formally define this task and prove that it is in the class of emph{non-polynomial} optimization problems. However, finding space-efficient competing patterns for real NLP tasks is feasible and gives efficient scalable solutions of segmentation task: segmentation is done in emph{constant} time with respect to the size of segmented dictionary.

Druh

D - Stať ve sborníku

CEP obor

IN - Informatika

OECD FORD obor

—

Projekt

<a href="/cs/project/LA09016" target="_blank" >LA09016: Účast ČR v European Research Consortium for Informatics and Mathematics (ERCIM)</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2012

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Sixth Workshop on Recent Advances in Slavonic Natural Languages Processing, RASLAN 2012

ISBN

9788026303138

ISSN

—

e-ISSN

—

Počet stran výsledku

11

Strana od-do

121-131

Název nakladatele

Tribun EU

Místo vydání

Brno

Místo konání akce

Karlova Studánka

Datum konání akce

7. 12. 2011

Typ akce podle státní příslušnosti

EUR - Evropská akce

Kód UT WoS článku

—