Slovak Language Models for Basic Preprocessing Tasks in Python

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F23%3ASV3PYY48" target="_blank" >RIV/00216208:11320/23:SV3PYY48 - isvavai.cz</a>

Výsledek na webu

<a href="https://www.ceeol.com/search/article-detail?id=1191258" target="_blank" >https://www.ceeol.com/search/article-detail?id=1191258</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.2478/jazcas-2023-0049" target="_blank" >10.2478/jazcas-2023-0049</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Slovak Language Models for Basic Preprocessing Tasks in Python

Popis výsledku v původním jazyce

"We propose a Slovak language model for the spaCy library in Python. These models are easy-to-use for basic natural language processing tasks in a single package. The package contains several components for basic preprocessing tasks, such as tokenization, sentence boundary detection, syntactic parsing, lemmatization, named entity recognition, morphology analysis, and word vectors. It is based on the state-of-the-art monolingual SlovakBERT model. Named entity recognition is trained on a separate, publicly available WikiAnn database. The other statistical classifiers use a Slovak Dependency Treebank corpus. Morphological tags are compatible with the conventions of the Slovak National Corpus. The part of speech tags use conventions of the Universal Dependencies framework. We trained a separate word vector model on a web-based corpus. The training uses fastText with Floret modification. We present a series of experiments that confirm that the model performs similarly to other languages for all tasks. Training scripts and data are publicly available."

Název v anglickém jazyce

Slovak Language Models for Basic Preprocessing Tasks in Python

Popis výsledku anglicky

"We propose a Slovak language model for the spaCy library in Python. These models are easy-to-use for basic natural language processing tasks in a single package. The package contains several components for basic preprocessing tasks, such as tokenization, sentence boundary detection, syntactic parsing, lemmatization, named entity recognition, morphology analysis, and word vectors. It is based on the state-of-the-art monolingual SlovakBERT model. Named entity recognition is trained on a separate, publicly available WikiAnn database. The other statistical classifiers use a Slovak Dependency Treebank corpus. Morphological tags are compatible with the conventions of the Slovak National Corpus. The part of speech tags use conventions of the Universal Dependencies framework. We trained a separate word vector model on a web-based corpus. The training uses fastText with Floret modification. We present a series of experiments that confirm that the model performs similarly to other languages for all tasks. Training scripts and data are publicly available."

Druh

J_ost - Ostatní články v recenzovaných periodicích

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

—

Rok uplatnění

2023

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

"Jazykovednỳ časopis"

ISSN

0021-5597

e-ISSN

—

Svazek periodika

2023

Číslo periodika v rámci svazku

74

Stát vydavatele periodika

US - Spojené státy americké

Počet stran výsledku

10

Strana od-do

323-332

Kód UT WoS článku

—

EID výsledku v databázi Scopus

—