Compressed FastText Models for Czech Tagger

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216224%3A14330%2F22%3A00127484" target="_blank" >RIV/00216224:14330/22:00127484 - isvavai.cz</a>
Result on the web
<a href="https://raslan2022.nlp-consulting.net/" target="_blank" >https://raslan2022.nlp-consulting.net/</a>
DOI - Digital Object Identifier
—

Result language
angličtina
Original language name
Compressed FastText Models for Czech Tagger
Original language description
We are building a new tagger for the Czech language that uses two models: the FastText model for word embeddings and a neural network that assigns tags to tokens. In the deployment, we are struggling with model sizes. Since the model size is a common obstacle in various tasks, several compression methods exist. Authors of the methods often claim that the impact on model performance is minimal. However, the evaluation is done on the two tasks the word embeddings are evaluated on: word analogy and word similarity. No information is provided for the evaluation of subsequent tasks. In this paper, we have trained a FastText word embedding model on more recent data. We retrained the tagger with the same parameters using compressed and uncompressed variants of the original FastText model and the new one. After comparing the results, we can see quantization methods are suitable, possibly together with pruning, without significant impact on the tagger performance. The precision dropped by 0.1 percentage point only in quantized models. All tested compression methods reduce the model size 10–100 times.
Czech name
—
Czech description
—

Project
<a href="/en/project/LM2018101" target="_blank" >LM2018101: Digital Research Infrastructure for the Language Technologies, Arts and Humanities</a><br>
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year
2022
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection
Proceedings of the Sixteenth Workshop on Recent Advances in Slavonic Natural Languages Processing, RASLAN 2022
ISBN
9788026317524
ISSN
2336-4289
e-ISSN
—
Number of pages
9
Pages from-to
79-87
Publisher name
Tribun EU
Place of publication
Brno
Event location
Brno
Event date
Jan 1, 2022
Type of event by nationality
CST - Celostátní akce
UT code for WoS article
—

Similar results(10)