Are the Multilingual Models Better? Improving Czech Sentiment with Transformers

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F21%3A43962571" target="_blank" >RIV/49777513:23520/21:43962571 - isvavai.cz</a>

Result on the web

<a href="https://aclanthology.org/2021.ranlp-1.128/" target="_blank" >https://aclanthology.org/2021.ranlp-1.128/</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.26615/978-954-452-072-4_128" target="_blank" >10.26615/978-954-452-072-4_128</a>

Result language

angličtina

Original language name

Are the Multilingual Models Better? Improving Czech Sentiment with Transformers

Original language description

In this paper, we aim at improving Czech sentiment with transformer-based models and their multilingual versions. More concretely, we study the task of polarity detection for the Czech language on three sentiment polarity datasets. We fine-tune and perform experiments with five multilingual and three monolingual models. We compare the monolingual and multilingual models&apos; performance, including comparison with the older approach based on recurrent neural networks. Furthermore, we test the multilingual models and their ability to transfer knowledge from English to Czech (and vice versa) with zero-shot cross-lingual classification. Our experiments show that the huge multilingual models can overcome the performance of the monolingual models. They are also able to detect polarity in another language without any training data, with performance not worse than 4.4 % compared to state-of-the-art monolingual trained models. Moreover, we achieved new state-of-the-art results on all three datasets.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

S - Specificky vyzkum na vysokych skolach

Publication year

2021

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Deep Learning for Natural Language Processing Methods and Applications

ISBN

978-954-452-072-4

ISSN

1313-8502

e-ISSN

2603-2813

Number of pages

12

Pages from-to

1138-1149

Publisher name

INCOMA Ltd.

Place of publication

Shoumen

Event location

online

Event date

Sep 1, 2021

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—