BERT-Based Sentiment Analysis Using Distillation

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F20%3A43959644" target="_blank" >RIV/49777513:23520/20:43959644 - isvavai.cz</a>
Result on the web
<a href="http://dx.doi.org/10.1007/978-3-030-59430-5_5" target="_blank" >http://dx.doi.org/10.1007/978-3-030-59430-5_5</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/978-3-030-59430-5_5" target="_blank" >10.1007/978-3-030-59430-5_5</a>

Result language
angličtina
Original language name
BERT-Based Sentiment Analysis Using Distillation
Original language description
In this paper, we present our experiments with BERT (Bidirectional Encoder Representations from Transformers) models in the task of sentiment analysis, which aims to predict the sentiment polarity for the given text. We trained an ensemble of BERT models from a large self-collected movie reviews dataset and distilled the knowledge into a single production model. Moreover, we proposed an improved BERT’s pooling layer architecture, which outperforms standard classification layer while enables per-token sentiment predictions. We demonstrate our improvements on a publicly available dataset with Czech movie reviews.
Czech name
—
Czech description
—

Project
<a href="/en/project/TN01000024" target="_blank" >TN01000024: National Competence Center - Cybernetics and Artificial Intelligence</a><br>
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year
2020
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection
Statistical Language and Speech Processing, SLSP 2020
ISBN
978-3-030-59429-9
ISSN
0302-9743
e-ISSN
1611-3349
Number of pages
13
Pages from-to
58-70
Publisher name
Springer
Place of publication
Cham
Event location
Cardiff, UK
Event date
Oct 14, 2020
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—

Similar results(10)