Lexical Normalization Using Generative Transformer Model (LN-GTM)
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F23%3AS6IHBN47" target="_blank" >RIV/00216208:11320/23:S6IHBN47 - isvavai.cz</a>
Result on the web
<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85176453443&doi=10.1007%2fs44196-023-00366-8&partnerID=40&md5=d14a281d74227cd320a3f33a9720193d" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85176453443&doi=10.1007%2fs44196-023-00366-8&partnerID=40&md5=d14a281d74227cd320a3f33a9720193d</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/s44196-023-00366-8" target="_blank" >10.1007/s44196-023-00366-8</a>
Alternative languages
Result language
angličtina
Original language name
Lexical Normalization Using Generative Transformer Model (LN-GTM)
Original language description
"Lexical Normalization (LN) aims to normalize a nonstandard text to a standard text. This problem is of extreme importance in natural language processing (NLP) when applying existing trained models to user-generated text on social media. Users of social media tend to use non-standard language. They heavily use abbreviations, phonetic substitutions, and colloquial language. Nevertheless, most existing NLP-based systems are often designed with the standard language in mind. However, they suffer from significant performance drops due to the many out-of-vocabulary words found in social media text. In this paper, we present a new (LN) technique by utilizing a transformer-based sequence-to-sequence (Seq2Seq) to build a multilingual characters-to-words machine translation model. Unlike the majority of current methods, the proposed model is capable of recognizing and generating previously unseen words. Also, it greatly reduces the difficulties involved in tokenizing and preprocessing the nonstandard text input and the standard text output. The proposed model outperforms the winning entry to the Multilingual Lexical Normalization (MultiLexNorm) shared task at W-NUT 2021 on both intrinsic and extrinsic evaluations. © 2023, The Author(s)."
Czech name
—
Czech description
—
Classification
Type
J<sub>SC</sub> - Article in a specialist periodical, which is included in the SCOPUS database
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
—
Others
Publication year
2023
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Name of the periodical
"International Journal of Computational Intelligence Systems"
ISSN
1875-6891
e-ISSN
—
Volume of the periodical
16
Issue of the periodical within the volume
1
Country of publishing house
US - UNITED STATES
Number of pages
11
Pages from-to
1-11
UT code for WoS article
—
EID of the result in the Scopus database
2-s2.0-85176453443