Character-level inclusive transformer architecture for information gain in low resource code-mixed language

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F26%3A53BDDJYW" target="_blank" >RIV/00216208:11320/26:53BDDJYW - isvavai.cz</a>

Result on the web

<a href="http://dx.doi.org/10.1007/s00521-022-06983-2" target="_blank" >http://dx.doi.org/10.1007/s00521-022-06983-2</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/s00521-022-06983-2" target="_blank" >10.1007/s00521-022-06983-2</a>

Result language

angličtina

Original language name

Character-level inclusive transformer architecture for information gain in low resource code-mixed language

Original language description

The use of code-mixed languages in social media platforms is very common to communicate in an informal way and has immense importance in a multilingual society, like India. Implementing various NLP tasks on code-mixed language for machine comprehension and NLP applications is the need of the hour. The implementation of complex learning models is difficult due to the scarcity of available code-mixed resources. Designing more effective architectures to perform learning from low resource dataset along with transfer learning settings are the possible solutions. We propose an improvised transformer network (Character Inclusion Transformer) that utilizes and learns character-level information available in the words of code-mixed sentences. The proposed model improves the performance of the transformer model when trained from scratch using low resource code-mixed datasets. We also propose two more architecture settings, useful for transfer learning strategy using the mBERT pre-trained model. Three basic word-level tagging NLP tasks, i.e., NER, POS Tagging, and Language Identification (LID) are considered in the paper where Language Identification is specific to code-mixed language. Six separate datasets, namely IIITH NER, LID FIRE, LID ICON, LID UD, POS ICON, POS UD, have been tested, and results are reported using weighted and macro-average while evaluating precision, recall and F1 score © The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2022.

Czech name

—

Czech description

—

Type

J_SC - Article in a specialist periodical, which is included in the SCOPUS database

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2025

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical

Neural Computing and Applications

ISSN

0941-0643

e-ISSN

—

Volume of the periodical

37

Issue of the periodical within the volume

2

Country of publishing house

US - UNITED STATES

Number of pages

19

Pages from-to

559-577

UT code for WoS article

—

EID of the result in the Scopus database

2-s2.0-85125946949