Multi-Task Learning in Natural Language Processing: An Overview
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3AD7U9WHXB" target="_blank" >RIV/00216208:11320/25:D7U9WHXB - isvavai.cz</a>
Result on the web
<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85203247218&doi=10.1145%2f3663363&partnerID=40&md5=56e5b2119a708636335fa8acb175ff47" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85203247218&doi=10.1145%2f3663363&partnerID=40&md5=56e5b2119a708636335fa8acb175ff47</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1145/3663363" target="_blank" >10.1145/3663363</a>
Alternative languages
Result language
angličtina
Original language name
Multi-Task Learning in Natural Language Processing: An Overview
Original language description
Deep learning approaches have achieved great success in the field of Natural Language Processing (NLP). However, directly training deep neural models often suffer from overfitting and data scarcity problems that are pervasive in NLP tasks. In recent years, Multi-Task Learning (MTL), which can leverage useful information of related tasks to achieve simultaneous performance improvement on these tasks, has been used to handle these problems. In this article, we give an overview of the use of MTL in NLP tasks. We first review MTL architectures used in NLP tasks and categorize them into four classes, including parallel architecture, hierarchical architecture, modular architecture, and generative adversarial architecture. Then we present optimization techniques on loss construction, gradient regularization, data sampling, and task scheduling to properly train a multi-task model. After presenting applications of MTL in a variety of NLP tasks, we introduce some benchmark datasets. Finally, we make a conclusion and discuss several possible research directions in this field. © 2024 Copyright held by the owner/author(s). Publication rights licensed to ACM.
Czech name
—
Czech description
—
Classification
Type
J<sub>SC</sub> - Article in a specialist periodical, which is included in the SCOPUS database
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
—
Others
Publication year
2024
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Name of the periodical
ACM Computing Surveys
ISSN
0360-0300
e-ISSN
—
Volume of the periodical
56
Issue of the periodical within the volume
12
Country of publishing house
US - UNITED STATES
Number of pages
31
Pages from-to
1-31
UT code for WoS article
—
EID of the result in the Scopus database
2-s2.0-85203247218