Towards Domain Robustness of Neural Language Models

Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216224%3A14330%2F21%3A00123248" target="_blank" >RIV/00216224:14330/21:00123248 - isvavai.cz</a>
Result on the web
<a href="https://nlp.fi.muni.cz/raslan/raslan21.pdf#page=99" target="_blank" >https://nlp.fi.muni.cz/raslan/raslan21.pdf#page=99</a>
DOI - Digital Object Identifier
—

Result language
angličtina
Original language name
Towards Domain Robustness of Neural Language Models
Original language description
This work summarises recent progress in generalization evaluation and training of deep neural networks, categorized in data-centric and model-centric overviews. Grounded in the results of the referenced work, we propose three future directions towards reaching higher robustness of language models to an unknown domain or its adaptation to an existing domain of interest. In the example propositions that practically complement each of the directions, we introduce novel ideas of a) dynamic objective selection, b) language modeling respecting the token similarities to the ground truth and c) a framework of additive component of the loss utilizing the well-performing generalization measures.
Czech name
—
Czech description
—

Publication year
2021
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection
Recent Advances in Slavonic Natural Language Processing (RASLAN 2021)
ISBN
9788026316701
ISSN
2336-4289
e-ISSN
—
Number of pages
13
Pages from-to
91-103
Publisher name
Tribun EU
Place of publication
Brno
Event location
Brno
Event date
Jan 1, 2021
Type of event by nationality
EUR - Evropská akce
UT code for WoS article
—

Similar results(10)