Transfer Learning of Transformers for Spoken Language Understanding
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F22%3A43965700" target="_blank" >RIV/49777513:23520/22:43965700 - isvavai.cz</a>
Result on the web
<a href="https://link.springer.com/chapter/10.1007/978-3-031-16270-1_40" target="_blank" >https://link.springer.com/chapter/10.1007/978-3-031-16270-1_40</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/978-3-031-16270-1_40" target="_blank" >10.1007/978-3-031-16270-1_40</a>
Alternative languages
Result language
angličtina
Original language name
Transfer Learning of Transformers for Spoken Language Understanding
Original language description
Pre-trained models used in the transfer-learning scenario are recently becoming very popular. Such models benefit from the availability of large sets of unlabeled data. Two kinds of such models include the Wav2Vec 2.0 speech recognizer and T5 text-to-text transformer. In this paper, we describe a novel application of such models for dialog systems, where both the speech recognizer and the spoken language understanding modules are represented as Transformer models. Such composition outperforms the baseline based on the DNN-HMM speech recognizer and CNN understanding.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
20205 - Automation and control systems
Result continuities
Project
<a href="/en/project/GA22-27800S" target="_blank" >GA22-27800S: Transformers of multiple modalities for more natural spoken dialog</a><br>
Continuities
P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)
Others
Publication year
2022
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Text, Speech, and Dialogue 25th International Conference, TSD 2022, Brno, Czech Republic, September 6–9, 2022, Proceedings
ISBN
978-3-031-16269-5
ISSN
0302-9743
e-ISSN
1611-3349
Number of pages
12
Pages from-to
489-500
Publisher name
Springer International Publishing
Place of publication
Cham
Event location
Brno, Czech Republic
Event date
Sep 6, 2022
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—