Abilities of Contrastive Soft Prompting for Open Domain Rhetorical Question Detection
Identifikátory výsledku
Kód výsledku v IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F25%3A43976642" target="_blank" >RIV/49777513:23520/25:43976642 - isvavai.cz</a>
Výsledek na webu
<a href="https://www.cai.sk/ojs/index.php/cai/article/view/2025_3_523" target="_blank" >https://www.cai.sk/ojs/index.php/cai/article/view/2025_3_523</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.31577/cai_2025_3_523" target="_blank" >10.31577/cai_2025_3_523</a>
Alternativní jazyky
Jazyk výsledku
angličtina
Název v původním jazyce
Abilities of Contrastive Soft Prompting for Open Domain Rhetorical Question Detection
Popis výsledku v původním jazyce
In this work, we start by demonstrating experimentally thatrhetorical question detection is still a challenging task, even for state-of-the-art Large Language Models (LLMs).We then propose an approach that boosts the performances of such LLMs by training a soft prompt in a waythat enables building a joint embedding space from multiple loosely related corpora.The advantages of using a soft-prompt compared to finetuning is to limit the training costs and combat overfittingand forgetting. Soft prompting is often viewed as a way to guide the model towards a specific known task, or tointroduce new knowledge into the model through in-context learning.We further show that soft prompting may also be used to modify the geometry of the embedding space, so thatthe distance between embeddings becomes semantically relevant for a target task, similarly to what is commonlyachieved with contrastive finetuning.We exploit this property to combat data scarcity for the task of rhetorical question detection bymerging several datasets into a joint semantic embedding space.We finally show on the standard Switchboard dataset that the resulting BERT-based model nearly divides by 2the number of errors as compared to Flan-T5-XXL with only 5 few-shot labeled samples, thanks to this jointembedding space. We have chosen in our experiments a BERT model because it has already been shown with S-BERT thatcontrastive finetuning of BERT leads to semantically meaningful representations. Therefore, we also show that thisproperty of BERT nicely transfers to the soft-prompting paradigm.Finally, we qualitatively analyze the resulting embedding space and propose a few heuristic criteria to selectappropriate related tasks for inclusion into the pool of training datasets.
Název v anglickém jazyce
Abilities of Contrastive Soft Prompting for Open Domain Rhetorical Question Detection
Popis výsledku anglicky
In this work, we start by demonstrating experimentally thatrhetorical question detection is still a challenging task, even for state-of-the-art Large Language Models (LLMs).We then propose an approach that boosts the performances of such LLMs by training a soft prompt in a waythat enables building a joint embedding space from multiple loosely related corpora.The advantages of using a soft-prompt compared to finetuning is to limit the training costs and combat overfittingand forgetting. Soft prompting is often viewed as a way to guide the model towards a specific known task, or tointroduce new knowledge into the model through in-context learning.We further show that soft prompting may also be used to modify the geometry of the embedding space, so thatthe distance between embeddings becomes semantically relevant for a target task, similarly to what is commonlyachieved with contrastive finetuning.We exploit this property to combat data scarcity for the task of rhetorical question detection bymerging several datasets into a joint semantic embedding space.We finally show on the standard Switchboard dataset that the resulting BERT-based model nearly divides by 2the number of errors as compared to Flan-T5-XXL with only 5 few-shot labeled samples, thanks to this jointembedding space. We have chosen in our experiments a BERT model because it has already been shown with S-BERT thatcontrastive finetuning of BERT leads to semantically meaningful representations. Therefore, we also show that thisproperty of BERT nicely transfers to the soft-prompting paradigm.Finally, we qualitatively analyze the resulting embedding space and propose a few heuristic criteria to selectappropriate related tasks for inclusion into the pool of training datasets.
Klasifikace
Druh
J<sub>imp</sub> - Článek v periodiku v databázi Web of Science
CEP obor
—
OECD FORD obor
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Návaznosti výsledku
Projekt
—
Návaznosti
S - Specificky vyzkum na vysokych skolach
Ostatní
Rok uplatnění
2025
Kód důvěrnosti údajů
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Údaje specifické pro druh výsledku
Název periodika
COMPUTING AND INFORMATICS
ISSN
1335-9150
e-ISSN
2585-8807
Svazek periodika
44
Číslo periodika v rámci svazku
3
Stát vydavatele periodika
SK - Slovenská republika
Počet stran výsledku
26
Strana od-do
523-548
Kód UT WoS článku
001621681300002
EID výsledku v databázi Scopus
2-s2.0-105022496658