Vše

Co hledáte?

Vše
Projekty
Výsledky výzkumu
Subjekty

Rychlé hledání

  • Projekty podpořené TA ČR
  • Významné projekty
  • Projekty s nejvyšší státní podporou
  • Aktuálně běžící projekty

Chytré vyhledávání

  • Takto najdu konkrétní +slovo
  • Takto z výsledků -slovo zcela vynechám
  • “Takto můžu najít celou frázi”

Abilities of Contrastive Soft Prompting for Open Domain Rhetorical Question Detection

Identifikátory výsledku

  • Kód výsledku v IS VaVaI

    <a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F25%3A43976642" target="_blank" >RIV/49777513:23520/25:43976642 - isvavai.cz</a>

  • Výsledek na webu

    <a href="https://www.cai.sk/ojs/index.php/cai/article/view/2025_3_523" target="_blank" >https://www.cai.sk/ojs/index.php/cai/article/view/2025_3_523</a>

  • DOI - Digital Object Identifier

    <a href="http://dx.doi.org/10.31577/cai_2025_3_523" target="_blank" >10.31577/cai_2025_3_523</a>

Alternativní jazyky

  • Jazyk výsledku

    angličtina

  • Název v původním jazyce

    Abilities of Contrastive Soft Prompting for Open Domain Rhetorical Question Detection

  • Popis výsledku v původním jazyce

    In this work, we start by demonstrating experimentally thatrhetorical question detection is still a challenging task, even for state-of-the-art Large Language Models (LLMs).We then propose an approach that boosts the performances of such LLMs by training a soft prompt in a waythat enables building a joint embedding space from multiple loosely related corpora.The advantages of using a soft-prompt compared to finetuning is to limit the training costs and combat overfittingand forgetting. Soft prompting is often viewed as a way to guide the model towards a specific known task, or tointroduce new knowledge into the model through in-context learning.We further show that soft prompting may also be used to modify the geometry of the embedding space, so thatthe distance between embeddings becomes semantically relevant for a target task, similarly to what is commonlyachieved with contrastive finetuning.We exploit this property to combat data scarcity for the task of rhetorical question detection bymerging several datasets into a joint semantic embedding space.We finally show on the standard Switchboard dataset that the resulting BERT-based model nearly divides by 2the number of errors as compared to Flan-T5-XXL with only 5 few-shot labeled samples, thanks to this jointembedding space. We have chosen in our experiments a BERT model because it has already been shown with S-BERT thatcontrastive finetuning of BERT leads to semantically meaningful representations. Therefore, we also show that thisproperty of BERT nicely transfers to the soft-prompting paradigm.Finally, we qualitatively analyze the resulting embedding space and propose a few heuristic criteria to selectappropriate related tasks for inclusion into the pool of training datasets.

  • Název v anglickém jazyce

    Abilities of Contrastive Soft Prompting for Open Domain Rhetorical Question Detection

  • Popis výsledku anglicky

    In this work, we start by demonstrating experimentally thatrhetorical question detection is still a challenging task, even for state-of-the-art Large Language Models (LLMs).We then propose an approach that boosts the performances of such LLMs by training a soft prompt in a waythat enables building a joint embedding space from multiple loosely related corpora.The advantages of using a soft-prompt compared to finetuning is to limit the training costs and combat overfittingand forgetting. Soft prompting is often viewed as a way to guide the model towards a specific known task, or tointroduce new knowledge into the model through in-context learning.We further show that soft prompting may also be used to modify the geometry of the embedding space, so thatthe distance between embeddings becomes semantically relevant for a target task, similarly to what is commonlyachieved with contrastive finetuning.We exploit this property to combat data scarcity for the task of rhetorical question detection bymerging several datasets into a joint semantic embedding space.We finally show on the standard Switchboard dataset that the resulting BERT-based model nearly divides by 2the number of errors as compared to Flan-T5-XXL with only 5 few-shot labeled samples, thanks to this jointembedding space. We have chosen in our experiments a BERT model because it has already been shown with S-BERT thatcontrastive finetuning of BERT leads to semantically meaningful representations. Therefore, we also show that thisproperty of BERT nicely transfers to the soft-prompting paradigm.Finally, we qualitatively analyze the resulting embedding space and propose a few heuristic criteria to selectappropriate related tasks for inclusion into the pool of training datasets.

Klasifikace

  • Druh

    J<sub>imp</sub> - Článek v periodiku v databázi Web of Science

  • CEP obor

    —

  • OECD FORD obor

    10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Návaznosti výsledku

  • Projekt

    —

  • Návaznosti

    S - Specificky vyzkum na vysokych skolach

Ostatní

  • Rok uplatnění

    2025

  • Kód důvěrnosti údajů

    S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Údaje specifické pro druh výsledku

  • Název periodika

    COMPUTING AND INFORMATICS

  • ISSN

    1335-9150

  • e-ISSN

    2585-8807

  • Svazek periodika

    44

  • Číslo periodika v rámci svazku

    3

  • Stát vydavatele periodika

    SK - Slovenská republika

  • Počet stran výsledku

    26

  • Strana od-do

    523-548

  • Kód UT WoS článku

    001621681300002

  • EID výsledku v databázi Scopus

    2-s2.0-105022496658