Argument Mining with Modular BERT and Transfer Learning

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985807%3A_____%2F23%3A00577167" target="_blank" >RIV/67985807:_____/23:00577167 - isvavai.cz</a>

Výsledek na webu

<a href="http://dx.doi.org/10.1109/IJCNN54540.2023.10191968" target="_blank" >http://dx.doi.org/10.1109/IJCNN54540.2023.10191968</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1109/IJCNN54540.2023.10191968" target="_blank" >10.1109/IJCNN54540.2023.10191968</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Argument Mining with Modular BERT and Transfer Learning

Popis výsledku v původním jazyce

We introduce BERT-MINUS, a modular, feature-enriched and transfer learning enabled model for Argument Mining. BERT-MINUS consists of: 1) a joint module which embeds the paragraph text, and 2) a dedicated module, consisting of three customized BERT models, which contextualize the argument markers, argument components and additional features given as text, respectively. BERT-MINUS implements two kinds of transfer learning - auto-transfer (transfer from a task to itself) and cross-transfer (classical transfer from one task to another) - via a novel Selective Fine-tuning mechanism. BERT-MINUS achieves state-of-the-art results on the Link Identification task and competitive results on the Argument Type Classification task. The synergy between the Features as Text and Selective Fine-tuning mechanisms significantly improves the performance of the model. Our work reveals the importance and potential of transfer learning via selective fine-tuning for modular Language Models. Moreover, this study dovetails naturally into the Prompt Engineering paradigm in NLP.

Název v anglickém jazyce

Argument Mining with Modular BERT and Transfer Learning

Popis výsledku anglicky

We introduce BERT-MINUS, a modular, feature-enriched and transfer learning enabled model for Argument Mining. BERT-MINUS consists of: 1) a joint module which embeds the paragraph text, and 2) a dedicated module, consisting of three customized BERT models, which contextualize the argument markers, argument components and additional features given as text, respectively. BERT-MINUS implements two kinds of transfer learning - auto-transfer (transfer from a task to itself) and cross-transfer (classical transfer from one task to another) - via a novel Selective Fine-tuning mechanism. BERT-MINUS achieves state-of-the-art results on the Link Identification task and competitive results on the Argument Type Classification task. The synergy between the Features as Text and Selective Fine-tuning mechanisms significantly improves the performance of the model. Our work reveals the importance and potential of transfer learning via selective fine-tuning for modular Language Models. Moreover, this study dovetails naturally into the Prompt Engineering paradigm in NLP.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

<a href="/cs/project/GA22-02067S" target="_blank" >GA22-02067S: AppNeCo: Aproximativní neurovýpočty</a><br>

Návaznosti

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Rok uplatnění

2023

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

IJCNN 2023 Conference Proceedings

ISBN

978-1-6654-8867-9

ISSN

—

e-ISSN

—

Počet stran výsledku

8

Strana od-do

191330

Název nakladatele

IEEE

Místo vydání

Piscataway

Místo konání akce

Queensland

Datum konání akce

18. 6. 2023

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

001046198707037