Argument Mining in BioMedicine: Zero-Shot, In-Context Learning and Fine-tuning with LLMs

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F67985807%3A_____%2F24%3A00602747" target="_blank" >RIV/67985807:_____/24:00602747 - isvavai.cz</a>

Result on the web

<a href="https://ceur-ws.org/Vol-3878/15_main_long.pdf" target="_blank" >https://ceur-ws.org/Vol-3878/15_main_long.pdf</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Argument Mining in BioMedicine: Zero-Shot, In-Context Learning and Fine-tuning with LLMs

Original language description

Argument Mining (AM) aims to extract the complex argumentative structure of a text and Argument Type Classification (ATC) is an essential sub-task of AM. Large Language Models (LLMs) have shown impressive capabilities in most NLP tasks and beyond. However, fine-tuning LLMs can be challenging. In-Context Learning (ICL) has been suggested as a bridging paradigm between training-free and fine-tuning settings for LLMs. In ICL, an LLM is conditioned to solve tasks using a few solved demonstration examples included in its prompt. We focuse on AM in the biomedical AbstRCT dataset. We address ATC using quantized and unquantized LLaMA-3 models through zero-shot learning, in-context learning, and fine-tuning approaches. We introduce a novel ICL strategy that combines kNN-based example selection with majority vote ensembling, along with a well-designed fine-tuning strategy for ATC. In zero-shot setting, we show that LLaMA-3 fails to achieve acceptable classification results, suggesting the need for additional training modalities. However, in our ICL training-free setting, LLaMA-3 can leverage relevant information from only a few demonstration examples to achieve very competitive results. Finally, in our fine-tuning setting, LLaMA-3 achieves state-of-the-art performance on ATC task in AbstRCT dataset.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/GA22-02067S" target="_blank" >GA22-02067S: AppNeCo: Approximate Neurocomputing</a><br>

Continuities

I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the Tenth Italian Conference on Computational Linguistics (CLiC-it 2024)

ISBN

—

ISSN

1613-0073

e-ISSN

—

Number of pages

10

Pages from-to

15

Publisher name

Technical University & CreateSpace Independent Publishing

Place of publication

Aachen

Event location

Pisa

Event date

Dec 4, 2024

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—