Multilevel Analysis of Biomedical Domain Adaptation of Llama 2: What Matters the Most? A Case Study

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3A6LB6FGDZ" target="_blank" >RIV/00216208:11320/25:6LB6FGDZ - isvavai.cz</a>

Result on the web

<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85204435756&partnerID=40&md5=5eb5fe3c36a68abaa812f0b2322f179f" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85204435756&partnerID=40&md5=5eb5fe3c36a68abaa812f0b2322f179f</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Multilevel Analysis of Biomedical Domain Adaptation of Llama 2: What Matters the Most? A Case Study

Original language description

Domain adaptation of Large Language Models (LLMs) leads to models better suited for a particular domain by capturing patterns from domain text which leads to improvements in downstream tasks. To the naked eye, these improvements are visible; however, the patterns are not so. How can we know which patterns and how much they contribute to changes in downstream scores? Through a Multilevel Analysis we discover and quantify the effect of text patterns on downstream scores of domain-adapted Llama 2 for the task of sentence similarity (BIOSSES dataset). We show that text patterns from PubMed abstracts such as clear writing and simplicity, as well as the amount of biomedical information, are the key for improving downstream scores. Also, we show how another factor not usually quantified contributes equally to downstream scores: choice of hyperparameters for both domain adaptation and fine-tuning.. ©2024 Association for Computational Linguistics.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

BioNLP - Meet. ACL Spec. Interest Group Biomed. Nat. Lang. Process., Proc. Workshop Shar. Tasks

ISBN

979-889176130-8

ISSN

—

e-ISSN

—

Number of pages

8

Pages from-to

449-456

Publisher name

Association for Computational Linguistics (ACL)

Place of publication

—

Event location

Bangkok

Event date

Jan 1, 2025

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—