Designing Informative Metrics for Few-Shot Example Selection
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3AXSKGA4N7" target="_blank" >RIV/00216208:11320/25:XSKGA4N7 - isvavai.cz</a>
Result on the web
<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85205323908&partnerID=40&md5=20b2032ec42ee691e33a06d21164ee16" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85205323908&partnerID=40&md5=20b2032ec42ee691e33a06d21164ee16</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
Designing Informative Metrics for Few-Shot Example Selection
Original language description
Pretrained language models (PLMs) have shown remarkable few-shot learning capabilities when provided with properly formatted examples. However, selecting the “best” examples remains an open challenge. We propose a complexity-based prompt selection approach for sequence tagging tasks. This approach avoids the training of a dedicated model for selection of examples, and instead uses certain metrics to align the syntactico-semantic complexity of test sentences and examples. We use both sentence- and word-level metrics to match the complexity of examples to the (test) sentence being considered. Our results demonstrate that our approach extracts greater performance from PLMs: it achieves state-of-the-art performance on few-shot NER, achieving a 5% absolute improvement in F1 score on the CoNLL2003 dataset for GPT-4. We also see large gains of upto 28.85 points (F1/Acc.) in smaller models like GPT-j-6B. © 2024 Association for Computational Linguistics.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
—
Others
Publication year
2024
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proc. Annu. Meet. Assoc. Comput Linguist.
ISBN
979-889176099-8
ISSN
0736-587X
e-ISSN
—
Number of pages
9
Pages from-to
10127-10135
Publisher name
Association for Computational Linguistics (ACL)
Place of publication
—
Event location
Hybrid, Bangkok
Event date
Jan 1, 2025
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—