Leveraging Codebook Knowledge with NLI and ChatGPT for Zero-Shot Political Relation Classification

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F25%3A6KY9TG3D" target="_blank" >RIV/00216208:11320/25:6KY9TG3D - isvavai.cz</a>

Result on the web

<a href="https://www.scopus.com/inward/record.uri?eid=2-s2.0-85204495330&partnerID=40&md5=f804cedfd171a383ae457d1c052471fd" target="_blank" >https://www.scopus.com/inward/record.uri?eid=2-s2.0-85204495330&partnerID=40&md5=f804cedfd171a383ae457d1c052471fd</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Leveraging Codebook Knowledge with NLI and ChatGPT for Zero-Shot Political Relation Classification

Original language description

Is it possible accurately classify political relations within evolving event ontologies without extensive annotations? This study investigates zero-shot learning methods that use expert knowledge from existing annotation codebook, and evaluates the performance of advanced ChatGPT (GPT-3.5/4) and a natural language inference (NLI)-based model called ZSP. ChatGPT uses codebook's labeled summaries as prompts, whereas ZSP breaks down the classification task into context, event mode, and class disambiguation to refine task-specific hypotheses. This decomposition enhances interpretability, efficiency, and adaptability to schema changes. The experiments reveal ChatGPT's strengths and limitations, and crucially show ZSP's outperformance of dictionary-based methods and its competitive edge over some supervised models. These findings affirm the value of ZSP for validating event records and advancing ontology development. Our study underscores the efficacy of leveraging transfer learning and existing domain expertise to enhance research efficiency and scalability. The code is publicly available. © 2024 Association for Computational Linguistics.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2024

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proc. Annu. Meet. Assoc. Comput Linguist.

ISBN

979-889176094-3

ISSN

0736-587X

e-ISSN

—

Number of pages

21

Pages from-to

583-603

Publisher name

Association for Computational Linguistics (ACL)

Place of publication

—

Event location

Bangkok

Event date

Jan 1, 2025

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—