A Monte Carlo Language Model Pipeline for Zero-Shot Sociopolitical Event Extraction

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F23%3ADFR3MMIF" target="_blank" >RIV/00216208:11320/23:DFR3MMIF - isvavai.cz</a>

Result on the web

<a href="http://arxiv.org/abs/2305.15051" target="_blank" >http://arxiv.org/abs/2305.15051</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

A Monte Carlo Language Model Pipeline for Zero-Shot Sociopolitical Event Extraction

Original language description

"We consider dyadic zero-shot event extraction (EE) to identify actions between pairs of actors. The emph{zero-shot} setting allows social scientists or other non-computational researchers to extract any customized, user-specified set of events without training, resulting in a emph{dyadic} event database, allowing insight into sociopolitical relational dynamics among actors and the higher level organizations or countries they represent. Unfortunately, we find that current zero-shot EE methods perform poorly for the task, with issues including word sense ambiguity, modality mismatch, and efficiency. Straightforward application of large language model prompting typically performs even worse. We address these challenges with a new fine-grained, multi-stage generative question-answer method, using a Monte Carlo approach to exploit and overcome the randomness of generative outputs. It performs 90% fewer queries than a previous approach, with strong performance on the widely-used Automatic Content Extraction dataset. Finally, we extend our method to extract affiliations of actor arguments and demonstrate our method and findings on a dyadic international relations case study."

Czech name

—

Czech description

—

Type

O - Miscellaneous

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2023

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů