Rethinking the Objectives of Extractive Question Answering

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F21%3APU142981" target="_blank" >RIV/00216305:26230/21:PU142981 - isvavai.cz</a>

Result on the web

<a href="https://aclanthology.org/2021.mrqa-1.2/" target="_blank" >https://aclanthology.org/2021.mrqa-1.2/</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Rethinking the Objectives of Extractive Question Answering

Original language description

This work demonstrates that using the objective with independence assumption for modelling the span probability P (a_s , a_e ) = P (a_s )P (a_e) of span starting at position a_s and ending at position a_e has adverse effects. Therefore we propose multiple approaches to modelling joint probability P (a_s , a_e) directly. Among those, we propose a  compound objective, composed from the joint probability while still keeping the objective with independence assumption as an auxiliary objective. We find that the compound objective is consistently superior or equal to other assumptions in exact match. Additionally, we identified common errors caused by the assumption of independence and manually checked the counterpart predictions, demonstrating the impact of the compound objective on the real examples. Our findings are supported via experiments with three extractive QA models (BIDAF, BERT, ALBERT) over six datasets and our code, individual results and manual analysis are available online.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/LTC18054" target="_blank" >LTC18054: Distant Reading for European Literary History</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2021

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the 3rd Workshop on Machine Reading for Question Answering

ISBN

978-1-954085-95-4

ISSN

—

e-ISSN

—

Number of pages

14

Pages from-to

14-27

Publisher name

Association for Computational Linguistics

Place of publication

Punta Cana

Event location

Punta Cana

Event date

Nov 7, 2021

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—