NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216305%3A26230%2F21%3APU142944" target="_blank" >RIV/00216305:26230/21:PU142944 - isvavai.cz</a>
Result on the web
<a href="http://proceedings.mlr.press/v133/min21a/min21a.pdf" target="_blank" >http://proceedings.mlr.press/v133/min21a/min21a.pdf</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
Original language description
We review the EfficientQA competition from NeurIPS 2020. The competition focused on open-domain question answering (QA), where systems take natural language questions as input and return natural language answers. The aim of the competition was to build systems that can predict correct answers while also satisfying strict on-disk memory budgets. These memory budgets were designed to encourage contestants to explore the trade-off between storing retrieval corpora or the parameters of learned models. In this report, we describe the motivation and organization of the competition, review the best submissions, and analyze system predictions to inform a discussion of evaluation for open-domain QA.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
R - Projekt Ramcoveho programu EK
Others
Publication year
2021
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the NeurIPS 2020 Competition and Demonstration Track
ISBN
—
ISSN
2640-3498
e-ISSN
—
Number of pages
25
Pages from-to
86-111
Publisher name
Proceedings of Machine Learning Research
Place of publication
online
Event location
online
Event date
Dec 6, 2020
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—