Text-in-Context: Token-Level Error Detection for Table-to-Text Generation
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F21%3A10440544" target="_blank" >RIV/00216208:11320/21:10440544 - isvavai.cz</a>
Result on the web
<a href="https://aclanthology.org/2021.inlg-1.25.pdf" target="_blank" >https://aclanthology.org/2021.inlg-1.25.pdf</a>
DOI - Digital Object Identifier
—
Alternative languages
Result language
angličtina
Original language name
Text-in-Context: Token-Level Error Detection for Table-to-Text Generation
Original language description
We present our Charles-UPF submission for the Shared Task on Evaluating Accuracy in Generated Texts at INLG 2021. Our system can detect the errors automatically using a combination of a rule-based natural language generation (NLG) system and pretrained language models (LMs). We first utilize a rule-based NLG system to generate sentences with facts that can be derived from the input. For each sentence we evaluate, we select a subset of facts which are relevant by measuring semantic similarity to the sentence in question. Finally, we finetune a pretrained language model on annotated data along with the relevant facts for fine-grained error detection. On the test set, we achieve 69% recall and 75% precision with a model trained on a mixture of human-annotated and synthetic data.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
S - Specificky vyzkum na vysokych skolach<br>I - Institucionalni podpora na dlouhodoby koncepcni rozvoj vyzkumne organizace
Others
Publication year
2021
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
Proceedings of the 14th International Conference on Natural Language Generation (INLG 2021)
ISBN
978-1-954085-51-0
ISSN
—
e-ISSN
—
Number of pages
7
Pages from-to
259-265
Publisher name
Association for Computational Linguistics
Place of publication
Stroudsburgh, PA, USA
Event location
Online
Event date
Sep 20, 2021
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
—