Comparing RNN and Transformer Context Representations in the Czech Answer Selection Task

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216224%3A14330%2F22%3A00125094" target="_blank" >RIV/00216224:14330/22:00125094 - isvavai.cz</a>

Výsledek na webu

<a href="http://dx.doi.org/10.5220/0000155600003116" target="_blank" >http://dx.doi.org/10.5220/0000155600003116</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.5220/0000155600003116" target="_blank" >10.5220/0000155600003116</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Comparing RNN and Transformer Context Representations in the Czech Answer Selection Task

Popis výsledku v původním jazyce

Open domain question answering now inevitably builds upon advanced neural models processing large unstructured textual sources serving as a kind of underlying knowledge base. In case of non-mainstream highly- inflected languages, the state-of-the-art approaches lack large training datasets emphasizing the need for other improvement techniques. In this paper, we present detailed evaluation of a new technique employing various context representations in the answer selection task where the best answer sentence from a candidate document is identified as the most relevant to the human entered question. The input data here consists not only of each sentence in isolation but also of its preceding sentence(s) as the context. We compare seven different context representations including direct recurrent network (RNN) embeddings and several BERT-model based sentence embedding vectors. All experiments are evaluated with a new version 3.1 of the Czech question answering benchmark dataset SQAD wit h possible multiple correct answers as a new feature. The comparison shows that the BERT-based sentence embeddings are able to offer the best context representations reaching the mean average precision results of 83.39% which is a new best score for this dataset.

Název v anglickém jazyce

Comparing RNN and Transformer Context Representations in the Czech Answer Selection Task

Popis výsledku anglicky

Open domain question answering now inevitably builds upon advanced neural models processing large unstructured textual sources serving as a kind of underlying knowledge base. In case of non-mainstream highly- inflected languages, the state-of-the-art approaches lack large training datasets emphasizing the need for other improvement techniques. In this paper, we present detailed evaluation of a new technique employing various context representations in the answer selection task where the best answer sentence from a candidate document is identified as the most relevant to the human entered question. The input data here consists not only of each sentence in isolation but also of its preceding sentence(s) as the context. We compare seven different context representations including direct recurrent network (RNN) embeddings and several BERT-model based sentence embedding vectors. All experiments are evaluated with a new version 3.1 of the Czech question answering benchmark dataset SQAD wit h possible multiple correct answers as a new feature. The comparison shows that the BERT-based sentence embeddings are able to offer the best context representations reaching the mean average precision results of 83.39% which is a new best score for this dataset.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10200 - Computer and information sciences

Projekt

<a href="/cs/project/LM2018101" target="_blank" >LM2018101: Digitální výzkumná infrastruktura pro jazykové technologie, umění a humanitní vědy</a><br>

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)<br>S - Specificky vyzkum na vysokych skolach

Rok uplatnění

2022

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

Proceedings of the 14th International Conference on Agents and Artificial Intelligence (ICAART)

ISBN

9789897585470

ISSN

—

e-ISSN

—

Počet stran výsledku

7

Strana od-do

388-394

Název nakladatele

SCITEPRESS

Místo vydání

Portugal

Místo konání akce

Portugal

Datum konání akce

1. 1. 2022

Typ akce podle státní příslušnosti

CST - Celostátní akce

Kód UT WoS článku

—