One Model is Not Enough: Ensembles for Isolated Sign Language Recognition

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F49777513%3A23520%2F22%3A43966108" target="_blank" >RIV/49777513:23520/22:43966108 - isvavai.cz</a>

Výsledek na webu

<a href="https://www.mdpi.com/1424-8220/22/13/5043/htm" target="_blank" >https://www.mdpi.com/1424-8220/22/13/5043/htm</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.3390/s22135043" target="_blank" >10.3390/s22135043</a>

Jazyk výsledku

angličtina

Název v původním jazyce

One Model is Not Enough: Ensembles for Isolated Sign Language Recognition

Popis výsledku v původním jazyce

In this paper, we dive into sign language recognition, focusing on the recognition of isolated signs. The task is defined as a classification problem, where a sequence of frames (i.e., images) is recognized as one of the given sign language glosses. We analyze two appearance-based approaches, I3D and TimeSformer, and one pose-based approach, SPOTER. The appearance-based approaches are trained on a few different data modalities, whereas the performance of SPOTER is evaluated on different types of preprocessing. All the methods are tested on two publicly available datasets: AUTSL and WLASL300. We experiment with ensemble techniques to achieve new state-of-the-art results of 73.84% accuracy on the WLASL300 dataset by using the CMA-ES optimization method to find the best ensemble weight parameters. Furthermore, we present an ensembling technique based on the Transformer model, which we call Neural Ensembler.

Název v anglickém jazyce

One Model is Not Enough: Ensembles for Isolated Sign Language Recognition

Popis výsledku anglicky

In this paper, we dive into sign language recognition, focusing on the recognition of isolated signs. The task is defined as a classification problem, where a sequence of frames (i.e., images) is recognized as one of the given sign language glosses. We analyze two appearance-based approaches, I3D and TimeSformer, and one pose-based approach, SPOTER. The appearance-based approaches are trained on a few different data modalities, whereas the performance of SPOTER is evaluated on different types of preprocessing. All the methods are tested on two publicly available datasets: AUTSL and WLASL300. We experiment with ensemble techniques to achieve new state-of-the-art results of 73.84% accuracy on the WLASL300 dataset by using the CMA-ES optimization method to find the best ensemble weight parameters. Furthermore, we present an ensembling technique based on the Transformer model, which we call Neural Ensembler.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

20205 - Automation and control systems

Projekt

Výsledek vznikl pri realizaci vícero projektů. Více informací v záložce Projekty.

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2022

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

SENSORS

ISSN

1424-8220

e-ISSN

1424-8220

Svazek periodika

22

Číslo periodika v rámci svazku

13

Stát vydavatele periodika

CH - Švýcarská konfederace

Počet stran výsledku

17

Strana od-do

nestrankovano

Kód UT WoS článku

000824167200001

EID výsledku v databázi Scopus

2-s2.0-85133217387