To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F22%3A9MDIFVDQ" target="_blank" >RIV/00216208:11320/22:9MDIFVDQ - isvavai.cz</a>

Result on the web

<a href="https://doi.org/10.1162/coli_a_00425" target="_blank" >https://doi.org/10.1162/coli_a_00425</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1162/coli_a_00425" target="_blank" >10.1162/coli_a_00425</a>

Result language

angličtina

Original language name

To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP

Original language description

Data-hungry deep neural networks have established themselves as the de facto standard for many NLP tasks, including the traditional sequence tagging ones. Despite their state-of-the-art performance on high-resource languages, they still fall behind their statistical counterparts in low-resource scenarios. One methodology to counterattack this problem is text augmentation, that is, generating new synthetic training data points from existing data. Although NLP has recently witnessed several new textual augmentation techniques, the field still lacks a systematic performance analysis on a diverse set of languages and sequence tagging tasks. To fill this gap, we investigate three categories of text augmentation methodologies that perform changes on the syntax (e.g., cropping sub-sentences), token (e.g., random word insertion), and character (e.g., character swapping) levels. We systematically compare the methods on part-of-speech tagging, dependency parsing, and semantic role labeling for a diverse set of language families using various models, including the architectures that rely on pretrained multilingual contextualized language models such as mBERT. Augmentation most significantly improves dependency parsing, followed by part-of-speech tagging and semantic role labeling. We find the experimented techniques to be effective on morphologically rich languages in general rather than analytic languages such as Vietnamese. Our results suggest that the augmentation techniques can further improve over strong baselines based on mBERT, especially for dependency parsing. We identify the character-level methods as the most consistent performers, while synonym replacement and syntactic augmenters provide inconsistent improvements. Finally, we discuss that the results most heavily depend on the task, language pair (e.g., syntactic-level techniques mostly benefit higher-level tasks and morphologically richer languages), and model type (e.g., token-level augmentation provides significant improvements for BPE, while character-level ones give generally higher scores for char and mBERT based models).

Czech name

—

Czech description

—

Type

J_SC - Article in a specialist periodical, which is included in the SCOPUS database

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2022

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical

Computational Linguistics

ISSN

0891-2017

e-ISSN

1530-9312

Volume of the periodical

48

Issue of the periodical within the volume

1

Country of publishing house

US - UNITED STATES

Number of pages

38

Pages from-to

5-42

UT code for WoS article

—

EID of the result in the Scopus database

2-s2.0-85128214382