Batch Active Learning for Text Classification and Sentiment Analysis

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F68407700%3A21230%2F22%3A00360609" target="_blank" >RIV/68407700:21230/22:00360609 - isvavai.cz</a>

Result on the web

<a href="https://doi.org/10.1145/3562007.3562028" target="_blank" >https://doi.org/10.1145/3562007.3562028</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1145/3562007.3562028" target="_blank" >10.1145/3562007.3562028</a>

Result language

angličtina

Original language name

Batch Active Learning for Text Classification and Sentiment Analysis

Original language description

Supervised learning of classifiers for text classification and sentiment analysis relies on the availability of labels that may be either difficult or expensive to obtain. A standard procedure is to add labels to the training dataset sequentially by querying an annotator until the model reaches a satisfactory performance. Active learning is a process that optimizes unlabeled data records selection for which the knowledge of the label would bring the highest discriminability of the dataset. Batch active learning is a generalization of a single instance active learning by selecting a batch of documents for labeling. This task is much more demanding because plenty of different factors come into consideration (i. e. batch size, batch evaluation, etc.). In this paper, we provide a large scale study by decomposing the existing algorithms into building blocks and systematically comparing meaningful combinations of these blocks with a subsequent evaluation on different text datasets. While each block is known (warm start weights initialization, Dropout MC, entropy sampling, etc.), many of their combinations like Bayesian strategies with agglomerative clustering are first proposed in our paper with excellent performance. Particularly, our extension of the warm start method to batch active learning is among the top performing strategies on all datasets. We studied the effect of this proposal comparing the outcomes of varying distinct factors of an active learning algorithm. Some of these factors include initialization of the algorithm, uncertainty representation, acquisition function, and batch selection strategy. Further, various combinations of these are tested on selected NLP problems with documents encoded using RoBERTa embeddings. Datasets cover context integrity (Gibberish Wackerow), fake news detection (Kaggle Fake News Detection), categorization of short texts by emotional context (Twitter Sentiment140), and sentiment classification (Amazon Reviews). Ultimately, we show that each of the active learning factors has advantages for certain datasets or experimental settings.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/TL05000057" target="_blank" >TL05000057: The Signal and the Noise in the Era of Journalism 5.0 - A Comparative Perspective of Journalistic Genres of Automated Content</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2022

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

CCRIS '22: Proceedings of the 2022 3rd International Conference on Control, Robotics and Intelligent System

ISBN

978-1-4503-9685-1

ISSN

—

e-ISSN

—

Number of pages

6

Pages from-to

111-116

Publisher name

Association for Computing Machinery

Place of publication

New York

Event location

Virtual Event

Event date

Aug 26, 2022

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—