Annotating the Tweebank Corpus on Named Entity Recognition and Building NLP Models for Social Media Analysis

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216208%3A11320%2F22%3AC8AF63B3" target="_blank" >RIV/00216208:11320/22:C8AF63B3 - isvavai.cz</a>

Result on the web

<a href="https://aclanthology.org/2022.lrec-1.780" target="_blank" >https://aclanthology.org/2022.lrec-1.780</a>

DOI - Digital Object Identifier

—

Result language

angličtina

Original language name

Annotating the Tweebank Corpus on Named Entity Recognition and Building NLP Models for Social Media Analysis

Original language description

Social media data such as Twitter messages (“tweets”) pose a particular challenge to NLP systems because of their short, noisy, and colloquial nature. Tasks such as Named Entity Recognition (NER) and syntactic parsing require highly domain-matched training data for good performance. To date, there is no complete training corpus for both NER and syntactic analysis (e.g., part of speech tagging, dependency parsing) of tweets. While there are some publicly available annotated NLP datasets of tweets, they are only designed for individual tasks. In this study, we aim to create Tweebank-NER, an English NER corpus based on Tweebank V2 (TB2), train state-of-the-art (SOTA) Tweet NLP models on TB2, and release an NLP pipeline called Twitter-Stanza. We annotate named entities in TB2 using Amazon Mechanical Turk and measure the quality of our annotations. We train the Stanza pipeline on TB2 and compare with alternative NLP frameworks (e.g., FLAIR, spaCy) and transformer-based models. The Stanza tokenizer and lemmatizer achieve SOTA performance on TB2, while the Stanza NER tagger, part-of-speech (POS) tagger, and dependency parser achieve competitive performance against non-transformer models. The transformer-based models establish a strong baseline in Tweebank-NER and achieve the new SOTA performance in POS tagging and dependency parsing on TB2. We release the dataset and make both the Stanza pipeline and BERTweet-based models available “off-the-shelf” for use in future Tweet NLP research. Our source code, data, and pre-trained models are available at: https://github.com/social-machines/TweebankNLP.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

—

Continuities

—

Publication year

2022

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Proceedings of the Thirteenth Language Resources and Evaluation Conference

ISBN

979-10-95546-72-6

ISSN

—

e-ISSN

—

Number of pages

10

Pages from-to

7199-7208

Publisher name

European Language Resources Association

Place of publication

—

Event location

Marseille, France

Event date

Jan 1, 2022

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—