A comprehensive social media data processing and analytics architecture by using big data platforms: a case study of twitter flood-risk messages

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F61989100%3A27740%2F21%3A10247634" target="_blank" >RIV/61989100:27740/21:10247634 - isvavai.cz</a>

Result on the web

<a href="https://link.springer.com/article/10.1007/s12145-021-00601-w" target="_blank" >https://link.springer.com/article/10.1007/s12145-021-00601-w</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/s12145-021-00601-w" target="_blank" >10.1007/s12145-021-00601-w</a>

Result language

angličtina

Original language name

A comprehensive social media data processing and analytics architecture by using big data platforms: a case study of twitter flood-risk messages

Original language description

The main objective of the article is to propose an advanced architecture and workflow based on Apache Hadoop and Apache Spark big data platforms. The primary purpose of the presented architecture is collecting, storing, processing, and analysing intensive data from social media streams. This paper presents how the proposed architecture and data workflow can be applied to analyse Tweets with a specific flood topic. The secondary objective, trying to describe the flood alert situation by using only Tweet messages and exploring the informative potential of such data is demonstrated as well. The predictive machine learning approach based on Bayes Theorem was utilized to classify flood and no flood messages. For this study, approximately 100,000 Twitter messages were processed and analysed. Messages were related to the flooding domain and collected over a period of 5 days (14 May - 18 May 2018). Spark application was developed to run data processing commands automatically and to generate the appropriate output data. Results confirmed the advantages of many well-known features of Spark and Hadoop in social media data processing. It was noted that such technologies are prepared to deal with social media data streams, but there are still challenges that one has to take into account. Based on the flood tweet analysis, it was observed that Twitter messages with some considerations are informative enough to be used to estimate general flood alert situations in particular regions. Text analysis techniques proved that Twitter messages contain valuable flood-spatial information. (C) 2021, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.

Czech name

—

Czech description

—

Type

J_imp - Article in a specialist periodical, which is included in the Web of Science database

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/LQ1602" target="_blank" >LQ1602: IT4Innovations excellence in science</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2021

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Name of the periodical

Earth Science Informatics

ISSN

1865-0473

e-ISSN

—

Volume of the periodical

14

Issue of the periodical within the volume

2

Country of publishing house

DE - GERMANY

Number of pages

17

Pages from-to

913-929

UT code for WoS article

000627653100001

EID of the result in the Scopus database

2-s2.0-85102495032