Creating large size of data with apache hadoop

Result code in IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F61989100%3A27240%2F17%3A10236715" target="_blank" >RIV/61989100:27240/17:10236715 - isvavai.cz</a>

Alternative codes found

RIV/61989100:27350/17:10236715 RIV/61989100:27740/17:10236715

Result on the web

<a href="http://dx.doi.org/10.1007/978-3-319-45123-7_22" target="_blank" >http://dx.doi.org/10.1007/978-3-319-45123-7_22</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/978-3-319-45123-7_22" target="_blank" >10.1007/978-3-319-45123-7_22</a>

Result language

angličtina

Original language name

Creating large size of data with apache hadoop

Original language description

The paper is focused on research in the area of building large datasets using Apache Hadoop. Our team is managing an information system that is able to calculate probability of existence of different objects in space and time. The system works with a lot of different data sources, including large datasets. The workflow of data processing is quite complicated and time consuming, so we were looking for some framework that could help with system management and, if possible, to speed up data processing as well. Apache Hadoop was selected as a platform for enhance our information system. Apache Hadoop is usually used for processing large datasets, but in a case of our information scystem is necessary to perform other types of tasks as well. The systems computes spatio-temporal relations between different types of objects. This means that from relatively small amount of records (thousands) are built relatively large datasets (millions of records). For this purposes is usually used PostgreSQL/PostGIS database or tools written in Java or other language. Our research was focused to determination if we could simply move some of this tasks to Apache Hadoop platform using simple SQL editor like Hive. We have selected two types of common tasks and tested them on PostgreSQL and Apache Hadoop (Hive) platform to be able compare time necessary to complete these tasks. The paper presents results of our research. © Springer International Publishing AG 2017.

Czech name

—

Czech description

—

Type

D - Article in proceedings

CEP classification

—

OECD FORD branch

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Project

<a href="/en/project/TB0500MD011" target="_blank" >TB0500MD011: Specific method of check-in and number of passengers</a><br>

Continuities

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Publication year

2017

Confidentiality

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Article name in the collection

Lecture Notes in Geoinformation and Cartography. Volume F3

ISBN

978-3-319-45122-0

ISSN

1863-2246

e-ISSN

1863-2351

Number of pages

8

Pages from-to

307-314

Publisher name

Springer

Place of publication

Cham

Event location

Ostrava

Event date

Mar 16, 2016

Type of event by nationality

WRD - Celosvětová akce

UT code for WoS article

—