Analysis of workflow schedulers in simulated distributed environments

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F61989100%3A27740%2F22%3A10249720" target="_blank" >RIV/61989100:27740/22:10249720 - isvavai.cz</a>

Výsledek na webu

<a href="https://link.springer.com/content/pdf/10.1007/s11227-022-04438-y.pdf" target="_blank" >https://link.springer.com/content/pdf/10.1007/s11227-022-04438-y.pdf</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1007/s11227-022-04438-y" target="_blank" >10.1007/s11227-022-04438-y</a>

Jazyk výsledku

angličtina

Název v původním jazyce

Analysis of workflow schedulers in simulated distributed environments

Popis výsledku v původním jazyce

Task graphs provide a simple way to describe scientific workflows (sets of tasks with dependencies) that can be executed on both HPC clusters and in the cloud. An important aspect of executing such graphs is the used scheduling algorithm. Many scheduling heuristics have been proposed in existing works; nevertheless, they are often tested in oversimplified environments. We provide an extensible simulation environment designed for prototyping and benchmarking task schedulers, which contains implementations of various scheduling algorithms and is open-sourced, in order to be fully reproducible. We use this environment to perform a comprehensive analysis of workflow scheduling algorithms with a focus on quantifying the effect of scheduling challenges that have so far been mostly neglected, such as delays between scheduler invocations or partially unknown task durations. Our results indicate that network models used by many previous works might produce results that are off by an order of magnitude in comparison to a more realistic model. Additionally, we show that certain implementation details of scheduling algorithms which are often neglected can have a large effect on the scheduler&apos;s performance, and they should thus be described in great detail to enable proper evaluation.

Název v anglickém jazyce

Analysis of workflow schedulers in simulated distributed environments

Popis výsledku anglicky

Task graphs provide a simple way to describe scientific workflows (sets of tasks with dependencies) that can be executed on both HPC clusters and in the cloud. An important aspect of executing such graphs is the used scheduling algorithm. Many scheduling heuristics have been proposed in existing works; nevertheless, they are often tested in oversimplified environments. We provide an extensible simulation environment designed for prototyping and benchmarking task schedulers, which contains implementations of various scheduling algorithms and is open-sourced, in order to be fully reproducible. We use this environment to perform a comprehensive analysis of workflow scheduling algorithms with a focus on quantifying the effect of scheduling challenges that have so far been mostly neglected, such as delays between scheduler invocations or partially unknown task durations. Our results indicate that network models used by many previous works might produce results that are off by an order of magnitude in comparison to a more realistic model. Additionally, we show that certain implementation details of scheduling algorithms which are often neglected can have a large effect on the scheduler&apos;s performance, and they should thus be described in great detail to enable proper evaluation.

Druh

J_imp - Článek v periodiku v databázi Web of Science

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

—

Návaznosti

V - Vyzkumna aktivita podporovana z jinych verejnych zdroju

Rok uplatnění

2022

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název periodika

Journal of Supercomputing

ISSN

0920-8542

e-ISSN

1573-0484

Svazek periodika

78

Číslo periodika v rámci svazku

13

Stát vydavatele periodika

NL - Nizozemsko

Počet stran výsledku

27

Strana od-do

15154-15180

Kód UT WoS článku

000782538600001

EID výsledku v databázi Scopus

2-s2.0-85128355415