On the Quality of Wall Time Estimates for Resource Allocation Prediction

Kód výsledku v IS VaVaI

<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F63839172%3A_____%2F19%3A10133225" target="_blank" >RIV/63839172:_____/19:10133225 - isvavai.cz</a>

Výsledek na webu

<a href="https://dl.acm.org/doi/abs/10.1145/3339186.3339204" target="_blank" >https://dl.acm.org/doi/abs/10.1145/3339186.3339204</a>

DOI - Digital Object Identifier

<a href="http://dx.doi.org/10.1145/3339186.3339204" target="_blank" >10.1145/3339186.3339204</a>

Jazyk výsledku

angličtina

Název v původním jazyce

On the Quality of Wall Time Estimates for Resource Allocation Prediction

Popis výsledku v původním jazyce

Today&apos;s HPC systems experience steadily increasing problems with the storage I/O bottleneck. At the same time, new storage technologies are emerging in the compute nodes of HPC systems. There are many ideas and approaches how compute-node local storage can be made usable for HPC systems. One consideration is to copy job data to the compute-node local disks in advance. To accomplish this, the allocated nodes must be known in advance. In this paper, we look at the node allocation behavior of a HPC batch scheduling system. Our goal is to determine whether it is possible to stage data in advance, based on scheduler predictions. We show that wall time estimates must be excellent to reliably predict node allocations. In reality, the required accuracy enabling advance data staging is hard to achieve. Therefore, the behavior of (standard) batch scheduler have to be modified in order to enable efficient advance data staging.

Název v anglickém jazyce

On the Quality of Wall Time Estimates for Resource Allocation Prediction

Popis výsledku anglicky

Today&apos;s HPC systems experience steadily increasing problems with the storage I/O bottleneck. At the same time, new storage technologies are emerging in the compute nodes of HPC systems. There are many ideas and approaches how compute-node local storage can be made usable for HPC systems. One consideration is to copy job data to the compute-node local disks in advance. To accomplish this, the allocated nodes must be known in advance. In this paper, we look at the node allocation behavior of a HPC batch scheduling system. Our goal is to determine whether it is possible to stage data in advance, based on scheduler predictions. We show that wall time estimates must be excellent to reliably predict node allocations. In reality, the required accuracy enabling advance data staging is hard to achieve. Therefore, the behavior of (standard) batch scheduler have to be modified in order to enable efficient advance data staging.

Druh

D - Stať ve sborníku

CEP obor

—

OECD FORD obor

10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Projekt

Výsledek vznikl pri realizaci vícero projektů. Více informací v záložce Projekty.

Návaznosti

P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Rok uplatnění

2019

Kód důvěrnosti údajů

S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Název statě ve sborníku

ICPP 2019: Proceedings of the 48th International Conference on Parallel Processing: Workshops

ISBN

978-1-4503-7196-4

ISSN

—

e-ISSN

—

Počet stran výsledku

8

Strana od-do

1-8

Název nakladatele

Association for Computing Machinery

Místo vydání

Neuveden

Místo konání akce

Kyoto, Japan

Datum konání akce

5. 8. 2019

Typ akce podle státní příslušnosti

WRD - Celosvětová akce

Kód UT WoS článku

—