Real-Life HPC Workload Trace Featuring Refined Job Runtime Estimates
Identifikátory výsledku
Kód výsledku v IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F63839172%3A_____%2F25%3A10133792" target="_blank" >RIV/63839172:_____/25:10133792 - isvavai.cz</a>
Výsledek na webu
<a href="https://doi.org/10.1007/978-3-031-74430-3_1" target="_blank" >https://doi.org/10.1007/978-3-031-74430-3_1</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/978-3-031-74430-3_1" target="_blank" >10.1007/978-3-031-74430-3_1</a>
Alternativní jazyky
Jazyk výsledku
angličtina
Název v původním jazyce
Real-Life HPC Workload Trace Featuring Refined Job Runtime Estimates
Popis výsledku v původním jazyce
User-provided job walltime (runtime) estimates are often used in current job schedulers to optimize performance and predictability when scheduling parallel jobs on computing resources. Since the walltime estimate is usually the upper runtime limit after which a job is forcibly terminated, user-provided estimates are usually overestimated, inaccurate, and rather coarse-grained. In this situation, the performance of crucial scheduling policies (e.g., the ability to backfill jobs) can be degraded. In this paper, we present a large real-life workload trace from the Czech National Grid Infrastructure MetaCentrum that contains automatically generated runtime estimates that improve original (user-provided) job walltime estimates. We also present our experience with using an automated system to improve user-provided estimates in an actual HPC system. Last but not least, we provide insights into the nature of the workload and also share several tools used to develop, debug, and analyze our predictor, scheduling policies, and the workload itself.
Název v anglickém jazyce
Real-Life HPC Workload Trace Featuring Refined Job Runtime Estimates
Popis výsledku anglicky
User-provided job walltime (runtime) estimates are often used in current job schedulers to optimize performance and predictability when scheduling parallel jobs on computing resources. Since the walltime estimate is usually the upper runtime limit after which a job is forcibly terminated, user-provided estimates are usually overestimated, inaccurate, and rather coarse-grained. In this situation, the performance of crucial scheduling policies (e.g., the ability to backfill jobs) can be degraded. In this paper, we present a large real-life workload trace from the Czech National Grid Infrastructure MetaCentrum that contains automatically generated runtime estimates that improve original (user-provided) job walltime estimates. We also present our experience with using an automated system to improve user-provided estimates in an actual HPC system. Last but not least, we provide insights into the nature of the workload and also share several tools used to develop, debug, and analyze our predictor, scheduling policies, and the workload itself.
Klasifikace
Druh
D - Stať ve sborníku
CEP obor
—
OECD FORD obor
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Návaznosti výsledku
Projekt
—
Návaznosti
—
Ostatní
Rok uplatnění
2025
Kód důvěrnosti údajů
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Údaje specifické pro druh výsledku
Název statě ve sborníku
JOB SCHEDULING STRATEGIES FOR PARALLEL PROCESSING, JSSPP 2024
ISBN
978-3-031-74429-7
ISSN
0302-9743
e-ISSN
1611-3349
Počet stran výsledku
19
Strana od-do
1-19
Název nakladatele
SPRINGER INTERNATIONAL PUBLISHING AG
Místo vydání
CHAM
Místo konání akce
San Francisco
Datum konání akce
31. 5. 2025
Typ akce podle státní příslušnosti
WRD - Celosvětová akce
Kód UT WoS článku
001532176000001