Real-Life HPC Workload Trace Featuring Refined Job Runtime Estimates
The result's identifiers
Result code in IS VaVaI
<a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F63839172%3A_____%2F25%3A10133792" target="_blank" >RIV/63839172:_____/25:10133792 - isvavai.cz</a>
Result on the web
<a href="https://doi.org/10.1007/978-3-031-74430-3_1" target="_blank" >https://doi.org/10.1007/978-3-031-74430-3_1</a>
DOI - Digital Object Identifier
<a href="http://dx.doi.org/10.1007/978-3-031-74430-3_1" target="_blank" >10.1007/978-3-031-74430-3_1</a>
Alternative languages
Result language
angličtina
Original language name
Real-Life HPC Workload Trace Featuring Refined Job Runtime Estimates
Original language description
User-provided job walltime (runtime) estimates are often used in current job schedulers to optimize performance and predictability when scheduling parallel jobs on computing resources. Since the walltime estimate is usually the upper runtime limit after which a job is forcibly terminated, user-provided estimates are usually overestimated, inaccurate, and rather coarse-grained. In this situation, the performance of crucial scheduling policies (e.g., the ability to backfill jobs) can be degraded. In this paper, we present a large real-life workload trace from the Czech National Grid Infrastructure MetaCentrum that contains automatically generated runtime estimates that improve original (user-provided) job walltime estimates. We also present our experience with using an automated system to improve user-provided estimates in an actual HPC system. Last but not least, we provide insights into the nature of the workload and also share several tools used to develop, debug, and analyze our predictor, scheduling policies, and the workload itself.
Czech name
—
Czech description
—
Classification
Type
D - Article in proceedings
CEP classification
—
OECD FORD branch
10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Result continuities
Project
—
Continuities
—
Others
Publication year
2025
Confidentiality
S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů
Data specific for result type
Article name in the collection
JOB SCHEDULING STRATEGIES FOR PARALLEL PROCESSING, JSSPP 2024
ISBN
978-3-031-74429-7
ISSN
0302-9743
e-ISSN
1611-3349
Number of pages
19
Pages from-to
1-19
Publisher name
SPRINGER INTERNATIONAL PUBLISHING AG
Place of publication
CHAM
Event location
San Francisco
Event date
May 31, 2025
Type of event by nationality
WRD - Celosvětová akce
UT code for WoS article
001532176000001