Vše

Co hledáte?

Vše
Projekty
Výsledky výzkumu
Subjekty

Rychlé hledání

  • Projekty podpořené TA ČR
  • Významné projekty
  • Projekty s nejvyšší státní podporou
  • Aktuálně běžící projekty

Chytré vyhledávání

  • Takto najdu konkrétní +slovo
  • Takto z výsledků -slovo zcela vynechám
  • “Takto můžu najít celou frázi”

Real-life Experience with Major Reconfiguration of Job Scheduling System

Identifikátory výsledku

  • Kód výsledku v IS VaVaI

    <a href="https://www.isvavai.cz/riv?ss=detail&h=RIV%2F00216224%3A14330%2F17%3A00094513" target="_blank" >RIV/00216224:14330/17:00094513 - isvavai.cz</a>

  • Výsledek na webu

    <a href="http://www.cs.huji.ac.il/~feit/parsched/jsspp15/p5-klusacek.pdf" target="_blank" >http://www.cs.huji.ac.il/~feit/parsched/jsspp15/p5-klusacek.pdf</a>

  • DOI - Digital Object Identifier

    <a href="http://dx.doi.org/10.1007/978-3-319-61756-5_5" target="_blank" >10.1007/978-3-319-61756-5_5</a>

Alternativní jazyky

  • Jazyk výsledku

    angličtina

  • Název v původním jazyce

    Real-life Experience with Major Reconfiguration of Job Scheduling System

  • Popis výsledku v původním jazyce

    This work describes the goals and impacts of a large reconfiguration of the job scheduling system, used in the Czech National Grid and Cloud infrastructure MetaCentrum, which was implemented in early 2014. MetaCentrum serves a varied user-base consisting of both individual users and research groups. This imposes strict requirements on the robustness of job scheduling algorithms being employed, as the system must be capable of assigning a highly heterogeneous set of workloads to a similarly heterogeneous set of computational resources. Primary goals for MetaCentrum were always to provide efficient and fair resource utilization with respect to different users in the system. During the last few years, MetaCentrum has gone through a period of rapid growth (1,500 CPU cores in 2009 vs. 10,600 CPU cores in 2014) forcing us to re-evaluate our scheduling approaches, as the ``old" configuration no longer satisfied our utilization and fairness demands. This re-evaluation was supported by a significant body of research, which included the proposal of new scheduling approaches as well as detailed simulations based on real-life complex workload traces. First, a new multi-resource aware fair-sharing algorithm (based on our recent research) was deployed, with the goal of improving fairness with respect to the growing heterogeneity of resources and users' workloads. Second, the queue configuration of the entire system was completely reworked in order to decrease resource fragmentation and improve the utilization and the impact of fairness policies. This paper summarizes the effects of these changes using real data from the production system. Moreover, we publish complex workload from MetaCentrum that was used in this paper, since it represents a valuable source of data concerning a highly heterogeneous production system. Last but not least, we present our advanced job scheduling simulator which is routinely used for testing of new scheduling strategies prior their deployment in the real system.

  • Název v anglickém jazyce

    Real-life Experience with Major Reconfiguration of Job Scheduling System

  • Popis výsledku anglicky

    This work describes the goals and impacts of a large reconfiguration of the job scheduling system, used in the Czech National Grid and Cloud infrastructure MetaCentrum, which was implemented in early 2014. MetaCentrum serves a varied user-base consisting of both individual users and research groups. This imposes strict requirements on the robustness of job scheduling algorithms being employed, as the system must be capable of assigning a highly heterogeneous set of workloads to a similarly heterogeneous set of computational resources. Primary goals for MetaCentrum were always to provide efficient and fair resource utilization with respect to different users in the system. During the last few years, MetaCentrum has gone through a period of rapid growth (1,500 CPU cores in 2009 vs. 10,600 CPU cores in 2014) forcing us to re-evaluate our scheduling approaches, as the ``old" configuration no longer satisfied our utilization and fairness demands. This re-evaluation was supported by a significant body of research, which included the proposal of new scheduling approaches as well as detailed simulations based on real-life complex workload traces. First, a new multi-resource aware fair-sharing algorithm (based on our recent research) was deployed, with the goal of improving fairness with respect to the growing heterogeneity of resources and users' workloads. Second, the queue configuration of the entire system was completely reworked in order to decrease resource fragmentation and improve the utilization and the impact of fairness policies. This paper summarizes the effects of these changes using real data from the production system. Moreover, we publish complex workload from MetaCentrum that was used in this paper, since it represents a valuable source of data concerning a highly heterogeneous production system. Last but not least, we present our advanced job scheduling simulator which is routinely used for testing of new scheduling strategies prior their deployment in the real system.

Klasifikace

  • Druh

    D - Stať ve sborníku

  • CEP obor

  • OECD FORD obor

    10201 - Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)

Návaznosti výsledku

  • Projekt

    <a href="/cs/project/GAP202%2F12%2F0306" target="_blank" >GAP202/12/0306: Dyschnet - Dynamické plánování a rozvrhování výpočetních a síťových zdrojů</a><br>

  • Návaznosti

    P - Projekt vyzkumu a vyvoje financovany z verejnych zdroju (s odkazem do CEP)

Ostatní

  • Rok uplatnění

    2017

  • Kód důvěrnosti údajů

    S - Úplné a pravdivé údaje o projektu nepodléhají ochraně podle zvláštních právních předpisů

Údaje specifické pro druh výsledku

  • Název statě ve sborníku

    20th International Workshop on Job Scheduling Strategies for Parallel Processing, JSSPP 2016

  • ISBN

    9783319617558

  • ISSN

    0302-9743

  • e-ISSN

  • Počet stran výsledku

    19

  • Strana od-do

    83-101

  • Název nakladatele

    Springer

  • Místo vydání

    Cham, Switzerland

  • Místo konání akce

    Hyderabad, India

  • Datum konání akce

    29. 5. 2015

  • Typ akce podle státní příslušnosti

    WRD - Celosvětová akce

  • Kód UT WoS článku

    000441213000005