A multi-criteria job scheduling framework for large computing farms

摘要

In this paper, we propose a new multi-criteria job scheduler for scheduling a continuous stream of batch jobs on large-scale computing farms. Our solution, called Convergent Scheduler, exploits a set of heuristics that drives the scheduler in taking decisions. Each heuristics manages a specific problem constraint, and contributes to compute a value that measures the degree of matching between a job and a machine. Scheduling choices are taken both to meet the Quality of Service requested by the submitted jobs and to optimize the usage of software and hardware resources. In order to validate the scheduler we propose, it has been compared versus two common job scheduling algorithms: Easy and Flexible backfilling. Convergent Scheduler demonstrated to be able to compute good assignments that allow a better exploitation of resources with respect to the other algorithms. Moreover, it has a simple modular structure that makes simple its extension and customization to meet the service goal of an installation.