SLURM Job Arrays
Introduction and Submitting Arrays
To submit a number of identical jobs without having drive the submission with an external script use the SLURM's feature of array jobs. You can learn how to submit them at Submitting Array Jobs.
Note: There is a maximum limit of 3000 jobs per user on HiPerGator.
Using the array ID Index
SLURM will provide a $SLURM_ARRAY_TASK_ID variable to each task. It can be used inside the job script to handle input and output files for that task. To learn how and see some examples, visit Array ID Indexes.
Running many short tasks
While SLURM array jobs make it easy to run many similar tasks, if each task is short (seconds or even a few minutes), array jobs quickly bog down the scheduler and more time is spent managing jobs than actually doing any work for you. This also negatively impacts other users.
If you have hundreds or thousands of tasks, it is unlikely that a simple array job is the best solution. That does not mean that array jobs are not helpful in these cases, but that a little more thought needs to go into them for efficient use of the resources.
As an example let's imagine I have 5,000 runs of a program to do, with each run taking about 30 seconds to complete. Rather than running an array job with 5,000 tasks, it would be much more efficient to run 5 tasks where each completes 1,000 runs.
Deleting job arrays and tasks
To delete all of the tasks of an array job, use
scancel with the job ID:
To delete a single task, add the task ID:
Controlling Job emails
By default in SLURM, the emails for events BEGIN, END and FAIL apply to the job array as a whole rather than individual tasks. So:
would result in one email per job, not per task. If you want per task emails, specify:
which will send emails for each task in the array.