Difference between revisions of "Temporary Directories"
| Line 14: | Line 14: | ||
SLURM will set <code>$TMPDIR</code> to the same value as <code>$SLURM_TMPDIR</code> unless <code>$TMPDIR</code> has already been set, in which case it will be ignored. Check your job script(s) and shell initialization files like .bashrc and .bash_profile to make sure you do not have <code>$TMPDIR</code> set. | SLURM will set <code>$TMPDIR</code> to the same value as <code>$SLURM_TMPDIR</code> unless <code>$TMPDIR</code> has already been set, in which case it will be ignored. Check your job script(s) and shell initialization files like .bashrc and .bash_profile to make sure you do not have <code>$TMPDIR</code> set. | ||
| − | If a personal Singularity container is used, make sure that the <code>$SINGULARITYENV_TMPDIR</code> variable is set within the job to export the local scratch location into the Singularity container. | + | If a personal Singularity/Apptainer container is used, make sure that the <code>$SINGULARITYENV_TMPDIR</code> (or <code>APPTAINERENV_TMPDIR</code>) variable is set within the job to export the local scratch location into the Singularity container. |
Revision as of 14:01, 19 July 2023
When a SLURM job starts, the scheduler creates a temporary directory for the job on the compute node's local hard drive. This directory accessible via the $SLURM_TMPDIR or $TMPDIR variable as a filesystem path is very useful for jobs that need to use or generate a large number of small files, as the main RC parallel filesystem is optimized for large file streaming and is less suitable for small files.
The directory is owned by the user running the job. The path to the temporary directory is made available as the $SLURM_TMPDIR variable. At the end of the job, the temporary directory is automatically removed unless the job fails at which time it's temporarily retained for troubleshooting.
You can use the ${SLURM_TMPDIR} variable in job scripts to copy temporary data to the temporary job directory. If necessary, it can also be used as argument for applications that accept a temporary directory argument.
Note - Default Paths
Many applications and programming languages use the $TMPDIR environment variable, if available, as the default temporary directory path. If this variable is not set, the applications will default to using the /tmp directory, which is not desirable. Users should not use /tmp when they run analyses on HiPerGator as /tmp is configured to be small and can be easily filled up causing a node to fail.
SLURM will set $TMPDIR to the same value as $SLURM_TMPDIR unless $TMPDIR has already been set, in which case it will be ignored. Check your job script(s) and shell initialization files like .bashrc and .bash_profile to make sure you do not have $TMPDIR set.
If a personal Singularity/Apptainer container is used, make sure that the $SINGULARITYENV_TMPDIR (or APPTAINERENV_TMPDIR) variable is set within the job to export the local scratch location into the Singularity container.