Difference between revisions of "Temporary Directories"

From UFRC
Jump to navigation Jump to search
m (Reverted edits by Cabreraruizdiazj (talk) to last revision by Israel.herrera)
Tag: Rollback
 
(7 intermediate revisions by 4 users not shown)
Line 1: Line 1:
[[Category:Docs]]
+
[[Category:Documentation‏]]
When a SLURM job starts the scheduler creates a temporary directory for the job on the local hard drive on the compute node. This directory is owned by the user running the job. The path to the temporary directory is made available as the <code>$SLURM_TMPDIR</code> variable. The temporary job directory is automatically removed at the end of the job. The $SLURM_TMPDIR directory is very useful for jobs that need to use or generate a large number of small files since the '<code>/ufrc</code>' parallel filesystem is optimized for large file streaming and is less suitable for small files.
+
{{Note|'''Note: Do not use /tmp directory for any analyses |warn}}
  
Use the '<code>${SLURM_TMPDIR}</code>' variable in job scripts to copy temporary data to the temporary job directory or as an argument for an application that accepts a temporary directory argument if necessary.
+
When a SLURM job starts, the scheduler creates a temporary directory for the job on the compute node's local hard drive. This directory accessible via the <code>$SLURM_TMPDIR</code> or <code>$TMPDIR</code> variable as a filesystem path is very useful for jobs that need to use or generate a large number of small files, as the main RC parallel filesystem is optimized for large file streaming and is less suitable for small files.  
  
Note that many applications and programming languages use the '<code>$TMPDIR</code>' environment variable, if available, as the default temporary directory path. If this variable is not set the applications will default to using the '<code>/tmp</code>' directory, which is not desirable. SLURM will set TMPDIR to the same value as SLURM_TMPDIR unless TMPDIR has already been set in which case it will be ignored. Check your job script(s) and shell initialization files like .bashrc and .bash_profile to make sure you do not have TMPDIR set.
+
The directory is owned by the user running the job. The path to the temporary directory is made available as the <code>$SLURM_TMPDIR</code> variable. At the end of the job, the temporary directory is automatically removed unless the job fails at which time it's temporarily retained for troubleshooting.
  
If a personal singularity container is used make sure that the SINGULARITYENV_TMPDIR variable is set within the job to export the local scratch location into the singularity container.
+
You can use the <code>${SLURM_TMPDIR}</code> variable in job scripts to copy temporary data to the temporary job directory. If necessary, it can also be used as argument for applications that accept a temporary directory argument.
 +
 
 +
====Note - Default Paths====
 +
 
 +
Many applications and programming languages use the <code>$TMPDIR</code> environment variable, if available, as the default temporary directory path. If this variable is not set, the applications will default to using the <code>/tmp</code> directory, which is not desirable. SLURM will set <code>$TMPDIR</code> to the same value as <code>$SLURM_TMPDIR</code> unless <code>$TMPDIR</code> has already been set, in which case it will be ignored. Check your job script(s) and shell initialization files like .bashrc and .bash_profile to make sure you do not have <code>$TMPDIR</code> set.
 +
 
 +
If a personal Singularity container is used, make sure that the <code>$SINGULARITYENV_TMPDIR</code> variable is set within the job to export the local scratch location into the Singularity container.

Latest revision as of 15:01, 19 July 2023

Note: Do not use /tmp directory for any analyses

When a SLURM job starts, the scheduler creates a temporary directory for the job on the compute node's local hard drive. This directory accessible via the $SLURM_TMPDIR or $TMPDIR variable as a filesystem path is very useful for jobs that need to use or generate a large number of small files, as the main RC parallel filesystem is optimized for large file streaming and is less suitable for small files.

The directory is owned by the user running the job. The path to the temporary directory is made available as the $SLURM_TMPDIR variable. At the end of the job, the temporary directory is automatically removed unless the job fails at which time it's temporarily retained for troubleshooting.

You can use the ${SLURM_TMPDIR} variable in job scripts to copy temporary data to the temporary job directory. If necessary, it can also be used as argument for applications that accept a temporary directory argument.

Note - Default Paths

Many applications and programming languages use the $TMPDIR environment variable, if available, as the default temporary directory path. If this variable is not set, the applications will default to using the /tmp directory, which is not desirable. SLURM will set $TMPDIR to the same value as $SLURM_TMPDIR unless $TMPDIR has already been set, in which case it will be ignored. Check your job script(s) and shell initialization files like .bashrc and .bash_profile to make sure you do not have $TMPDIR set.

If a personal Singularity container is used, make sure that the $SINGULARITYENV_TMPDIR variable is set within the job to export the local scratch location into the Singularity container.