Difference between revisions of "PyTorch"

Latest revision as of 16:39, 14 December 2022

Description

PyTorch is a machine learning library with strong support for neural networks and deep learning. PyTorch also has a large user base and software ecosystem.

Environment Modules

To use PyTorch on HiPerGator, you first need to load one of the PyTorch environment modules. Run module spider pytorch to find out what environment modules are available for PyTorch.

Once you determine which module you want to use, load it as follows (using pytorch/1.7.1 as an example):

module load pytorch/1.7.1

Without a version number, the latest installed version will load. ngc-pytorch is the pytorch software environment from the Nvidia GPU Cloud (NGC) with Apex optimizers.

Additional Information

To use PyTorch with a GPU or GPUs on HiPerGator, you must request the --gpus or --gpus-per-task resource and specify the gpu partition in your job script or on the command line as described in the GPU Access help page. For example, to start an interactive session with access to a single GPU, you might run the following command.

 srun --partition=gpu --gpus=1 --ntasks=1 --mem=4gb --time=08:00:00 --pty bash -i

Job Script Examples

To help you get started, here is an example SLURM script for running a PyTorch application on a single GPU on HiPerGator. If you are new to writing SLURM scripts and scheduling SLURM jobs, you will want to first read our help information on using SLURM and writing SLURM scripts. For information about using GPUs on HiPerGator, please see GPU Access.

Note that lines beginning with #SBATCH are instructions to the SLURM scheduler. Lines beginning with # are comments to help you understand the script; feel free to delete them if you adapt this script for your own use.

Expand to view example script.

#!/bin/sh
# The job name: you can choose whatever you want for this.
#SBATCH --job-name=my_pytorch_job

# Your email address and the events for which you want to receive email
# notification (NONE, BEGIN, END, FAIL, ALL).
#SBATCH --mail-user=username@ufl.edu
#SBATCH --mail-type=ALL

# The compute configuration for the job. For a job that uses GPUs, the
# partition must be set to "gpu". This example script requests access
# to a single GPU, 16 CPUs, and 30 GB of RAM for a single PyTorch task.
#SBATCH --nodes=1
#SBATCH --partition=gpu
#SBATCH --ntasks=1
#SBATCH --gpus-per-task=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=30gb

# Specifies how long the job will be allowed to run in HH:MM:SS.
#SBATCH --time=05:05:05

# The log file for all job output. Note the special string "%j", which
# represents the job number.
#SBATCH --output=job_output_%j.out

# Prints the working directory, name of the assigned node, and
# date/time at the top of the output log.
pwd; hostname; date

module load pytorch/1.7.1

# This should be the command you would use if you were running your PyTorch application from the terminal.
python my_pytorch_script.py

date

@@ Line 2: / Line 2: @@
 {|<!--CONFIGURATION: REQUIRED-->
 |{{#vardefine:app|pytorch}}
-|{{#vardefine:url|https://github.com/pytorch/pytorch}}
+|{{#vardefine:url|https://pytorch.org/}}
 <!--CONFIGURATION: OPTIONAL (|1}} means it's ON)-->
 |{{#vardefine:conf|}}           <!--CONFIGURATION-->
-|{{#vardefine:exe|}}            <!--ADDITIONAL INFO-->
+|{{#vardefine:exe|1}}            <!--ADDITIONAL INFO-->
-|{{#vardefine:job|}}            <!--JOB SCRIPTS-->
+|{{#vardefine:job|1}}            <!--JOB SCRIPTS-->
 |{{#vardefine:policy|}}         <!--POLICY-->
 |{{#vardefine:testing|}}       <!--PROFILING-->
@@ Line 22: / Line 22: @@
 <!--Modules-->
 ==Environment Modules==
-Run <code>module spider {{#var:app}}</code> to find out what environment modules are available for this application.
+To use PyTorch on HiPerGator, you first need to load one of the PyTorch [[Modules_Basic_Usage|environment modules]].
-==System Variables==
+Run <code>module spider {{#var:app}}</code> to find out what environment modules are available for PyTorch.
-* HPC_{{uc:{{#var:app}}}}_DIR - installation directory
-* HPC_{{uc:{{#var:app}}}}_BIN - executable directory
+Once you determine which module you want to use, load it as follows (using <code>pytorch/1.7.1</code> as an example):
+ <nowiki>
+module load pytorch/1.7.1</nowiki>
+Without a version number, the latest installed version will load. ngc-pytorch is the pytorch software environment from the Nvidia GPU Cloud (NGC) with Apex optimizers.
 <!--Configuration-->
@@ Line 33: / Line 37: @@
 <!--Run-->
 {{#if: {{#var: exe}}|==Additional Information==
+To use PyTorch with a GPU or GPUs on HiPerGator, you must request the <code>--gpus</code> or <code>--gpus-per-task</code> resource and specify the <code>gpu</code> partition in your job script or on the command line as described in the [[GPU_Access|GPU Access]] help page. For example, to start an interactive session with access to a single GPU, you might run the following command.
-WRITE_ADDITIONAL_INSTRUCTIONS_ON_RUNNING_THE_SOFTWARE_IF_NECESSARY
+ <nowiki>
+ srun --partition=gpu --gpus=1 --ntasks=1 --mem=4gb --time=08:00:00 --pty bash -i</nowiki>
 |}}
 <!--Job Scripts-->
 {{#if: {{#var: job}}|==Job Script Examples==
-See the [[{{PAGENAME}}_Job_Scripts]] page for {{#var: app}} Job script examples.
+To help you get started, here is an example SLURM script for running a PyTorch application on a single GPU on HiPerGator.  If you are new to writing SLURM scripts and scheduling SLURM jobs, you will want to first read our help information on [[Getting_Started#Scheduling_jobs_using_SLURM|using SLURM and writing SLURM scripts]]. For information about using GPUs on HiPerGator, please see [[GPU Access]].
+Note that lines beginning with <code>#SBATCH</code> are instructions to the SLURM scheduler.  Lines beginning with <code># </code> are comments to help you understand the script; feel free to delete them if you adapt this script for your own use.
+<div class="mw-collapsible mw-collapsed" style="width:70%; padding: 5px; border: 1px solid gray;">
+''Expand to view example script.''
+<div class="mw-collapsible-content" style="padding: 5px;">
+ <nowiki>
+#!/bin/sh
+# The job name: you can choose whatever you want for this.
+#SBATCH --job-name=my_pytorch_job
+# Your email address and the events for which you want to receive email
+# notification (NONE, BEGIN, END, FAIL, ALL).
+#SBATCH --mail-user=username@ufl.edu
+#SBATCH --mail-type=ALL
+# The compute configuration for the job. For a job that uses GPUs, the
+# partition must be set to "gpu". This example script requests access
+# to a single GPU, 16 CPUs, and 30 GB of RAM for a single PyTorch task.
+#SBATCH --nodes=1
+#SBATCH --partition=gpu
+#SBATCH --ntasks=1
+#SBATCH --gpus-per-task=1
+#SBATCH --cpus-per-task=16
+#SBATCH --mem=30gb
+# Specifies how long the job will be allowed to run in HH:MM:SS.
+#SBATCH --time=05:05:05
+# The log file for all job output. Note the special string "%j", which
+# represents the job number.
+#SBATCH --output=job_output_%j.out
+# Prints the working directory, name of the assigned node, and
+# date/time at the top of the output log.
+pwd; hostname; date
+module load pytorch/1.7.1
+# This should be the command you would use if you were running your PyTorch application from the terminal.
+python my_pytorch_script.py
+date</nowiki>
+</div>
+</div>
 |}}
 <!--Policy-->
 {{#if: {{#var: policy}}|==Usage Policy==