Difference between revisions of "GPU Access"

From UFRC
Jump to navigation Jump to search
Line 24: Line 24:
 
==== Hardware Specifications for the GPU Partition====
 
==== Hardware Specifications for the GPU Partition====
 
We have the following types of NVIDIA GPU nodes available in the "gpu" partition:
 
We have the following types of NVIDIA GPU nodes available in the "gpu" partition:
 
* NVIDIA GeForce GTX 1080Ti, with 2 GPUs per node. See [https://www.geforce.com/hardware/desktop-gpus/geforce-gtx-1080-ti/specifications technical specifications] for reference.
 
* NVIDIA GeForce RTX 2080Ti, with 8 GPUs per node. See [https://www.nvidia.com/en-us/geforce/graphics-cards/rtx-2080-ti technical specifications] for reference.
 
* NVIDIA Quadro RTX 6000, with 8 GPUs per node. These GPUs have SLI bridging See [https://www.nvidia.com/en-us/design-visualization/quadro/rtx-6000/ technical specifications] for reference.
 
* AI NVIDIA DGX A100 SuperPod, with 8 GPUs per node. These GPUs have [https://www.nvidia.com/en-us/data-center/nvlink/ NVSWITCH] interconnects See [https://www.nvidia.com/en-us/data-center/a100/ technical specifications] for reference.
 
 
 
{| style="margin-left: 5px; width:80%"
 
{| style="margin-left: 5px; width:80%"
 
|
 
|
 
{| class="wikitable" style="text-align: center"
 
{| class="wikitable" style="text-align: center"
!GPU!!Host Quantity!!Host Architecture!!Host Memory!!Host Interconnect!!CPUs per Host!!CPUS per Socket!!GPUs per Host!!CPUs per GPU!!Memory per GPU!!SLURM Feature!!GRES GPU type
+
!GPU Specs!!Host Quantity!!Host Architecture!!Host Memory!!Host Interconnect!!CPUs per Host!!CPUS per Socket!!GPUs per Host!!CPUs per GPU!!Memory per GPU!!SLURM Feature!!GRES GPU type!!Technical Ref
 
|-
 
|-
| style="width: 14%;"|GeForce 1080Ti||1||Intel Haswell||128 GB||FDR IB||28||14||2||14||11GB||n/a||geforce
+
| style="width: 14%;"|GeForce 1080Ti||1||Intel Haswell||128 GB||FDR IB||28||14||2||14||11GB||n/a||geforce||[https://www.geforce.com/hardware/desktop-gpus/geforce-gtx-1080-ti/specifications Specifications]
 
|-
 
|-
| style="width: 14%;"|GeForce 2080Ti||32||Intel Skylake||187 GB||EDR IB||32||16||8||4||11GB||2080ti||geforce
+
| style="width: 14%;"|GeForce 2080Ti||32||Intel Skylake||187 GB||EDR IB||32||16||8||4||11GB||2080ti||geforce||[https://www.nvidia.com/en-us/geforce/graphics-cards/rtx-2080-ti Specifications]
 
|-
 
|-
| style="width: 14%;"|GeForce 2080Ti||38||Intel Cascade Lake||187 GB||EDR IB||32||16||8||4||11GB||2080ti||geforce
+
| style="width: 14%;"|GeForce 2080Ti||38||Intel Cascade Lake||187 GB||EDR IB||32||16||8||4||11GB||2080ti||geforce||[https://www.nvidia.com/en-us/geforce/graphics-cards/rtx-2080-ti Specifications]
 
|-
 
|-
| style="width: 14%;"|Quadro RTX 6000||6||Intel Cascade Lake||187 GB||EDR IB||32||16||8||4||23GB||rtx6000||quadro
+
| style="width: 14%;"|Quadro RTX 6000 SLI||6||Intel Cascade Lake||187 GB||EDR IB||32||16||8||4||23GB||rtx6000||quadro||[https://www.nvidia.com/en-us/design-visualization/quadro/rtx-6000/ Specifications]
 
|-
 
|-
| style="width: 14%;"|NVIDIA A100 ||140||AMD EPYC ROME||2 TB||HDR IB||128||16||8||16||80GB||a100||a100
+
| style="width: 14%;"|NVIDIA A100 [https://www.nvidia.com/en-us/data-center/nvlink/ NVSWITCH]||140||AMD EPYC ROME||2 TB||HDR IB||128||16||8||16||80GB||a100||a100||[https://www.nvidia.com/en-us/data-center/a100/ Specifications]
 
|}
 
|}
 
|}
 
|}

Revision as of 21:32, 18 January 2023

Interactive Jobs in the GPU partition are limited to 12 hrs

Normalized Graphics Processor Units (NGUs) include all of the infrastructure (memory, network, rack space, cooling) necessary for GPU-accelerated computation. Each NGU is equivalent to 1 GPU presently, however newer GPUs such as the A100s may require more than 1 NGU to access in the future.

Researchers can add NGUs to their allocations by filling out the Purchase Form or requesting a Trial Allocation.

GPU-enabled Services

Types of GPUs are listed below. Two partitions contain GPUs - the hwgui partition for visualization and the gpu partition for general computation.

Hardware Accelerated GUI

GPUs in these servers are used to accelerate rendering for graphical applications. These servers are in the SLURM "hwgui" partition. Refer to the Hardware Accelerated GUI Sessions page for more information on available resources and usage.

Please note: The original HWGUI system based on xpra is being deprecated, please use our Open-On-Demand service as a replacement: UFRC Open OnDemand

GPU Assisted Computation

A number of high performance applications installed on HiPerGator implement GPU-accelerated computing functions via CUDA to achieve significant speed-up over CPU calculations. These servers are in the SLURM "gpu" partition (--partition=gpu).

Hardware Specifications for the GPU Partition

We have the following types of NVIDIA GPU nodes available in the "gpu" partition:

GPU Specs Host Quantity Host Architecture Host Memory Host Interconnect CPUs per Host CPUS per Socket GPUs per Host CPUs per GPU Memory per GPU SLURM Feature GRES GPU type Technical Ref
GeForce 1080Ti 1 Intel Haswell 128 GB FDR IB 28 14 2 14 11GB n/a geforce Specifications
GeForce 2080Ti 32 Intel Skylake 187 GB EDR IB 32 16 8 4 11GB 2080ti geforce Specifications
GeForce 2080Ti 38 Intel Cascade Lake 187 GB EDR IB 32 16 8 4 11GB 2080ti geforce Specifications
Quadro RTX 6000 SLI 6 Intel Cascade Lake 187 GB EDR IB 32 16 8 4 23GB rtx6000 quadro Specifications
NVIDIA A100 NVSWITCH 140 AMD EPYC ROME 2 TB HDR IB 128 16 8 16 80GB a100 a100 Specifications

For a list of additional node features, see the Available Node Features page.

To select a specific type of GPU within a partition please use either a SLURM constraint (e.g. --constraint=rtx6000) or a GRES with the needed GPU type (--gres or --gpu=a100:1). See more examples below.

Compiling CUDA Enabled Programs

The most direct way to develop a custom GPU accelerated algorithm is with the CUDA programming, please refer to the Nvidia CUDA Toolkit page. The current CUDA environment is cuda/11. However, C++ or Python packages numba and PyCuda are other ways to program GPU algorithms.

Slurm and GPU Use

View instructions for using GPUs and scheduling GPU jobs with SLURM at Slurm and GPU Use