Quick Start Guide

Revision as of 17:48, 5 August 2021 by Magitz (talk | contribs) (→‎HiPerGator Etiquette)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

HiPerGator Etiquette

  • Only run workloads on compute nodes. Do not run scripts or applications on the login nodes beyond a small quick test. Use sbatch, srundev, salloc, or srun to start a session on a compute node instead.

  • Only run workloads from blue storage. This is a fast storage systems that can handle the I/O involved in research workloads. Before using sbatch or launching a workload interactively, make sure your working directory is a blue file path, e.g. /blue/<group>/<user>, and not your home directory (~ or /home/<user>). Use pwd to print working directory.

  • Home directory is only for storing user readable files. Your 40GB home directory is the only storage on HiPerGator for which file recovery may be possible, so keep copies of scripts, configurations, or other important files here. Versions of files may be available for the previous 7 days (see information on home directory snapshots and recovering files), if you need to recover a file from your home directory. If you need back ups of important data, backup services will need to be purchased.

  • Do not run workloads from orange storage. Orange is intended as long-term, archival storage of data you currently do not use. It cannot handle the high-throughput requirements of high-performance computing workloads.

  • Do not request excessive resources. This includes CPU, GPU and memory. Job emails include summary estimations of memory use, however, active monitoring will help you understand resource requirements. Applications often require special commands, arguments, or configurations to run in parallel. Therefore, you will likely need to do more than request multiple CPUs or GPUs for a workload to put those resources to use.

Logging in

Via command line interface: On macOS and Linux use the terminal. On Windows use an SSH client such as Putty or Tabby. You may omit GatorLinkUsername if the username on your local computer matches your GatorLink username e.g.

$ ssh <GatorLink Username>@hpg.rc.ufl.edu
$ ssh hpg.rc.ufl.edu

After logging in, you are connected to a login node and your working directory will be /home/<GatorLink>.

Via web browser:

  • Services like OpenOnDemand (OOD) and JupyterHub require the users to be on the UF network. If the user is off the campus network they need to be connected to the UF VPN.
  • To start a GUI application go to https://ood.rc.ufl.edu and choose the application you would like to use, for example HiPerGator Desktop is a linux GUI.
    • To start a Jupyter Notebook with OOD, select the “Jupyter Notebook” application server under the “Interactive Applications” tab on the main dashboard. Specify the resources you need in the resource request and click “Launch” at the bottom of the form. Once your job starts, click “Connect to Jupyter” to start your Notebook.
  • To start a Jupyter Notebook go to http://jhub.rc.ufl.edu and select the resources you need. This JupyterHub server has preset resource options.
    • Jupyterhub and OutOfMemory (OOM) – if your Notebook stops responding it might be an indication of the job running out of memory. Currently this does not produce an error message. You can check your /home/<user>/jupyter slurm spawner logs to be sure. You will need to restart the job with additional RAM.
  • To start a Galaxy instance go to https://galaxy.rc.ufl.edu/. Galaxy is a web-based framework for accessible, reproducible, and transparent biological computing.

File Transfers

  • For small or medium file transfers use sftp, scp, or rsync to login to sftp.rc.ufl.edu, hpg.rc.ufl.edu, or rsync.rc.ufl.edu.
  • For large file transfers or transfers with many small files, use our Globus service.

For more in-depth information see Transfer Data.

Using Installed Software

Numerous scientific applications are installed and available for use on HiPerGator. To load an application use the module load command e.g.

$ module load [module_name]

For more in-depth information see Modules Basic Usage

In Jupyter Notebooks, kernels are available with our most popular software stacks. If you are unable to find what you need or would like software installed, please fill out a help request.

Monitoring Your Workloads

You can see presently running workloads with the squeue command e.g.

$ squeuemine

OpenOnDemand offers a method to monitor jobs using the Jobs menu in the upper toolbar on your dashboard. This will show your current running, pending, and recently completed jobs. Select: Jobs -> Active Jobs from the upper dashboard menu.

We provide a number of helpful commands in the UFRC module. The ufrc module is loaded by default at login, but you can also load the ufrc module with the following command:

$ module load ufrc

Examples of commands for SLURM or HiPerGator specific UFRC environment module

$ slurmInfo           - displays resource usage for your group
$ showQos             - displays your available QoS
$ home_quota          - displays your /home quota
$ blue_quota          - displays your /blue quota
$ orange_quota        - displays your /orange quota
$ sacct               - displays job id and state of your recent workloads
$ nodeInfo            - displays partitions by node types, showing total RAM and other features
$ sinfo -p partition  - displays the status of nodes in a partition
$ jobhtop             - displays resource usage for jobs
$ jobnvtop            - displays resource usage for GPU jobs
$ which python        - displays path to the Python install of the environment modules you have loaded