Singularity

We already use and welcome your use of Singularity Containers on HiPerGator. As the Singularity website says you can make and customize your containers locally, copy them to your or your group's shared /ufrc space on HiPerGator and execute programs installed within whether in a dev session or in batch jobs. In essence it would allow you to develop your analytical workflow on your local machine and bring it as is - Bring Your Own Environment to the cluster.

While Singularity Documentation is the best place to find up-to-date and detailed information about using Singularity there some HiPerGator specific details that we'd like to point out to reduce the number of warnings you see and potential pitfalls you might encounter.

Image Builds

When building a new container image create the following directories that our Singularity install tries to automatically mount to avoid warnings or running into unavailability of those filesystems in the course of your normal work.

mkdir -p /ufrc /bio /rlts /scratch/local

Image Location

The /ufrc Blue Storage filesystem is the best suited for container image storage since it highly performant when reading large files such as container images in parallel. You are unlikely to be able to fit container image into your /home quote, but we'd like to emphasize that you should not do it even if you can to avoid performance issues and impacting other users.

Hub image downloads

When you run a container from an image stored in the Singularity Hub the container image will be downloaded to the current working directory. Make sure you run the command while on /ufrc filesystem and not in your home directory. E.g.

$ module load singularity
$ singularity run shub://GodloveD/lolcow

...

$ ls *.simg

GodloveD-lolcow-master.simg

Docker hub caching

When you run a Docker container from Docker Hub using Singularity the contents of the container image will be cached in your home directory in the '~/.singularity/docker'. You should either monitor the /home quota usage by the cache if you frequently use docker hub images or symlink that directory to your /ufrc directory. E.g.

ln -s /ufrc/my_group/$USER/singularity/ ~/.singularity

Singularity will detect the symlink and set the cache to the target directory in /ufrc.