For help on creating and managing personal environments whether for command-tool use or python package use in SLURM jobs or Jupyter kernels see Managing Python environments and Jupyter kernels
Conda is an open-source package management system and environment management system that runs on Windows, macOS, and Linux. Conda quickly installs, runs, and updates packages and their dependencies. Conda easily creates, saves, loads, and switches between environments. Separating applications in separate conda environments allows installation of incompatible dependencies - python2 and python3 for example.
Note: For a faster conda see Mamba.
module spider conda to find out what environment modules are available for this application.
- HPC_CONDA_DIR - installation directory
- HPC_CONDA_BIN - executable directory
Many projects that use Python code require careful management of the respective Python environments. Rapid changes in package dependencies, package version conflicts, deprecation of APIs (function calls) by individual projects, and obsolescence of system drivers and libraries make it virtually impossible to use an arbitrary set of packages or create one all-encompassing environment that will serve everyone's needs over long periods of time. The high velocity of changes in the popular ML/DL frameworks and packages and GPU computing exacerbates the problem.
The problem with
Create and activate your first environment
Export or import an environment
It is possible to create a shared environment accessed by a group on HiPerGator, storing the environment in, for example,
/blue/group/share/conda. In general, this works best if only one user has write access to the environment. All installs should be made by that one user and should be communicated with the other users in the group.