Difference between revisions of "Conda"
Moskalenko (talk | contribs) |
Moskalenko (talk | contribs) |
||
| Line 35: | Line 35: | ||
{{#if: {{#var: exe}}|==Additional Information== | {{#if: {{#var: exe}}|==Additional Information== | ||
UF Research Computing Applications Team uses conda for many application installs behind the scenes. We are happy to [https://support.rc.ufl.edu install applications on request] for you. However, if you would like to use conda to create multiple environments for your personal projects we encourage you to do so. Here are some recommendations for successful conda use on HiPerGator. | UF Research Computing Applications Team uses conda for many application installs behind the scenes. We are happy to [https://support.rc.ufl.edu install applications on request] for you. However, if you would like to use conda to create multiple environments for your personal projects we encourage you to do so. Here are some recommendations for successful conda use on HiPerGator. | ||
| + | |||
| + | * We suggest not installing your own copy of miniconda or anaconda, but using one of our conda or [[Mamba|mamba]] environment modules to get the 'conda engine' ''for free''. The only difference is that a personal miniconda3 install will overwrite your ~/.bashrc to make creating environments by name (with the -n argument) in your ~/.conda/envs directory more straightforward and the 'conda activate' command available in your shell environment. We don't use that approach for our work and do not recommend it. | ||
| + | |||
| + | We would like to suggest using '-p' argument to create environments by 'path', so they wouldn't fill up your home directory (check quota with home_quota) and could be located conveniently close to the project(s) they serve in /blue for better tracking of installs | ||
| + | |||
| + | Use | ||
| + | conda create -yp /path/to/the/environment | ||
| + | command instead of 'conda create -n name_of_the_env' | ||
| + | |||
| + | The above has the advantage over the default conda behavior of creating envs by name in your home directory that you can create an environment in blue and not run the risk of filling up your home_quota. It's also much easier to keep track of the many environments if they are located closer to the projects they serve rather than being dumped into ~/.conda/envs as conda usually does by default. Note, as described in | ||
* We would like to point out that there is a clear distinction between using conda environments to install packages and using applications installed in those environments. To use a conda environment you need to add its 'bin' directory to your shell $PATH whether in an interactive session or a job script submitted to the scheduler. To modify a conda environment by installing or removing packages you need to 'activate' the environment. Because of that distinction we strongly recommend against allowing miniconda to insert its activation code into your shell initialization file ~/.bashrc. If you already let it do so please remove the offending code from ~/.bashrc. It's the text within and including the | * We would like to point out that there is a clear distinction between using conda environments to install packages and using applications installed in those environments. To use a conda environment you need to add its 'bin' directory to your shell $PATH whether in an interactive session or a job script submitted to the scheduler. To modify a conda environment by installing or removing packages you need to 'activate' the environment. Because of that distinction we strongly recommend against allowing miniconda to insert its activation code into your shell initialization file ~/.bashrc. If you already let it do so please remove the offending code from ~/.bashrc. It's the text within and including the | ||
| Line 50: | Line 60: | ||
$ module load conda | $ module load conda | ||
| − | $ source activate | + | $ source activate /path/to/the/environment |
Once you are done installing packages inside the environment you can use | Once you are done installing packages inside the environment you can use | ||
Revision as of 14:20, 5 March 2021
Description
Conda is an open-source package management system and environment management system that runs on Windows, macOS, and Linux. Conda quickly installs, runs, and updates packages and their dependencies. Conda easily creates, saves, loads, and switches between environments. Separating applications in separate conda environments allows installation of incompatible dependencies - python2 and python3 for example.
Miniconda is a free minimal installer for conda. It is a small, bootstrap version of Anaconda that includes only conda, Python, the packages they depend on, and a small number of other useful packages, including pip, zlib and a few others. Use the conda install command to install additional conda packages from repositories (channels) like bioconda.
Environment Modules
Run module spider conda to find out what environment modules are available for this application.
System Variables
- HPC_CONDA_DIR - installation directory
- HPC_CONDA_BIN - executable directory
Additional Information
UF Research Computing Applications Team uses conda for many application installs behind the scenes. We are happy to install applications on request for you. However, if you would like to use conda to create multiple environments for your personal projects we encourage you to do so. Here are some recommendations for successful conda use on HiPerGator.
- We suggest not installing your own copy of miniconda or anaconda, but using one of our conda or mamba environment modules to get the 'conda engine' for free. The only difference is that a personal miniconda3 install will overwrite your ~/.bashrc to make creating environments by name (with the -n argument) in your ~/.conda/envs directory more straightforward and the 'conda activate' command available in your shell environment. We don't use that approach for our work and do not recommend it.
We would like to suggest using '-p' argument to create environments by 'path', so they wouldn't fill up your home directory (check quota with home_quota) and could be located conveniently close to the project(s) they serve in /blue for better tracking of installs
Use
conda create -yp /path/to/the/environment
command instead of 'conda create -n name_of_the_env'
The above has the advantage over the default conda behavior of creating envs by name in your home directory that you can create an environment in blue and not run the risk of filling up your home_quota. It's also much easier to keep track of the many environments if they are located closer to the projects they serve rather than being dumped into ~/.conda/envs as conda usually does by default. Note, as described in
- We would like to point out that there is a clear distinction between using conda environments to install packages and using applications installed in those environments. To use a conda environment you need to add its 'bin' directory to your shell $PATH whether in an interactive session or a job script submitted to the scheduler. To modify a conda environment by installing or removing packages you need to 'activate' the environment. Because of that distinction we strongly recommend against allowing miniconda to insert its activation code into your shell initialization file ~/.bashrc. If you already let it do so please remove the offending code from ~/.bashrc. It's the text within and including the
# >>> conda initialize >>> ... # <<< conda initialize <<<
lines.
Instead, load our conda environment module, which we keep up-to-date with conda releases any time you need to create conda environments and to install packages in them. The difference is that 'conda activate' command will not be available, so you will have to use 'source activate' command instead.
E.g.
$ module load conda $ source activate /path/to/the/environment
Once you are done installing packages inside the environment you can use
$ conda deactivate
We do not recommend activating conda environments when _using_ them i.e. running programs installed in the environments. Please prepend the path to that environment to your $PATH instead.
E.g. If you have a project-specific conda environment at '/home/myuser/envs/project1/' add the following into your job script before executing any commands
export PATH=/home/myuser/envs/project1/bin:$PATH