Ollama
Jump to navigation
Jump to search
Description
Get up and running with large language models.
Environment Modules
Run module spider ollama to find out what environment modules are available for this application.
System Variables
- HPC_OLLAMA_DIR - installation directory
Additional Information
Users need to start two interactive sessions on the same GPU node to use the application - one session to start the ollama server and the other to chat with LLMs.
There are two options for starting the ollama server:
1. $ ollama serve (use default environmental variables).
2. $ env {options} ollama serve (pass environmental variables to server).
For example: set host to 127.0.0.1:11435, keep models in memory for 1 hour, and utilize all assigned gpus:
$ env OLLAMA_HOST=127.0.0.1:11435 OLLAMA_KEEP_ALIVE=60m OLLAMA_SCHED_SPREAD=T ollama serve