Ollama
Revision as of 15:55, 1 October 2024 by Achander (talk | contribs) (Created page with "Category:SoftwareCategory:AI {|<!--CONFIGURATION: REQUIRED--> |{{#vardefine:app|ollama}} |{{#vardefine:url|https://github.com/ollama/ollama}} <!--CONFIGURATION: OPTION...")
Description
Get up and running with large language models.
Environment Modules
Run module spider ollama to find out what environment modules are available for this application.
System Variables
- HPC_OLLAMA_DIR - installation directory
Additional Information
Users need to start two interactive sessions on the same GPU node to use the application - one session to start the ollama server and the other to chat with LLMs.
There are two options for starting the ollama server:
1. $ ollama serve (use default environmental variables).
2. $ env {options} ollama serve (pass environmental variables to server).
For example: set host to 127.0.0.1:11435, keep models in memory for 1 hour, and utilize all assigned gpus:
$ env OLLAMA_HOST=127.0.0.1:11435 OLLAMA_KEEP_ALIVE=60m OLLAMA_SCHED_SPREAD=T ollama serve