Ollama

From UFRC
Revision as of 15:55, 1 October 2024 by Achander (talk | contribs) (Created page with "Category:SoftwareCategory:AI {|<!--CONFIGURATION: REQUIRED--> |{{#vardefine:app|ollama}} |{{#vardefine:url|https://github.com/ollama/ollama}} <!--CONFIGURATION: OPTION...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Description

ollama website  

Get up and running with large language models.

Environment Modules

Run module spider ollama to find out what environment modules are available for this application.

System Variables

  • HPC_OLLAMA_DIR - installation directory

Additional Information

Users need to start two interactive sessions on the same GPU node to use the application - one session to start the ollama server and the other to chat with LLMs.

There are two options for starting the ollama server:

1. $ ollama serve (use default environmental variables).
2. $ env {options} ollama serve (pass environmental variables to server). 
   For example: set host to 127.0.0.1:11435, keep models in memory for 1 hour, and utilize all assigned gpus:
   $ env OLLAMA_HOST=127.0.0.1:11435 OLLAMA_KEEP_ALIVE=60m OLLAMA_SCHED_SPREAD=T ollama serve