View source for Ollama

[[Category:Software]][[Category:AI]]
{|<!--CONFIGURATION: REQUIRED-->
|{{#vardefine:app|ollama}}
|{{#vardefine:url|https://github.com/ollama/ollama}}
<!--CONFIGURATION: OPTIONAL (|1}} means it's ON)-->
|{{#vardefine:conf|}}           <!--CONFIGURATION-->
|{{#vardefine:exe|1}}            <!--ADDITIONAL INFO-->
|{{#vardefine:job|}}            <!--JOB SCRIPTS-->
|{{#vardefine:policy|}}         <!--POLICY-->
|{{#vardefine:testing|}}       <!--PROFILING-->
|{{#vardefine:faq|}}             <!--FAQ-->
|{{#vardefine:citation|}}       <!--CITATION-->
|{{#vardefine:installation|}} <!--INSTALLATION-->
|}
<!--BODY-->
<!--Description-->
{{#if: {{#var: url}}|
{{App_Description|app={{#var:app}}|url={{#var:url}}|name={{#var:app}}}}|}}

Get up and running with large language models.

<!--Modules-->
==Environment Modules==
Run <code>module spider {{#var:app}}</code> to find out what environment modules are available for this application.
==System Variables==
* HPC_{{uc:{{#var:app}}}}_DIR - installation directory
<!--Configuration-->
{{#if: {{#var: conf}}|==Configuration==
See the [[{{PAGENAME}}_Configuration]] page for {{#var: app}} configuration details.
|}}
<!--Run-->
{{#if: {{#var: exe}}|==Additional Information==

{{Note | HiPerGator users should be aware that the State of Florida prohibits the use of '''DeepSeek''' models, like '''R1'''. Please consult the [https://www.dms.myflorida.com/prohibited_applications_list list of prohibited applications]. This prohibition extends to HiPerGator and all state-owned devices.|warn}}

=== Interactive OLLAMA use ===
Users need to start an interactive HiperGator Desktop session session on a GPU node at Open Ondemand (https://ood.rc.ufl.edu/) and launch two terminals, one to start the ollama server and the other to chat with LLMs. 

In terminal 1, load the ollama module and start the server with either default or custom environmental settings: 

 1. Default settings
    $ ml ollama
    $ ollama serve (default environmental variables).

 2. Custom settings
    $ ml ollama
    $ env {options} ollama serve (pass environmental variables to server). 
    
    For example: set custom path to LLMs models:
    
    $ env OLLAMA_MODELS=/blue/group/$USER/ollama/models ollama serve

In terminal 2, pull a model and start chatting. For example, llama3.2:

   $ ml ollama
   $ ollama pull llama3.2
   $ ollama run llama3.2

=== OLLAMA as a Slurm job ===

 #!/bin/bash
 #SBATCH --job-name=ollama
 #SBATCH --output=ollama_%j.log
 #SBATCH --ntasks=1
 #SBATCH --mem=8gb
 #SBATCH --partition=gpu
 #SBATCH --gpus=a100:1
 #SBATCH --time=01:00:00
 date;hostname;pwd
 module load ollama
 env_path=/my/conda/env/bin 
 #add conda env with langchain to path
 export PATH=$env_path:$PATH
 ollama serve &
 ollama pull mistral
 python my_ollama_python_script.py >> my_ollama_output.txt

Example python script:
 
 from langchain.llms import Ollama
 ollama = Ollama(model="mistral")
 print(ollama.invoke("why is the sky blue"))


|}}
<!--Job Scripts-->
{{#if: {{#var: job}}|==Job Script Examples==
See the [[{{PAGENAME}}_Job_Scripts]] page for {{#var: app}} Job script examples.
|}}
<!--Policy-->
{{#if: {{#var: policy}}|==Usage Policy==

WRITE USAGE POLICY HERE (Licensing, usage, access).

|}}
<!--Performance-->
{{#if: {{#var: testing}}|==Performance==

WRITE_PERFORMANCE_TESTING_RESULTS_HERE

|}}
<!--Faq-->
{{#if: {{#var: faq}}|==FAQ==
*'''Q:''' **'''A:'''|}}
<!--Citation-->
{{#if: {{#var: citation}}|==Citation==
If you publish research that uses {{#var:app}} you have to cite it as follows:

WRITE_CITATION_HERE

|}}
<!--Installation-->
{{#if: {{#var: installation}}|==Installation==
See the [[{{PAGENAME}}_Install]] page for {{#var: app}} installation notes.|}}
<!--Turn the Table of Contents and Edit paragraph links ON/OFF-->
__NOTOC____NOEDITSECTION__