Ollama, get up and running with large language models, locally.

This Community Chart is for deploying Ollama.

Requirements

• Kubernetes: >= 1.16.0-0 for CPU only

• Kubernetes: >= 1.26.0-0 for GPU stable support (NVIDIA and AMD)

Not all GPUs are currently supported with ollama (especially with AMD)

Deploying Ollama chart

To install the ollama chart in the ollama namespace:

helm repo add ollama-helm https://otwld.github.io/ollama-helm/
helm repo update
helm install ollama ollama-helm/ollama --namespace ollama

Upgrading Ollama chart

First please read the release notes of Ollama to make sure there are no backwards incompatible changes.

Make adjustments to your values as needed, then run helm upgrade:

# -- This pulls the latest version of the ollama chart from the repo.
helm repo update
helm upgrade ollama ollama-helm/ollama --namespace ollama --values values.yaml

Uninstalling Ollama chart

To uninstall/delete the ollama deployment in the ollama namespace:

helm delete ollama --namespace ollama

Substitute your values if they differ from the examples. See helm delete --help for a full reference on delete parameters and flags.

Interact with Ollama

• Ollama documentation can be found HERE
• Interact with RESTful API: Ollama API
• Interact with official clients libraries: ollama-js and ollama-python
• Interact with langchain: langchain-js and langchain-python

Examples

• It's highly recommended to run an updated version of Kubernetes for deploying ollama with GPU

Basic values.yaml example with GPU and two models pulled at startup

ollama:
  gpu:
    # -- Enable GPU integration
    enabled: true
    
    # -- GPU type: 'nvidia' or 'amd'
    type: 'nvidia'
    
    # -- Specify the number of GPU to 1
    number: 1
   
  # -- List of models to pull at container startup
  models: 
    - mistral
    - llama2

Basic values.yaml example with Ingress

ollama:
  models:
    - llama2
  
ingress:
  enabled: true
  hosts:
  - host: ollama.domain.lan
    paths:
      - path: /
        pathType: Prefix

• API is now reachable at ollama.domain.lan

Helm Values

• See values.yaml to see the Chart's default values.

Core team

Jean Baptiste Detroyes

Nathan Tréhout

Support

• For questions, suggestions, and discussion about Ollama please refer to the Ollama issue page
• For questions, suggestions, and discussion about this chart please visit Ollama-Helm issue page or join our OTWLD Discord