Portainer Templates logo

Portainer Templates

Ollama Ollama

Stack

LLM Infrastructure

Ollama - run open large language models locally behind a simple HTTP API. Pull a model (Llama, Mistral, Qwen, Gemma, Phi, and many more) and serve it from your own infrastructure, with OpenAI-compatible endpoints.

Image details

Pulls: 178.7M
Architecture: amd64, arm64
Image size: 3.5 GB
Latest: 0.34.4
User: ollama
Created: Jun 29, 2023
Updated: 10 hours ago
Status: active

Source details

Repo: ollama/ollama
Stars: 182k
Forks: 18.0k
Language: Go
License: MIT
Latest: v0.34.4 · 1 day ago
Updated: 13 minutes ago
Website: ollama.com/

Configuration

Type
Compose
Platform
linux
Image
ollama/ollama:0.30.8
Ports
5000:5000
Volumes
/root/.ollama : models
Env vars
OLLAMA_HOST=0.0.0.0:5000
Restart
unless-stopped
Source
Repo

Template by deployable-sh·Source

Report issue

Standalone Install

Select an install method, to see config/commands for deploying Ollama

Installation method

Install on Portainer

Import all app templates into your Portainer instance, for easy 1-click deploys

  1. Ensure both Docker and Portainer are installed, and up-to-date
  2. Log into your Portainer web UI
  3. Under Settings → App Templates, paste the below URL
  4. Head to Home → App Templates, and the list of apps will show up
  5. Select Ollama, fill in any config options, and hit Deploy

Template Import URL

https://raw.githubusercontent.com/Lissy93/portainer-templates/main/templates.json
Show Me demo
Original stackfile

The compose file this template deploys, straight from its repo:

name: ollama

services:
  ollama:
    image: ollama/ollama:0.30.8
    restart: unless-stopped
    environment:
      OLLAMA_HOST: "0.0.0.0:5000"
    ports:
      - "5000:5000"
    volumes:
      - models:/root/.ollama

volumes:
  models:

Or deploy it directly from the source:

git clone https://github.com/deployable-sh/stacks
cd stacks
docker compose -f ollama/compose.yaml up -d

More install options in our documentation, or see ollama/ollama for app-specific guidance.

Ollama Docker image

Discord
Ollama makes it easy to get up and running with large language models locally.

CPU only

docker run -d -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

Nvidia GPU

Install the NVIDIA Container Toolkit.

Install with Apt

  1. Configure the repository
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
    | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
    | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
    | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
  1. Install the NVIDIA Container Toolkit packages
sudo apt-get install -y nvidia-container-toolkit

Install with Yum or Dnf

  1. Configure the repository
curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo \
    | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
  1. Install the NVIDIA Container Toolkit packages
sudo yum install -y nvidia-container-toolkit

Configure Docker to use Nvidia driver 

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Start the container

docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

AMD GPU

To run Ollama using Docker with AMD GPUs, use the rocm tag and the following command:
docker run -d --device /dev/kfd --device /dev/dri -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama:rocm

Run model locally

Now you can run a model:
docker exec -it ollama ollama run llama3

Try different models

More models can be found on the Ollama library.

Learn more

https://github.com/ollama/ollama

Serve Ollama on your own domain behind Caddy, Nginx or Traefik. Fill in your domain and copy the result. It's a starting point, some apps need their own base URL or extra headers set too.

Proxying ollama.example.com to http://ollama:5000

Add this to your Caddyfile

ollama.example.com {
	reverse_proxy http://ollama:5000
}

Check the logs first

Nine times out of ten the logs tell you exactly what went wrong.

  • In Portainer, go to Containers, click the container, then Logs. Or run docker logs <container>
  • Exit codes help too: 137 means killed, usually out of memory. 126 or 127 means the command inside the image is broken.

Port already in use

If deployment fails with "Bind for 0.0.0.0:5000 failed: port is already allocated", something else on your server is using that port.

  • Find what's using it: sudo ss -tlnp | grep :5000
  • Stop the other service, or pick a different host port. In 5000:5000 only the left number is yours to change, the right one belongs to the app.

Running but the page won't load

The container is up but nothing appears in your browser.

  • Use your server's real IP: http://your-server-ip:5000. The 0.0.0.0 link Portainer shows isn't a real address.
  • Give it a minute after first deploy, ollama can take a while to initialise.
  • Make sure your firewall allows the port, e.g. sudo ufw allow 5000

Image won't pull

Test the pull directly on the host: docker pull ollama/ollama:0.30.8

  • "manifest unknown" means the tag no longer exists.
  • "toomanyrequests" is the Docker Hub rate limit. Log in with docker login to raise it.
  • "no space left on device" means a full disk. Reclaim space with docker system prune

"exec format error"

This means the image was built for a different CPU architecture than your server.

  • This image supports: amd64, arm64
  • Check yours with uname -m: x86_64 is amd64, aarch64 is arm64. Raspberry Pi and other ARM boards are the usual culprits.

Container keeps restarting

The unless-stopped restart policy relaunches the app after every crash, so the real error can scroll past.

  • Check the logs right after a restart, the last few lines before it died are the useful ones.
  • Get the exit code with docker inspect <container> --format '{{.State.ExitCode}}'
  • Still stuck? Redeploy once with the restart policy set to no so the failure stays visible.

Stack won't deploy

Compose stacks fail fast on small mistakes, and Portainer shows the reason just above the editor.

  • YAML only accepts spaces for indentation, a single tab breaks the whole file.

Raise an issue

Found something which isn't working as it should? Here's how to report it.

A Compose stack

Ollama is a Compose stack, a set of containers defined in one file and brought up together by Portainer, then started and stopped as a single app.

The app image

An image is the app packed up ready to go, everything Ollama needs bundled into one download. This template pulls ollama/ollama:0.30.8, which Docker fetches once (about 3.5 GB) and then starts your own copy from.

Where the image comes from

Docker pulls its images from registries, public libraries of ready-built apps. Ollama's comes from Docker Hub, published by ollama.

Version tags

The bit after the colon in the image name is the version tag. This one pins 0.30.8, so every redeploy gives you that exact build until you bump it yourself.

Which machines it runs on

Every image is built for particular CPU types. This one ships for amd64, arm64, so it runs on both regular x86 servers and ARM boards like a Raspberry Pi.

Ports

A port is the door the app answers on. A mapping like 5000:5000 means it's reachable on port 5000 of your server, where the left number is yours to change and the right one belongs to the app. It opens:

  • 5000:5000

Volumes

A volume is where Ollama keeps its files so they survive an update or a restart. Without one, anything it saves would sit inside the container and vanish the moment it's recreated. This template mounts:

  • /root/.ollama kept in the models volume Docker manages

Environment variables

Environment variables are the settings you hand over when you deploy, things like a password or a timezone. Ollama takes 1 of them, all with defaults you can leave alone or tweak:

  • OLLAMA_HOST, defaults to 0.0.0.0:5000

Restart policy

The restart policy here is unless-stopped, so Docker restarts Ollama after a crash or reboot, but leaves it off when you stop it on purpose. You can change this on the deploy screen. The choices are no (never restart), on-failure (only after a crash), unless-stopped (restart unless you stop it), and always (bring it back no matter what).

Networking

Nothing custom is set, so Ollama sits on Docker's default bridge network: its own private space that reaches the outside world only through the ports it publishes.

Container name

Once it's deployed, Portainer names the container ollama. That's what you'll spot in the containers list and use in commands like docker logs ollama.

Platform

The platform is linux, the kind of system the container is built to run on. Docker and Portainer handle this on a normal Linux server.

Open source license

Ollama is open source, released under the MIT license. In plain terms the code is out in the open, so you're free to run it and change it to fit what you need.

Portainer app templates

Zooming out, this whole page comes from a Portainer app template: a short recipe telling Portainer how to set Ollama up. Add the template list to Portainer once, then deploying Ollama is a click rather than a wall of config.