Portainer Templates logo

Portainer Templates

LiteLLM LiteLLM

Stack

LLM Infrastructure

LiteLLM proxy - one OpenAI-compatible endpoint in front of every LLM provider (Anthropic, OpenAI, Gemini, Bedrock, Mistral, local, …) with virtual keys, per-key budgets, spend tracking, fallbacks and retries.

Configuration

Type
Compose
Platform
linux
Env vars
LITELLM_MASTER_KEY=''LITELLM_SALT_KEY=''
Source
Repo

Template by deployable-sh·Source

Report issue

Services

litellm

Configuration

Image
ghcr.io/berriai/litellm-database:v1.93.0
Command
--port,5000
Ports
5000:5000
Env vars
DATABASE_URL=postgresql://litellm:litellm-local-only@db:5432/litellmLITELLM_MASTER_KEY=${LITELLM_MASTER_KEY}LITELLM_SALT_KEY=${LITELLM_SALT_KEY}STORE_MODEL_IN_DB=True
Restart
unless-stopped

Image details

User: berriai

db

Configuration

Image
postgres:17
Volumes
/var/lib/postgresql/data : dbdata
Env vars
POSTGRES_USER=litellmPOSTGRES_PASSWORD=litellm-local-onlyPOSTGRES_DB=litellm
Restart
unless-stopped

Image details

Pulls: 11.6B
User: stackbrew
Created: Jun 05, 2014
Updated: 1 hour ago
Status: active

Standalone Install

Select an install method, to see config/commands for deploying LiteLLM

Installation method

Install on Portainer

Import all app templates into your Portainer instance, for easy 1-click deploys

  1. Ensure both Docker and Portainer are installed, and up-to-date
  2. Log into your Portainer web UI
  3. Under Settings → App Templates, paste the below URL
  4. Head to Home → App Templates, and the list of apps will show up
  5. Select LiteLLM, fill in any config options, and hit Deploy

Template Import URL

https://raw.githubusercontent.com/Lissy93/portainer-templates/main/templates.json
Show Me demo
Original stackfile

The compose file this template deploys, straight from its repo:

name: litellm

services:
  litellm:
    healthcheck:
      test: ["CMD-SHELL", "python3 -c 'import urllib.request as u; u.urlopen(\"http://127.0.0.1:5000/health/liveliness\", timeout=4)'"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s
    image: ghcr.io/berriai/litellm-database:v1.93.0
    restart: unless-stopped
    command: ["--port", "5000"]
    environment:
      DATABASE_URL: postgresql://litellm:litellm-local-only@db:5432/litellm
      LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}
      LITELLM_SALT_KEY: ${LITELLM_SALT_KEY}
      STORE_MODEL_IN_DB: "True"
    ports:
      - "5000:5000"
    depends_on:
      - db

  db:
    healthcheck:
      test: ["CMD", "pg_isready", "-h", "127.0.0.1"]
      interval: 10s
      timeout: 5s
      retries: 5
    image: postgres:17
    restart: unless-stopped
    environment:
      POSTGRES_USER: litellm
      POSTGRES_PASSWORD: litellm-local-only
      POSTGRES_DB: litellm
    volumes:
      - dbdata:/var/lib/postgresql/data

volumes:
  dbdata:

Or deploy it directly from the source:

git clone https://github.com/deployable-sh/stacks
cd stacks
docker compose -f litellm/compose.yaml up -d

More install options in our documentation.

Container Documentation

db Documentation

The PostgreSQL object-relational database system provides reliability and data integrity.

Serve LiteLLM on your own domain behind Caddy, Nginx or Traefik. Fill in your domain and copy the result. It's a starting point, some apps need their own base URL or extra headers set too.

Proxying litellm.example.com to http://litellm:5000

Add this to your Caddyfile

litellm.example.com {
	reverse_proxy http://litellm:5000
}

Check the logs first

Nine times out of ten the logs tell you exactly what went wrong.

  • In Portainer, go to Containers, click the container, then Logs. Or run docker logs <container>
  • Exit codes help too: 137 means killed, usually out of memory. 126 or 127 means the command inside the image is broken.

Port already in use

If deployment fails with "Bind for 0.0.0.0:5000 failed: port is already allocated", something else on your server is using that port.

  • Find what's using it: sudo ss -tlnp | grep :5000
  • Stop the other service, or pick a different host port. In 5000:5000 only the left number is yours to change, the right one belongs to the app.

Running but the page won't load

The container is up but nothing appears in your browser.

  • Use your server's real IP: http://your-server-ip:5000. The 0.0.0.0 link Portainer shows isn't a real address.
  • Give it a minute after first deploy, litellm can take a while to initialise.
  • Make sure your firewall allows the port, e.g. sudo ufw allow 5000

Image won't pull

Test the pull directly on the host: docker pull ghcr.io/berriai/litellm-database:v1.93.0

  • "manifest unknown" means the tag no longer exists.
  • "toomanyrequests" is the Docker Hub rate limit. Log in with docker login to raise it.
  • "no space left on device" means a full disk. Reclaim space with docker system prune

"exec format error"

This means the image was built for a different CPU architecture than your server.

  • Check yours with uname -m: x86_64 is amd64, aarch64 is arm64. Raspberry Pi and other ARM boards are the usual culprits.

Container keeps restarting

The unless-stopped restart policy relaunches the app after every crash, so the real error can scroll past.

  • Check the logs right after a restart, the last few lines before it died are the useful ones.
  • Get the exit code with docker inspect <container> --format '{{.State.ExitCode}}'
  • Still stuck? Redeploy once with the restart policy set to no so the failure stays visible.

Stack won't deploy

Compose stacks fail fast on small mistakes, and Portainer shows the reason just above the editor.

  • YAML only accepts spaces for indentation, a single tab breaks the whole file.

Raise an issue

Found something which isn't working as it should? Here's how to report it.

A Compose stack

LiteLLM is a Compose stack, a set of containers (2 of them) defined in one file and brought up together by Portainer, then started and stopped as a single app.

The services

This stack is built from 2 containers that run side by side. Here's each one, with the image it runs and anything it waits for first:

  • litellm runs ghcr.io/berriai/litellm-database:v1.93.0, starts after db
  • db runs postgres:17

Ports

A port is the door the app answers on. A mapping like 5000:5000 means it's reachable on port 5000 of your server, where the left number is yours to change and the right one belongs to the app. It opens:

  • 5000:5000

Volumes

A volume is where LiteLLM keeps its files so they survive an update or a restart. Without one, anything it saves would sit inside the container and vanish the moment it's recreated. This template mounts:

  • /var/lib/postgresql/data kept in the dbdata volume Docker manages

Environment variables

Environment variables are the settings you hand over when you deploy, things like a password or a timezone. LiteLLM takes 7 of them, all with defaults you can leave alone or tweak:

  • DATABASE_URL, defaults to postgresql://litellm:litellm-local-only@db:5432/litellm
  • LITELLM_MASTER_KEY, pulled from your own environment
  • LITELLM_SALT_KEY, pulled from your own environment
  • STORE_MODEL_IN_DB, defaults to True
  • POSTGRES_USER, defaults to litellm
  • POSTGRES_PASSWORD, defaults to litellm-local-only
  • POSTGRES_DB, defaults to litellm

Restart policy

The restart policy here is unless-stopped, so Docker restarts LiteLLM after a crash or reboot, but leaves it off when you stop it on purpose. You can change this on the deploy screen. The choices are no (never restart), on-failure (only after a crash), unless-stopped (restart unless you stop it), and always (bring it back no matter what).

Networking

Portainer puts these services on one shared private network, so they can find each other by name (like litellm) while only the ports above are open to you.

Platform

The platform is linux, the kind of system the container is built to run on. Docker and Portainer handle this on a normal Linux server.

Portainer app templates

Zooming out, this whole page comes from a Portainer app template: a short recipe telling Portainer how to set LiteLLM up. Add the template list to Portainer once, then deploying LiteLLM is a click rather than a wall of config.