Portainer Templates logo

Portainer Templates

Buildkite Agent

Stack

Dev Tools

A Buildkite agent: Buildkite hosts the UI and orchestration, your agents run the jobs - the hybrid-CI model where compute was always meant to be yours. This is the cleanest runner in the catalogue: official image, one token, done.

Image details

Pulls: 9.8M
Architecture: amd64, arm64
Image size: 30 MB
Latest: 4.0.6-sidecar
User: buildkite
Created: Feb 25, 2015
Updated: 15 hours ago
Status: active

Source details

Repo: buildkite/agent
Stars: 1k
Forks: 383
Language: Go
License: MIT
Latest: v4.0.6 · 1 day ago
Updated: 7 hours ago
Website: buildkite.com/

Configuration

Type
Compose
Platform
linux
Image
buildkite/agent:3-ubuntu
Volumes
/buildkite/builds : builds
Env vars
BUILDKITE_AGENT_TOKEN=${BUILDKITE_AGENT_TOKEN}BUILDKITE_AGENT_TAGS=${BUILDKITE_AGENT_TAGS:-queue=miget}
Restart
unless-stopped
Source
Repo

Template by deployable-sh·Source

Report issue

Standalone Install

Select an install method, to see config/commands for deploying Buildkite Agent

Installation method

Install on Portainer

Import all app templates into your Portainer instance, for easy 1-click deploys

  1. Ensure both Docker and Portainer are installed, and up-to-date
  2. Log into your Portainer web UI
  3. Under Settings → App Templates, paste the below URL
  4. Head to Home → App Templates, and the list of apps will show up
  5. Select Buildkite Agent, fill in any config options, and hit Deploy

Template Import URL

https://raw.githubusercontent.com/Lissy93/portainer-templates/main/templates.json
Show Me demo
Original stackfile

The compose file this template deploys, straight from its repo:

name: buildkite-agent

services:
  agent:
    image: buildkite/agent:3-ubuntu
    restart: unless-stopped
    environment:
      BUILDKITE_AGENT_TOKEN: ${BUILDKITE_AGENT_TOKEN}
      BUILDKITE_AGENT_TAGS: ${BUILDKITE_AGENT_TAGS:-queue=miget}
    volumes:
      - builds:/buildkite/builds

volumes:
  builds:

Or deploy it directly from the source:

git clone https://github.com/deployable-sh/stacks
cd stacks
docker compose -f buildkite-agent/compose.yaml up -d

More install options in our documentation, or see buildkite/agent for app-specific guidance.

Usage

docker run --rm buildkite/agent:3 --help

Versioning

The default tag (i.e. buildkite/agent and buildkite/agent:latest) will always point to the latest stable release (currently 3.x).
We recommend you use buildkite/agent:3 for new setups.
If you want to use an exact version of Buildkite Agent you can use the corresponding tag, such as buildkite/agent:3.0.1.
You can see all the available versions on https://hub.docker.com/r/buildkite/agent/tags/.

Configuring the agent

Most agent configuration settings can be set with environment variables. Setting sensitive data, (such as the agent token) via environment variables or command line arguments is not recommended as they are exposed in commands such as docker inspect.
In addition to environment variables, you can copy or mount a configuration file to /buildkite/buildkite-agent.cfg, for example:
docker run -it \
  -v "$HOME/buildkite-agent.cfg:/buildkite/buildkite-agent.cfg:ro" \
  buildkite/agent:3

Adding hooks

You can add custom agent hooks by mounting or copying them into the /buildkite/hooks directory (and ensuring they are executable).
For example, this is how you'd mount the hooks directory using a read-only host volume:
docker run -it \
  -v "$HOME/buildkite-hooks:/buildkite/hooks:ro" \
  buildkite/agent:3

Alternatively, if you create your own image based off buildkite/agent, you can copy your hooks into the correct location:
FROM buildkite/agent

ADD hooks /buildkite/hooks/

Exposing build secrets into the container

There are many approaches to exposing secrets to Docker containers. In addition, many Docker platforms have their own methods for exposing secrets. If you’re running your own Docker containers, we recommend using a read-only host volume.
The following example mounts a directory containing secrets on the host machine ($HOME/buildkite-secrets) into the container as a read-only data volume at /buildkite-secrets:
docker run -it \
  -v "$HOME/buildkite-secrets:/buildkite-secrets:ro" \
  buildkite/agent:3

You can then use an environment agent hook to expose those secrets via environment variables, or to configure tools such as git or ssh.

Authenticating private git repositories

To configure a git-credentials file located at /buildkite-secrets/git-credentials, you could use the following environment agent hook mounted to /buildkite/hooks/environment:
#!/bin/bash

set -euo pipefail

git config --global credential.helper "store --file=/buildkite-secrets/git-credentials"

# You can export other secrets here too
# export FOO=bar

To configure a private SSH key located at /buildkite-secrets/id_rsa_buildkite_git you could use the following environment agent hook mounted to /buildkite/hooks/environment:
#!/bin/bash

set -euo pipefail

eval "$(ssh-agent -s)"
ssh-add -k /buildkite-secrets/id_rsa_buildkite_git

# You can export other secrets here too
# export FOO=bar

Other options for configuring Git and SSH include:
  • Running ssh-agent on the host machine and mounting the ssh-agent socket into the containers. See the Buildkite Agent SSH keys documentation for examples on using ssh-agent.
  • The least-secure approach: the built-in docker-ssh-env-config support allows you to pass in keys via environment variables.

Invoking Docker from within a build

To invoke Docker from within builds you'll need to mount the Docker socket into the container:
docker run -it \
  -v /var/run/docker.sock:/var/run/docker.sock \
  buildkite/agent:3

Note that this gives builds the same access to the host system as docker has, which is generally root.

Entrypoint customizations

The entrypoint uses tini to correctly pass signals to, and kill, sub-processes. Instead of redefining ENTRYPOINT we recommend you copy executable scripts into /docker-entrypoint.d/. All executable scripts should not contain any file extension, and will be executed in alphanumeric order.

Check the logs first

Nine times out of ten the logs tell you exactly what went wrong.

  • In Portainer, go to Containers, click the container, then Logs. Or run docker logs <container>
  • Exit codes help too: 137 means killed, usually out of memory. 126 or 127 means the command inside the image is broken.

Image won't pull

Test the pull directly on the host: docker pull buildkite/agent:3-ubuntu

  • "manifest unknown" means the tag no longer exists.
  • "toomanyrequests" is the Docker Hub rate limit. Log in with docker login to raise it.
  • "no space left on device" means a full disk. Reclaim space with docker system prune

"exec format error"

This means the image was built for a different CPU architecture than your server.

  • This image supports: amd64, arm64
  • Check yours with uname -m: x86_64 is amd64, aarch64 is arm64. Raspberry Pi and other ARM boards are the usual culprits.

Container keeps restarting

The unless-stopped restart policy relaunches the app after every crash, so the real error can scroll past.

  • Check the logs right after a restart, the last few lines before it died are the useful ones.
  • Get the exit code with docker inspect <container> --format '{{.State.ExitCode}}'
  • Still stuck? Redeploy once with the restart policy set to no so the failure stays visible.

Stack won't deploy

Compose stacks fail fast on small mistakes, and Portainer shows the reason just above the editor.

  • YAML only accepts spaces for indentation, a single tab breaks the whole file.

Raise an issue

Found something which isn't working as it should? Here's how to report it.

A Compose stack

Buildkite Agent is a Compose stack, a set of containers defined in one file and brought up together by Portainer, then started and stopped as a single app.

The app image

An image is the app packed up ready to go, everything Buildkite Agent needs bundled into one download. This template pulls buildkite/agent:3-ubuntu, which Docker fetches once (about 30 MB) and then starts your own copy from.

Where the image comes from

Docker pulls its images from registries, public libraries of ready-built apps. Buildkite Agent's comes from Docker Hub, published by buildkite.

Version tags

The bit after the colon in the image name is the version tag. This one pins 3-ubuntu, so every redeploy gives you that exact build until you bump it yourself.

Which machines it runs on

Every image is built for particular CPU types. This one ships for amd64, arm64, so it runs on both regular x86 servers and ARM boards like a Raspberry Pi.

Volumes

A volume is where Buildkite Agent keeps its files so they survive an update or a restart. Without one, anything it saves would sit inside the container and vanish the moment it's recreated. This template mounts:

  • /buildkite/builds kept in the builds volume Docker manages

Environment variables

Environment variables are the settings you hand over when you deploy, things like a password or a timezone. Buildkite Agent takes 2 of them, all with defaults you can leave alone or tweak:

  • BUILDKITE_AGENT_TOKEN, pulled from your own environment. Agent token: Buildkite > Agents > Reveal Agent Token.
  • BUILDKITE_AGENT_TAGS, defaults to queue=miget. Pipelines target agents by tags, e.g.

Restart policy

The restart policy here is unless-stopped, so Docker restarts Buildkite Agent after a crash or reboot, but leaves it off when you stop it on purpose. You can change this on the deploy screen. The choices are no (never restart), on-failure (only after a crash), unless-stopped (restart unless you stop it), and always (bring it back no matter what).

Networking

Nothing custom is set, so Buildkite Agent sits on Docker's default bridge network: its own private space that reaches the outside world only through the ports it publishes.

Container name

Once it's deployed, Portainer names the container agent. That's what you'll spot in the containers list and use in commands like docker logs agent.

Platform

The platform is linux, the kind of system the container is built to run on. Docker and Portainer handle this on a normal Linux server.

Open source license

Buildkite Agent is open source, released under the MIT license. In plain terms the code is out in the open, so you're free to run it and change it to fit what you need.

Portainer app templates

Zooming out, this whole page comes from a Portainer app template: a short recipe telling Portainer how to set Buildkite Agent up. Add the template list to Portainer once, then deploying Buildkite Agent is a click rather than a wall of config.