Self-hosted installation

OvalBee can run in your cloud account, dedicated VPC or data center. The self-hosted stack uses prebuilt container images; it does not build this checkout. Images, annotations, models and node runs stay in the infrastructure you operate.

The OvalBee CLI installs, upgrades and operates both deployment options below, and prepares Helm values for GitOps. To modify the source or rebuild images, use the DEVELOPMENT guide and ovalbee dev up; only that workflow uses the development Compose overlay. Do not add the development overlay to a self-hosted deployment.

Deployment options

With Kubernetes, node runs can also go to a separate GPU cluster through an executor-only installation of the same chart, while the platform runs elsewhere. See Compute.

Starting node runs and deployments as Kubernetes pods needs the Business plan. A Community installation from the Helm chart runs the platform, but refuses node runs and deployments until it is upgraded; on Community, use Docker Compose to run them.

Before you deploy

• Choose and pin the image tag you intend to run. Compose reads OVALBEE_IMAGE_REGISTRY and OVALBEE_IMAGE_TAG from .env; the chart uses global.imageRegistry and global.imageTag. The defaults name the docker.io/ovalbee namespace, and either setting can point to an internal registry mirror. Ensure that your hosts or cluster can pull every image the selected deployment needs; add Kubernetes pull credentials through global.imagePullSecrets or imagePullSecrets when needed.
• Keep the installation directory and its .env file private. ovalbee install generates every secret and password in it, including OVALBEE_APP_SECRET and OVALBEE_QDRANT_API_KEY, and never replaces them when run again.
• Treat the application secret as durable: it encrypts credentials stored in platform settings. Back it up and do not change it after credentials have been saved. The Qdrant key must remain the same in the API and Qdrant.
• The OvalBee core is licensed under the GNU AGPLv3. The source-available enterprise code in api/src/ee and web/src/ee is proprietary: development and testing are permitted, while production use requires the applicable Enterprise License and subscription. The prebuilt labeling-tool image, third-party libraries, and model weights have their own licenses; the labeling tool’s source is not in this repository.

What gets installed

PostgreSQL, Redis and object storage can each run inside the installation or be services you already operate, such as a managed database or an existing bucket.

What to prepare

• A place to run it: a Kubernetes cluster, or a Linux host with Docker for the Compose option.
• GPU capacity for training and inference: GPU nodes in the cluster with the NVIDIA device plugin, or a GPU host with the NVIDIA container toolkit.
• A DNS name and TLS certificate for the address people will use. On Kubernetes the chart can attach an HTTPRoute to your Gateway API Gateway (httpRoute.enabled, httpRoute.parentRefs, httpRoute.hostnames); the Gateway, its TLS and DNS stay yours.
• Storage: an S3-compatible bucket, Azure container or Google Cloud Storage bucket, unless you use the bundled storage.
• Optionally, managed PostgreSQL and Redis, your identity provider’s OIDC details for single sign-on, and SMTP for email.
• Network access from the cluster to the OvalBee container registry, or a mirror of the images in your own registry.

System requirements

OvalBee images are built for Linux on x86-64. ARM hosts aren’t supported.

The platform itself (web app, API, databases and storage) without any node runs:

Images, annotations and models are kept in object storage, so size it for your datasets and models with room to grow. The Helm chart requests persistent volumes of 20 GB for PostgreSQL, 5 GB for Redis, 20 GB for Qdrant and, when the bundled object storage is enabled, 50 GB for it. Each size is a chart value.

Hosts that run training and inference need:

• NVIDIA driver 570 or newer. The node images are built on CUDA 12.8.
• GPU access for containers. With Docker Compose, install the NVIDIA Container Toolkit and register its nvidia runtime in Docker. On Kubernetes, install the NVIDIA device plugin.
• Disk for node images. One model family’s image takes 15–40 GB, and the whole Node Library downloads about 120 GB of compressed layers. Plan at least 500 GB of fast disk for container images.
• Memory for shared memory. Each GPU run gets 5 GB of shared memory, on top of what the run itself uses.

On Kubernetes, each GPU run requests one GPU. With Docker Compose, a run gets no GPU of its own and no CPU or memory limit, so runs started at the same time share the host.

The cluster also needs a storage class for the volumes and a network plugin that enforces NetworkPolicy. Without one, the policy that keeps node runs away from Qdrant restricts nothing, and Qdrant’s API key is the only protection left.

Configuration

With Docker Compose, ovalbee install writes docker-compose.yml and .env into the installation directory (--dir) and the data into --data-dir; ovalbee upgrade replaces docker-compose.yml and only adds new settings to .env, so keep your own changes in a docker-compose.override.yml beside it. The docker-compose.yml is the image-only self-hosted stack. It mounts the Docker socket for the Docker executor and publishes the configured ingress port; put it only on a host where that access is acceptable. Do not set COMPOSE_FILE=docker-compose.yml:docker-compose.dev.yml for a self-hosted deployment.

The Helm chart generates and retains OVALBEE_APP_SECRET and OVALBEE_QDRANT_API_KEY across normal upgrades when secrets.create is enabled. For GitOps rendering, pin secrets.appSecret and secrets.qdrantApiKey, or provide secrets.existingSecret; a rendered manifest cannot look up a previously generated Secret. An existingSecret replaces the generated Secret rather than extending it, so it must include both keys and the database, Redis, and storage connection values required by your chosen services.

Application settings are separate from process configuration. In Helm, put settings under config.settings (or name a config.existingConfigMap); the chart mounts the resulting file and reapplies every named setting at deployment. File-managed settings are locked in Admin → Configuration. Keep credentials out of values.yaml: the configuration file can reference ${NAME} or ${NAME}_FILE, and config.env can source those values from a Secret. Supplying a setting replaces its whole value, so include all of its credential fields when managing it this way. For settings managed in the UI, see Platform configuration.

Request logs and the audit log record the client’s IP address from the X-Forwarded-For header set by the proxies in front of the API. The API believes only proxies on private addresses (OVALBEE_TRUST_PROXY=loopback,uniquelocal), which covers the Compose network and most Kubernetes pod ranges. If your pods use another range, such as 100.64.0.0/10, list it in OVALBEE_TRUST_PROXY. Behind a cloud load balancer, also make your ingress controller pass the real client address through, for example with use-forwarded-headers or the PROXY protocol in ingress-nginx; otherwise every request is recorded with the load balancer’s address.

With Compose, configure platform settings in the UI after startup unless you deliberately arrange a mounted YAML file and OVALBEE_CONFIG_FILE yourself. Environment variables configure the process and its bundled connections; they are not a general replacement for the platform settings.

Once installed, sign in with the initial administrator account and follow the first-day checklist.

Logs

Task log panels show only what a task printed, plus short status lines. Everything else an installation logs about itself goes to a diagnostic log: Kubernetes and Docker events in short form (reason, message, image, node, container), warnings and errors printed by tasks, with their tracebacks, and warnings and errors from every service of the installation. The diagnostic log leaves out ordinary task output, progress bars, the labels on task pods and request bodies.

The diagnostic log is kept in daily compressed files for 14 days: with Docker Compose under ${OVALBEE_DATA_DIR}/logs, on Kubernetes in the ovalbee-vector pod, on a volume only with logging.files.persistence.enabled (otherwise the files go with the pod).

Every installation also sends the errors from its diagnostic log to OvalBee, at https://input-logs.ovalbee.com, so the team can help when something goes wrong. Only records at error level or worse are sent, so warnings stay in the local files. Among Kubernetes and Docker events, that means the ones where a workload failed: it ran out of memory, exited with an error, was evicted, or its image couldn’t be downloaded. Waiting for a machine, restarts and routine events stay local. Each record sent loses its user id and any address, host or node name, user name or email field, and email and IP addresses in its text are replaced by [email] and [ip]. What remains is the message with its level and time, the task, deployment or service it came from, and technical context such as a traceback, an error type, a request path, or an event’s reason, image and machine class.

To turn sending off, set logging.remote.enabled: false in the chart or OVALBEE_LOGS_REMOTE_ENABLED=false in the logger’s environment for Compose (see the override below). A key from the OvalBee team only tells OvalBee which installation the records come from: set it as logging.remote.key (or a Secret named by logging.remote.existingSecret) in the chart, or OVALBEE_LOGS_REMOTE_KEY for Compose. Without a key, the records are pooled anonymously with those of every other installation without one. If the address can’t be reached, records that don’t fit in a small buffer are dropped; task logs and the local files aren’t held up.

Two settings change what is sent, and the defaults above suit most installations: logging.remote.level (OVALBEE_LOGS_REMOTE_LEVEL for Compose, default error) is the lowest level sent, and logging.remote.anonymize: false (OVALBEE_LOGS_REMOTE_ANONYMIZE=false, default true) keeps user ids, addresses and emails in the records. Credentials in URLs and failed queries stay masked either way. A level below warn also lowers what the local files keep, so at info they get ordinary task output and service lines as well, but never progress bars.

For troubleshooting, debug mode puts every log record with all its metadata in the files, including ordinary task output, and sends OvalBee that same full log, warnings, events, user ids and addresses included, with nothing masked. It stays on until you turn it off.

On Kubernetes the chart’s logging values cover all of this: debug, files.retentionDays, remote.enabled, and vector.env for HTTPS_PROXY and NO_PROXY behind a proxy. With Docker Compose, put the logger’s settings in a docker-compose.override.yml next to docker-compose.yml, which Compose reads on every docker compose up:

services:
  logger:
    environment:
      - OVALBEE_LOGS_REMOTE_KEY=<key from the OvalBee team, optional>
      - OVALBEE_LOGS_DEBUG=true
      - HTTPS_PROXY=http://proxy.example.com:3128

OVALBEE_LOGS_RETENTION_DAYS changes how long the files are kept, and NO_PROXY lists the hosts to reach without the proxy. The logger never reaches the API through the proxy.

Sending logs to your own storage

The logger runs Vector, and you can add any Vector sink to it: S3, CloudWatch, Elasticsearch, Splunk, another Loki and more. A sink reads one of two streams by name:

• diagnostic_logs: the diagnostic log described above, exactly as the files and OvalBee get it, including everything when debug mode is on.
• task_logs: what the platform receives about tasks and deployments: their output, as shown in the log panels, plus the few lifecycle events that move their status: waiting for a machine, pulling the image, started, finished, and failures such as an image that can’t be downloaded, running out of memory or eviction.

Put the sink in a .toml, .yaml or .json file. The file holds sinks only, each reading diagnostic_logs or task_logs: no sources, transforms or global options. Every sink needs buffer.when_full = "drop_newest" and must leave acknowledgements off, so that a destination that can’t keep up loses records instead of holding up task logs. The logger loads the file next to its own configuration and checks both when it starts. If a file breaks any of these rules or doesn’t validate, the logger starts without any of your files and names the file and the reason in its log, so task logs keep flowing. Credentials come from the logger’s environment: the AWS sinks read AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY themselves, and any setting can name a variable as ${NAME}.

Some problems only show once Vector runs your sink, such as a port another component already uses. If Vector exits while your files are loaded, the logger restarts it without them and says so in its log. Save the fixed file to load it again.

This file sends the diagnostic log to S3:

[sinks.s3]
type = "aws_s3"
inputs = ["diagnostic_logs"]
bucket = "my-ovalbee-logs"
region = "eu-central-1"
key_prefix = "ovalbee/%Y-%m-%d/"
compression = "gzip"
encoding.codec = "json"
buffer.when_full = "drop_newest"

With Docker Compose, save it as vector.d/s3.toml next to docker-compose.yml and mount the folder in docker-compose.override.yml:

services:
  logger:
    volumes:
      - ./vector.d:/etc/ovalbee/vector.d:ro
    environment:
      - AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
      - AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}

On Kubernetes, pass the same file with --set-file 'logging.vector.extraConfig.s3\.toml=s3.toml', and the credentials from a Secret through logging.vector.env:

logging:
  vector:
    env:
      - name: AWS_ACCESS_KEY_ID
        valueFrom:
          secretKeyRef: {name: ovalbee-logs-s3, key: AWS_ACCESS_KEY_ID}
      - name: AWS_SECRET_ACCESS_KEY
        valueFrom:
          secretKeyRef: {name: ovalbee-logs-s3, key: AWS_SECRET_ACCESS_KEY}

Behind a proxy, the HTTPS_PROXY set for the logger applies to your sinks too.

HTTPS

OvalBee itself serves plain HTTP: with Docker Compose its router listens on OVALBEE_INGRESS_PORT, and on Kubernetes traffic reaches it through your ingress controller. Terminate TLS in front of it, at your reverse proxy, load balancer or ingress, and redirect HTTP to HTTPS there.

Once the site answers on its HTTPS address:

1. Under Admin → Configuration → General, set Public URL to the https:// address.
2. Under Authentication, turn on Secure cookies. Do it only after HTTPS works: browsers drop secure cookies on a plain HTTP address, and people are signed out as soon as their sign-in needs refreshing.

OvalBee doesn’t add browser security headers to its responses, so set them where TLS terminates:

A strict Content-Security-Policy isn’t on the list: the interface loads some content from other hosts, such as dataset previews from Dataset Ninja, and a policy that doesn’t allow them breaks those parts.

Monitoring

Two addresses report the platform’s state:

• /api/health answers as long as the API is serving. The Helm chart’s liveness and readiness probes use it.
• /api/health/dependencies also checks PostgreSQL, Redis, object storage and Qdrant, and answers 503 with the status of each when one of them doesn’t respond within 2 seconds. Point your monitoring and alerts at it.

Don’t use /api/health/dependencies for a liveness or readiness probe or for a container health check: a short outage of one dependency would then restart the API or take it out of routing while it still works.

Command-line administration

The API container includes ovalbee-admin, a command-line tool for the tasks you may need before you can sign in, or without the web app at all. Run it inside the API container:

# Docker Compose, from the installation directory
docker compose exec -T api ovalbee-admin status
# Kubernetes
kubectl -n <namespace> exec -i deploy/<release>-api -- ovalbee-admin status

Passwords, licenses and secrets are read from standard input, never from the command line, so they don’t appear in your shell history or the process list. Pipe them in from a file, and keep the -T flag (with kubectl exec, -i without -t) so the input can end:

docker compose exec -T api ovalbee-admin admin set-password < new-password.txt
docker compose exec -T api ovalbee-admin license set < license.txt

Each command prints a single JSON document and exits with 0 on success, 3 when there was nothing to change, 4 when the input was rejected, 5 when the platform isn’t ready yet (safe to retry) and 1 on any other error, so it can be used from scripts.

Changing the application secret

OVALBEE_APP_SECRET encrypts the credentials OvalBee stores: secret platform settings such as object storage keys, SSO client secrets and workspace secrets. If it changes, the API still starts, but settings it can’t decrypt silently fall back to their defaults and the rest fail when used. To change it on purpose:

1. Set the new secret and restart the API.
2. Pipe the previous secret into ovalbee-admin secrets rotate, then run ovalbee-admin secrets check to confirm nothing is left.
3. Restart the API again so it reloads its settings.

Everyone is signed out, because sessions are signed with the same secret.

Upgrades

Keep the image tag and chart version under your change control. Before changing either, review the corresponding configuration defaults and preserve durable secrets, storage, and database backups. A Compose update pulls the configured images; a Helm upgrade applies the selected chart and image values. Database migrations run at startup. Each placed node stays on the template version it was added from and picks up changes to that version when its image is updated.

Related pages

• Compute — executor backends and GPU machine classes.
• Platform configuration — settings after installation.
• Sign-in and accounts — connecting your identity provider.