Go to production
The one-command installer gives you a working Orchard instance on one Linux machine. That machine holds everything: the cluster, the disks, the registry, and the backups. Because of this, the installer suits a home lab, a side project, a small team, or a trial of Orchard. For a production setup, Orchard recommends the Helm chart on a multi-node cluster that you build for it, with the bundled components replaced by ones you operate.
This page explains the difference, lists the components you can replace, and then walks through the settings to check before other people depend on your instance.
Choose a layout
Section titled “Choose a layout”The installer builds a single-node k3s cluster. k3s is a certified Kubernetes distribution and is used in production. The limitation is the installer’s single-node layout, not k3s:
- One machine is a single point of failure. If the machine stops, every app, database, and the dashboard stop with it.
- Volumes use
local-path. Each volume is a directory on that machine’s disk, with no replication. - The registry is bound to loopback. The bundled zot registry listens on
127.0.0.1on the install host, and the node pulls tenant images through it. Another machine cannot pull from it, so an installer host cannot take more nodes. Refer to Add and remove nodes. - Backups stay on the same disk. Database backups go to the store on the server, which does not survive the loss of the machine unless you add S3 storage.
| Layout | Suits | You operate |
|---|---|---|
| One-command installer | A home lab, a side project, a small team, or a trial | One Linux machine. The installer and orchardctl manage the rest. |
| Helm chart on a multi-node cluster | Production, and any instance where people depend on uptime and backups | The cluster, its storage, its registry, and its object storage, as you choose. |
For production, run the Helm chart on a multi-node cluster built for it, such as one running Talos Linux. Hack Club runs Orchard this way, on six Talos Linux nodes. Talos is an operating system made only to run Kubernetes, and it offers the gVisor and Kata runtimes that kernel isolation needs as system extensions. For the chart install, refer to Install on an existing cluster.
When you build the cluster, plan for the following:
- A CNI that enforces NetworkPolicy. Orchard recommends Cilium, which Hack Club runs on its Talos nodes. Talos deploys Flannel unless you configure another CNI, and Flannel does not enforce NetworkPolicy. For the supported CNIs, refer to Supported CNIs, and for Cilium on Talos, refer to Deploying Cilium CNI.
- Kubernetes 1.30 or later, for the admission guard. For the distributions, CNIs, ingress controllers, and storage that Orchard supports, refer to Compatibility.
- The components below, either the ones the chart expects or your replacements.
The chart’s defaults describe the cluster that the Orchard project runs itself, with Harbor as the registry and the HAProxy ingress controller in the network namespace. The chart’s own quick install was run end to end on a single node, and Hack Club runs the chart on a six-node cluster. Before you plan a cluster, read which configurations were tested in the chart README.
Replace bundled components
Section titled “Replace bundled components”On a Helm cluster, you decide which components Orchard runs and which ones you bring. The sections below list each bundled component, the replacements that the chart and the server support, and the settings for each. Replace what you already operate, and keep the rest.
Registry
Section titled “Registry”The installer runs zot, bound to loopback on its host. On a cluster, use a registry that every node can pull from, such as Harbor or any other OCI registry.
| Setting | Where | Value |
|---|---|---|
builders.harbor.username, password, hosts |
Chart values | The account that builds push with, and the registry hosts it applies to. |
builders.registryEgress |
Chart values | The registry’s namespace and port, so build pods can reach it. |
builders.httpRegistries |
Chart values | Only for a registry that serves plain HTTP. Anything not listed uses HTTPS. |
DOCKER_REGISTRY_PUSH, DOCKER_REGISTRY_PULL |
server.env |
The address builds push to, and the address nodes pull from. |
REGISTRY_PROJECT |
server.env |
The project or path prefix that holds every tenant image. The default is orchard. |
HARBOR_USERNAME, HARBOR_PASSWORD |
server.env |
The same account. The server uses it to delete old images and to read scan results. |
| Registry credentials | Every node runtime | Orchard does not add a pull secret to app pods, so each node needs its own credentials. |
Use a low-privilege robot account, not the Harbor admin account, because every tenant’s builds share this login. When the registry is Harbor, Orchard detects it through the Harbor API and can use Harbor’s own scanner. For the steps, refer to Turn on image builds, Use your own registry, and Set up Harbor.
Object storage
Section titled “Object storage”The installer runs an S3-compatible store inside the cluster (objectStorage.local.enabled: true). The chart leaves it off. Use external S3-compatible storage, such as Amazon S3 or Cloudflare R2, on infrastructure separate from the cluster.
- Leave
objectStorage.local.enabledatfalse. - Add the bucket at Instance admin > Configuration > Object storage. No chart change is needed.
- To manage storage from your values file instead, set
r2Proxy.upstream.*withr2Proxy.defaultBucketorr2Proxy.allowedBuckets, and theR2_*variables inserver.env.
For the settings for each provider, refer to Add S3 storage.
Storage class
Section titled “Storage class”The installer uses local-path, which keeps each volume on one node’s disk. On a cluster, use a replicated StorageClass, so that a volume survives the loss of a node.
tenant.storageClass(K8S_STORAGE_CLASS) sets the class for tenant app volumes and databases. Empty uses the cluster default.- Each platform volume has its own
storageClasskey, such aspostgres.storage.storageClassandredis.storageClass. Empty uses the cluster default. The simplest option is to make your replicated class the cluster default. admissionPolicies.storageClasseslimits which classes a tenant volume can name.
A StorageClass cannot change on an existing volume, so choose the class before you install. For the full list of volume keys, refer to the chart README.
Ingress controller
Section titled “Ingress controller”Tenant apps use standard Kubernetes Ingress objects, so they route through the controller you name in tenant.ingressClass (K8S_INGRESS_CLASS). Request, error, and latency charts and access-log statistics come only from two controllers: Traefik and the HAProxy Kubernetes Ingress Controller. With another controller, apps still route, but those charts stay empty.
| Setting | Value |
|---|---|
tenant.ingressClass |
Your ingress class. Empty falls back to haproxy. |
tenant.ingressNamespaces |
The controller’s namespace, then orchard. Refer to Let the ingress controller reach apps. |
metrics.ingress.traefik.* or metrics.ingress.haproxy.* |
The namespace and pod labels of your controller. Refer to Point app metrics at your ingress controller. |
INGRESS_METRICS |
traefik or haproxy, to force one adapter. Empty detects it for each app. |
The chart routes the dashboard and the MCP endpoint with Traefik IngressRoute objects. Without Traefik, set ingress.enabled and mcp.ingress.enabled to false, and route the dashboard hostname yourself: /apiv1 to the server Service on port 3001, and everything else to the frontend Service on port 3000.
The chart runs one Redis pod. To use a Redis that you operate instead, set redis.enabled: false and add REDIS_URL to server.env. If you set redis.url and leave redis.enabled on, the server uses your URL, but the chart still deploys its own Redis pod.
Control-plane Postgres
Section titled “Control-plane Postgres”The chart creates the control-plane database as a CloudNativePG Cluster with one instance. Install the CloudNativePG operator first, and set tenant.cnpgOperatorNamespace to its namespace.
| Setting | Value |
|---|---|
postgres.instances |
More than 1 adds replicas, and CloudNativePG fails over to one if the primary is lost. |
postgres.storage.storageClass |
Your replicated class. Empty uses the cluster default. |
postgres.pooler.enabled |
true puts PgBouncer in front of the database, which caps connections from many server pods. |
postgres.backup.* |
Continuous backups to S3. Off by default. Refer to step 5. |
Metrics and logs
Section titled “Metrics and logs”The metrics and log stores (VictoriaMetrics, vmagent, VictoriaLogs, and Vector) are part of the chart and cannot be replaced with your own monitoring stack. You can resize them, or turn them off with metrics.enabled: false and logs.enabled: false. Off, the dashboard has no charts or log history, and everything else keeps working. For sizes, refer to Scale the control plane.
To send server traces and profiles to Datadog, set datadog.enabled: true. This is in addition to the bundled stack, not a replacement for it.
1. Check your secrets
Section titled “1. Check your secrets”The installer generates secrets. An instance assembled by hand may still use example values.
| Variable | Purpose |
|---|---|
JWT_SECRET |
Signs sessions. Anyone who knows this value can create a session. |
SECRETS_ENCRYPTION_KEY |
Encrypts stored secrets. Orchard cannot save secrets without it. |
Store both in a Kubernetes Secret. Do not store them in a values file in a repository.
2. Close signups
Section titled “2. Close signups”The default for the chart and the installer is invite_only. Claiming the instance works in every mode.
signupMode: invite_only # or: closedWith open on a public URL, anyone who finds the URL can create an account and run workloads on your hardware.
3. Check your superadmins
Section titled “3. Check your superadmins”A new instance has a single-use setup token that is valid for 24 hours. Redeem it with the account that should own the instance. For more information, refer to Claim the instance.
After you claim the instance, go to Instance admin > Users and confirm that the list of superadmins is correct.
4. Check TLS settings
Section titled “4. Check TLS settings”Set the following server variables:
SESSION_COOKIE_SECURE=trueTRUST_PROXY=1SESSION_COOKIE_SECURE=truestops browsers from sending session cookies over plain HTTP.TRUST_PROXYis the number of proxy hops to trust when Orchard finds the client IP address, or a comma-separated list of proxy CIDRs. The chart sets it to1, for a single ingress in front of the API. Unset, the server trusts no forwarded header. A value that is too high lets a client spoof its IP address with a header, which weakens per-IP rate limits. A private address is never trusted only because it is private.
Behind a Cloudflare Tunnel, also set CLIENT_IP_HEADER=cf-connecting-ip and set CLIENT_IP_HEADER_FROM to the pod CIDR. The installer does this in tunnel mode. For more information, refer to Request headers.
5. Back up the control-plane database
Section titled “5. Back up the control-plane database”The control-plane database holds every organization, project, app definition, environment variable, and audit record. Tenant database backups do not include it.
A new installer host also writes a daily pg_dump of this database to the object store on the same machine. That dump undoes a mistake, but it shares a disk with the database, so it does not protect you if the machine is lost. For dumps to another location, refer to Schedule database dumps.
Continuous backups need S3-compatible storage on another machine.
-
Create a bucket, and a key that can read, write, and delete in it.
-
Add the key to the namespace that Orchard runs in. The chart reads the names
ACCESS_KEY_IDandACCESS_SECRET_KEY.kubectl -n orchard create secret generic orchard-db-backup \--from-literal=ACCESS_KEY_ID=<KEY_ID> \--from-literal=ACCESS_SECRET_KEY=<SECRET> -
Set the chart values. On an installer host, run the following commands:
sudo orchardctl config set postgres.backup.destinationPath s3://my-bucket/orchard-dbsudo orchardctl config set postgres.backup.endpointURL https://s3.example.comsudo orchardctl config set postgres.backup.enabled true --applyOn a Helm install, add the following to your values file:
postgres:backup:enabled: truedestinationPath: s3://my-bucket/orchard-dbendpointURL: https://s3.example.com # leave out for AWS S3secretName: orchard-db-backup # the defaultretentionPolicy: 30d # the defaultschedule: '0 30 1 * * *' # the default, six fields, with seconds -
Check that the first base backup finished and that WAL is arriving. The first backup starts when the
ScheduledBackupexists.kubectl -n orchard get backups.postgresql.cnpg.iokubectl -n orchard get clusters.postgresql.cnpg.io orchard-db \-o jsonpath='{.status.firstRecoverabilityPoint}{"\n"}{.status.conditions[?(@.type=="ContinuousArchiving")].status}{"\n"}' -
To test a restore, create a scratch namespace and copy the Secret into it.
-
Apply the following cluster, then check the data:
apiVersion: postgresql.cnpg.io/v1kind: Clustermetadata:name: orchard-db-restorenamespace: orchard-restore-testspec:instances: 1storage:size: 10Gibootstrap:recovery:source: orchard-dbexternalClusters:- name: orchard-dbbarmanObjectStore:serverName: orchard-dbdestinationPath: s3://my-bucket/orchard-dbendpointURL: https://s3.example.coms3Credentials:accessKeyId:name: orchard-db-backupkey: ACCESS_KEY_IDsecretAccessKey:name: orchard-db-backupkey: ACCESS_SECRET_KEY
If the barman cloud plugin is already installed in the cluster, you can set postgres.backup.method to plugin. The restore example applies to the default method.
A restore test proves that the backup is readable. Bringing a whole instance back on a new machine takes more steps, and Orchard has not tested that procedure end to end. For the steps, refer to Back up and restore an instance.
6. Configure object storage for tenant backups
Section titled “6. Configure object storage for tenant backups”Managed databases need a backup bucket. An installer host uses the store on the server by default, which shares a disk with the databases.
To use S3 or R2, go to Instance admin > Configuration > Object storage. An organization can also set its own bucket at Organization settings > Backups. For more information, refer to Add S3 storage.
7. Choose your isolation settings
Section titled “7. Choose your isolation settings”If anyone other than you can deploy, read Security and isolation. Network isolation is on by default. Kernel isolation is off. To run untrusted code, install gVisor and Kata, and turn on tenantSandbox and builderSandbox.
8. Set quotas
Section titled “8. Set quotas”Set organization caps and member allowances before people start to deploy. For more information, refer to Set quotas.
9. Set up email
Section titled “9. Set up email”Set SMTP_* so that sign-in codes, invitations, and notifications arrive. For more information, refer to Set up email.
Without email, Orchard hides email codes and a new member can only sign in through an identity provider. For an existing account, an operator can run sudo orchardctl login --email <ADDRESS> to print a one-time link.
10. Use a wildcard certificate
Section titled “10. Use a wildcard certificate”By default, Orchard issues a certificate for each hostname over HTTP-01, so each new app waits for an ACME request. A wildcard certificate covers every generated hostname at once, and needs a DNS-01 issuer. For more information, refer to Set up wildcard certificates.
11. Pin your version
Section titled “11. Pin your version”An unpinned instance moves to the newest release on every update. On an installer host with automatic updates, that happens every night. Pin a version, read the release notes, and upgrade when you decide to.
On a Helm install, pass --version on every upgrade:
helm upgrade orchard oci://ghcr.io/hackclub/orchard/charts/orchard \ --version <VERSION> -n orchard -f my-values.yaml --server-side=falseOn Helm 3, leave out --server-side=false. A published chart sets the server, frontend, MCP, and backup proxy images to the versions in that release, so pinning the chart pins them too. Do not set server.image.tag or frontend.image.tag to latest in your values.
On an installer host, refer to Pin a version on an installer host.
Release notes are on GitHub Releases. Each service has its own release, tagged <service>/v<version>, for example server/v2.95.4. The chart has no release page of its own. Its version is the highest version among the services it ships, so the release with that number names the newest change.
Verify the configuration
Section titled “Verify the configuration”Run the following commands:
kubectl -n orchard exec deploy/server -- printenv SIGNUP_MODEkubectl get runtimeclasskubectl -n orchard get scheduledbackups.postgresql.cnpg.iohelm list -n orchard- The first command prints
invite_onlyorclosed. - If the second command prints nothing and other people can deploy, return to Security and isolation.
- The third command lists a
ScheduledBackup. If it prints nothing, control-plane backups are off. Return to step 5. - The last command shows the chart version you run.
You now have an instance that you can hand to other people. To turn on the optional parts of the setup, refer to Finish instance setup.
Related resources
Section titled “Related resources”- Upgrade Orchard - Upgrade an installer host or a Helm install, and pin a version.
- Back up and restore an instance - Learn what to back up and how to bring back the control plane.
- Manage nodes and pools - Add nodes to a cluster and control where workloads run.