Kubernetes guide
Multi-cluster and on-premises Kubernetes
One well-run cluster is better than five neglected ones. More clusters make sense for clear reasons: separating production, meeting data rules, or running close to users. When you do need several, manage them as one fleet.
Start withAs few clusters as possible
Split forEnvironment, region, rules
Manage asA fleet
On-premOpenStack or bare metal
When to add a cluster
| Reason | Example | Alternative to consider first |
|---|---|---|
| Separate production | Production and non-production on different clusters | Usually worth it |
| Data rules | Customer data must stay in a specific location | Usually worth it |
| Regions or sites | Users in two regions, or disaster recovery | Usually worth it |
| Each team wants its own | Ten teams, ten clusters | Namespaces with quotas and policies on a shared cluster |
| Different Kubernetes versions | One application cannot upgrade yet | Fix the application; old clusters become a risk |
Managing a fleet
- Build every cluster from the same code (Cluster API, Terraform or your distribution's tooling).
- Apply the same baseline policies and add-ons to every cluster from Git, for example with Argo CD ApplicationSets.
- Send metrics and logs from all clusters to one place.
- Upgrade the fleet in waves, with the same routine every time.
Kubernetes on-premises
| Topic | What to plan |
|---|---|
| On OpenStack | Clusters created by Cluster API or Magnum, using Cinder for volumes and Octavia for load balancers. See private cloud on OpenStack. |
| On bare metal | Best performance and GPU access; needs MetalLB or similar for load balancers and a plan for replacing failed hardware. |
| Storage | Rook-Ceph or a vendor CSI driver for persistent volumes. |
| Backup | Velero for cluster resources and volumes, with copies off the cluster. Disaster recovery for the platform: see DR. |
More Kubernetes guides: Platform design · GitOps delivery · Security · Kubernetes FAQ · Use case: festive-season scale
Planning a Kubernetes platform, or fixing one?
Tell us how many applications and teams you have, how you deploy today and where it should run. We will come back with an honest view: whether Kubernetes fits, what a lean platform looks like, and what it takes to run.