300 lines
14 KiB
Markdown
300 lines
14 KiB
Markdown
# ☸ Kubernetes — architektura, platformy, Cluster API
|
|
|
|
## Přehled
|
|
|
|
Kubernetes (K8s) je open-source orchestrátor kontejnerů — de facto standard pro nasazování, škálování a správu containerizovaných aplikací. Postaven na modelu deklarativní konfigurace a control loopů (reconciliation).
|
|
|
|
## Způsoby nasazení Kubernetes
|
|
|
|
| Metoda | Popis | Správa control plane | Vhodné pro |
|
|
|--------|-------|---------------------|------------|
|
|
| **kubeadm** | Oficiální nástroj pro bootstrap K8s clusteru | Self-managed (stacked/external etcd) | On-prem, lab, learning |
|
|
| **K3s** | Lightweight K8s (Rancher), single binary, embedded etcd/SQLite | Self-managed | Edge, IoT, low-resource, HA s embedded etcd |
|
|
| **RKE2** | Rancher Kubernetes Engine 2, CIS-hardened, FIPS-ready | Self-managed | Enterprise on-prem, air-gapped, regulatory |
|
|
| **OpenShift** | Red Hat enterprise K8s + operator lifecycle + SDN + routing | Self-managed (RHCOS) | Enterprise, multicluster, platform engineering |
|
|
| **Vanilla K8s (CAPI)** | Cluster API — deklarativní provisioning a lifecycle management | Self-managed (CAPI managed) | Fleet management, GitOps, multi-provider |
|
|
| **EKS** (AWS) | Managed K8s | AWS managed | AWS cloud-native, nejméně ops |
|
|
| **AKS** (Azure) | Managed K8s | Azure managed | Azure cloud-native |
|
|
| **GKE** (GCP) | Managed K8s, auto-pilot, autopilot modes | GCP managed | GCP cloud-native |
|
|
| **SKE** (Sangfor) | Managed K8s on Sangfor HCI | Vendor managed | Sangfor HCI ekosystém |
|
|
|
|
---
|
|
|
|
## Cluster API (CAPI)
|
|
|
|
### Co je Cluster API
|
|
|
|
Cluster API je Kubernetes sub-projekt (SIG Cluster-Lifecycle), který přináší deklarativní API pro provisioning, upgrade a operace Kubernetes clusterů. Místo Terraform skriptů nebo manuálního `kubeadm` definujete cluster jako Kubernetes Custom Resources — `Cluster`, `Machine`, `MachineDeployment` atd.
|
|
|
|
Princip: **Kubernetes cluster, který spravuje Kubernetes clustery.**
|
|
|
|
### Architektura
|
|
|
|
```
|
|
┌─────────────────────────────────────────┐
|
|
│ Management Cluster │
|
|
│ │
|
|
│ ┌──────────────────────────────────┐ │
|
|
│ │ CAPI Controllers │ │
|
|
│ │ ┌──────┐ ┌──────┐ ┌─────────┐ │ │
|
|
│ │ │ Infra│ │Bootstrap│ │Control │ │ │
|
|
│ │ │ Prov │ │ Prov │ │Plane Pr │ │ │
|
|
│ │ └──────┘ └──────┘ └─────────┘ │ │
|
|
│ └──────────────────────────────────┘ │
|
|
│ │
|
|
│ CR: Cluster, Machine, MachineDeployment│
|
|
│ ... │
|
|
└────────────────┬────────────────────────┘
|
|
│ CAPI controller
|
|
│ vytváří / spravuje
|
|
┌────────┴────────┐
|
|
▼ ▼
|
|
┌───────────────┐ ┌───────────────┐
|
|
│ Workload │ │ Workload │
|
|
│ Cluster (dev) │ │ Cluster (prod)│
|
|
│ ┌───┐ ┌───┐ │ │ ┌───┐ ┌───┐ │
|
|
│ │ CP│ │ W │ │ │ │ CP│ │ W │ │
|
|
│ └───┘ └───┘ │ │ └───┘ └───┘ │
|
|
└───────────────┘ └───────────────┘
|
|
```
|
|
|
|
- **Management cluster** — Kubernetes cluster, kde běží CAPI controllery. Může to být vyhrazený "admin" cluster (často velmi malý).
|
|
- **Workload (managed) cluster** — Kubernetes clustery, které CAPI spravuje. Každý je reprezentován jako CRD v management clusteru.
|
|
- **Machine** — abstrakce compute jednotky (VM, bare metal), která se stane K8s uzlem.
|
|
|
|
### Klíčové CRD (Custom Resource Definitions)
|
|
|
|
| CRD | API skupina | Účel |
|
|
|-----|------------|------|
|
|
| **Cluster** | `cluster.x-k8s.io` | Reprezentace clusteru (infra reference, control plane ref, networking) |
|
|
| **Machine** | `cluster.x-k8s.io` | Jednotlivý uzel (VM/BM instance) |
|
|
| **MachineDeployment** | `cluster.x-k8s.io` | Deklarativní škálování a rolling update workerů |
|
|
| **MachineSet** | `cluster.x-k8s.io` | Replica set pro Machiny (lower-level) |
|
|
| **MachineHealthCheck** | `cluster.x-k8s.io` | Auto-remediaci (automatické nahrazení unhealthy uzlu) |
|
|
| **ClusterClass** | `cluster.x-k8s.io` | Šablona pro vytváření clusterů |
|
|
| **KubeadmControlPlane** | `controlplane.cluster.x-k8s.io` | Control plane managed kubeadm (stacked/external etcd) |
|
|
| **KubeadmConfig / KubeadmConfigTemplate** | `bootstrap.cluster.x-k8s.io` | Bootstrap konfigurace (kubeadm init/join) |
|
|
|
|
### Provider model
|
|
|
|
CAPI používá třívrstvý provider model:
|
|
|
|
#### 1. Infrastructure Provider
|
|
Vytváří a spravuje infrastrukturu (VM, sítě, LB, storage).
|
|
|
|
| Provider | Platforma | Status |
|
|
|----------|-----------|--------|
|
|
| **AWS (CAPA)** | AWS EC2, VPC, ELB, EKS | Stable, SIG-sponsored |
|
|
| **Azure (CAPZ)** | Azure VM, VNet, LB, AKS | Stable, SIG-sponsored |
|
|
| **GCP (CAPG)** | GCP Compute, VPC, GKE | Beta |
|
|
| **vSphere (CAPV)** | VMware vSphere | Stable |
|
|
| **OpenStack (CAPO)** | OpenStack compute/network | Stable |
|
|
| **Metal3** | Bare metal (Ironic) | Stable |
|
|
| **Docker (CAPD)** | Docker containers (development) | Tilt/Dev only |
|
|
| **Akamai (Linode)** | Linode | Community |
|
|
| **Azure Stack HCI** | Azure Stack HCI | Community |
|
|
| **cloudscale** | cloudscale.ch | Community |
|
|
| **Exoscale** | Exoscale | Community |
|
|
| **IBM Cloud** | IBM Cloud | Community |
|
|
| **Equinix Metal** | Equinix (ex Packet) | Community |
|
|
| **Hetzner** | Hetzner Cloud | Community |
|
|
| **OpenNebula** | OpenNebula | Community |
|
|
|
|
#### 2. Bootstrap Provider
|
|
Zajišťuje inicializaci K8s na node (kubeadm init/join, TLS certs, tokeny).
|
|
|
|
| Provider | Popis |
|
|
|----------|-------|
|
|
| **Kubeadm** (vestavěný) | Standardní kubeadm init/join, podpora stacked/external etcd |
|
|
| **EKS** | Bootstrap pro EKS managed control plane (AWS) |
|
|
| **K3s** | Lightweight K8s bootstrap (edge, IoT) |
|
|
| **RKE2** | Rancher K8s bootstrap, CIS-hardened |
|
|
| **Talos** | API-driven bootstrap (Sidero Labs), immutable OS |
|
|
| **k0smotron** | K0s-based bootstrap + hosted control plane |
|
|
| **MicroK8s** | Canonical MicroK8s bootstrap |
|
|
| **Canonical Kubernetes** | Canonical K8s (snap-based) |
|
|
|
|
#### 3. Control Plane Provider
|
|
Spravuje control plane uzly.
|
|
|
|
| Provider | Popis |
|
|
|----------|-------|
|
|
| **KubeadmControlPlane** (vestavěný) | Kubeadm-managed CP, stacked/external etcd |
|
|
| **EKS** | AWS EKS managed control plane |
|
|
| **Kamaji** | Hosted control plane (CP běží jako deployment v management clusteru) |
|
|
| **K3s** | K3s control plane (edge-optimized) |
|
|
| **RKE2** | RKE2 control plane |
|
|
| **Talos** | Talos control plane, API-based management |
|
|
| **k0smotron** | Hosted control plane (k0s-based) |
|
|
| **Nested** | Nested virtualization control plane |
|
|
|
|
### ClusterClass a Managed Topologies
|
|
|
|
ClusterClass (stabilní od CAPI v1beta1, CAPI v1.0+) umožňuje definovat **šablonu clusteru**:
|
|
|
|
```yaml
|
|
apiVersion: cluster.x-k8s.io/v1beta1
|
|
kind: ClusterClass
|
|
metadata:
|
|
name: standard-aws-cluster
|
|
spec:
|
|
controlPlane:
|
|
ref:
|
|
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
|
|
kind: KubeadmControlPlaneTemplate
|
|
name: aws-cp-tmpl
|
|
machineInfrastructure:
|
|
ref:
|
|
kind: AWSMachineTemplate
|
|
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
|
|
name: aws-cp-machine-tmpl
|
|
workers:
|
|
machineDeployments:
|
|
- class: default-worker
|
|
template:
|
|
bootstrap:
|
|
ref:
|
|
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
|
|
kind: KubeadmConfigTemplate
|
|
name: aws-worker-bootstrap-tmpl
|
|
infrastructure:
|
|
ref:
|
|
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
|
|
kind: AWSMachineTemplate
|
|
name: aws-worker-machine-tmpl
|
|
variables:
|
|
- name: instanceType
|
|
required: true
|
|
schema:
|
|
openAPIV3Schema:
|
|
type: string
|
|
enum: ["t3.large", "m5.large", "m5.xlarge"]
|
|
```
|
|
|
|
Pak lze vytvořit cluster s přetížením proměnných:
|
|
|
|
```yaml
|
|
apiVersion: cluster.x-k8s.io/v1beta1
|
|
kind: Cluster
|
|
metadata:
|
|
name: dev-team-alpha
|
|
namespace: clusters
|
|
spec:
|
|
topology:
|
|
class: standard-aws-cluster
|
|
version: v1.30.2
|
|
controlPlane:
|
|
replicas: 1
|
|
workers:
|
|
machineDeployments:
|
|
- class: default-worker
|
|
name: md-0
|
|
replicas: 2
|
|
variables:
|
|
- name: instanceType
|
|
value: "m5.xlarge"
|
|
```
|
|
|
|
### Životní cyklus clusteru s CAPI
|
|
|
|
| Fáze | Akce | CAPI mechanismus |
|
|
|------|------|------------------|
|
|
| **Create** | `kubectl apply -f cluster.yaml` | Controller vytvoří infra (VM, network), provede bootstrap kubeadm init/join |
|
|
| **Scale** | Upravit `replicas` v MachineDeployment | Controller vytvoří/odstraní Machine → VM → node join/drain |
|
|
| **Upgrade** | Změnit `version` v KubeadmControlPlane / MachineDeployment | Rolling update: nový CP node → upgrade → starý drain a delete. Workers: MachineDeployment rolling update |
|
|
| **Health check** | MachineHealthCheck | Pokud node unhealthy > timeout, controller vytvoří náhradní Machine |
|
|
| **Delete** | `kubectl delete cluster` | Controller provede drain, delete VMs, cleanup infrastruktury |
|
|
| **Template update** | Změna AWSMachineTemplate / KubeadmConfigTemplate | Stroj se vytvoří s novou šablonou; stávající Machiny se dotýká jen přes rolling update |
|
|
|
|
### Auto-remediace (MachineHealthCheck)
|
|
|
|
```yaml
|
|
apiVersion: cluster.x-k8s.io/v1beta1
|
|
kind: MachineHealthCheck
|
|
metadata:
|
|
name: prod-mhc
|
|
namespace: clusters
|
|
spec:
|
|
clusterName: prod-us-east
|
|
selector:
|
|
matchLabels:
|
|
cluster.x-k8s.io/deployment-name: prod-us-east-workers
|
|
unhealthyConditions:
|
|
- type: Ready
|
|
status: "False"
|
|
timeout: 5m
|
|
- type: Ready
|
|
status: Unknown
|
|
timeout: 5m
|
|
maxUnhealthy: "40%"
|
|
nodeStartupTimeout: 10m
|
|
```
|
|
|
|
### CAPI + GitOps
|
|
|
|
CAPI se přirozeně integruje s GitOps:
|
|
|
|
- **ArgoCD** — Cluster a MachineDeployment manifesty v Git repozitáři, ArgoCD je aplikuje na management cluster
|
|
- **Flux** — `Kustomization` + `OCIRepository` pro CAPI objekty
|
|
- **Crossplane** — lze kombinovat: Crossplane pro provisioning cloud resources (VPC, subnets), CAPI pro K8s cluster na nich
|
|
|
|
Vzor: vyhrazený "fleet management" cluster, na kterém běží CAPI + ArgoCD. Všechny workload clustery jsou definované jako YAML v Gitu.
|
|
|
|
### CAPI pro on-prem
|
|
|
|
| Provider | Use case | Poznámka |
|
|
|----------|----------|----------|
|
|
| **Metal3** (Ironic) | Bare metal provisioning (PXE, IPMI, Redfish) | Automatické provisionování BM serverů jako K8s nodes |
|
|
| **CAPV (vSphere)** | VMware VM jako K8s nodes | Většina enterprise on-prem |
|
|
| **CAPO (OpenStack)** | OpenStack VM jako K8s nodes | OpenStack-native |
|
|
| **Nutanix (CAPNX)** | Nutanix AHV/Prism | Community provider |
|
|
|
|
### CAPI pro edge
|
|
|
|
| Provider | Use case | Poznámka |
|
|
|----------|----------|----------|
|
|
| **K3s bootstrap + control plane** | Lightweight K8s na edge zařízeních | Single binary, SQLite/embedded etcd |
|
|
| **RKE2 bootstrap + control plane** | Enterprise edge, air-gapped | CIS-hardened, FIPS |
|
|
| **Talos** | Immutable OS, API-driven | Minimal footprint, no SSH |
|
|
| **k0smotron** | Hosted control plane pro edge clustery | CP běží v management clusteru, worker na edge |
|
|
|
|
### CAPI vs alternativy
|
|
|
|
| Nástroj | Přístup | CAPI výhoda | CAPI nevýhoda |
|
|
|---------|---------|-------------|---------------|
|
|
| **Terraform/Pulumi** | Imperativní/declarativní IaC | CAPI je K8s-native — stejný nástroj pro appky i clustery; GitOps ready | Terraform má širší podporu non-K8s resources |
|
|
| **kubeadm** | Manuální nebo skriptovaný | CAPI automatizuje celý lifecycle včetně upgradů a remediací | Vyšší komplexita, nutný management cluster |
|
|
| **Rancher** | Web UI + API pro správu K8s clusterů | CAPI je open-source, vendor-neutral | Rancher má GUI, monitoring, katalog appek |
|
|
| **OpenShift Hive/ACM** | Red Hat Advanced Cluster Management | CAPI je standardní (SIG) — širší provider ecosystem | ACM má governance, policy, compliance |
|
|
|
|
### Limitations a maturity
|
|
|
|
- **Management cluster je SPOF** — musí mít vlastní HA a backup (etcd zálohy, certifikáty)
|
|
- **CAPI není cluster autoscaler** — řeší lifecycle clusterů, ne auto-scaling podů v rámci clusteru (používá se Cluster Autoscaler samostatně)
|
|
- **Provider maturity se liší** — AWS/Azure/vSphere stabilní, GCP/OpenStack beta, některé community providers alpha
|
|
- **etcd backup není built-in** — nutné řešit externě (Velero, etcd snapshot)
|
|
- **CAPI neřeší aplikace** — pouze lifecycle K8s clusterů (monitoring, logging, ingress si řídí uživatel)
|
|
- **Learning curve** — nutnost management clusteru, pochopení provider modelu, CRDs
|
|
- **CAPI v1.13+ (2026)** — stable release, v1beta1 API je GA, ClusterClass stable, EKS/AKS/GKE managed control plane podpora
|
|
|
|
### Doporučený stack pro CAPI v produkci
|
|
|
|
| Komponenta | Doporučení |
|
|
|------------|------------|
|
|
| **Management cluster** | K3s (malý footprint) nebo kubeadm (3 nodes HA) |
|
|
| **Infra provider** | CAPA (AWS) / CAPV (vSphere) / CAPO (OpenStack) — dle platformy |
|
|
| **Bootstrap/CP provider** | Kubeadm nebo RKE2 |
|
|
| **GitOps** | ArgoCD nebo Flux |
|
|
| **Backup** | Velero + restic/Ceph |
|
|
| **Cluster autoscaler** | Cluster Autoscaler (přes CAPI integration) |
|
|
| **Network** | Cilium (CAPI-native, podpora) |
|
|
| **Secrets** | External Secrets Operator / Sealed Secrets |
|
|
| **Monitoring** | Prometheus + Grafana (kube-prometheus-stack) |
|
|
| **Ingress** | ingress-nginx / Kong / Traefik |
|
|
|
|
## Zdroje
|
|
|
|
Odkazy, knihy a standardy: [sources/infrastructure/sources.md](sources/infrastructure/sources.md)
|
|
|
|
*Poslední revize: 2026-06-18*
|