18.6.2026
This commit is contained in:
Stanislav Hubacek
299
KUBERNETES.md
Normal file
299
KUBERNETES.md
Normal file
@@ -0,0 +1,299 @@
|
||||
# ☸ Kubernetes — architektura, platformy, Cluster API
|
||||
|
||||
## Přehled
|
||||
|
||||
Kubernetes (K8s) je open-source orchestrátor kontejnerů — de facto standard pro nasazování, škálování a správu containerizovaných aplikací. Postaven na modelu deklarativní konfigurace a control loopů (reconciliation).
|
||||
|
||||
## Způsoby nasazení Kubernetes
|
||||
|
||||
| Metoda | Popis | Správa control plane | Vhodné pro |
|
||||
|--------|-------|---------------------|------------|
|
||||
| **kubeadm** | Oficiální nástroj pro bootstrap K8s clusteru | Self-managed (stacked/external etcd) | On-prem, lab, learning |
|
||||
| **K3s** | Lightweight K8s (Rancher), single binary, embedded etcd/SQLite | Self-managed | Edge, IoT, low-resource, HA s embedded etcd |
|
||||
| **RKE2** | Rancher Kubernetes Engine 2, CIS-hardened, FIPS-ready | Self-managed | Enterprise on-prem, air-gapped, regulatory |
|
||||
| **OpenShift** | Red Hat enterprise K8s + operator lifecycle + SDN + routing | Self-managed (RHCOS) | Enterprise, multicluster, platform engineering |
|
||||
| **Vanilla K8s (CAPI)** | Cluster API — deklarativní provisioning a lifecycle management | Self-managed (CAPI managed) | Fleet management, GitOps, multi-provider |
|
||||
| **EKS** (AWS) | Managed K8s | AWS managed | AWS cloud-native, nejméně ops |
|
||||
| **AKS** (Azure) | Managed K8s | Azure managed | Azure cloud-native |
|
||||
| **GKE** (GCP) | Managed K8s, auto-pilot, autopilot modes | GCP managed | GCP cloud-native |
|
||||
| **SKE** (Sangfor) | Managed K8s on Sangfor HCI | Vendor managed | Sangfor HCI ekosystém |
|
||||
|
||||
---
|
||||
|
||||
## Cluster API (CAPI)
|
||||
|
||||
### Co je Cluster API
|
||||
|
||||
Cluster API je Kubernetes sub-projekt (SIG Cluster-Lifecycle), který přináší deklarativní API pro provisioning, upgrade a operace Kubernetes clusterů. Místo Terraform skriptů nebo manuálního `kubeadm` definujete cluster jako Kubernetes Custom Resources — `Cluster`, `Machine`, `MachineDeployment` atd.
|
||||
|
||||
Princip: **Kubernetes cluster, který spravuje Kubernetes clustery.**
|
||||
|
||||
### Architektura
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────┐
|
||||
│ Management Cluster │
|
||||
│ │
|
||||
│ ┌──────────────────────────────────┐ │
|
||||
│ │ CAPI Controllers │ │
|
||||
│ │ ┌──────┐ ┌──────┐ ┌─────────┐ │ │
|
||||
│ │ │ Infra│ │Bootstrap│ │Control │ │ │
|
||||
│ │ │ Prov │ │ Prov │ │Plane Pr │ │ │
|
||||
│ │ └──────┘ └──────┘ └─────────┘ │ │
|
||||
│ └──────────────────────────────────┘ │
|
||||
│ │
|
||||
│ CR: Cluster, Machine, MachineDeployment│
|
||||
│ ... │
|
||||
└────────────────┬────────────────────────┘
|
||||
│ CAPI controller
|
||||
│ vytváří / spravuje
|
||||
┌────────┴────────┐
|
||||
▼ ▼
|
||||
┌───────────────┐ ┌───────────────┐
|
||||
│ Workload │ │ Workload │
|
||||
│ Cluster (dev) │ │ Cluster (prod)│
|
||||
│ ┌───┐ ┌───┐ │ │ ┌───┐ ┌───┐ │
|
||||
│ │ CP│ │ W │ │ │ │ CP│ │ W │ │
|
||||
│ └───┘ └───┘ │ │ └───┘ └───┘ │
|
||||
└───────────────┘ └───────────────┘
|
||||
```
|
||||
|
||||
- **Management cluster** — Kubernetes cluster, kde běží CAPI controllery. Může to být vyhrazený "admin" cluster (často velmi malý).
|
||||
- **Workload (managed) cluster** — Kubernetes clustery, které CAPI spravuje. Každý je reprezentován jako CRD v management clusteru.
|
||||
- **Machine** — abstrakce compute jednotky (VM, bare metal), která se stane K8s uzlem.
|
||||
|
||||
### Klíčové CRD (Custom Resource Definitions)
|
||||
|
||||
| CRD | API skupina | Účel |
|
||||
|-----|------------|------|
|
||||
| **Cluster** | `cluster.x-k8s.io` | Reprezentace clusteru (infra reference, control plane ref, networking) |
|
||||
| **Machine** | `cluster.x-k8s.io` | Jednotlivý uzel (VM/BM instance) |
|
||||
| **MachineDeployment** | `cluster.x-k8s.io` | Deklarativní škálování a rolling update workerů |
|
||||
| **MachineSet** | `cluster.x-k8s.io` | Replica set pro Machiny (lower-level) |
|
||||
| **MachineHealthCheck** | `cluster.x-k8s.io` | Auto-remediaci (automatické nahrazení unhealthy uzlu) |
|
||||
| **ClusterClass** | `cluster.x-k8s.io` | Šablona pro vytváření clusterů |
|
||||
| **KubeadmControlPlane** | `controlplane.cluster.x-k8s.io` | Control plane managed kubeadm (stacked/external etcd) |
|
||||
| **KubeadmConfig / KubeadmConfigTemplate** | `bootstrap.cluster.x-k8s.io` | Bootstrap konfigurace (kubeadm init/join) |
|
||||
|
||||
### Provider model
|
||||
|
||||
CAPI používá třívrstvý provider model:
|
||||
|
||||
#### 1. Infrastructure Provider
|
||||
Vytváří a spravuje infrastrukturu (VM, sítě, LB, storage).
|
||||
|
||||
| Provider | Platforma | Status |
|
||||
|----------|-----------|--------|
|
||||
| **AWS (CAPA)** | AWS EC2, VPC, ELB, EKS | Stable, SIG-sponsored |
|
||||
| **Azure (CAPZ)** | Azure VM, VNet, LB, AKS | Stable, SIG-sponsored |
|
||||
| **GCP (CAPG)** | GCP Compute, VPC, GKE | Beta |
|
||||
| **vSphere (CAPV)** | VMware vSphere | Stable |
|
||||
| **OpenStack (CAPO)** | OpenStack compute/network | Stable |
|
||||
| **Metal3** | Bare metal (Ironic) | Stable |
|
||||
| **Docker (CAPD)** | Docker containers (development) | Tilt/Dev only |
|
||||
| **Akamai (Linode)** | Linode | Community |
|
||||
| **Azure Stack HCI** | Azure Stack HCI | Community |
|
||||
| **cloudscale** | cloudscale.ch | Community |
|
||||
| **Exoscale** | Exoscale | Community |
|
||||
| **IBM Cloud** | IBM Cloud | Community |
|
||||
| **Equinix Metal** | Equinix (ex Packet) | Community |
|
||||
| **Hetzner** | Hetzner Cloud | Community |
|
||||
| **OpenNebula** | OpenNebula | Community |
|
||||
|
||||
#### 2. Bootstrap Provider
|
||||
Zajišťuje inicializaci K8s na node (kubeadm init/join, TLS certs, tokeny).
|
||||
|
||||
| Provider | Popis |
|
||||
|----------|-------|
|
||||
| **Kubeadm** (vestavěný) | Standardní kubeadm init/join, podpora stacked/external etcd |
|
||||
| **EKS** | Bootstrap pro EKS managed control plane (AWS) |
|
||||
| **K3s** | Lightweight K8s bootstrap (edge, IoT) |
|
||||
| **RKE2** | Rancher K8s bootstrap, CIS-hardened |
|
||||
| **Talos** | API-driven bootstrap (Sidero Labs), immutable OS |
|
||||
| **k0smotron** | K0s-based bootstrap + hosted control plane |
|
||||
| **MicroK8s** | Canonical MicroK8s bootstrap |
|
||||
| **Canonical Kubernetes** | Canonical K8s (snap-based) |
|
||||
|
||||
#### 3. Control Plane Provider
|
||||
Spravuje control plane uzly.
|
||||
|
||||
| Provider | Popis |
|
||||
|----------|-------|
|
||||
| **KubeadmControlPlane** (vestavěný) | Kubeadm-managed CP, stacked/external etcd |
|
||||
| **EKS** | AWS EKS managed control plane |
|
||||
| **Kamaji** | Hosted control plane (CP běží jako deployment v management clusteru) |
|
||||
| **K3s** | K3s control plane (edge-optimized) |
|
||||
| **RKE2** | RKE2 control plane |
|
||||
| **Talos** | Talos control plane, API-based management |
|
||||
| **k0smotron** | Hosted control plane (k0s-based) |
|
||||
| **Nested** | Nested virtualization control plane |
|
||||
|
||||
### ClusterClass a Managed Topologies
|
||||
|
||||
ClusterClass (stabilní od CAPI v1beta1, CAPI v1.0+) umožňuje definovat **šablonu clusteru**:
|
||||
|
||||
```yaml
|
||||
apiVersion: cluster.x-k8s.io/v1beta1
|
||||
kind: ClusterClass
|
||||
metadata:
|
||||
name: standard-aws-cluster
|
||||
spec:
|
||||
controlPlane:
|
||||
ref:
|
||||
apiVersion: controlplane.cluster.x-k8s.io/v1beta1
|
||||
kind: KubeadmControlPlaneTemplate
|
||||
name: aws-cp-tmpl
|
||||
machineInfrastructure:
|
||||
ref:
|
||||
kind: AWSMachineTemplate
|
||||
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
|
||||
name: aws-cp-machine-tmpl
|
||||
workers:
|
||||
machineDeployments:
|
||||
- class: default-worker
|
||||
template:
|
||||
bootstrap:
|
||||
ref:
|
||||
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
|
||||
kind: KubeadmConfigTemplate
|
||||
name: aws-worker-bootstrap-tmpl
|
||||
infrastructure:
|
||||
ref:
|
||||
apiVersion: infrastructure.cluster.x-k8s.io/v1beta2
|
||||
kind: AWSMachineTemplate
|
||||
name: aws-worker-machine-tmpl
|
||||
variables:
|
||||
- name: instanceType
|
||||
required: true
|
||||
schema:
|
||||
openAPIV3Schema:
|
||||
type: string
|
||||
enum: ["t3.large", "m5.large", "m5.xlarge"]
|
||||
```
|
||||
|
||||
Pak lze vytvořit cluster s přetížením proměnných:
|
||||
|
||||
```yaml
|
||||
apiVersion: cluster.x-k8s.io/v1beta1
|
||||
kind: Cluster
|
||||
metadata:
|
||||
name: dev-team-alpha
|
||||
namespace: clusters
|
||||
spec:
|
||||
topology:
|
||||
class: standard-aws-cluster
|
||||
version: v1.30.2
|
||||
controlPlane:
|
||||
replicas: 1
|
||||
workers:
|
||||
machineDeployments:
|
||||
- class: default-worker
|
||||
name: md-0
|
||||
replicas: 2
|
||||
variables:
|
||||
- name: instanceType
|
||||
value: "m5.xlarge"
|
||||
```
|
||||
|
||||
### Životní cyklus clusteru s CAPI
|
||||
|
||||
| Fáze | Akce | CAPI mechanismus |
|
||||
|------|------|------------------|
|
||||
| **Create** | `kubectl apply -f cluster.yaml` | Controller vytvoří infra (VM, network), provede bootstrap kubeadm init/join |
|
||||
| **Scale** | Upravit `replicas` v MachineDeployment | Controller vytvoří/odstraní Machine → VM → node join/drain |
|
||||
| **Upgrade** | Změnit `version` v KubeadmControlPlane / MachineDeployment | Rolling update: nový CP node → upgrade → starý drain a delete. Workers: MachineDeployment rolling update |
|
||||
| **Health check** | MachineHealthCheck | Pokud node unhealthy > timeout, controller vytvoří náhradní Machine |
|
||||
| **Delete** | `kubectl delete cluster` | Controller provede drain, delete VMs, cleanup infrastruktury |
|
||||
| **Template update** | Změna AWSMachineTemplate / KubeadmConfigTemplate | Stroj se vytvoří s novou šablonou; stávající Machiny se dotýká jen přes rolling update |
|
||||
|
||||
### Auto-remediace (MachineHealthCheck)
|
||||
|
||||
```yaml
|
||||
apiVersion: cluster.x-k8s.io/v1beta1
|
||||
kind: MachineHealthCheck
|
||||
metadata:
|
||||
name: prod-mhc
|
||||
namespace: clusters
|
||||
spec:
|
||||
clusterName: prod-us-east
|
||||
selector:
|
||||
matchLabels:
|
||||
cluster.x-k8s.io/deployment-name: prod-us-east-workers
|
||||
unhealthyConditions:
|
||||
- type: Ready
|
||||
status: "False"
|
||||
timeout: 5m
|
||||
- type: Ready
|
||||
status: Unknown
|
||||
timeout: 5m
|
||||
maxUnhealthy: "40%"
|
||||
nodeStartupTimeout: 10m
|
||||
```
|
||||
|
||||
### CAPI + GitOps
|
||||
|
||||
CAPI se přirozeně integruje s GitOps:
|
||||
|
||||
- **ArgoCD** — Cluster a MachineDeployment manifesty v Git repozitáři, ArgoCD je aplikuje na management cluster
|
||||
- **Flux** — `Kustomization` + `OCIRepository` pro CAPI objekty
|
||||
- **Crossplane** — lze kombinovat: Crossplane pro provisioning cloud resources (VPC, subnets), CAPI pro K8s cluster na nich
|
||||
|
||||
Vzor: vyhrazený "fleet management" cluster, na kterém běží CAPI + ArgoCD. Všechny workload clustery jsou definované jako YAML v Gitu.
|
||||
|
||||
### CAPI pro on-prem
|
||||
|
||||
| Provider | Use case | Poznámka |
|
||||
|----------|----------|----------|
|
||||
| **Metal3** (Ironic) | Bare metal provisioning (PXE, IPMI, Redfish) | Automatické provisionování BM serverů jako K8s nodes |
|
||||
| **CAPV (vSphere)** | VMware VM jako K8s nodes | Většina enterprise on-prem |
|
||||
| **CAPO (OpenStack)** | OpenStack VM jako K8s nodes | OpenStack-native |
|
||||
| **Nutanix (CAPNX)** | Nutanix AHV/Prism | Community provider |
|
||||
|
||||
### CAPI pro edge
|
||||
|
||||
| Provider | Use case | Poznámka |
|
||||
|----------|----------|----------|
|
||||
| **K3s bootstrap + control plane** | Lightweight K8s na edge zařízeních | Single binary, SQLite/embedded etcd |
|
||||
| **RKE2 bootstrap + control plane** | Enterprise edge, air-gapped | CIS-hardened, FIPS |
|
||||
| **Talos** | Immutable OS, API-driven | Minimal footprint, no SSH |
|
||||
| **k0smotron** | Hosted control plane pro edge clustery | CP běží v management clusteru, worker na edge |
|
||||
|
||||
### CAPI vs alternativy
|
||||
|
||||
| Nástroj | Přístup | CAPI výhoda | CAPI nevýhoda |
|
||||
|---------|---------|-------------|---------------|
|
||||
| **Terraform/Pulumi** | Imperativní/declarativní IaC | CAPI je K8s-native — stejný nástroj pro appky i clustery; GitOps ready | Terraform má širší podporu non-K8s resources |
|
||||
| **kubeadm** | Manuální nebo skriptovaný | CAPI automatizuje celý lifecycle včetně upgradů a remediací | Vyšší komplexita, nutný management cluster |
|
||||
| **Rancher** | Web UI + API pro správu K8s clusterů | CAPI je open-source, vendor-neutral | Rancher má GUI, monitoring, katalog appek |
|
||||
| **OpenShift Hive/ACM** | Red Hat Advanced Cluster Management | CAPI je standardní (SIG) — širší provider ecosystem | ACM má governance, policy, compliance |
|
||||
|
||||
### Limitations a maturity
|
||||
|
||||
- **Management cluster je SPOF** — musí mít vlastní HA a backup (etcd zálohy, certifikáty)
|
||||
- **CAPI není cluster autoscaler** — řeší lifecycle clusterů, ne auto-scaling podů v rámci clusteru (používá se Cluster Autoscaler samostatně)
|
||||
- **Provider maturity se liší** — AWS/Azure/vSphere stabilní, GCP/OpenStack beta, některé community providers alpha
|
||||
- **etcd backup není built-in** — nutné řešit externě (Velero, etcd snapshot)
|
||||
- **CAPI neřeší aplikace** — pouze lifecycle K8s clusterů (monitoring, logging, ingress si řídí uživatel)
|
||||
- **Learning curve** — nutnost management clusteru, pochopení provider modelu, CRDs
|
||||
- **CAPI v1.13+ (2026)** — stable release, v1beta1 API je GA, ClusterClass stable, EKS/AKS/GKE managed control plane podpora
|
||||
|
||||
### Doporučený stack pro CAPI v produkci
|
||||
|
||||
| Komponenta | Doporučení |
|
||||
|------------|------------|
|
||||
| **Management cluster** | K3s (malý footprint) nebo kubeadm (3 nodes HA) |
|
||||
| **Infra provider** | CAPA (AWS) / CAPV (vSphere) / CAPO (OpenStack) — dle platformy |
|
||||
| **Bootstrap/CP provider** | Kubeadm nebo RKE2 |
|
||||
| **GitOps** | ArgoCD nebo Flux |
|
||||
| **Backup** | Velero + restic/Ceph |
|
||||
| **Cluster autoscaler** | Cluster Autoscaler (přes CAPI integration) |
|
||||
| **Network** | Cilium (CAPI-native, podpora) |
|
||||
| **Secrets** | External Secrets Operator / Sealed Secrets |
|
||||
| **Monitoring** | Prometheus + Grafana (kube-prometheus-stack) |
|
||||
| **Ingress** | ingress-nginx / Kong / Traefik |
|
||||
|
||||
## Zdroje
|
||||
|
||||
Odkazy, knihy a standardy: [sources/infrastructure/sources.md](sources/infrastructure/sources.md)
|
||||
|
||||
*Poslední revize: 2026-06-18*
|
||||
Reference in New Issue
Block a user