Fossil/knowledge-base
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
ef3c2f75b16e5ac57555c62441fd99306082f54e
knowledge-base/HYPERVISORS.en.md
Stanislav Hubacek ef3c2f75b1 18.6.2026
2026-06-18 16:25:33 +02:00

36 KiB
Raw Blame History

🖥️ Hypervisors and Virtualization Platforms

Hypervisor Types

Type Description Examples
Type 1 (bare-metal) Runs directly on hardware VMware ESXi, Microsoft Hyper-V, KVM, Xen
Type 2 (hosted) Runs on top of host OS VirtualBox, VMware Workstation, Parallels

Platform Overview

Platform Hypervisor License Note
VMware vSphere ESXi Proprietary (Subscription from 2024) Market leader, wide adoption. After Broadcom acquisition (2023), switched to per-core subscription, perpetual license discontinued
Microsoft Hyper-V Hyper-V Windows Server / standalone Integration with Azure, SCVMM
Proxmox VE KVM + LXC Open source Debian-based, web UI, low cost
Red Hat OpenStack / oVirt KVM Open source Open alternative, complex
Nutanix AHV KVM (fork) Part of Nutanix Integrated HCI solution
XCP-ng / Xen Server Xen Open source Successor to Citrix Hypervisor
Oracle VM Xen Proprietary Oracle ecosystem

Key Concepts

  • VM — Virtual Machine — full virtualization, own kernel
  • Container — shared host kernel, lighter (Docker, LXC)
  • Paravirtualization — guest OS knows it runs in a VM (better I/O performance)
  • NUMA — Non-Uniform Memory Access, CPU/memory allocation optimization (see SERVER-HW.en.md)
  • Overcommit — allocating more vCPU/RAM than physically available (ratio management)
  • Live Migration — moving a running VM between hosts (vSphere vMotion, Hyper-V Live Migration)
  • HA (High Availability) — VM restart on another host upon failure
  • DRS / Load Balancing — automatic VM distribution based on load

VMware vSphere

VMware licensing (post-Broadcom 2024+)

Since 2024, VMware only sells subscription licenses; perpetual + SnS (Support & Subscription) have been discontinued.

Product Metric Price (indicative) What it includes
vSphere Standard Per core (min 16 cores/CPU) ~$140/core/year ESXi, vCenter, vMotion, HA, DRS basic
vSphere Enterprise Plus Per core ~$220/core/year All above + DRS advanced, SIOC, NIOC, Big Data Extensions
vSphere Foundation Per core (bundle) ~$350/core/year vSphere Enterprise Plus + Aria Operations, Aria Operations for Logs, Aria Automation
VMware Cloud Foundation (VCF) Per core (bundle) ~$700/core/year vSphere + vSAN + NSX + Aria full suite. Required for vSAN and NSX from 2025
vSAN Per core (only as part of VCF from 2025) No longer standalone Storage virtualization, dedup, compression, encryption
NSX Per core (only as part of VCF from 2025) No longer standalone SDN, micro-segmentation, firewall, load balancing

Key changes after Broadcom acquisition:

  • Discontinued perpetual license sales (May 2024)
  • Discontinued standalone products: vSAN and NSX can no longer be purchased standalone (only within VCF)
  • Desktop and ROBO variants cancelled (migrated to VCF)
  • Average cost increase: 2–5× compared to the previous model (depends on size and product mix)
  • Impact: Many customers are migrating to Proxmox VE, Nutanix AHV, or Hyper-V

Per-core calculation:

Server: 2× EPYC 9654 (96C each) = 192 cores
  vSphere Standard: 192 × $140 = $26,880/year
  VCF: 192 × $700 = $134,400/year (incl. vSAN and NSX)
For comparison: previously perpetual + SnS ≈ $15,000 one-time + $3,000/year

VMware Exit Strategy (post-Broadcom 2024+)

Context

After Broadcom's acquisition of VMware (completed November 2023), the virtualization market experienced the biggest upheaval in its history. Changes include:

  • Discontinuation of perpetual licenses (February 2024) — mandatory subscription model
  • Forced bundling — 8,000+ SKUs reduced to 4 bundles (VCF, VVF, vSphere Standard/Foundation)
  • Minimum 72-core commitment (from April 2025) — small servers can no longer be licensed economically
  • 20% late renewal penalty — no tolerance
  • Price increase of 150–1,500% depending on size and product mix
  • Standalone products discontinued — vSAN and NSX only within VCF
  • Collapse of the partner ecosystem — from 4,500+ partners to ~300 Premier

According to Foundry/CIO.com survey (2025): 56% of organizations plan to reduce VMware usage, 71% are actively looking for on-premise alternatives. Gartner predicts a loss of ~35% of workloads within 3 years.

Three Strategies

Strategy Description Suitable for
Stay Accept new pricing, renew VCF/VVF subscription Large organizations with deep integration where migration costs more than new licenses
Reduce Reduce VMware footprint, migrate part of workloads to alternatives, optimize the rest Medium and large enterprises with heterogeneous environments
Exit Complete migration to an alternative platform SMEs, organizations facing 3–6× cost increases, greenfield projects

Target Platforms — Comparison

Criterion Proxmox VE Nutanix AHV Microsoft Hyper-V Red Hat OpenShift Virtualization Sangfor aSV (HCI)
Hypervisor KVM + LXC KVM (fork) Hyper-V KVM (KubeVirt) KVM (aSV)
License Open source (free), support ~€500/host/year Per node subscription (30–60% savings vs VCF) Windows Server license (Standard/Datacenter) OpenShift subscription (core-based) Per node (Enterprise Pro), all-inclusive
Live Migration Live Migration (Proxmox 8+) AHV Live Migration Live Migration (SMB/RDMA) KubeVirt (VMI live migration) Yes
HA Proxmox HA (watchdog, fencing) Built-in HA (Prism) Hyper-V HA (WS Failover Cluster) OpenShift HA (self-healing) Built-in HA
Storage ZFS, Ceph, LVM AOS (hybrid/SSD, erasure coding) S2D, CSV, ReFS OCS, Ceph, LSO aSAN (distributed SDS, locality-aware)
Backup Proxmox Backup Server (free) Native snapshot + DR Windows Server Backup / Veeam OpenShift APIs + OADP Built-in backup + CDP
Price (3 years, 3 hosts) $0 + support $1,500 ~$45,000–60,000 $0 (Hyper-V Server free) or Windows Server license ~$90,000+ (OpenShift) ~$15,000–25,000
Price (3 years, 10 hosts) $0 + support $5,000 ~$150,000–200,000 Windows Server Datacenter for unlimited VMs ~$300,000+ (OpenShift) ~$50,000–80,000
Migration difficulty Medium (VMDK → QCOW2, VirtIO drivers) Low (Nutanix Move tool) Medium (V2V converter, SCVMM) High (Kubernetes learning curve) Low (VMware import tool)
Linux support Excellent (native KVM) Excellent (KVM-based) Good (LIS drivers) Excellent (KVM + OpenShift) Excellent (KVM-based)
Windows support Good (VirtIO drivers) Excellent (ALAS drivers, svpd) Excellent (native) Good (KubeVirt + VirtIO) Good (VirtIO drivers)
GPU passthrough VFIO (excellent) GPU passthrough DDA (Direct Device Assignment) VFIO + GPU Operator vGPU support (standard)
Integrated security — — — — Yes (NGFW, IPS, WAF, EDR — aSEC)
Min. cluster (3 copies) 3 (Ceph) 3 2–3 3 3

Migration Tools

Tool Source Platform Target Platform Method
Proxmox VMware Import Wizard VMware ESXi Proxmox VE Web GUI import via NFS/ESXi API. Limitation: snapshots must be removed, UEFI not supported before Proxmox 8.1
Nutanix Move VMware ESXi, Hyper-V Nutanix AHV Virtual appliance, automated migration with minimal downtime, UEFI support, can retain IP/MAC
Veeam Backup & Replication v12.2+ VMware ESXi Proxmox VE Backup/restore via Veeam, hot migration, Proxmox support from v12.2
StarWind V2V Converter VMware ESXi Proxmox, Hyper-V, XCP-ng Free GUI tool, VMDK → QCOW2/raw/VHDX, CLI support, hot migrations
virt-v2v VMware ESXi, Xen, Hyper-V KVM (libvirt) Open source CLI tool, disk + driver conversion (virtio), suitable for bulk migration
Windows Admin Center VM Conversion Extension VMware ESXi Hyper-V Microsoft WAC extension, free, GUI-based, bulk migration
Platform9 vJailbreak VMware ESXi OpenStack / KVM In-place migration (no swing gear), open source
Sangfor VMware Import Tool VMware ESXi Sangfor aSV (HCI) VMware import tool, disk + driver conversion, can retain network config

Cross-Hypervisor Migration Matrix

Comprehensive overview of all source→target pairs with methods, tools, limitations, and complexity.

Source → Target Method Tools Complexity Limitations
VMware → Proxmox Disk conversion VMDK→QCOW2, driver reinstall Proxmox Import Wizard, Veeam, StarWind, virt-v2v Medium VirtIO drivers required, UEFI not supported in Import Wizard (< 8.1), snapshots must be removed
VMware → Hyper-V Disk conversion VMDK→VHDX, driver reinstall StarWind, WAC Converter, SCVMM, Microsoft MTC Medium Integration Services required, network config differences (VMXNET3 → Hyper-V Synthetic)
VMware → KVM/XCP-ng Disk conversion VMDK→raw/QCOW2, driver swap virt-v2v, StarWind Medium VirtIO drivers, UEFI support (OVMF), host passthrough compatibility
VMware → Nutanix AHV Automated migration via Move appliance Nutanix Move, Veeam Low AHV is also KVM — minimal issues, retain IP/MAC, UEFI support
VMware → Sangfor aSV Import via VMware Import Tool, disk + driver conversion Sangfor VMware Import Tool Low Built-in tool, retain network config, UEFI support
VMware → OpenStack In-place or swing Platform9 vJailbreak, virt-v2v + Glance High Network redesign (Neutron), storage (Cinder), image format (Glance) required
Hyper-V → VMware Disk conversion VHDX→VMDK, driver reinstall StarWind, virt-v2v, VMware vCenter Converter (standalone) Medium VMware Tools required, network driver change (VMXNET3), UEFI/secure boot issues
Hyper-V → Proxmox Disk conversion VHDX→QCOW2, driver swap StarWind, virt-v2v, qemu-img Medium–High VirtIO drivers, integration services → guest agent, secure boot issues
Hyper-V → KVM/XCP-ng Disk conversion VHDX→raw/QCOW2 virt-v2v, qemu-img Medium VirtIO drivers, Linux generic drivers usually work
Hyper-V → Nutanix AHV Automated migration Nutanix Move Low–Medium Similar to VMware→Nutanix, UEFI support, retain IP
Proxmox → VMware Export OVF/OVA, qemu-img convert qemu-img (QCOW2→VMDK), ovftool, manual OVF export High VMware Tools required, storage format differences, no live migration, downtime required
Proxmox → Hyper-V qemu-img convert, driver reinstall qemu-img, manual VHDX conversion High Hyper-V Integration Services required, no automated tool, edge case
Proxmox → KVM/XCP-ng Direct QCOW2 (same format), XML edit libvirt, virsh dumpxml/define Medium libvirt XML/QEMU args differences (storage pool, network), validation required
Proxmox → Nutanix AHV qemu-img + manual import qemu-img, Nutanix Image Service CLI High No hot tool, conversion + manual VM reconfiguration required
XCP-ng → VMware Disk conversion VHD→VMDK qemu-img, StarWind, virt-v2v High VMware Tools required, paravirtualization differences (Xen PV vs VMware)
XCP-ng → Proxmox Disk conversion or direct VHD qemu-img, manual import Medium Disk conversion, VHD format not native in Proxmox
XCP-ng → Hyper-V Disk conversion VHD→VHDX (direct) StarWind, qemu-img Medium VHD/VHDX compatible, Integration Services required
Nutanix AHV → VMware Export + conversion qemu-img, Nutanix Export, VMware vCenter Converter High VMware Tools, AHV is KVM → usually easier than Hyper-V→VMware
Nutanix AHV → Proxmox qemu-img + manual import qemu-img, Nutanix self-service restore Medium AFS disks → QCOW2, metadata must be reconstructed
Nutanix AHV → Hyper-V qemu-img + manual qemu-img, StarWind High Edge case, no hot tool
OpenStack → (any) Glance export + qemu-img glance image-download, qemu-img, ovftool Medium–High Image format (raw/QCOW2), metadata (flavor, security groups) must be recreated
Sangfor aSV → (any) qemu-img conversion + manual qemu-img, manual OVF/OVA export Medium–High KVM-based → conversion to QCOW2/VMDK/VHDX via qemu-img, metadata must be recreated
(any) → Sangfor aSV aSV API import + VMware Import Tool Sangfor VMware Import Tool (for VMware), manual qemu-img import for others Medium KVM-based → standard formats supported, import tool for VMware only

Migration success keys:

  • Drivers — each platform requires its own paravirtual drivers (VMware Tools, VirtIO, Hyper-V Integration Services, Xen Tools). Always swap after migration.
  • UEFI / Secure Boot — not all combinations support UEFI (Proxmox Import Wizard < 8.1 does not). Test UEFI VMs before migration.
  • Snapshots — snapshots must be removed (merged) before migration. Most tools only migrate flat disks.
  • Network — MAC addresses, IP addresses, VLAN tagging — verify after migration. Some tools (Nutanix Move, VMware Converter) can retain MAC.
  • Storage format — VMDK ↔ VHDX ↔ QCOW2 ↔ raw are inter-convertible via qemu-img, but metadata differs (snapshots, backing files).
  • Live migration — no live migration exists between different hypervisors. Downtime is always required (minutes to hours depending on VM size).
  • Migration temperature — the "colder" the VM (fewer changes), the easier the migration. Real-time database applications require a separate DB migration plan.
Platform Year 1 3 Years Total Note
VMware VVF (1-year rate) $22,800 $68,400 120 cores × $190/core/year
VMware VCF $42,000 $126,000 120 cores × $350/core/year
Proxmox VE (support) $1,500 $4,500 3× €500/host/year
Nutanix AHV (average) ~$18,000 ~$54,000 Per node subscription, estimate
Hyper-V (Windows Server Datacenter) $12,400 $37,200 One-time license per core, without SA
Hyper-V (Azure Stack HCI) ~$7,200 ~$21,600 ~$10/core/month, 120 cores
Sangfor HCI (Enterprise Pro) ~$5,000–8,000 ~$15,000–25,000 Per node, all-inclusive, 3 nodes

Real-world example from Spiceworks (2026): A user reports VMware Essentials+ increasing from $1,900/year to $14,000/year (VVF) — a 7.4× increase.

Decision Framework

1. Audit VMware environment
   ├─ Number of hosts, core count, utilization
   ├─ Feature dependency (vSAN, NSX, SRM)
   ├─ Workload profile (Windows vs Linux, DB, GPU)
   └─ Hardware refresh cycle

2. Calculate TCO for VMware renewal (3 years)
   ├─ VVF vs VCF vs current model
   └─ Include audit risk, late renewal penalty

3. Select target platform (1-2 candidates)
   ├─ Proxmox: lowest TCO, Linux-heavy shops
   ├─ Nutanix: enterprise HCI, low migration difficulty
    ├─ Hyper-V: Windows-centric, Azure hybrid
    ├─ Sangfor: HCI all-in-one, security-first, VMware exit (SMB/mid-market)
    └─ OpenShift: Kubernetes-first, platform engineering

4. Plan migration phases
   ├─ Wave 1: non-critical (dev/test, 1-2 months)
   ├─ Wave 2: standard production (3-6 months)
   ├─ Wave 3: mission-critical (6-12 months)
   └─ Coexistence: VMware + target running in parallel

5. Allow 18-48 months for complete exit (Gartner)

Real-World Case Studies

Organization Starting Point Target Scale Result
Stanford University VMware (60+ nodes) Proxmox VE (6 clusters) 1,500 VMs Completed 2025, increased automation, lower costs
Michelin VMware Platform9 + OpenStack Dozens of nodes Platform engineering team, production workload migration
Czech enterprise (50-100 servers) VMware Proxmox VE ~100 VMs Annual savings of ~340,000–500,000 CZK on licenses

Timing — Key Deadlines

Event Date Impact
Discontinuation of perpetual licenses February 2024 Already done
72-core minimum April 2025 Small server licensing became more expensive
vSphere 7 EOS April 2025 Upgrade to 8.x required
ESXi 8.0 EOS October 2027 Last supported version, migration deadline
Windows Server 2025 Hyper-V December 2025 64-host cluster, 2,048 vCPU per VM
Proxmox VE 9 + Datacenter Manager 2026 Enterprise features, vCenter alternative

Recommendations

Scenario Action
Small company (< 10 hosts), Linux workloads Migrate to Proxmox VE — immediate 100% license savings
Medium company (10-50 hosts), mixed workloads Evaluate Nutanix AHV (easy migration) or Proxmox (lower TCO)
Enterprise (50+ hosts), deep VMware integration Reduce strategy: optimize existing VMware + migrate selected workloads to OpenShift / Hyper-V
Microsoft shop Hyper-V / Azure Stack HCI — native Azure hybrid, no additional hypervisor licenses
Kubernetes-native team OpenShift Virtualization / KubeVirt — unify VM and container management
MSP / hosting provider Nutanix or OpenStack — multi-tenancy, vCloud Director alternative

Cluster Design

  • Max cluster size: 64 hosts (vSphere 8/9), 96 hosts (vSphere 8 + enhanced)
  • Datastore limits: max 256 datastores per host, max 65 TB per VMFS-6 datastore
  • vSAN ready capacity: recommended max 60–64 hosts per vSAN cluster
  • Fault domains — cluster division into host groups (rack awareness), min 3 fault domains for stretched cluster
  • Admission control — resource reservation for HA failover:
    • Host failures cluster tolerates — most common (1–4 hosts)
    • Percentage of cluster resources — reserve % of CPU/memory
    • Dedicated failover hosts — dedicated host(s) for HA
  • Cluster limits (vSphere 8/9):
    • 960 VMs per host (vSphere 9 max)
    • 15,000 VMs per cluster (vCenter max)
    • 300 hosts per cluster (vSphere 8/9, hardware vMotion)

Microsoft Hyper-V Licensing

Variant Metric Price What it includes
Windows Server Standard Per core (min 16 licenses/server) + CAL ~$1,000/core (one-time) + $200/CAL 2 VM licenses (each with full Windows Server license)
Windows Server Datacenter Per core (min 16 licenses/server) + CAL ~$6,200/core (one-time) + $200/CAL Unlimited VMs, Storage Spaces Direct, Shielded VMs
Azure Stack HCI Per core (monthly) ~$10–20/core/month (Azure hybrid benefit) Hyper-V + S2D + Azure management, part of Azure subscription
Hyper-V Server Free $0 Standalone hypervisor (no management, no GUI, limited support) — no longer distributed as of 2025

Important:

  • Windows Server Standard = 2 VMs per license. If you need 3 VMs on a 2-socket server, you need 2× Standard license (4 VMs) or Datacenter
  • Azure Hybrid Benefit — if you have Windows Server with SA (Software Assurance), you can use licenses in Azure at no additional cost
  • CAL (Client Access License) — every user or device accessing Windows Server must have a CAL (except Azure Hybrid Benefit)

Microsoft Hyper-V

Feature Hyper-V Note
Max hosts per cluster 64 (Windows Server 2025) Shared Nothing Live Migration
Max VMs per host 1,024 (WS 2022+) Generation 2 VMs
Max vCPU per VM 240 (WS 2022+) 64-host cluster
Max RAM per VM 12 TB (WS 2022+) Dynamic memory
Live Migration SMB, CSV, RDMA Compressed or RDMA
Storage CSV (Cluster Shared Volumes), ReFS S2D for HCI
Nested Virtualization Yes Intel VT-x / AMD-V
SCVMM System Center VMM Enterprise management, fabric, P2V

Hyper-V vs VMware Comparison

Feature VMware vSphere Microsoft Hyper-V
OS VMware ESXi (VMkernel) Windows Server / Hyper-V Server
License Per CPU (subscription) Windows Server license / Datacenter
Storage VMFS, NFS, vSAN, HCI NTFS, ReFS, SMB, S2D
Live Migration vMotion (cross-vSwitch, long distance) Live Migration (SMB/RDMA)
Storage Migration Storage vMotion (online) Shared Nothing (data disk)
Replication vSphere Replication Hyper-V Replica (ASR)
Management vCenter, vSphere Client SCVMM, Hyper-V Manager, Admin Center
Linux support Excellent (open-vm-tools) Good (Linux Integration Services)
TCO Higher Lower (with Windows license)

KVM

Architecture

Hardware ──> QEMU (I/O emulation) + KVM (kernel module, virtualization)
                │
           libvirt (API + management)
                │
        ┌───────┼───────────┐
  virt-manager  virsh    openstack/proxmox

Tuning

  • CPU pinning — virsh vcpupin vm1 0 2 (vCPU 0 → physical core 2), prevents context switching
  • Huge pages — 2 MB / 1 GB pages instead of 4 KB, reduces TLB misses (VMs with large RAM): echo 2048 > /proc/sys/vm/nr_hugepages
  • NUMA affinity — VM pinned to one NUMA node (minimizes cross-NUMA memory access)
    • numactl --cpunodebind=0 --membind=0
    • virsh numatune vm1 --nodeset 0
  • VirtIO — paravirtualized I/O (virtio-net, virtio-blk, virtio-scsi) for better performance
  • IO threads — dedicated threads for QEMU I/O emulation

KVM Tuning Checklist

  • Verify HW virtualization: lscpu | grep Virtualization
  • Load KVM modules: kvm, kvm_intel/kvm_amd, vfio-pci
  • Optimize storage: raw/LVM (avoid qcow2 for performance workloads)

Sangfor aSV (HCI)

Chinese vendor — KVM-based hypervisor, part of Sangfor HCI stack (aSV + aSAN + aNet + aSEC). Distributed through partners in EMEA.

Stack architecture

Component Role
aSV Hypervisor (KVM-based)
aSAN Distributed SDS (locality-aware, data tiering, dedup, compression)
aNet Network virtualization (distributed switches and routers, WYDIWYG visual editor)
aSEC Security (NGFW, IPS, WAF, EDR, east-west segmentation)
Sangfor Cloud Platform Management orchestrator, unified dashboard

Key features

Feature Detail
Hypervisor KVM (aSV) — custom fork with HCI extensions
License Enterprise Pro — per node, all-inclusive (compute + storage + network + security)
Min. cluster 3 nodes (3 data copies)
Live Migration Yes
HA Built-in HA
Storage aSAN — locality-aware, data tiering (SSD + HDD), dedup, compression, erasure coding
Backup Built-in backup + CDP — no 3rd party needed
Security Integrated NGFW, IPS, WAF, EDR — no external appliances
VDI aDesk — integrated VDI solution
Kubernetes SKE (Sangfor Kubernetes Engine)
Migration Sangfor VMware Import Tool (from vCenter), qemu-img for others
vGPU Standard support (no extra license)

Comparison with VMware

Feature Sangfor VMware
License Per node, all-inclusive Multi-tier (vSphere + vSAN + NSX + Aria)
vGPU Included (standard) Enterprise Plus only
Backup + CDP Built-in 3rd party or extra license
Security (NGFW, IPS, WAF) Built-in (aSEC) NSX + 3rd party
Network management WYDIWYG visual editor NSX Manager (more complex)
Min. cluster (3 copies) 3 nodes 5 nodes (vSAN)
Data locality Yes No
SSD life prediction Yes No

Use case

  • VMware exit — VMware replacement for SMB and mid-market
  • Greenfield HCI — new DCs, branch offices, remote sites
  • VDI — aDesk integrated with HCI
  • Security-first — organizations requiring integrated security
  • Asia-Pacific / EMEA — strongest in Asia, expanding to Europe

Risks and limitations

Risk Detail
Geopolitical Chinese vendor — possible regulatory restrictions (GDPR, EU, NATO, government)
Ecosystem Smaller community than VMware/Proxmox, less documentation and ISV certifications
Support Primary support from Asia, local partner critical
Vendor lock-in Closed ecosystem (aSV + aSAN + aNet + aSEC), harder to mix with 3rd party
References in CZ/EU Very limited — pilot required before production

Storage in Hypervisors

See also: STORAGE.en.md — detailed overview of storage protocols and configurations.

Type Description Protocols
Local storage Disks directly in the server SATA, SAS, NVMe
Shared storage SAN / NAS accessible to all hosts Fibre Channel, iSCSI, NFS, SMB
vSAN / HCI Hyperconverged storage (server disks = single pool) VMware vSAN, Nutanix, StarWind
Software-Defined SDS separates storage software from hardware Ceph, GlusterFS, MinIO

HCI Details

Feature Nutanix (AOS + AHV) VMware vSAN Azure Stack HCI
Hypervisor AHV (KVM fork), ESXi optional ESXi (required) Hyper-V
Min. nodes 3 2 (witness) 2 (witness)
Max nodes 80+ 64 16 (typical)
Replication 2 or 3 copies + erasure coding Mirroring (RAID 1), erasure coding Mirroring + parity
Deduplication Cluster-level (post-process) Disk-level (capacity tier) ReFS (real-time)
Compression Inline (AOS 6+) Dedup + compression combined ReFS
Management Prism (web UI) vCenter + vSAN UI Windows Admin Center
Licensing Per node subscription Per CPU subscription Per core subscription
Ecosystem Built-in DR, backup, security Broad ISV ecosystem Azure integration
Use case Enterprise VDI, general VM VMware-centric shops Azure hybrid, branch offices

Virtualization Platforms — Comparison

Capability VMware vSphere Microsoft Hyper-V Proxmox VE Nutanix AHV
Live Migration vMotion Live Migration Live Migration Live Migration
HA vSphere HA Hyper-V HA Proxmox HA Built-in
DRS/balancing DRS SCVMM / AKS HA groups Built-in
Storage vMotion yes when VM is off ZFS send/recv Built-in
Snapshots yes yes yes yes
Backup API CBT (Changed Block Tracking) Hyper-V WMI / RCT Proxmox Backup Server Native
GPU passthrough vGPU (NVIDIA Grid) DDA VFIO passthrough GPU passthrough
Licensing Per CPU / subscription Windows Server license Open source (free) Per node subscription

OpenStack

  • Distributions: Red Hat OpenStack, Canonical Charmed OpenStack
  • Services: Nova (compute), Cinder (block), Neutron (networking), Glance (images), Swift (object)
  • Use case: Telco, large private clouds, MNO (MANO, NFVI)
  • Complexity: High — complex deployment and maintenance

Variant Hypervisor Configurations by Size and Storage Type

Platform Selection by Use Case

Use Case Primary Choice Alternative Rationale
VMware shop, enterprise vSphere 8/9 Hyper-V Most comprehensive ecosystem, vSAN, SRM, broadest ISV support
Microsoft shop, Azure hybrid Hyper-V / Azure Stack HCI vSphere Windows Server CAL already in place, S2D, Azure Arc, native Hyper-V Replica
SME / low budget Proxmox VE XCP-ng / Hyper-V (free) Open source, built-in Ceph, ZFS, PBS, no license costs
HCI greenfield Nutanix AHV VMware vSAN All-in-one, simple management, built-in DR and backup
Hyperscale / telco OpenStack (RHOSP) — Multi-tenancy, NFVI, MANO, Neutron SDN, Ceph integration

Variant A: Small Deployment (2-3 hosts, local storage)

For small companies, branch offices, edge, dev/test. No shared storage — HA provided at the application level or via VM replication.

Parameter Proxmox VE VMware vSphere Hyper-V
CPU 1× EPYC 9124-9224 / Xeon 4410Y (8-16C) 1× EPYC 9124-9224 / Xeon 4410Y 1× Xeon 4410Y / EPYC 9124
RAM 64-128 GB (DDR5-4800, 1DPC) 64-128 GB 64-128 GB
OS disk 2× SATA SSD RAID1 (240-480 GB) 2× SATA SSD RAID1 2× SATA SSD RAID1
VM storage ZFS RAID10 (4-6× NVMe/SATA SSD) VMFS local (4-6× SSD RAID5/10) ReFS CSV (4-6× SSD RAID10)
Network 2× 10/25 GbE LACP 2× 10/25 GbE LACP + management 2× 10/25 GbE LACP
Management Proxmox web UI (1× node) vCSA / vCenter (1× appliance) Windows Admin Center / SCVMM
HA Proxmox HA (watchdog, fencing) vSphere HA (1 host failure) Hyper-V HA (WS Failover Cluster)
Backup Proxmox Backup Server Veeam B&R (Community) Windows Server Backup / Veeam
License Free (support ~€500/host/year) vSphere Essentials (~$600/3 hosts) Windows Server Standard (2 VMs)

Use case: Startup, branch office, dev/test, < 200 VMs, no SAN, minimal budget.

Advantages: Low cost, simple management. Disadvantages: Limited scalability, host failure = VM unavailability.

Variant B: Medium HCI (3-6 hosts, vSAN / Ceph)

Hyperconverged infrastructure — storage runs on the same hosts as VMs.

Parameter VMware vSAN Proxmox + Ceph Nutanix AHV
CPU 1-2× EPYC 9334-9654 (16-32C) 1-2× EPYC 9224-9334 (12-24C) 1-2× EPYC 9334-9654
RAM 256-512 GB 128-256 GB 256-512 GB
Cache tier 1-2× NVMe cache (write buffer) — (Ceph uses RAM/OSD) 1-2× NVMe (oplog)
Capacity tier 4-8× SSD (SAS/SATA) 4-8× HBA NVMe/SSD (OSD) 4-6× SSD (extent store)
Network 4× 25 GbE (vSAN + VM + mgmt) 4× 25 GbE (Ceph public + cluster) 4× 25 GbE (storage + VM)
Fault domain Rack awareness (3 racks min) CRUSH rack level Rack awareness
Replication RAID-1 mirroring (FTT=1) 3× replication / EC 8+3 2× copies + EC
Dedupe/Compress Dedup + compression (capacity) ZFS / Ceph compression (inline) Inline compression
HA limit 1-3 host failures 1-2 host failures (replication) 1-2 host failures
Min. hosts 2 + witness 3 (MON + OSD) 3

Use case: Medium company, VDI, general virtualization, 50-500 VMs.

Recommendation: For vSAN → min. 4 hosts for FTT=1 with erasure coding. For Ceph → min. 3 hosts, ideally 5+, each OSD host = 1 OSD per NVMe for maximum IOPS.

Variant C: Enterprise FC SAN (6+ hosts)

Classic 3-tier architecture — compute (hosts) + storage (SAN) + network separated.

Parameter VMware vSphere Hyper-V
CPU 2× EPYC 9654-9965 (32-64C) 2× EPYC 9654-9965 / Xeon 8592+
RAM 512-2048 GB (DDR5) 512-2048 GB
OS disk 2× SATA SSD RAID1 (480 GB) 2× SATA SSD RAID1
Storage FC SAN LUN (2× FC HBA 32/64G) FC SAN LUN or CSV over SMB
App network 2-4× 25/100 GbE LACP 2-4× 25/100 GbE LACP
Storage network 2× FC 32/64G (multipath) 2× FC 32/64G or SMB Multichannel
vMotion / Live Migration 2× 25 GbE dedicated (vMotion) 2× 25 GbE dedicated (SMB/RDMA)
Management vCenter (VCSA), NSX, Aria SCVMM, Azure Arc
Cluster max 64-96 hosts (vSphere 8/9) 64 hosts (WS 2025)
Admission control 1-4 host failures Nodes reserve
DRS / Balancing DRS (fully automated) SCVMM / AKS load balancing

Use case: Enterprise, databases, critical applications, 500-5000 VMs.

Storage variants: FC SAN (lowest latency), iSCSI (lower CAPEX), NFS (simpler management).

FC SAN topology:

                 ┌─────────────────────────────────────┐
                 │              FC Fabric              │
                 │   ┌─────────┐       ┌─────────┐    │
                 │   │ Switch 1│       │ Switch 2│    │
                 │   └────┬────┘       └────┬────┘    │
                 └────────┼─────────────────┼──────────┘
                    ┌─────┴─────┐     ┌─────┴─────┐
               ┌───┤ FC HBA 1  ├─┐ ┌─┤ FC HBA 2  ├───┐
               │   └───────────┘ │ │ └───────────┘   │
            ┌──┴──┐          ┌──┴──┴──┐          ┌──┴──┐
            │Host1│          │Host2   │          │Host3│ ...
            └─────┘          └────────┘          └─────┘

Variant D: Hyperscale OpenStack (20+ hosts)

For telco, large private clouds, MANO/NFVI environments.

Parameter Red Hat OpenStack Canonical Charmed OpenStack
Compute Nova + KVM Nova + KVM
Storage Ceph (Cinder/RBD) + Swift Ceph + Swift
Network Neutron + OVN/OVS + DPDK Neutron + OVN/OVS
CPU per host 2× EPYC 9654-9965 (64-128C) 2× EPYC 9654-9965
RAM per host 512-1024 GB 512-1024 GB
Storage per host Ceph OSD (4-12× NVMe/SSD) Ceph OSD
Network per host 4-8× 100 GbE (DPDK/VPP) 4× 100 GbE
Control plane 3-9× control node (HA) 3-7× control node
Orchestration TripleO / OpenStack Kolla Juju + charms
SDN OVN, OpenDaylight OVN
NFVI ready Yes (SR-IOV, NUMA, huge pages) Yes
Min. size 9 nodes (3 ctl + 3 compute + 3 ceph) 7 nodes

Use case: Telco (5G UPF, MNO), hyperscale private cloud, > 5000 VMs.

Connectivity Summary by Platform

Platform App / VM Network Storage Network Replication / HA Management
Proxmox small 2× 10/25 GbE LACP — (local ZFS) — 1× 1 GbE
vSAN (3-6) 2× 25 GbE LACP 2× 25 GbE (vSAN) vSAN traffic 1× 1 GbE
Proxmox Ceph (3-6) 2× 25 GbE 2× 25 GbE (Ceph public) 2× 25 GbE (Ceph cluster) 1× 1 GbE
Nutanix (3-6) 2× 25 GbE Dedicated storage VLAN Replication traffic 1× 1 GbE
vSphere FC SAN (6+) 2-4× 25/100 GbE LACP 2× FC 32/64G multipath 2× 25 GbE (vMotion) 1× 1 GbE + SAN mgmt
Hyper-V FC SAN (6+) 2-4× 25/100 GbE LACP 2× FC 32/64G or SMB 2× 25 GbE (Live Migration) 1× 1 GbE
OpenStack (20+) 2-4× 100 GbE 2× 100 GbE (Ceph) 2× 100 GbE (OVN) 1× 1 GbE

Resources

Links, books and standards: sources/infrastructure/sources.en.md

Recommended Reading

Book Authors ISBN Description
Virtualization Essentials (3rd ed., 2023) Matthew Portnoy 978-1119481513 Practical guide to virtualization: from hypervisor basics (Type 1/Type 2), VM configuration (CPU, memory, storage, networking) to cloud computing and DevOps. "Learning-by-doing" approach with tutorials. Author is a Senior System Engineer at VMware/Splunk.
VMware vSphere Design (2nd ed.) Guthrie, Lowe, Coleman 978-1119130312 Comprehensive guide to vSphere infrastructure design: hardware selection, network layout, security, storage and hypervisors. Describes a framework for design, decision analysis and best practices from experienced VMware architects.

Last revision: 2026-06-04

Reference in New Issue View Git Blame Copy Permalink