How to build a private cloud
What a private cloud really needs, the hardware and networking to plan, the software options, and a sensible order for building one on servers you own.
Updated
What counts as a private cloud
NIST defines a private cloud as infrastructure dedicated to one organization, on or off its premises, run by that organization or a third party. The important word in its definition of cloud computing is self-service: people get resources when they need them, without asking someone to do it by hand.
That is the usual gap. A cluster where an administrator creates VMs on request is useful virtualization, but it is not yet a cloud. Keep that test in mind for every decision below.
Decide these first
- Who uses it? One team has very different needs from many teams or outside customers.
- What do they need? VMs only, or also Kubernetes, load balancers, object storage and databases?
- Who runs it? Count the people and hours available for upgrades and on-call.
- What must it survive? The loss of a disk, of a server, or of a whole room lead to very different designs.
Plan the hardware and network
- Start with three servers if you want high availability, since clusters need a majority to agree on who is alive.
- Buy servers with a BMC (iLO, iDRAC or IPMI) so you can power and reinstall them remotely.
- Separate your traffic. Keep management, storage and workload traffic apart, on VLANs or separate interfaces.
- Choose local or replicated storage. Local disks are simple and fast. Replicated storage survives a server loss but needs fast networking, typically 10 Gbps or more.
- Keep hardware consistent where you can, because mixed CPUs and disks make capacity and migration harder.
Pick the software
| Effort to run | Good for | |
|---|---|---|
| Proxmox VE | Low | VMs and containers run by a small team of administrators |
| Harvester | Medium | Organizations already working in Kubernetes |
| OpenStack | High | Large multi-tenant clouds, with a platform team |
| OpenShift with Virtualization | High | Container platforms that also need to run VMs |
| CloudOps | Low; we run the control plane | Self-service VMs, Kubernetes, networks and storage for your teams |
For closer comparisons, see Proxmox vs OpenStack, Proxmox vs Harvester and OpenStack vs OpenShift.
A sensible build order
- Rack the servers, and confirm that remote power and the console work.
- Build the network, with the VLANs and addressing you planned.
- Install the operating system over the network with PXE, so a server can be rebuilt the same way later.
- Install your platform and form the cluster.
- Set up storage, then test what happens when a disk or a node fails.
- Create private networks and firewall rules.
- Prepare VM images, so nothing is built by hand.
- Add users, roles and quotas, which is where it starts to become a cloud.
- Describe resources in Terraform or through the API.
- Move one real but unimportant workload, and run it for a few weeks.
What people forget
- Upgrades. Rehearse one before you depend on the platform.
- Monitoring and alerting, including failing disks and fans, before users notice.
- Backups stored off the cluster, and a restore you have actually tested.
- Capacity. Self-service fills servers quickly, so quotas and growth planning matter.
If you would rather not build the control plane
CloudOps covers most of this list for servers you already own: network installs over PXE, self-service VMs and Kubernetes, private networks, security groups, load balancers and S3-compatible storage, from a console, an API and Terraform. We run the control plane as a service.
The trade-offs are real. CloudOps has no live migration between hosts and no automatic VM restart on another host, and you cannot install the control plane yourself, so it does not suit networks that must stay fully disconnected.