← All posts

Private Bare Metal Cloud V1

At Whyle Labs, we are interested in reducing our reliance on large tech companies - especially those who have become gatekeepers. This means fewer companies that we need to seek approval from and more independence.

The long-term vision

A proper private bare-metal cloud includes:

  • Owning your own machines (either in a colocation or your own building)
  • Having IP transit or IP peering with multiple providers (for reliability, speed and contract bargaining power)
  • Owning your own edge routers
  • Announce your own IP blocks with BGP
  • Running your own Anycast DNS servers
  • Running your own private container registry and cache of public images
  • Stateful workloads (databases, block storage, object storage) are backed by these machines, including (most) backups
  • Identity and Access Management are managed internally, not dependent on an external service
  • Securing data on the machines with TPMs and Secure Boot

Current status

As of July 2026, Whyle Labs is still pretty far away from this, specifically:

  • Using rented dedicated machines from OVH
  • OVH announces our IP space for us, though we manually route our IPs to specific nodes (toilsome)
  • Using external DNS
  • Using an external private container registry and directly referencing public images
  • Our database runs directly on our dedicated machines, using local disk with SecureBoot and TPMs though we do not use block or object storage yet
  • Using an external Identity and Access Management

But even with this setup, other than the on-going physical maintenance, it would now be feasible to deploy to a colocation without too many more changes.

Operating System and Deployment

While this deserves its own blog post, it has been awesome to use Talos - a minimal Linux distribution made for declarative state and Kubernetes (there is no SSH, for instance).

The rough steps are:

  1. Provision a dedicated machine
  2. Install the Talos OS (either over the network with IPMI or bootable media if you are next to the server)
  3. Add the IP address of the machine to your Pulumi/Terraform configuration to bootstrap the cluster or add the machine to an existing cluster
  4. Use that cluster in your Pulumi to deploy Kubernetes deployments and such

There are some gotchas, like the dance to set-up TPM and SecureBoot and how to handle loadbalancers on your private cloud.

Next Steps

Currently all routing is done by static (but Pulumi controlled) DNS settings. Once we drop the dependency on external DNS, we would also start doing things like health checks for our DNS. Of course external providers can do health checks too, but we are looking for fewer dependencies.

Some of the trickiness for DNS that we need to handle:

  • Support cert-manager webhook for TLS certificates
  • Be able to route across multiple DNS servers with a single IP via Anycast
  • Support for health checks
Consulting

Need extra hands on your next build?

We take on select consulting engagements - architecture reviews, hands-on implementation, or unblocking a stuck project. Book a free intro call.

Book a free call