There's a Dan Garfield talk from KubeCon EU 2021 — "The Quest for the Ultimate Kubernetes Homelab" — that put an idea in a lot of platform engineers' minds: the cluster in your closet isn't a toy, it's where you learn what you do later at work. Five years later, the question has changed: can the homelab cluster now run a useful LLM, the same way it runs your media server and your password manager?

For me, yes. With three Kubernetes-native projects: HAMi for GPU sharing, Kaito for declarative model deployment, and LLM-d for KV-cache-aware routing. Do you have more than one Gaming PC? Maybe yours and you son's, or maybe you kept your older GPU? If you add them to your Kubernetes cluster, you are a couple of minutes away from your own a distributed inference endpoint.

Does it make sense against running ollama or LM Studio? I will give you an honest answer.

You'll leave knowing exactly what to install on your homelab tonight — and whether you should.