Session
Curl a GPU Into Your Cluster: Hybrid Kubernetes for AI Development
AI work puts a model and a GPU inside the developer's inner loop, and a laptop cannot hold the GPU. This talk shows a platform pattern that keeps the fast parts of the loop local and brings GPU capacity in from outside. A vind cluster (vCluster in Docker) runs the control plane and workers as containers on the laptop, with LoadBalancers and images served through the laptop's Docker. A cloud VM with an NVIDIA T4 then joins the same cluster as a private node: one curl command installs kubelet and connects it over a WireGuard VPN, so the same kube context reaches local workers and the remote GPU.
The demo is live. A Stack declared in vcluster.yaml waits for the GPU node, loads a 12B model into the T4's memory behind Ollama, and runs a health check before developers get the endpoint. The audience then calls the model from their phones.
We close the loop by moving the app to Git and Argo CD, and with what self-service GPU access still needs: auto nodes for on-demand capacity, tenant boundaries, quotas, idle GPU cost, and telemetry.
Piotr Zaniewski
🌐 Cloud Platforms Architect | 📢 Speaker ✍️ Blogger | 🚀 Digital Transformation
Frankfurt am Main, Germany
Links
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top