In this talk, we will demonstrate how to set up your own private, on-premise, container-based LLM with Terraform and Kubernetes. We will delve deeper and show how to add gigabytes of documents and interact with your private, sensitive data. During the live demo we will also show how to connect it to HolmesGPT and let the private LLM auto solve your Kubernetes issues.

Source Code: https://github.com/iits-consulting/otc-terraform-template/tree/ollama

Keywords: LLM, HolmesGPT, OpenTelekomCloud, Kubernetes, Terraform, AI, Ollama, Airbyte, RAG