As AI becomes increasingly embedded in our daily lives and critical systems, the need for data sovereignty and compute efficiency is more urgent than ever. Cloud-based AI models, while powerful, often raise concerns about privacy, control, and long-term sustainability — especially in regions with strict data regulations or limited internet access.

In this talk, I’ll demonstrate how developers can take back control by fine-tuning and deploying small yet capable language models (like Qwen3) completely offline, using fully open-source tools. We’ll cover the full pipeline — from fine-tuning on CPU-based hardware to exporting optimized GGUF models and running them on edge devices using llama.cpp. I'll also showcase a real-world use case where these models are integrated into an Android app to perform tool-calling tasks (e.g., setting alarms, sending WhatsApp messages) without ever needing to touch the cloud.