Privacy-conscious users and business needs are pushing a shift toward "Local-First AI." Running a large language model (LLM) on a device is a good beginning, but the real challenge is making that model smart with your own private data without using the cloud. This session looks at the setup of Offline Retrieval-Augmented Generation (RAG) on Android.

We will explore how to implement a local vector search using high-performance databases like ObjectBox or Couchbase Lite to store and query embeddings. You will learn how to create a smooth pipeline that sends real-time local information into open models (on device ), such as Gemma, through the MediaPipe LLM Inference API. We will also address an important architectural question: when should you invest in LoRA fine-tuning instead of choosing the flexibility of RAG?

Key Takeaways:
- Architecting Local RAG
- Vector DB Implementation
- Real-time Context Injection
- LoRA vs. RAG
- Performance Optimisation