Context and Background
As language models get smaller and more efficient, the frontier of innovation shifts from massive cloud inference to edge-level intelligence. Google’s Gemma models open new possibilities for developers to fine-tune and deploy LLMs directly on resource-constrained environments. In this session, I’ll share my experience building an end-to-end workflow — from lightweight model adaptation using LoRA/PEFT to evaluating performance locally with Ollama and deploying the resulting model seamlessly to an Android app.

Session Abstract
This session demonstrates how to efficiently fine-tune and deploy a small language model for real-world mobile use. We’ll explore how to adapt Gemma using parameter-efficient fine-tuning (PEFT) on a sample dataset, validate it in a local inference environment, and run it natively on Android. The talk blends technical insight with practical deployment lessons — perfect for developers aiming to bridge the gap between model training and mobile integration.

What the Session Covers
• Choosing Gemma as a base model for mobile-scale applications.
• Applying LoRA/PEFT for efficient, low-compute fine-tuning.
• Setting up evaluation and benchmarking using Ollama.
• Integrating the model into an Android app, covering quantization, inference flow, and performance optimization.
• Lessons learned from data preparation, adapter merging, and balancing model quality vs. latency.

Key Takeaways
• A practical roadmap from fine-tuning → evaluation → Android deployment.
• How to achieve meaningful LLM adaptation without large compute resources.
• Design strategies for on-device AI with low latency and strong UX.
• A real example of bringing small language models to production at the edge.