0ms Latency Dreams vs. NPU Reality: Engineering a Production-Ready AI Search with Gem nano

While "Hello World" demos make On-Device AI look effortless, shipping it into a production app like Strollby reveals a different reality. Traditional search filters are predictable, but they lack the nuance of human intent.

In this session, we go behind the scenes of how we replaced rigid UI filters with an AI-driven Search Parser using the ML Kit Prompt API.We will walk through a candid production case study, documenting the journey of optimizing our natural language entity extraction. You’ll see exactly how we slashed inference time from a sluggish 18.3s to a lightning-fast <1s by mastering the "Art of Prompting".

Key Production Insights:

The Prompt Evolution: Compare the "Polite Conversation" approach (18s) vs. the "Structured Constraint" approach (5s).

Zero-Inference Latency: Implementing Prefix Caching to skip static instructions and jump straight to the user query. (Techniques for passing relative time (like "tomorrow") by injecting a dynamic TODAY_DATE context into cached prefixes.)

The Logic Bridge: Using Kotlin to chain calls and map Nano’s JSON output directly to our experience booking modules.

Streaming for Speed: Using Kotlin Flows and generateContentStream to build a responsive UI that feels instantaneous to the user

A Production Reality Check: Addressing the "Fragmented Availability" and "Inconsistent Behavior" across the current Android AI ecosystem.

How we track down the failing conditions and integrated a hybrid architecture as fallback upon user feedback.

From Component to Catalog: Multiplatform Design Systems with Storytale

Building reusable UI components in Jetpack Compose is powerful, but previewing, testing, and cataloging those components across a growing team can quickly become a bottleneck. Enter Storytale an incubator Gradle plugin from JetBrains designed to bring a true "Storybook" experience to the Compose Multiplatform ecosystem.

In this session, we’ll explore how Storytale enables developers to build composables in complete isolation by automatically generating an interactive gallery of project components. We will walk through:

Setting the Stage: Configuring the Storytale plugin and structuring your commonStories source sets for Compose Multiplatform projects.

Crafting Stories: Writing isolated states for your UI elements, from simple buttons to complex, data-rich layouts.

The Developer Workflow: Leveraging gallery generation to speed up UI iterations, review cycles, and bridge the gap between design and engineering.

Real-World Scaling: Why isolated component development is a game-changer when managing large, unified design systems across multi-module applications.

Whether you're migrating an existing architecture to Compose or building a multiplatform design system from scratch, you'll walk away from this session with a practical blueprint for integrating Storytale into your pipeline to build highly maintainable, testable UIs.

Agentic Android: Exposing and Consuming Capabilities with App Functions

Android 16 marks a massive shift in how the OS interacts with installed applications. With the introduction of the android.app.appfunctions package, we are moving away from the rigid deep-linking of App Actions and Slices toward a truly agentic model. Apps are no longer just destinations to be opened; they are headless toolboxes of capabilities that system agents can invoke directly.

In this session, we will move beyond the documentation and dive into the practical implementation of App Functions. We will explore how to wire up an app to expose its features using the new Jetpack androidx.appfunctions library, focusing on the @AppFunction annotation and secure cross-process data handling with AppFunctionSerializable.

Finally, we will look at the other side of the equation: consuming these capabilities. I will demonstrate how to build a custom Android launcher that queries AppFunctionStaticMetadata and natively executes tasks from third-party apps via the AppFunctionManager — effectively turning a standard UI into an AI-ready intent engine.

Key Takeaways:

The Agentic OS Shift: Understand the architectural differences between legacy App Actions and the new headless execution model of App Functions.

Exposing Capabilities: Learn how to implement the androidx.appfunctions Jetpack library, handle KSP code generation, and manage the PlatformAppFunctionService system entry point.

Schema & Serialization: Best practices for defining structured parameters that system agents can easily index and consume.

Live Launcher Demo: A practical walkthrough of querying the on-device index and executing App Functions dynamically from a custom Android launcher.

Architecting the Shared Layer: GraphQL Best Practices in Kotlin Multiplatform

Implementing Clean Architecture in Kotlin Multiplatform (KMP) requires a robust data layer that abstracts complexity away from the UI. When adding GraphQL into the mix, the challenge lies in maintaining strict separation of concerns while leveraging the power of the graph across Android and iOS. This lightning talk explores architectural patterns for integrating Apollo Kotlin directly into the commonMain source set.

We will examine how to treat the KMP shared module as the authoritative source of truth, ensuring that platform-specific UIs remain purely reactive.

Takeaways

Shared Normalized Cache: Leveraging Apollo's cache within the shared Data layer to provide a unified, offline-first source of truth that keeps Android and iOS states perfectly synchronized.

Schema-Driven Modularity: Strategies for isolating generated GraphQL network models within the Data layer and using mappers to expose pure Domain entities, preventing API details from leaking into business logic.

Type-Safe Error Boundaries: Implementing a unified Result wrapper pattern that intercepts GraphQL partial failures in the Shared Repository, translating them into clean Domain states for safe UI consumption

Building Private Brain on Android: Offline RAG with Vector Databases & MediaPipe

Privacy-conscious users and business needs are pushing a shift toward "Local-First AI." Running a large language model (LLM) on a device is a good beginning, but the real challenge is making that model smart with your own private data without using the cloud. This session looks at the setup of Offline Retrieval-Augmented Generation (RAG) on Android.

We will explore how to implement a local vector search using high-performance databases like ObjectBox or Couchbase Lite to store and query embeddings. You will learn how to create a smooth pipeline that sends real-time local information into open models (on device ), such as Gemma, through the MediaPipe LLM Inference API. We will also address an important architectural question: when should you invest in LoRA fine-tuning instead of choosing the flexibility of RAG?

Key Takeaways:
- Architecting Local RAG
- Vector DB Implementation
- Real-time Context Injection
- LoRA vs. RAG
- Performance Optimisation

Write Once, Query Everywhere: Designing a Shared GraphQL Architecture in Kotlin Multiplatform

As Kotlin Multiplatform (KMP) grows, the challenge moves from sharing simple utility code to handling complex data layers across Android, iOS, Desktop, and Web. GraphQL provides a type-safe and efficient solution for modern APIs. With Apollo Kotlin’s strong multiplatform support, developers can now share all their networking and data logic.

In this session, we will focus on how to implement Apollo GraphQL in a KMP architecture. Using our experience in building scalable Android apps with Jetpack Compose, we will look at how to apply those skills in a multi-platform setup.

Key takeaways include:

- Architecture: Setting up a shared data module that serves Android (Compose), iOS (SwiftUI), and Desktop.

- Type-Safety: Using Apollo’s code generation to keep a single source of truth for your API schema across all platforms.

- Best Practices: Managing caching, authentication headers, and reactive queries with Kotlin Coroutines and Flow.

- Real-world hurdles: Handling platform-specific configurations and improving network performance for a consistent user experience.

Beyond the Grid: Crafting a Custom Android Launcher from Scratch

As the creator of Simple Launcher, a minimalist text-based launcher, I've navigated the unique challenges of replacing one of Android's most fundamental components. This lightning talk will demystify the process of building a custom launcher experience that sits at the powerful intersection of the Android system and the user.

We'll cut through the complexity and get straight to the essentials:

What is a launcher, how it sit between us and android system? How can we build a custom launcher? Things to consider while building custom experience !!

The Core Foundation: We'll explore how to leverage key Android APIs like PackageManager to query and manage installed applications, and how a Clean Architecture approach can keep your launcher robust and maintainable.

Staying in Sync: A launcher must be a living part of the system. I'll demonstrate how to effectively use BroadcastReceivers to respond instantly to app installations, updates, and uninstalls, ensuring your UI is never out of date.

Navigating the Maze: Building a launcher isn't just about code; it's about compliance and compatibility. We'll cover the necessary permissions (including the crucial QUERY_ALL_PACKAGES) and discuss the real-world complexities of dealing with OEM-specific quirks and Android version fragmentation.

The Accessibility "Hack": In a surprising twist, I'll share how I creatively leveraged Accessibility Services—often a tool for assistance—to implement powerful features like an app blocker, and discuss the ethical considerations and pitfalls of this unconventional approach.

Build Your Pocket Brain: Open On-Device LLMs via MediaPipe on Android

MediaPipe's new LLM Inference API is leading the charge for Android developers. beyond simply running pre-trained models like Gemma on a device, we will dive deep into the power of customisation using Low-Rank Adaptation (LoRA), a parameter-efficient fine-tuning (PEFT) technique.

You'll learn the end-to-end workflow for creating a specialised, on-device LLM tailored to your app's unique domain. We will cover how to take a base model (like Gemma or Phi-2), fine-tune it with your own dataset using the PEFT library in Python, convert both the base model and the LoRA weights into the MediaPipe-compatible FlatBuffer format, and finally, integrate this custom-tuned model into an Android application.

we will demonstrate how to configure LlmInferenceOptions in kotlin to load both the base model and the .tflite LoRA file, unlocking hyper-personalised AI experiences that are fast, offline-capable, and completely private.

Key Takeaways

* Understanding when to use MediaPipe LLM Inference / gemini nano on-device generative AI solutions.

* Setup and configure MediaPipe LLM Inference for on-device generative AI.

* Expertise in LoRA fine-tuning to adapt LLMs like Gemma-2B or Phi-2 for specific use cases cost-effectively.

* Dive into Configuration options and multimodal prompting.

* Knowledge of deployment workflows, GPU-accelerated LoRA inference, and ethical AI practices that we should follow.

Supercharging Android Apps with On-Device AI: Gemini Nano & Prompt API

Mobile AI revolution is increasingly moving on-device, driven by demands for privacy, low latency, and offline capability. In this session, I’ll demonstrate how to leverage cutting-edge on-device AI tools including Gemini Nano, Prompt API, ML Kit Gen AI API and MediaPipe LLM Inference APIs to build intelligent Android apps entirely on-device.

Key Takeaways:

* On-Device AI Landscape - Understand the shift from cloud to on-device AI, the privacy benefits, and real-world use cases for features like smart reply, summarisation, image analysis, and breaks down how Android abstracts on-device intelligence through AI Core, LoRA fine-tuning, Private Compute, and hardware accelerators like NPUs..

* Getting Started with Gemini Nano - Walk through integrating Google’s Gemini Nano generative model into a modern Android app, highlighting both ML Kit Gen-AI APIs and Prompt API for custom scenarios.

* Prompt API demo - Through real-world demo of AI-driven activity booking search system built on Gemini Nano using prompt api (optimising using prompt design strategies and best practice from 18s -> 3s ), shows how to design, optimize, and productionise on-device AI features for modern Android apps.

* Production Considerations - Address model size, device compatibility, privacy, and performance optimisation lessons learned from deploying AI features at scale in consumer and enterprise Android apps.

* Beyond Gemini - MediaPipe, LiteRT, and Custom Models: Explore the MediaPipe ecosystem for LLM (large language model) inference on-device, and how to bring your own models using LiteRT/TensorFlow Lite for specialised tasks.

Session presented on Droidcon Uganda 2025 on 10th November.