Context and Background
Generative AI has opened up new creative possibilities, but most applications rely on cloud-based models, which can introduce latency, privacy, and cost challenges. In this session, we explore how to shift from server-side story generation to efficient on-device personalization. By using Google’s Gemini API and Gemma models, we demonstrate a hybrid approach that maintains quality while optimizing for responsiveness, adaptability, and user control.

Abstract:
Join us as we journey from the cloud to your pocket, using Google’s Gemini API and Gemma on-device model to refine story generation. We’ll start with cloud-based story creation and then shift to running models directly on-device, showcasing how to enhance outputs with tuning techniques. By the end, you'll see how to measure and compare the results of each stage.

What the Session Covers:
This session dives deep into the practical use of language models for story generation, moving from a Gemini model to on-device Gemma models and refining them for richer output. We'll explore the transition to on-device models that can be progressively tuned.

* Starting with a Gemini API to generate initial stories.
* Transitioning to on-device models for offline and personalized story generation.
* Demonstrating three stages of on-device model use: the base model, a prompt-tuned version, and a fine-tuned version using adapter tuning.
* Evaluation methods to compare and measure the improvements in story quality across these different models.

Session Highlights:

* Practical steps for moving from API-based generation to on-device models.
* A detailed look at how prompt tuning and fine-tuning can enhance on-device model outputs.
* Simple evaluation metrics to assess the quality of generated stories.

Key Takeaways:
Attendees will gain insight into how to evolve from cloud-based to on-device models, how to refine model outputs through tuning, and how to evaluate the improvements effectively.