The future of AI inference lies not only in the cloud—execution on end devices is becoming a reality.

Modern computers already feature powerful NPUs (e.g., Qualcomm Snapdragon), and manufacturers such as Intel and AMD are ready to launch their own solutions. This hardware opens up new possibilities for high-performance on-device AI. One of the major trends in the dynamic world of AI is becoming tangible.

In this presentation, Aydin will show how “Microsoft Foundry Local” can be used to leverage large language models (LLMs) in new ways: with privacy, sustainability, and cost efficiency – and how we can integrate this technology into our own applications.

Foundry Local offers a coordinated solution: from the runtime environment (Windows ML, ONNX Runtime, Genie) and execution unit (NPU, GPU, CPU) to optimized models. We will talk about architectures, model transformation, tool calling, and APIs, as well as the limitations and potential of on-device AI.

Takeaways for participants
• Understand why on-device AI is a game changer.
• Gain insight into the possibilities of Microsoft Foundry Local.
• Practical tips for integrating LLMs into your own applications.
• Implementation of tool calling and local inference.
• Clear overview of the potential and limitations of on-device AI.