On-device AI is slowly but surely becoming a reality for LLM inference.
Modern computers already feature powerful NPUs, such as those from Qualcomm, Intel or AMD. This hardware enables sufficiently high-performance on-device AI. One of the major trends in the dynamic world of AI is becoming a tangible reality.

The advantages are clear: AI becomes significantly more sustainable, privacy is automatically guaranteed, latency is very low, and costs are negligible.

In this talk, Aydin demonstrates how to build AI applications for this new approach using ‘Microsoft AI Foundry Local’. Foundry Local offers a fully integrated solution: from the runtime environment (Windows ML, ONNX Runtime, Genie) and execution units (NPU, GPU, CPU) right through to curated and optimised models. We will discuss architectures, model transformation, tool calling and APIs, as well as the limitations and potential of on-device AI.

Using practical examples, we will learn about the integration of MCP servers, the use of the inference API, and an outlook on the integration of RAG.

This presentation focuses on aspects of on-device AI; that said, this also applies to the use of dedicated hardware on-premises (local AI).

I am an experienced speaker in the field of .Net, Cloud and Green Software.
Many years of repeated presentations on:
Developer Week (https://www.developer-week.de/)
We Are Developers (https://www.wearedevelopers.com/world-congress)
.NET Conf (https://www.dotnetconf.net)
IT-Tage (https://www.ittage.informatik-aktuell.de/)
NDC Oslo (https://ndcoslo.com)
Developer Day (https://entwicklertag.de/)
And many other community conferences and meetups