What happens when an AI agent can observe the physical world, not just documents?

In this session, you will see how to build a troubleshooting experience that combines Copilot Studio, Raspberry Pi, Azure IoT, and Power Pages into a single intelligent system.

The demo centers on a physical device containing an array of lights and switches that represent a malfunctioning system. The objective is to restore the system by placing the switches in the correct configuration. Attendees will be able to interact with the device live during the session, toggling switches and observing how the system responds. In many real environments, technicians and operators must rely on large binders or lengthy procedures to diagnose problems. Finding the correct steps takes time, mistakes are easy to make, and delays can increase risk, downtime, or cost.

Mounted with the device is a user panel displaying a web portal built with Power Pages that remains open at all times, acting as an on-demand support interface. The embedded Copilot agent is grounded in the manual’s procedures while also receiving live telemetry from the device through the Raspberry Pi and Azure IoT, including the current state of lights and switches. By combining documented knowledge with real-time context, the agent guides users step by step toward the solution. The portal also visualizes the device state as it changes in response to physical interaction.

This session explores the architecture, agent design, IoT integration, and user experience decisions required to make AI guidance actionable, not just informative. Attendees will leave with practical patterns for building agents that interact directly with physical systems in manufacturing, facilities, field operations, training environments, and other high-impact scenarios.