Building a Native visionOS App with a Live Physiology Engine for Medical Training

Most medical simulation VR runs pre-scripted scenarios on mobile chips with handheld controllers. This talk covers how I built a fully native visionOS app for a major health system that runs a real-time physiology engine, where patient vitals respond dynamically to every trainee action.

The physiology engine powering the simulation is a C++ library, which meant bridging native Swift and visionOS code with C++ in a clean, performant way. Every Swift developer working with legacy systems, game engines, scientific libraries, or any third-party C++ dependency will be able to take this knowledge and apply it directly to their own projects.

The app uses native hand tracking and speech recognition that validates verbal announcements at each step to enforce AHA ACLS protocol compliance in real time. With limited documentation and few people who had tackled anything like this before, the Swift and visionOS community was one of the most important resources throughout the entire build.