Robotic Process Automation is difficult to get right. Current tools rely on specifying the “How”: Write text into a textfield for example. Instead, you should specify the “What”: Fill out the form with the data. Automation is therefore often implemented by specialized teams with technical expertise but lack domain knowledge. To add to this, tools often only target specific platforms and operating systems. This leads to automation that is brittle, and flaky and relies on outside support for maintenance.

This talk will present the bright future of RPA due to current advancements in Artificial Intelligence and Computer Vision. In the future, domain experts will formulate automation as an intent in the form of natural language and let the tool figure out a way to achieve the desired outcome with the appropriate actions. This will break the dependence on tool specialists and empower domain experts to automate workflows autonomously, which is cost-effective and faster.

A live demo will show the practical implications by leveraging GPT for task understanding, Computer Vision models for object detection, and human-like interaction with the operating system to write a poem about RPA and save it to a desktop application. All of this is initiated by a single prompt and no further user interference.