In the rapidly evolving landscape of incident management and response, PagerDuty stands at the forefront, integrating Generative AI (GenAI) features to enhance operational efficiency and user experience. This session delves into the intricate design and implementation of the API that power these GenAI features, focusing on the seamless integration of Large Language Models (LLMs) to automate and improve various aspects of the PagerDuty platform.

The core of our discussion revolves around the PagerDuty Copilot AI Assistant, which, through Slack integration, enables users to submit queries directly, thereby initiating the GenAI functionalities. At the heart of these operations is the Engine Core, which, upon receiving user queries, determines the appropriate task, fetches relevant data through the Data Fetcher from sources like the PagerDuty Operation Cloud and 3rd Party Services, and directs the request to the suitable backend service. This process is crucial for generating accurate, context-aware responses and actions.

Central to our discussion is the Data Science LLM API, which manages the interactions with various LLMs providers, such as OpenAI and AWS Bedrock. This API is pivotal in managing prompts, ensuring data security, and adhering to legal and security guidelines, thereby streamlining the implementation of GenAI features across the platform.

This session aims to provide a comprehensive overview of the backend processes and API designs that enable the GenAI features of PagerDuty, illustrating the platform's commitment to leveraging cutting-edge technology to enhance user experience and operational efficiency in incident management.