Smoke testing a portfolio of Shiny applications is a significant and often underestimated quality assurance challenge in pharmaceutical data science. Manual testing — navigating tabs, exercising dropdowns, sliders, and checkboxes, waiting for tables and plots to render, and consolidating findings into reports — is time-consuming, inconsistent, and difficult to scale. As Shiny app portfolios grow across clinical, statistical, and analytical workflows, teams face a difficult choice: invest increasing manual effort per release cycle or accept lower testing confidence.

This talk presents an open-source, agentic workflow that automates Shiny smoke testing using three complementary technologies:

smolagents — a lightweight Python framework for building LLM-orchestrated agents
Playwright — for robust, browser-level interaction with live Shiny applications
MCP (Model Context Protocol) server — a custom, domain-specific layer that exposes four core Shiny testing capabilities as reusable tools
The MCP server wraps Shiny-specific testing operations into four tools: SAML-based authentication into the hosting environment (saml_login_posit_connect), programmatic tab and section discovery (find_all_tabs_and_sections), systematic tab walking with screenshot capture (walk_all_tabs_with_screenshots), and automated HTML report generation (generate_test_report). An LLM-orchestrated agent calls these tools in sequence, visiting every tab, waiting for tables and plots to render, capturing visual evidence, and producing a structured test report — automatically and repeatably.

The talk will address:

Why existing R testing frameworks are not enough — testthat and shinytest2 are excellent for unit and snapshot testing within individual apps, but they were not designed for portfolio-level, cross-app smoke testing with authentication, tab orchestration, and consolidated reporting.

MCP vs Playwright MCP — why a domain-specific Shiny MCP provides more value than using a generic Playwright MCP directly, and how the two layers can work together effectively.

Architecture walkthrough — a clear, visual explanation of how the MCP server, agent, and Playwright automation layer fit together, and how the workflow bridges R Shiny applications with a Python-based testing stack.

Live demo — the agent running against a Shiny app in real time: one URL in, an HTML report and screenshots out.

LLM cost analysis — using Azure OpenAI models (GPT-5 and gpt-4o-mini), a full smoke test costs approximately USD 0.12 and USD 0.01 per app respectively, making LLM-assisted automated testing economically viable at any portfolio scale.

Roadmap — the path from smoke testing towards full end-to-end (E2E) Shiny testing, visual regression, and data-driven output validation.

Attendees will leave with a clear understanding of how to build agentic, MCP-based testing workflows for their own Shiny apps, and why this Python + R approach represents a practical, scalable solution to a real, recurring quality assurance problem in pharma data science.