As patients increasingly turn to large language models (LLMs) and AI assistants for medical guidance before consulting traditional healthcare channels, a new governance challenge is emerging. These systems do far more than generate answers—they actively influence decisions by ranking, filtering, and prioritizing medical information, treatment options, and healthcare providers.

Beneath every AI-generated response lies a complex recommendation stack shaped by training data, retrieval mechanisms, system prompts, safety guardrails, and alignment layers. This hidden recommendation layer determines what information is surfaced, suppressed, or emphasized, yet it remains largely opaque and difficult to audit.

The session examines how AI recommendation systems in healthcare operate, where control points exist, and why vulnerabilities such as prompt injection, content poisoning, and recommendation manipulation represent more than technical risks—they directly impact patient safety, trust, and clinical outcomes. It also explores the growing governance gap created by AI-driven healthcare advice, where accountability and transparency mechanisms have not kept pace with adoption.

Ultimately, the discussion raises a fundamental question for healthcare leaders, regulators, and technologists: Who controls what AI recommends in healthcare, and how can those decisions be made transparent, auditable, and accountable?