Patients are increasingly turning to AI systems and large language models (LLMs) for medical guidance before engaging with traditional healthcare channels. However, these systems do not merely answer questions—they shape recommendations by selecting, ranking, filtering, and prioritizing information about symptoms, treatments, medications, and healthcare providers.

Behind every AI-generated response lies a set of hidden control points: training data, retrieval-augmented generation (RAG) pipelines, system prompts, and safety and alignment layers. These components collectively determine what information is surfaced, what is omitted, and how recommendations are framed. Despite their growing influence on healthcare decisions, these mechanisms remain largely opaque and difficult to audit.

This session explores how these hidden recommendation layers operate, where control and influence exist within AI systems, and why emerging risks such as prompt injection, content poisoning, and recommendation manipulation have direct implications for patient safety and trust. It also examines the widening governance gap created by AI-driven healthcare advice, where transparency, accountability, and auditability have not kept pace with adoption.

The discussion ultimately addresses a critical question for healthcare organizations, regulators, and technology providers: Who controls what AI recommends, and how can these hidden control points be made transparent, auditable, and accountable in healthcare?