Security products now promise autonomous analysts, intelligent triage, predictive detection, AI-generated remediation, agentic response, and machine-speed investigation.

But what is the system actually doing?

Behind the AI label may be deterministic rules, classical machine learning, retrieval, an LLM summarizer, a scripted workflow, hidden human review, or a model operating within conditions far narrower than the product demonstration suggests.

This session presents a technical methodology for testing those claims.

We will decompose AI-enabled security systems into their data sources, model boundaries, decision authority, orchestration layers, human dependencies, and failure paths. A demonstration harness will test representative security workflows for hallucination, nondeterminism, prompt sensitivity, adversarial inputs, weak ground truth, out-of-distribution behavior, latency, privacy exposure, and silent dependence on manual intervention.

The session introduces the AI Security Capability Verification Matrix, a reusable framework for determining what a product detects, predicts, generates, recommends, or actually controls.

Rather than asking whether a security product “uses AI,” attendees will learn to ask which decisions the model makes, what evidence supports them, how failures become visible, whether results can be reproduced, and who must intervene when the system is wrong.

Attendees will leave with practical tests for separating genuine capability from automation, rebranding, and polished theater.

Session format

Conference session

Level

300: Advanced

Session duration

45 minutes

Technical content
Deterministic rules versus statistical models
Classical machine learning versus generative AI
Retrieval-augmented generation
Tool use and agentic orchestration
System boundaries and decision authority
Benchmark and ground-truth design
Hallucination and unsupported inference
Nondeterminism and reproducibility
Prompt sensitivity and prompt-injection exposure
Adversarial and out-of-distribution testing
Hidden human-in-the-loop dependencies
Model drift and update risk
Data retention and training exposure
Latency, cost, and failure under realistic workloads
Logging, auditability, rollback, and escalation
Demonstration

A vendor-neutral evaluation harness will test several representative AI-security behaviors, such as:

Alert summarization
Incident classification
Threat prioritization
Recommended remediation
Natural-language investigation
Agentic task execution

The demonstration will intentionally vary evidence quality, prompt wording, attack context, and unavailable ground truth to show where apparent capability breaks.

Attendee takeaways
Distinguish rules, automation, machine learning, generative AI, and agentic behavior
Build realistic tests for hallucination, nondeterminism, prompt sensitivity, privacy, latency, and hidden human intervention
Use the AI Security Capability Verification Matrix during technical reviews, proofs of concept, and architecture decisions
Original framework

The verification matrix evaluates systems across six layers:

Input integrity: What evidence reaches the system, and what can contaminate it?
Model behavior: What is predicted, generated, classified, or inferred?
Decision authority: Does the system advise, approve, initiate, or act?
Failure visibility: How do users know the system is wrong?
Human dependency: What review or correction remains hidden?
Operational consequence: What happens when the output is trusted?
Speaker notes

The session is vendor-neutral. It will not name or ridicule individual products. The focus is a reproducible technical method attendees can apply to any AI-enabled security system.