Security vendors now promise AI analysts, autonomous triage, predictive detection, intelligent remediation, and machine-speed response. But beneath the label may be a rule engine, a legacy statistical model, an LLM summarizer, or a workflow still dependent on human operators.

This session shows how to test those claims.

We will break down the technical differences between rules, automation, classical machine learning, generative AI, and agentic systems, then examine how vendors blur those boundaries in demos and marketing. Attendees will learn how to design practical tests for hallucination, nondeterminism, prompt sensitivity, model drift, hidden human intervention, weak ground truth, out-of-distribution inputs, latency, privacy exposure, and failure under realistic workloads.

The session introduces an AI Security Capability Verification Matrix that teams can use during proofs of concept, architecture reviews, and purchasing decisions. Rather than asking whether a product “uses AI,” the framework asks what decisions the system actually makes, what evidence supports those decisions, how failures appear, and who must intervene when the model is wrong.

Attendees will leave with a repeatable method for separating genuine capability from automation, rebranding, and polished theater.

Session Format

1-hour technical talk

Topics

AI Security
Security Tooling
Security Testing
Architecture
Blue Team
GRC
Machine Learning
Vendor Evaluation

What will attendees walk away with?

• A practical method for distinguishing rules, automation, machine learning, generative AI, and agentic behavior
• Reusable tests for hallucination, nondeterminism, prompt sensitivity, privacy, latency, drift, and hidden human intervention
• A verification matrix for evaluating AI security tools during proof-of-concept testing and procurement

Level

Intermediate

Can we record your talk?

Yes.

First-Time Speaker?

No.

Can you present in person in Huntsville March 20?

Yes.

Workshop Logistics

Not applicable. This proposal is for a one-hour conference talk.