Sawyer Bowerman
AI Developer Advocate
Boston, Massachusetts, United States
Actions
Sawyer Bowerman is an AI Engineer and Developer Advocate on Red Hat’s AI team based in Boston, MA. He specializes in high-performance model serving and inference, focusing on scaling open-source ecosystems like vLLM and llm-d to make large language models more efficient and accessible for developers. He is dedicated to bridging the gap between raw model performance and real-world developer productivity through open-source innovation.
Area of Expertise
Topics
Your Hardware Is a Shared Apartment: The Secret to Serving One Model to 10,000 Users
Every demo serves one user. Production serves hundreds at once, and that's where things get ugly. One tenant fires off an 8K-token prompt and suddenly everyone else's chat completions are crawling, latency jumps 5x and your on-call gets paged at 2am.
We've been running multi-tenant inference using vLLM and llm-d, and honestly, the tricks that actually matter aren't the ones you'd expect. Disaggregated prefill and decode is the big one - splitting the compute-bound prompt crunching from the latency-sensitive token generation so they stop fighting each other. Prefix-cache-aware routing is the other, because if ten tenants share the same system prompt, you shouldn't process it ten times.
Beyond the single node, llm-d (now in CNCF Sandbox) lets you stretch vLLM across a Kubernetes cluster with KV offloading, cache-aware LoRA routing, and autoscaling that actually scales to zero when nobody's talking. We've seen ~3.1k tok/s per B200 GPU and TTFT improvements that are frankly hard to believe until you reproduce them.
In this talk, we’ll walk through the architecture we landed on, the mistakes we made getting there, and the observability techniques that saved us from shipping something embarrassing.
Audience:
Platform engineers, MLOps practitioners, and anyone shipping LLM-powered features to more than one customer.
Tool Calling Grew Up: Building Agents on vLLM
There's a moment in every agent project where you go from "this is amazing" to "this is terrifying," usually right when the model confidently calls a function that doesn't exist with
arguments that don't parse. Tool calling demos look magical. Tool calling in production looks like a lot of error handling.
We want to talk about what it actually takes to ship reliable tool-calling agents on vLLM. Under the hood, vLLM has model-specific parsers that know how each model formats its tool calls: XML tags for some, DSML tokens for DeepSeek, JSON for others. Knowing which parser does what saves you hours of debugging why your tool calls suddenly broke after a model swap.
The auto, required, and none modes for tool choice give you real control over when the model reaches for a tool versus just answering, and the strict calling toggle keeps it from improvising.
The piece we're most excited about is MCP integration: connecting vLLM directly to external tool servers so your model can browse, run code, and hit APIs without you writing the glue. Pair that with reasoning model support (DeepSeek R1, Qwen3), where the chain-of-thought is separated from the tool invocation, and you can finally see why the model picked that tool, not just that it did.
Audience:
Backend engineers building AI agents, platform teams deploying agentic workloads, and anyone who's tired of wrapping LLMs in try/except blocks.
The Context Wall: Giving Long-Running Agents Memory Beyond GPU RAM
Ask an agent to read a 200-page contract and then hold a conversation about it, and watch what happens to your GPU memory. Every token of context the model has ever seen sits in a KV cache, and that cache grows with every turn, every tool call, every document you feed it. GPU memory doesn't grow to match. Eventually you hit the context wall: either you buy more
GPUs you don't need for compute, or you start truncating context.
We've been running long-context and agentic workloads, and the answer is being smarter about where the memory lives. A lot of an agentic workload is repeated prefix, and once you see that, you can budget context very differently. Here's what makes the difference:
- Tiered KV cache offloading keeps recent context fast and lets cold context get cheap.
- Prefill and decode disaggregation splits the compute-bound job of ingesting a huge prompt from the memory-bandwidth-bound job of generating tokens one at a time.
- KV-cache-aware routing sends a request to whichever replica already holds the relevant cache instead of recomputing it cold.
- Distributed autoscaling that watches KV-cache saturation and queue depth instead of GPU utilization, which can be a red herring under continuous batching.
We'll go through the architecture, how to move the bottleneck off memory bandwidth, and the monitoring signals that tell you you're approaching the wall before your agent starts silently forgetting things.
Audience:
Platform engineers building agentic systems, decision makers at companies with hardware available
Quantize, Then Prove It: Compressing LLMs That Hold Up on Real Evals
Quantization has a credibility problem. Most tutorials show you how to shrink a model, then stop right before checking whether it still works. We've all seen the "4-bit with no quality
loss!" claims that fall apart the second you run a real eval.
We've been working with LLM Compressor, vLLM's compression library, and we want to share the honest picture. Which formats hold up: FP8 is the safe bet, and NVFP4 gets a 70B model onto
one GPU if you understand the tradeoffs. Which algorithms actually earn their place: GPTQ, AWQ, and newer microscaling formats like MXFP4 for Blackwell. And where accuracy-aware
transforms like SpinQuant and QuIP-style rotations let you push to 4-bit without falling off a cliff.
Then we get to the part most talks skip: benchmarking. Not just perplexity, but HumanEval pass rates, real inference throughput on H100 and H200, and the gap between "works on paper"
and "works under load." We'll show where FP8 KV-cache quantization genuinely helps long-context workloads, and where it quietly makes them worse.
You'll leave with a workflow you can take back to your team: compress, evaluate, deploy, and know exactly what you gave up.
Audience:
ML engineers, model owners, and anyone who's been asked to "make the model cheaper" without making it worse.
Attack the Model Before Your Users Do: Automated Red-Teaming for LLMs
When you deploy a model, you need to know whether it will hand a stranger someone else's PII if they phrase the question cleverly enough, or whether "ignore previous instructions" still works in 2026. Right now, most teams find that out in production. No good.
A model that can't be gamed by a sneaky prompter needs a different set of tools. Security scans fire jailbreak probes, prompt injection, PII extraction attempts, and toxicity triggers at a model automatically. A model doesn't get approved after passing enough evals. It gets approved after surviving enough attacks.
We'll demo these scans at two levels. First, pre-deployment: how to wire the scans into deployment pipelines as a CI/CD gate. Second, at runtime: scanning every input and output live at the inference boundary, and adding conversational rails so no matter how long someone keeps at it, they can't slowly coax your agent into doing something malicious. You need both layers, and they answer different questions.
We'll walk through what actually broke the first time we ran these scans, what a "model risk" view next to your catalog listing could look like, and why a clean benchmark score is
exactly the kind of false confidence that gets regulated industries in trouble. We'll use a set of open-source tools, notably Garak, TrustyAI, and NeMo Guardrails
Audience:
Platform and security teams, company leaders using customer facing models for enterprise, MLOps engineers
AI Agents You Can Actually Trust: Platform Best Practices for Safe AI
Your AI agent works in testing. In production, it makes unpredictable calls and breaks things. Four platform practices make agents safe: isolation, Identity, Observability, and Governance. This talk shows how to add them, and why infrastructure, not just prompting, is how you control AI at scale.
Audience:
Platform engineers building agentic systems, decision makers at companies with hardware available, company leaders using customer facing models for enterprise, MLOps engineers, Backend engineers building AI agents, platform teams deploying agentic workloads
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top