Andrew Schwabe
CEO & Founder, QCoda. Building AI engineering governance infrastructure
Ho Chi Minh City, Vietnam
Actions
Andrew Schwabe has spent twenty-five-plus years shipping software across insurance, financial services, healthcare, and manufacturing, the industries where "move fast and break things" gets you a compliance violation, not a promotion. He's the founder of QCoda, where he's building governance infrastructure for AI-assisted software development, and the creator of LLMesh, an open-source inference layer that runs large language models inside private environments with zero third-party API calls. He writes about the gap between what AI tooling promises and what it actually delivers on HackerNoon. Based in Ho Chi Minh City.
Area of Expertise
Topics
Your AI Agents Need a Runtime Security Policy: Governance Lessons from Cloud Native for the Agentic
The cloud native community spent five years building runtime guardrails after painful lessons. Software Bills of Materials, image signing, admission controllers, network policies, Open Policy Agent. That work assumed deterministic workloads. AI agents break the assumption.
Today's agentic pipelines fan out across Model Context Protocol tool chains, spawn sub-agents, and move data laterally through infrastructure that was never part of the original architecture review. The agent calls an API. That API calls a model. That model's output triggers another agent. Nobody authorized the full chain, and nobody can reconstruct it after the fact. I've been calling this the "openclaw moment," autonomous agent-to-agent fanout where the blast radius is invisible until something goes wrong.
This talk maps governance patterns from cloud native infrastructure onto AI agent workflows. Not as metaphor, but as implementation. I'll walk through what it actually looks like to run a multi-agent governance pipeline inside containers: deploying open-source large language models (8B to 32B parameter) on Docker and AWS Fargate with zero egress to third-party model APIs, enforcing an inference perimeter the way you'd enforce a network policy, building decision traceability as a first-class observability signal (not just metrics and traces, but what model, what prompt, what human approved it), and applying the sidecar pattern to embed governance into existing development workflows without adding a new control plane.
Concrete takeaways: a reference architecture for containerized AI governance pipelines, the three failure modes we hit that don't exist in traditional microservice observability, and an open-source inference layer (LLMesh) you can deploy today to keep model calls inside your perimeter. Whether you're a platform engineer adding AI workloads to your fleet or an AI builder who's never thought about runtime security, this is the infrastructure layer that's missing from the agentic stack.
We Needed a Mesh for AI Inference. So We Built One. And Open-Sourced It.
Your local LLM setup works. Then you push to staging and it breaks, because your app was pointing at localhost. So you switch to a cloud API, pay for tokens you didn't need, and send data you didn't intend to share.
We hit this building QCoda, an AI orchestration platform. Running agents at scale, the local-to-staging gap was a blocker. So we built LLMesh: an open-source inference mesh where compute nodes (laptops, GPU boxes, anything running Ollama) connect outward to a central hub. Your app hits one endpoint. Config never changes between environments. The mesh self-assembles from hardware your team already owns.
This talk covers what we needed, why nothing fit, the architectural flip that solved it (nodes reach the hub, not the reverse), and why we open-sourced it. I'll walk through pooling a small team's GPU capacity with no DevOps hire, no cloud contract, and zero third-party API calls. And tease the kind of LLM orchestration it unlocks.
Takeaways:
- The connection-direction pattern that kills the local-to-staging config gap
- Pooling existing hardware into a distributed inference mesh behind one endpoint
- Why open source, and what that decision actually costs
- Built-in token analytics for making real local-vs-cloud decisions
- Going from single-dev setup to team-wide mesh without a cloud API
- What you can do with a mesh that provides truly private small and large models available to your whole team
Notes for reviewers
LLMesh is fully open source (MIT, github.com/qcoda-ai/llmesh). No sales pitch. QCoda is origin-story context only. All tech discussed is open source. Possible to live demo, can discuss.
Andrew Schwabe
CEO & Founder, QCoda. Building AI engineering governance infrastructure
Ho Chi Minh City, Vietnam
Actions
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top