Akkhil Gupta
DeduceTheLogic
Hyderābād, India
Actions
I am a senior technology and AI leader with over 17 years of experience building and scaling large-scale platforms across AI/ML, data engineering, distributed systems, and enterprise product engineering. He currently serves as a Vice President – Technical, where he works at the intersection of AI strategy, platform architecture, and business outcomes.
Links
Area of Expertise
Topics
Zero-Trust, Zero Connectivity: Identity and Sandboxing for Offline MCP Agent
MCP's security story today assumes a network in the loop — OAuth 2.1 round-trips to an Authorization Server for delegation, WASM sandboxes with cycles to spare for capability metering, audit trails that land on a central log. That story breaks the moment your MCP host is a phone with no connectivity.
This talk is about what happens when the MCP host is the disconnected edge device. Drawing on a production on-device deployment (Phi-3 Mini, int4-quantized, running via ExecuTorch on Snapdragon's Hexagon NPU, resolving ~80% of queries in under 300ms with zero cloud egress), we work through three unsolved seams:
Delegation without a live token exchange — how On-Behalf-Of semantics (user token + actor token, scope narrowing per hop) survive being pre-provisioned at deployment instead of negotiated per-request, and what you trade away (revocation latency, replay window) to get there.
Sandboxing on a shared NPU budget — what fuel-metered WASM capability boundaries actually cost when they're competing with your inference pipeline for the same silicon, and where rigor gets traded for a sub-300ms latency budget.
Zero Egress, Full Intelligence: An In-Process MCP Orchestrator for Mobile AI
A fully self-contained Android application demonstrating Model Context Protocol (MCP) orchestration running entirely on-device, powered by ExecuTorch with Qualcomm HTP and XNNPACK backends.
This talk presents a working architecture that collapses the MCP client, transport, and server into a single in-process orchestrator running on a commodity Android phone. User requests are routed through a confidence-gated three-tier pipeline: a small language model (Phi-3 Mini, INT4-quantized via ExecuTorch) handles the request first; if its confidence falls below a calibrated per-domain threshold, the query escalates to an on-device RAG layer (FAISS over a local corpus); only structurally, never over the network. MCP remains the contract between tiers — tool discovery, capability tokens, and audit logging all work exactly as they would in a distributed deployment, but the "transport" is a coroutine channel, not HTTP.
The result: no data egress, sub-second p50 latency without network round-trips, and GDPR/DPDP compliance by architecture rather than by policy — there is simply nothing to consent to sending, because nothing is sent.
Zero Cloud, No IAM: Governing Agentic Identity at the Edge"
Enterprise IAM assumes a static actor: one human, one credential, one stable role. AI agents break this — they're provisioned dynamically, inherit permissions from invokers, and spawn sub-agents that delegate further, with no attribution chain or credential lifecycle model to match. On-device deployment makes this worse: there's no network authority in the loop to authenticate or log the agent's actions in real time.
This talk uses EdgeMind — an on-device agentic system running Phi-3 Mini via ExecuTorch on Snapdragon HTP, resolving most queries in under 300ms with zero cloud egress — as a case study. We walk through its five-agent EdgeForge pipeline to show how identity and authorization can be scoped per agent stage, using PyTorch/ExecuTorch-level techniques (export/quantization provenance, runtime integrity checks, constrained action surfaces) as substitutes for network identity controls when offline. Attendees leave with a concrete architecture pattern, not just the problem statement.
From GPU Efficiency to Governance: Protecting Latency SLOs in the Blackwell Era
NVIDIA Blackwell and NVFP4 fundamentally change the economics of LLM inference by dramatically increasing effective HBM capacity and enabling much higher concurrency per GPU. Frameworks like PyTorch and inference engines such as vLLM can now drive unprecedented utilization through continuous batching, long-context decoding, and dense multi-tenancy.
However, as memory bandwidth pressure recedes, a new bottleneck emerges: cross-workload interference.
This talk argues that in the Blackwell era, resource governance becomes as critical as kernel optimization. We explore why traditional best-effort batching is no longer sufficient, and why OpenReg-style concepts—explicit admission control, KV-cache isolation, priority-aware scheduling, and policy-driven backpressure—must become first-class concerns in PyTorch inference stacks.
The session concludes with a concrete roadmap for integrating governance into PyTorch-based inference without sacrificing performance—turning raw GPU efficiency into predictable, SLO-compliant systems
From Tokens to Throughput: Operating vLLM at Production Scale
As LLMs move into production, inference has become a systems engineering problem rather than a modeling challenge. While vLLM enables high-throughput serving via PagedAttention and continuous batching, operating it reliably at scale exposes new bottlenecks. This talk explores real-world failure modes in production vLLM deployments, including tail-latency spikes, KV-cache fragmentation, and GPU under-utilization driven by memory bandwidth pressure. We walk through the inference request lifecycle, highlight where latency hides, and introduce vLLM-specific observability signals that traditional metrics miss. Attendees will gain a systems-level mental model and practical guidance for building reliable, high-performance LLM serving platforms with PyTorch.
Scaling ML Models in Production: When to Move from Flask APIs to vLLM
This talk presents a practical, experience-driven evolution path for ML deployment, focusing on when and why teams should transition from general-purpose frameworks like Flask to production-grade inference engines such as vLLM.
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top