Karthik Ravi
Senior Software Engineer, AI/ML Infrastructure at PayPal
Mountain View, California, United States
Actions
Karthik Ravi is a senior software engineer focused on AI/ML infrastructure and distributed systems. He works on Kubernetes-based workload scheduling, multi-cluster compute, and reliable self-service platforms for data and AI workloads. His engineering interests include heterogeneous workload controllers, accelerator resource management, admission control, and turning integration failures into repeatable platform-readiness tests.
Area of Expertise
Topics
Right-Sized GPUs for JupyterHub: Sharing Accelerators Across Interactive Notebooks
Interactive notebook users often reserve an entire GPU when exploration needs only a fraction of its memory and compute capacity. This reduces availability and forces lightweight notebooks to compete with workloads that need a full device.
This talk presents how we added right-sized GPU profiles to JupyterHub. Users select a profile based on the memory and compute they need, while KubeSpawner translates that choice into Kubernetes resources for the notebook pod. A GPU-sharing layer implemented with HAMi selects a compatible device and enforces the allocation without exposing scheduler-specific details to users.
We will cover profile design, resource translation, spawn-time validation, clear failures when capacity is unavailable, visibility into assigned resources, persistence across notebook restarts, and reliable release during shutdown or failed launches. We will also explain the isolation boundary: GPU memory can be capped, while compute sharing uses time slicing and is not equivalent to hardware partitioning.
Your AI Workload Was Admitted. Why Didn't It Run?
A Kubernetes workload can be admitted and still fail to become useful compute. Across validation of Job, RayJob, PyTorchJob, JobSet, and AppWrapper, we found that apparently similar "stuck" workloads can come from very different layers: API and CRD readiness, controller reconciliation and RBAC, or Kueue admission and quota accounting.
This technical session organizes those implemented integration lessons into a three-gate diagnostic method. It explains how API and CRD readiness, controller reconciliation, and Kueue admission can be checked independently, and how the status returned at each layer narrows the failure domain. It also covers verified implementation findings: framework-specific placement behavior, an RBAC failure that prevented reconciliation, and a DRA quota-accounting gap where a workload could receive a GPU without consuming the expected Kueue quota.
Attendees leave with a practical test sequence and failure taxonomy for onboarding heterogeneous AI workload controllers without treating admission as proof that useful compute will run.
One Notebook Interface, Many Systems: Where Governance Actually Lives
Notebook users want one concise interface even when the systems behind it use different clients, credentials, policies, and execution models. Our notebook magic libraries support database queries and cloud services through shared IPython entry points, connection management, authentication helpers, and metrics instrumentation. The implementation includes integrations for BigQuery, Bigtable, Dataproc, and MySQL, alongside additional data systems.
This session examines where the common platform boundary ends and a backend-specific connector begins. It covers the implemented patterns used to route commands to connection adapters, reuse and close connections, obtain credentials through centralized helpers, attach operational and billing metadata, retrieve classification context, and instrument notebook operations. It also explains why authentication, query execution, managed compute, and metadata lookup cannot all be forced behind one identical abstraction.
Attendees leave with a practical way to divide responsibilities between a shared notebook interface and backend adapters, plus the design trade-offs to consider when adding another governed data or compute integration.
One Click, Three Decisions: Quota, Cluster, and GPU Sharing Behind a Jupyter Notebook
Starting a Jupyter notebook looks like one click. At roughly 5,000 daily sessions, a GPU-backed server may cross several control boundaries before it is ready. This poster visualizes that path across multiple Kubernetes clusters.
A user selects a profile containing CPU, memory, GPU, image, and storage requirements. A multi-cluster KubeSpawner converts it into Kubernetes resources and manages the lifecycle across manager and worker contexts. Kueue evaluates the request through LocalQueue, ClusterQueue, quota, priority, and ResourceFlavor policies. MultiKueue dispatches the admitted workload to an eligible bare-metal or managed cloud cluster. On a GPU worker, HAMi selects a compatible device and applies the requested GPU-memory and compute share.
The poster separates three decisions that are often conflated: when quota permits the notebook to run, which cluster receives it, and how accelerator capacity is allocated there. It also maps user-visible progress, failure states, timeout recovery, and cross-cluster cleanup.
One Notebook Interface, Many Systems: A Reusable Governance Pattern
Notebook users want one simple command whether the destination is MySQL, BigQuery, Spanner, or a Dataproc cluster. Platform teams, however, must consistently handle identity, authorization, connection lifecycle, metadata, auditability, failure behavior, and observability across systems with very different APIs. Reimplementing those controls inside every connector creates drift and makes new backends expensive to govern.
This lightning talk presents a connector-harness design pattern for notebook-native infrastructure. A thin system adapter supplies backend-specific operations, while a shared control layer enforces capability declarations, policy hooks, structured events, redacted errors, lifecycle telemetry, and metadata context. We show how the same contract supports data queries and managed compute without exposing credentials or binding users to backend-specific setup.
Attendees leave with an adapter interface, governance hooks, common event model, and conformance checklist they can reuse when adding the next data or compute system.
Invisible Infrastructure at Scale: Operating 5,000 Daily Notebook Servers Across Multiple Kubernetes
Starting a Jupyter notebook appears to be one simple action. At approximately 5,000 real notebook-server sessions on a typical day, that action becomes considerably more complex - especially because the workloads are not identical. Each notebook may require a different combination of CPU, memory, GPU capacity, container image, storage, and runtime configuration, with some resources available only in particular clusters.
This talk presents how we extended KubeSpawner to separate the JupyterHub control plane from the Kubernetes clusters where notebook servers run. We follow a notebook through its complete lifecycle: interpreting its resource requirements, selecting a suitable cluster, creating its pod and service in the correct context, observing startup, retrieving its endpoint, monitoring health, and terminating it safely.
We cover context-aware Kubernetes clients, per-cluster state tracking, progress reporting, timeout recovery, stale-state detection, and cross-cluster cleanup. The result is a reusable architecture that lets JupyterHub operate across bare-metal and managed cloud clusters without exposing infrastructure complexity to notebook users.
Interactive Is Not Batch: Scheduling Multi-Cluster JupyterHub with Kueue
Batch schedulers assume a submitted job can wait quietly. Interactive notebooks cannot: users need visible admission progress, bounded startup behavior, cancellation, cleanup, and reliable reconnects while capacity may live in another Kubernetes cluster. Treating a notebook Pod like an ordinary queued job creates failure modes at the seams between JupyterHub, Kueue, and multi-cluster placement.
This session builds an upstream-only control-plane model for Kueue-aware interactive sessions. We trace a spawn request through context selection, queue admission, workload progress, notebook startup, proxy routing, cancellation, and stale-event cleanup. We then separate four problems that are often conflated: admission state, user-visible progress, event-loop responsiveness, and cross-cluster lifecycle ownership.
Attendees leave with a lifecycle state machine, failure taxonomy, and synthetic test plan for validating interactive workloads across Kubernetes clusters—without relying on private topology or vendor-specific components.
Installed Is Not Schedulable: Three Gates for Kubernetes AI Workloads
Installing a CRD and controller does not prove that an AI workload will become useful compute. Across implemented validation of Kubernetes Job, RayJob, PyTorchJob, JobSet, and AppWrapper with Kueue, apparently similar "stuck" workloads came from different layers: API and schema readiness, controller reconciliation and RBAC, or scheduler admission and quota accounting.
This session turns those integration lessons into a three-gate diagnostic method. It explains how to test API readiness, controller execution, and Kueue admission independently; how the status returned at each layer narrows the failure domain; and why admission alone is not proof that the intended compute will run. The examples are grounded in implemented findings, including framework-specific placement behavior, an RBAC gap that prevented controller reconciliation, and a Dynamic Resource Allocation quota-accounting gap where a workload could receive a GPU without consuming the expected Kueue quota.
Attendees will leave with a practical test sequence and failure taxonomy for onboarding heterogeneous AI workload controllers before investing in GPU packing, platform UX, or performance tuning.
Beyond GPU Counts: Proving Kubernetes DRA Workloads Are Actually Schedulable
Requesting nvidia.com/gpu: 2 treats accelerators as interchangeable device counts. Modern AI clusters need a richer contract for partitioned, topology-sensitive, or driver-managed devices. Kubernetes Dynamic Resource Allocation (DRA) provides that contract, but installing a DRA driver does not prove that real workloads can be admitted, allocated, run, and cleaned up correctly.
This session presents an upstream-only validation method for the path from ResourceClaim to accelerator compute. We separate readiness into six observable gates: driver availability, claim binding, Kueue admission, concurrent allocation, real GPU execution, and workload-controller lifecycle. We apply the same contract to a Job, JobSet, RayJob, and PyTorchJob to expose failures that installation checks miss.
The key lesson is that allocation and queue accounting are different correctness problems. Attendees leave with a reusable compatibility matrix, failure taxonomy, and rollout sequence for Kubernetes-based AI infrastructure.
Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.
Jump to top