Our systems work where the internet doesn't: air-gapped sites where vehicles collect sensor data with no cloud connectivity at all. Occasionally during maintenance window they get to offload.

This talk walks through the end-to-end pipeline we built for that reality, in five acts:

1. Offload: vehicles push archives through a self-hosted Supabase stack (Kong, GoTrue, PostgREST, Storage API) that doubles as the OLTP backend for our customer-facing applications.

2. Ingest: once archives land in S3, Databricks Auto Loader streams them into Delta tables under Unity Catalog — exactly-once, MERGE upserts, checkpoint recovery.

3. Post-process: machine-learning and map-processing jobs consume the lakehouse and publish trained models and processed maps back into it.

4. Compute: these jobs run as Argo Workflows on Kubernetes, scheduled across AWS (EKS + Karpenter) and Nebius (scale-to-zero GPU pools) — picking the cheapest viable backend per workload.

5. Consolidate: merging these three organically grown stacks into one VPC planning, to turn internal infrastructure into a real product.