"Where did I leave my keys?" "What did we agree in that meeting?" "Find my Amsterdam ticket, it's somewhere in these photos." "When did this pump start sounding wrong?" Different devices, different sensors, one problem: something has been collecting data, and now it has to answer a question about what it collected. The hard part isn't the model — it's that the device has to remember, search what it remembers, and answer with no network, in under a second, inside a power budget.

This talk follows that problem through a stack that is open end to end. Modality lives in the encoder — an embedder for images, an ASR model for speech, whatever fits for sensor streams, all running under the usual on-device runtimes — ONNX Runtime, LiteRT, and LiteRT-LM where the answer itself is generated locally — while the retrieval layer underneath only ever sees vectors. That separation is what makes these results transferable: swap the encoder, keep the index. Underneath sits a local index, built and queried entirely offline. How it fills up differs by device: a sensor that never stops, a photo library imported once, a chat log that appends a little at a time. Then the same pipeline runs down a ladder of hardware — a Raspberry Pi or any Linux SBC, iOS and Android handsets, robots, and wearables — because that is where the interesting result is.

On a device, retrieval is a different engineering problem than on a server: no daemon to supervise, memory in megabytes, and flash with a wear budget. And the ingest pattern decides a great deal: a device that can afford one bulk build has options that a device where data never stops arriving — and reindexing is permanently off the table — simply does not. But none of these constraints are uniform across the ladder. An SBC gives you mmap and page cache. A robot runs Linux too, but the index competes for CPU with control loops that cannot be starved. A handset takes the daemon away and can kill your process mid-write, making crash consistency and cold start design constraints rather than optimizations. Smart glasses are the hardest case in the set: a few cores shared between camera, detection and embedding, and a thermal and power envelope that decides how much indexing you get to do at all — on that class of part a query can pull enough current to drop the device. The engine that wins at the top of the ladder is not the one that wins at the bottom.

I'll present a hands-on comparison of the embeddable engines — sqlite-vec, Qdrant Edge, ObjectBox — across all of them: footprint, bulk build versus incremental insert, cold start, quantization and recall, metadata filtering, and how each is actually reachable from an embedded application. Licensing belongs in that comparison rather than in a footnote: they range from a fully open core to an Apache-2.0 binding wrapped around a proprietary engine, and on an unfamiliar target that difference decides whether you can build at all. *Disclosure: I contribute to Qdrant.* The pipeline also runs across several inference backends — CPU, GPU delegate, and vendor NPU — which puts a number on what each path actually buys you.

And it runs live, on three of them: Qualcomm-based smart glasses, a Raspberry Pi, and a phone. All three offline, with the network off, on stage.

Every path was implemented and measured on real hardware, and the numbers are on the slides. The most useful of them is a negative result: on constrained parts the accelerator is often the wrong answer for a transformer graph, and not because the accelerator is slow.