Session

A copilot inside a structured document: why naive RAG isn't enough

A screenplay is not linear text. It's a graph of scenes, characters and relations. The first copilot I put inside one ran on standard RAG: chunk the document, compute embeddings, retrieve the most similar pieces. The answers were plausible and wrong. The model didn't know that the line on page 58 belongs to the character who walked in on page 12, that the scene had already been rewritten three times, or that the user was looking right at it.

I'll walk through the architecture that came out of that failure, built while working on Typewrite, a writing environment for screenwriters.

The document becomes a graph instead of a string. Every node carries an id that survives rewrites, and context is navigated by relation: who speaks, in which scene, after what.

Between the editor and the model there's a typed structure. The editor decides what enters the context, because the editor is the one that knows where the user is working, and every piece that reaches the model can be traced back to its origin.

Above that sits a gateway, and under the gateway sit the evals. I benchmarked several models from four providers on the same structured-extraction task over a screenplay, against a hand-annotated gold set: I bring the accuracy, latency and cost numbers. What surprised me is how much the winner changes with the task, and how different the native structured-output mechanisms are from each other. That's why routing ended up in the architecture instead of in a config file.

Part of the session is about the approaches I threw away and the bills that blew up in my hands.

On the state of the project: the editor is TypeScript on ProseMirror and TipTap, with a page engine written from scratch, working and tested. The AI layer is under implementation. This talk is about the architecture decisions and the benchmark that drove them, not about a system already in production.


Available in Italian and English. Works as a 25-30 min talk or a 45 min version with a longer benchmark section. Audience: intermediate — assumes familiarity with LLM APIs and basic retrieval. Code and benchmark data shown live. First delivery: submitted to DevFest Alps 2026.

Salvatore Contino

Founder & CEO at Animora · Cursor Ambassador · Microsoft Developer Insider · Startup Geeks mentor

Turin, Italy

Actions

Please note that Sessionize is not responsible for the accuracy or validity of the data provided by speakers. If you suspect this profile to be fake or spam, please let us know.

Jump to top