Organizations face escalating costs from high-volume application logs—most never used. In response, teams aggressively filter logs, risking incident blind spots.
This talk introduces the Flight Recorder pattern for OTel Logs: instead of continuously exporting logs via OTLP, applications export them to ring buffers—ideally backed by OS-native mechanisms like Windows ETW or Linux Tracepoints, but also implementable in user space. Logs remain local, continuously overwritten until triggers—critical errors or operator requests—cause buffered logs to be exported via OTLP to OTel Collector, maintaining existing workflows.
Once triggered, the system also enters a temporary streaming mode, exporting all new logs continuously for a brief period. This approach eliminates unnecessary logging costs while ensuring logs are readily available when it matters. We’ll present a design and working prototype that shows how this model delivers substantial cost savings without sacrificing observability.