As the complexity of large language model (LLM) applications continues to grow, so does the challenge of debugging and iterating on these applications. Traditional monitoring tools often fall short in providing the level of visibility and context needed to understand the behavior of LLM apps.

In this talk, we will discuss how Open Source TruLens used OpenTelemetry (OTel) spans to enable richer visualization of LLM app traces, and enable the ability to compute evaluations against particular span types and attributes. By leveraging OTel's ability to capture and propagate context across distributed systems, we were able to gain deeper insights into the behavior of our LLM apps.

Specifically, we will discuss how we used OTel spans to:
Identify and visualize the different stages of LLM app execution
Evaluate the performance of specific LLM app components
We will also share how the combination of OTel spans and TruLens's visualization capabilities enabled us to iterate on our LLM apps more quickly and efficiently.

This talk will be of interest to anyone who is interested in using OTel to improve the observability and performance of their LLM applications.