learning-it by example

Back to the video

AI Agent Observability: OpenTelemetry Traces, Cost, Replay

When an agent gives a bad answer, the hard part is getting that exact run back.

Video #245 ยท Enterprise architecture and AI agents

Steps in the video

  1. 0:00 A bad answer, and a way to run it again without calling the model
  2. 0:27 The idea
  3. 1:09 Setup
  4. 1:33 An agent with a trace
  5. 2:07 Read it, then replay it
  6. 2:34 Run it, and run it again
  7. 3:37 What this does not show
  8. 4:17 Mistakes to avoid
  9. 4:46 My take and next steps
  10. 5:16 Recap

Files

  • agent.py (52 lines)
  • compare.py (19 lines)
  • replay.py (30 lines)

Download the code (zip)

Tested with

Python 3.12.15 | opentelemetry-api==1.45.1, opentelemetry-sdk==1.45.1

What this does not show

One request with a made-up model and tools, traced in memory; the replay returns the recorded outputs, so the same answer and zero real calls are by design; it tests agent code, not the model.

Sources

Learning IT by Example.