#245 Enterprise architecture and AI agents 5.8 min
AI Agent Observability: OpenTelemetry Traces, Cost, Replay
When an agent gives a bad answer, the hard part is getting that exact run back.
Find the six spans
Copy the code folder first. Open agent.py and find the place where the root span agent.run starts, the place where each model.call span starts, and the place where each tool.call span starts. Using the SCRIPT dictionary, work out why one request makes three model.call spans, two tool.call spans and one root span, which is six spans in all. This is a reading task.
A hint
The first two steps ask for a tool, and the third step gives the answer.
Predict a changed cost
In agent.py, change the token number in SCRIPT step 1 from 520 to 1000. Predict the total tokens and the total credits for model.call, using the price CREDITS_PER_1K = 3.0 in the file. Then, if you have the two OpenTelemetry packages from the video installed, run python compare.py and compare. I did not run this change, so treat the numbers as your own prediction.
A hint
Credits are tokens divided by 1000 and multiplied by 3.0.
Predict the replay with a different question
In compare.py, change the question "Please refund my order A-1" to another sentence. Predict whether the made-up model's answer changes and whether the replay still gives the same answer. Explain your reasoning from the SCRIPT dictionary. If you have the packages installed, run python compare.py to check. I did not run this change.
A hint
Look at how the made-up model chooses what to say at each step.
Say what the replay leaves out
Read replay.py and the paragraph in the video about what a replay cannot do. Write two sentences saying why this replay returns the same answer by design, and what kinds of real-life differences, such as time or randomness, a replay built only from recorded outputs would not reproduce. This is a reading task.
A hint
The replay feeds recorded outputs to the agent instead of asking the model.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
One request with a made-up model and tools, traced in memory; the replay returns the recorded outputs, so the same answer and zero real calls are by design; it tests agent code, not the model.
Up next in Enterprise architecture and AI agents
More in Enterprise architecture and AI agents
- #229 A Reference Architecture for Enterprise AI Agents, as Code
- #230 Agent Gateway: One Front Door for Models, Tools, Agents
- #231 MCP Gateway: Per-Tool Policy for Agent Tools (Python)
- #232 Give Every AI Agent Its Own Identity
- #233 Delegated Authority: Let an AI Agent Act for a User, Safely
- #234 Authorisation for AI Agents With Policy as Code (Cedar)
Related videos
- #236 Agent Orchestration Patterns: Supervisor, Pipeline, Swarm Enterprise architecture and AI agents
- #237 Where Should AI Agent State and Memory Live? A Crash Test Enterprise architecture and AI agents
- #238 Event-Driven AI Agents: Queues, Crashes and Duplicates Enterprise architecture and AI agents
- #239 Async AI Agents: Design for Timeouts With Job Polling Enterprise architecture and AI agents
