#244 Enterprise architecture and AI agents 5.5 min
Cost Control for AI Agents: A Budget for Each Agent
One agent in a loop used 88 percent of a made-up day's spend, and nothing stopped it.
Read the three lines
Copy the code folder first. Run python compare.py. For each of the three lines, write down the credits, the share taken by research and the number of refused calls. Then say in one sentence which change made the biggest drop in credits and which made a smaller one.
A hint
The first change adds budgets. The second change sends easy calls to the small model.
Shorten the loop
In traffic.py, change range(60) to range(20) in the research line, so that the loop makes 20 calls. Predict the no-controls line, then run python compare.py. When I ran it the no-controls line showed 42,600 tokens and 127.8 credits with research 70% of it, and 14 calls were refused in the budget lines.
A hint
Each research call is 1500 tokens on the large model, which costs 4.5 credits.
Give research a bigger budget
In traffic.py, change the research limit from 30 to 60 in LIMITS. Predict whether more or fewer calls are refused, then run python compare.py. When I ran it the budget line showed 96.3 credits and 47 refused, with research 61% of it.
A hint
A bigger limit lets more of the loop's calls through before the gate closes.
Make the small model dearer
In budget.py, change "small": 0.2 to "small": 1.0 in PRICE. Predict whether the last line gets cheaper or dearer, then run python compare.py. When I ran it the last line showed 44.4 credits, with research 61% of it. Say why the first two lines did not change.
A hint
Only the third line sends easy calls to the small model.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
Made-up prices, traffic and budgets on one laptop; the 88 percent and the 54 refusals follow from the loop length and the limits I chose; no real agent, model or router was run.
Up next in Enterprise architecture and AI agents
More in Enterprise architecture and AI agents
- #229 A Reference Architecture for Enterprise AI Agents, as Code
- #230 Agent Gateway: One Front Door for Models, Tools, Agents
- #231 MCP Gateway: Per-Tool Policy for Agent Tools (Python)
- #232 Give Every AI Agent Its Own Identity
- #233 Delegated Authority: Let an AI Agent Act for a User, Safely
- #234 Authorisation for AI Agents With Policy as Code (Cedar)
Related videos
- #71 Model Routing: Send Easy Questions to a Small Model LLM engineering and evals
- #236 Agent Orchestration Patterns: Supervisor, Pipeline, Swarm Enterprise architecture and AI agents
- #237 Where Should AI Agent State and Memory Live? A Crash Test Enterprise architecture and AI agents
- #238 Event-Driven AI Agents: Queues, Crashes and Duplicates Enterprise architecture and AI agents
