learning-it by example

#244 Enterprise architecture and AI agents 5.5 min

Cost Control for AI Agents: A Budget for Each Agent

One agent in a loop used 88 percent of a made-up day's spend, and nothing stopped it.

1. How does the per-agent budget in the video decide whether a model call may go ahead?
2. In the replay of one made-up day with no controls, how many credits were spent and what share did the looping research agent take?
3. With a budget for each agent in the made-up day, how many credits were spent and how many calls were refused?
4. When does the budget count a warning, and when does it refuse a call?
5. Why is the cheaper-model result of 36.2 credits not proof that a real router would save as much?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Read the three lines

    Copy the code folder first. Run python compare.py. For each of the three lines, write down the credits, the share taken by research and the number of refused calls. Then say in one sentence which change made the biggest drop in credits and which made a smaller one.

    A hint

    The first change adds budgets. The second change sends easy calls to the small model.

  2. Shorten the loop

    In traffic.py, change range(60) to range(20) in the research line, so that the loop makes 20 calls. Predict the no-controls line, then run python compare.py. When I ran it the no-controls line showed 42,600 tokens and 127.8 credits with research 70% of it, and 14 calls were refused in the budget lines.

    A hint

    Each research call is 1500 tokens on the large model, which costs 4.5 credits.

  3. Give research a bigger budget

    In traffic.py, change the research limit from 30 to 60 in LIMITS. Predict whether more or fewer calls are refused, then run python compare.py. When I ran it the budget line showed 96.3 credits and 47 refused, with research 61% of it.

    A hint

    A bigger limit lets more of the loop's calls through before the gate closes.

  4. Make the small model dearer

    In budget.py, change "small": 0.2 to "small": 1.0 in PRICE. Predict whether the last line gets cheaper or dearer, then run python compare.py. When I ran it the last line showed 44.4 credits, with research 61% of it. Say why the first two lines did not change.

    A hint

    Only the third line sends easy calls to the small model.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

Made-up prices, traffic and budgets on one laptop; the 88 percent and the 54 refusals follow from the loop length and the limits I chose; no real agent, model or router was run.

Up next in Enterprise architecture and AI agents

More in Enterprise architecture and AI agents

All Enterprise architecture and AI agents videos

Related videos