Cost Control for AI Apps: A Token Meter With Caps
In a made-up day, a 40-cent user cap and a 60-cent daily cap each refused a call before it ran.
Video #162 ยท MLOps and deployment
Steps in the video
- 0:00 A made-up day: a 40-cent user cap and a 60-cent daily cap each refused a call.
- 0:39 The idea
- 1:10 Setup
- 1:29 The stand-in model
- 1:50 Prices and caps
- 2:18 Check before the call
- 2:49 What is logged
- 3:13 The made-up requests
- 4:36 What this does not show
- 5:17 Mistakes to avoid
- 5:42 My take and next steps
- 6:08 Recap
Files
- meter.py (38 lines)
- run.py (34 lines)
- stand_in.py (12 lines)
Tested with
Python 3.12.15
What this does not show
No model or API is called: the model is a stand-in, tokens are counted as words, prices and users are made up, and the totals live in memory only.
Sources
- https://cheatsheetseries.owasp.org/cheatsheets/Logging_Cheat_Sheet.html
- https://docs.python.org/3/tutorial/floatingpoint.html
- https://genai.owasp.org/llmrisk/llm102025-unbounded-consumption/
- https://platform.claude.com/docs/en/api/messages
- https://platform.claude.com/docs/en/api/rate-limits
- https://platform.claude.com/docs/en/build-with-claude/token-counting
Learning IT by Example.