learning-it by example

Back to the video

Cost Control for AI Apps: A Token Meter With Caps

In a made-up day, a 40-cent user cap and a 60-cent daily cap each refused a call before it ran.

Video #162 ยท MLOps and deployment

Steps in the video

  1. 0:00 A made-up day: a 40-cent user cap and a 60-cent daily cap each refused a call.
  2. 0:39 The idea
  3. 1:10 Setup
  4. 1:29 The stand-in model
  5. 1:50 Prices and caps
  6. 2:18 Check before the call
  7. 2:49 What is logged
  8. 3:13 The made-up requests
  9. 4:36 What this does not show
  10. 5:17 Mistakes to avoid
  11. 5:42 My take and next steps
  12. 6:08 Recap

Files

  • meter.py (38 lines)
  • run.py (34 lines)
  • stand_in.py (12 lines)

Download the code (zip)

Tested with

Python 3.12.15

What this does not show

No model or API is called: the model is a stand-in, tokens are counted as words, prices and users are made up, and the totals live in memory only.

Sources

Learning IT by Example.