Model Routing: Send Easy Questions to a Small Model
Routing cost 19.1 made-up cents instead of 40, and got one more of 20 questions wrong.
Video #71 ยท LLM engineering and evals
Steps in the video
Files
- models.py (7 lines)
- questions.py (24 lines)
- router.py (13 lines)
- run.py (43 lines)
Tested with
Python 3.12.15
What this does not show
No model is called: questions, difficulties, prices and model skill are made up, so right and wrong are true by construction, and the threshold and signal words were tuned on the same twenty questions.
Sources
Learning IT by Example.