learning-it by example

Back to the video

Too Many MCP Tools: Shortlist, Validate and Log (Python)

A plain keyword router put the right tool in a list of 5 for 28 of 30 made-up requests.

Video #330 ยท AI agents and MCP

Steps in the video

  1. 0:00 Right tool in a list of five: 28 of 30, from 539 words to 45
  2. 0:31 The idea
  3. 1:08 Setup
  4. 1:23 The catalogue
  5. 1:46 Thirty requests
  6. 1:55 Scoring
  7. 2:15 Route, then shortlist
  8. 2:54 Count the hits
  9. 3:41 Required and unknown fields
  10. 3:56 Type, enum and range
  11. 4:33 The call log
  12. 5:14 What this does not show
  13. 5:43 Mistakes to avoid
  14. 6:07 My take and next steps
  15. 6:26 Recap

Files

  • calllog.py (36 lines)
  • catalog.py (275 lines)
  • labelled.py (55 lines)
  • router.py (66 lines)
  • run.py (70 lines)
  • validate.py (37 lines)

Download the code (zip)

Tested with

Python 3.12.15

What this does not show

No model is run, so model accuracy is not measured: the example only checks whether the right tool is in a keyword-router shortlist. Tools, thirty requests and the right-tool labels are made up; the server step made no difference here and a bigger catalogue was not tried. No real MCP server or client is used; the tools are plain Python dicts in the MCP tool shape.

Sources

Learning IT by Example.