Too Many MCP Tools: Shortlist, Validate and Log (Python)
A plain keyword router put the right tool in a list of 5 for 28 of 30 made-up requests.
Video #330 ยท AI agents and MCP
Steps in the video
- 0:00 Right tool in a list of five: 28 of 30, from 539 words to 45
- 0:31 The idea
- 1:08 Setup
- 1:23 The catalogue
- 1:46 Thirty requests
- 1:55 Scoring
- 2:15 Route, then shortlist
- 2:54 Count the hits
- 3:41 Required and unknown fields
- 3:56 Type, enum and range
- 4:33 The call log
- 5:14 What this does not show
- 5:43 Mistakes to avoid
- 6:07 My take and next steps
- 6:26 Recap
Files
- calllog.py (36 lines)
- catalog.py (275 lines)
- labelled.py (55 lines)
- router.py (66 lines)
- run.py (70 lines)
- validate.py (37 lines)
Tested with
Python 3.12.15
What this does not show
No model is run, so model accuracy is not measured: the example only checks whether the right tool is in a keyword-router shortlist. Tools, thirty requests and the right-tool labels are made up; the server step made no difference here and a bigger catalogue was not tried. No real MCP server or client is used; the tools are plain Python dicts in the MCP tool shape.
Sources
- https://arxiv.org/abs/2503.01763
- https://json-schema.org/understanding-json-schema/reference/enum
- https://json-schema.org/understanding-json-schema/reference/numeric
- https://json-schema.org/understanding-json-schema/reference/object
- https://modelcontextprotocol.io/specification/latest/server/tools
- https://www.anthropic.com/engineering/writing-tools-for-agents
Learning IT by Example.