#230 Enterprise architecture and AI agents 5.6 min
Agent Gateway: One Front Door for Models, Tools, Agents
I sent eight requests through one gateway, and five were stopped at the door.
Allow a longer prompt
Copy the code folder first. This code needs fastapi, starlette and httpx at the versions in requirements.txt. In gateway.py, change MAX_PROMPT from 200 to 600. Predict the status of the request with the 500-character prompt and the last line it prints, then run python compare.py and compare. The model service answers any request that reaches it with a status 200.
A hint
The 413 check compares the length of the prompt with MAX_PROMPT.
Give the refund bot a model scope
In gateway.py, add "llm:chat" to the scopes of bot-key. Predict what happens to the bot-key request to /llm/chat, and how many requests now show as passed on. Run python compare.py to check your prediction.
A hint
The scope check compares the scope the destination needs with the list of scopes for the caller.
Add a ninth request
In compare.py, add a ninth request to REQUESTS: the key stolen-key sent to /payments/send with the body {"amount": 5}. Predict which status the gateway returns, even though that destination does not exist. Think about which of the four checks runs first. Then predict how many lines the audit log has. Run python compare.py to check.
A hint
The destination is only checked after the key has been looked up.
Change who may call agents
In compare.py, add a request that sends bot-key to /agents/ask with the body {"prompt": "hi"}. Predict the status and the message. Then in gateway.py give bot-key the scope "agents:call" and predict again. Run python compare.py after each change and write down what the audit log count says each time.
A hint
The scope each destination needs is stored next to its service in UPSTREAMS.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
Two made-up callers with keys in a dictionary, and services called in the same process. Only the prompt field is size-checked, a body that is not a JSON object fails with no audit line, and the audit records the gateway's verdict, not the reply.
Up next in Enterprise architecture and AI agents
More in Enterprise architecture and AI agents
Related videos
- #249 Build a Minimal AI Agent Platform: Gateway, Registry, Audit Enterprise architecture and AI agents
- #229 A Reference Architecture for Enterprise AI Agents, as Code Enterprise architecture and AI agents
- #234 Authorisation for AI Agents With Policy as Code (Cedar) Enterprise architecture and AI agents
- #235 An End-to-End Audit Trail for AI Agent Actions in Python Enterprise architecture and AI agents
