learning-it by example

Back to the video

Agent Gateway: One Front Door for Models, Tools, Agents

I sent eight requests through one gateway, and five were stopped at the door.

Video #230 ยท Enterprise architecture and AI agents

Steps in the video

  1. 0:00 Eight requests, one door, and five of them stopped there
  2. 0:35 The idea
  3. 1:09 Setup
  4. 1:32 The checks
  5. 2:03 Log first, then pass on
  6. 2:36 Eight requests
  7. 3:42 What this does not show
  8. 4:09 Mistakes to avoid
  9. 4:38 My take and next steps
  10. 5:07 Recap

Files

  • compare.py (31 lines)
  • gateway.py (41 lines)
  • upstreams.py (32 lines)

Download the code (zip)

Tested with

Python 3.12.15 | fastapi==0.161.1, starlette==1.9.0, httpx==0.28.1

What this does not show

Two made-up callers with keys in a dictionary, and services called in the same process. Only the prompt field is size-checked, a body that is not a JSON object fails with no audit line, and the audit records the gateway's verdict, not the reply.

Sources

Learning IT by Example.