#91 · AI security and governance · 5.3 minRate Limiting an LLM API: Token Bucket in FastAPI
In my test, one simulated user sent 400 big requests in 20 seconds, and my API answered all of them.
Python 3.12.15 | fastapi==0.161.1, starlette==1.9.0, httpx==0.28.1