Deploy an LLM API with FastAPI and Docker: The 0.0.0.0 Trap
Put a local LLM behind a FastAPI endpoint, pack it in a Docker container, and reproduce the bind-address mistake that makes it unreachable.
Video #16 ยท MLOps and deployment
Steps in the video
Files
- Dockerfile (7 lines)
- app.py (30 lines)
- call_api.py (28 lines)
- requirements.txt (4 lines)
Tested with
Python 3.12.15 | fastapi==0.162.0, uvicorn==0.54.0, ollama==0.6.3, pydantic==2.14.1
What this does not show
One container on one laptop calling Ollama on the same machine; no cloud deployment, sign-in, load test or scaling. host.docker.internal works out of the box with Docker Desktop; plain Docker on Linux needs an extra setting. One mistake (the bind address) is reproduced; the summary text itself is not shown or checked.
Sources
- https://calcsnippets-web.pages.dev/blog/uvicorn-host-0-0-0-0-is-the-flag-you-need-when-your-fastapi-app-works-locally-and-is-still-invisible-from-everything-else.html
- https://docs.docker.com/desktop/features/networking/networking-how-tos/
- https://docs.docker.com/reference/cli/docker/container/run/
- https://hub.docker.com/_/python
- https://pypi.org/project/fastapi/
- https://pypi.org/project/ollama/0.6.3/
- https://pypi.org/project/pydantic/2.14.1/
- https://pypi.org/project/uvicorn/
- https://uvicorn.dev/settings/
Learning IT by Example.