Safe Rollouts: Canary and Rollback, Simulated
The same bug reached 100, 30 or 830 requests. The difference was a canary that someone watched.
Video #315 ยท Forward deployed engineer interviews
Steps in the video
Files
- rollout.py (28 lines)
- run.py (13 lines)
Tested with
Python 3.12.15
What this does not show
Simulated traffic, made-up failure rates, one seed. In a side check the watched canary's 30 ranged from 10 to 143 over 1,000 seeds; 830 follows from the plan. Version 1's failures are not simulated. A bug smaller than the 4 percent limit is not caught. Real canaries watch more than errors.
Sources
Learning IT by Example.