learning-it by example

#322 Forward deployed engineer interviews 5.5 min

SLOs and Error Budgets for an AI Feature, Explained

Both promises were kept. One afternoon burned the budget at 57 times the allowed rate.

1. In SLO and error-budget terms, what is burn rate, as the video defines it?
2. Both objectives were kept over the simulated month. What did the burn rate show for the three-hour outage on day 17?
3. In the simulated AI-feature month, the video says both objectives were kept, but with little margin. What does that mean?
4. In the SLO error-budget code, how are the allowed failures worked out for an objective of 99%?
5. Why does the video list an objective of 100 percent as a mistake?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Set a stricter quality objective

    Copy the code folder first. In slo.py change QUALITY from 0.95 to 0.97. Predict the number of failures allowed and whether the quality objective is still kept. Run python run.py and compare. Then change QUALITY back to 0.95, because the third exercise asks whether both objectives are kept.

    A hint

    Allowed failures are the requests times one minus the target. I saw 369 bad of 300 allowed.

  2. Shorten the outage

    In traffic.py, in the line that sets outage, change the 16 at the end to 14, so the outage lasts one hour instead of three. Predict whether the availability budget left will go up or down, and whether the day 17 burn rate will go up or down. Run python run.py. The run.py window still looks at 13:00 to 16:00, and the comment on that line still says 3 hours. When you have finished, change 14 back to 16, because the next exercise needs the original.

    A hint

    A shorter outage means fewer failures in the month, but the window in run.py now includes some calm hours. I saw a burn rate of 19 and a budget that lasts 38 hours.

  3. Try another month of traffic

    In traffic.py change month(seed=4, per_day=333) to month(seed=5, per_day=333). Predict whether both objectives will still be kept, and whether the day 17 burn rate will be exactly 57 again. Run python run.py. Then write one sentence on what this tells you about one simulated month.

    A hint

    The failures are drawn at random from the seed. I saw a day 17 burn rate of 71, and a budget lasting 10 hours.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

Made-up traffic with one clean three-hour outage; a single window, not the multi-window alerting real systems use; the quality check is only a label.

Up next in Forward deployed engineer interviews

More in Forward deployed engineer interviews

All Forward deployed engineer interviews videos

Related videos