#157 MLOps and deployment 6.6 min
Model Registry by Hand: Versions, Promote, Roll Back
Version 3 scored 0.867 against 0.897 in production: the registry refused it, a forced push went through, and one call rolled it back.
Give Cy more rows
Copy the code folder first. In run.py change "cy": NEW[:40] to "cy": NEW[:200]. Predict what version 3 will score and whether the gate will still refuse it. Run python run.py register, then python run.py promote, and look for a refused line. On my machine version 3 got threshold 250 and test accuracy 0.897, and no refused line appeared.
A hint
Compare version 3's accuracy with version 2's.
Refuse ties as well
Keep the change from the last task. In registry.py, in promote, the gate refuses when the new accuracy is below the old one. Change that test so that it also refuses when the two are equal (use less-than-or-equal). Predict what python run.py promote prints first, then run it. On my machine a refused line appeared even though the two accuracies were equal.
A hint
The gate compares the new version's test accuracy with the one in production.
Rewrite the file with the same bytes
In run.py the tamper mode overwrites version 2's file with b'{"threshold": 0}'. Replace those bytes with b'{"threshold": 250}' and run python run.py tamper, then try b'{"threshold":250}' with no space. Predict for each whether the load is refused. On my machine the first was not refused (no refused line means the load was accepted) and the second was.
A hint
The check compares hashes of bytes, not the meaning of the content.
Roll back twice
In run.py in the promote function, add a second call reg.rollback("fraud", "production", "ana") followed by another show line. Predict which version production is on after the second rollback and what the audit will list, then run python run.py promote. On my machine production stayed on version 2 and a second rollback row appeared.
A hint
Read what rollback looks up in the audit table: the last promote or force row.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
A made-up one-number fraud model on made-up payments; a hand-written SQLite registry with one metric as the gate, a counter instead of a clock, one process and no logins; this is not MLflow.
Up next in MLOps and deployment
More in MLOps and deployment
Related videos
- #133 A/B Test Analysis in SQL and Python: z Test and Peeking SQL and analytics
- #208 Right to Erasure Across a Pipeline: Delete, Scan, Restore Data governance
- #235 An End-to-End Audit Trail for AI Agent Actions in Python Enterprise architecture and AI agents
- #238 Event-Driven AI Agents: Queues, Crashes and Duplicates Enterprise architecture and AI agents