Never Trust Model Output: Validate It in Python
All ten of my model replies passed every check, and three were still wrong.
Video #95 ยท AI security and governance
Steps in the video
Files
- ask.py (15 lines)
- check.py (20 lines)
- compare.py (30 lines)
- schema.py (10 lines)
- tickets.py (16 lines)
Tested with
Python 3.12.15 | ollama==0.6.3
What this does not show
Ten made-up tickets, one seed, one small local model (llama3.2). Right means only that the category matches the one the author chose; priority and refund are not scored. The refund rule never fired in this run, so it is not shown working on a bad reply. Validation only checks what you thought to check.
Sources
Learning IT by Example.