Content Moderation With a Small Open Model vs a Word List
My banned-word list blocked a made-up test comment about killing a Python process.
Video #96 ยท AI security and governance
Steps in the video
Files
- compare.py (23 lines)
- judge.py (26 lines)
- messages.py (23 lines)
Tested with
Python 3.12.15 | ollama==0.6.3
What this does not show
Twenty comments I wrote and labelled myself, in English and mild; one seed, one small local model (llama3.2). The prompt was written after seeing the word list's mistakes, so the model had a head start. One of the word list's nine catches is a substring accident (skills contains kill). No dedicated safeguard model such as Llama Guard was tested.
Sources
Learning IT by Example.