learning-it by example

Back to the video

Content Moderation With a Small Open Model vs a Word List

My banned-word list blocked a made-up test comment about killing a Python process.

Video #96 ยท AI security and governance

Steps in the video

  1. 0:00 My word list blocked a comment about killing a process
  2. 0:30 The idea
  3. 1:03 Setup
  4. 1:27 Twenty comments
  5. 1:55 Two moderators
  6. 2:20 Score both
  7. 3:10 What this does not show
  8. 3:39 Mistakes to avoid
  9. 4:08 My take and next steps
  10. 4:33 Recap

Files

  • compare.py (23 lines)
  • judge.py (26 lines)
  • messages.py (23 lines)

Download the code (zip)

Tested with

Python 3.12.15 | ollama==0.6.3

What this does not show

Twenty comments I wrote and labelled myself, in English and mild; one seed, one small local model (llama3.2). The prompt was written after seeing the word list's mistakes, so the model had a head start. One of the word list's nine catches is a substring accident (skills contains kill). No dedicated safeguard model such as Llama Guard was tested.

Sources

Learning IT by Example.