learning-it by example

Back to the video

Jailbreak-Test Your Own App: Eight Wrappers, Counted

My bot refused a plain question, then answered four of eight wrapped versions of it.

Video #89 ยท AI security and governance

Steps in the video

  1. 0:00 The bot refused my plain question, and answered four wrapped ones
  2. 0:27 The idea
  3. 1:11 Setup
  4. 1:37 Eight wrappers
  5. 2:09 Two builds
  6. 2:36 Count the leaks
  7. 3:26 What this does not show
  8. 4:03 Mistakes to avoid
  9. 4:29 My take and next steps
  10. 4:56 Recap

Files

  • attacks.py (16 lines)
  • bot.py (22 lines)
  • compare.py (10 lines)

Download the code (zip)

Tested with

Python 3.12.15 | ollama==0.6.3

What this does not show

One bot, one harmless off-topic question, eight wrappers I wrote, one seed and one small local model (llama3.2); success is the single word Paris in the reply; no real cooking question was sent, so the cost of the check is not measured.

Sources

Learning IT by Example.