learning-it by example

Back to the video

Indirect Prompt Injection: A Note That Gave Orders

A sentence I planted in a made-up note told my assistant what to say, and it said it.

Video #88 ยท AI security and governance

Steps in the video

  1. 0:00 A planted note told my assistant what to say
  2. 0:29 The idea
  3. 1:06 Setup
  4. 1:29 Notes and attacks
  5. 1:58 Three designs
  6. 2:23 Count two things
  7. 3:15 What this does not show
  8. 4:00 Mistakes to avoid
  9. 4:25 My take and next steps
  10. 4:54 Recap

Files

  • assist.py (28 lines)
  • compare.py (21 lines)
  • notes.py (18 lines)

Download the code (zip)

Tested with

Python 3.12.15 | ollama==0.6.3

What this does not show

Six attacks I wrote, one question, one seed and one small local model (llama3.2); a reply counts as obeyed if it contains a marker word and as answered if it contains the word Thursday; the assistant has no tools; none obeyed with the fence here does not make a fence safe.

Sources

Learning IT by Example.