learning-it by example

Back to the video

Prompt Injection Live: Attacking My Own LLM App

I attacked my own language model app and a secret leaked. Then I ran the same attacks against three defences, and in this small test one of them made it worse.

Video #18 ยท AI security and governance

Steps in the video

  1. 0:00 I attacked my own app
  2. 0:27 What prompt injection is
  3. 0:55 Setup
  4. 1:22 The app and the emails
  5. 2:05 The attack
  6. 2:46 Three defences
  7. 3:15 Count the leaks
  8. 3:51 What this does not show
  9. 4:22 Mistakes to avoid
  10. 4:43 My take and next steps
  11. 5:06 Recap

Files

  • attack.py (10 lines)
  • defend.py (26 lines)
  • shop.py (38 lines)

Download the code (zip)

Tested with

Python 3.12.15 | ollama==0.6.3

What this does not show

One small local model and three attack emails, one run each, so the counts are not a general result for any model or defence. The discount code is made up. The output filter's zero is true by construction (the score uses the same string check) and an exact-string filter misses reworded leaks.

Sources

Learning IT by Example.