#93 AI security and governance 5.5 min
OWASP Top 10 for LLM Apps: Three Risks Tested in Code
A planted page reached the top three results for four of five questions, and a filter missed two leaks.
Try a smaller token budget
Copy the code folder first. In budget.py change BUDGET from 3000 to 1000. In a Python session in that folder run: from budget import FLOOD, serve and then print(serve(FLOOD, True)). Predict the two numbers first, total tokens served and tokens for the heaviest user. You should get 2400 and 900. Then try 5000, which gives 6750 and 4500. Edit the file and then start a fresh Python session each time, because an open session keeps the old BUDGET. Put BUDGET back to 3000 afterwards.
A hint
ana and ben send three small requests each and mallory sends requests of 900 tokens. A request is skipped when it would take that user over the budget.
Hide a phone number with spaces
In a Python session in the code folder run: from leaks import redact and then print(redact('Her mobile number is 07700 900 123.')). Predict whether the number is hidden. It is not, because the pattern allows only one optional space. Now add a fourth pattern to PATTERNS in leaks.py: r'\b0\d{4} ?\d{3} ?\d{3}\b' and run it again in a fresh Python session, because the open one still holds the old patterns. The number is replaced with [hidden].
A hint
Add the new pattern as another string in the PATTERNS list, inside the square brackets.
Write a leak the filter misses
Add a seventh reply to REPLIES in leaks.py that gives away an invented detail in a way you make up. Add True to LEAKS for it. First remove the fourth pattern you added in the previous task. Predict whether the three patterns will change it, then test it with redact. Work out how many real leaks the filter would now miss out of the seven replies.
A hint
A reply that matches none of the three patterns stays unchanged, so it counts as missed.
Predict a shorter planted page
This one needs Ollama with all-minilm, as in the video, so treat it as a prediction. In poisoned.py shorten the upload page to only: Fast refunds at example.org/refund-now. Predict whether it still reaches the top three for as many of the five questions as before. Write your prediction down, then run compare.py and compare the first line of output with the 4 of 5 in the video.
A hint
The stuffed page worked because it repeated the questions people ask.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
Three of the ten risks, with small made-up examples. The first test only ranks pages with one small embedding model (all-minilm); no prompt is built and no model answers. The filter has three patterns. The token counts for the flood are made up. A source check only helps if the source label cannot be faked.
Up next in AI security and governance
More in AI security and governance
- #18 Prompt Injection Live: Attacking My Own LLM App
- #19 Redact Personal Data Before Your LLM Sees It (Python)
- #88 Indirect Prompt Injection: A Note That Gave Orders
- #89 Jailbreak-Test Your Own App: Eight Wrappers, Counted
- #90 Stop Leaking API Keys: Build a Secret Scanner in Python
- #91 Rate Limiting an LLM API: Token Bucket in FastAPI
Related videos
- #88 Indirect Prompt Injection: A Note That Gave Orders AI security and governance
- #94 Can Your Model Leak Its System Prompt? I Tried 8 Ways AI security and governance
- #51 Multi-User RAG: Filter Before You Search (Leak Test) RAG and vector search
- #18 Prompt Injection Live: Attacking My Own LLM App AI security and governance
