learning-it by example

#93 AI security and governance 5.5 min

OWASP Top 10 for LLM Apps: Three Risks Tested in Code

A planted page reached the top three results for four of five questions, and a filter missed two leaks.

1. In the video 'OWASP Top 10 for LLM Apps: Three Risks Tested in Code', which three entries of the 2025 list are tested with small made-up code examples?
2. In a test with six made-up handbook pages, one stuffed upload page and five questions, ranked by the small all-minilm embedding model, how often did the stuffed upload reach the top three results when every page was indexed, and when only handbook pages were allowed?
3. When does keeping uploaded pages out of a search index by their source label help against a planted page?
4. In a demo with six made-up replies, a privacy filter with three patterns changed three replies, but five replies were real leaks. Why did it miss two?
5. How does the per-user token budget work in the flood demo, and what did it do to the total?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Try a smaller token budget

    Copy the code folder first. In budget.py change BUDGET from 3000 to 1000. In a Python session in that folder run: from budget import FLOOD, serve and then print(serve(FLOOD, True)). Predict the two numbers first, total tokens served and tokens for the heaviest user. You should get 2400 and 900. Then try 5000, which gives 6750 and 4500. Edit the file and then start a fresh Python session each time, because an open session keeps the old BUDGET. Put BUDGET back to 3000 afterwards.

    A hint

    ana and ben send three small requests each and mallory sends requests of 900 tokens. A request is skipped when it would take that user over the budget.

  2. Hide a phone number with spaces

    In a Python session in the code folder run: from leaks import redact and then print(redact('Her mobile number is 07700 900 123.')). Predict whether the number is hidden. It is not, because the pattern allows only one optional space. Now add a fourth pattern to PATTERNS in leaks.py: r'\b0\d{4} ?\d{3} ?\d{3}\b' and run it again in a fresh Python session, because the open one still holds the old patterns. The number is replaced with [hidden].

    A hint

    Add the new pattern as another string in the PATTERNS list, inside the square brackets.

  3. Write a leak the filter misses

    Add a seventh reply to REPLIES in leaks.py that gives away an invented detail in a way you make up. Add True to LEAKS for it. First remove the fourth pattern you added in the previous task. Predict whether the three patterns will change it, then test it with redact. Work out how many real leaks the filter would now miss out of the seven replies.

    A hint

    A reply that matches none of the three patterns stays unchanged, so it counts as missed.

  4. Predict a shorter planted page

    This one needs Ollama with all-minilm, as in the video, so treat it as a prediction. In poisoned.py shorten the upload page to only: Fast refunds at example.org/refund-now. Predict whether it still reaches the top three for as many of the five questions as before. Write your prediction down, then run compare.py and compare the first line of output with the 4 of 5 in the video.

    A hint

    The stuffed page worked because it repeated the questions people ask.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

Three of the ten risks, with small made-up examples. The first test only ranks pages with one small embedding model (all-minilm); no prompt is built and no model answers. The filter has three patterns. The token counts for the flood are made up. A source check only helps if the source label cannot be faked.

Up next in AI security and governance

More in AI security and governance

All AI security and governance videos

Related videos