#19 AI security and governance 6.5 min
Redact Personal Data Before Your LLM Sees It (Python)
Patterns found 5 of 8 pieces of personal data, a local model found the rest, and one wrong answer taught me something. A redaction step you can score.
Score the pattern finder
Copy the code folder first. Run python score_regex.py. This script does not need the Ollama model. Check that you see 5 of 8 items found both with and without the card check, with 1 false alarm without it and 0 with it. Then write down which three items are listed as missed and why the patterns cannot find them.
A hint
The missed items are the two names and the street address, which have no single shape.
Break the card number on purpose
In tickets.py change the card number 4111 1111 1111 1111 to 4111 1111 1111 1112 in both the text and the shaped list of ticket 2. Predict what the with card check line will say, then run python score_regex.py. When we ran it, it said found 4 of 8 items, and the missed list included the changed card number.
A hint
The last digit now fails the Luhn check, so the card is no longer kept.
Add a fifth ticket
In tickets.py add a new ticket before the kettle one, with the text Please write to jo.b@example.com about my parcel. Give it shaped set to that email and free set to an empty list. Predict the new totals, then run python score_regex.py. When we ran it, both lines said 6 of 9 items found.
A hint
Copy the layout of an existing ticket: text, shaped and free.
Test the longest-first rule
In a Python prompt, set text to 'Contact: sam.o@example.org.' and items to the set {'sam', 'sam.o@example.org'}. Replace each item with [REDACTED] twice: once looping over sorted(items, key=len, reverse=True) and once over sorted(items, key=len). Predict both results, then compare. When we ran it, the longest first result was 'Contact: [REDACTED].' and the shortest first result was 'Contact: [REDACTED].o@example.org.'
A hint
Use text.replace(item, '[REDACTED]') in each loop and start the second loop again from the original text.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
Four made-up tickets with eight known items, one small local model, UK formats only, one run. A passing score on this set is not compliance and not legal advice. The model returned one wrong item and can miss or reword names. The article on pattern limits is from Protecto, a vendor of data privacy software.
Up next in AI security and governance
More in AI security and governance
Related videos
- #01 Local AI Agent in Python: No API Key (Ollama) AI agents and MCP
- #06 Test Your LLM App with pytest: An Eval Harness in 40 Lines LLM engineering and evals
- #12 Multi-Agent Patterns in Plain Python: Pipeline, Router, Loop AI agents and MCP
- #20 AI Coding Assistant Workflow: Copy, Test, Read the Diff AI coding and productivity
