learning-it by example

#19 AI security and governance 6.5 min

Redact Personal Data Before Your LLM Sees It (Python)

Patterns found 5 of 8 pieces of personal data, a local model found the rest, and one wrong answer taught me something. A redaction step you can score.

1. Why does the personal data redaction step run before a support ticket is sent to a language model?
2. In the pattern finder run with the Luhn card check, how many of the 8 known personal data items were found, and how many false alarms were there?
3. What does the Luhn check do to a long digit string in the pattern finder?
4. In the combined redaction step, why are the found items replaced longest first?
5. A redaction step removed all 8 known personal data items from four made-up tickets. What does the video say that result shows?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Score the pattern finder

    Copy the code folder first. Run python score_regex.py. This script does not need the Ollama model. Check that you see 5 of 8 items found both with and without the card check, with 1 false alarm without it and 0 with it. Then write down which three items are listed as missed and why the patterns cannot find them.

    A hint

    The missed items are the two names and the street address, which have no single shape.

  2. Break the card number on purpose

    In tickets.py change the card number 4111 1111 1111 1111 to 4111 1111 1111 1112 in both the text and the shaped list of ticket 2. Predict what the with card check line will say, then run python score_regex.py. When we ran it, it said found 4 of 8 items, and the missed list included the changed card number.

    A hint

    The last digit now fails the Luhn check, so the card is no longer kept.

  3. Add a fifth ticket

    In tickets.py add a new ticket before the kettle one, with the text Please write to jo.b@example.com about my parcel. Give it shaped set to that email and free set to an empty list. Predict the new totals, then run python score_regex.py. When we ran it, both lines said 6 of 9 items found.

    A hint

    Copy the layout of an existing ticket: text, shaped and free.

  4. Test the longest-first rule

    In a Python prompt, set text to 'Contact: sam.o@example.org.' and items to the set {'sam', 'sam.o@example.org'}. Replace each item with [REDACTED] twice: once looping over sorted(items, key=len, reverse=True) and once over sorted(items, key=len). Predict both results, then compare. When we ran it, the longest first result was 'Contact: [REDACTED].' and the shortest first result was 'Contact: [REDACTED].o@example.org.'

    A hint

    Use text.replace(item, '[REDACTED]') in each loop and start the second loop again from the original text.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

Four made-up tickets with eight known items, one small local model, UK formats only, one run. A passing score on this set is not compliance and not legal advice. The model returned one wrong item and can miss or reword names. The article on pattern limits is from Protecto, a vendor of data privacy software.

Up next in AI security and governance

More in AI security and governance

All AI security and governance videos

Related videos