#101 AI security and governance 5.4 min
Differential Privacy Basics: Noisy Counts in Python
A made-up count of 412 told a simulated attacker whether one person was in the data.
Try epsilon 0.5
Copy the code folder first. In compare.py change the loop for epsilon in (10, 1, 0.1) to for epsilon in (0.5,) and run it. Predict the typical error and how often the attacker is right, then compare with the lines for 1 and 0.1 in the video. When I ran it the typical error was 2.0 and the attacker was right 62% of the time.
A hint
Epsilon 0.5 sits between epsilon 1 and epsilon 0.1.
Ask five times instead of twenty
In compare.py change guess_right(1, 20) to guess_right(1, 5) and the words in the print text to match. Predict whether the attacker does better or worse than with 20 questions and than with 1 question. When I ran it the attacker was right 80% of the time at epsilon 1 with 5 questions, between 70% for one question and 94% for twenty.
A hint
More questions means more fresh noise to average away.
Change the seed
In compare.py change random.Random(7) to random.Random(8) and run it. Predict which lines change. When I ran it only the typical error at epsilon 0.1 changed, from 9.9 to 10.2, and the percentages stayed the same. Then explain why the video says the seed makes the numbers repeat, but a real system needs a carefully built noise source.
A hint
The numbers in the video repeat only on the same Python version with the same seed.
Spend the budget on a tiny epsilon
Change guess_right(1, 20) in the last print to guess_right(0.1, 20), leaving the printed words alone, and run it. Predict whether twenty averaged answers at epsilon 0.1 help the attacker as much as at epsilon 1. When I ran it the attacker was right 56% of the time. Compare it with 52% for one answer at epsilon 0.1.
A hint
A smaller epsilon means much more noise in every answer.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
One made-up count and one made-up person; a seeded teaching simulation in plain Python against an attacker who knows both possible counts; not noise you should use in production.
Up next in AI security and governance
More in AI security and governance
- #18 Prompt Injection Live: Attacking My Own LLM App
- #19 Redact Personal Data Before Your LLM Sees It (Python)
- #88 Indirect Prompt Injection: A Note That Gave Orders
- #89 Jailbreak-Test Your Own App: Eight Wrappers, Counted
- #90 Stop Leaking API Keys: Build a Secret Scanner in Python
- #91 Rate Limiting an LLM API: Token Bucket in FastAPI
Related videos
- #100 Fake but Realistic Test Data in Python With Faker AI security and governance
- #208 Right to Erasure Across a Pipeline: Delete, Scan, Restore Data governance
- #214 Build an AI Model Register and Risk Score Data governance
- #229 A Reference Architecture for Enterprise AI Agents, as Code Enterprise architecture and AI agents
