learning-it by example

#215 Data governance 5.5 min

Pseudonymise or Anonymise? Test It on Fake Data

I hashed a thousand emails. A list of guesses gave all of them back. Then I dropped the names, and 998 rows were still unique.

1. Why is hashing email addresses with plain SHA-256 not anonymising them?
2. In the Python test that hashed 1000 made-up emails and tried to reverse them with a list of guesses, how many emails were recovered from each kind of published hash?
3. In the Python k-anonymity test on 1000 made-up people, after the names were dropped, exact age, postcode and sex were left. How many rows could be picked out?
4. In the Python k-anonymity test on 1000 made-up people, after generalising and removing groups under 5, no row was alone. What risk remained?
5. In the Python hash-guessing test on 1000 made-up emails, the 1000 of 1000 result was certain. Why should a real attacker be expected to recover fewer?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Hide the salt from the attacker

    Copy the code folder first. In pseudo.py, change the salted line in SCHEMES so the attacker uses plain instead of salted, as if the attacker never saw the salt. Predict the three results, then run python pseudo.py. When I ran it, plain recovered 1000 of 1000, and salted and keyed both recovered 0 of 1000. Explain in one sentence why the video's version recovered all 1000 for salted. Undo this change afterwards, so the next exercises give the numbers they state.

    A hint

    In the video the salt is stored with the data, so the attacker can add it to every guess.

  2. Shrink the guess list

    In pseudo.py, change the guesses line so it only uses the first 20 first names, by writing FIRST[:20] in place of FIRST. Predict whether the plain and salted results stay at 1000, then run it. When I ran it, plain and salted each recovered 506 of 1000 and keyed still recovered 0.

    A hint

    The guess list now holds fewer addresses than the data contains.

  3. Raise the smallest allowed group

    In kanon.py, change the number 5 in the safe line to 10. Before you run python kanon.py, predict whether more or fewer rows will remain in the groups removed line. When I ran it, 778 rows remained and the smallest group was k = 10, but the riskiest group of 13 still showed asthma 77%. Why does the last line not change? Undo this change afterwards, so the next exercises give the numbers they state.

    A hint

    The riskiest group has 13 rows, which is above both thresholds.

  4. Drop a column from the key

    In kanon.py, remove sex from both the raw and the coarse functions, so each returns only age and postcode. Predict how the rows alone figure changes for the age band and postcode district case, then run it. When I ran it, rows alone became 0, the smallest group was k = 2, and the riskiest group was 5 rows with diabetes 60%. Write one sentence on what that tells you about the choice of columns.

    A hint

    Fewer columns in the key make groups bigger, so fewer rows are alone.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

Made-up data. The guess list holds every possible email and the join uses the same thousand people, so real results will differ. Not legal advice; the ICO guidance is under review.

Up next in Data governance

More in Data governance

All Data governance videos

Related videos