learning-it by example

Back to the video

Tag PII Columns in Python, Including a False Positive

A 25-line PII scanner tagged email, phone and postcode right, and order references wrong.

Video #204 ยท Data governance

Steps in the video

  1. 0:00 A 25-line scanner tagged most columns right and one wrong, and I left the wrong one in
  2. 0:36 The idea
  3. 1:06 Setup
  4. 1:38 The sample table
  5. 2:10 The patterns
  6. 2:44 Count the matches
  7. 3:02 Decide the tag
  8. 3:28 Run the scan
  9. 4:13 What this does not show
  10. 4:49 Mistakes to avoid
  11. 5:12 My take and next steps
  12. 5:38 Recap

Files

  • detect.py (27 lines)
  • run.py (6 lines)
  • sample.py (14 lines)

Download the code (zip)

Tested with

Python 3.12.15

What this does not show

Made-up table of six rows; the patterns and the 60 percent line are my choices, and names or addresses in free text are missed.

Sources

Learning IT by Example.