#213 Data governance 5.5 min
Govern AI Training Data: Provenance, Consent, Dataset Card
Ten records, five used. A dataset card says why the other five were left out.
Record consent for record 6
Copy the code folder first. In records.py, change the consent value of record 6 from None to True, then run python run.py. Predict first how many records are used and whether the fingerprint changes. When I ran it, 6 were used, 4 were left out, the reasons showed no consent 1, and the fingerprint started 131ee1e4e612. Undo this change afterwards, so the next exercises give the numbers they state.
A hint
Record 6 holds personal data. Unknown consent counts as no consent, but True is accepted.
Allow the partner licence
In curate.py, add "partner-only" to the ALLOWED set, then run python run.py. Predict which record changes and what the Licences line gains. When I ran it, 6 records were used, the card listed partner-only 1 under Licences, and licence not allowed disappeared from the reasons. Undo this change afterwards, so the next exercises give the numbers they state.
A hint
Record 5 is the only one with the licence partner-only.
Give record 4 an opt out
At the end of records.py, add the line RECORDS[3]["opt_out"] = True. Record 4 has no licence. Predict which reason the log shows for record 4 now, and whether the fingerprint changes. When I ran it, record 4 showed opt-out, the reasons were licence not allowed 1, no consent 2, opt-out 2, and the fingerprint stayed 6f31a401470b.
A hint
Look at the order of the if statements in decide, and at which records go into the hash.
Change the order of the checks
In curate.py, move the no licence check above the opt-out check, with record 4 still marked as an opt out from the previous task. Predict what the log shows for record 4, then run it and compare. Write one sentence on why the order of checks changes the reasons you report.
A hint
The first check that fails is the only reason returned.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
Ten made-up records and my own rules; whether a licence or consent covers training is a legal question, the fingerprint covers record ids only, and this is not legal advice.
Up next in Data governance
More in Data governance
- #203 A Business Glossary in YAML With Four Python Checks
- #204 Tag PII Columns in Python, Including a False Positive
- #205 Column-Level Lineage from SQL with SQLGlot
- #208 Right to Erasure Across a Pipeline: Delete, Scan, Restore
- #209 UK GDPR Data Map as Code: A Practical Checklist
- #210 A Data Quality Scorecard That Shows a Hidden Failure
