learning-it by example

Back to the video

Audit Evidence From Pipeline Logs in Python (SHA-256)

Six made-up pipeline runs, three gaps: an evidence report built straight from the logs.

Video #216 ยท Data governance

Steps in the video

  1. 0:00 Three gaps in six pipeline runs
  2. 0:23 The idea
  3. 1:07 Setup
  4. 1:28 The log
  5. 2:00 Control one
  6. 2:14 Control two
  7. 2:38 Control three
  8. 3:00 The report and the hash
  9. 3:37 Run it
  10. 4:13 The copy and the check
  11. 4:33 What this does not show
  12. 5:07 Mistakes to avoid
  13. 5:40 My take and next steps
  14. 6:06 Recap

Files

  • controls.py (60 lines)
  • report.py (40 lines)
  • run.py (24 lines)
  • runs.jsonl (16 lines)

Download the code (zip)

Tested with

Python 3.12.15

What this does not show

Made-up runs and invented controls. This is evidence, not compliance: the hash shows the log did not change after hashing, not that it was true.

Sources

Learning IT by Example.