learning-it by example

Back to the video

Medallion Architecture: Bronze, Silver, Gold in Python

A messy CSV summed straight from raw says 99 pounds; after bronze, silver and gold it is 86.

Video #263 ยท Data modelling

Steps in the video

  1. 0:00 A messy CSV summed straight from raw says 99. After three layers: 86.
  2. 0:27 The idea
  3. 1:02 Setup
  4. 1:34 Bronze: land it as text
  5. 1:56 Silver rules: dates and countries
  6. 2:30 Silver rules: amounts
  7. 2:52 Silver: keep one row per order
  8. 3:19 Gold: a table for the report
  9. 3:34 Run bronze
  10. 5:00 What this does not show
  11. 5:28 Mistakes to avoid
  12. 6:00 My take and next steps
  13. 6:22 Recap

Files

  • layers.py (64 lines)
  • orders_raw.csv (10 lines)
  • run.py (62 lines)

Download the code (zip)

Tested with

Python 3.12.15

What this does not show

Nine made-up rows in SQLite, not a lakehouse; no Delta tables, Spark or incremental loads, and the pattern is described by vendors.

Sources

Learning IT by Example.