learning-it by example

Back to the video

Schema Drift: Map a Customer's Data and Catch Changes

A guessed fix for a changed export made two totals 100 times too big, with no error.

Video #307 ยท Forward deployed engineer interviews

Steps in the video

  1. 0:00 The customer's export changed, and a guessed fix made it worse
  2. 0:34 The idea
  3. 1:10 Setup
  4. 1:30 March, and April
  5. 2:03 The mapping
  6. 2:34 The drift check
  7. 3:00 The tempting fix
  8. 4:05 What this does not show
  9. 4:47 Mistakes to avoid
  10. 5:14 My take and next steps
  11. 5:45 Recap

Files

  • drift.py (52 lines)
  • exports.py (16 lines)
  • mapping.py (20 lines)
  • run.py (18 lines)

Download the code (zip)

Tested with

Python 3.12.15

What this does not show

Tiny made-up exports and mapping, five rows each. The rename hint is a spelling guess that a person must confirm. The check only counts values that fail to convert: a value that converts to the wrong number (1.250 meant as 1,250) passes. A program cannot settle day-first against month-first dates. Sources are vendor pages, not standards.

Sources

Learning IT by Example.