#107 · Data engineering and pipelines · 6.9 minData Contracts: Stop Breaking Downstream
In a made-up feed, a renamed column broke the dashboard; a contract check found 10 problems before any load.
Python 3.12.1531 videos tagged "python sqlite3".
#107 · Data engineering and pipelines · 6.9 minIn a made-up feed, a renamed column broke the dashboard; a contract check found 10 problems before any load.
Python 3.12.15
#110 · Data modelling · 6.9 minThe same seven sales put 105 pounds in the West with type 1, and 40 with type 2.
Python 3.12.15
#111 · Data engineering and pipelines · 6.7 minFull reloads copied 3,011 rows over three loads; a watermark copied 1,010 and missed one late update.
Python 3.12.15
#112 · Data engineering and pipelines · 6.5 minLoading the same day of sales twice with a plain append doubled it from 63.35 to 126.70; three other strategies stayed at 6 rows.
Python 3.12.15
#115 · Data engineering and pipelines · 6.7 minThree changes to a made-up feed: a positional reader loaded all 8 rows without an error, and 4 were wrong.
Python 3.12.15
#122 · SQL and analytics · 6.6 minThree queries asked which customers have no orders; NOT IN returned 0 rows because of one NULL.
Python 3.12.15
#124 · SQL and analytics · 6.6 minA 20-line SQL query builds a cohort retention table; remove DISTINCT and retention shows 150 percent.
Python 3.12.15
#125 · SQL and analytics · 6.7 minSix made-up sales pivot into 3 rows in SQLite, which has no PIVOT or LATERAL keyword.
Python 3.12.15
#126 · SQL and analytics · 6.8 minOne recursive SQL query finds that Asha has 9 people under her; another goes round a loop until a LIMIT stops it.
Python 3.12.15
#127 · SQL and analytics · 6.6 minSix made-up orders total 220, but two joins make the same SUM say 505, with no error.
Python 3.12.15
#128 · SQL and analytics · 6.9 minIn SQLite, a total stored as the text 9.50 passed a filter for totals over 20; a cast fixed it.
Python 3.12.15
#130 · Data modelling · 6.7 minTwo dashboards said 279 and 249 for one March; defined once, both said 150.
Python 3.12.15
#134 · SQL and analytics · 6.4 minTwo missing days made SQLite report 156.6 in sales a day for a week that averaged 111.9 once the gaps were filled.
Python 3.12.15
#135 · SQL and analytics · 6.7 minOn 200,000 made-up orders, one index turned a full scan into a search that was at least 20 times faster.
Python 3.12.15
#250 · Data modelling · 6.6 minWith foreign key checks off, SQLite accepted a repair for bike 99, which does not exist.
Python 3.12.15
#252 · Data modelling · 6.9 minThree ordinary changes broke a flat table of 5 rows; four tables took all three.
Python 3.12.15
#253 · Data modelling · 6.5 minA source renumbered 3 customers; natural and hash keys ended up with 6 rows, a surrogate key kept 3.
Python 3.12.15
#254 · Data modelling · 6.8 minWithout constraints SQLite loaded all 8 rows, 5 of them bad; with constraints only 3 got in.
Python 3.12.15
#255 · Data modelling · 6.5 minFour students' fees add up to 3,200 pounds; summed after joining through a bridge table they came out as 5,500.
Python 3.12.15
#256 · Data modelling · 6.6 minNine order lines, one fact table, three reports, and every report adds up to 64.
Python 3.12.15
#257 · Data modelling · 6.5 minA daily sales report said 108 pounds when the shop sold 54, and one rule fixed it.
Python 3.12.15
#258 · Data modelling · 6.4 minThe same nine order events become three fact tables, and the shipped orders took 2 days on average.
Python 3.12.15
#259 · Data modelling · 7 minJoining two fact tables made 25 pounds of refunds look like 50, and drilling across fixed it.
Python 3.12.15
#261 · Data modelling · 6.9 minOne big table took almost 5 times the space, and renaming one category touched 66,082 rows.
Python 3.12.15
#262 · Data modelling · 6.6 minThree loads into a tiny Data Vault in SQLite, and the third, a repeat, added 0 rows.
Python 3.12.15
#265 · Data modelling · 6.6 minOne price correction stored three ways; only the bitemporal table answers all 4 questions as asked.
Python 3.12.15
#266 · Data modelling · 6.4 minTwo late clicks made an append-only job report 7 sessions; rebuilt from event time there were 4.
Python 3.12.15
#267 · Data modelling · 6.9 minThree orders' shipping adds up to 9 pounds; after flattening the JSON into item rows it summed to 17.
Python 3.12.15
#272 · Data modelling · 6.7 minA retry loaded 3 payments twice into a table with no key, and 75 pounds became 150.
Python 3.12.15
#276 · How it works, by example · 6.5 min200,000 rows: one CREATE INDEX turned a full scan into a search, same answer.
Python 3.12.15
#297 · SQL and analytics · 6.8 minTen SQL queries on one small shop: a left join finds the 1 customer who never ordered.
Python 3.12.15