#111 · Data engineering and pipelines · 6.7 minIncremental Loads: Watermarks and the Late Update Trap
Full reloads copied 3,011 rows over three loads; a watermark copied 1,010 and missed one late update.
Python 3.12.154 videos tagged "data pipeline".
#111 · Data engineering and pipelines · 6.7 minFull reloads copied 3,011 rows over three loads; a watermark copied 1,010 and missed one late update.
Python 3.12.15
#112 · Data engineering and pipelines · 6.5 minLoading the same day of sales twice with a plain append doubled it from 63.35 to 126.70; three other strategies stayed at 6 rows.
Python 3.12.15
#115 · Data engineering and pipelines · 6.7 minThree changes to a made-up feed: a positional reader loaded all 8 rows without an error, and 4 were wrong.
Python 3.12.15
#263 · Data modelling · 6.8 minA messy CSV summed straight from raw says 99 pounds; after bronze, silver and gold it is 86.
Python 3.12.15