#145 Python for data 5.9 min
Type Hints and mypy: Catch Data Bugs Before You Run
Mypy reported 4 errors from 3 bugs without running the code, and missed a wrong average.
Fix the blind bug
Copy the code folder first. In blind.py change the divisor (len(rows) + 1) to len(rows), then run python blind.py. Predict the average quantity before you run it. When I ran it, the average printed 2.33, the same as the by-hand line.
A hint
The by-hand line is 7 divided by 3.
A lookup that does find the order
In run.py change the order id "9" to "3" in the region_of call, then run python run.py bad and python run.py good. Predict which of the two runs still crash. When I ran it, the region line printed North in both and the bad run still crashed on revenue. The label still says order 9, which you can change too. This shows why a bug that only appears for a missing order can hide.
A hint
The None problem only happens when find_order finds nothing.
Look at an empty quantity
In run.py change mod.quantity(rows[0]) to mod.quantity(rows[1]). Order 2 has an empty qty field. Predict what quantity prints for bad and for good. When I ran it, bad printed an empty string in quotes, and good printed 0. The label still says order 1.
A hint
A real project should choose the default of 0 on purpose.
Predict a new mypy error
First set the order id in run.py back to "9" if you changed it in the second exercise. In good.py change the end of region_of from else "unknown" to else 0. Before you run mypy, predict whether mypy will report an error and which part of the message will mention str and int. Then, if mypy is installed as in the video, run python -m mypy good.py to check. Running python run.py good still works because Python does not enforce the hint. When I ran the program, the region line printed 0.
A hint
The function says it returns str.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
Made-up orders and three bugs I wrote on purpose; standard library only, so no data frames were tested; one mypy version, two runs; mypy skips functions without hints by default.
Up next in Python for data
More in Python for data
- #08 Pandas vs Polars vs DuckDB on 10M Rows, Timed and Checked
- #09 Clean Messy Data with Pandas 3: End to End, 3 Mistakes
- #137 Polars for pandas Users: One Report, Translated
- #138 Seven pandas Changes Tested on 1M Rows: Speed and Memory
- #139 Validate Data with Pydantic: Catch Bad Rows
- #140 Python Dates and Time Zones: 5 Bugs Reproduced
Related videos
- #113 Parquet Explained: Why a Reader Has Less to Read Data engineering and pipelines
- #114 Partitioning Parquet: Which Layout Opens Fewer Files? Data engineering and pipelines
- #205 Column-Level Lineage from SQL with SQLGlot Data governance
- #216 Audit Evidence From Pipeline Logs in Python (SHA-256) Data governance
