#146 Python for data 6 min
Test Data Code with pytest: Fixtures and Frames
Six pytest tests pass, and one catches a column that came back float64 instead of int64.
Predict the failing line
Copy the code folder first. Read clean.py and test_naive.py without running them. Predict which column and which attribute the failing test reports, and what the left and right values will be. Then, with pandas, NumPy and pytest installed as in the video, run python -m pytest -qq --tb=short test_naive.py and compare. The video's run reported column qty with dtype float64 on the left and int64 on the right.
A hint
The values 2, 0, 0, 5 are right in both frames. Only one attribute differs.
Fix the naive fill
In clean.py change fill_qty_naive so that it ends with .astype("int64") after fillna(0), the same last step that clean_orders uses. Predict whether test_naive.py now passes. Then run python -m pytest -qq test_naive.py to check.
A hint
The only difference between the naive fill and the real one is the type step.
Add a fifth spelling
In test_clean.py add one more spelling to the parametrize list, for example "North " or " nOrth". Predict how many tests pytest reports for test_clean.py, given that the run in the video showed six. Then run python -m pytest -qq test_clean.py to check.
A hint
Each spelling counts as its own test.
Break a rule on purpose
In test_clean.py change the rule set(out["region"]) == {"North", "South", "East"} so the set also contains "West". Predict which test fails, and whether the small example test still passes. Then run python -m pytest -qq test_clean.py to check.
A hint
The small example test does not touch the thousand-row fixture.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
Made-up data and a tiny function; the failing test shows a bug set up on purpose, not one found in real work.
Up next in Python for data
More in Python for data
- #08 Pandas vs Polars vs DuckDB on 10M Rows, Timed and Checked
- #09 Clean Messy Data with Pandas 3: End to End, 3 Mistakes
- #137 Polars for pandas Users: One Report, Translated
- #138 Seven pandas Changes Tested on 1M Rows: Speed and Memory
- #139 Validate Data with Pydantic: Catch Bad Rows
- #140 Python Dates and Time Zones: 5 Bugs Reproduced
Related videos
- #139 Validate Data with Pydantic: Catch Bad Rows Python for data
- #20 AI Coding Assistant Workflow: Copy, Test, Read the Diff AI coding and productivity
- #203 A Business Glossary in YAML With Four Python Checks Data governance
- #307 Schema Drift: Map a Customer's Data and Catch Changes Forward deployed engineer interviews