Exploratory Data Analysis in pandas: A Rerunnable Checklist
Seven pandas checks found 100 gaps, 25 repeats, 12 region spellings and 7 bad rows in made-up orders.
Video #148 ยท Python for data
Steps in the video
Files
- checks.py (47 lines)
- data.py (37 lines)
- run.py (7 lines)
Tested with
Python 3.12.15 | pandas==3.1.0, numpy==2.5.4
What this does not show
Made-up orders with problems I planted; no charts; cut-offs are my own choices; seven checks are not a complete analysis; pandas 3.1.0 and NumPy 2.5.4, two runs with identical output.
Sources
- https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.duplicated.html
- https://pandas.pydata.org/docs/reference/api/pandas.Series.corr.html
- https://pandas.pydata.org/docs/user_guide/missing_data.html
- https://pypi.org/pypi/numpy/2.5.4/json
- https://pypi.org/pypi/pandas/3.1.0/json
- https://www.itl.nist.gov/div898/handbook/eda/section1/eda11.htm
Learning IT by Example.