learning-it by example

Back to the video

Exploratory Data Analysis in pandas: A Rerunnable Checklist

Seven pandas checks found 100 gaps, 25 repeats, 12 region spellings and 7 bad rows in made-up orders.

Video #148 ยท Python for data

Steps in the video

  1. 0:00 A checklist found 100 gaps, 25 repeats, 12 spellings and 7 bad rows
  2. 0:30 The idea
  3. 1:02 Setup
  4. 1:25 The planted problems
  5. 1:54 Shape, types, missing, repeats
  6. 3:09 Spellings and ranges
  7. 4:06 Tidy, then one relationship
  8. 5:08 What this does not show
  9. 5:35 Mistakes to avoid
  10. 6:00 My take and next steps
  11. 6:21 Recap

Files

  • checks.py (47 lines)
  • data.py (37 lines)
  • run.py (7 lines)

Download the code (zip)

Tested with

Python 3.12.15 | pandas==3.1.0, numpy==2.5.4

What this does not show

Made-up orders with problems I planted; no charts; cut-offs are my own choices; seven checks are not a complete analysis; pandas 3.1.0 and NumPy 2.5.4, two runs with identical output.

Sources

Learning IT by Example.