learning-it by example

Back to the video

Missing Data in pandas: Drop, Fill or Model It?

On made-up data, dropping rows moved a mean from 40,057 to 37,458; a model on age got back to 39,972.

Video #149 ยท Python for data

Steps in the video

  1. 0:00 Dropping rows moved a made-up mean from 40,057 to 37,458
  2. 0:36 The idea
  3. 1:05 Setup
  4. 1:33 The made-up people
  5. 1:51 Three ways to lose values
  6. 2:20 Drop, fill or model
  7. 2:47 What is printed
  8. 4:23 What this does not show
  9. 4:50 Mistakes to avoid
  10. 5:29 My take and next steps
  11. 5:56 Recap

Files

  • data.py (22 lines)
  • fixes.py (16 lines)
  • run.py (17 lines)

Download the code (zip)

Tested with

Python 3.12.15 | numpy==2.5.4, pandas==3.1.0, python-dateutil==3.0.0, six==1.17.0, tzdata==2026.5

What this does not show

Made-up data where the true answer and the reason for each gap are known by design; one simple model, filled once, and no multiple imputation.

Sources

Learning IT by Example.