learning-it by example

Back to the video

Anomaly Detection Basics: z-Score, IQR, Isolation Forest

A z-score found 1 of 3 planted bad readings; a median-based check found all 3.

Video #194 ยท Machine learning and data science

Steps in the video

  1. 0:00 A z-score found 1 of 3 planted bad rows
  2. 0:22 The idea
  3. 0:48 Setup
  4. 1:06 The made-up readings
  5. 1:27 The z-score
  6. 1:53 The median version
  7. 2:20 The box plot fences
  8. 3:17 Two columns
  9. 3:41 An isolation forest
  10. 4:34 What this does not show
  11. 5:05 Mistakes to avoid
  12. 5:34 My take and next steps
  13. 5:59 Recap

Files

  • data.py (22 lines)
  • detect.py (22 lines)
  • run.py (39 lines)

Download the code (zip)

Tested with

Python 3.12.15 | numpy==2.5.4, scikit-learn==1.9.1, scipy==1.18.1, joblib==1.6.0, narwhals==2.27.1, threadpoolctl==3.7.0, cloudpickle==3.1.2

What this does not show

Made-up data with planted anomalies, one seed for the isolation forest, and a contamination value chosen by hand.

Sources

Learning IT by Example.