Anomaly Detection Basics: z-Score, IQR, Isolation Forest
A z-score found 1 of 3 planted bad readings; a median-based check found all 3.
Video #194 ยท Machine learning and data science
Steps in the video
Files
- data.py (22 lines)
- detect.py (22 lines)
- run.py (39 lines)
Tested with
Python 3.12.15 | numpy==2.5.4, scikit-learn==1.9.1, scipy==1.18.1, joblib==1.6.0, narwhals==2.27.1, threadpoolctl==3.7.0, cloudpickle==3.1.2
What this does not show
Made-up data with planted anomalies, one seed for the isolation forest, and a contamination value chosen by hand.
Sources
- https://pypi.org/pypi/numpy/json
- https://pypi.org/pypi/scikit-learn/json
- https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.IsolationForest.html
- https://scikit-learn.org/stable/modules/outlier_detection.html
- https://www.itl.nist.gov/div898/handbook/eda/section3/eda35h.htm
- https://www.itl.nist.gov/div898/handbook/prc/section1/prc16.htm
Learning IT by Example.