Precision, Recall, F1: Which Metric Matters?
A model that never flags fraud scored 98% accuracy and caught 0 of 20 frauds.
Video #183 ยท Machine learning and data science
Steps in the video
Files
- data.py (12 lines)
- metrics.py (20 lines)
- run.py (26 lines)
Tested with
Python 3.12.15 | numpy==2.5.4, scikit-learn==1.9.1, scipy==1.18.1, joblib==1.6.0, narwhals==2.27.1, threadpoolctl==3.7.0, cloudpickle==3.1.2
What this does not show
Made-up payments scored by a pretend model with two thresholds and made-up prices, compared on the same rows.
Sources
- https://developers.google.com/machine-learning/crash-course/classification/accuracy-precision-recall
- https://pypi.org/pypi/numpy/json
- https://pypi.org/pypi/scikit-learn/json
- https://scikit-learn.org/stable/modules/generated/sklearn.metrics.precision_score.html
- https://scikit-learn.org/stable/modules/model_evaluation.html
Learning IT by Example.