learning-it by example

Back to the video

Imbalanced Data: Class Weights vs Threshold vs Resampling

A plain model caught 8 of 39 made-up frauds; class weights, a tuned threshold and resampling each caught 30 to 34.

Video #200 ยท Machine learning and data science

Steps in the video

  1. 0:00 A plain model caught 8 of 39 frauds
  2. 0:31 The idea
  3. 0:57 Setup
  4. 1:18 Three splits
  5. 1:44 Resampling by hand
  6. 2:13 A threshold, tuned on validation
  7. 2:40 Train four models
  8. 3:49 Reading the table
  9. 4:23 What this does not show
  10. 5:02 Mistakes to avoid
  11. 5:35 My take and next steps
  12. 6:06 Recap

Files

  • data.py (11 lines)
  • fixes.py (25 lines)
  • run.py (35 lines)

Download the code (zip)

Tested with

Python 3.12.15 | numpy==2.5.4, scikit-learn==1.9.1, scipy==1.18.1, joblib==1.6.0, narwhals==2.27.1, threadpoolctl==3.7.0, cloudpickle==3.1.2

What this does not show

Made-up data with two columns, one model, one seed and one split, with only 39 frauds in the test set.

Sources

Learning IT by Example.