learning-it by example

Back to the video

Feature Engineering Beat 500 Trees, on Made-Up Data

On made-up rents, a straight line with one extra column missed by 31.2 a month; a 500-tree forest on the raw columns missed by 39.4.

Video #187 ยท Machine learning and data science

Steps in the video

  1. 0:00 One extra column: a miss of 31.2, not 39.4
  2. 0:33 The idea
  3. 1:04 Setup
  4. 1:26 The made-up flats
  5. 1:57 One new column
  6. 2:22 Score each model
  7. 2:38 Four models
  8. 4:03 What this does not show
  9. 4:38 Mistakes to avoid
  10. 5:09 My take and next steps
  11. 5:42 Recap

Files

  • data.py (13 lines)
  • features.py (12 lines)
  • run.py (32 lines)

Download the code (zip)

Tested with

Python 3.12.15 | numpy==2.5.4, scikit-learn==1.9.1, scipy==1.18.1, joblib==1.6.0, narwhals==2.27.1, threadpoolctl==3.7.0, cloudpickle==3.1.2

What this does not show

Made-up data where one product explains the rent by design; one seed, one test set, no tuning, no boosted models.

Sources

Learning IT by Example.