Feature Engineering Beat 500 Trees, on Made-Up Data
On made-up rents, a straight line with one extra column missed by 31.2 a month; a 500-tree forest on the raw columns missed by 39.4.
Video #187 ยท Machine learning and data science
Steps in the video
Files
- data.py (13 lines)
- features.py (12 lines)
- run.py (32 lines)
Tested with
Python 3.12.15 | numpy==2.5.4, scikit-learn==1.9.1, scipy==1.18.1, joblib==1.6.0, narwhals==2.27.1, threadpoolctl==3.7.0, cloudpickle==3.1.2
What this does not show
Made-up data where one product explains the rent by design; one seed, one test set, no tuning, no boosted models.
Sources
- https://developers.google.com/machine-learning/crash-course/categorical-data/feature-crosses
- https://en.wikipedia.org/wiki/Feature_engineering
- https://pypi.org/pypi/numpy/json
- https://pypi.org/pypi/scikit-learn/json
- https://scikit-learn.org/stable/auto_examples/applications/plot_cyclical_feature_engineering.html
- https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html
- https://scikit-learn.org/stable/modules/tree.html
Learning IT by Example.