#187 Machine learning and data science 6.2 min
Feature Engineering Beat 500 Trees, on Made-Up Data
On made-up rents, a straight line with one extra column missed by 31.2 a month; a 500-tree forest on the raw columns missed by 39.4.
Run the four models
Copy the code folder first. This video needs numpy and scikit-learn, as listed in requirements.txt. Run python run.py 200. It grows a thousand trees, so give it a minute or two. Predict which of the four models misses rent by the least. In the video's run, the line with the area column missed by 31.2 and the forest on raw columns missed by 39.4.
A hint
The noise floor line is not a model, it is the made-up rule used as the prediction.
Turn the noise down
In data.py, change rng.normal(0, 40, n) to rng.normal(0, 10, n). Predict what happens to the noise floor and to the line with the area column, then run python run.py 200. In my check of the line models and the noise floor, the line on raw columns missed by 41.1, the line with the area column by 7.8, and the noise floor by 7.7.
A hint
The noise floor is how far off even the true rule is.
Try a width over length column
In features.py, add def with_ratio(X) that returns np.column_stack([X, X[:, 0] / X[:, 1]]). In run.py, import it and add one report line using LinearRegression and with_ratio. Predict whether a ratio helps this line as much as the area did. In my check of the line, the ratio gave a miss of 47.7, far worse than the 31.2 from the area column.
A hint
The made-up rent follows width times length, not width divided by length.
Keep only the area and the floor
In features.py, add a function that returns just two columns: the area, width times length, and the floor. Add a line model using it in run.py. Predict whether dropping the raw width and length columns hurts the line. In my check, it missed by 30.9, which is no worse than the 31.2 with all four columns.
A hint
The rent rule is 300 plus 12 per square metre plus 5 per floor.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
Made-up data where one product explains the rent by design; one seed, one test set, no tuning, no boosted models.
Up next in Machine learning and data science
More in Machine learning and data science
Related videos
- #182 Linear Regression From Scratch, Then scikit-learn Machine learning and data science
- #186 Data Leakage: The Bug That Fakes Accuracy, Reproduced Machine learning and data science
- #193 Clustering: k-means and When It Fails, in Python Machine learning and data science
- #183 Precision, Recall, F1: Which Metric Matters? Machine learning and data science
