learning-it by example

#187 Machine learning and data science 6.2 min

Feature Engineering Beat 500 Trees, on Made-Up Data

On made-up rents, a straight line with one extra column missed by 31.2 a month; a 500-tree forest on the raw columns missed by 39.4.

1. In the made-up rent data, the rent follows the floor area, but the model only gets the two sides of the room. What is the one piece of feature engineering the video adds?
2. With 200 training rows, how far did each model miss the monthly rent on average?
3. Why can a plain straight-line model not use the width and length of the room well on its own to predict rent that follows the area?
4. When the training rows grew from 200 to 2000, what happened to the gap between the forest on raw columns and the line with the area column?
5. Why is the made-up rent example easy for feature engineering?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Run the four models

    Copy the code folder first. This video needs numpy and scikit-learn, as listed in requirements.txt. Run python run.py 200. It grows a thousand trees, so give it a minute or two. Predict which of the four models misses rent by the least. In the video's run, the line with the area column missed by 31.2 and the forest on raw columns missed by 39.4.

    A hint

    The noise floor line is not a model, it is the made-up rule used as the prediction.

  2. Turn the noise down

    In data.py, change rng.normal(0, 40, n) to rng.normal(0, 10, n). Predict what happens to the noise floor and to the line with the area column, then run python run.py 200. In my check of the line models and the noise floor, the line on raw columns missed by 41.1, the line with the area column by 7.8, and the noise floor by 7.7.

    A hint

    The noise floor is how far off even the true rule is.

  3. Try a width over length column

    In features.py, add def with_ratio(X) that returns np.column_stack([X, X[:, 0] / X[:, 1]]). In run.py, import it and add one report line using LinearRegression and with_ratio. Predict whether a ratio helps this line as much as the area did. In my check of the line, the ratio gave a miss of 47.7, far worse than the 31.2 from the area column.

    A hint

    The made-up rent follows width times length, not width divided by length.

  4. Keep only the area and the floor

    In features.py, add a function that returns just two columns: the area, width times length, and the floor. Add a line model using it in run.py. Predict whether dropping the raw width and length columns hurts the line. In my check, it missed by 30.9, which is no worse than the 31.2 with all four columns.

    A hint

    The rent rule is 300 plus 12 per square metre plus 5 per floor.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

Made-up data where one product explains the rent by design; one seed, one test set, no tuning, no boosted models.

Up next in Machine learning and data science

More in Machine learning and data science

All Machine learning and data science videos

Related videos