#189 Machine learning and data science 6.3 min
Random Forest Explained in Code: Bagging by Hand
One deep tree scored 0.728 on test; a hundred random trees together scored 0.842.
Run and compare the forests
Copy the code folder first. This video needs numpy and scikit-learn, as listed in requirements.txt. Run python run.py. Compare the test accuracy of the one full tree with your forest of 1, 5, 25 and 100 trees. In the video's run, the single tree scored 0.728 on test and your hundred-tree forest scored 0.842.
A hint
The first line after the data line also shows the training accuracy of the single tree.
Predict more trees
In run.py, change the list [1, 5, 25, 100] to [1, 5, 25, 100, 400]. Predict whether the accuracy with 400 trees will be much higher than with 100, then run it and compare. The video quotes the docs: results stop getting much better beyond a critical number of trees.
A hint
Look at how much the score changed from 25 to 100 trees.
Predict with no flipped labels
In data.py, change flip=0.1 to flip=0.0, so no label is wrong on purpose. Predict what happens to the test accuracy of the single tree and of the forest of 100 trees, then run python run.py and compare. Say in one sentence why the flipped labels made things harder.
A hint
With flipped labels, even a perfect model would miss about one in ten.
Change the forest seed
In forest.py, the function fit_forest has seed=0. Change it to seed=1 and run python run.py again. Predict which lines change, then compare with the first run. What does this tell you about comparing two forests on one seed?
A hint
The seed decides which rows each tree draws.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
Made-up data, one train and test split, one seed, and no tuning; the trees themselves come from scikit-learn.
Up next in Machine learning and data science
More in Machine learning and data science
- #182 Linear Regression From Scratch, Then scikit-learn
- #183 Precision, Recall, F1: Which Metric Matters?
- #184 Overfitting You Can See: Train vs Validation Error
- #185 Cross-Validation Done Right: The Leakage Mistake
- #186 Data Leakage: The Bug That Fakes Accuracy, Reproduced
- #187 Feature Engineering Beat 500 Trees, on Made-Up Data
Related videos
- #182 Linear Regression From Scratch, Then scikit-learn Machine learning and data science
- #183 Precision, Recall, F1: Which Metric Matters? Machine learning and data science
- #186 Data Leakage: The Bug That Fakes Accuracy, Reproduced Machine learning and data science
- #187 Feature Engineering Beat 500 Trees, on Made-Up Data Machine learning and data science
