learning-it by example

Back to the video

Random Forest Explained in Code: Bagging by Hand

One deep tree scored 0.728 on test; a hundred random trees together scored 0.842.

Video #189 ยท Machine learning and data science

Steps in the video

  1. 0:00 One tree 0.728, a hundred trees 0.842
  2. 0:33 The idea
  3. 1:04 Setup
  4. 1:23 The made-up data
  5. 1:50 Grow the trees
  6. 2:26 Average the answers
  7. 2:48 The run
  8. 3:52 What this does not show
  9. 4:36 Mistakes to avoid
  10. 5:15 My take and next steps
  11. 5:49 Recap

Files

  • data.py (11 lines)
  • forest.py (19 lines)
  • run.py (20 lines)

Download the code (zip)

Tested with

Python 3.12.15 | numpy==2.5.4, scikit-learn==1.9.1, scipy==1.18.1, joblib==1.6.0, narwhals==2.27.1, threadpoolctl==3.7.0, cloudpickle==3.1.2

What this does not show

Made-up data, one train and test split, one seed, and no tuning; the trees themselves come from scikit-learn.

Sources

Learning IT by Example.