Decision Trees & Feature Importance
A decision tree learns a flowchart of yes/no questions: "petal length < 2.5? → setosa." It's the most human-readable model in ML — you can literally print the rules and hand them to a domain expert.
Why trees earn their keep
- No scaling needed. Trees split on thresholds, so features in different units (cm vs kg vs rupees) work as-is.
- Non-linear. They capture "if A and B but not C" patterns a straight line can't.
- Feature importance for free.
tree.feature_importances_ranks how much each feature reduced uncertainty. This is often the real value of training a tree — not the predictions, but learning which inputs matter.
The knob that controls everything: max_depth
A deep tree can carve the training data into tiny perfect boxes — and memorise noise. That's overfitting (next lesson). max_depth=3 forces the tree to stay simple and generalise. Shallow trees underfit; deep trees overfit; the right depth is found by testing.
Try it
- Print the feature importances. Which single measurement separates the species best?
- Set
max_depth=1. Accuracy drops — the tree is now a single question. What is it splitting on?
3 graded exercises
Write the code, click Check. We’ll tell you exactly what to fix.
- Exercise 1
Write
fit_tree(X, y, depth)returning aDecisionTreeClassifierfitted withmax_depth=depthandrandom_state=0. - Exercise 2
Write
top_feature(model, names)returning the name of the most important feature.namesis the list of feature names. - Exercise 3
Trees don't need feature scaling. Set
needs_scalingto the boolean that answers: does a decision tree require you to scale features?