Train/Test Split: Don't Fool Yourself
Here is the mistake every beginner makes: they train a model, test it on the same data, see 99% accuracy, and ship it. Then it fails on real users. Why? The model memorised the examples instead of learning the pattern. Grading it on data it already saw is like giving students the exam answers beforehand.
The fix: hold data back
Split your data into two piles before training:
- Training set — the model learns only from this.
- Test set — locked away; used once, at the end, to measure honest accuracy.
train_test_split does it for you. test_size=0.25 keeps 25% for testing; random_state=0 makes the split reproducible so your results don't wander every run.
Measure error, not vibes
For a number you predict (regression), MAE (mean absolute error) tells you, on average, how many units you're off by. Lower is better, and MAE is in the same unit as the label (here, lakhs) so it's easy to explain to a human.
The golden rule
Never let the test set touch training — not the fit, not the feature scaling, not the tuning. The moment it leaks, your accuracy number becomes a lie.
Try it
- Change
test_sizeto0.5. Does test MAE get noisier with fewer training rows? - Remove
random_state. Run twice — why do the numbers change?
3 graded exercises
Write the code, click Check. We’ll tell you exactly what to fix.
- Exercise 1
Write
split_sizes(X, y, test_size)that splits withrandom_state=0and returns(n_train, n_test). - Exercise 2
Write
test_mae(X, y)that splits (25% test,random_state=0), fitsLinearRegressionon train, and returns the MAE on the test set. - Exercise 3
A colleague fits and scores on the same data and brags about 100% accuracy. In one line, set
mistaketo the string"data leakage".