My First Kaggle Notebook: What Building a “Simple” House Price Model Actually Taught Me
A few weeks ago, I did something a lot of people learning machine learning eventually do: I opened Kaggle, picked the House Prices competition, and told myself “how hard can this be?”
Turns out — not hard to start. Genuinely hard to do well.
Why I picked this one
House Prices: Advanced Regression Techniques is one of Kaggle’s “Getting Started” competitions. No prize money, no medals, just a dataset of home sales and a target: predict SalePrice. It’s the kind of competition people recommend to beginners because the data is small, well-documented, and forgiving.
I didn’t pick it to chase a leaderboard rank. I picked it because I wanted to build something the way I imagined a real ML engineer would — not just a notebook full of df.fillna(0) and a linear regression, but something structured, testable, and reproducible.
Getting to know the data
Before writing a single line of model code, I spent time just looking at the data — plotting distributions, checking what correlated with SalePrice, hunting for outliers. That scatter plot up top (ground living area vs. sale price) was one of the first things I looked at, and it’s a good example of what EDA is actually for: it told me the relationship was mostly linear but not perfectly so, with a handful of large, cheap homes that would later turn out to be genuine outliers worth dropping.
The correlation heatmap was even more useful for feature selection :
A few things jumped out immediately — OverallQual, GrLivArea, and the garage-related features were clearly the strongest predictors of price, while things like MiscVal and PoolArea barely moved the needle. That gave me a rough priority order before I’d written any feature engineering code at all.
The first mistake: treating it like a school assignment
My first instinct was to open a single notebook and just start writing cells top to bottom — load data, clean data, engineer features, train, predict, submit. It worked. It also meant that if I wanted to change one thing about how I handled missing values, I had to scroll through the whole notebook, find every place that logic touched, and hope I didn’t break something three cells down.
That’s when it clicked: notebooks are great for exploring, but terrible for building. So I started over, this time splitting things properly:
├── data/
├── notebooks/
│ └── 01_EDA.ipynb # exploration only
├── src/
│ ├── feature_engineering.py
│ ├── preprocessing.py
│ ├── models.py
│ ├── train.py
│ └── predict.py
├── tests/
└── outputs/The notebook became just for exploring the data — checking distributions, hunting for missing values, looking at correlations with SalePrice. Everything that actually mattered for the model lived in versioned, testable Python files.
The second mistake: not thinking about leakage
Early on, I was doing feature engineering directly on the full dataset before splitting into train/validation. It “worked” — validation scores looked great. Too great. I eventually realized I was letting information from my validation set quietly leak into training through things like scaling and encoding fit on the combined data.
The fix was to make sure every transformation — imputing missing values, encoding categories, scaling — happened inside an sklearn Pipeline, fit only on the training fold. It’s a small technical change, but it’s the difference between a model that looks good in a notebook and a model that actually generalizes.
The third mistake: no tests, no confidence
Halfway through, I changed how I engineered one feature and my validation RMSE quietly got worse. I had no idea why until I spent an hour manually re-checking outputs. That was the moment I added a tests/ folder and wrote unit tests for the feature engineering logic specifically — since that’s the part most likely to break silently when you’re iterating fast.
Get Pratik Bothra’s stories in your inbox
Join Medium for free to get updates from this writer.
It felt like overkill for a Kaggle side project. It wasn’t. It saved me from shipping a broken submission at least twice.
What I actually built
By the end, the project had:
- A LightGBM regression pipeline — chosen for its strong out-of-the-box performance on tabular data and native handling of categorical features
- A proper CLI for training and inference (python -m src.train, python -m src.predict) instead of manually running notebook cells
- Reproducibility baked in — fixed random seeds, leak-proof preprocessing pipelines, pinned dependencies (I had to deliberately keep NumPy below 2.0 to avoid breaking compatibility with older scientific-Python packages — a real issue I only found by hitting it)
- Unit tests for the parts of the code most likely to silently fail
The result
My model scored 0.12574 RMSLE on the public leaderboard, landing around rank 1431 out of roughly 4,500+ teams — solidly mid-pack, nothing to write home about in leaderboard terms.
And honestly? That’s fine. This competition’s top scores are widely known to be inflated — since the real sale prices are public real estate records, some leaderboard-topping submissions aren’t really “modeling” anything. Chasing rank here would’ve been chasing the wrong thing.
What I actually got out of it
Not a top percentile. What I got was:
- A pipeline I could actually explain, line by line, to someone else
- An instinct for where data leakage creeps in
- A habit of writing tests before I trust my own code
- A project structure I’ll reuse on the next one, instead of starting from a blank notebook again
If you’re starting your first Kaggle project too — my honest advice is to worry less about your leaderboard position and more about whether you can explain every decision in your pipeline. That’s the skill that actually transfers.
Kaggle :
Github Repo :
