Home Credit already owns most of the signal needed for default prediction.
So why should they still purchase external credit scores?
Every time someone applies for a loan, the lender quietly pays a credit bureau for a risk score. They do so for every applicant, as a recurring fee, forever. This is reasonable, because the scores actually help predict default. However, “it works” and “it is worth buying” are not the same thing, so I set out to answer:
How much of a credit model’s power actually comes from the scores a lender buys, versus the data they already own?
To answer this, I used Home Credit’s public dataset, which contains over 300,000 real loan applications across seven linked tables. These include the application record, credit bureau records, past loans, monthly balances, and payment histories. The data also includes three external credit scores, which, for the sake of argument, are assumed to have been purchased from a credit bureau and are the main focus of this project. Before building anything, I performed exploratory data analysis (EDA) on each raw table to understand what I was working with, including distributions, missingness, cardinality, and obvious data-quality issues. During data exploration, two key findings emerged: the signal in the external score and the signal in missingness.
The external signal is real
The first thing I wanted to know during EDA was whether the purchased scores were correlated with default rate, or were just a noisy waste of money. I found that when applicants are sorted by their external score, the default rate climbs steeply: the lowest-scoring group defaults about six times as often as the highest. This tells us that the scores genuinely separate good borrowers from bad ones, and, in fact, I discovered later on that they are the strongest predictors of default in the entire dataset. This meant, from the start, the question was never “are the scores useless?” It was “how much of that strength are credit lenders paying for that could be rebuilt for free internally?”
Missing data is a signal too
The second finding is the kind of thing an EDA profiler surfaces, and most people clean away without a second thought. The data was full of holes. 25% of the application fields were blank, 18% of prior application data was missing, 13% of credit bureau data was missing, and 7% of credit card data was missing, among others.
For most, the reflex is either to impute the missing data or drop applications with gaps. Before I did anything, I dug deeper, and I’m glad I did because the blanks were not random. The more fields an applicant left empty, the more likely they were to default.
Missing data was itself a predictor.
That finding had an immediate impact on my model selection. I used a gradient-boosted tree, specifically because a tree can split on a blank and treat “missing” as its own value. I chose LightGBM over XGBoost and other gradient boosting models for its faster training times and native categorical handling, which removed an encoding step from the pipeline. A regression or a neural network would have forced me to either impute/fill the missing values or remove applicants with missing data entirely, potentially erasing the exact signal I had found.
One model, only the data changes
The trap with a question like this is that it can accidentally be answered using model optimization rather than the data. Build a fancier model for the internal features, and of course the numbers move. When the model is optimized for each feature set, there is no way to tell whether the improvement was due to the data or to better tuning.
So, I did the opposite. I locked one model to the exact same configuration in every single run and only ever changed the data it was trained on. There were three setups: the purchased scores alone, the features I engineered from the bank’s own data alone, and the two combined. Because the model itself remained fixed, any differences in the results can be attributed to changes in the features it was trained on. I scored every version with AUC: a single number for how well the model ranks applicants from safest to riskiest, which is exactly the decision a lender makes (approve the safe ones, decline the risky ones). Additionally, AUC is the standard metric for this problem, so the results are comparable to other public solutions.
On the internal data side, I engineered new features, bringing the total to over 3,500. This covered per-table summary statistics (means, maxes, sums, and standard deviations across each applicant’s history), hand-built financial ratios like credit-to-income and annuity-to-income, recency windows and trends over past loans and payments, active-versus-closed and approved-versus-refused breakdowns, cross-table signals that only exist when tables are joined together, and a final round of machine-learned features that included principal components, KMeans cluster distances, and an autoencoder’s compressed embedding plus its reconstruction error.
Then, instead of guessing which of those 3,500+ features mattered, I selected the valuable ones using null importance. The idea is simple: fit the model once on the real target to get each feature’s true importance, then refit it many times on a randomly shuffled target and record how much importance each feature picks up purely by chance. Anything whose real importance does not clearly beat its own by-chance score is noise, and gets cut. That reduced the roughly 3,570 candidates to 442. Then a final backward-elimination pass was done repeatedly dropping features and keeping only those whose removal actually hurt the model. This step removed just one feature, which was itself a sign the null-importance screen had already been tight. That left 441 final features.
Before settling on null importance, I tried Boruta, a stricter, shadow-feature version of the same permutation idea, in which a feature must beat the best of thousands of shuffled copies to survive. The issue I found with using Boruta for this dataset was that it was far too aggressive. It immediately cut the set to around 50 features, and the model’s accuracy dropped. Because of this, I reverted back to null importance, which preserves the long tail of weak-but-real signal that Boruta throws away.
Furthermore, none of the internally engineered features ever interacted with the external scores, so the comparison between internal and external signals remained clean.
The purchased scores look unbeatable
Before I trusted any comparison, I looked at what the model actually leans on when all the features were combined. Global feature importance ranks every feature by how much it drives the model’s decisions, and the three externally purchased scores sat right at the top, towering over everything I had engineered.
Feature importance is the most common way people judge what a model needs, and by that measure the purchased scores are non-negotiable. Anyone glancing at this chart would conclude the lender has to keep paying, full stop.
Get Morgan Cooper’s stories in your inbox
Join Medium for free to get updates from this writer.
Is this really the right conclusion to come to?
Built beats bought
When separating the three external scores from my internally engineered data, my features alone predicted default better than the external scores did, with an AUC of ~0.77 versus ~0.72. Adding the purchased scores back on top raised it to ~0.79. The external scores look dominant in isolation because each one is a dense, pre-packaged summary, but spread across hundreds of features, the data Home Credit already owns simply carries the majority of the default-prediction power.
This completely flipped the intuition I started with. Most of the predictive power is not something a credit lending desk should buy, since it is already owned. This led to the next question.
If the internal lending data is that strong, can I just rebuild the external scores from it?
The scores are only half-rebuildable
Instead of just predicting default, I trained a model on the purchased scores themselves and tried to reconstruct each one from the internal data. The logic is simple: if I can rebuild a score cleanly from data the lending desk already has, then they were never really paying for anything unique.
I found that the model could reconstruct part of each score, but never more than about half of any single score, so none were fully reproducible. Two came close to that halfway mark, leaning on the obvious data the bank already collects, such as age, region, and phone history. The third barely moved: no matter what I fed it, I could only reconstruct about a fifth. The other four-fifths comes from entirely outside the lender’s walls, genuinely unique information I could not recreate at any strength.
That gives a clear picture, all in one test. Even with all the credit desk’s internal data, there is a small, stubborn slice of predictive power we never close in on. This is what the lending desk is really paying for when it purchases an external score, even though most of its predictive power is already owned.
The lift is worth real money
Here is the tension that makes this project interesting. Adding the scores was “only” a two-point jump in AUC, from ~0.77 to ~0.79, which just sounds like a rounding error. That is exactly why I did not stop at the accuracy figure: a metric that looks like a rounding error can still be worth a fortune, and the only way to know is to put a price on it.
So I simulated the lending decision across the 300,000+ loans, approving the safest 90% of applicants under each model, and then counted how many of those approved loans actually defaulted. Better risk ranking means fewer bad loans slip into the book.
With the external scores in the mix, about 850 fewer approved loans default: 16,159 without them, 15,311 with them. That said, the external signal is not magic. At the same approval rate, the model trained with the external signal on top reshuffles the book, correctly removing 2,471 bad loans that the internal-only model would have approved, but incorrectly allowing 1,623 other bad loans to be approved that would otherwise have been declined. Even so, it catches far more than it lets in, for a net gain of about 850 fewer defaults.
Since it’s so good, what if we skipped engineering anything from the internal data and just bought the scores?
On their own, the scores are weakest
I tested this too, and a model trained only on the three external scores approves 1,453 more bad loans (17,612 versus 16,159) than a model built only on the internal data. On their own, the purchased scores are the weakest of the three options.
The score helps in every segment
The score’s value is not uniform, but it is never zero. I broke its lift out across the book more than a dozen ways, by file thickness, age, loan size, employment length, education, region, income, and contract type, and it stayed positive in every single group. This is what makes “buy it for everyone” safe. There is no slice of the book where an external score stops paying.
The gains cluster where the bank’s own data is weakest. The clearest case is loan size: the internal model reads the largest loans worst (0.739, its weakest slice by loan value), and those are exactly the loans where a default costs the most — so the score earns its single largest lift in the table, +0.031, right where the money is. The same shape holds for applicants with short employment histories and thin internal records. Where the bank already reads someone well, the score adds less, but it always adds something.
One caveat worth naming: on its own, the model is weakest for the oldest applicants and pensioners (an AUC of 0.724, below the rest of the book). The external score helps this group more than the average (+0.028 versus +0.024 for the book), but it does not close the gap — even with it, over-60 applicants score 0.752 versus 0.794 overall. That residual is a fairness issue a lender would need to handle separately, not something the external score fixes.
Buy it across the board
For this illustration, assume a defaulted loan loses 60% of its value. Since the dataset masks the currency type, let’s discuss it in terms of currency units (CU). On that basis, adding the external scores is worth an average of ~951 CU per applicant, which, across the book’s 307,511 applicants, comes to about ~292.5 million CU in prevented losses. That ~951 is the average over the whole book (about 0.159% of an average loan). What it is worth on any given loan depends on the loan’s size:
- On a small 10,000 CU loan, 0.159% is ~16 CU.
- On the median loan in this dataset (~513,500 CU), ~815 CU.
- On the largest loans (top 1%, averaging ~2.14M CU), ~3,400 CU.
So the break-even is ~951 CU per applicant, or about 0.16% of the loan being underwritten. Below that, the score pays for itself, and it pays for itself everywhere. The lift stayed positive in every segment I tested, so there is no slice of the book where the calculation flips and no group you would want to skip.
Most of the model is built from internal data Home Credit already owns. The purchased score is a valuable layer on top of it — not a foundation to build on, but not something to ration either.
If you want the full build, the code and write-up are on my GitHub: https://github.com/cooper-rm/bought-or-built-credit-signal
