Deep Learning
Exploring Loan Defaults with Neural Networks
What is Deep Learning?
Deep learning, which is a broader part of machine learning methods, has had some major advancements over the years. Architectures such as deep neural networks, convolutional neural networks, and recurrent neural networks have helped provide insights and solutions in the complicated fields of natural language processing, computer vision, image analysis, and much more.
What is unique about deep learning is how these neural networks were inspired by the way that biological systems use nodes to communicate and process information. Although not as complex, deep learning is often depicted to resemble the neurons in our brains. The same way neurons fire in the brain to make connections is similar to the methodology behind deep learning and how hidden layers are created with nodes in order to find deeper connections and patterns for the task at hand.
The remainder of this blog will be exploring a loan dataset provided by LendingClub to construct a multilayer perceptron (MLP) in order to attempt at classifying whether a loan was charged off or fully paid.
For those who don’t know, LendingClub is a peer-to-peer lending company that provides a marketplace to potential investors and borrowers. Borrowers can go to their website and after going through a screening process, are serviced a loan that can be used for debt consolidation, home improvement, finance a purchase, and much more.
On the other hand, investors can go to LendingClub to purchase notes, which represent fractions of these loans from the borrowers. By investing in these loans, they have the potential to earn on the interest rates being paid by the borrowers.
Data cleaning:
I thought that this would be a cool dataset to explore how deep learning can be used for classification. The dataset provided by LendingClub is from 2007 up until the third quarter of 2017 which had over 1.6 million loans and 150 features. Here is a look at the target variable:
Since there were many current loans, these were dropped since they would not help in classifying if a loan was charged off versus fully paid. This data set also had many features that were missing more than 30% of the data and so these were also dropped.
Get Sabatino Chen’s stories in your inbox
Join Medium for free to get updates from this writer.
In order to narrow down the features further, I wanted to make a model that would be useful to potential investors. So the features I used were based off the information that is provided when a potential investor is on the website looking through different loans.
After further data scrubbing by imputing null values log transforming, assigning columns to their appropriate data types, and feature selection, I was left with the remaining features:
EDA:
In order to gain more insight about the dataset, exploratory data analysis was done. Here are some of the interesting charts that were made during the EDA process.
This last chart was very eye opening. Although very obvious how the financial crisis impacted the whole world economy, it’s still just shocking to see a chart like this that basically shows how events leading up to 2008 were such a mess that you see an inversion in average annual income for loan status. Maybe this can be some form of an indicator for a future financial crisis…
Modeling:
After exploring more of the dataset, further feature selection was done by checking for collinearity and also dropping features that were redundant. Next I set the training features and target variable and split the data.
Right away you can see that our data is fairly imbalanced. In order to solve this issue, SMOTE is used to upsample to a more balanced data for training. Then usual scaling is done before running any model using StandardScaler.
Although this is a deep learning project, it is good practice to see if your problem can be solved with much simpler machine learning models such as tree models or logistic regression. So before constructing the MLP, I ran an XGBoost model.
Some slight overfitting but overall pretty solid accuracy. Now onto building an MLP!
An MLP, which again stands for multilayer perceptron, is a type of feedforward artificial neural network that consists of at least three layers of nodes: an input layer, a hidden layer, and an output layer.
Neural networks can be very computationally expensive depending on how many layers deep you go and also the size of these layers. So during the iterating process I only trained my models for 5–10 epochs. Since this is a binary classification problem, sigmoid activation is used for the final output layer.
In the end, my final model looked like this:
Additional dropout layers and L2 regularization was used to try and limit overfitting. In deep learning there are many more hyper parameters that can be tuned such as tuning the kernel initializer or optimizer functions.
Conclusion
Unfortunately in the end, even after training for 100 epochs, the accuracy results were about 80% on the validation set. These are solid results but are very similar to the results yielded from the basic XGBoost model.
Possibly with more data being provided by LendingClub and further data cleaning and feature engineering, improved results could be achieved. But for now it seems this problem is not entirely dependent on deep learning. Never the less, being a Junior Data Scientist, it was great experience looking at how a neural network can be used for classification in the financial world.
For those interested, my notebook can be found here. Thanks for reading and I hope you enjoyed!
