Improving CNN
Sharing a unpublished blog from the series, Dedicated to the the draft posts and quiet projects cut short when life takes over.
In this article, we will study some deep learning techniques and will try to improve the accuracy of the previous model using these techniques. Github link of this article series is here. Link to the jupyter notebook of this tutorial is here.
Index
- Introduction to machine learning and deep learning.
- Introduction to neural networks.
- Introduction to Python.
- Building our first neural network in Keras.
- A comprehensive guide to CNN.
- Image classification with CNN
- Improving CNN < — You are here
We will implement all the changes one by one and will visualize improvements in the accuracy of the model after each change. I will share the link to code with every change as there will be multiple notebooks. First, we will study some theories and will try to implement these changes. Let’s begin
Dropout
We have already discussed what overfitting means. If you are not aware of it this image will make it clear.
If our model is overfitting, it fits the training data well and gives good results on training data but it is unable to give good results at the time of validation or testing. If you have seen the notebook of the last article, the accuracy of our model at training time was 98% after 20 epochs but the validation accuracy was only 90% which is good but the model is overfitting.
Get Sanchit Tanwar’s stories in your inbox
Join Medium for free to get updates from this writer.
So using some techniques we can reduce the overfitting of our model, such techniques are called regularization. There are several regularization techniques such as L1/L2 regularization, early stopping, dropout, etc but we will discuss dropout here. If you want to study more techniques in detail refer here.
Dropout is a very simple yet effective regularization technique. Dropout randomly turns off some nodes from a layer of neural network. The number of nodes turned off depends on the dropout rate. At every iteration, the nodes to be turned off are chosen randomly.
Here the second figure is with dropout and we can see half of the nodes are closed. Here the dropout rate is 0.5 in this case.
Dropout in Keras -
We can add dropout between layers using a single line
model.add(Dropout(0.5))Here 0.5 is the dropout rate. I will change the model and add dropout layers in it and we will plot the training results.
