FPN Paper Walkthrough: Leveraging the Internal Pyramid

In my previous article I was talking about YOLOv3 [1]. One of the factors that makes this YOLO version better than its predecessors is its ability in detecting small objects thanks to its FPN-like neck adopted by the model. Unfortunately, my explanation about FPN in that article was not quite thorough since I was focusing more on the YOLOv3 itself. Thus, in this article I decided to write specifically about FPN from its original paper titled “Feature Pyramid Networks for Object Detection” [2] so that you can get a better understanding of what it actually is and how it works. Not only that, here I will also demonstrate how to implement FPN from scratch and how to connect it with a CNN backbone and an RPN head.

Backbone, Neck, and Head

Before we get into FPN, we first need to know that the structure of an object detection model is different from that of the classification model, in which the main difference lies in the very last layer. In a typical classification model, the last layer comprises a number of neurons where each of them corresponds to every single class available in the dataset. Or, in the…