The Simple Approach to Multi-label Image Classification Using Transfer Learning

Abstract

The article deals with the problem of image classification on a relatively small dataset. The training deep convolutional neural net from scratch requires a large amount of data. In many cases, the solution to this problem is to use the pretrained network on another big dataset (e.g. ImageNet) and fine-tune it on available data. In the article, we apply this approach to classify advertising banners images. Initially, we reset the weights of the last layer and change its size to match a number of classes in our dataset. Then we train all network, but the learning rate for the last layer is several times more than for other layers. We use Adam optimization algorithm with some modifications. Firstly, applying weight decay instead of L2 regularization (for Adam they are not same) improves the result. Secondly, the division learning rate on the maximum of gradients squares sum instead of just gradients squares sum makes the training process more stable. Experiments have shown that this approach is appropriate for classifying relatively small datasets. Used metrics and test time augmentation are discussed. Particularly we find that confusion matrix is very useful because it gives an understanding of how to modify the train set to increase model quality.