Negative Log Likelihood Ratio Loss for Deep Neural Network Classification
In deep neural network, the cross-entropy loss function is commonly used for classification. Minimizing cross-entropy is equivalent to maximizing likelihood under assumptions of uniform feature and class distributions. It belongs to generative training criteria which does not directly discriminate correct class from competing classes. We propose a discriminative loss function with negative log likelihood ratio between correct and competing classes. It significantly outperforms the cross-entropy loss on the CIFAR-10 image classification task.
KeywordsLoss function Cross entropy Likelihood ratio Deep neural network
- 1.He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)Google Scholar
- 7.Casella, G., Berger, R.L.: Statistical Inference, Duxbury Thomson Learning (2002)Google Scholar
- 8.Krizhevsky, A.: Learning Multiple Layers of Features from Tiny Images (2009)Google Scholar
- 10.Keras examples. https://github.com/keras-team/keras/tree/master/examples