Abstract
In recent years, deep convolutional neural network (CNN) has demonstrated its great power in image classification. In real world, there are many images contain abundant contents so that they have multiple labels. Moreover, there are correlations between labels. Traditional deep methods for such data rarely take into account such correlations. In this paper, we propose a dual-CNN model, i.e., Dual-CNN model for Multi-Label classification (Dual-CNN-ML), which can make full use of the dependencies of labels to enhance classification performance. Specifically, we first obtain co-occurrence dependency matrix from training datasets; then, we merge the co-occurrence dependency matrix and image representation together; finally, we use the new representation to predict labels of samples. Extensive experiments on public benchmark datasets demonstrate that the proposed method obtains satisfying results and outperforms several state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chen, Q., Song, Z., Dong, J., Huang, Z., Hua, Y., Yan, S.: Contextualizing object detection and classification. IEEE Trans. Pattern Anal. Mach. Intell. 37(1), 13–27 (2015)
Dong, J., Xia, W., Chen, Q.. Feng, J., Huang, Z., Yan, S.: Subcategory-aware object classification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 827–834 (2013)
Everingham, M., Van Gool, L., Williams, C.K., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vis. 111(1), 117–176 (2010)
Gong, Y., Jia, Y., Leung, T., Toshev, A., Ioffe, S.: Deep convolutional ranking for multilabel image annotation. arXiv (2013)
Guillaumin, M., Mensink, T., Verbeek, J., Schmid, C.: Tagprop: Discriminative metric learning in nearest neighbor models for image auto-annotation. In: Proceedings of IEEE International Conference on Computer Vision, pp. 309–316 (2009)
Guo, Y., Gu, S.: Multi-label classification using conditional dependency networks. In: Proceedings of International Joint Conference on Artificial Intelligence, pp. 1300–1305 (2011)
Harzallah, H., Jurie, F., Schmid, C.: Combining efficient object localization and image classification. In: Proceedings of IEEE International Conference on Computer Vision, pp. 237–244 (2009)
Kan, M., Shan, S., Chen, X.: Multi-view deep network for cross-view classification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 4847–4855 (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
LeCun, Y., Ranzato, M.: Deep learning tutorial. In: Proceedings of International Conference on Machine Learning (2013)
Li, X., Zhao, F., Guo, Y.: Multi-label image classification with a probabilistic label enhancement model. In: Proceedings of International Conference on Uncertainty in Artificial Intelligence, pp. 430–439 (2014)
Murthy, V.N., Singh, V., Chen, T., Manmatha, R., Comaniciu, D.: Deep decision network for multi-class image classification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2240–2248 (2016)
Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1717–1724 (2014)
Read, J., Pfahringer, B., Holmes, G., Frank, E.: Classifier chains for multi-label classification. Mach. Learn. 85, 333–359 (2011)
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., LeCun, Y.: Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv (2013)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv (2014)
Song, X., Jiang, S., Herranz, L.: Multi-scale multi-feature context modeling for scene recognition in the semantic manifold. IEEE Trans. Image Process. 26(6), 2721–2735 (2017)
Song, X., Jiang, S., Herranz, L., Kong, Y., Zheng, K.: Category co-occurrence modeling for large scale scene recognition. Pattern Recogn. 59, 98–111 (2016)
Wang, J., Yang, Y., Mao, J., Huang, Z., Huang, C., Xu, W.: CNN-RNN: A unified framework for multi-label image classification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 2285–2294 (2016)
Wei, Y., Xia, W., Huang, J., Ni, B., Dong, J., Zhao, Y., Yan, S.: CNN: Single-label to multi-label. arXiv (2014)
Wu, J., Yu, Y., Huang, C., Yu, K.: Deep multiple instance learning for image classification and auto-annotation. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3460–3469 (2015)
Xu, X.-S., Jiang, Y., Peng, L., Xue, X., Zhou, Z.-H.: Ensemble approach based on conditional random field for multi-label image and video annotation. In: Proceedings of ACM International Conference on Multimedia, pp. 1377–1380 (2011)
Xu, X.-S., Jiang, Y., Xue, X., Zhou, Z.-H.: Semi-supervised multi-instance multi-label learning approach for video annotation task. In: Proceedings of ACM International Conference on Multimedia, pp. 737–740 (2012)
Xue, X., Zhang, W., Zhang, J., Wu, B., Fan, J., Lu, Y.: Correlative multi-label multi-instance image annotation. In: Proceedings of IEEE International Conference on Computer Vision, pp. 651–658 (2011)
Zhou, F., Lin, Y.: Fine-grained image classification by exploring bipartite-graph labels. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1124–1133 (2016)
Acknowledgement
This work was partially supported by National Natural Science Foundation of China (61173068, 61573212), Program for New Century Excellent Talents in University of the Ministry of Education, Key Research and Development Program of Shandong Province (2016GGX101044).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Zhang, PF., Wu, HY., Xu, XS. (2018). A Dual-CNN Model for Multi-label Classification by Leveraging Co-occurrence Dependencies Between Labels. In: Zeng, B., Huang, Q., El Saddik, A., Li, H., Jiang, S., Fan, X. (eds) Advances in Multimedia Information Processing – PCM 2017. PCM 2017. Lecture Notes in Computer Science(), vol 10735. Springer, Cham. https://doi.org/10.1007/978-3-319-77380-3_30
Download citation
DOI: https://doi.org/10.1007/978-3-319-77380-3_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-77379-7
Online ISBN: 978-3-319-77380-3
eBook Packages: Computer ScienceComputer Science (R0)