A Robust Indoor Scene Recognition Method Based on Sparse Representation

  • Guilherme Nascimento
  • Camila Laranjeira
  • Vinicius Braz
  • Anisio Lacerda
  • Erickson R. Nascimento
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10657)

Abstract

In this paper, we present a robust method for scene recognition, which leverages Convolutional Neural Networks (CNNs) features and Sparse Coding setting by creating a new representation of indoor scenes. Although CNNs highly benefited the fields of computer vision and pattern recognition, convolutional layers adjust weights on a global-approach, which might lead to losing important local details such as objects and small structures. Our proposed scene representation relies on both: global features that mostly refers to environment’s structure, and local features that are sparsely combined to capture characteristics of common objects of a given scene. This new representation is based on fragments of the scene and leverages features extracted by CNNs. The experimental evaluation shows that the resulting representation outperforms previous scene recognition methods on Scene15 and MIT67 datasets, and performs competitively on SUN397, while being highly robust to perturbations in the input image such as noise and occlusion.

Keywords

Indoor scene recognition Sparse coding Convolutional Neural Networks 

References

  1. 1.
    Dixit, M., Chen, S., Gao, D., Rasiwasia, N., Vasconcelos, N.: Scene classification with semantic fisher vectors. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2974–2983. IEEE (2015)Google Scholar
  2. 2.
    Gao, S., Tsang, I.W.H., Chia, L.T., Zhao, P.: Local features are not lonely - Laplacian sparse coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3555–3561. IEEE Computer Society (2010)Google Scholar
  3. 3.
    Herranz, L., Jiang, S., Li, X.: Scene recognition with CNNs: objects, scales and dataset bias. In: IEEE Conference on Computer Vision and Pattern Recognition, June 2016Google Scholar
  4. 4.
    Jaakkola, T., Haussler, D.: Exploiting generative models in discriminative classifiers. In: Advances in Neural Information Processing Systems, vol. 11, pp. 487–493. MIT Press (1998)Google Scholar
  5. 5.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2169–2178. IEEE Computer Society (2006)Google Scholar
  6. 6.
    Li, F.F., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 524–531. IEEE Computer Society, Washington, DC (2005)Google Scholar
  7. 7.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  8. 8.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online dictionary learning for sparse coding. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, pp. 689–696, ACM, New York (2009)Google Scholar
  9. 9.
    Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115(3), 211–252 (2015)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001)CrossRefMATHGoogle Scholar
  11. 11.
    Oliveira, G., Nascimento, E., Vieira, A., Campos, M.: Sparse spatial coding: a novel approach to visual recognition. IEEE Trans. Image Process. 23(6), 2719–2731 (2014)MathSciNetCrossRefMATHGoogle Scholar
  12. 12.
    Quattoni, A., Torralba, A.: Recognizing indoor scenes. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 413–420 (2009)Google Scholar
  13. 13.
    Sánchez, J., Perronnin, F., Mensink, T., Verbeek, J.: Image classification with the fisher vector: theory and practice. Int. J. Comput. Vis. 105(3), 222–245 (2013)MathSciNetCrossRefMATHGoogle Scholar
  14. 14.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR abs/1409.1556 (2014)Google Scholar
  15. 15.
    Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B 58, 267–288 (1994)MathSciNetMATHGoogle Scholar
  16. 16.
    Tropp, J.A., Gilbert, A.C.: Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inf. Theory 53(12), 4655–4666 (2007)MathSciNetCrossRefMATHGoogle Scholar
  17. 17.
    Wang, Z., Wang, L., Wang, Y., Zhang, B., Qiao, Y.: Weakly supervised PatchNets: describing and aggregating local patches for scene recognition. IEEE Trans. Image Process. 26, 2028–2041 (2017)MathSciNetCrossRefGoogle Scholar
  18. 18.
    Xiao, J., Hays, J., Ehinger, K.A., Oliva, A., Torralba, A.: SUN database: large-scale scene recognition from abbey to zoo. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3485–3492. IEEE Computer Society (2010)Google Scholar
  19. 19.
    Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (2009)Google Scholar
  20. 20.
    Gong, Y., Wang, L., Guo, R., Lazebnik, S.: Multi-scale orderless pooling of deep convolutional activation features. CoRR abs/1403.1840 (2014)Google Scholar
  21. 21.
    Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features for scene recognition using places database. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 487–495. Curran Associates, Inc. (2014)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Universidade Federal de Minas Gerais (UFMG)Belo HorizonteBrazil

Personalised recommendations