Structured dictionary learning using mixed-norms and group-sparsity constraint

  • Zivar Ataee
  • Hadis MohseniEmail author
Original Article


Recently, sparse representation and dictionary learning have shown significant performance in machine vision. In particular, several supervised dictionary learning methods have been proposed for classification aim and increasing its accuracy. Among them, structured dictionary learning is an interesting approach which captures the discriminative properties of each class and common features among all classes in class-specific sub-dictionaries and a distinct shared sub-dictionary, respectively. It extracts the structural information that exists in samples of each class to increase the classification accuracy. Therefore, in this paper, a group-based structured dictionary learning method is proposed that captures structural information in each class and learns class-specific and shared sub-dictionaries based on mixed \(l_{2,1}\) norm. Also, mixed \(l_{2,1}\) norm is used for acquiring the sparse coefficients of data samples based on the learned sub-dictionaries. Then, classification is done by finding the class with (1) minimum reconstruction error or (2) maximum number of nonzero groups based on \(l_{1,0}\) norm. The proposed method is evaluated by conducting experiments on Extended YaleB, AR and CMU-PIE face databases and the USPS handwritten digits database. The experimental results demonstrate the effectiveness of the proposed method in data representation and classification.


Supervised dictionary learning Sparse representation Structured sparsity Mixed norms Classification 



We received no funding for this research.

Compliance with ethical standards

Conflict of interest

Author Zivar Ataee declares that she has no conflict of interest. Author Hadis Mohseni declares that she has no conflict of interest.


  1. 1.
    Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15(12), 3736 (2006)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Yang, M., Dai, D., Shen, L., Van Gool, L.: Latent dictionary learning for sparse representation based classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4138–4145 (2014)Google Scholar
  3. 3.
    Xu, Y., Li, Z., Yang, J., Zhang, D.: A survey of dictionary learning algorithms for face recognition. In: IEEE Access, vol. 5, pp. 8502–8514 (2017) CrossRefGoogle Scholar
  4. 4.
    Mallat, S.: A Wavelet Tour of Signal Processing. Academic Press, Cambridge (1999)zbMATHGoogle Scholar
  5. 5.
    Le Pennec, E., Mallat, S.: Sparse geometric image representations with bandelets. IEEE Trans. Image Process. 14(4), 423 (2005)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Aharon, M., Elad, M., Bruckstein, A.: K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans Signal Process 54, 4311–4322 (2006). CrossRefzbMATHGoogle Scholar
  7. 7.
    Bryt, O., Elad, M.: Compression of facial images using the K-SVD algorithm. J. Vis. Commun. Image Represent. 19(4), 270 (2008)CrossRefGoogle Scholar
  8. 8.
    Sprechmann, P., Sapiro, G.: Dictionary learning and sparse coding for unsupervised clustering. In: 2010 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP), pp. 2042–2045. IEEE (2010)Google Scholar
  9. 9.
    Mairal, J., Ponce, J., Sapiro, G., Zisserman, A., Bach, F.R.: Supervised dictionary learning. In: Advances in Neural Information Processing Systems, pp. 1033–1040 (2009)Google Scholar
  10. 10.
    Zhang, Q., Li, B.: Discriminative K-SVD for dictionary learning in face recognition. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2691–2698. IEEE (2010)Google Scholar
  11. 11.
    Jiang, Z., Lin, Z., Davis, L.S.: Label consistent K-SVD: learning a discriminative dictionary for recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2651 (2013)CrossRefGoogle Scholar
  12. 12.
    Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210 (2009)CrossRefGoogle Scholar
  13. 13.
    Yang, M., Zhang, L., Yang, J., Zhang, D.: Metaface learning for sparse representation based face recognition. In: 2010 17th IEEE International Conference on Image Processing (ICIP), pp. 1601–1604. IEEE (2010)Google Scholar
  14. 14.
    Ramirez, I., Sprechmann, P., Sapiro, G.: Classification and clustering via dictionary learning with structured incoherence and shared features. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3501–3508. IEEE (2010)Google Scholar
  15. 15.
    Yang, M., Zhang, L., Feng, X., Zhang, D.: Sparse representation based fisher discrimination dictionary learning for image classification. Int. J. Comput. Vis. 109(3), 209 (2014)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Lu, J., Wang, G., Zhou, J.: Simultaneous feature and dictionary learning for image set based face recognition. IEEE Trans. Image Process. 26(8), 4042 (2017). MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Foroughi, H., Ray, N., Zhang, H.: Object classification with joint projection and low-rank dictionary learning. IEEE Trans. Image Process. 27(2), 806 (2018). MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Wang, X., Lid, Y., You, S., Li, H., Wang, S.: Unidirectional representation based efficient dictionary learning. IEEE Trans. Circuits Syst. Video Technol. (2018). CrossRefGoogle Scholar
  19. 19.
    Wen, Z., Hou, B., Jiao, L.: Discriminative dictionary learning with two-level low rank and group sparse decomposition for image classification. IEEE Trans Cybernetics 47, 3758–3771 (2017). CrossRefGoogle Scholar
  20. 20.
    Wang, D., Kong, S.: A classification-oriented dictionary learning model: explicitly learning the particularity and commonality across categories. Pattern Recognit. 47(2), 885 (2014)CrossRefGoogle Scholar
  21. 21.
    Wang, C.P., Wei, W., Zhang, J.S., Song, H.B.: Robust face recognition via discriminative and common hybrid dictionary learning. Appl. Intell. 48(1), 156 (2018)CrossRefGoogle Scholar
  22. 22.
    Xu, Y., Quan, Y., Zheng, B.: Discriminative structured dictionary learning with hierarchical group sparsity. Comput. Vis. Image Underst. 136, 59 (2015)CrossRefGoogle Scholar
  23. 23.
    Pati, Y.C., Rezaiifar, R., Krishnaprasad, P.S.: Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: 1993 Conference Record of The 27th Asilomar Conference on Signals, Systems and Computers, 1993, pp. 40–44. IEEE (1993)Google Scholar
  24. 24.
    Efron, B., Hastie, T., Johnstone, I., Tibshirani, R., et al.: Least angle regression. Ann. Stat. 32(2), 407 (2004)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM Rev. 43(1), 129 (2001)MathSciNetCrossRefGoogle Scholar
  26. 26.
    Elhamifar, E., Vidal, R.: Robust classification using structured sparse representation. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1873–1879. IEEE (2011)Google Scholar
  27. 27.
    Sun, Y., Quan, Y., Fu, J.: Sparse coding and dictionary learning with class-specific group sparsity. Neural Comput. Appl. 30(4), 1265–1275 (2018). CrossRefGoogle Scholar
  28. 28.
    Sun, Y., Liu, Q., Tang, J., Tao, D.: Learning discriminative dictionary for group sparse representation. IEEE Trans. Image Process. 23(9), 3816 (2014)MathSciNetCrossRefGoogle Scholar
  29. 29.
    Zhao, L., Zhang, Y., Yin, B., Sun, Y., Hu, Y., Piao, X., Wu, Q.: Fisher discrimination-based L2,1-norm sparse representation for face recognition. Vis. Comput. 32(9), 1165 (2016). CrossRefGoogle Scholar
  30. 30.
    Fan, C., Hu, C., Liu, B.: Linearized kernel dictionary learning with group sparse priors for action recognition. Vis. Comput. (2018). CrossRefGoogle Scholar
  31. 31.
    Jacob, L., Obozinski, G., Vert, J.P.: Group lasso with overlap and graph lasso. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 433–440. ACM (2009)Google Scholar
  32. 32.
    Kim, S., Xing, E.P.: Tree-guided group lasso for multi-response regression with structured sparsity, with an application to eQTL mapping. Ann. Appl. Stat. 6(3), 1095–1117 (2012)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Bengio, S., Pereira, F., Singer, Y., Strelow, D.: Group sparse coding. In: Proceedings of the 22nd International Conference on Neural Information Processing Systems, NIPS’09. Curran Associates Inc Google Scholar
  34. 34.
    Huang, J., Zhang, T., Metaxas, D.: Learning with structured sparsity. J. Mach. Learn. Res. 12(Nov), 3371 (2011)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Gu, G., Hou, Z., Chen, C., Zhao, Y.: A dimensionality reduction method based on structured sparse representation for face recognition. Artif. Intell. Rev. 46(4), 431 (2016)CrossRefGoogle Scholar
  36. 36.
    Fan, B., Cong, Y., Tang, Y.: Online structured sparse learning with labeled information for robust object tracking. J. Electron. Imaging 26(1), 013007 (2017)CrossRefGoogle Scholar
  37. 37.
    Zhang, Z., Xu, Y., Yang, J., Li, X., Zhang, D.: A survey of sparse representation: algorithms and applications. IEEE Access 3, 490 (2015)CrossRefGoogle Scholar
  38. 38.
    Suo, Y., Dao, M., Tran, T., Mousavi, H., Srinivas, U., Monga, V.: Group structured dirty dictionary learning for classification. In: 2014 IEEE International Conference on Image Processing, ICIP 2014, pp. 150–154 (2014)Google Scholar
  39. 39.
    Wright, S.J., Nowak, R.D., Figueiredo, M.A.: Sparse reconstruction by separable approximation. IEEE Trans. Signal Process. 57(7), 2479 (2009)MathSciNetCrossRefGoogle Scholar
  40. 40.
    Chi, Y.T., Ali, M., Rajwade, A., Ho, J.: Block and Group Regularized Sparse Modeling for Dictionary Learning. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2013)Google Scholar
  41. 41.
    Lee, Kuang-Chih, Ho, J., Kriegman, D.J.: Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 684 (2005)CrossRefGoogle Scholar
  42. 42.
    Martinez, A.M.: The ar face database. Tech. rep. CVC technical report (1998)Google Scholar
  43. 43.
    Sim, T., Baker, S., Bsat, M.: The CMU pose, illumination, and expression (PIE) database. 5, vol. 4, pp. 53–58. IEEE (2002)Google Scholar
  44. 44.
    Hull, J.J.: A database for handwritten text recognition research. IEEE Trans. Pattern Anal. Mach. Intell. 16(5), 550 (1994)CrossRefGoogle Scholar
  45. 45.
    Liu, H.D., Yang, M., Gao, Y., Yin, Y., Chen, L.: Bilinear discriminative dictionary learning for face recognition. Pattern Recognit. 47(5), 1835 (2014)CrossRefGoogle Scholar
  46. 46.
    Akhtar, N., Mian, A., Porikli, F.: Joint discriminative bayesian dictionary and classifier learning, In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3919–3928. (2017)

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Computer Engineering DepartmentShahid Bahonar University of KermanKermanIran

Personalised recommendations