Advertisement

Dictionary Reduction: Automatic Compact Dictionary Learning for Classification

  • Yang SongEmail author
  • Zhifei Zhang
  • Liu Liu
  • Alireza Rahimpour
  • Hairong Qi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10111)

Abstract

A complete and discriminative dictionary can achieve superior performance. However, it also consumes extra processing time and memory, especially for large datasets. Most existing compact dictionary learning methods need to set the dictionary size manually, therefore an appropriate dictionary size is usually obtained in an exhaustive search manner. How to automatically learn a compact dictionary with high fidelity is still an open challenge. We propose an automatic compact dictionary learning (ACDL) method which can guarantee a more compact and discriminative dictionary while at the same time maintaining the state-of-the-art classification performance. We incorporate two innovative components in the formulation of the dictionary learning algorithm. First, an indicator function is introduced that automatically removes highly correlated dictionary atoms with weak discrimination capacity. Second, two additional constraints, namely, the sum-to-one and the non-negative constraints are imposed on the sparse coefficients. On one hand, this achieves the same functionality as the \(L_2\)-normalization on the raw data to maintain a stable sparsity threshold. On the other hand, this effectively preserves the geometric structure of the raw data which would be otherwise destroyed by the \(L_2\)-normalization. Extensive evaluations have shown that the preservation of geometric structure of the raw data plays an important role in achieving high classification performance with smallest dictionary size. Experimental results conducted on four recognition problems demonstrate the proposed ACDL can achieve competitive classification performance using a drastically reduced dictionary (https://github.com/susanqq/ACDL.git).

Keywords

Dictionary Learning Sparse Coefficient Dictionary Size Overcomplete Dictionary Dictionary Atom 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

References

  1. 1.
    Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: a strategy employed by V1? Vis. Res. 37, 3311–3325 (1997)CrossRefGoogle Scholar
  2. 2.
    Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15, 3736–3745 (2006)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Aharon, M., Elad, M., Bruckstein, A.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Sig. Process. 54, 4311 (2006)CrossRefGoogle Scholar
  4. 4.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online dictionary learning for sparse coding. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 689–696. ACM (2009)Google Scholar
  5. 5.
    Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8. IEEE (2008)Google Scholar
  6. 6.
    Ramirez, I., Sprechmann, P., Sapiro, G.: Classification and clustering via dictionary learning with structured incoherence and shared features. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3501–3508. IEEE (2010)Google Scholar
  7. 7.
    Zhang, Q., Li, B.: Discriminative K-SVD for dictionary learning in face recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2691–2698. IEEE (2010)Google Scholar
  8. 8.
    Jiang, Z., Lin, Z., Davis, L.S.: Label consistent K-SVD: learning a discriminative dictionary for recognition. IEEE Trans. Pattern Anal. Mach. Intell. 35, 2651–2664 (2013)CrossRefGoogle Scholar
  9. 9.
    Yang, M., Zhang, L., Feng, X., Zhang, D.: Fisher discrimination dictionary learning for sparse representation. In: IEEE International Conference on Computer Vision (ICCV), pp. 543–550. IEEE (2011)Google Scholar
  10. 10.
    Mairal, J., Ponce, J., Sapiro, G., Zisserman, A., Bach, F.R.: Supervised dictionary learning. In: Advances in Neural Information Processing Systems, pp. 1033–1040 (2009)Google Scholar
  11. 11.
    Mairal, J., Bach, F., Ponce, J.: Task-driven dictionary learning. IEEE Trans. Pattern Anal. Mach. Intell. 34, 791–804 (2012)CrossRefGoogle Scholar
  12. 12.
    Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31, 210–227 (2009)CrossRefGoogle Scholar
  13. 13.
    Rahimpour, A., Taalimi, A., Luo, J., Qi, H.: Distributed object recognition in smart camera networks. In: IEEE International Conference on Image Processing, Phoenix, Arizona, USA. IEEE (2016)Google Scholar
  14. 14.
    Lazebnik, S., Raginsky, M.: Supervised learning of quantizer codebooks by information loss minimization. IEEE Trans. Pattern Anal. Mach. Intell. 31, 1294–1309 (2009)CrossRefGoogle Scholar
  15. 15.
    Liu, J., Shah, M.: Learning human actions via information maximization. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–8. IEEE (2008)Google Scholar
  16. 16.
    Kong, S., Wang, D.: A dictionary learning approach for classification: separating the particularity and the commonality. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7572, pp. 186–199. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-33718-5_14 CrossRefGoogle Scholar
  17. 17.
    Jiang, Z., Zhang, G., Davis, L.S.: Submodular dictionary learning for sparse coding. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3418–3425. IEEE (2012)Google Scholar
  18. 18.
    Yang, M., Dai, D., Shen, L., Van Gool, L.: Latent dictionary learning for sparse representation based classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4138–4145. IEEE (2014)Google Scholar
  19. 19.
    Qiu, Q., Patel, V.M., Chellappa, R.: Information-theoretic dictionary learning for image classification. IEEE Trans. Pattern Anal. Mach. Intell. 36, 2173–2184 (2014)CrossRefGoogle Scholar
  20. 20.
    Lu, C., Shi, J., Jia, J.: Scale adaptive dictionary learning. IEEE Trans. Image Process. 23, 837–847 (2014)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Parikh, N., Boyd, S.: Proximal algorithms. Found. Trends Optim. 1, 123–231 (2013)Google Scholar
  22. 22.
    Rakotomamonjy, A.: Direct optimization of the dictionary learning problem. IEEE Trans. Sig. Process. 61, 5495–5506 (2013)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Naikal, N., Yang, A.Y., Sastry, S.S.: Towards an efficient distributed object recognition system in wireless smart camera networks. In: 13th Conference on Information Fusion (FUSION), pp. 1–8. IEEE (2010)Google Scholar
  24. 24.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, pp. 2169–2178. IEEE (2006)Google Scholar
  25. 25.
    Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. In: CVPR Workshop on Generative-Mode Based Vision. IEEE (2004)Google Scholar
  26. 26.
    Naikal, N., Yang, A.Y., Sastry, S.S.: Informative feature selection for object recognition via sparse PCA. In: IEEE International Conference on Computer Vision (ICCV), pp. 818–825. IEEE (2011)Google Scholar
  27. 27.
    Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., Darrell, T.: DECAF: a deep convolutional activation feature for generic visual recognition. In: Proceedings of the 31st International Conference on Machine Learning (ICML-2014), pp. 647–655 (2014)Google Scholar
  28. 28.
    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Heidelberg (2014). doi: 10.1007/978-3-319-10590-1_53 Google Scholar
  29. 29.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86, 2278–2324 (1998)CrossRefGoogle Scholar
  30. 30.
    Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features for scene recognition using places database. In: Advances in Neural Information Processing Systems, pp. 487–495 (2014)Google Scholar
  31. 31.
    Gao, S., Tsang, I.W.H., Chia, L.T.: Laplacian sparse coding, hypergraph laplacian sparse coding, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 35, 92–104 (2013)CrossRefGoogle Scholar
  32. 32.
    Jarrett, K., Kavukcuoglu, K., Ranzato, M., LeCun, Y.: What is the best multi-stage architecture for object recognition? In: IEEE International Conference on Computer Vision (ICCV), pp. 2146–2153. IEEE (2009)Google Scholar
  33. 33.
    Ciregan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (2012)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Yang Song
    • 1
    Email author
  • Zhifei Zhang
    • 1
  • Liu Liu
    • 1
  • Alireza Rahimpour
    • 1
  • Hairong Qi
    • 1
  1. 1.Department of Electrical Engineering and Computer ScienceUniversity of TennesseeKnoxvilleUSA

Personalised recommendations