Efficient Sparsity Estimation via Marginal-Lasso Coding

  • Tzu-Yi Hung
  • Jiwen Lu
  • Yap-Peng Tan
  • Shenghua Gao
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8692)


This paper presents a generic optimization framework for efficient feature quantization using sparse coding which can be applied to many computer vision tasks. While there are many works working on sparse coding and dictionary learning, none of them has exploited the advantages of the marginal regression and the lasso simultaneously to provide more efficient and effective solutions. In our work, we provide such an approach with a theoretical support. Therefore, the computational complexity of the proposed method can be two orders faster than that of the lasso with sacrificing the inevitable quantization error. On the other hand, the proposed method is more robust than the conventional marginal regression based methods. We also provide an adaptive regularization parameter selection scheme and a dictionary learning method incorporated with the proposed sparsity estimation algorithm. Experimental results and detailed model analysis are presented to demonstrate the efficacy of our proposed methods.


Sparsity estimation marginal regression sparse coding lasso dictionary learning adaptive regularization parameter 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Balasubramanian, K., Yu, K., Lebanon, G.: Smooth sparse coding via marginal regression for learning sparse representations. In: ICML (2013)Google Scholar
  2. 2.
    Chang, C.C., Lin, C.J.: Libsvm: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27:1–27:27 (2011)Google Scholar
  3. 3.
    Donoho, D.L.: For most large underdetermined systems of linear equations the minimal l1-norm solution is also the sparsest solution. Comm. Pure Appl. Math. 59, 797–829 (2004)CrossRefMathSciNetGoogle Scholar
  4. 4.
    Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Annals of Statistics (2004)Google Scholar
  5. 5.
    Elhamifar, E., Vidal, R.: Robust classification using structured sparse representation. In: CVPR (2011)Google Scholar
  6. 6.
    Fan, J., Lv, J.: Sure independence screening for ultrahigh dimensional feature space. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 70(5), 849–911 (2008)CrossRefMathSciNetGoogle Scholar
  7. 7.
    Friedman, J., Hastie, T., Hofling, H., Tibshirani, R.: Pathwise coordinate optimization (2007)Google Scholar
  8. 8.
    Gao, S., Tsang, I., Chia, L.: Laplacian sparse coding, hypergraph laplacian sparse coding, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence(2012)Google Scholar
  9. 9.
    Gao, S., Tsang, I.W.H., Chia, L.T., Zhao, P.: Local features are not lonely - laplacian sparse coding for image classification. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (2010)Google Scholar
  10. 10.
    van Gemert, J.C., Geusebroek, J.-M., Veenman, C.J., Smeulders, A.W.M.: Kernel codebooks for scene categorization. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 696–709. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  11. 11.
    Genovese, C.R., Jin, J., Wasserman, L., Yao, Z.: A comparison of the lasso and marginal regression. J. Mach. Learn. Res. 13(1), 2107–2143 (2012)zbMATHMathSciNetGoogle Scholar
  12. 12.
    Gkalelis, N., Tefas, A., Pitas, I.: Human identification from human movements. In: ICIP, pp. 2585–2588 (2009)Google Scholar
  13. 13.
    Gorelick, L., Blank, M., Shechtman, E., Irani, M., Basri, R.: Action as space-time shapes. PAMI 29(12), 224–2253 (2007)CrossRefGoogle Scholar
  14. 14.
    Hull, J.: A database for handwritten text recognition research. PAMI 16(5), 550–554 (1994)CrossRefGoogle Scholar
  15. 15.
    Hung, T.-Y., Lu, J., Tan, Y.-P.: Graph-based sparse coding and embedding for activity-based human identification. In: ICME (2013)Google Scholar
  16. 16.
    Iosifidis, A., Tefas, A., Pitas, I.: Activity based person identification using fuzzy representation and discriminant learning. TIFS 7(2), 530–542 (2012)Google Scholar
  17. 17.
    Laptev, I.: On space-time interest points. International Journal of Computer Vision 64(2-3), 107–123 (2005)CrossRefGoogle Scholar
  18. 18.
    Laptev, I., Marszalek, M., Schmid, C., Rozenfeld, B.: Learning realistic human actions from movies. In: CVPR (2008)Google Scholar
  19. 19.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: CVPR, pp. 2169–2178 (2006)Google Scholar
  20. 20.
    Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Advances in Neural Information Processing Systems, vol. 19, pp. 801–808. MIT Press, Cambridge (2007)Google Scholar
  21. 21.
    Li, F.F., Perona, P.: A bayesian hierarchical model for learning natural scene categories. In: CVPR, pp. 524–531 (2005)Google Scholar
  22. 22.
    Lu, J., Hu, J., Zhou, X., Shang, Y.: Activity-based human identification using sparse coding and discriminative metric learning. In: ACM Multimedia (2012)Google Scholar
  23. 23.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online learning for matrix factorization and sparse coding. J. Mach. Learn. Res. 11, 19–60 (2010)zbMATHMathSciNetGoogle Scholar
  24. 24.
    Oliva, A., Torralba, A.: Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV 42(3), 145–175 (2001)CrossRefzbMATHGoogle Scholar
  25. 25.
    Sarkar, S., Phillips, P.J., Liu, Z., Vega, I.R., Grother, P., Bowyer, K.W.: The humanID gait challenge problem: data sets, performance, and analysis. PAMI 27(2), 162–177 (2005)CrossRefGoogle Scholar
  26. 26.
    Schuldt, C., Laptev, I., Caputo, B.: Recognizing human actions: A local svm approach. In: ICPR, pp. 32–36 (2004)Google Scholar
  27. 27.
    Tibshirani, R.: Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological) 58(1), 267–288 (1996)zbMATHMathSciNetGoogle Scholar
  28. 28.
    Wright, J., Yang, A., Ganesh, A., Sastry, S., Ma, Y.: Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(2), 210–227 (2009)CrossRefGoogle Scholar
  29. 29.
    Wu, T., Lange, K.: Coordinate descent algorithms for lasso penalized regression. Annals of Applied Statistics (2008)Google Scholar
  30. 30.
    Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1794–1801 (June 2009)Google Scholar
  31. 31.
    Zheng, M., Bu, J., Chen, C., Wang, C., Zhang, L., Qiu, G., Cai, D.: Graph regularized sparse coding for image representation. IEEE Transactions on Image Processing 20(5), 1327–1336 (2011)CrossRefMathSciNetGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Tzu-Yi Hung
    • 1
  • Jiwen Lu
    • 2
  • Yap-Peng Tan
    • 1
  • Shenghua Gao
    • 3
  1. 1.School of Electrical and Electronic EngineeringNanyang Technological UniversitySingapore
  2. 2.Advanced Digital Sciences CenterSingapore
  3. 3.ShanghaiTech UniversityShanghaiChina

Personalised recommendations