Skip to main content
SpringerLink
Log in

A Brief Survey of Dimension Reduction

  • Conference paper
  • First Online:
Intelligence Science and Big Data Engineering (IScIDE 2018)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11266))

Abstract

Dimension reduction problem is a big concern which can reduce the scale of a database and keep the main features of these data simultaneously. This paper aims at reviewing and comparing different dimension reduction algorithms. Mainly, the performances of four basic algorithms (PCA, LDA, LLE and LE), their improved methods and deep learning methods are compared by reviewing the previous work. Their recognition accuracy and running time are carefully analyzed. We conclude that PCA and LDA are used more frequently in related fields. Combined methods usually perform better than original methods. Besides, deep learning method is also an approach developed in recent years, which outperforms existing traditional algorithms, though there are many barriers at present, such as obtaining huge labeled database, the computing and power limitation of different systems etc. Future research should focus on the processing of larger database. Finally, some new applications of dimension reduction are reviewed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Idan, M., Shaviv, G.E.: Robust control design strategy with parameter-dominated uncertainty. J. Guid. Control Dyn. 19(3), 605–611 (1996)

    Article  Google Scholar 

  2. Al-Arashi, W., Ibrahim, H., Suandi, S.: Optimizing principal component analysis performance for face recognition using genetic algorithm. Neurocomputing 128, 415–420 (2014)

    Article  Google Scholar 

  3. Tang, H., Fang, T., Shi, P.F.: Laplacian linear discriminant analysis. Pattern Recogn. 39, 136–139 (2006)

    Article  Google Scholar 

  4. Chen, J., Liu, Y.: Locally linear embedding: a survey. Artif. Intell. Rev. 36, 29–48 (2011)

    Article  Google Scholar 

  5. Deng, T., Deng, Y., Shi, Y., Zhou, X.: Research on ımproved locally linear embedding algorithm. In: Pan, L., Păun, G., Pérez-Jiménez, M.J., Song, T. (eds.) BIC-TA 2014. CCIS, vol. 472, pp. 88–92. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-662-45049-9_15

    Chapter  Google Scholar 

  6. Hou, C.P., Zhang, C.S., Wu, Y., Jiao, Y.Y.: Stable local dimensionality reduction approaches. Pattern Recogn. 42, 2054–2066 (2009)

    Article  Google Scholar 

  7. Kim, K.I., Jung, K., Kim, H.J.: Face recognition using kernel principal component analysis. IEEE Sig. Process. Lett. 9(2), 40–42 (2002)

    Article  Google Scholar 

  8. Meng, H., Ke, X.: Further research on principal component analysis method of face recognition. In: Proceedings of 2008 IEEE International Conference on Mechatronics and Automation, pp. 421–425 (2008)

    Google Scholar 

  9. Xu, Y., Goodacre, R.: Multiblock principal component analysis: an efficient tool for analyzing metabolomics data which contain two influential factors. Metabolomics 8, 37–51 (2012)

    Article  Google Scholar 

  10. Kokiopoulou, E., Saad, Y.: PCA without eigenvalue calculations: a case study on face recognition. University of Minnesota (2005)

    Google Scholar 

  11. DuPont, E.M., Moore, C.A., Roberts, R.G.: Terrain classification for mobile robots traveling at various speeds: an eigenspace manifold approach. In: 2008 IEEE International Conference on Robotics and Automation, pp. 3284–3289 (2008)

    Google Scholar 

  12. Ma, Z.L., Wen, J., Liang, X.M., et al.: Extraction and recognition of features from multi-types of surface targets for visual systems in unmanned surface vehicle. J. Xi’an Jiaotong Univ. 48(8), 60–66 (2014)

    Google Scholar 

  13. Ames, B.P.W., Hong, M.Y.: Alternating direction method of multipliers for penalized zero-variance discriminant analysis. Comput. Optim. Appl. 64, 725–754 (2016)

    Article  MathSciNet  Google Scholar 

  14. Mahmoudi, N., Duman, E.: Detecting credit card fraud by modified fisher discriminant analysis. Expert Syst. Appl. 42, 2510–2516 (2015)

    Article  Google Scholar 

  15. Dai, D., Yuen, P.C.: Face recognition by regularized discriminant analysis. IEEE Trans. Syst. Man Cybern. Part B: Cybern. 37(4), 1080–1085 (2007)

    Article  Google Scholar 

  16. Wang, H.X., Lu, X.S., Hu, Z.L., Zheng, W.M.: Fisher discriminant analysis with L1-norm. IEEE Trans. Cybern. 44(6), 828–842 (2014)

    Article  Google Scholar 

  17. Bose, S., Pal, A., SahaRay, R., Nayak, J.: Generalized quadratic discriminant analysis. Pattern Recogn. 48, 2676–2684 (2015)

    Article  Google Scholar 

  18. Bai, J.Q., Yan, G., Wang, C.: Modal identification method following locally linear embedding. J. Xi’an Jiaotong Univ. 47(1), 85–100 (2013)

    MathSciNet  Google Scholar 

  19. Zhang, S.: Enhanced supervised locally linear embedding. Pattern Recogn. Lett. 30, 1208–1218 (2009)

    Article  Google Scholar 

  20. Alipanahi, B., Ghodsi, A.: Guided locally linear embedding. Pattern Recogn. Lett. 32, 1029–1035 (2011)

    Article  Google Scholar 

  21. Deng, T., Deng, Y.N., Shi, Y., Zhou, X.Q.: Research on improved locally linear embedding algorithm. Commun. Comput. Inf. Sci. 472, 88–92 (2014)

    Google Scholar 

  22. Pang, Y.H., Teoh, A.B.J., Wong, E.K., Abas, F.S.: Supervised locally linear embedding in face recognition. In: International Symposium on Biometrics and Security Technologies, pp. 1–6 (2008)

    Google Scholar 

  23. Zhao, Z.Q., Li, J.Z., Gao, J., Wu, X.D.: A modified semi-supervised learning algorithm on Laplacian eigenmaps. Neural Process Lett. 32, 75–82 (2010)

    Article  Google Scholar 

  24. Tompkins, F., Wolfe, P.J.: Image analysis with regularized Laplacian eigenmaps. In: Proceedings of 2010 IEEE 17th International Conference on Image Processing, pp. 1913–1916 (2010)

    Google Scholar 

  25. Belkin, M., Niyogi, P.: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput. 15, 1373–1396 (2003)

    Article  Google Scholar 

  26. Zhong, G.Q., Hou, X.W., Liu, C.L.: Relative distance-based Laplacian eigenmaps. In: IEEE Explore, pp. 1–5 (2009)

    Google Scholar 

  27. Jafari, A., Almasganj, F.: Using Laplacian eigenmaps latent variable model and manifold learning to improve speech recognition accuracy. Speech Commun. 52, 725–735 (2010)

    Article  Google Scholar 

  28. Qi, Y.F., Zhang, J.: (2D)2 PCALDA: an efficient approach for face recognition. Appl. Math. Comput. 213, 1–7 (2009)

    MathSciNet  MATH  Google Scholar 

  29. Ge, Z.H., Sharma, S.R., Smith, M.J.T.: PCA/LDA approach for text-independent speaker recognition. arXiv (2016)

    Google Scholar 

  30. Zhang, W.W., Ding, W.R., Liu, C.H.: Prediction of interference effect on UAV data link in complex environment. Syst. Eng. Electron. 28(4), 760–766 (2016)

    Google Scholar 

  31. He, S.J., Liu, D.T., Yu, P.: Flight mode recognition method of the unmanned aerial vehicle based on telemetric data. Chin. J. Sci. Instrum. 37(9), 2004–2013 (2016)

    Google Scholar 

  32. An, N., Yan, B., Xiong, J.: A multi-scale insulator tracking algorithm based on compressive sensing. Transducer Microsyst. Technol. 35(2), 140–143 (2016)

    Google Scholar 

  33. Ren, J., Jiang, X.: Regularized 2-D complex-log spectral analysis and subspace reliability analysis of micro-Doppler signature for UAV detection. Pattern Recogn. 69, 225–237 (2017)

    Article  Google Scholar 

  34. Zong, Q., Ji, Y.H., Dou, L.Q., Zeng, F.L.: LPV model reduction of UAV lateral system. Control Decis. 25(6), 948–952 (2010)

    MathSciNet  Google Scholar 

  35. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  Google Scholar 

  36. Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. IEEE Trans. Neutral Comput. 18(7), 1527–1554 (2006)

    MathSciNet  MATH  Google Scholar 

  37. Sutskever, I., Hinton, G.E.: Deep narrow sigmoid belief networks are universal approximators. IEEE Trans. Neutral Comput. 20(11), 2629–2636 (2008)

    MATH  Google Scholar 

  38. Gao, Q., Ma, Y.M.: Research and application of the level of the deep belief network. Sci. Technol. Eng. 16(23), 234–238 (2016)

    Google Scholar 

  39. Shen, F., Luo, X., Chen, Y.: Text classification dimension reduction algorithm for Chinese web page based on deep learning. In: International Conference on Cyberspace Technology, pp. 451–456 (2013)

    Google Scholar 

  40. Sun, M., Tan, Q., Ding, R.W., Liu, H.: Cross-domain sentiment classification using deep learning approach. In: IEEE International Conference on Cloud Computing & Intelligence Systems, pp. 60–64 (2014)

    Google Scholar 

  41. Li, T., Zeng, X.Q., Xu, S.J.: A deep learning method for Braille recognition. In: 2014 International Conference on Computational Intelligence and Communication Networks, pp. 1092–1095 (2014)

    Google Scholar 

  42. Yi, J.K., Zhang, Y.C., Zhao, X.H., Wan, J.: A novel text clustering approach using deep-learning vocabulary network. Math. Probl. Eng. 2017(1), 1–13 (2017)

    MathSciNet  Google Scholar 

  43. Ahmad, A., Abbes, A., Naeem, R.: Content-based image retrieval with compact deep convolutional features. Neurocomputing 249, 95–105 (2017)

    Article  Google Scholar 

  44. Ackley, D.H., Hinton, G.E., Sejnowski, T.J.: A learning algorithm for Boltzmann machines. Cogn. Sci. 9(1), 147–169 (1985)

    Article  Google Scholar 

  45. Vanhulle, M.M., Orban, G.A.: Entropy driven artificial neuronal networks and sensorial representation - a proposal. J. Parallel Distrib. Comput. 6(2), 264–290 (1989)

    Article  Google Scholar 

  46. Fukushima, K.: Neocognitron - a self-organizing neural network model for a mechanism of pattern-recognition unaffected by shift in position. Biol. Cybern. 36(4), 193–202 (1980)

    Article  Google Scholar 

Download references

Acknowledgment

This work is partially supported by National Natural Science Foundation (NSFC) under Grants 61473038 and 91648117. And this work is also partially supported by Beijing Natural Science Foundation (BJNSF) under Grant 4172055.

Author information

Authors and Affiliations

  1. School of Xuteli, Beijing Institute of Technology, Beijing, China

    Li Song & Zilong Zhou

  2. School of Automation, Beijing Institute of Technology, Beijing, China

    Hongbin Ma & Mei Wu

  3. State Key Laboratory of Intelligent Control and Decision of Complex System, Beijing Institute of Technology, Beijing, China

    Hongbin Ma & Mengyin Fu

Authors
  1. Li Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongbin Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zilong Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mengyin Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongbin Ma .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Yuxin Peng

  2. Shanghai Jiao Tong University, Shanghai, China

    Kai Yu

  3. Tsinghua University, Beijing, China

    Jiwen Lu

  4. Central China Normal University, Wuhan, China

    Xingpeng Jiang

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Song, L., Ma, H., Wu, M., Zhou, Z., Fu, M. (2018). A Brief Survey of Dimension Reduction. In: Peng, Y., Yu, K., Lu, J., Jiang, X. (eds) Intelligence Science and Big Data Engineering. IScIDE 2018. Lecture Notes in Computer Science(), vol 11266. Springer, Cham. https://doi.org/10.1007/978-3-030-02698-1_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-02698-1_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-02697-4

  • Online ISBN: 978-3-030-02698-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation