Skip to main content
SpringerLink
Log in

Outlier-Resistant Data Processing with L1-Norm Principal Component Analysis

  • Chapter
  • First Online:
Advances in Principal Component Analysis

Abstract

Principal Component Analysis (PCA) has been a cornerstone of data analysis for more than a century, with important applications across most fields of science and engineering. However, despite its many strengths, PCA is known to have a major drawback: it is very sensitive to the presence of outliers among the processed data. To counteract the impact of outliers in data analysis, researchers have been long working on robust modifications of PCA. One of the most successful (and promising) PCA alternatives is L1-PCA. L1-PCA relies on the L1-norm of the processed data and, thus, tames any outliers that may exist in the dataset. Experimental studies in various applications have shown that L1-PCA (i) attains similar performance to PCA when the processed data are outlier-free and (ii) maintains sturdy resistance against outliers when the processed data are corrupted. Thus, L1-PCA is expected to play a significant role in the big-data era, when large datasets are often outlier corrupted. In this chapter, we present the theoretical foundations of L1-PCA, optimal and state-of-the-art approximate algorithms for its implementation, and some numerical studies that demonstrate its favorable performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  1. Barnett, V., Lewis, T.: Outliers in Statistical Data. Wiley, New York, NY (1994)

    MATH  Google Scholar 

  2. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York, NY (2006)

    MATH  Google Scholar 

  3. Brooks, J.P., Dulá, J.H.: The L1-norm best-fit hyperplane problem. Appl. Math. Lett. 26, 51–55 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  4. Brooks, J.P., Dulá, J.H., Boone, E.L.: A pure L1-norm principal component analysis. J. Comput. Stat. Data Anal. 61, 83–98 (2013)

    Article  MATH  Google Scholar 

  5. Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust principal component analysis? J. ACM 58(3), 37 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  6. Chamadia, S., Pados, D.A.: Optimal sparse l1-norm principal-component analysis. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp. 2686–2690 (2017a)

    Google Scholar 

  7. Chamadia, S., Pados, D.A.: Outlier processing via l1-principal subspaces. In: Proceedings of Florida Artificial Intelligence Research Society (FLAIRS), Marco Island, FL, pp 508–513 (2017b)

    Google Scholar 

  8. Ding, C., Zhou, D., He, X., Zha, H.: \(r_1\)-PCA: Rotational invariant L1-norm principal component analysis for robust subspace factorization. In: Proceedings of International Conference on Machine Learning (ICML 2006), Pittsburgh, PA, pp. 281–288 (2006)

    Google Scholar 

  9. Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley, New York, NY (2001)

    MATH  Google Scholar 

  10. Eriksson, A., van den Hengel, A.: In: Efficient computation of robust low-rank matrix approximations in the presence of missing data using the L1 norm. In: Proceedings of IEEE Conference on Computer Vision Pattern Recognition (CVPR), San Francisco, CA, USA, pp. 771–778 (2010)

    Google Scholar 

  11. Golub, G.H.: Some modified matrix eigenvalue problems. SIAM Rev. 15, 318–334 (1973)

    Article  MathSciNet  MATH  Google Scholar 

  12. Johnson, M., Savakis, A.: Fast L1-eigenfaces for robust face recognition. In: Proceedings of IEEE Western New York Image Signal Processing Workshop (WNYISPW), Rochester, NY, pp. 1–5 (2014)

    Google Scholar 

  13. Jolliffe, I.T.: Principal Component Analysis. Springer, New York, NY (1986)

    Book  MATH  Google Scholar 

  14. Ke, Q., Kanade, T.: Robust subspace computation using L1 norm. Techincal report Internal Technical Report, Computer Science Department, Carnegie Mellon University, Pittsburgh, PA, USA, CMU-CS-03-172 (2003)

    Google Scholar 

  15. Ke, Q., Kanade, T.: Robust L1 norm factorization in the presence of outliers and missing data by alternative convex programming. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), San Diego, CA, pp. 739–746 (2005)

    Google Scholar 

  16. Kundu, S., Markopoulos, PP., Pados, D.A.: Fast computation of the L1-principal component of real-valued data. In: Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy, pp. 8028–8032 (2014)

    Google Scholar 

  17. Kwak, N.: Principal component analysis based on L1-norm maximization. IEEE Trans. Patt. Anal. Mach. Intell. 30, 1672–1680 (2008)

    Article  Google Scholar 

  18. Kwak, N., Oh, J.: Feature extraction for one-class classification problems: enhancements to biased discriminant analysis. Patt. Recog. 42, 17–26 (2009)

    Article  MATH  Google Scholar 

  19. Liu, Y., Pados, D.A.: Compressed-sensed-domain L1-PCA video surveillance. IEEE Trans. Multimedia 18(3), 351–363 (2016)

    Article  Google Scholar 

  20. Markopoulos, P.: Optimal algorithms for L1-norm principal component analysis: new tools for signal processing and machine learning with few and/or faulty training data. Ph.D. thesis, State University of New York at Buffalo (2015)

    Google Scholar 

  21. Markopoulos, P.P.: Reduced-rank filtering on L1-norm subspaces. In: Proceedings of IEEE Sensor Array Multichannel Signal Processing Workshop (SAM), Rio de Janeiro, Brazil, pp.1–5 (2016)

    Google Scholar 

  22. Markopoulos, P.P., Ahmad, F.: Indoor human motion classification by L1-norm subspaces of micro-doppler signatures. In: Proceedings of IEEE Radar Conference (Radarcon), Seattle, WA, pp. 1807–1810 (2017)

    Google Scholar 

  23. Markopoulos, P.P., Karystinos, G.N., Pados, D.A.: Some options for L1-subspace signal processing. In: Proceedings of 10th International Sympoisum on Wireless Communication System (ISWCS), Ilmenau, Germany, pp. 622–626 (2013)

    Google Scholar 

  24. Markopoulos, P.P., Karystinos, G.N., Pados, D.A.: Optimal algorithms for L1-subspace signal processing. IEEE Trans. Signal Process. 62, 5046–5058 (2014a)

    Article  MathSciNet  Google Scholar 

  25. Markopoulos, P.P., Tsagkarakis, N., Pados, D.A., Karystinos, G.N.: Direction finding with L1-norm subspaces. In: Proceedings of Commercial Sensing Conference on SPIE Defence Security Sensors (DSS), Baltimore MD, pp. 91-090J1–91-090J11 (2014b)

    Google Scholar 

  26. Markopoulos, P.P., Kundu, S., Pados, D.A.: L1-fusion: Robust linear-time image recovery from few severely corrupted copies. In: Proceedings of IEEE International Conference on Image Processing (ICIP), Quebec City, Canada, pp. 1225–1229 (2015)

    Google Scholar 

  27. Markopoulos, P.P., Kundu, S., Chamadia, S., Pados, D.A.: L1-norm principal-component analysis via bit flipping. In: Proceedings of IEEE International Conference on Machine Learning Applications (ICMLA), Anaheim, CA, pp. 326–332 (2016a)

    Google Scholar 

  28. Markopoulos, P.P., Tsagkarakis, N., Pados, D.A., Karystinos, G.N.: Direction-of-arrival estimation from l1-norm principal components. In: Proceedings of IEEE International Sympoisum on Phased Array Systems and Technology (PAST), Boston, MA, pp. 1–6 (2016b)

    Google Scholar 

  29. Markopoulos, P.P., Kundu, S., Chamadia, S., Pados, D.: Efficient l1-norm principal-component analysis via bit flipping. IEEE Transactions on Signal Processing (2016b)

    Google Scholar 

  30. McCoy, M., Tropp, J.A.: Two proposals for robust PCA using semidefinite programming. Electron. J. Stat. 5, 1123–1160 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  31. Meng, D., Zhao, Q., Xu, Z.: Improve robustness of sparse PCA by L1-norm maximization. Patt. Recog. 45, 487–497 (2012)

    Article  MATH  Google Scholar 

  32. Nie, F., Huang, H., Ding, C., Luo, D., Wang, H.: Robust principal component analysis with non-greedy L1-norm maximization. In: Proceedings of International Joint Conference on Artificial intelligence (IJCAI), Barcelona, Spain, pp. 1433–1438 (2011)

    Google Scholar 

  33. Pearson, K.: On lines and planes of closest fit to systems of points in space. Philos. Mag. 2, 559–572 (1901)

    Article  MATH  Google Scholar 

  34. Schönemann, P.H.: A generalized solution of the orthogonal procrustes problem. Psychometrika 31(1), 1–10 (1966)

    Article  MathSciNet  MATH  Google Scholar 

  35. Tsagkarakis, N., Markopoulos, P.P., Pados, D.A.: Direction finding by complex L1-principal component analysis. In: Proceedings of IEEE 16th International Workshop on Signal Processing Advances in Wireless Communications (SPAWC), Stockholm, Sweden, pp. 475–479 (2015)

    Google Scholar 

  36. Tsagkarakis, N., Markopoulos, P.P., Pados, D.A.: On the l1-norm approximation of a matrix by another of lower rank. In: Proceedings of IEEE International Conference on Machine Learning Applications (IEEE ICMLA 2016), IEEE, pp. 768–773 (2016)

    Google Scholar 

  37. Wang, H.: Block principal component analysis with L1-norm for image analysis. Patt. Recog. Lett. 33, 537–542 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical and Microelectronic Engineering, Rochester Institute of Technology, Rochester, NY, 14623, USA

    Panos P. Markopoulos

  2. Qualcomm Technologies, Inc., San Jose, CA, 95110, USA

    Sandipan Kundu

  3. Department of Anesthesia, Critical Care, and Pain Medicine, Harvard Medical School, Massachusetts General Hospital, Boston, MA, 02114, USA

    Shubham Chamadia

  4. College of Engineering and Computing, University of South Carolina, Columbia, SC, 29208, USA

    Nicholas Tsagkarakis

  5. Computer and Electrical Engineering & Computer Science, Florida Atlantic University, Boca Raton, FL, 33431, USA

    Dimitris A. Pados

Authors
  1. Panos P. Markopoulos
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sandipan Kundu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shubham Chamadia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nicholas Tsagkarakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dimitris A. Pados
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Panos P. Markopoulos .

Editor information

Editors and Affiliations

  1. BENS Research Group, MARCS Institute, Western Sydney University, Kingswood, Australia

    Ganesh R. Naik

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Markopoulos, P.P., Kundu, S., Chamadia, S., Tsagkarakis, N., Pados, D.A. (2018). Outlier-Resistant Data Processing with L1-Norm Principal Component Analysis. In: Naik, G. (eds) Advances in Principal Component Analysis. Springer, Singapore. https://doi.org/10.1007/978-981-10-6704-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-6704-4_6

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-6703-7

  • Online ISBN: 978-981-10-6704-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation