Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Machine Learning, Optimization, and Big Data

MOD 2017: Machine Learning, Optimization, and Big Data pp 234–243Cite as

An Elementary Approach to the Problem of Column Selection in a Rectangular Matrix

  • 2912 Accesses

Part of the Lecture Notes in Computer Science book series (LNISA,volume 10710)

Abstract

The problem of extracting a well conditioned submatrix from any rectangular matrix (with e.g. normalized columns) has been a subject of extensive research with applications to machine learning (rank revealing factorization, sparse solutions to least squares regression problems, clustering, \(\cdots \)), optimisation (low stretch spanning trees, \(\cdots \)), and is also connected with problems in functional and harmonic analysis (Bourgain-Tzafriri restricted invertibility problem).

In this paper, we provide a deterministic algorithm which extracts a submatrix \(X_S\) from any matrix X with guaranteed individual lower and upper bounds on each singular value of \(X_S\). We are also able to deduce a slightly weaker (up to a \(\log \)) version of the Bourgain-Tzafriri theorem as an immediate side result.

We end the paper with a description of how our method applies to the analysis of a large data set and how its numerical efficiency compares with the method of Spieman and Srivastava.

Keywords

  • Bourgain Tzafriri theorem
  • Restricted invertibility
  • Column selection problems

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://www.cad.zju.edu.cn/home/dengcai/Data/Yale/Yale_64x64.mat.

References

  1. Arthur, D., Vassilvitskii, S.: k-means++: the advantages of careful seeding. In: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1027–1035. Society for Industrial and Applied Mathematics (2007)

    Google Scholar 

  2. Avron, H., Boutsidis, C.: Faster subset selection for matrices and applications. SIAM J. Matrix Anal. Appl. 34(4), 1464–1499 (2013)

    CrossRef  MathSciNet  Google Scholar 

  3. Bourgain, J., Tzafriri, L.: Invertibility of “large” submatrices with applications to the geometry of Banach spaces and harmonic analysis. Israel J. Math. 57(2), 137–224 (1987)

    CrossRef  MathSciNet  Google Scholar 

  4. Boutsidis, C., Drineas, P., Magdon-Ismail, M.: Near-optimal column-based matrix reconstruction. SIAM J. Comput. 43(2), 687–717 (2014)

    CrossRef  MathSciNet  Google Scholar 

  5. Boutsidis, C., Drineas, P., Mahoney, M.: On selecting exactly k columns from a matrix (2008, in press)

    Google Scholar 

  6. d’Aspremont, A., Ghaoui, L.E., Jordan, M.I., Lanckriet, G.R.: A direct formulation for sparse PCA using semidefinite programming. In: Advances in Neural Information Processing Systems, pp. 41–48 (2005)

    Google Scholar 

  7. Farahat, A.K., Elgohary, A., Ghodsi, A., Kamel, M.S.: Greedy column subset selection for large-scale data sets. Knowl. Inform. Syst. 45(1), 1–34 (2015)

    CrossRef  Google Scholar 

  8. Mallat, S.: Group invariant scattering. Commun. Pure Appl. Math. 65(10), 1331–1398 (2012)

    CrossRef  MathSciNet  Google Scholar 

  9. Bruna, J., Mallat, S.: Invariant scattering convolution networks. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1872–1886 (2013)

    CrossRef  Google Scholar 

  10. Naor, A.: Sparse quadratic forms and their geometric applications [following Batson, Spielman and Srivastava]. Séminaire Bourbaki: Vol. 2010/2011. Exposés 1027–1042. Astérisque No. 348 (2012), Exp. No. 1033, viii, 189–217

    Google Scholar 

  11. Nikolov, A.: Randomized rounding for the largest simplex problem. In: Proceedings of the Forty-Seventh Annual ACM on Symposium on Theory of Computing, pp. 861–870 (2015)

    Google Scholar 

  12. Nikolov, A., Singh, M.: Maximizing determinants under partition constraints. In: STOC 2016, pp. 192–201 (2016)

    Google Scholar 

  13. Spielman, D.A., Srivastava, N.: An elementary proof of the restricted invertibility theorem. Israel J. Math. 190, 83–91 (2012)

    CrossRef  MathSciNet  Google Scholar 

  14. Tropp, J.A.: The random paving property for uniformly bounded matrices. Studia Math. 185(1), 67–82 (2008)

    CrossRef  MathSciNet  Google Scholar 

  15. Tropp, J.A.: Norms of random submatrices and sparse approximation. C. R. Acad. Sci. Paris, Ser. I 346, 1271–1274 (2008)

    CrossRef  MathSciNet  Google Scholar 

  16. Tropp, J.A.: Column subset selection, matrix factorization, and eigenvalue optimization. In: Proceedings of the Twentieth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 978–986. Society for Industrial and Applied Mathematics (2009)

    Google Scholar 

  17. Vershynin, R.: John’s decompositions: selecting a large part. Israel J. Math. 122, 253–277 (2001)

    CrossRef  MathSciNet  Google Scholar 

  18. Youssef, P.: A note on column subset selection. Int. Math. Res. Not. IMRN 23, 6431–6447 (2014)

    CrossRef  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Physical Laboratory, Hampton Road, Teddington, TW11 0LW, UK

    Stéphane Chrétien

  2. Aix Marseille Université, CNRS, Centrale Marseille, I2M UMR 7373, 13453, Marseille, France

    Sébastien Darses

Authors
  1. Stéphane Chrétien
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sébastien Darses
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stéphane Chrétien .

Editor information

Editors and Affiliations

  1. University of Catania, Catania, Italy

    Giuseppe Nicosia

  2. University of Florida, Gainesville, FL, USA

    Dr. Panos Pardalos

  3. University of Catania, Catania, Italy

    Prof. Giovanni Giuffrida

  4. Harvard University, Cambridge, MA, USA

    Renato Umeton

Rights and permissions

Reprints and Permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chrétien, S., Darses, S. (2018). An Elementary Approach to the Problem of Column Selection in a Rectangular Matrix. In: Nicosia, G., Pardalos, P., Giuffrida, G., Umeton, R. (eds) Machine Learning, Optimization, and Big Data. MOD 2017. Lecture Notes in Computer Science(), vol 10710. Springer, Cham. https://doi.org/10.1007/978-3-319-72926-8_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-72926-8_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-72925-1

  • Online ISBN: 978-3-319-72926-8

  • eBook Packages: Computer ScienceComputer Science (R0)

search

Navigation