Advertisement

Computing the spark: mixed-integer programming for the (vector) matroid girth problem

  • Andreas M. TillmannEmail author
Article
  • 3 Downloads

Abstract

We investigate the NP-hard problem of computing the spark of a matrix (i.e., the smallest number of linearly dependent columns), a key parameter in compressed sensing and sparse signal recovery. To that end, we identify polynomially solvable special cases, gather upper and lower bounding procedures, and propose several exact (mixed-)integer programming models and linear programming heuristics. In particular, we develop a branch and cut scheme to determine the girth of a matroid, focussing on the vector matroid case, for which the girth is precisely the spark of the representation matrix. Extensive numerical experiments demonstrate the effectiveness of our specialized algorithms compared to general-purpose black-box solvers applied to several mixed-integer programming models.

Keywords

Matroid girth Spark Sparse recovery Compressed sensing Branch-and-cut Mixed-integer programming 

Mathematics Subject Classification

05B35 68Q17 90C10 90C11 90C57 94A15 

Notes

Acknowledgements

The author would like to thank Marc Pfetsch for inspiring discussions in the early stages of this research effort and his support in getting started working with the SCIP framework, the authors of [38, 75] for kindly providing code for their respective deterministic compressed sensing matrix construction routines, as well as the anonymous reviewer whose comments helped improve the paper.

References

  1. 1.
    Alexeev, B., Cahill, J., Mixon, D.G.: Full spark frames. J. Fourier Anal. Appl. 18(6), 1167–1194 (2012)MathSciNetzbMATHGoogle Scholar
  2. 2.
    Amaldi, E., Kann, V.: The complexity and approximability of finding maximum feasible subsystems of linear relations. Theor. Comput. Sci. 147, 181–210 (1995)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Amaldi, E., Kann, V.: On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theor. Comput. Sci. 209(1–2), 237–260 (1998)MathSciNetzbMATHGoogle Scholar
  4. 4.
    Arellano, J.D.: Algorithms to find the girth and cogirth of a linear matroid. Ph.D. thesis, Rice University (2014)Google Scholar
  5. 5.
    Arellano, J.D., Hicks, I.V.: Degree of redundancy of linear systems using implicit set covering. IEEE Trans. Autom. Sci. Eng. 11(1), 274–279 (2014)Google Scholar
  6. 6.
    Arora, S., Babai, L., Stern, J., Sweedyk, Z.: The hardness of approximate optima in lattices, codes, and systems of linear equations. J. Comput. Syst. Sci. 54(2), 317–331 (1997)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Aveila, P., Boccia, M., Vasilyev, I.: Computational experience with general cutting planes for the set covering problem. Oper. Res. Lett. 37(1), 16–20 (2009)MathSciNetzbMATHGoogle Scholar
  8. 8.
    Aykanat, C., Pinar, A., Çatalyürek, U.: Permuting sparse rectangular matrices into block-diagonal form. SIAM J. Sci. Comput. 25(6), 1860–1879 (2004)MathSciNetzbMATHGoogle Scholar
  9. 9.
    Balas, E.: Cutting planes from conditional bounds: a new approach to set covering. Math. Prog. 12, 19–36 (1980)MathSciNetzbMATHGoogle Scholar
  10. 10.
    Balas, E., Ho, A.: Set covering algorithms using cutting planes, heuristics, and subgradient optimization: a computational study. Math. Prog. 12, 37–60 (1980)MathSciNetzbMATHGoogle Scholar
  11. 11.
    Balas, E., Jeroslow, R.: Canonical cuts on the unit hypercube. SIAM J. Appl. Math. 23(1), 61–69 (1972)MathSciNetzbMATHGoogle Scholar
  12. 12.
    Balas, E., Ng, S.M.: On the set covering polytope: I. All the facets with coefficients in 0, 1, 2. Math. Program. 43(1–3), 57–69 (1989)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Bausal, M., Kianfar, K., Ding, Y., Moreno-Centeno, E.: Hybridization of bound-and-decompose and mixed integer feasibility checking to measure redundancy in structured linear systems. IEEE Trans. Autom. Sci. Eng. 10(4), 1151–1157 (2013)Google Scholar
  14. 14.
    Beasley, J.E., Jornsten, K.: Enhancing an algorithm for set covering problems. Eur. J. Oper. Res. 58(2), 293–300 (1992)zbMATHGoogle Scholar
  15. 15.
    Berry, M.W., Heath, M.T., Kaneka, I., Lawo, M., Plemmons, R.J., Ward, R.C.: An algorithm to compute a sparse basis of the null space. Numer. Math. 47(4), 483–504 (1985)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Bixby, R.E., Saltzman, M.J.: Recovering an optimal LP basis from an interior point solution. Oper. Res. Lett. 15(4), 169–178 (1994)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Borndörfer, R.: Aspects of set packing, partitioning, and covering. Doctoral dissertation. TU Berlin, Germany (1998)Google Scholar
  18. 18.
    Cameron, P.J.: Combinatorics: topics, techniques. Algorithms. Cambridge University Press, Cambridge (1994)zbMATHGoogle Scholar
  19. 19.
    Candès, E., Tao, T.: Decoding by linear programming. IEEE Trans. Inform. Theory 51(12), 4203–4215 (2005)MathSciNetzbMATHGoogle Scholar
  20. 20.
    Caprara, A., Toth, P., Fischetti, M.: Algorithms for the set covering problem. Ann. Oper. Res. 98, 353–371 (2000)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Chang, S.F., McCormick, S.T.: A hierarchical algorithm for making sparse matrices sparser. Math. Program. 56(1–3), 1–30 (1992)MathSciNetzbMATHGoogle Scholar
  22. 22.
    Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20(1), 33–61 (1998)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Chistov, A., Fournier, H., Gurvits, L., Koiran, P.: Vandermonde matrices, NP-completeness, and transversal subspaces. Found. Comput. Math. 3(4), 421–427 (2003)MathSciNetzbMATHGoogle Scholar
  24. 24.
    Cho, J.J., Chen, Y., Ding, Y.: On the (co)girth of a connected matroid. Discrete Appl. Math. 155(18), 2456–2470 (2007)MathSciNetzbMATHGoogle Scholar
  25. 25.
    Cho, M., Mishra, K.V., Xu, W.: Computable performance guarantees for compressed sensing matrices. EURASIP J. Adv. Signal Process. 2018, 16 (2018)Google Scholar
  26. 26.
    Coleman, T.F., Pothen, A.: The sparse null space basis problem. Technical Report TR 84-598, Cornell University, Ithaca, NY, USA (1984)Google Scholar
  27. 27.
    Damaschke, P., Eğecioğlu, O., Molokov, L.: Fixed-Parameter Tractability of Error Correction in Graphical Linear Systems. In: Ghosh, S.K., Tokuyama, T. (eds.) Proceedings of the WALCOM. 2013, LNCS, vol. 7748, pp. 245–256. Springer (2013)Google Scholar
  28. 28.
    d’Aspremont, A., Bach, F., Ghaoui, L.E.: Optimal solutions for sparse principal component analysis. J. Mach. Learn. Res. 9, 1269–1294 (2008)MathSciNetzbMATHGoogle Scholar
  29. 29.
    d’Aspremont, A., Ghaoui, L.E.: Testing the nullspace property using semidefinite programming. Math. Program. 127(1), 123–144 (2011)MathSciNetzbMATHGoogle Scholar
  30. 30.
    d’Aspremont, A., Ghaoui, L.E., Jordan, M.I., Lanckriet, G.R.G.: A direct formulation for sparse PCA using semidefinite programming. SIAM Rev. 49(3), 434–448 (2007)MathSciNetzbMATHGoogle Scholar
  31. 31.
    DeVore, R.A.: Deterministic constructions of compressed sensing matrices. J. Complex. 23(4–6), 918–925 (2007)MathSciNetzbMATHGoogle Scholar
  32. 32.
    Dimakis, A.G., Smarandache, R., Vontobel, P.O.: LDPC codes for compressed sensing. IEEE Trans. Inf. Theory 58(5), 3093–3114 (2012)MathSciNetzbMATHGoogle Scholar
  33. 33.
    Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)MathSciNetzbMATHGoogle Scholar
  34. 34.
    Donoho, D.L., Elad, M.: Optimally sparse representation in general (non-orthogonal) dictionaries via \(\ell ^1\) minimization. Proc. Natl. Acad. Sci. USA 100(5), 2197–2202 (2003)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Egner, S., Minkwitz, T.: Sparsification of rectangular matrices. J. Symb. Comput. 26(2), 135–149 (1998)MathSciNetzbMATHGoogle Scholar
  36. 36.
    Elad, M.: Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer, Heidelberg (2010)zbMATHGoogle Scholar
  37. 37.
    Ferris, M.C., Pataki, G., Schmieta, S.: Solving the seymour problem. Optima 66, 2–6 (2001)Google Scholar
  38. 38.
    Fischer, T.: Konstruktion von dünn besetzten Sensing-Matrizen. Diploma Thesis, TU Darmstadt, Germany (2012) (in German) Google Scholar
  39. 39.
    Foucart, S., Rauhut, H.: A Mathematical Introduction to Compressive Sensing. Applied and Numeric Harmonic Analysis. Birkhäuser, Basel (2013)zbMATHGoogle Scholar
  40. 40.
    Gally, T., Pfetsch, M.E.: Computing Restricted Isometry Constants via Mixed-Integer Semidefinite Programming. Optimization Online E-print (2016). http://www.optimization-online.org/DB_HTML/2016/04/5395.html. Accessed 20 Apr 2018
  41. 41.
    Garey, M.R., Johnson, D.S.: Computers and Intractability. A Guide to the Theory of NP-Completeness. W. H. Freeman and Company, New York (1979)zbMATHGoogle Scholar
  42. 42.
    Gharibi, W.: An improved lower bound of the spark with application. Int. J. Distrib. Parallel Syst. 3(5), 1–8 (2012)Google Scholar
  43. 43.
    Gilbert, J.R., Heath, M.T.: Computing a sparse basis for the null space. SIAM J. Algebraic Discrete Methods 8(3), 446–459 (1987)MathSciNetzbMATHGoogle Scholar
  44. 44.
    Gottlieb, L.A., Neylon, T.: Matrix sparsification and the sparse null space problem. Algorithmica 76(2), 426–444 (2016)MathSciNetzbMATHGoogle Scholar
  45. 45.
    Gribonval, R., Nielsen, M.: Sparse representations in unions of bases. IEEE Trans. Inf. Theory 49(2), 3320–3325 (2003)MathSciNetzbMATHGoogle Scholar
  46. 46.
    Grossman, J.W., Kulkarni, D.M., Schochetman, I.E.: Algebraic graph theory without orientation. Linear Algebra Appl. 212–213, 289–307 (1994)MathSciNetzbMATHGoogle Scholar
  47. 47.
    Grossman, J.W., Kulkarni, D.M., Schochetman, I.E.: On the minors of an incidence matrix and its Smith normal form. Linear Algebra Appl. 218, 213–224 (1995)MathSciNetzbMATHGoogle Scholar
  48. 48.
    Grötschel, M., Lovász, L., Schrijver, A.: Geometric Algorithms and Combinatorial Optimization, Algorithms and Combinatorics, vol. 2, 2nd edn. Springer, Berlin (1993)zbMATHGoogle Scholar
  49. 49.
    Helmling, M., Ruzika, S., Tanatmis, A.: Mathematical programming decoding of binary linear codes: theory and algorithms. IEEE Trans. Inf. Theory 58(7), 4753–4769 (2012)MathSciNetzbMATHGoogle Scholar
  50. 50.
    Helmling, M., Scholl, S., Gensheimer, F., Dietz, T., Kraft, K., Ruzika, S., Wehn, N.: Database of channel codes and ML simulation results (2017). https://www.uni-kl.de/channel-codes/channelcodes. Accessed 20 Apr 2018
  51. 51.
    Hoffman, A.J., McCormick, S.T.: A fast algorithm that makes matrices optimally sparse. In: Pulleyblank, W.R. (ed.) Progress in Combinatorial Optimization, pp. 185–196. Academic Press, Amsterdam (1984)Google Scholar
  52. 52.
    Itai, A., Rodeh, M.: Finding a minimum circuit in a graph. SIAM J. Comput. 7(4), 413–423 (1978)MathSciNetzbMATHGoogle Scholar
  53. 53.
    Iwen, M.A.: Simple deterministically constructible RIP matrices with sublinear Fourier sampling requirements. Proc. CISS 2009, 870–875 (2009)Google Scholar
  54. 54.
    Jensen, P.M., Korte, B.: Complexity of matroid property algorithms. SIAM J. Comput. 11(1), 184–190 (1982)MathSciNetzbMATHGoogle Scholar
  55. 55.
    Jokar, S., Pfetsch, M.E.: Exact and approximate sparse solutions of underdetermined linear equations. SIAM J. Sci. Comput. 31(1), 23–44 (2008)MathSciNetzbMATHGoogle Scholar
  56. 56.
    Juditsky, A., Nemirovski, A.: On verifiable sufficient conditions for sparse signal recovery via \(\ell 1\) minimization. Math. Program. 127(1), 57–88 (2011)MathSciNetzbMATHGoogle Scholar
  57. 57.
    Keha, A.B., Duman, T.: Minimum distance computation of LDPC codes using a branch and cut algorithm. IEEE Trans. Commun. 58(4), 1072–1079 (2010)Google Scholar
  58. 58.
    Khachiyan, L., Boros, E., Elbassioni, K., Gurvich, V., Makino, K.: On the complexity of some enumeration problems for matroids. SIAM J. Discrete Math. 19(4), 966–984 (2006)MathSciNetzbMATHGoogle Scholar
  59. 59.
    Kianfar, K., Pourhabib, A., Ding, Y.: An integer programming approach for analyzing the measurement redundancy in structured linear systems. IEEE Trans. Autom. Sci. Eng. 8(2), 447–450 (2011)Google Scholar
  60. 60.
    King, E.: Algebraic and geometric spread in finite frames. In: Proceedings of the SPIE 9597, Wavelets and Sparsity XVI, Article no. 95970B (2015)Google Scholar
  61. 61.
    Koiran, P., Zouzias, A.: On the Certification of the Restricted Isometry Property. arXiv:1103.4984 [cs.CC] (2011). Accessed 20 Apr 2018
  62. 62.
    Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM Rev. 51(3), 455–500 (2009)MathSciNetzbMATHGoogle Scholar
  63. 63.
    Kruskal, J.B.: Three-way arrays: rank and uniqueness of trilinear decompositions. Linear Algebra Appl. 18(2), 95–138 (1977)MathSciNetzbMATHGoogle Scholar
  64. 64.
    Lange, J.H., Pfetsch, M.E., Seib, B.M., Tillmann, A.M.: Sparse Recovery with Integrality Constraints. arXiv:1608.08678 [cs.IT] (2016). Accessed 20 Apr 2018
  65. 65.
    Lisoněk, P., Trummer, L.: Algorithms for the minimum weight of linear codes. Adv. Math. Commun. 10(1), 195–207 (2016)MathSciNetzbMATHGoogle Scholar
  66. 66.
    Liu, X.J., Xia, S.T.: Constructions of quasi-cyclic measurement matrices based on array codes. Proc. ISIT 2013, 479–483 (2013)Google Scholar
  67. 67.
    Maculan, N., Macambira, E.M., de Souza, C.C.: Geometrical Cuts for 0–1 Integer Programming. Technical Report IC-02-006, Instituto de Computacao, Universidade Estadual de Campinas (2002)Google Scholar
  68. 68.
    Maher, S.J., Fischer, T., Gally, T., Gamrath, G., Gleixner, A., Gottwald, R.L., Hendel, G., Koch, T., Lübbecke, M.E., Miltenberger, M., Müller, B., Pfetsch, M.E., Puchert, C., Rehfeldt, D., Schenker, S., Schwarz, R., Serrano, F., Shinano, Y., Weninger, D., Witt, J.T., Witzig, J.: The SCIP Optimization Suite 4.0. Technical Report 17-12, ZIB (2017)Google Scholar
  69. 69.
    Manikas, A., Proukakis, C.: Modeling and estimation of ambiguities in linear arrays. IEEE Trans. Signal Process. 46(8), 2166–2179 (1998)Google Scholar
  70. 70.
    McCormick, S.T.: A Combinatorial Approach to Some Sparse Matrix Problems. Ph.D. thesis, Stanford University (1983)Google Scholar
  71. 71.
    McCormick, S.T.: Making sparse matrices sparser: computational results. Math. Program. 49(1–3), 91–111 (1990)MathSciNetzbMATHGoogle Scholar
  72. 72.
    Minieka, E.: Finding the circuits of a matroid. J. Res. Natl. Bur. Stand. B 80B(3), 337–342 (1976)MathSciNetzbMATHGoogle Scholar
  73. 73.
    Mitchell, J.E.: Branch-and-cut algorithms for combinatorial optimization problems. In: Pardalos, P.M., Resende, M.G.C. (eds.) Handbook of Applied Optimization, pp. 65–77. Oxford Univ. Press, Oxford (2002)Google Scholar
  74. 74.
    Monien, B.: The complexity of determining a shortest cycle of even length. Computing 31(4), 355–369 (1983)MathSciNetzbMATHGoogle Scholar
  75. 75.
    Naidu, R.R., Jampana, P., Sastry, C.S.: Deterministic compressed sensing matrices: construction via Euler squares and applications. IEEE Trans. Signal Process. 64(14), 3566–3575 (2016)MathSciNetzbMATHGoogle Scholar
  76. 76.
    Oxley, J.G.: Matroid Theory. Oxford Graduate Texts Mathematics, 2nd edn. Oxford Univ. Press, Oxford (2011)Google Scholar
  77. 77.
    Robinson, G.C., Welsh, D.J.A.: The computational complexity of matroid properties. Math. Proc. Camb. Philos. Soc. 87(29), 29–45 (1980)MathSciNetzbMATHGoogle Scholar
  78. 78.
    Schrijver, A.: Combinatorial optimization: polyhedra and efficiency, volume A algorithms and combinatorics, vol. 24. Springer, Berlin (2003)zbMATHGoogle Scholar
  79. 79.
    Seymour, P.D.: Decomposition of regular matroids. J. Combin. Theory Ser. B 28, 305–359 (1980)MathSciNetzbMATHGoogle Scholar
  80. 80.
    Seymour, P.D.: A note on hyperplane generation. J. Combin. Theory Ser. B 61, 88–91 (1994)MathSciNetzbMATHGoogle Scholar
  81. 81.
    Tanatmis, A., Ruzika, S., Hamacher, H.W., Punekar, M., Kienle, F., Wehn, N.: Valid inequalities for binary linear codes. Proc. ISIT 2009, 2216–2220 (2009)zbMATHGoogle Scholar
  82. 82.
    Tillmann, A.M.: Computational Aspects of Compressed Sensing. Doctoral dissertation. TU Darmstadt, Germany (2013)Google Scholar
  83. 83.
    Tillmann, A.M.: On the computational intractability of exact and approximate dictionary learning. IEEE Signal Process. Lett. 22(1), 45–49 (2015)Google Scholar
  84. 84.
    Tillmann, A.M., Gribonval, R., Pfetsch, M.E.: Projection onto the cosparse set is NP-hard. Proc. ICASSP 2014, 7148–7152 (2014)Google Scholar
  85. 85.
    Tillmann, A.M., Pfetsch, M.E.: The computational complexity of the restricted isometry property, the nullspace property, and related concepts in compressed sensing. IEEE Trans. Inf. Theory 60(2), 1248–1259 (2014)MathSciNetzbMATHGoogle Scholar
  86. 86.
    Truemper, K.: A decomposition theory for matroids. V. Testing of matrix total unimodularity. J. Combin. Theory Ser. B 49, 241–281 (1990)MathSciNetzbMATHGoogle Scholar
  87. 87.
    Vardy, A.: The intractability of computing the minimum distance of a code. IEEE Trans. Inf. Theory 43(6), 1757–1766 (1997)MathSciNetzbMATHGoogle Scholar
  88. 88.
    Walter, M., Truemper, K.: Implementation of a unimodularity test. Math. Program. Comput. 5(1), 57–73 (2013)MathSciNetzbMATHGoogle Scholar
  89. 89.
    Zhang, J., Han, G., Fang, Y.: Deterministic construction of compressed sensing matrices from protograph LDPC codes. IEEE Signal Process. Lett. 22(11), 1960–1964 (2015)Google Scholar
  90. 90.
    Zhang, X.D.: Matrix Analysis and Applications. Cambridge Univ. Press, Cambridge (2017)zbMATHGoogle Scholar
  91. 91.
    Zhu, Z., So, A.M.C., Ye, Y.: Fast and near-optimal matrix completion via randomized basis pursuit. In: Ji, L., Poon, Y.S., Yang, L., Yau, S.T. (eds.) Proceedings of the 5th ICCM. AMS/IP Studies in Advanced Mathematics, vol. 51, pp. 859–882. AMS and International Press (2012)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Visual Computing Institute and Chair of Operations ResearchRWTH Aachen UniversityAachenGermany

Personalised recommendations