A Bidirectional Greedy Heuristic for the Subspace Selection Problem

  • Dag Haugland
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4638)


The Subspace Selection Problem (SSP) amounts to selecting t out of n given vectors of dimension m, such that they span a subspace in which a given target \(b\in\Re^m\) has a closest possible approximation. This model has numerous applications in e.g. signal compression and statistical regression. It is well known that the problem is NP-hard. Based on elements from a forward and a backward greedy method, we develop a randomized search heuristic, which in some sense resembles variable neighborhood search, for SSP. Through numerical experiments we demonstrate that this approach has good promise, as it produces good results at modest computational cost.


Subspace selection dimension reduction greedy methods computational experiments 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Miller, A.J.: Subset Selection in Regression, 2nd edn. Chapman and Hall, London, U.K (2002)zbMATHGoogle Scholar
  2. 2.
    Couvreur, C., Bresler, Y.: On the optimality of the backward greedy algorithm for the subset selection problem. SIAM Journal on Matrix Analysis and Applications 21, 797–808 (2000)zbMATHCrossRefGoogle Scholar
  3. 3.
    Mallat, S., Zhang, Z.: Matching Pursuit in a Time-frequency Dictionary. IEEE Transactions on Signal Processing 41, 3397–3415 (1993)zbMATHCrossRefGoogle Scholar
  4. 4.
    Pati, Y.C., Rezaiifar, R., Krishnaprasad, P.S.: Orthogonal Matching Pursuit: Recursive function approximation with applications to wavelet decomposition. In: Proc. 27th Annu Asilomar Conf. Signals, Systems and Computers, Pacific Grove, CA, pp. 40–44 (1993)Google Scholar
  5. 5.
    Gharavi-Alkhansari, M., Huang, T.S.: A Fast Orthogonal Matching Pursuit Algorithm. In: Proc. ICASSP 1998, Seattle, Washington, USA, pp. 1389–1392 (1998)Google Scholar
  6. 6.
    Natarajan, B.K: Sparse approximate solutions to linear systems. SIAM J. Comput. 24, 227–234 (1995)zbMATHCrossRefGoogle Scholar
  7. 7.
    Haugland, D., Storøy, S.: Local search methods for ℓ1-minimization in frame based signal compression. Optimization and Engineering 7, 81–96 (2006)CrossRefGoogle Scholar
  8. 8.
    Garey, M.R., Johnson, D.S.: Computers and Intractability: A Guide to the Theory of NP-completeness. W.H. Freeman, New York (1979)zbMATHGoogle Scholar
  9. 9.
    Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by Basis Pursuit. SIAM Journal on Scientific Computing 20, 33–61 (1998)CrossRefGoogle Scholar
  10. 10.
    Golub, G.H., Van Loan, C.F.: Matrix Computations, 3rd edn. Johns Hopkins Univ. Press, Baltimore, MD (1996)zbMATHGoogle Scholar
  11. 11.
    Reeves, S.J.: An Efficient Implementation of the Backward Greedy Algorithm for Sparse Signal Reconstruction. IEEE Signal Processing Letters 6, 266–268 (1999)CrossRefGoogle Scholar
  12. 12.
    Mladenović, N., Hansen, P.: Variable Neighborhood Search. Computers and Operations Research 24, 1097–1100 (1997)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Dag Haugland
    • 1
  1. 1.Department of Informatics, University of Bergen, BergenNorway

Personalised recommendations