Abstract
The paper considers several optimization problem statements for sparse signal reconstruction problems. We tested the performance of AORDA portfolio safeguard (PSG) package with different problem formulations. We solved several medium-size test problems with cardinality functions: (a) minimize L1-error of regression subject to a constraint on cardinality of the solution vector; (b) minimize cardinality of the solution vector subject to a constraint on L1-error of regression. We compared performance of PSG and IBM CPLEX solvers on these problems. Although cardinality formulations are very appealing because of the direct control of the number of nonzero variables, large problems are beyond the reach of the tested commercial solvers. Step-down from the cardinality formulations is the formulation with the constraint on the sum of absolute values of the solution vector. This constraint is a relaxation of the cardinality constraint. Medium and large problems (from SPARCO toolbox for testing sparse reconstruction algorithms) were solved with PSG in the following formulation: minimize L1-error subject to a constraint on the sum of absolute values of the solution vector. The further step-down in the sparse reconstruction problem formulations is the LASSO approach which does not have any constraints on functions. With the LASSO approach you do not know in advance the cardinality of the solution vector and you solve many problems with different regularization parameters. Then you select a solution with appropriate regression error and cardinality. Definitely, it is a time-consuming process, but an advantage of LASSO approach is that optimization problems can be solved quite quickly even for very large problems. We have solved with PSG several medium and large problems from the SPARCO toolbox in LASSO formulation (minimize L2-error plus the weighted sum of absolute values of the solution vector).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
American Optimal Decisions (AORDA), Portfolio Safeguard (PSG) (2009). http://www.aorda.com
L1-magic (2009). http://www.acm.caltech.edu/l1magic/
Berg, E.v., Friedlander, M.P.: SPARCO: a toolbox for testing sparse reconstruction algorithms (2008). http://www.cs.ubc.ca/labs/scl/sparco/
Berg, E.v., Friedlander, M.P., Hennenfent, G., Herrmann, F., Saab, R., Yılmaz, Ö.: Sparco: a testing framework for sparse reconstruction. Technical Report TR-2007-20, Department of Computer Science, University of British Columbia, Vancouver (2007)
Buckheit, J., Donoho, D.L.: Wavelab and reproducible research. In: Wavelets and Statistics. Springer, Berlin (1995). http://citeseer.ist.psu.edu/article/buckheit95wavelab.html
Candés, E.J., Romberg, J.: Practical signal recovery from random projections. In: Proceedings of SPIE Conference Wavelet Applications in Signal and Image Processing XI, vol. 5914, San Diego (2004)
Candès, E., Romberg, J.: L1-magic (2007). http://www.l1-magic.org/
Candes, E., Tao, T.: The dantzig selector: statistical estimation when p is much larger than n. Ann. Stat. 35, 2313 (2007). doi: 10.1214/009053606000001523
Candes, E., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)
Candes, E., Romberg, J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Commun. Pure Appl. Math. 59(8), 1207–1223 (2006)
Chen, S.S., Donoho, D.L., Saunders, M.A.: Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20(1), 33–61 (1998). http://epubs.siam.org/SISC/volume-20/art30401.html
Compressed Sensing Resources (2009). http://www.dsp.ece.rice.edu/cs/
Database of Creative Commons licensed sounds (2007). http://freesound.iua.upf.edu/
Donoho, D.L.: Compressed sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)
Donoho, D.L.: For most large underdetermined systems of equations, the minimal ℓ 1-norm near-solution approximates the sparsest near-solution. Commun. Pure Appl. Math. 59(7), 907–934 (2006)
Donoho, D.L., Johnstone, I.M.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3), 425–455 (1994). http://citeseer.ist.psu.edu/donoho93ideal.html
Donoho, D.L., Tsaig, Y.: Fast solution of ℓ 1-norm minimization problems when the solution may be sparse. Technical Report, Institute for Computational and Mathematical Engineering, Stanford University (2006). http://www-stat.stanford.edu/~donoho/
Dossal, C., Mallat, S.: Sparse spike deconvolution with minimum scale. In: Proceedings of Signal Processing with Adaptive Sparse Structured Representations, pp. 123–126, Rennes, France (2005). http://spars05.irisa.fr/ACTES/PS2-11.pdf
Figueiredo, M., Nowak, R., Wright, S.: Gradient projection for sparse reconstruction: application to compressed sensing and other inverse problems. IEEE J. Sel. Topics Signal Process. 1(4), 586–597 (2007). DOI 10.1109/JSTSP.2007.910281. http://www.lx.it.pt/~mtf/GPSR
Hennenfent, G., Herrmann, F.J.: Sparseness-constrained data continuation with frames: applications to missing traces and aliased signals in 2/3-D. In: SEG International Exposition and 75th Annual Meeting. Tulsa, OK, USA (2005). http://slim.eos.ubc.ca/Publications/Public/Conferences/SEG/hennenfent05seg.pdf
Hennenfent, G., Herrmann, F.J.: Simply denoise: wavefield reconstruction via coarse nonuniform sampling. Technical Report, UBC Earth & Ocean Sciences (2007)
Herrmann, F.J., Hennenfent, G.: Non-parametric seismic data recovery with curvelet frames. Technical Report, UBC Earth & Ocean Sciences Department (2007). http://slim.eos.ubc.ca/Publications/Public/Journals/CRSI.pdf. [TR-2007-1]
IBM CPLEX 11.2 (2009). http://www.ilog.com/products/cplex/
Eldar, Y.C., Kutyniok, G.: Compressed sensing: Theory and applications. Cambridge University Press (2012)
Takhar, D., Laska, J.N., Wakin, M., Duarte, M., Baron, D., Sarvotham, S., Kelly, K.K., Baraniuk, R.G.: A new camera architecture based on optical-domain compression. In: Proceedings of the IS&T/SPIE Symposium on Electronic Imaging: Computational Imaging, vol. 6065, Springfield, VA, USA and Bellingham, WA, USA (2006)
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B 58, 267–288 (1994)
Wang, L., Gordon, M.D., Zhu, J.: Regularized least absolute deviations regression and an efficient algorithm for parameter tuning. In: ICDM ’06: Proceedings of the 6th International Conference on Data Mining, pp. 690–700, IEEE Computer Society, Washington, DC, USA (2006). DOI http://dx.doi.org/10.1109/ICDM.2006.134
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Boyko, N., Karamemis, G., Kuzmenko, V., Uryasev, S. (2014). Sparse Signal Reconstruction: LASSO and Cardinality Approaches. In: Vogiatzis, C., Walteros, J., Pardalos, P. (eds) Dynamics of Information Systems. Springer Proceedings in Mathematics & Statistics, vol 105. Springer, Cham. https://doi.org/10.1007/978-3-319-10046-3_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-10046-3_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10045-6
Online ISBN: 978-3-319-10046-3
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)