A filtered bucket-clustering method for projection onto the simplex and the \(\ell _1\) ball

Abstract

We propose in this paper a new method processing the projection of an arbitrary size vector onto the probabilistic simplex or the \(\ell _1\) ball. Our method merges two principles. The first one is an original search of the projection using a bucket algorithm. The second one is a filtering, on the fly, of the values that cannot be part of the projection. The combination of these two principles offers a simple and efficient algorithm whose worst-case complexity is linear with respect to the vector size. Furthermore, the proposed algorithm exploits the representation of numeric values in digital computers to define the number of buckets and to accelerate the filtering.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Notes

  1. 1.

    The code used for these experiments is available at www.perezguillau.me/code/projection.zip. Note that part of this code is based on the generously provided code from https://www.gipsa-lab.grenoble-inp.fr/~laurent.condat/software.html.

References

  1. 1.

    Barlaud, M., Belhajali, W., Combettes, P.L., Fillatre, L.: Classification and regression using an outer approximation projection-gradient method. IEEE Trans. Signal Process. 65(17), 4635–4644 (2017)

    MathSciNet  Article  Google Scholar 

  2. 2.

    Candès, E.J., Tao, T.: Decoding by linear programming. IEEE Trans. Inf. Theory 51(12), 4203–4215 (2005)

    MathSciNet  Article  Google Scholar 

  3. 3.

    Condat, L.: Fast projection onto the simplex and the \(\ell _1\) ball. Math. Program. Ser. A 158(1), 575–585 (2016)

    MathSciNet  Article  Google Scholar 

  4. 4.

    Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: Introduction to Algorithms., vol. 6. MIT Press, Cambridge (2001)

    Google Scholar 

  5. 5.

    Donoho, D.L.: De-noising by soft-thresholding. IEEE Trans. Inf. Theory 41(3), 613–627 (1995)

    MathSciNet  Article  Google Scholar 

  6. 6.

    Donoho, D.L., Elad, M.: Optimally sparse representation in general (nonorthogonal) dictionaries via \(\ell _1\) minimization. Proc. Natl. Acad. Sci. 100(5), 2197–2202 (2003)

    MathSciNet  Article  Google Scholar 

  7. 7.

    Duchi, J., Shalev-Shwartz, S., Singer, Y., Chandra, T.: Efficient projections onto the \(\ell _1\)-ball for learning in high dimensions. In: Proceedings of the 25th International Conference on Machine Learning. ACM (2008), pp. 272–279

  8. 8.

    Held, M., Wolfe, P., Crowder, H.P.: Validation of subgradient optimization. Math. Program. 6(1), 62–88 (1974)

    MathSciNet  Article  Google Scholar 

  9. 9.

    Kiwiel, K.C.: Breakpoint searching algorithms for the continuous quadratic knapsack problem. Math. Program. 112(2), 473–491 (2008)

    MathSciNet  Article  Google Scholar 

  10. 10.

    Michelot, C.: A finite algorithm for finding the projection of a point onto the canonical simplex of \({\mathbb{R}}^n\). J. Optim. Theory Appl. 50(1), 195–200 (1986)

    MathSciNet  Article  Google Scholar 

  11. 11.

    Natarajan, B.: Sparse approximate solutions to linear systems. SIAM J. Comput. 24(2), 227–234 (1995)

    MathSciNet  Article  Google Scholar 

  12. 12.

    Perez, G., Barlaud, M., Fillatre, l., Régin, J.-C.: A filtered bucket-clustering method for Projection onto the Simplex and the \(\ell _1\) ball. In: XXVIème Colloque Gretsi, Juan-Les-Pins (2017)

  13. 13.

    Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.) 267–288 (1996)

  14. 14.

    Tibshirani, R.: The lasso method for variable selection in the COX model. Stat. Med. (1997)

  15. 15.

    Van Den Berg, E., Friedlander, M.P.: Probing the pareto frontier for basis pursuit solutions. SIAM J. Sci. Comput. 31(2), 890–912 (2008)

    MathSciNet  Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Guillaume Perez.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Part of this work has been already published [12].

Appendix A

Appendix A

See Table 3.

Table 3 (Top) Projection time using Gaussian generation with \(\mu =a/d\), and \(\sigma =10^{-3}\), and \(a=1\). (Bottom) Projection time using Gaussian generation with \(\mu =a/d\), and \(\sigma =10^{-3}\), and \(a=64\)

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Perez, G., Barlaud, M., Fillatre, L. et al. A filtered bucket-clustering method for projection onto the simplex and the \(\ell _1\) ball. Math. Program. 182, 445–464 (2020). https://doi.org/10.1007/s10107-019-01401-3

Download citation

Mathematics Subject Classification

  • 49M30
  • 65C60
  • 65K05
  • 90C25