Skip to main content
SpringerLink
Log in

An Improved Deterministic Rescaling for Linear Programming Algorithms

  • Conference paper
  • First Online:
Integer Programming and Combinatorial Optimization (IPCO 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10328))

Abstract

The perceptron algorithm for linear programming, arising from machine learning, has been around since the 1950s. While not a polynomial-time algorithm, it is useful in practice due to its simplicity and robustness. In 2004, Dunagan and Vempala showed that a randomized rescaling turns the perceptron method into a polynomial time algorithm, and later Peña and Soheili gave a deterministic rescaling. In this paper, we give a deterministic rescaling for the perceptron algorithm that improves upon the previous rescaling methods by making it possible to rescale much earlier. This results in a faster running time for the rescaled perceptron algorithm. We will also demonstrate that the same rescaling methods yield a polynomial time algorithm based on the multiplicative weights update method. This draws a connection to an area that has received a lot of recent attention in theoretical computer science.

T. Rothvoss—Supported by an Alfred P. Sloan Research Fellowship. Both authors supported by NSF grant 1420180 with title “Limitations of convex relaxations in combinatorial optimization”.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The \(\tilde{O}\)-notation suppresses any \({\text {polylog}}(m,n)\) terms.

  2. 2.

    It suffices here to consider the trivial example with \(\lambda _1=\ldots = \lambda _n = \frac{1}{n}\) and \(A_i = e_i\) being the standard basis. Then \(\Vert \sum _{i \in J} \lambda _i A_i\Vert _2 \le \frac{1}{\sqrt{n}}\) for any subset J. The optimality of our rescaling can also be seen since the cone in the last iteration is \(\tilde{O}(n)\)-well rounded, which is optimal up to \(\tilde{O}\)-terms.

  3. 3.

    Recall that a function \(F : {\mathbb {R}}^n \rightarrow {\mathbb {R}}\) is Lipschitz with Lipschitz constant 1 if \(|F(x)-F(y)| \le \Vert x-y\Vert _2\) for all \(x,y \in {\mathbb {R}}^n\). A famous concentration inequality by Sudakov, Tsirelson, Borell states that \(\Pr [|F(g)-\mu | \ge t] \le e^{-t^2/\pi ^2}\), where g is a random Gaussian and \(\mu \) is the mean of F under g.

References

  1. Agmon, S.: The relaxation method for linear inequalities. Can. J. Math. 6, 382–392 (1954)

    Article  MathSciNet  MATH  Google Scholar 

  2. Arora, S., Hazan, E., Kale, S.: Fast algorithms for approximate semidefinite programming using the multiplicative weights update method. In: 46th IEEE FOCS, pp. 339–348 (2005)

    Google Scholar 

  3. Arora, S., Hazan, E., Kale, S.: The multiplicative weights update method: a meta-algorithm and applications. Theor. Comp. 8, 121–164 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  4. Alon, N., Spencer, J.H.: The Probabilistic Method. Wiley Series in Discrete Mathematics and Optimization. Wiley, Hoboken (2004)

    MATH  Google Scholar 

  5. Ball, K.: An elementary introduction to modern convex geometry. In: Silvio, L. (ed.) Flavors of Geometry, pp. 1–58. University Press, Cambridge (1997)

    Google Scholar 

  6. Betke, U.: Relaxation, new combinatorial and polynomial algorithms for the linear feasibility problem. Discrete Comput. Geom. 32(3), 317–338 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  7. Conforti, M., Cornuejols, G., Zambelli, G.: Integer Programming. Springer Publishing Company Inc., Heidelberg (2014)

    Book  MATH  Google Scholar 

  8. Chubanov, S.: A strongly polynomial algorithm for linear systems having a binary solution. Math. Program. 134(2), 533–570 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  9. Chubanov, S.: A polynomial projection algorithm for linear feasibility problems. Math. Program. 153(2), 687–713 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  10. Christiano, P., Kelner, J.A., Madry, A., Spielman, D.A., Teng, S.: Electrical flows, laplacian systems, and faster approximation of maximum flow in undirected graphs. In: Proceedings of the 43rd ACM Symposium on Theory of Computing, New York, NY, USA, pp. 273–282 (2011)

    Google Scholar 

  11. Dantzig, G.B.: Maximization of a linear function of variables subject to linear inequalities. In: Activity Analysis of Production and Allocation, Cowles Commission Monograph, vol. 13, pp. 339–347. John Wiley & Sons Inc., Chapman & Hall Ltd., New York (1951)

    Google Scholar 

  12. Dunagan, J., Vempala, S.: A simple polynomial-time rescaling algorithm for solving linear programs. Math. Program. 114(1), 101–114 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  13. Dadush, D., Végh, L.A., Zambelli, G.: Rescaling algorithms for linear programming - part I: conic feasibility. CoRR, abs/1611.06427 (2016)

    Google Scholar 

  14. Garg, N., Könemann, J.: Faster and simpler algorithms for multicommodity flow and other fractional packing problems. SIAM J. Comput. 37(2), 630–652 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  15. Hačijan, L.G.: A polynomial algorithm in linear programming. Dokl. Akad. Nauk SSSR 244(5), 1093–1096 (1979)

    MathSciNet  MATH  Google Scholar 

  16. John, F.: Extremum problems with inequalities as subsidiary conditions. In: Friedrichs, K.O., Neugebauer, O.E., Stoker, J.J. (eds.) Studies and Essays presented to R. Courant on his 60th Birthday, pp. 187–204. Interscience Publishers, New York (1948)

    Google Scholar 

  17. Karmarkar, N.: A new polynomial-time algorithm for linear programming. Combinatorica 4(4), 373–395 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  18. Klee, V., Minty, G.: How good is the simplex algorithm? In: Inequalities, III (Proceedings Third Symposium, UCLA, 1969; Dedicated to the Memory of Theodore S. Motzkin), pp. 159–175. Academic Press, New York (1972)

    Google Scholar 

  19. Lee, Y., Sinford, A.: A new polynomial-time algorithm for linear programming (2015). https://arxiv.org/abs/1312.6677

  20. Madry, A.: Faster approximation schemes for fractional multicommodity flow problems via dynamic graph algorithms. In: Proceedings of the 42nd ACM Symposium on Theory of Computing, New York, NY, pp. 121–130 (2010)

    Google Scholar 

  21. Nesterov, Y.: Excessive gap technique in nonsmooth convex minimization. SIAM J. Optim. 16(1), 235–249 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  22. Peña, J., Soheili, N.: A smooth perceptron algorithm. SIAM J. Optim. 22(2), 728–737 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  23. Peña, J., Soheili, N.: A deterministic rescaled perceptron algorithm. Math. Program. 155(1–2), 497–510 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  24. Plotkin, S.A., Shmoys, D.B., Tardos, E.: Fast approximation algorithms for fractional packing and covering problems. Math. Oper. Res. 20(2), 257–301 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  25. Schrijver, A.: Theory of linear and integer programming. Wiley-Interscience Series in Discrete Mathematics. John Wiley and Sons, Inc., New York (1986)

    MATH  Google Scholar 

  26. Vazirani, V.: Approximation Algorithms. Springer, Heidelberg (2001)

    MATH  Google Scholar 

  27. Williamson, D.P., Shmoys, D.B.: The Design of Approximation Algorithms. University Press, Cambridge (2011)

    Book  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Washington, Seattle, WA, 98105, USA

    Rebecca Hoberg & Thomas Rothvoss

Authors
  1. Rebecca Hoberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Rothvoss
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rebecca Hoberg .

Editor information

Editors and Affiliations

  1. Ecole Polytechnique Federale de Lausanne - EPFL, Lausanne, Switzerland

    Friedrich Eisenbrand

  2. University of Waterloo, Waterloo, Ontario, Canada

    Jochen Koenemann

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Hoberg, R., Rothvoss, T. (2017). An Improved Deterministic Rescaling for Linear Programming Algorithms. In: Eisenbrand, F., Koenemann, J. (eds) Integer Programming and Combinatorial Optimization. IPCO 2017. Lecture Notes in Computer Science(), vol 10328. Springer, Cham. https://doi.org/10.1007/978-3-319-59250-3_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-59250-3_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-59249-7

  • Online ISBN: 978-3-319-59250-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation