Skip to main content
SpringerLink
Log in

Algorithms for Sparse k-Monotone Regression

  • Conference paper
  • First Online:
Integration of Constraint Programming, Artificial Intelligence, and Operations Research (CPAIOR 2018)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10848))

Abstract

The problem of constructing k-monotone regression is to find a vector \(z\in \mathbb {R}^n\) with the lowest square error of approximation to a given vector \(y\in \mathbb {R}^n\) (not necessary k-monotone) under condition of k-monotonicity of z. The problem can be rewritten in the form of a convex programming problem with linear constraints. The paper proposes two different approaches for finding a sparse k-monotone regression (Frank-Wolfe-type algorithm and k-monotone pool adjacent violators algorithm). A software package for this problem is developed and implemented in R. The proposed algorithms are compared using simulated data.

The work was supported by RFBR (grant 18-37-00060).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahuja, R., Orlin, J.: A fast scaling algorithm for minimizing separable convex functions subject to chain constraints. Oper. Res. 49(1), 784–789 (2001)

    Article  MathSciNet  Google Scholar 

  2. Altmann, D., Grycko, E., Hochstättler, W., Klützke, G.: Monotone smoothing of noisy data. Diskrete Mathematik und Optimierung. Technical report feu-dmo034.15. Fern Universität in Hagen, Fakultät für Mathematik und Informatik (2014)

    Google Scholar 

  3. Bach, F.: Efficient algorithms for non-convex isotonic regression through submodular optimization (2017), Working paper or preprint

    Google Scholar 

  4. Balabdaoui, F., Rufibach, K., Santambrogio, F.: Least-squares estimation of two-ordered monotone regression curves. J. Nonparametr. Stat. 22(8), 1019–1037 (2010)

    Article  MathSciNet  Google Scholar 

  5. Barlow, R., Brunk, H.: The isotonic regression problem and its dual. J. Am. Stat. Assoc. 67(337), 140–147 (1972)

    Article  MathSciNet  Google Scholar 

  6. Best, M.J., Chakravarti, N.: Active set algorithms for isotonic regression: a unifying framework. Math. Progr.: Ser. A B 47(3), 425–439 (1990)

    Article  MathSciNet  Google Scholar 

  7. Best, M., Chakravarti, N., Ubhaya, V.: Minimizing separable convex functions subject to simple chain constraints. SIAM J. Optim. 10(3), 658–672 (2000)

    Article  MathSciNet  Google Scholar 

  8. Bor, H.: A study on local properties of Fourier series. Nonlinear Anal.: Theory Methods Appl. 57(2), 191–197 (2004)

    Article  MathSciNet  Google Scholar 

  9. Bor, H.: A note on local property of factored Fourier series. Nonlinear Anal.: Theory Methods Appl. 64(3), 513–517 (2006)

    Article  MathSciNet  Google Scholar 

  10. Boytsov, D.I., Sidorov, S.P.: Linear approximation method preserving \(k\)-monotonicity. Sib. Electron. Math. Rep. 12, 21–27 (2015)

    MathSciNet  MATH  Google Scholar 

  11. Brezger, A., Steiner, W.J.: Monotonic regression based on bayesian P-splines. J. Bus. Econ. Stat. 26(1), 90–104 (2008)

    Article  Google Scholar 

  12. Burdakov, O., Grimvall, A., Hussian, M.: A generalised PAV algorithm for monotonic regression in several variables. In: Antoch, J. (ed.) Proceedings of the 16th Symposium in Computational Statistics, COMPSTAT, vol. 10, no. 1, pp. 761–767 (2004)

    Google Scholar 

  13. Burdakov, O., Sysoev, O.: A dual active-set algorithm for regularized monotonic regression. J. Optim. Theory Appl. 172(3), 929–949 (2017)

    Article  MathSciNet  Google Scholar 

  14. Cai, Y., Judd, K.L.: Shape-preserving dynamic programming. Math. Methods Oper. Res. 77, 407–421 (2013)

    Article  MathSciNet  Google Scholar 

  15. Cai, Y., Judd, K.L.: Advances in numerical dynamic programming and new applications. In: Handbook of Computational Economics, vol. 3, pp. 479–516. Elsevier (2014)

    Chapter  Google Scholar 

  16. Chen, Y.: Aspects of shape-constrained estimation in statistics. Ph.D. thesis. University of Cambridge (2013)

    Google Scholar 

  17. Chepoi, V., Cogneau, D., Fichet, B.: Polynomial algorithms for isotonic regression. Lect. Notes-Monogr. Ser. 31(1), 147–160 (1997)

    Article  MathSciNet  Google Scholar 

  18. Cullinan, M.P.: Piecewise convex-concave approximation in the minimax norm. In: Demetriou, I., Pardalos, P. (eds.) Abstracts of Conference on Approximation and Optimization: Algorithms, Complexity, and Applications, Athens, Greece, 29–30 June 2017, p. 4. National and Kapodistrian University of Athens (2017)

    Google Scholar 

  19. Diggle Peter, M.S., Tony, M.J.: Case-control isotonic regression for investigation of elevation in risk around a point source. Stat. Med. 18(1), 1605–1613 (1999)

    Article  Google Scholar 

  20. Dykstra, R.: An isotonic regression algorithm. J. Stat. Plan. Inference 5(1), 355–363 (1981)

    Article  MathSciNet  Google Scholar 

  21. Dykstra, R., Robertson, T.: An algorithm for isotonic regression for two or more independent variables. Ann. Stat. 10(1), 708–719 (1982)

    Article  MathSciNet  Google Scholar 

  22. Faizliev, A.R., Gudkov, A.A., Mironov, S.V., Levshunov, M.A.: Greedy algorithm for sparse monotone regression. In: CEUR Workshop Proceedings, vol. 2018, pp. 23–31 (2017)

    Google Scholar 

  23. Frank, M., Wolfe, P.: An algorithm for quadratic programming. Nav. Res. Logist. Q. 3(1–2), 95–110 (1956)

    Article  MathSciNet  Google Scholar 

  24. Gal, S.G.: Shape-Preserving Approximation by Real and Complex Polynomials. Birkhäuser, Boston (2008)

    Book  Google Scholar 

  25. Gorinevsky, D.: Monotonic regression filters for trending deterioration faults. In: Proceedings of the American Control Conference, vol. 6, pp. 5394–5399 (2004)

    Google Scholar 

  26. Gorinevsky, D.: Efficient filtering using monotonic walk model. In: Proceedings of the American Control Conference, pp. 2816–2821. IEEE (2008)

    Google Scholar 

  27. Gudkov, A.A., Mironov, S.V., Faizliev, A.R.: On the convergence of a greedy algorithm for the solution of the problem for the construction of monotone regression. Izv. Sarat. Univ. (N. S.) Ser. Math. Mech. Inform. 17(4), 431–440 (2017)

    Article  Google Scholar 

  28. Hansohm, J.: Algorithms and error estimations for monotone regression on partially preordered sets. J. Multivar. Anal. 98(5), 1043–1050 (2007)

    Article  MathSciNet  Google Scholar 

  29. Hastie, T., Tibshirani, R., Wainwright, M.: Statistical Learning with Sparsity. Chapman and Hall/CRC, New York (2015)

    MATH  Google Scholar 

  30. Hazelton, M., Turlach, B.: Semiparametric regression with shape-constrained penalized splines. Comput. Stat. Data Anal. 55(10), 2871–2879 (2011)

    Article  MathSciNet  Google Scholar 

  31. Jaggi, M.: Revisiting Frank-Wolfe: projection-free sparse convex optimization. In: Proceedings of the 30th International Conference on Machine Learning, ICML 2013, pp. 427–435 (2013)

    Google Scholar 

  32. Judd, K.: Numerical Methods in Economics. The MIT Press, Cambridge (1998)

    MATH  Google Scholar 

  33. Latreuch, Z., Belaïdi, B.: New inequalities for convex sequences with applications. Int. J. Open Probl. Comput. Math. 5(3), 15–27 (2012)

    Article  Google Scholar 

  34. Leeuw, J., Hornik, K., Mair, P.: Isotone optimization in R: pool-adjacent-violators algorithm (PAVA) and active set methods. J. Stat. Softw. 32(5), 1–24 (2009)

    Article  Google Scholar 

  35. Leindler, L.: A new extension of monotone sequences and its applications. J. Inequal. Pure Appl. Math. 7(1), 7 (2006). Paper No. 39 electronic only. http://eudml.org/doc/128520

    MathSciNet  MATH  Google Scholar 

  36. Leitenstorfer, F., Tutz, G.: Generalized monotonic regression based on B-splines with an application to air pollution data. Biostatistics 8(3), 654–673 (2007)

    Article  Google Scholar 

  37. Lu, M.: Spline estimation of generalised monotonic regression. J. Nonparametr. Stat. 27(1), 19–39 (2014)

    Article  MathSciNet  Google Scholar 

  38. Gorinevsky, D., Kim, S.J., Beard, S., Boyd, S., Gordon, G.: Optimal estimation of deterioration from diagnostic image sequence. IEEE Trans. Signal Process. 57(3), 1030–1043 (2009)

    Article  MathSciNet  Google Scholar 

  39. Marshall, A.W., Olkin, I., Arnold, B.C.: Inequalities: Theory of Majorization and Its Applications. Springer, New York (2011). https://doi.org/10.1007/978-0-387-68276-1

    Book  MATH  Google Scholar 

  40. Milovanović, I.Z., Milovanović, E.I.: Some properties of \(l_p^k\)-convex sequences. Bull. Int. Math. Virtual Inst. 5(1), 33–36 (2015)

    MathSciNet  MATH  Google Scholar 

  41. Niezgoda, M.: Inequalities for convex sequences and nondecreasing convex functions. Aequ. Math. 91(1), 1–20 (2017)

    Article  MathSciNet  Google Scholar 

  42. Robertson, T., Wright, F., Dykstra, R.: Order Restricted Statistical Inference. Wiley, New York (1988)

    MATH  Google Scholar 

  43. Shevaldin, V.T.: Local approximation by splines. UrO RAN, Ekaterinburg (2014)

    Google Scholar 

  44. Sidorov, S.P., Mironov, S.V.: Duality gap analysis of weak relaxed greedy algorithms. In: Battiti, R., Kvasov, D.E., Sergeyev, Y.D. (eds.) LION 2017. LNCS, vol. 10556, pp. 251–262. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69404-7_18

    Chapter  Google Scholar 

  45. Sidorov, S.P., Mironov, S.V., Pleshakov, M.G.: Dual convergence estimates for a family of greedy algorithms in Banach spaces. In: Nicosia, G., Pardalos, P., Giuffrida, G., Umeton, R. (eds.) MOD 2017. LNCS, vol. 10710, pp. 109–120. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-72926-8_10

    Chapter  Google Scholar 

  46. Sidorov, S.: On the saturation effect for linear shape-preserving approximation in Sobolev spaces. Miskolc Math. Notes 16(2), 1191–1197 (2015)

    Article  MathSciNet  Google Scholar 

  47. Sidorov, S.P.: Linear k-monotonicity preserving algorithms and their approximation properties. In: Kotsireas, I.S., Rump, S.M., Yap, C.K. (eds.) MACIS 2015. LNCS, vol. 9582, pp. 93–106. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-32859-1_7

    Chapter  Google Scholar 

  48. Siem, A.Y.D., den Hertog, D., Hoffmann, A.L.: Multivariate convex approximation and least-norm convex data-smoothing. In: Gavrilova, M., Gervasi, O., Kumar, V., Tan, C.J.K., Taniar, D., Laganá, A., Mun, Y., Choo, H. (eds.) ICCSA 2006. LNCS, vol. 3982, pp. 812–821. Springer, Heidelberg (2006). https://doi.org/10.1007/11751595_86

    Chapter  MATH  Google Scholar 

  49. Stromberg, U.: An algorithm for isotonic regression with arbitrary convex distance function. Comput. Stat. Data Anal. 11(1), 205–219 (1991)

    Article  MathSciNet  Google Scholar 

  50. Toader, G.: The representation of \(n\)-convex sequences. L’Anal. Numér. et la Théorie de L’Approx. 10(1), 113–118 (1981)

    MathSciNet  MATH  Google Scholar 

  51. Wu, S., Debnath, L.: Inequalities for convex sequences and their applications. Comput. Math. Appl. 54(4), 525–534 (2007)

    Article  MathSciNet  Google Scholar 

  52. Wu, W.B., Woodroofe, M., Mentz, G.: Isotonic regression: another look at the changepoint problem. Biometrika 88(3), 793–804 (2001)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Saratov State University, Saratov, Russian Federation

    Sergei P. Sidorov, Alexey R. Faizliev, Alexander A. Gudkov & Sergei V. Mironov

Authors
  1. Sergei P. Sidorov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alexey R. Faizliev
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander A. Gudkov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sergei V. Mironov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sergei P. Sidorov .

Editor information

Editors and Affiliations

  1. Carnegie Mellon University, Pittsburgh, Pennsylvania, USA

    Willem-Jan van Hoeve

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sidorov, S.P., Faizliev, A.R., Gudkov, A.A., Mironov, S.V. (2018). Algorithms for Sparse k-Monotone Regression. In: van Hoeve, WJ. (eds) Integration of Constraint Programming, Artificial Intelligence, and Operations Research. CPAIOR 2018. Lecture Notes in Computer Science(), vol 10848. Springer, Cham. https://doi.org/10.1007/978-3-319-93031-2_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-93031-2_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-93030-5

  • Online ISBN: 978-3-319-93031-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation