Computational Optimization and Applications

, Volume 45, Issue 1, pp 159–179 | Cite as

An algorithm for the estimation of a regression function by continuous piecewise linear functions

  • Adil Bagirov
  • Conny Clausen
  • Michael Kohler


The problem of the estimation of a regression function by continuous piecewise linear functions is formulated as a nonconvex, nonsmooth optimization problem. Estimates are defined by minimization of the empirical L 2 risk over a class of functions, which are defined as maxima of minima of linear functions. An algorithm for finding continuous piecewise linear functions is presented. We observe that the objective function in the optimization problem is semismooth, quasidifferentiable and piecewise partially separable. The use of these properties allow us to design an efficient algorithm for approximation of subgradients of the objective function and to apply the discrete gradient method for its minimization. We present computational results with some simulated data and compare the new estimator with a number of existing ones.


Nonsmooth optimization Nonparametric regression Subdifferential Semismooth functions 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bagirov, A.M.: Minimization methods for one class of nonsmooth functions and calculation of semi-equilibrium prices. In: Eberhard, A., et al. (eds.) Progress in Optimization: Contribution from Australia, pp. 147–175. Kluwer Academic, Dordrecht (1999) Google Scholar
  2. 2.
    Bagirov, A.M., Ugon, J.: Piecewise partially separable functions and a derivative-free method for large-scale nonsmooth optimization. J. Glob. Optim. 35, 163–195 (2006) MATHCrossRefMathSciNetGoogle Scholar
  3. 3.
    Bagirov, A.M., Ghosh, M., Webb, D.: A derivative-free method for linearly constrained nonsmooth optimization. J. Ind. Manag. Optim. 2(3), 319–338 (2006) MATHMathSciNetGoogle Scholar
  4. 4.
    Bagirov, A.M., Karasozen, B., Sezer, M.: Discrete gradient method: a derivative free method for nonsmooth optimization. J. Optim. Theory Appl. (2008). DOI:  10.1007/s10957-007-9335-5 MathSciNetGoogle Scholar
  5. 5.
    Bartels, S.G., Kuntz, L., Sholtes, S.: Continuous selections of linear functions and nonsmooth critical point theory. Nonlinear Anal. TMA 24, 385–407 (1995) MATHCrossRefGoogle Scholar
  6. 6.
    Breiman, L., Friedman, J.H., Olshen, R.H., Stone, C.J.: Classification and Regression Trees. Wadsworth, Belmont (1984) MATHGoogle Scholar
  7. 7.
    Clarke, F.H.: Generalized gradients and applications. Trans. Am. Math. Soc. 205, 247–262 (1975) MATHCrossRefGoogle Scholar
  8. 8.
    Clarke, F.H.: Optimization and Nonsmooth Analysis. Wiley, New York (1983) MATHGoogle Scholar
  9. 9.
    Demyanov, V.F., Rubinov, A.M.: Constructive Nonsmooth Analysis. Peter Lang, Frankfurt am Main (1995) MATHGoogle Scholar
  10. 10.
    Friedman, J.H.: Multivariate adaptive regression splines (with discussion). Ann. Stat. 19, 1–141 (1991) MATHCrossRefGoogle Scholar
  11. 11.
    Gorokhovik, V.V., Zorko, O.I., Birkhoff, G.: Piecewise affine functions and polyhedral sets. Optimization 31(3), 209–221 (1994) MATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Györfi, L., Kohler, M., Krzyżak, A., Walk, H.: A Distribution-Free Theory of Nonparametric Regression. Springer Series in Statistics. Springer, Heldelberg (2002) MATHGoogle Scholar
  13. 13.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning. Springer, New York (2001) MATHGoogle Scholar
  14. 14.
    Mifflin, R.: Semismooth and semiconvex functions in constrained optimization. SIAM J. Control Optim. 15, 957–972 (1977) CrossRefMathSciNetGoogle Scholar
  15. 15.
    The R Project for Statistical Computing. Available on:
  16. 16.
    Wolfe, P.H.: Finding the nearest point in a polytope. Math. Program. 11, 128–149 (1976) MATHCrossRefMathSciNetGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  1. 1.School of Information Technology and Mathematical SciencesUniversity of BallaratBallaratAustralia
  2. 2.Department of MathematicsUniversität des SaarlandesSaarbrückenGermany
  3. 3.Department of MathematicsTechnische Universität DarmstadtDarmstadtGermany

Personalised recommendations