On the Least-Squares Fitting of Data by Sinusoids

  • Yaroslav D. SergeyevEmail author
  • Dmitri E. Kvasov
  • Marat S. Mukhametzhanov
Part of the Springer Optimization and Its Applications book series (SOIA, volume 107)


The sinusoidal parameter estimation problem is considered to fit a sum of damped sinusoids to a series of noisy observations. It is formulated as a nonlinear least-squares global optimization problem. A one-parametric case study is examined to determine an unknown frequency of a signal. Univariate Lipschitz-based deterministic methods are used for solving such problems within a limited computational budget. It is shown that the usage of local information in these methods (such as local tuning on the objective function behavior and/or evaluating the function first derivatives) can significantly accelerate the search for the problem solution with a required guarantee. Results of a numerical comparison with metaheuristic techniques frequently used in engineering design are also reported and commented on.


Nonlinear regression Least-squares fitting Lipschitz-based deterministic methods Metaheuristics Numerical comparison 



This work was supported by the Russian Science Foundation, project number 15-11-30022 “Global optimization, supercomputing computations, and applications.”


  1. 1.
    Barkalov, K., Polovinkin, A., Meyerov, I., Sidorov, S., Zolotykh, N.: SVM regression parameters optimization using parallel global search algorithm. In: Parallel Computing Technologies. LNCS, vol. 7979, pp. 154–166. Springer, Heidelberg (2013)Google Scholar
  2. 2.
    Bloomfield, P.: Fourier Analysis of Time Series: An Introduction. Wiley, New York (2000)CrossRefzbMATHGoogle Scholar
  3. 3.
    Broomhead, D.S., King, G.P.: Extracting qualitative dynamics from experimental data. Phys. D Nonlinear Phenom. 20 (2–3), 217–236 (1986)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Calvin, J.M., Žilinskas, A.: One-dimensional global optimization for observations with noise. Comput. Math. Appl. 50 (1–2), 157–169 (2005)MathSciNetCrossRefzbMATHGoogle Scholar
  5. 5.
    Carnì, D.L., Fedele, G.: Multi-sine fitting algorithm enhancement for sinusoidal signal characterization. Comput. Stand. Interfaces 34 (6), 535–540 (2012)CrossRefGoogle Scholar
  6. 6.
    Costanzo, S.: Synthesis of multi-step coplanar waveguide-to-microstrip transition. Prog. Electromagn. Res. 113, 111–126 (2011)CrossRefGoogle Scholar
  7. 7.
    Elsner, J.B., Tsonis, A.A.: Singular Spectrum Analysis: A New Tool in Time Series Analysis. Springer, New York (1996)CrossRefzbMATHGoogle Scholar
  8. 8.
    Evtushenko, Y.G.: Numerical Optimization Techniques. Translations Series in Mathematics and Engineering. Springer, Berlin (1985)CrossRefGoogle Scholar
  9. 9.
    Fedele, G., Ferrise, A.: A frequency-locked-loop filter for biased multi-sinusoidal estimation. IEEE Trans. Signal Process. 62 (5), 1125–1134 (2014)MathSciNetCrossRefGoogle Scholar
  10. 10.
    Garnier, H., Wang, L. (eds.): Identification of Continuous-Time Models from Sampled Data. Springer, London (2008)Google Scholar
  11. 11.
    Gergel, V.P., Sergeyev, Y.D.: Sequential and parallel algorithms for global minimizing functions with Lipschitzian derivatives. Comput. Math. Appl. 37 (4–5), 163–179 (1999)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Gergel, V.P., Grishagin, V.A., Gergel, A.V.: Adaptive nested optimization scheme for multidimensional global search. J. Glob. Optim. (2015, to appear). doi  10.1007/s10898-015-0355-7
  13. 13.
    Gergel, V.P., Grishagin, V.A., Israfilov, R.A.: Local tuning in nested scheme of global optimization. Proc. Comput. Sci. 51, 865–874 (2015). (International Conference on Computational Science ICCS 2015 – Computational Science at the Gates of Nature)Google Scholar
  14. 14.
    Gillard, J.W.: Cadzow’s basic algorithm, alternating projections and singular spectrum analysis. Stat. Interface 3 (3), 335–343 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  15. 15.
    Gillard, J.W., Kvasov, D.E.: Lipschitz optimization methods for fitting a sum of damped sinusoids to a series of observations. Stat. Interface (2016, to appear)Google Scholar
  16. 16.
    Gillard, J.W., Zhigljavsky, A.: Analysis of structured low rank approximation as an optimisation problem. Informatica 22 (4), 489–505 (2011)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Gillard, J.W., Zhigljavsky, A.: Optimization challenges in the structured low rank approximation problem. J. Glob. Optim. 57 (3), 733–751 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Gillard, J.W., Zhigljavsky, A.: Stochastic algorithms for solving structured low-rank matrix approximation problems. Commun. Nonlinear Sci. Numer. Simul. 21, 70–88 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Golyandina, N., Nekrutkin, V., Zhigljavsky, A.: Analysis of Time Series Structure: SSA and Related Techniques. Chapman & Hall/CRC, Boca Raton (2001)CrossRefzbMATHGoogle Scholar
  20. 20.
    Grishagin, V.A., Strongin, R.G.: Optimization of multi-extremal functions subject to monotonically unimodal constraints. Eng. Cybern. 22 (5), 117–122 (1984)MathSciNetzbMATHGoogle Scholar
  21. 21.
    Grishagin, V.A., Sergeyev, Y.D., Strongin, R.G.: Parallel characteristic algorithms for solving problems of global optimization. J. Glob. Optim. 10 (2), 185–206 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Holmström, K., Petersson, J.: A review of the parameter estimation problem of fitting positive exponential sums to empirical data. Appl. Math. Comput. 126 (1), 31–61 (2002)MathSciNetzbMATHGoogle Scholar
  23. 23.
    Kvasov, D.E.: Diagonal numerical methods for solving Lipschitz global optimization problems. Boll. Unione Mat. Ital. I (Serie IX) (3), 857–871 (2008)Google Scholar
  24. 24.
    Kvasov, D.E., Mukhametzhanov, M.S.: One-dimensional global search: nature-inspired vs. Lipschitz methods. In: Proceedings of the ICNAAM2015 Conference, AIP Conference Proceedings. AIP Publishing LLC, New York (2015).Google Scholar
  25. 25.
    Kvasov, D.E., Sergeyev, Y.D.: Univariate geometric Lipschitz global optimization algorithms. Numer. Algebra Control Optim. 2 (1), 69–90 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Kvasov, D.E., Sergeyev, Y.D.: Lipschitz global optimization methods in control problems. Autom. Remote Control 74 (9), 1435–1448 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  27. 27.
    Kvasov, D.E., Sergeyev, Y.D.: Deterministic approaches for solving practical black-box global optimization problems. Adv. Eng. Softw. 80, 58–66 (2015)CrossRefGoogle Scholar
  28. 28.
    Kvasov, D.E., Mukhametzhanov, M.S., Sergeyev, Y.D.: Solving univariate global optimization problems by nature-inspired and deterministic algorithms. Adv. Eng. Softw. (2015, submitted)Google Scholar
  29. 29.
    Lemmerling, P., Van Huffel, S.: Analysis of the structured total least squares problem for Hankel∕Toeplitz matrices. Numer. Algorithms 27 (1), 89–114 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Lera, D., Sergeyev, Y.D.: Acceleration of univariate global optimization algorithms working with Lipschitz functions and Lipschitz first derivatives. SIAM J. Optim. 23 (1), 508–529 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Li, Y., Liu, K., Razavilar, J.: A parameter estimation scheme for damped sinusoidal signals based on low-rank Hankel approximation. IEEE Trans. Signal Process. 45 (2), 481–486 (1997)CrossRefGoogle Scholar
  32. 32.
    Liuzzi, G., Lucidi, S., Piccialli, V.: A partition-based global optimization algorithm. J. Glob. Optim. 48 (1), 113–128 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  33. 33.
    Markovsky, I.: Bibliography on total least squares and related methods. Stat. Interface 3 (3), 329–334 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Mockus, J.: Bayesian Approach to Global Optimization. Kluwer Academic, Dordrecht (1989)CrossRefzbMATHGoogle Scholar
  35. 35.
    Paulavičius, R., Žilinskas, J.: Simplicial Global Optimization. SpringerBriefs in Optimization. Springer, New York (2014)CrossRefzbMATHGoogle Scholar
  36. 36.
    Paulavičius, R., Žilinskas, J.: Simplicial Lipschitz optimization without the Lipschitz constant. J. Glob. Optim. 59 (1), 23–40 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Paulavičius, R., Sergeyev, Y.D., Kvasov, D.E., Žilinskas, J.: Globally-biased Disimpl algorithm for expensive global optimization. J. Glob. Optim. 59 (2–3), 545–567 (2014)MathSciNetCrossRefzbMATHGoogle Scholar
  38. 38.
    Pintér, J.D.: Global Optimization in Action. Continuous and Lipschitz Optimization: Algorithms, Implementations and Applications. Kluwer Academic, Dordrecht (1996)CrossRefzbMATHGoogle Scholar
  39. 39.
    Pollock, D.: A Handbook of Time Series Analysis, Signal Processing, and Dynamics. Academic, London (1999)zbMATHGoogle Scholar
  40. 40.
    Sergeyev, Y.D.: An information global optimization algorithm with local tuning. SIAM J. Optim. 5 (4), 858–870 (1995)MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Sergeyev, Y.D.: A one-dimensional deterministic global minimization algorithm. Comput. Math. Math. Phys. 35 (5), 705–717 (1995)MathSciNetGoogle Scholar
  42. 42.
    Sergeyev, Y.D.: Global one-dimensional optimization using smooth auxiliary functions. Math. Program. 81 (1), 127–146 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Sergeyev, Y.D.: On convergence of “Divide the Best” global optimization algorithms. Optimization 44 (3), 303–325 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Sergeyev, Y.D.: Multidimensional global optimization using the first derivatives. Comput. Math. Math. Phys. 39 (5), 711–720 (1999)MathSciNetzbMATHGoogle Scholar
  45. 45.
    Sergeyev, Y.D., Grishagin, V.A.: A parallel method for finding the global minimum of univariate functions. J. Optim. Theory Appl. 80 (3), 513–536 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Sergeyev, Y.D., Kvasov, D.E.: Diagonal Global Optimization Methods. FizMatLit, Moscow (2008) [in Russian]zbMATHGoogle Scholar
  47. 47.
    Sergeyev, Y.D., Kvasov, D.E.: Lipschitz global optimization. In: Cochran, J.J. (ed.) Wiley Encyclopedia of Operations Research and Management Science, vol. 4, pp. 2812–2828. Wiley, New York (2011)Google Scholar
  48. 48.
    Sergeyev, Y.D., Khalaf, F.M.H., Kvasov, D.E.: Univariate algorithms for solving global optimization problems with multiextremal non-differentiable constraints. In: A. Törn, J. Žilinskas (eds.) Models and Algorithms for Global Optimization, pp. 123–140. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  49. 49.
    Sergeyev, Y.D., Strongin, R.G., Lera, D.: Introduction to Global Optimization Exploiting Space-Filling Curves. SpringerBriefs in Optimization. Springer, New York (2013)CrossRefzbMATHGoogle Scholar
  50. 50.
    Sergeyev, Y.D., Mukhametzhanov, M.S., Kvasov, D.E., Lera, D.: Derivative-free local tuning and local improvement techniques embedded in the univariate global optimization. J. Optim. Theory Appl. (2016, to appear)Google Scholar
  51. 51.
    Strekalovsky, A.S.: Elements of Nonconvex Optimization. Nauka, Novosibirsk (2003) [in Russian]Google Scholar
  52. 52.
    Strongin, R.G., Sergeyev, Y.D.: Global Optimization with Non-Convex Constraints: Sequential and Parallel Algorithms. Kluwer Academic, Dordrecht (2000). 3rd edn. by Springer, Berlin (2014)Google Scholar
  53. 53.
    Törn, A., Žilinskas, A.: Global Optimization. Lecture Notes in Computer Science, vol. 350. Springer, Berlin (1989)Google Scholar
  54. 54.
    Van Huffel, S., Vandewalle, J.: The Total Least Squares Problem: Computational Aspects and Analysis. SIAM, Philadelphia (1991)CrossRefzbMATHGoogle Scholar
  55. 55.
    Zhigljavsky, A., Žilinskas, A.: Stochastic Global Optimization. Springer, New York (2008)zbMATHGoogle Scholar
  56. 56.
    Žilinskas, A.: Global Optimization. Axiomatics of Statistical Models, Algorithms, and Applications. Mokslas, Vilnius (1986) [in Russian]zbMATHGoogle Scholar
  57. 57.
    Žilinskas, A.: On similarities between two models of global optimization: statistical models and radial basis functions. J. Glob. Optim. 48 (1), 173–182 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  58. 58.
    Žilinskas, A., Žilinskas, J.: Global optimization based on a statistical model and simplicial partitioning. Comput. Math. Appl. 44 (7), 957–967 (2002)MathSciNetCrossRefzbMATHGoogle Scholar
  59. 59.
    Žilinskas, A., Žilinskas, J.: Interval arithmetic based optimization in nonlinear regression. Informatica 21 (1), 149–158 (2010)MathSciNetzbMATHGoogle Scholar
  60. 60.
    Žilinskas, A., Žilinskas, J.: A hybrid global optimization algorithm for non-linear least squares regression. J. Glob. Optim. 56 (2), 265–277 (2013)MathSciNetCrossRefzbMATHGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Yaroslav D. Sergeyev
    • 1
    • 2
    Email author
  • Dmitri E. Kvasov
    • 1
    • 2
  • Marat S. Mukhametzhanov
    • 1
    • 2
  1. 1.Dipartimento di Ingegneria Informatica, Modellistica, Elettronica e SistemisticaUniversità della CalabriaRendeItaly
  2. 2.Department of Software and Supercomputing TechnologiesLobachevsky State University of Nizhni NovgorodNizhny NovgorodRussia

Personalised recommendations