Skip to main content
SpringerLink
Log in
Book cover

Encyclopedia of Systems and Control pp 1–12Cite as

System Identification Techniques: Convexification, Regularization, Relaxation

  • Living reference work entry
  • Latest version View entry history
  • First Online:

  • 234 Accesses

Abstract

System identification has been developed, by and large, following the classical parametric approach. In this entry we discuss how regularization theory can be employed to tackle the system identification problem from a nonparametric (or semi-parametric) point of view. Both regularization for smoothness and regularization for sparseness are discussed, as flexible means to face the bias/variance dilemma and to perform model selection. These techniques have also advantages from the computational point of view, leading sometimes to convex optimization problems.

This is a preview of subscription content, log in via an institution.

References

  • Aravkin A, Burke J, Chiuso A, Pillonetto G (2014) Convex vs non-convex estimators for regression and sparse estimation: the mean squared error properties of ARD and GLASSO. J Mach Learn Res 15:217–252

    MathSciNet  MATH  Google Scholar 

  • Bach F, Lanckriet G, Jordan M (2004) Multiple kernel learning, conic duality, and the SMO algorithm. In: Proceedings of the 21st international conference on machine learning, Banff, pp 41–48

    Google Scholar 

  • Banbura M, Giannone D, Reichlin L (2010) Large Bayesian VARs. J Appl Econom 25:71–92

    Article  Google Scholar 

  • Chen T, Ohlsson H, Ljung L (2012) On the estimation of transfer functions, regularizations and Gaussian processes – revisited. Automatica 48:1525–1535

    Article  MathSciNet  Google Scholar 

  • Chiuso A (2016) Regularization and Bayesian learning in dynamical systems: past, present and future. Annu Rev Control 41:24–38

    Article  Google Scholar 

  • Chiuso A, Pillonetto G (2012) A Bayesian approach to sparse dynamic network identification. Automatica 48:1553–1565

    Article  MathSciNet  Google Scholar 

  • Daniel M, Robert JP, Thomas S, Claudio M, Dario F, Gustavo S (2010) Revealing strengths and weaknesses of methods for gene network inference. Proc Natl Acad Sci 107:6286–6291

    Article  Google Scholar 

  • Dankers A, Van den Hof PMJ, Bombois X, Heuberger PSC (2016) Identification of dynamic models in complex networks with prediction error methods: predictor input selection. IEEE Trans Autom Control 61:937–952

    Article  MathSciNet  Google Scholar 

  • Doan T, Litterman R, Sims C (1984) Forecasting and conditional projection using realistic prior distributions. Econom Rev 3:1–100

    Article  Google Scholar 

  • Donoho D (2006) Compressed sensing. IEEE Trans Inf Theory 52:1289–1306

    Article  MathSciNet  Google Scholar 

  • Fan J, Li R (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 96:1348–1360

    Article  MathSciNet  Google Scholar 

  • Fazel M, Hindi H, Boyd S (2001) A rank minimization heuristic with application to minimum order system approximation. In: Proceedings of the 2001 American control conference, Arlington, vol 6, pp 4734–4739

    Google Scholar 

  • Formentin S, Chiuso A (2018) CoRe: control-oriented regularization for system identification. In: 2018 IEEE conference on decision and control (CDC), pp 2253–2258

    Google Scholar 

  • Hayden D, Chang YH, Goncalves J, Tomlin CJ (2016) Sparse network identifiability via compressed sensing. Automatica 68:9–17

    Article  MathSciNet  Google Scholar 

  • Hocking RR (1976) A biometrics invited paper. The analysis and selection of variables in linear regression. Biometrics 32:1–49

    Google Scholar 

  • Kitagawa G, Gersh H (1984) A smoothness priors-state space modeling of time series with trends and seasonalities. J Am Stat Assoc 79:378–389

    Google Scholar 

  • Leeb H, Pötscher B (2005) Model selection and inference: facts and fiction. Econom Theory 21:21–59

    Article  MathSciNet  Google Scholar 

  • Ljung L (1999) System identification – theory for the user. Prentice Hall, Upper Saddle River

    MATH  Google Scholar 

  • Mackay D (1994) Bayesian non-linear modelling for the prediction competition. ASHRAE Trans 100:3704–3716

    Google Scholar 

  • Ohlsson H, Ljung L (2013) Identification of switched linear regression models using sum-of-norms regularization. Automatica 49:1045–1050

    Article  MathSciNet  Google Scholar 

  • Ozay N, Sznaier M, Lagoa C, Camps O (2012) A sparsification approach to set membership identification of switched affine systems. IEEE Trans Autom Control 57:634–648

    Article  MathSciNet  Google Scholar 

  • Pillonetto G, Chiuso A (2015) Tuning complexity in regularized kernel-based regression and linear system identification: the robustness of the marginal likelihood estimator. Automatica 58:106–117

    Article  MathSciNet  Google Scholar 

  • Pillonetto G, De Nicolao G (2010) A new kernel-based approach for linear system identification. Automatica 46:81–93

    Article  MathSciNet  Google Scholar 

  • Pillonetto G, Chiuso A, De Nicolao G (2011) Prediction error identification of linear systems: a nonparametric Gaussian regression approach. Automatica 47:291–305

    Article  MathSciNet  Google Scholar 

  • Pillonetto G, Chen T, Chiuso A, Nicolao GD, Ljung L (2016) Regularized linear system identification using atomic, nuclear and kernel-based norms: the role of the stability constraint. Automatica 69:137–149

    Article  MathSciNet  Google Scholar 

  • Prando G, Chiuso A, Pillonetto G (2017a) Maximum entropy vector kernels for MIMO system identification. Automatica 79:326–339

    Article  MathSciNet  Google Scholar 

  • Prando G, Zorzi M, Bertoldo A, Chiuso A (2017b) Estimating effective connectivity in linear brain network models. In: 2017 IEEE 56th annual conference on decision and control (CDC), pp 5931–5936

    Google Scholar 

  • Rasmussen C, Williams C (2006) Gaussian processes for machine learning. MIT, Cambridge

    MATH  Google Scholar 

  • Razi A, Seghier ML, Zhou Y, McColgan P, Zeidman P, Park H-J, Sporns O, Rees G, Friston KJ (2017) Large-scale DCMs for resting-state fMRI. Netw Neurosci 1:222–241

    Article  Google Scholar 

  • Romeres D, Zorzi M, Camoriano R, Traversaro S, Chiuso A (2019, in press) Derivative-free online learning of inverse dynamics models. IEEE Trans Control Syst Technol

    Google Scholar 

  • Tibshirani R (1996) Regression shrinkage and selection via the LASSO. J R Stat Soc Ser B 58:267–288

    MathSciNet  MATH  Google Scholar 

  • Tipping M (2001) Sparse Bayesian learning and the relevance vector machine. J Mach Learn Res 1:211–244

    MathSciNet  MATH  Google Scholar 

  • Wang H, Li G, Tsai C (2007) Regression coefficient and autoregressive order shrinkage and selection via the LASSO. J R Stat Soc Ser B 69:63–78

    MathSciNet  Google Scholar 

  • Wipf D, Rao B, Nagarajan S (2011) Latent variable Bayesian models for promoting sparsity. IEEE Trans Inf Theory 57:6236–6255

    Article  MathSciNet  Google Scholar 

  • Yuan M, Lin Y (2006) Model selection and estimation in regression with grouped variables. J R Stat Soc Ser B 68:49–67

    Article  MathSciNet  Google Scholar 

  • Zorzi M, Chiuso A (2017) Sparse plus low rank network identification: a nonparametric approach. Automatica 76:355–366

    Article  MathSciNet  Google Scholar 

  • Zorzi M, Chiuso A (2018) The harmonic analysis of kernel functions. Automatica 94:125–137

    Article  MathSciNet  Google Scholar 

  • Zou H (2006) The adaptive Lasso and it oracle properties. J Am Stat Assoc 101:1418–1429

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Information Engineering, University of Padova, Padova, Italy

    Alessandro Chiuso

Authors
  1. Alessandro Chiuso
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alessandro Chiuso .

Editor information

Editors and Affiliations

  1. Electrical and Computer Engineering, Boston University, Boston, MA, USA

    John Baillieul

  2. Automation and Control Solutions, Honeywell, Golden Valley, MN, USA

    Tariq Samad

Section Editor information

  1. Division of Automatic Control, Department of Electrical Engineering, Linköping University, Linköping, Sweden

    Lennart Ljung

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer-Verlag London Ltd., part of Springer Nature

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Chiuso, A. (2019). System Identification Techniques: Convexification, Regularization, Relaxation. In: Baillieul, J., Samad, T. (eds) Encyclopedia of Systems and Control. Springer, London. https://doi.org/10.1007/978-1-4471-5102-9_101-3

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-5102-9_101-3

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-5102-9

  • Online ISBN: 978-1-4471-5102-9

  • eBook Packages: Springer Reference EngineeringReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Chapter history

  1. Latest

    System Identification Techniques: Convexification, Regularization, Relaxation
    Published:
    11 October 2019

    DOI: https://doi.org/10.1007/978-1-4471-5102-9_101-3

  2. Original

    System Identification Techniques: Convexification, Regularization, and Relaxation
    Published:
    25 March 2014

    DOI: https://doi.org/10.1007/978-1-4471-5102-9_101-1

Search

Navigation