Skip to main content
SpringerLink
Log in
Book cover

Gene Regulatory Networks pp 385–422Cite as

Scalable Inference of Ordinary Differential Equation Models of Biochemical Processes

  • Protocol
  • First Online:

Part of the book series: Methods in Molecular Biology ((MIMB,volume 1883))

Abstract

Ordinary differential equation models have become a standard tool for the mechanistic description of biochemical processes. If parameters are inferred from experimental data, such mechanistic models can provide accurate predictions about the behavior of latent variables or the process under new experimental conditions. Complementarily, inference of model structure can be used to identify the most plausible model structure from a set of candidates, and, thus, gain novel biological insight. Several toolboxes can infer model parameters and structure for small- to medium-scale mechanistic models out of the box. However, models for highly multiplexed datasets can require hundreds to thousands of state variables and parameters. For the analysis of such large-scale models, most algorithms require intractably high computation times. This chapter provides an overview of the state-of-the-art methods for parameter and model inference, with an emphasis on scalability.

This is a preview of subscription content, log in via an institution.

Protocol
USD   49.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD   199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Springer Nature is developing a new tool to find and evaluate Protocols. Learn more

References

  1. Klipp E, Herwig R, Kowald A, Wierling C, Lehrach H (2005) Systems biology in practice. Wiley-VCH, Weinheim

    Book  Google Scholar 

  2. Kitano H (2002) Systems biology: a brief overview. Science 295(5560):1662–1664

    Article  CAS  PubMed  Google Scholar 

  3. Kitano H (2002) Computational systems biology. Nature 420(6912):206–210

    Article  CAS  PubMed  Google Scholar 

  4. Adlung L, Kar S, Wagner MC, She B, Chakraborty S, Bao J, Lattermann S, Boerries M, Busch H, Wuchter P, Ho AD, Timmer J, Schilling M, Höfer T, Klingmüller U (2017) Protein abundance of AKT and ERK pathway components governs cell type-specific regulation of proliferation. Mol Syst Biol 13(1):904

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  5. Buchholz VR, Flossdorf M, Hensel I, Kretschmer L, Weissbrich B, Gräf P, Verschoor A, Schiemann M, Höfer T, Busch DH (2013) Disparate individual fates compose robust CD8+ t cell immunity. Science 340(6132):630–635

    Article  CAS  PubMed  Google Scholar 

  6. Intosalmi J, Nousiainen K, Ahlfors H, Läähdesmäki H (2016) Data-driven mechanistic analysis method to reveal dynamically evolving regulatory networks. Bioinformatics 32(12):i288–i296

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  7. Hug S, Schwarzfischer M, Hasenauer J, Marr C, Theis FJ (2016) An adaptive scheduling scheme for calculating Bayes factors with thermodynamic integration using Simpson’s rule. Stat Comput 26(3):663–677

    Article  Google Scholar 

  8. Hross S, Fiedler A, Theis FJ, Hasenauer J (2016) Quantitative comparison of competing PDE models for Pom1p dynamics in fission yeast. In: Findeisen R, Bullinger E, Balsa-Canto E, Bernaerts K (eds) Proc. 6th IFAC conf. found. syst. biol. eng., IFAC-PapersOnLine, vol 49, pp 264–269

    Article  Google Scholar 

  9. Toni T, Ozaki Yi, Kirk P, Kuroda S, Stumpf MPH (2012) Elucidating the in vivo phosphorylation dynamics of the ERK MAP kinase using quantitative proteomics data and Bayesian model selection. Mol Biosyst 8:1921–1929

    Article  CAS  PubMed  Google Scholar 

  10. Molinelli EJ, Korkut A, Wang W, Miller ML, Gauthier NP, Jing X, Kaushik P, He Q, Mills G, Solit DB, Pratilas CA, Weigt M, Braunstein A, Pagnani A, Zecchina R, Sander C (2013) Perturbation biology: inferring signaling networks in cellular systems. PLoS Comput Biol 9(12):e1003290

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  11. Schilling M, Maiwald T, Hengl S, Winter D, Kreutz C, Kolch W, Lehmann WD, Timmer J, Klingmüller U (2009) Theoretical and experimental analysis links isoform-specific ERK signalling to cell fate decisions. Mol Syst Biol 5:334

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  12. Fey D, Halasz M, Dreidax D, Kennedy SP, Hastings JF, Rauch N, Munoz AG, Pilkington R, Fischer M, Westermann F, Kolch W, Kholodenko BN, Croucher DR (2015) Signaling pathway models as biomarkers: patient-specific simulations of JNK activity predict the survival of neuroblastoma patients. Sci Signal 8(408):ra130

    Article  PubMed  CAS  Google Scholar 

  13. Eduati F, Doldàn-Martelli V, Klinger B, Cokelaer T, Sieber A, Kogera F, Dorel M, Garnett MJ, Blüthgen N, Saez-Rodriguez J (2017) Drug resistance mechanisms in colorectal cancer dissected with cell Type–Specific dynamic logic models. Cancer Res 77(12):3364–3375

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Hass H, Masson K, Wohlgemuth S, Paragas V, Allen JE, Sevecka M, Pace E, Timmer J, Stelling J, MacBeath G, Schoeberl B, Raue A (2017) Predicting ligand-dependent tumors from multi-dimensional signaling features. NPJ Syst Biol Appl 3(1):27

    Article  PubMed  PubMed Central  Google Scholar 

  15. Maiwald T, Hass H, Steiert B, Vanlier J, Engesser R, Raue A, Kipkeew F, Bock HH, Kaschek D, Kreutz C, Timmer J (2016) Driving the model to its limit: Profile likelihood based model reduction. PLoS One 11(9):e0162366

    Article  PubMed  PubMed Central  Google Scholar 

  16. Snowden TJ, van der Graaf PH, Tindall MJ (2017) Methods of model reduction for large-scale biological systems: a surveyof current methods and trends. B Math Biol 79(7):1449–1486

    Article  CAS  Google Scholar 

  17. Transtrum MK, Qiu P (2016) Bridging mechanistic and phenomenological models of complex biological systems. PLoS Comput Biol 12(5):1–34

    Article  CAS  Google Scholar 

  18. Dano S, Madsen MF, Schmidt H, Cedersund G (2006) Reduction of a biochemical model with preservation of its basic dynamic properties. FEBS J 273(21):4862–4877

    Article  PubMed  CAS  Google Scholar 

  19. Klonowski W (1983) Simplifying principles for chemical and enzyme reaction kinetics. Biophys Chem 18(2):73–87

    Article  CAS  PubMed  Google Scholar 

  20. Hoops S, Sahle S, Gauges R, Lee C, Pahle J, Simus N, Singhal M, Xu L, Mendes P, Kummer U (2006) COPASI – a COmplex PAthway SImulator. Bioinformatics 22(24):3067–3074

    Article  CAS  PubMed  Google Scholar 

  21. Raue A, Steiert B, Schelker M, Kreutz C, Maiwald T, Hass H, Vanlier J, Tönsing C, Adlung L, Engesser R, Mader W, Heinemann T, Hasenauer J, Schilling M, Höfer T, Klipp E, Theis FJ, Klingmüller U, Schöberl B, JTimmer (2015) Data2Dynamics: a modeling environment tailored to parameter estimation in dynamical systems. Bioinformatics 31(21):3558–3560

    Article  CAS  PubMed  Google Scholar 

  22. Somogyi ET, Bouteiller JM, Glazier JA, König M, Medley JK, Swat MH, Sauro HM (2015) libRoadRunner: a high performance SBML simulation and analysis library. Bioinformatics 31(20):3315–3321

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Santos SDM, Verveer PJ, Bastiaens PIH (2007) Growth factor-induced MAPK network topology shapes Erk response determining PC-12 cell fate. Nat Cell Biol 9(3):324–330

    Article  CAS  PubMed  Google Scholar 

  24. Yao J, Pilko A, Wollman R (2016) Distinct cellular states determine calcium signaling response. Mol Syst Biol 12(12):894

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  25. Ogilvie LA, Kovachev A, Wierling C, Lange BMH, Lehrach H (2017) Models of models: a translational route for cancer treatment and drug development. Front Oncol 7: 219

    Article  PubMed  PubMed Central  Google Scholar 

  26. Schillings C, Sunnåker M, Stelling J, Schwab C (2015) Efficient characterization of parametric uncertainty of complex (bio)chemical networks. PLoS Comput Biol 11(8):e1004457

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  27. Babtie AC, Stumpf MPH (2017) How to deal with parameters for whole-cell modelling. J R Soc Interface 14(133):20130505

    Article  Google Scholar 

  28. Ocone A, Haghverdi L, Mueller NS, Theis FJ (2015) Reconstructing gene regulatory dynamics from high-dimensional single-cell snapshot data. Bioinformatics 31(12):i89–i96

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Kondofersky I, Fuchs C, Theis FJ (2015) Identifying latent dynamic components in biological systems. IET Syst Biol 9(5): 193–203

    Article  PubMed  PubMed Central  Google Scholar 

  30. Akaike H (1973) Information theory and an extension of the maximum likelihood principle. In: 2nd international symposium on information theory, Tsahkadsor, Armenian SSR, Akademiai Kiado, vol 1, pp 267–281

    Google Scholar 

  31. Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464

    Article  Google Scholar 

  32. Steiert B, Timmer J, Kreutz C (2016) L1 regularization facilitates detection of cell type-specific parameters in dynamical systems. Bioinformatics 32(17):i718–i726

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Klimovskaia A, Ganscha S, Claassen M (2016) Sparse regression based structure learning of stochastic reaction networks from single cell snapshot time series. PLoS Comput Biol 12(12):e1005234

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  34. Loos C, Moeller K, Fröhlich F, Hucho T, Hasenauer J (2018) A hierarchical, data-driven approach to modeling single-cell populations predicts latent causes of cell-to-cell variablity. Cell Syst 6(5):593–603

    Article  CAS  PubMed  Google Scholar 

  35. Hock S, Hasenauer J, Theis FJ (2013) Modeling of 2D diffusion processes based on microscopy data: Parameter estimation and practical identifiability analysis. BMC Bioinf 14(Suppl 10):S7

    Article  Google Scholar 

  36. Hross S (2016) Parameter estimation and uncertainty quantification for image based systems biology. Ph.D. thesis, Technische Universität München

    Google Scholar 

  37. Menshykau D, Germann P, Lemereux L, Iber D (2013) Simulating organogenesis in COMSOL: Parameter optimization for PDE-based models. In: Proceedings of COMSOL conference, Rotterdam

    Google Scholar 

  38. Hross S, Hasenauer J (2016) Analysis of CFSE time-series data using division-, age- and label-structured population models. Bioinformatics 32(15):2321–2329

    Article  CAS  PubMed  Google Scholar 

  39. Fröhlich F, Thomas P, Kazeroonian A, Theis FJ, Grima R, Hasenauer J (2016) Inference for stochastic chemical kinetics using moment equations and system size expansion. PLoS Comput Biol 12(7):e1005030

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  40. Ruess J, Lygeros J (2015) Moment-based methods for parameter inference and experiment design for stochastic biochemical reaction networks. ACM T Math Softwares Model Comput S 25(2):8

    Google Scholar 

  41. Munsky B, Trinh B, Khammash M (2009) Listening to the noise: random fluctuations reveal gene network parameters. Mol Syst Biol 5:318

    Article  PubMed  PubMed Central  Google Scholar 

  42. Kreutz C, Bartolome Rodriguez MM, Maiwald T, Seidl M, Blum HE, Mohr L, Timmer J (2007) An error model for protein quantification. Bioinformatics 23(20):2747–2753

    Article  CAS  PubMed  Google Scholar 

  43. Maier C, Loos C, Hasenauer J (2017) Robust parameter estimation for dynamical systems from outlier-corrupted data. Bioinformatics 33(5):718–725

    CAS  PubMed  Google Scholar 

  44. Puga JL, Krzywinski M, Altman N (2015) Bayes’ theorem. Nat Methods 12(3):277–278

    Article  CAS  Google Scholar 

  45. Dauphin YN, Pascanu R, Gulcehre C, Cho K (2014) Identifying and attacking the saddle point problem in high-dimensional non-convex optimization. In: Advances in neural information processing systems 26 (NIPS 2014), pp 2933–2941

    Google Scholar 

  46. Amestoy PR, Davis TA, Duff IS (1996) An approximate minimum degree ordering algorithm. SIAM J Matrix Anal A 17(4):886–905

    Article  Google Scholar 

  47. Anandkumar A, Ge R (2016) Efficient approaches for escaping higher order saddle points in non-convex optimization. In: Conference on learning theory, pp 81–102

    Google Scholar 

  48. Kirk P, Rolando DM, MacLean AL, Stumpf MP (2015) Conditional random matrix ensembles and the stability of dynamical systems. New J Phys 17(8):083025

    Article  Google Scholar 

  49. Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. IEEE Trans Evol Comput 1(1):67–82

    Article  Google Scholar 

  50. Goffe WL, Ferrier GD, Rogers J (1994) Global optimization of statistical functions with simulated annealing. J Econom 60(1–2):65–99

    Article  Google Scholar 

  51. Neumaier A (2004) Complete search in continuous global optimization and constraint satisfaction. Acta Numer 13:271–369

    Article  Google Scholar 

  52. Johnson DS, McGeoch LA (1997) The traveling salesman problem: a case study in local optimization. In: Local search in combinatorial optimization, vol 1, pp 215–310

    Google Scholar 

  53. Ashyraliyev M, Fomekong-Nanfack Y, Kaandorp JA, Blom JG (2009) Systems biology: parameter estimation for biochemical models. FEBS J 276(4):886–902

    Article  CAS  PubMed  Google Scholar 

  54. Hooke R, Jeeves TA (1961) “Direct Search” solution of numerical and statistical problems. J ACM 8(2):212–229

    Article  Google Scholar 

  55. Fister Jr I, Yang XS, Fister I, Brest J, Fister D (2013) A brief review of nature-inspired algorithms for optimization. Elektrotehniski Vestnik 80(3):116–122

    Google Scholar 

  56. Nocedal J, Wright S (2006) Numerical optimization. Springer Science & Business Media, Berlin/Heidelberg

    Google Scholar 

  57. Nelder JA, Mead R (1965) A simplex method for function minimization. Comput J 7(4):308–313

    Article  Google Scholar 

  58. De La Maza M, Yuret D (1994) Dynamic hill climbing. AI expert 9:26–26

    Google Scholar 

  59. Levenberg K (1944) A method for the solution of certain non-linear problems in least squares. Q Appl Math 2(2):164–168

    Article  Google Scholar 

  60. Marquardt DW (1963) An algorithm for least-squares estimation of non-linear parameters. SIAM J Appl Math 11(22):431–441

    Article  Google Scholar 

  61. Holland JH (1992) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT Press, Cambridge

    Google Scholar 

  62. Kennedy J (2011) Particle swarm optimization. In: Encyclopedia of machine learning. Springer, Boston, pp 760–766

    Google Scholar 

  63. Egea JA, Rodriguez-Fernandez M, Banga JR, Marti R (2007) Scatter search for chemical and bio-process optimization. J Global Optim 37(3):481–503

    Article  Google Scholar 

  64. Kirkpatrick S, Gelatt˜ CD Jr, Vecchi MP (1983) Optimization by simulated annealing. Science 220(4598):671–680

    Article  CAS  PubMed  Google Scholar 

  65. Kan AR, Timmer GT (1987) Stochastic global optimization methods part I: clustering methods. Math Program 39(1):27–56

    Article  Google Scholar 

  66. Lawler EL, Wood DE (1966) Branch-and-bound methods: A survey. Oper Res 14(4):699–719

    Article  Google Scholar 

  67. Törn A, Zilinskas A (1989) Global optimization. Lecture notes in computer science, vol 350. Springer-Verlag, Berlin

    Google Scholar 

  68. Rios LM, Sahinidis NV (2013) Derivative-free optimization: a review of algorithms and comparison of software implementations. J Global Optim 56(3):1247–1293

    Article  Google Scholar 

  69. Rudolph G (1994) Convergence analysis of canonical genetic algorithms. IEEE Trans Neural Netw 5(1):96–101

    Article  CAS  PubMed  Google Scholar 

  70. Moles CG, Mendes P, Banga JR (2003) Parameter estimation in biochemical pathways: a comparison of global optimization methods. Genome Res 13:2467–2474

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  71. Raue A, Schilling M, Bachmann J, Matteson A, Schelke M, Kaschek D, Hug S, Kreutz C, Harms BD, Theis FJ, Klingmüller U, Timmer J (2013) Lessons learned from quantitative dynamical modeling in systems biology. PLoS One 8(9):e74335

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  72. Banga JR (2008) Optimization in computational systems biology. BMC Syst Biol 2:47

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  73. Boender CGE, Rinnooy Kan AHG (1987) Bayesian stopping rules for multistart global optimization methods. Math Program 37(1):59–80

    Article  Google Scholar 

  74. Törn A, Ali MM, Viitanen S (1999) Stochastic global optimization: problem classes and solution techniques. J Global Optim 14(4):437–447

    Article  Google Scholar 

  75. Fröhlich F, Kaltenbacher B, Theis FJ, Hasenauer J (2017) Scalable parameter estimation for genome-scale biochemical reaction networks. PLoS Comput Biol 13(1):e1005331

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  76. Penas DR, González P, Egea JA, Banga JR, Doallo R (2015) Parallel metaheuristics in computational biology: an asynchronous cooperative enhanced scatter search method. Procedia Comput Sci 51:630–639

    Article  Google Scholar 

  77. Armijo L (1966) Minimization of functions having Lipschitz continuous first partial derivatives. Pac J Math 16(1):1–3

    Article  Google Scholar 

  78. Wolfe P (1969) Convergence conditions for ascent methods. SIAM Rev 11(2):226–235

    Article  Google Scholar 

  79. Kolda TG, Lewis RM, Torczon V (2003) Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev 45(3):385–482

    Article  Google Scholar 

  80. Lewis RM, Torczon V, Trosset MW (2000) Direct search methods: then and now. J Comput Appl Math 124(1):191–207

    Article  Google Scholar 

  81. Rosenbrock HH (1960) An automatic method for finding the greatest or least value of a function. Comput J 3(3):175–184

    Article  Google Scholar 

  82. Yuan Yx (2015) Recent advances in trust region algorithms. Math Program 151(1):249–281

    Article  Google Scholar 

  83. Hartley HO (1961) The modified Gauss-Newton method for the fitting of non-linear regression functions by least squares. Technometrics 3(2):269–280

    Article  Google Scholar 

  84. Nesterov Y, Polyak B (2006) Cubic regularization of newton method and its global performance. Math Program 108(1): 177–205

    Article  Google Scholar 

  85. Lanczos C (1950) An iteration method for the solution of the eigenvalue problem of linear differential and integral operators. United States Governm. Press Office, Los Angeles

    Book  Google Scholar 

  86. Nash SG (1984) Newton-type minimization via the Lanczos method. SIAM J Numer Anal 21(4):770–788

    Article  Google Scholar 

  87. Byrd RH, Schnabel RB, Shultz GA (1987) A trust region algorithm for nonlinearly constrained optimization. SIAM J Numer Anal 24(5):1152–1170

    Article  Google Scholar 

  88. Sorensen DC (1982) Newton’s method with a model trust region modification. SIAM J Numer Anal 19(2):409–426

    Article  Google Scholar 

  89. Wild SM, Regis RG, Shoemaker CA (2008) ORBIT: Optimization by radial basis function interpolation in trust-regions. SIAM J Sci Comput 30(6):3197–3219

    Article  Google Scholar 

  90. Steihaug T (1983) The conjugate gradient method and trust regions in large scale optimization. SIAM J Numer Anal 20(3):626–637

    Article  Google Scholar 

  91. Byrd RH, Schnabel RB, Shultz GA (1988) Approximate solution of the trust region problem by minimization over two-dimensional subspaces. Math Program 40(1):247–263

    Article  Google Scholar 

  92. Branch MA, Coleman TF, Li Y (1999) A subspace, interior, and conjugate gradient method for large-scale bound-constrained minimization problems. SIAM J Sci Comput 21(1):1–23

    Article  Google Scholar 

  93. Stapor P, Weindl D, Ballnus B, Hug S, Loos C, Fiedler A, Krause S, Hross S, Fröhlich F, Hasenauer J (2017) PESTO: Parameter EStimation TOolbox. Bioinformatics btx676

    Article  PubMed Central  CAS  Google Scholar 

  94. Egea JA, Henriques D, Cokelaer T, Villaverde AF, MacNamara A, Danciu DP, Banga JR, Saez-Rodriguez J (2014) MEIGO: an open-source software suite based on metaheuristics for global optimization in systems biology and bioinformatics. BMC Bioinf 15:136

    Article  Google Scholar 

  95. Fogel DB, Fogel LJ, Atmar JW (1991) Meta-evolutionary programming. In: 1991 Conference record of the twenty-fifth Asilomar conference on signals, systems and computers, 1991, pp 540–545. IEEE, New York

    Chapter  Google Scholar 

  96. Michalewicz Z (2013) Genetic Algorithms + Data Structures = Evolution Programs. Springer Science & Business Media, Berlin/Heidelberg

    Google Scholar 

  97. Brent RP (2013) Algorithms for minimization without derivatives. Courier Corporation, North Chelmsford

    Google Scholar 

  98. Dembo RS, Steihaug T (1983) Truncated-newtono algorithms for large-scale unconstrained optimization. Math Program 26(2):190–212

    Article  Google Scholar 

  99. Solis FJ, Wets RJB (1981) Minimization by random search techniques. Math Oper Res 6(1):19–30

    Article  Google Scholar 

  100. Runarsson TP, Yao X (2000) Stochastic ranking for constrained evolutionary optimization. IEEE Trans Evolut Comput 4(3): 284–294

    Article  Google Scholar 

  101. Bellavia S, Macconi M, Morini B (2004) STRSCNE: a scaled trust-region solver for constrained nonlinear equations. Comput Optim Appl 28(1):31–50

    Article  Google Scholar 

  102. Morini B, Porcelli M (2012) TRESNEI, a Matlab trust-region solver for systems of nonlinear equalities and inequalities. Comput Optim Appl 51(1):27–49

    Article  Google Scholar 

  103. Le Digabel S (2011) Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans Math Software 37(4):44

    Article  Google Scholar 

  104. Kelley CT (1999) Iterative methods for optimization. SIAM, Philadelphia

    Book  Google Scholar 

  105. Wächter A, Biegler LT (2006) On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math Program 106(1):25–57

    Article  Google Scholar 

  106. Ye Y (1989) SOLNP users’ guide. Tech. rep., Department of Management Sciences, University of Iowa

    Google Scholar 

  107. Exler O, Lehmann T, Schittkowski K (2012) MISQP: a Fortran subroutine of a trust region SQP algorithm for mixed-integer nonlinear programming-user’s guide. Tech. rep., Department of Computer Science, University of Bayreuth

    Google Scholar 

  108. Dennis JE Jr, Gay DM, Welsch RE (1981) Algorithm 573: Nl2sol—an adaptive nonlinear least-squares algorithm [E4]. ACM Trans Math Software 7(3):369–383

    Article  Google Scholar 

  109. Powell MJ (2009) The BOBYQA algorithm for bound constrained optimization without derivatives. Tech. rep., Cambridge NA Report NA2009/06, University of Cambridge, Cambridge

    Google Scholar 

  110. Vaz AIF, Vicente LN (2009) PSwarm: a hybrid solver for linearly constrained global derivative-free optimization. Optim Method Softw 24(4–5):669–685

    Article  Google Scholar 

  111. Degasperi A, Fey D, Kholodenko BN (2017) Performance of objective functions and optimisation procedures for parameter estimation in system biology models. NPJ Syst Biol Appl 3(1):20

    Article  PubMed  PubMed Central  Google Scholar 

  112. Wieland FG (2016) Implementation and assessment of optimization approaches for parameter estimation in systems biology. Tech. rep., University of Freiburg

    Google Scholar 

  113. Villaverde AF, Henriques D, Smallbone K, Bongard S, Schmid J, Cicin-Sain D, Crombach A, Saez-Rodriguez J, Mauch K, Balsa-Canto E, Mendes P, Jaeger J, Banga JR (2015) BioPreDyn-bench: a suite of benchmark problems for dynamic modelling in systems biology. BMC Syst Biol 9(8)

    Article  PubMed  PubMed Central  Google Scholar 

  114. Kreutz C (2016) New concepts for evaluating the performance of computational methods. IFAC-PapersOnLine 49(26):63–70

    Article  Google Scholar 

  115. Shamir M, Bar-On Y, Phillips R, Milo R (2016) SnapShot: timescales in cell biology. Cell 164(6):1302–1302

    Article  CAS  PubMed  Google Scholar 

  116. Hasenauer J, Jagiella N, Hross S, Theis FJ (2015) Data-driven modelling of biological multi-scale processes. J Coupled Syst Multiscale Dynam 3(2):101–121

    Article  Google Scholar 

  117. Smallbone K, Mendes P (2013) Large-scale metabolic models: from reconstruction to differential equations. Ind Biotechnol 9(4):179–184

    Article  CAS  Google Scholar 

  118. Resat H, Petzold L, Pettigrew MF (2009) Kinetic modeling of biological systems. Methods Mol Biol 541:311–335

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  119. Gonnet P, Dimopoulos S, Widmer L, Stelling J (2012) A specialized ODE integrator for the efficient computation of parameter sensitivities. BMC Syst Biol 6:46

    Article  PubMed  PubMed Central  Google Scholar 

  120. Butcher JC (1964) Implicit Runge-Kutta processes. Math Comp 18(85):50–64

    Article  Google Scholar 

  121. Alexander R (1977) Diagonally implicit Runge–Kutta methods for stiff O.D.E.’s. SIAM J Numer Anal 14(6):1006–1021

    Article  Google Scholar 

  122. Rosenbrock HH (1963) Some general implicit processes for the numerical solution of differential equations. Comput J 5(4):329–330

    Article  Google Scholar 

  123. Zhang H, Sandu A (2014) FATODE: a library for forward, adjoint, and tangent linear integration of ODEs. SIAM J Sci Comput 36(5):C504–C523

    Article  Google Scholar 

  124. Serban R, Hindmarsh AC (2005) CVODES: an ODE solver with sensitivity analysis capabilities. ACM Trans Math Software 31(3):363–396

    Article  Google Scholar 

  125. Hindmarsh AC, Brown PN, Grant KE, Lee SL, Serban R, Shumaker DE, Woodward CS (2005) SUNDIALS: Suite of Nonlinear and Differential/Algebraic Equation Solvers. ACM Trans Math Software 31(3):363–396

    Article  Google Scholar 

  126. Coppersmith D, Winograd S (1990) Matrix multiplication via arithmetic progressions. J Symb Comp 9(3):251–280

    Article  Google Scholar 

  127. Thorson J (1979) Gaussian elimination on a banded matrix

    Google Scholar 

  128. Barabasi AL, Oltvai ZN (2004) Network biology: understanding the cell’s functional organization. Nat Rev Genet 5(2):101–113

    Article  CAS  PubMed  Google Scholar 

  129. Davis TA, Palamadai Natarajan E (2010) Algorithm 907: KLU, a direct sparse solver for circuit simulation problems. ACM Trans Math Software 37(3):36

    Article  Google Scholar 

  130. Petzold L (1983) Automatic selection of methods for solving stiff and nonstiff systems of ordinary differential equations. SIAM J Sci Stat Comp 4(1):136–148

    Article  Google Scholar 

  131. Tangherloni A, Nobile MS, Besozzi D, Mauri G, Cazzaniga P (2017) LASSIE: simulating large-scale models of biochemical systems on GPUs. BMC Bioinformatics 18(1):246

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  132. Demmel JW, Gilbert JR, Li XS (1999) An asynchronous parallel supernodal algorithm for sparse Gaussian elimination. SIAM J Matrix Anal A 20(4):915–952

    Article  Google Scholar 

  133. Hucka M, Finney A, Sauro HM, Bolouri H, Doyle JC, Kitano H, Arkin AP, Bornstein BJ, Bray D, Cornish-Bowden A, Cuellar AA, Dronov S, Gilles ED, Ginkel M, Gor V, Goryanin II, Hedley WJ, Hodgman TC, Hofmeyr JH, Hunter PJ, Juty NS, Kasberger JL, Kremling A, Kummer U, Le˜Novère N, Loew LM, Lucio D, Mendes P, Minch E, Mjolsness ED, Nakayama Y, Nelson MR, Nielsen PF, Sakurada T, Schaff JC, Shapiro BE, Shimizu TS, Spence HD, Stelling J, Takahashi K, Tomita M, Wagner J, Wang J (2003) The systems biology markup language (SBML): a medium for representation and exchange of biochemical network models. Bioinformatics 19(4):524–531

    Google Scholar 

  134. Griewank A, Walther A (2008) Evaluating derivatives, 2nd edn. Society for Industrial and Applied Mathematics, Philadelphia

    Book  Google Scholar 

  135. Milne-Thompson L (1933) The calculus of finite differences. Macmillan, London

    Google Scholar 

  136. Dickinson RP, Gelinas RJ (1976) Sensitivity analysis of ordinary differential equation systems—a direct method. J Comput Phys 21(2):123–143

    Article  Google Scholar 

  137. Kokotovic P, Heller J (1967) Direct and adjoint sensitivity equations for parameter optimization. IEEE Trans Autom Contr 12(5):609–610

    Article  Google Scholar 

  138. Lu J, Muller S, Machné R, Flamm C (2008) SBML ODE solver library: extensions for inverse analysis. In: Proc. 5th int. W. comp. syst. biol.

    Google Scholar 

  139. Fujarewicz K, Kimmel M, Swierniak A (2005) On fitting of mathematical models of cell signaling pathways using adjoint systems. Math Bio Eng 2(3):527–534

    Article  Google Scholar 

  140. Lu J, August E, Koeppl H (2012) Inverse problems from biomedicine: inference of putative disease mechanisms and robust therapeutic strategies. J Math Biol 67(1):143–168

    Article  PubMed  Google Scholar 

  141. Plessix RE (2006) A review of the adjoint-state method for computing the gradient of a functional with geophysical applications. Geophys J Int 167(2):495–503

    Article  Google Scholar 

  142. Balsa-Canto E, Banga JR, Alonso AA, Vassiliadis VS (2001) Dynamic optimization of chemical and biochemical processes using restricted second-order information. Comput Chem Eng 25(4):539–546

    Article  CAS  Google Scholar 

  143. Vassiliadis VS, Canto EB, Banga JR (1999) Second-order sensitivities of general dynamic systems with application to optimal control problems. Chem Eng Sci 54(17):3851–3860

    Article  CAS  Google Scholar 

  144. Özyurt DB, Barton PI (2005) Cheap second order directional derivatives of stiff ODE embedded functionals. SIAM J Sci Comput 26(5):1725–1743

    Article  Google Scholar 

  145. Björck Å (1996) Numerical methods for least squares problems. SIAM, Philadelphia

    Book  Google Scholar 

  146. Fisher RA (1922) On the mathematical foundations of theoretical statistics. Philos Trans R Soc London, Ser A 222:309–368

    Article  Google Scholar 

  147. Fletcher R, Powell MJ (1963) A rapidly convergent descent method for minimization. Comp J 6(2):163–168

    Article  Google Scholar 

  148. Goldfarb D (1970) A family of variable-metric methods derived by variational means. Math Comp 24(109):23–26

    Article  Google Scholar 

  149. Byrd RH, Khalfan HF, Schnabel RB (1996) Analysis of a symmetric rank-one trust region method. SIAM J Optim 6(4):1025–1039

    Article  Google Scholar 

  150. Ramamurthy V, Duffy N (2017) L-SR1: A second order optimization method for deep learning, under review as a conference paper at ICLR 2017

    Google Scholar 

  151. Nocedal J (1980) Updating quasi-newton matrices with limited storage. Mathematics of computation 35(151):773–782

    Article  Google Scholar 

  152. Liu DC, Nocedal J (1989) On the limited memory BFGS method for large scale optimization. Math Program 45(1):503–528

    Article  Google Scholar 

  153. Andrew G, Gao J (2007) Scalable training of l1-regularized log-linear models. In: Proceedings of the 24th international conference on Machine learning - ICML ’07. ACM, Philadelphia, pp 33–40

    Chapter  Google Scholar 

  154. Raue A, Kreutz C, Maiwald T, Bachmann J, Schilling M, Klingmüller U, Timmer J (2009) Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. Bioinformatics 25(25):1923–1929

    Article  CAS  PubMed  Google Scholar 

  155. Girolami M, Calderhead B (2011) Riemann manifold Langevin and Hamiltonian Monte Carlo methods. J R Statist Soc B 73(2):123–214

    Article  Google Scholar 

  156. Vanlier J, Tiemann CA, Hilbers PAJ, van Riel NAW (2012) A Bayesian approach to targeted experiment design. Bioinformatics 28(8):1136–1142

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  157. Blumer A, Ehrenfeucht A, Haussler D, Warmuth MK (1987) Occam’s razor. Inform Process Lett 24(6):377–380

    Article  Google Scholar 

  158. Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning, vol 2. Springer, New York

    Book  Google Scholar 

  159. Shibata R (1980) Asymptotically efficient selection of the order of the model for estimating parameters of a linear process. Ann Stat 8(1):147–164

    Article  Google Scholar 

  160. Kass RE, Raftery AE (1995) Bayes factors. J Am Stat Assoc 90(430):773–795

    Article  Google Scholar 

  161. Wang M, Sun X (2014) Bayes factor consistency for nested linear models with a growing number of parameters. J Stat Plan Infer 147:95–105

    Article  Google Scholar 

  162. Choi T, Rousseau J (2015) A note on Bayes factor consistency in partial linear models. J Stat Plan Infer 166:158–170

    Article  Google Scholar 

  163. Jeffreys H (1961) Theory of probability, 3rd edn. Oxford University Press, Oxford

    Google Scholar 

  164. Meng XL, Wong WH (1996) Simulating ratios of normalizing constants via a simple identity: a theoretical exploration. Stat Sin 6(4):831–860

    Google Scholar 

  165. Skilling J (2006) Nested sampling for general Bayesian computation. Bayesian Anal 1(4):833–359

    Article  Google Scholar 

  166. Burnham KP, Anderson DR (2002) Model selection and multimodel inference: a practical information-theoretic approach, 2nd edn. Springer, New York, NY

    Google Scholar 

  167. Shibata R (1981) An optimal selection of regression variables. Biometrika 68(1):45–54

    Article  Google Scholar 

  168. Kuha J (2004) AIC and BIC: comparisons of assumptions and performance. Sociol Method Res 33(2):188–229

    Article  Google Scholar 

  169. Acquah HDG (2010) Comparison of Akaike information criterion (AIC) and Bayesian information criterion (BIC) in selection of an asymmetric price relationship. J Dev Agric Econ 2(1):001–006

    Google Scholar 

  170. Hurvich C, Tsia CL (1989) Regression and time series model selection in small samples. Biometrika 76(2):297–307

    Article  Google Scholar 

  171. Chen J, Chen Z (2008) Extended Bayesian information criteria for model selection with large model spaces. Biometrika 95(3):759–771

    Article  Google Scholar 

  172. Wilks SS (1938) The large-sample distribution of the likelihood ratio for testing composite hypotheses. Ann Math Stat 9(1):60–62

    Article  Google Scholar 

  173. Neyman J, Pearson ES (1992) On the problem of the most efficient tests of statistical hypotheses. In: Breakthroughs in statistics. Springer, New York, pp 73–108

    Chapter  Google Scholar 

  174. Gelman A, Hwang J, Vehtari A (2014) Understanding predictive information criteria for Bayesian models. Stat Comp 24(6):997–1016

    Article  Google Scholar 

  175. Arlot S, Celisse A (2010) A survey of cross-validation procedures for model selection. Stat Surv 4:40–79

    Article  Google Scholar 

  176. Vyshemirsky V, Girolami M (2008) BioBayes: a software package for Bayesian inference in systems biology. Bioinformatics 24(17):1933–1934

    Article  CAS  PubMed  Google Scholar 

  177. Feroz F, Hobson M, Bridges M (2009) Multinest: an efficient and robust Bayesian inference tool for cosmology and particle physics. Mon Not R Astron Soc 398(4):1601–1614

    Article  Google Scholar 

  178. Thijssen B, Dijkstra TM, Heskes T, Wessels LF (2016) BCM: toolkit for Bayesian analysis of computational models using samplers. BMC Syst Biol 10(1):100

    Article  PubMed  PubMed Central  Google Scholar 

  179. Kaltenbacher B, Offtermatt J (2011) A refinement and coarsening indicator algorithm for finding sparse solutions of inverse problems. Inverse Probl Imaging 5(2):391–406

    Article  Google Scholar 

  180. Liu Z, Li G (2016) Efficient regularized regression with L0 penalty for variable selection and network construction. Comput Math Methods Med, 3456153

    Google Scholar 

  181. Rodriguez-Fernandez M, Rehberg M, Kremling A, Banga JR (2013) Simultaneous model discrimination and parameter estimation in dynamic models of cellular systems. BMC Sys Biol 7(1):76

    Article  Google Scholar 

  182. Henriques DR, M Saez-Rodriguez J, Banga JR (2015) Reverse engineering of logic-based differential equation models using a mixed-integer dynamic optimization approach. Bioinformatics 31(18):2999–3007

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  183. Daubechies I, DeVore R, Fornasier M, Güntürk CS (2010) Iteratively reweighted least squares minimization for sparse recovery. Commun Pur Appl Math 63(1):1–38

    Article  Google Scholar 

  184. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Statist Soc B 58(1):267–288

    Google Scholar 

  185. Chen SS, Donoho DL, Saunders MA (2001) Atomic decomposition by basis pursuit. SIAM Rev 43(1):129–159

    Article  Google Scholar 

  186. Efron B, Hastie T, Johnstone I, Tibshirani R (2004) Least angle regression. Ann Stat 32(2):407–499

    Article  Google Scholar 

  187. Wassermann L (2000) Bayesian model selection and model averaging. J Math Psychol 44(1):92–107

    Article  Google Scholar 

  188. Mannakee BK, Ragsdale AP, Transtrum MK, Gutenkunst RN (2016) Sloppiness and the geometry of parameter space. Springer International Publishing, Cham, pp 271–299

    Google Scholar 

  189. Transtrum MK, Machta BB, Sethna JP (2011) Geometry of nonlinear least squares with applications to sloppy models and optimization. Phys Rev E 83:036701

    Article  CAS  Google Scholar 

  190. Holland PW, Welsch RE (1977) Robust regression using iteratively reweighted least-squares. Commun Stat Theory 6(9): 813–827

    Article  Google Scholar 

  191. Karr JR, Sanghvi JC, Macklin DN, Gutschow MV, Jacobs JM, Bolival˜Jr B, Assad-Garcia N, Glass JI, Covert MW (2012) A whole-cell computational model predicts phenotype from genotype. Cell 150(2): 389–401

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  192. Tomita M, Hashimoto K, Takahashi K, Shimizu TS, Matsuzaki Y, Miyoshi F, Saito K, Tanida S, Yugi K, Venter JC, Hutchision˜III CA (1999) E-CELL: software environment for whole-cell simulation. Bioinformatics 15(1):72–84

    Article  CAS  PubMed  Google Scholar 

  193. Fröhlich F, Kessler T, Weindl D, Shadrin A, Schmiester L, Hache H, Muradyan A, Schuette M, Lim JH, Heinig M, Theis F, Lehrach H, Wierling C, Lange B, Hasenauer J (2017) Efficient parameterization of large-scale mechanistic models enables drug response prediction for cancer cell lines. bioRxiv, 174094

    Google Scholar 

  194. Büchel F, Rodriguez N, Swainston N, Wrzodek C, Czauderna T, Keller R, Mittag F, Schubert M, Glont M, Golebiewski M, van˜Iersel M, Keating S, Rall M, Wybrow M, Hermjakob H, Hucka M, Kell DB, Müller W, Mendes P, Zell A, Chaouiya C, Saez-Rodriguez J, Schreiber F, Laibe C, Dräger A, Novére NL (2013) Path2Models: Large-scale generation of computational models from biochemical pathway maps. BMC Syst Biol 7(116)

    Article  PubMed  PubMed Central  CAS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computational Biology, Helmholtz Zentrum München, Neuherberg, Germany

    Fabian Fröhlich, Carolin Loos & Jan Hasenauer

  2. Center for Mathematics, Technische Universität München, Garching, Germany

    Fabian Fröhlich, Carolin Loos & Jan Hasenauer

Authors
  1. Fabian Fröhlich
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carolin Loos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jan Hasenauer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jan Hasenauer .

Editor information

Editors and Affiliations

  1. School of Informatics, University of Edinburgh, Edinburgh, UK

    Guido Sanguinetti

  2. Department of Electrical Engineering and Computer Science, University of Liège, Liège, Belgium

    Vân Anh Huynh-Thu

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Science+Business Media, LLC, part of Springer Nature

About this protocol

Check for updates. Verify currency and authenticity via CrossMark

Cite this protocol

Fröhlich, F., Loos, C., Hasenauer, J. (2019). Scalable Inference of Ordinary Differential Equation Models of Biochemical Processes. In: Sanguinetti, G., Huynh-Thu, V. (eds) Gene Regulatory Networks. Methods in Molecular Biology, vol 1883. Humana Press, New York, NY. https://doi.org/10.1007/978-1-4939-8882-2_16

Download citation

  • DOI: https://doi.org/10.1007/978-1-4939-8882-2_16

  • Published:

  • Publisher Name: Humana Press, New York, NY

  • Print ISBN: 978-1-4939-8881-5

  • Online ISBN: 978-1-4939-8882-2

  • eBook Packages: Springer Protocols

Publish with us

Policies and ethics

Search

Navigation