Quantitative Biology

, Volume 6, Issue 4, pp 287–306 | Cite as

Experimental design and model reduction in systems biology

  • Jenny E. Jeong
  • Qinwei Zhuang
  • Mark K. Transtrum
  • Enlu Zhou
  • Peng QiuEmail author



In systems biology, the dynamics of biological networks are often modeled with ordinary differential equations (ODEs) that encode interacting components in the systems, resulting in highly complex models. In contrast, the amount of experimentally available data is almost always limited, and insufficient to constrain the parameters. In this situation, parameter estimation is a very challenging problem. To address this challenge, two intuitive approaches are to perform experimental design to generate more data, and to perform model reduction to simplify the model. Experimental design and model reduction have been traditionally viewed as two distinct areas, and an extensive literature and excellent reviews exist on each of the two areas. Intriguingly, however, the intrinsic connections between the two areas have not been recognized.


Experimental design and model reduction are deeply related, and can be considered as one unified framework. There are two recent methods that can tackle both areas, one based on model manifold and the other based on profile likelihood. We use a simple sum-of-two-exponentials example to discuss the concepts and algorithmic details of both methods, and provide Matlab-based code and implementation which are useful resources for the dissemination and adoption of experimental design and model reduction in the biology community.


From a geometric perspective, we consider the experimental data as a point in a high-dimensional data space and the mathematical model as a manifold living in this space. Parameter estimation can be viewed as a projection of the data point onto the manifold. By examining the singularity around the projected point on the manifold, we can perform both experimental design and model reduction. Experimental design identifies new experiments that expand the manifold and remove the singularity, whereas model reduction identifies the nearest boundary, which is the nearest singularity that suggests an appropriate form of a reduced model. This geometric interpretation represents one step toward the convergence of experimental design and model reduction as a unified framework.


experimental design model reduction model manifold profile likelihood 



This work is supported by funding from the National Science Foundation (CCF1552784). Peng Qiu is an ISAC Marylou Ingram Scholar.


  1. 1.
    Lander, A. D. (2004) A calculus of purpose. PLoS Biol., 2, e164CrossRefGoogle Scholar
  2. 2.
    Sobie, E. A., Lee, Y. S., Jenkins, S. L. and Iyengar, R. (2011) Systems biology—biomedical modeling. Sci. Signal., 4, tr2CrossRefGoogle Scholar
  3. 3.
    Fages, F., Gay, S. and Soliman, S. (2015) Inferring reaction systems from ordinary differential equations. Theor. Comput. Sci., 599, 64–78CrossRefGoogle Scholar
  4. 4.
    Jha, S. K. and Langmead, C. J. (2012) Exploring behaviors of stochastic differential equation models of biological systems using change of measures. BMC Bioinformatics, 13, S8CrossRefGoogle Scholar
  5. 5.
    Kauffman, S. A. (1969) Metabolic stability and epigenesis in randomly constructed genetic nets. J. Theor. Biol., 22, 437–467CrossRefGoogle Scholar
  6. 6.
    Sachs, K., Gifford, D., Jaakkola, T., Sorger, P. and Lauffenburger, D. A. (2002) Bayesian network approach to cell signaling pathway modeling. Sci. STKE, 2002, pe38Google Scholar
  7. 7.
    Koch, I. (2015) Petri nets in systems biology. Soft. Syst. Model., 14, 703–710CrossRefGoogle Scholar
  8. 8.
    Materi, W. and Wishart, D. S. (2007) Computational systems biology in drug discovery and development: methods and applications. Drug Discov. Today, 12, 295–303CrossRefGoogle Scholar
  9. 9.
    Machado, D., Costa, R. S., Rocha, M., Ferreira, E. C., Tidor, B. and Rocha, I. (2011) Modeling formalisms in systems biology. AMB Express, 1, 45CrossRefGoogle Scholar
  10. 10.
    Bartocci, E. and Lió, P. (2016) Computational modeling, formal analysis, and tools for systems biology. PLoS Comput. Biol., 12, e1004591CrossRefGoogle Scholar
  11. 11.
    Kitano, H. (2002) Computational systems biology. Nature, 420, 206–210CrossRefGoogle Scholar
  12. 12.
    Aldridge, B. B., Burke, J. M., Lauffenburger, D. A. and Sorger, P. K. (2006) Physicochemical modelling of cell signalling pathways. Nat. Cell Biol., 8, 1195–1203CrossRefGoogle Scholar
  13. 13.
    Anderson, J., Chang, Y. C. and Papachristodoulou, A. (2011) Model decomposition and reduction tools for large-scale networks in systems biology. Automatica, 47, 1165–1174CrossRefGoogle Scholar
  14. 14.
    Quaiser, T., Dittrich, A., Schaper, F. and Mönnigmann, M. (2011) A simple work flow for biologically inspired model reduction—application to early JAK-STAT signaling. BMC Syst. Biol., 5, 30CrossRefGoogle Scholar
  15. 15.
    Villaverde, A. F., Henriques, D., Smallbone, K., Bongard, S., Schmid, J., Cicin-Sain, D., Crombach, A., Saez-Rodriguez, J., Mauch, K., Balsa-Canto, E., et al. (2015) BioPreDyn-bench: a suite of benchmark problems for dynamic modelling in systems biology. BMC Syst. Biol., 9, 8CrossRefGoogle Scholar
  16. 16.
    Machta, B. B., Chachra, R., Transtrum, M. K. and Sethna, J. P. (2013) Parameter space compression underlies emergent theories and predictive models. Science, 342, 604–607CrossRefGoogle Scholar
  17. 17.
    Boyd, S. and Vandenberghe, L. (2004) Convex Optimization. New York: Cambridge University PressCrossRefGoogle Scholar
  18. 18.
    Moles, C. G., Mendes, P. and Banga, J. R. (2003) Parameter estimation in biochemical pathways: a comparison of global optimization methods. Genome Res., 13, 2467–2474CrossRefGoogle Scholar
  19. 19.
    Ramsay, J. O., Hooker, G., Campbell, D. and Cao, J. (2007) Parameter estimation for differential equations: a generalized smoothing approach. J. R. Stat. Soc. Series B Stat. Methodol., 69, 741–796.CrossRefGoogle Scholar
  20. 20.
    Zenker, S., Rubin, J. and Clermont, G. (2007) From inverse problems in mathematical physiology to quantitative differential diagnoses. PLoS Comput. Biol., 3, e204CrossRefGoogle Scholar
  21. 21.
    Campbell, D. A. and Chkrebtii, O. (2013) Maximum profile likelihood estimation of differential equation parameters through model based smoothing state estimates. Math. Biosci., 246, 283–292CrossRefGoogle Scholar
  22. 22.
    Banga, J. R. and Balsa-Canto, E. (2008) Parameter estimation and optimal experimental design. Essays Biochem., 45, 195–210CrossRefGoogle Scholar
  23. 23.
    Kreutz, C. and Timmer, J. (2009) Systems biology: experimental design. FEBS J., 276, 923–942CrossRefGoogle Scholar
  24. 24.
    Meyer, P., Cokelaer, T., Chandran, D., Kim, K. H., Loh, P. R., Tucker, G., Lipson, M., Berger, B., Kreutz, C., Raue, A. (2014) Network topology and parameter estimation: from experimental design methods to gene regulatory network kinetics using a community based approach. BMC Syst. Biol., 8, 13CrossRefGoogle Scholar
  25. 25.
    Apri, M., de Gee, M. and Molenaar, J. (2012) Complexity reduction preserving dynamical behavior of biochemical networks. J. Theor. Biol., 304, 16–26CrossRefGoogle Scholar
  26. 26.
    Danø, S., Madsen, M. F., Schmidt, H. and Cedersund, G. (2006) Reduction of a biochemical model with preservation of its basic dynamic properties. FEBS J., 273, 4862–4877CrossRefGoogle Scholar
  27. 27.
    Kourdis, P. D., Palasantza, A. G. and Goussis, D. A. (2013) Algorithmic asymptotic analysis of the NF-κB signaling system. Comput. Math. Appl., 65, 1516–1534CrossRefGoogle Scholar
  28. 28.
    Radulescu, O., Gorban, A. N., Zinovyev, A. and Noel, V. (2012) Reduction of dynamical biochemical reactions networks in computational biology. Front. Genet., 3, 131CrossRefGoogle Scholar
  29. 29.
    Vanlier, J., Tiemann, C. A., Hilbers, P. A. J. and van Riel, N. A. W. (2012) An integrated strategy for prediction uncertainty analysis. Bioinformatics, 28, 1130–1135CrossRefGoogle Scholar
  30. 30.
    Vanlier, J., Tiemann, C. A., Hilbers, P. A. J. and van Riel, N. A. W. (2012) A Bayesian approach to targeted experiment design. Bioinformatics, 28, 1136–1142CrossRefGoogle Scholar
  31. 31.
    Huan, X. and Marzouk, Y. M. (2013) Simulation-based optimal Bayesian experimental design for nonlinear systems. J. Comput. Phys., 232, 288–317CrossRefGoogle Scholar
  32. 32.
    Pauwels, E., Lajaunie, C. and Vert, J. P. (2014) A Bayesian active learning strategy for sequential experimental design in systems biology. BMC Syst. Biol., 8, 102CrossRefGoogle Scholar
  33. 33.
    Liepe, J., Filippi, S., Komorowski, M. and Stumpf, M. P. H. (2013) Maximizing the information content of experiments in systems biology. PLoS Comput. Biol., 9, e1002888CrossRefGoogle Scholar
  34. 34.
    Busetto, A. G., Hauser, A., Krummenacher, G., Sunnåker, M., Dimopoulos, S., Ong, C. S., Stelling, J. and Buhmann, J. M. (2013) Near-optimal experimental design for model selection in systems biology. Bioinformatics, 29, 2625–2632CrossRefGoogle Scholar
  35. 35.
    Faller, D., Klingmüller, U. and Timmer, J. (2003) Simulation methods for optimal experimental design in systems biology. Simulation, 79, 717–725CrossRefGoogle Scholar
  36. 36.
    Casey, F. P., Baird, D., Feng, Q., Gutenkunst, R. N., Waterfall, J. J., Myers, C. R., Brown, K. S., Cerione, R. A. and Sethna, J. P. (2007) Optimal experimental design in an epidermal growth factor receptor signalling and down-regulation model. IET Syst. Biol., 1, 190–202CrossRefGoogle Scholar
  37. 37.
    Krüger, R. and Heinrich, R. (2004) Model reduction and analysis of robustness for the Wnt/β-Catenin signal transduction pathway. Genome Inform., 15, 138–148Google Scholar
  38. 38.
    Gerdtzen, Z. P., Daoutidis, P. and Hu, W. S. (2004) Non-linear reduction for kinetic models of metabolic reaction networks. Metab. Eng., 6, 140–154CrossRefGoogle Scholar
  39. 39.
    Vora, N. and Daoutidis, P. (2001) Nonlinear model reduction of chemical reaction systems. Aiche J., 47, 2320–2332CrossRefGoogle Scholar
  40. 40.
    Lam, S. H. (2013) Model reductions with special CSP data. Combust. Flame, 160, 2707–2711CrossRefGoogle Scholar
  41. 41.
    Kuo, J. C. W. and Wei, J. (1969) Lumping analysis in monomolecular reaction systems. analysis of approximately lumpable system. Ind. Eng. Chem. Fundam., 8, 124–133CrossRefGoogle Scholar
  42. 42.
    Liao, J. C. and Lightfoot, E. N. Jr. (1988) Lumping analysis of biochemical reaction systems with time scale separation. Biotechnol. Bioeng., 31, 869–879CrossRefGoogle Scholar
  43. 43.
    Brochot, C., Tóth, J. and Bois, F. Y. (2005) Lumping in pharmacokinetics. J. Pharmacokinet. Pharmacodyn., 32, 719–736CrossRefGoogle Scholar
  44. 44.
    Dokoumetzidis A, Aarons L (2009) Proper lumping in systems biology models. IET Syst. Biol., 3, 40–51CrossRefGoogle Scholar
  45. 45.
    Seigneur, C., Stephanopoulos, G. and Carr Jr., R. W. (1982) Dynamic sensitivity analysis of chemical reaction systems: a variational method. Chem. Eng. Sci., 37, 845–853CrossRefGoogle Scholar
  46. 46.
    Turányi, T., Bérces, T. and Vajda, S. (1989) Reaction rate analysis of complex kinetic systems. Int. J. Chem. Kinet., 21, 83–99CrossRefGoogle Scholar
  47. 47.
    Petzold, L. and Zhu, W. (1999) Model reduction for chemical kinetics: an optimization approach. Aiche J., 45, 869–886CrossRefGoogle Scholar
  48. 48.
    Liu, G., Swihart, M. T. and Neelamegham, S. (2005) Sensitivity, principal component and flux analysis applied to signal transduction: the case of epidermal growth factor mediated signaling. Bioinformatics, 21, 1194–1202CrossRefGoogle Scholar
  49. 49.
    Schmidt, H., Madsen, M. F., Danø, S. and Cedersund, G. (2008) Complexity reduction of biochemical rate expressions. Bioinformatics, 24, 848–854CrossRefGoogle Scholar
  50. 50.
    Steiert, B., Raue, A., Timmer, J. and Kreutz, C. (2012) Experimental design for parameter estimation of gene regulatory networks. PLoS One, 7, e40052CrossRefGoogle Scholar
  51. 51.
    Maiwald, T., Hass, H., Steiert, B., Vanlier, J., Engesser, R., Raue, A., Kipkeew, F., Bock, H. H., Kaschek, D., Kreutz, C., et al. (2016) Driving the model to its limit: profile likelihood based model reduction. PLoS One, 11, e0162366CrossRefGoogle Scholar
  52. 52.
    Transtrum, M. K. and Qiu, P. (2012) Optimal experiment selection for parameter estimation in biological differential equation models. BMC Bioinformatics, 13, 181CrossRefGoogle Scholar
  53. 53.
    Transtrum, M. K. and Qiu, P. (2014) Model reduction by manifold boundaries. Phys. Rev. Lett., 113, 098701CrossRefGoogle Scholar
  54. 54.
    Transtrum, M. K. and Qiu, P. (2016) Bridging mechanistic and phenomenological models of complex biological systems. PLoS Comput. Biol., 12, e1004915CrossRefGoogle Scholar
  55. 55.
    Kutalik, Z., Cho, K. H. and Wolkenhauer, O. (2004) Optimal sampling time selection for parameter estimation in dynamic pathway modeling. Biosystems, 75, 43–55CrossRefGoogle Scholar
  56. 56.
    Bandara, S., Schlöder, J. P., Eils, R., Bock, H. G. and Meyer, T. (2009) Optimal experimental design for parameter estimation of a cell signaling model. PLoS Comput. Biol., 5, e1000558CrossRefGoogle Scholar
  57. 57.
    Hagen, D. R., White, J. K. and Tidor, B. (2013) Convergence in parameters and predictions using computational experimental design. Interface Focus, 3, 20130008CrossRefGoogle Scholar
  58. 58.
    Toni, T., Welch, D., Strelkowa, N., Ipsen, A. and Stumpf, M. P. (2009) Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. J. R. Soc. Interface, 6, 187–202CrossRefGoogle Scholar
  59. 59.
    Frieden, B.R. (2000) Physics from fisher information: a unification. Am. J. Phys., 68, 1064–1065CrossRefGoogle Scholar
  60. 60.
    Transtrum, M. K., Machta, B. B. and Sethna, J. P. (2011) Geometry of nonlinear least squares with applications to sloppy models and optimization. Phys. Rev. E Stat. Nonlin. Soft Matter Phys., 83, 036701CrossRefGoogle Scholar
  61. 61.
    Leis, J. R. and Kramer, M. A. (1988) The simultaneous solution and sensitivity analysis of systems described by ordinary differential equations. ACM Trans. Math. Softw., 14, 45–60CrossRefGoogle Scholar
  62. 62.
    Kumar, A., Christofides, P. D. and Daoutidis, P. (1998) Singular perturbation modeling of nonlinear processes with nonexplicit time-scale multiplicity. Chem. Eng. Sci., 53, 1491–1504CrossRefGoogle Scholar
  63. 63.
    Snowden, T. J., van der Graaf, P. H. and Tindall, M. J. (2017) Methods of model reduction for large-scale biological systems: a survey of current methods and trends. Bull. Math. Biol., 79, 1449–1486CrossRefGoogle Scholar
  64. 64.
    Heinrich, R. and Schuster, S. (1996) The Regulation of Cellular Systems. Springer: New YorkCrossRefGoogle Scholar
  65. 65.
    Voit, E. (2012) A First Course in Systems Biology. 1st ed., Garland Science: New YorkGoogle Scholar
  66. 66.
    Okino, M. S. and Mavrovouniotis, M. L. (1998) Simplification of mathematical models of chemical reaction systems. Chem. Rev., 98, 391–408CrossRefGoogle Scholar
  67. 67.
    Wolf, J. and Heinrich, R. (2000) Effect of cellular interaction on glycolytic oscillations in yeast: a theoretical investigation. Biochem. J., 345, 321–334CrossRefGoogle Scholar
  68. 68.
    Sauter, T., Gilles, E. D., Allgöwer, F., Saez-Rodriguez, J., Conzelmann, H. and Bullinger, E. (2004) Reduction of mathematical models of signal transduction networks: simulation-based approach applied to EGF receptor signalling. Syst. Biol. (Stevenage), 1, 159–169CrossRefGoogle Scholar
  69. 69.
    Liebermeister, W., Baur, U. and Klipp, E. (2005) Biochemical network models simplified by balanced truncation. FEBS J., 272, 4034–4043CrossRefGoogle Scholar
  70. 70.
    Maertens, J., Donckels, B., Lequeux, G. and Vanrolleghem, P. (2005) Metabolic model reduction by metabolite pooling on the basis of dynamic phase planes and metabolite correlation analysis. In Proceedings of the Conference on Modeling and Simulation in Biology, Medicine and Biomedical Engineering. Linkping, SwedenGoogle Scholar

Copyright information

© Higher Education Press and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Jenny E. Jeong
    • 1
  • Qinwei Zhuang
    • 2
  • Mark K. Transtrum
    • 3
  • Enlu Zhou
    • 4
  • Peng Qiu
    • 5
    • 6
    Email author
  1. 1.School of Electrical and Computer EngineeringGeorgia Institute of TechnologyAtlantaUSA
  2. 2.School of Biological SciencesGeorgia Institute of TechnologyAtlantaUSA
  3. 3.Department of Physics and AstronomyBrigham Young UniversityProvoUSA
  4. 4.School of Industrial and Systems EngineeringGeorgia Institute of TechnologyAtlantaUSA
  5. 5.Department of Biomedical EngineeringGeorgia Institute of TechnologyAtlantaUSA
  6. 6.Emory UniversityAtlantaUSA

Personalised recommendations