Reconstructing Genetic Regulatory Networks Using Two-Step Algorithms with the Differential Equation Models of Neural Networks

  • Chi-Kan ChenEmail author
Original Research Article



The identification of genetic regulatory networks (GRNs) provides insights into complex cellular processes. A class of recurrent neural networks (RNNs) captures the dynamics of GRN. Algorithms combining the RNN and machine learning schemes were proposed to reconstruct small-scale GRNs using gene expression time series.


We present new GRN reconstruction methods with neural networks. The RNN is extended to a class of recurrent multilayer perceptrons (RMLPs) with latent nodes. Our methods contain two steps: the edge rank assignment step and the network construction step. The former assigns ranks to all possible edges by a recursive procedure based on the estimated weights of wires of RNN/RMLP (RERNN/RERMLP), and the latter constructs a network consisting of top-ranked edges under which the optimized RNN simulates the gene expression time series. The particle swarm optimization (PSO) is applied to optimize the parameters of RNNs and RMLPs in a two-step algorithm. The proposed RERNN-RNN and RERMLP-RNN algorithms are tested on synthetic and experimental gene expression time series of small GRNs of about 10 genes. The experimental time series are from the studies of yeast cell cycle regulated genes and E. coli DNA repair genes.


The unstable estimation of RNN using experimental time series having limited data points can lead to fairly arbitrary predicted GRNs. Our methods incorporate RNN and RMLP into a two-step structure learning procedure. Results show that the RERMLP using the RMLP with a suitable number of latent nodes to reduce the parameter dimension often result in more accurate edge ranks than the RERNN using the regularized RNN on short simulated time series. Combining by a weighted majority voting rule the networks derived by the RERMLP-RNN using different numbers of latent nodes in step one to infer the GRN, the method performs consistently and outperforms published algorithms for GRN reconstruction on most benchmark time series. The framework of two-step algorithms can potentially incorporate with different nonlinear differential equation models to reconstruct the GRN.


GRN Reconstruction Gene expression time series RNN RMLP Two-step algorithm 


  1. 1.
    Berestovsky N, Nakhleh L (2013) An evaluation of methods for inferring boolean networks from time-series data. PLoS One 8(6):e66031CrossRefGoogle Scholar
  2. 2.
    Liang S, Fuhrman S, Somogyi R (1998) REVEAL, a general reverse engineering algorithm for inference of genetic network architectures. Pac Symp Biocomput 3:18–29Google Scholar
  3. 3.
    Friedman N et al (2000) Using Bayesian networks to analyze expression data. J Comput Biol 7(3–4):601–620CrossRefGoogle Scholar
  4. 4.
    Werhli AV and Husmeier, D (2007) Reconstructing gene regulatory networks with bayesian networks by combining expression data with multiple sources of prior knowledge. Stat Appl Genet Mol Biol 6 (Article 15)Google Scholar
  5. 5.
    Bansal M, Gatta GD, di Bernardo D (2006) Inference of gene regulatory networks and compound mode of action from time course gene expression profiles. Bioinformatics 22(7):815–822CrossRefGoogle Scholar
  6. 6.
    Guo S et al (2016) Gene regulatory network inference using PLS-based methods. BMC Bioinform 17(1):545CrossRefGoogle Scholar
  7. 7.
    Lo K et al (2012) Integrating external biological knowledge in the construction of regulatory networks from time-series expression data. BMC Syst Biol 6:101CrossRefGoogle Scholar
  8. 8.
    Young WC, Raftery AE, Yeung KY (2014) Fast Bayesian inference for gene regulatory networks using ScanBMA. BMC Syst Biol 8:47CrossRefGoogle Scholar
  9. 9.
    Kikuchi S et al (2003) Dynamic modelling of genetic networks using genetic algorithm and S-system. Bioinformatics 19:643–650CrossRefGoogle Scholar
  10. 10.
    Kimura S et al (2005) Inference of S-system models of genetic networks using a cooperative coevolutionary algorithm. Bioinformatics 21:1154–1163CrossRefGoogle Scholar
  11. 11.
    Margolin AA et al (2006) ARACNE: an algorithm for the reconstruction of gene regulatory networks in a mammalian cellular context. BMC Bioinform 7(Suppl 1):S7CrossRefGoogle Scholar
  12. 12.
    Meyer PE et al (2007) Information-theoretic inference of large transcriptional regulatory networks. EURASIP J Bioinform Syst Biol 27:79879Google Scholar
  13. 13.
    Bansal M, di Bernardo D (2007) Inference of gene networks from temporal gene expression profiles. IET Syst Biol 1(5):306–312CrossRefGoogle Scholar
  14. 14.
    Bezerianos A, Maraziotis IA (2008) Computational models reconstruct gene regulatory networks. Mol BioSyst 4(10):993–1000CrossRefGoogle Scholar
  15. 15.
    Hecker M et al (2009) Gene regulatory network inference: data integration in dynamic models-a review. Biosystems 96(1):86–103CrossRefGoogle Scholar
  16. 16.
    Omony J (2014) Biological network inference: a review of methods and assessment of tools and techniques. Annu Res Rev Biol 4(4):577–601CrossRefGoogle Scholar
  17. 17.
    Vohradsky J (2001) Neural model of the genetic network. J Biol Chem 276(39):36168–36173CrossRefGoogle Scholar
  18. 18.
    Werbos PJ (1990) Backpropagation through time: what it does and how to do it. Proc IEEE 78(10):1550–1560CrossRefGoogle Scholar
  19. 19.
    Chiang JH, Chao SY (2007) Modeling human cancer-related regulatory modules by GA-RNN hybrid algorithms. BMC Bioinform 8:91CrossRefGoogle Scholar
  20. 20.
    Wahde M, Hertz J (2000) Coarse-grained reverse engineering of genetic regulatory networks. BioSystems 55:129–136CrossRefGoogle Scholar
  21. 21.
    Xu R, Wunsch DC II, Frank RL (2007) Inference of genetic regulatory networks with recurrent neural network models using particle swarm optimization. IEEE/ACM Trans Comput Biol Bioinform 4(4):681–692CrossRefGoogle Scholar
  22. 22.
    Zhang Y et al (2009) Reverse engineering module networks by PSO-RNN hybrid modeling. BMC Genom 10(Suppl 1):S15CrossRefGoogle Scholar
  23. 23.
    Noman N, Palafox L, Iba H (2013) Reconstruction of gene regulatory networks from gene expression data using decoupled recurrent neural network model. In: Suzuki Y, Nakagaki T (eds) Natural computing and beyond, Winter school Hakodate 2011, Hakodate, Japan, March 2011 and 6th international workshop on natural computing, Tokyo, Japan, March 2012, proceedings. Springer Japan, Tokyo, pp 93–103Google Scholar
  24. 24.
    Khan A et al (2016) Construction of gene regulatory networks using recurrent neural networks and swarm intelligence. Scientifica 2016:1060843Google Scholar
  25. 25.
    Maraziotis I, Dragomirn A, and Bezerianos A (2005) Recurrent neural-fuzzy network models for reverse engineering gene regulatory interactions. In: First international symposium, CompLife 2005. 2005, Konstanz, Germany, SpringerGoogle Scholar
  26. 26.
    Eberhart RC and Kennedy J (1995) A new optimizer using particle swarm theory. In: Proceedings of the sixth international symposium on micromachine and human science, 1995, pp 39–43Google Scholar
  27. 27.
    Gupta M, Jin L, Homma N (2003) Static and dynamic neural networks: from fundamentals to advanced theory. Wiley, New JerseyCrossRefGoogle Scholar
  28. 28.
    Nocedal J, Wright SJ (2006) Numerical optimization, 2nd edn. Springer-Verlag, New YorkGoogle Scholar
  29. 29.
    R Core Team (2017) R: A language and environment for statistical computing R. Foundation for Statistical Computing, Vienna, AustriaGoogle Scholar
  30. 30.
    Clerc M (2012) Standard Particle Swarm Optimisation. Particle Swarm Central, Tech. Rep. Accessed 24 Sept 2012
  31. 31.
    Barabasi AL, Albert R (1999) Emergence of scaling in random networks. Science 286:509–512CrossRefGoogle Scholar
  32. 32.
    Mendes P, Sha W, Ye K (2003) Artificial gene networks for objective comparison of analysis algorithms. Bioinformatics 19(Suppl 2):122–129CrossRefGoogle Scholar
  33. 33.
    Marbach D et al (2009) Generating realistic in silico gene networks for performance assessment of reverse engineering methods. J Comput Biol 16(2):229–239CrossRefGoogle Scholar
  34. 34.
    Schaffter T, Marbach D, Floreano D (2011) GeneNetWeaver: in silico benchmark generation and performance profiling of network inference methods. Bioinformatics 27(16):2263–2270CrossRefGoogle Scholar
  35. 35.
    GNW DREAM Challenges. Accessed 7 Jan 2017
  36. 36.
    Marbach D, Mattiussi C, Floreano D (2009) Combining multiple results of a reverse-engineering algorithm: application to the DREAM five-gene network challenge. Ann NY Acad Sci 1158:102–113CrossRefGoogle Scholar
  37. 37.
    Li F et al (2004) The yeast cell-cycle network is robustly designed. Proc Natl Acad Sci USA 101(14):4781–4786CrossRefGoogle Scholar
  38. 38.
    Spellman PT et al (1998) Comprehensive identification of cell cycle-regulated genes of the yeast Saccharomyces cerevisiae by microarray hybridization. Mol Biol Cell 9(12):3273–3297CrossRefGoogle Scholar
  39. 39.
    Yeast cell cycle analysis project. Accessed 7 Jan 2017
  40. 40.
    Ronen M et al (2002) Assigning numbers to the arrows: Parameterizing a gene regulation network by using accurate expression kinetics. Proc Natl Acad Sci USA 99(16):10555–10560CrossRefGoogle Scholar
  41. 41.
  42. 42.
    Marsaglia G, Tsang WW, Wang J (2003) Evaluating Kolmogorov’s distribution. J Stat Softw 18(8):1–4Google Scholar
  43. 43.
    Mazur J et al (2009) Reconstructing nonlinear dynamic models of gene regulation using stochastic sampling. BMC Bioinform 10:448CrossRefGoogle Scholar
  44. 44.
    Bock M et al (2012) Hub-centered gene network reconstruction using automatic relevance determination. PLoS One 7(5):e35077CrossRefGoogle Scholar
  45. 45.
    Charbonnier C, Chiquet J and Ambroise C (2010) Weighted-LASSO for structured network inference from time course data. Stat Appl Genet Mol 9 (Article 15)Google Scholar
  46. 46.
    Kaderali L et al (2006) CASPAR: a hierarchical Bayesian approach to predict survival times in cancer from gene expression data. Bioinformatics 22(12):1495–1502CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.Department of Applied MathematicsNational Chung Hsing UniversityTaichung CityTaiwan

Personalised recommendations