Skip to main content
SpringerLink
Log in

A Review of Deep Learning Architectures and Their Application

  • Conference paper
  • First Online:
Modeling, Design and Simulation of Systems (AsiaSim 2017)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 752))

Included in the following conference series:

Abstract

Deep Learning is a new era of machine learning research that are making major advances in solving problem with powerful computational models. Currently, this new machine learning method is widely used in object detection, visual object and speech recognition and also for making prediction of regulatory genomic and cellular imaging. Here, we review the methodology and applications of deep learning architectures including deep neural network, convolutional neural network and recurrent neural network. Next, we review several existing prediction tools in genomic sequences analysis that use deep learning architectures. In addition, we discuss the future research directions of deep learning.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  2. Larranaga, P., Calvo, B., Santana, R., Bielza, C., Galdiano, J., Inza, I., Lozano, J.A., Armañanzas, R., Santafé, G., Pérez, A., Robles, V.: Machine learning in bioinformatics. Brief. Bioinform. 7(1), 86–112 (2006)

    Article  Google Scholar 

  3. Abdullah, A., Deris, S., Hashim, S.Z.M., Jamil, H.M.: Graph partitioning method for functional module detections of protein interaction network. In: International Conference on Computer Technology and Development, ICCTD 2009, vol. 1, pp. 230–234. IEEE, November 2009

    Google Scholar 

  4. Ismail, M.A., Deris, S., Mohamad, M.S., Abdullah, A.: A Newton cooperative genetic algorithm method for in Silico optimization of metabolic pathway production. PLoS ONE 10(5), e0126199 (2015)

    Article  Google Scholar 

  5. Hayashi, N., Matsumae, M., Yatsushiro, S., Hirayama, A., Abdullah, A., Kuroda, K.: Quantitative analysis of cerebrospinal fluid pressure gradients in healthy volunteers and patients with normal pressure hydrocephalus. Neurol. Med. Chir. 55(8), 657–662 (2015)

    Article  Google Scholar 

  6. Abdullah, A., Hirayama, A., Yatsushiro, S., Matsumae, M., Kuroda, K.: Cerebrospinal fluid image segmentation using spatial fuzzy clustering method with improved evolutionary expectation maximization. In: 2013 35th Annual International Conference of IEEE Engineering in Medicine and Biology Society (EMBC), pp. 3359–3362. IEEE, July 2013

    Google Scholar 

  7. Farabet, C., Couprie, C., Najman, L., LeCun, Y.: Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1915–1929 (2013)

    Article  Google Scholar 

  8. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

    Google Scholar 

  9. Tompson, J.J., Jain, A., LeCun, Y., Bregler, C.: Joint training of a convolutional network and a graphical model for human pose estimation. In: Advances in Neural Information Processing Systems, pp. 1799–1807 (2014)

    Google Scholar 

  10. Liu, N., Han, J., Zhang, D., Wen, S., Liu, T.: Predicting eye fixations using convolutional neural networks. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 362–370 (2015)

    Google Scholar 

  11. Hinton, G., Deng, L., Yu, D., Dahl, G.E., Mohamed, A.R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T.N., Kingsbury, B.: Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Sig. Process. Mag. 29(6), 82–97 (2012)

    Article  Google Scholar 

  12. Sainath, T.N., Mohamed A.-R., Kingsbury, B., et al.: Deep convolutional neural networks for LVCSR. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), p. 8614–8. IEEE (2013)

    Google Scholar 

  13. Min, S., Lee, B., Yoon, S. Deep learning in bioinformatics. Brief. Bioinform. bbw068 (2016)

    Google Scholar 

  14. Kiros, R., Zhu, Y., Salakhutdinov, R.R., Zemel, R., Urtasun, R., Torralba, A., Fidler, S.: Skip-thought vectors. In: Advances in Neural Information Processing Systems, pp. 3294–3302 (2015)

    Google Scholar 

  15. Li, J., Luong, M.T., Jurafsky, D.: A hierarchical neural autoencoder for paragraphs and documents. arXiv preprint arXiv:1506.01057 (2015)

  16. Luong, M.T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 (2015)

  17. Zhang, S., Zhou, J., Hu, H., Gong, H., Chen, L., Cheng, C., Zeng, J.: A deep learning framework for modeling structural features of RNA-binding protein targets. Nucl. Acids Res. 44(4), e32–e32 (2016)

    Article  Google Scholar 

  18. Suk, H.-I., Shen, D.: Deep learning-based feature representation for AD/MCI classification. In: Mori, K., Sakuma, I., Sato, Y., Barillot, C., Navab, N. (eds.) MICCAI 2013. LNCS, vol. 8150, pp. 583–590. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40763-5_72

    Chapter  Google Scholar 

  19. Alipanahi, B., Delong, A., Weirauch, M.T., Frey, B.J.: Predicting the sequence specificities of DNA-and RNA-binding proteins by deep learning. Nat. Biotechnol. 33(8), 831–838 (2015)

    Article  Google Scholar 

  20. Zhou, J., Troyanskaya, O.G.: Predicting effects of noncoding variants with deep learning-based sequence model. Nat. Methods 12(10), 931–934 (2015)

    Article  Google Scholar 

  21. Hua, K.L., Hsu, C.H., Hidayati, S.C., Cheng, W.H., Chen, Y.J.: Computer-aided classification of lung nodules on computed tomography images via deep learning technique. OncoTargets Ther. 8, 2015–2022 (2014)

    Google Scholar 

  22. Roth, H.R., Lu, L., Liu, J., Yao, J., Seff, A., Cherry, K., Summers, R.M.: Improving computer-aided detection using <? Pub _newline ?> convolutional neural networks and random view aggregation. IEEE Trans. Med. Imaging 35(5), 1170–1181 (2016)

    Article  Google Scholar 

  23. Roth, H.R., Yao, J., Lu, L., Stieger, J., Burns, J.E., Summers, R.M.: Detection of sclerotic spine metastases via random aggregation of deep convolutional neural network classifications. In: Yao, J., Glocker, B., Klinder, T., Li, S. (eds.) Recent Advances in Computational Methods and Clinical Applications for Spine Imaging. LNCVB, vol. 20, pp. 3–12. Springer, Cham (2013). doi:10.1007/978-3-319-14148-0_1

    Chapter  Google Scholar 

  24. Baldi, P., Pollastri, G., Andersen, C.A., Brunak, S.: Matching protein b-sheet partners by feedforward and recurrent neural networks. In: ISMB, pp. 25–36 (2000)

    Google Scholar 

  25. Ogawa, S., Lee, T.M., Kay, A.R., Tank, D.W.: Brain magnetic resonance imaging with contrast dependent on blood oxygenation. Proc. Natl. Acad. Sci. 87(24), 9868–9872 (1990)

    Article  Google Scholar 

  26. Hsieh, J.: Computed Tomography: Principles, Design, Artifacts, and Recent Advances, vol. 114. SPIE Press, Bellingham (2003)

    Google Scholar 

  27. McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5(4), 115–133 (1943)

    Article  MathSciNet  MATH  Google Scholar 

  28. Farley, B.W.A.C., Clark, W.: Simulation of self-organizing systems by digital computer. Trans. IRE Prof. Group Inf. Theory 4(4), 76–84 (1954)

    Article  MathSciNet  Google Scholar 

  29. Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65, 386 (1958)

    Article  Google Scholar 

  30. Svozil, D., Kvasnicka, V., Pospichal, J.: Introduction to multi-layer feed-forward neural networks. Chemometr. Intell. Lab. Syst. 39(1), 43–62 (1997)

    Article  Google Scholar 

  31. Heffernan, R., Paliwal, K., Lyons, J., Dehzangi, A., Sharma, A., Wang, J., Sattar, A., Yang, Y., Zhou, Y.: Improving prediction of secondary structure, local backbone angles, and solvent accessible surface area of proteins by iterative deep learning. Sci. Rep. 5, 11476 (2015)

    Article  Google Scholar 

  32. Chen, Y., Li, Y., Narayan, R., Subramanian, A., Xie, X.: Gene expression inference with deep learning. Bioinformatics 32(12), 1832–1839 (2016)

    Article  Google Scholar 

  33. Fakoor, R., Ladhak, F., Nazi, A., Huber, M.: Using deep learning to enhance cancer diagnosis and classification. In: Proceedings of International Conference on Machine Learning (2013)

    Google Scholar 

  34. van Gerven, M.A., de Lange, F.P., Heskes, T.: Neural decoding with hierarchical generative models. Neural Comput. 22(12), 3127–3142 (2010)

    Article  MATH  Google Scholar 

  35. Koyamada, S., Shikauchi, Y., Nakae, K., Koyama, M., Ishii, S.: Deep learning of fMRI big data: a novel approach to subject-transfer decoding. arXiv preprint arXiv:1502.00093 (2010)

  36. Li, Q., Feng, B., Xie, L., Liang, P., Zhang, H., Wang, T.: A cross-modality learning approach for vessel segmentation in retinal images. IEEE Trans. Med. Imaging 35(1), 109–118 (2016)

    Article  Google Scholar 

  37. Xu, J., Xiang, L., Liu, Q., Gilmore, H., Wu, J., Tang, J., Madabhushi, A.: Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images. IEEE Trans. Med. Imaging 35(1), 119–130 (2016)

    Article  Google Scholar 

  38. Plis, S.M., Hjelm, D.R., Salakhutdinov, R., Calhoun, V.D.: Deep learning for neuroimaging: a validation study. arXiv preprint arXiv:1312.5847 (2013)

  39. Hubel, D.H., Wiesel, T.N.: Shape and arrangement of columns in cat’s striate cortex. J. Physiol. 165(3), 559 (1963)

    Article  Google Scholar 

  40. Hubel, D.H., Wiesel, T.N.: The period of susceptibility to the physiological effects of unilateral eye closure in kittens. J. Physiol. 206(2), 419 (1970)

    Article  Google Scholar 

  41. Angermueller, C., Pärnamaa, T., Parts, L., Stegle, O.: Deep learning for computational biology. Mol. Syst. Biol. 12(7), 878 (2016)

    Article  Google Scholar 

  42. Kelley, D.R., Snoek, J., Rinn, J.: Basset: learning the regulatory code of the accessible genome with deep convolutional neural networks. Genome Res. (2016). doi:10.1101/gr.200535.115

  43. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: Advances in Neural Information Processing Systems, pp. 3104–3112 (2014)

    Google Scholar 

  44. Liwicki, M., Graves, A., Bunke, H., Schmidhuber, J.: A novel approach to on-line handwriting recognition based on bidirectional long short-term memory networks. In: Proceedings of 9th International Conference on Document Analysis and Recognition, vol. 1, pp. 367–371 (2007)

    Google Scholar 

  45. Graves, A., Liwicki, M., Fernández, S., Bertolami, R., Bunke, H., Schmidhuber, J.: A novel connectionist system for unconstrained handwriting recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(5), 855–868 (2009)

    Article  Google Scholar 

  46. Wulsin, D.F., Gupta, J.R., Mani, R., Blanco, J.A., Litt, B.: Modeling electroencephalography waveforms with semi-supervised deep belief nets: fast classification and anomaly measurement. J. Neural Eng. 8(3), 036015 (2011)

    Article  Google Scholar 

  47. Luong, M.T., Pham, H., Manning, C.D.: Effective approaches to attention-based neural machine translation. arXiv preprint arXiv:1508.04025 (2015)

  48. Vincent, P., Larochelle, H., Bengio, Y., Manzagol, P.A.: Extracting and composing robust features with denoising autoencoders. In: Proceedings of 25th International Conference on Machine Learning, pp. 1096–1103. ACM (2008)

    Google Scholar 

  49. Quang, D., Xie, X.: DanQ: a hybrid convolutional and recurrent deep neural network for quantifying the function of DNA sequences. Nucl. Acids Res. 44(11), e107–e107 (2016)

    Article  Google Scholar 

  50. Tripathi, R., Patel, S., Kumari, V., Chakraborty, P., Varadwaj, P.K.: DeepLNC, a long non-coding RNA prediction tool using deep neural network. Netw. Model. Anal. Health Inform. Bioinform. 5(1), 1–14 (2016)

    Article  Google Scholar 

  51. Lanchantin, J., Singh, R., Lin, Z., Qi, Y.: Deep motif: visualizing genomic sequence classifications. arXiv preprint arXiv:1605.01133 (2016)

  52. Leung, M.K., Delong, A., Alipanahi, B., Frey, B.J.: Machine learning in genomic medicine: a review of computational problems and data sets. Proc. IEEE 104(1), 176–197 (2016)

    Article  Google Scholar 

  53. Abdullah, A., Deris, S., Mohamad, M.S., Anwar, S.: An improved swarm optimization for parameter estimation and biological model selection. PLoS ONE 8(4), e61258 (2013)

    Article  Google Scholar 

  54. Abdullah, A., Deris, S., Hashim, S.Z.M., Mohamad, M.S., Arjunan, S.N.V.: An improved local best searching in particle swarm optimization using differential evolution. In: 2011 11th International Conference on Hybrid Intelligent Systems (HIS), pp. 115–120. IEEE, December 2011

    Google Scholar 

Download references

Acknowledgments

We would like to express gratitude to the editor and reviewers for helpful suggestions and Malaysian Ministry of Higher Education (MOHE) for sponsoring this research. This research was supported by Fundamental Research Grant Scheme (FRGS), vot number 4F738 and managed by Research Management Centre (RMC), Universiti Teknologi Malaysia (UTM).

Author information

Authors and Affiliations

  1. Synthetic Biology Research Group, Faculty of Computing, Universiti Teknologi Malaysia, 81310, Johor Bharu, Johor, Malaysia

    Jalilah Arijah Mohd Kamarudin & Afnizanfaizal Abdullah

  2. Soft Computing Research Group, Faculty of Computing, Universiti Teknologi Malaysia, 81310, Johor Bharu, Johor, Malaysia

    Roselina Sallehuddin

Authors
  1. Jalilah Arijah Mohd Kamarudin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Afnizanfaizal Abdullah
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Roselina Sallehuddin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jalilah Arijah Mohd Kamarudin .

Editor information

Editors and Affiliations

  1. Universiti Teknologi Malaysia, Skudai, Malaysia

    Mohamed Sultan Mohamed Ali

  2. Universiti Teknologi Malaysia, Skudai, Malaysia

    Herman Wahid

  3. Universiti Teknologi Malaysia, Skudai, Malaysia

    Nurul Adilla Mohd Subha

  4. Universiti Teknologi Malaysia, Skudai, Malaysia

    Shafishuhaza Sahlan

  5. Universiti Teknologi Malaysia, Skudai, Malaysia

    Mohd Amri Md. Yunus

  6. Universiti Teknologi Malaysia, Skudai, Malaysia

    Ahmad Ridhwan Wahap

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper

Mohd Kamarudin, J.A., Abdullah, A., Sallehuddin, R. (2017). A Review of Deep Learning Architectures and Their Application. In: Mohamed Ali, M., Wahid, H., Mohd Subha, N., Sahlan, S., Md. Yunus, M., Wahap, A. (eds) Modeling, Design and Simulation of Systems. AsiaSim 2017. Communications in Computer and Information Science, vol 752. Springer, Singapore. https://doi.org/10.1007/978-981-10-6502-6_7

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-6502-6_7

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-6501-9

  • Online ISBN: 978-981-10-6502-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation