Abstract
Predictions on sequential data, when both the upstream and downstream information is important, is a difficult and challenging task. The Bidirectional Recurrent Neural Network (BRNN) architecture has been designed to deal with this class of problems. In this paper, we present the development and implementation of the Scaled Conjugate Gradient (SCG) learning algorithm for BRNN architectures. The model has been tested on the Protein Secondary Structure Prediction (PSSP) and Transmembrane Protein Topology Prediction problems (TMPTP). Our method currently achieves preliminary results close to 73 % correct predictions for the PSSP problem and close to 79 % for the TMPTP problem, which are expected to increase with larger datasets, external rules, ensemble methods and filtering techniques. Importantly, the SCG algorithm is training the BRNN architecture approximately 3 times faster than the Backpropagation Through Time (BPTT) algorithm.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Schuster, M., Paliwal, K.K.: IEEE Trans. Signal Proces. 45, 2673–2681 (1997)
Dietterich, T.G.: Machine learning for sequential data: a review. In: Caelli, T.M., Amin, A., Duin, R.P.W., Kamel, M.S., de Ridder, D. (eds.) SSPR&SPR 2002. LNCS, vol. 2396, pp. 15–30. Springer, Heidelberg (2002)
Elman, J.L.: Cogn. Sci. 14, 179–211 (1990)
Werbos, P.J.: Proc. IEEE 78(10), 1550–1560 (1990)
Frasconi, P., Gori, M., Sperduti, A.: IEEE Trans. Neural Netw. 9, 768–786 (1998)
Møller, M.F.: Neural Netw. 6, 525–533 (1993)
Hochreiter, S., Schmidhuber, J.: Neural Comput. 9, 1735–1780 (1997)
Baldi, P., Brunak, S., Frasconi, P., Soda, G., Pollastri, G.: Bioinformatics 15, 937–946 (1999)
Kountouris, P., Agathocleous, M., Promponas, V., Christodoulou, G., Hadjicostas, S., Vassiliades, V., Christodoulou, C.: IEEE ACM Trans. Comput. Biol. Bioinform. 9, 731–739 (2012)
Agathocleous, M., Christodoulou, G., Promponas, V., Christodoulou, C., Vassiliades, V., Antoniou, A.: Protein secondary structure prediction with Bidirectional recurrent neural nets: can weight updating for each residue enhance performance? In: Papadopoulos, H., Andreou, A.S., Bramer, M. (eds.) AIAI 2010. IFIP AICT, vol. 339, pp. 128–137. Springer, Heidelberg (2010)
Nugent, T., Jones, D.T.: BMC Bioinf. 10, 159 (2009)
Altschul, S.F., Madden, T.L., Schäffer, A.A., Zhang, A., Zhang, Z., Miller, W., Lipman, D.J.: Nucleic Acids Res. 25, 3389–3402 (1997)
Cuff, J.A., Barton, G.J.: Proteins 34, 508–519 (1999)
Richards, F., Kundrot, C.: Proteins 3, 71–84 (1988)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Agathocleous, M., Christodoulou, C., Promponas, V., Kountouris, P., Vassiliades, V. (2016). Training Bidirectional Recurrent Neural Network Architectures with the Scaled Conjugate Gradient Algorithm. In: Villa, A., Masulli, P., Pons Rivero, A. (eds) Artificial Neural Networks and Machine Learning – ICANN 2016. ICANN 2016. Lecture Notes in Computer Science(), vol 9886. Springer, Cham. https://doi.org/10.1007/978-3-319-44778-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-44778-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-44777-3
Online ISBN: 978-3-319-44778-0
eBook Packages: Computer ScienceComputer Science (R0)