Abstract
Recognition of difficult patterns with the accuracy comparable to that of the human brain is a challenging problem. The ability of the human to excel at this task has motivated the use of Artificial Neural Networks (ANNs) which under certain conditions provide efficient solutions. ANNs are still unable to use the full potential of modular and holistic operations of biological neurons and their networks. The ability of neurons to transfer learned behaviour has inspired an idea to train ANN for a new task by using the behaviour patterns learnt from a related task. The useful patterns transferred from one task to another can significantly reduce the time needed to learn new patterns, and gives the neurons the ability to generalise instead of memorising patterns. In this paper we explore the ability of transfer learning for a face recognition problem by using Group Method of Data Handling (GMDH) type of Deep Neural Networks. In our experiments we show that the transfer learning of a GMDH-type neural network has reduced the training time by 31% on a face recognition benchmark.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Ciresan, D.C., Meier, U., Masci, J., Gambardella, M.: Flexible, high performance convolutional neural networks for image classification. In: International Joint Conference on Artificial Intelligence (IJCAI), pp. 1237–1242 (2011)
Dhawan, P., Dongre, S., Tidke, D.J.: Hybrid GMDH model for handwritten character recognition. In: International Mutli-conference on Automation, Computing, Communication, Control and Compressed Sensing (iMac4s), pp. 698–703 (2013)
Farfade, S.S., Saberian, M.J., Li, L.J.: Multi-view face detection using deep convolutional neural networks. In: The 5th ACM on International Conference on Multimedia Retrieval (ICMR), pp. 643–650. ACM (2015)
Ijjina, E.P., Mohan, C.K.: Facial expression recognition using Kinect depth sensor and convolutional neural networks. In: IEEE International Conference on Machine Learning and Applications (ICMLA), vol. 2014, pp. 392–396 (2014)
Ivakhnenko, A.: Polynomial theory of complex systems. IEEE Trans. Syst. Man Cybern. SMC–1(4), 364–378 (1971)
Jakaite, L., Schetinin, V.: Feature selection for Bayesian evaluation of trauma death risk. In: 14th Nordic-Baltic Conference on Biomedical Engineering and Medical Physics (NBC), pp. 123–126 (2008). https://doi.org/10.1007/978-3-540-69367-3_33
Jakaite, L., Schetinin, V., Maple, C.: Bayesian assessment of newborn brain maturity from two-channel sleep electroencephalograms. Comput. Math. Meth. Med. 2012, 1–7 (2012)
Jakaite, L., Schetinin, V., Schult, J.: Feature extraction from electroencephalograms for Bayesian assessment of newborn brain maturity. In: 24th International Symposium on Computer-Based Medical Systems (CBMS), pp. 1–6 (2011). https://doi.org/10.1109/CBMS.2011.5999109
Kawewong, A., Tangruamsub, S., Kankuekul, P., Hasegawa, O.: Fast online incremental transfer learning for unseen object classification using self-organizing incremental neural networks. In: IEEE International Conference on Neural Networks (IJCNN), pp. 749–756 (2011)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Nebauer, C.: Evaluation of convolutional neural networks for visual recognition. IEEE Trans. Neural Netw. 9(4), 685–696 (1998)
Nyah, N., Jakaite, L., Schetinin, V., Sant, P., Aggoun, A.: Learning polynomial neural networks of a near-optimal connectivity for detecting abnormal patterns in biometric data. In: 2016 SAI Computing Conference, pp. 409–413 (2016). https://doi.org/10.1109/SAI.2016.7556014
Raina, R., Ng, A.Y., Koller, D.: Constructing informative priors using transfer learning. In: The 23rd International Conference on Machine Learning (ICML), pp. 713–720 (2006)
Samaria, F.S., Harter, A.C.: Parameterisation of a stochastic model for human face identification. In: The IEEE Workshop on Applications of Computer Vision, pp. 138–142 (1994)
Schetinin, V., Jakaite, L.: Extraction of features from sleep EEG for Bayesian assessment of brain development. PLOS ONE 12(3), e0174027 (2017). https://doi.org/10.1371/journal.pone.0174027
Schetinin, V., Jakaite, L., Krzanowski, W.J.: Prediction of survival probabilities with Bayesian decision trees. Expert Syst. Appl. 40(14), 5466–5476 (2013). https://doi.org/10.1016/j.eswa.2013.04.009
Schetinin, V., Jakaite, L., Nyah, N., Novakovic, D., Krzanowski, W.: Feature extraction with GMDH-type neural networks for EEG-based person identification. Int. J. Neural Syst. 28(6), 1750064 (2018). https://doi.org/10.1142/S0129065717500642
Schetinin, V., Schult, J.: A neural-network technique to learn concepts from electroencephalograms. Theory Biosci. 124(1), 41–53 (2005). https://doi.org/10.1016/j.thbio.2005.05.004
Schetinin, V., Schult, J.: Learning polynomial networks for classification of clinical electroencephalograms. Soft Comput. 10(4), 397–403 (2006). https://doi.org/10.1007/s00500-005-0499-3
Schetinin, V., Schult, J., Scheidt, B., Kuriakin, V.: Learning multi-class neural-network models from electroencephalograms. In: Knowledge-Based Intelligent Information and Engineering Systems (KES), pp. 155–162 (2003)
Schönborn, S., Egger, B., Morel-Forster, A., Vetter, T.: Markov chain monte carlo for automated face image analysis. Int. J. Comput. Vis. 123(2), 160–183 (2017)
Sun, Y., Wang, X., Tang, X.: Deep convolutional network cascade for facial point detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3476–3483 (2013)
Taigman, Y., Yang, M., Ranzato, M., Wolf, L.: Deepface: closing the gap to human-level performance in face verification. In: The IEEE Conference on Computer Vision and Pattern Recognition, pp. 1701–1708 (2014)
Uglov, J., Jakaite, L., Schetinin, V., Maple, C.: Comparing robustness of pairwise and multiclass neural-network systems for face recognition. EURASIP J. Adv. Signal Process. 2008, 7 (2008)
Valenti, R., Sebe, N., Gevers, T., Cohen, I.: Machine learning techniques for face analysis. In: Anonymous Machine Learning Techniques for Multimedia, pp. 159–187 (2008)
Xu, Y., Du, J., Dai, L., Lee, C.: Cross-language transfer learning for deep neural network based speech enhancement. In: IEEE International Symposium on Chinese Spoken Language Processing (ISCSLP), vol. 9, pp. 336–340 (2014)
Zhao, W., Chellappa, R., Phillips, P.J., Rosenfeld, A.: Face recognition: a literature survey. ACM Comput. Surv. 35(4), 399–458 (2003). https://doi.org/10.1145/954339.954342
Zharkova, V.V., Schetinin, V.: A neural-network technique for recognition of filaments in solar images. In: 7th International Conference on Knowledge-Based Intelligent Information and Engineering Systems, (KES), pp. 148–154 (2003). https://doi.org/10.1007/978-3-540-45224-9_22
Acknowledgements
The authors would like to thank Dr Livija Jakaite, a member of the supervisory team at the School of Computer Science of University of Bedfordshire, for useful and constructive comments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Abdullahi, A., Akter, M. (2019). Transfer Learning in GMDH-Type Neural Networks. In: Choroś, K., Kopel, M., Kukla, E., Siemiński, A. (eds) Multimedia and Network Information Systems. MISSI 2018. Advances in Intelligent Systems and Computing, vol 833. Springer, Cham. https://doi.org/10.1007/978-3-319-98678-4_18
Download citation
DOI: https://doi.org/10.1007/978-3-319-98678-4_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-98677-7
Online ISBN: 978-3-319-98678-4
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)