Higher Order Neural Network and Its Applications: A Comprehensive Survey

  • Radha Mohan Pattanayak
  • H. S. Behera
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 710)


Over the years, neural networks have shown its strength in various fields of research. There is a vast improvement in the efficiency and effectiveness of various classification techniques mainly with the introduction of higher order neural networks. Due to great learning and storage capacity with grater computational ability than the existing traditional neural networks, nowadays, researchers are very much attracted toward the higher order neural network due to their nonlinear mapping ability with less number of input units. In this paper, a comprehensive survey on Pi-Sigma higher order neural network and its different applications to various domains over more than a decade has been reviewed. These techniques are vastly used in classification and regression in several domains including medical, time series forecasting, image processing, and engineering. The extensive survey provides a recent development in higher order neural network and its applications in several application domains.


Pi-Sigma neural network (PSNN) Jordan Pi-Sigma neural network (JPSNN) Ridge polynomial neural network (RPNN) Dynamic ridge polynomial neural network (DRPNN) Recurrent Pi-Sigma neural network (RPSNN) 


  1. 1.
    Lippmann R.P. (1989) Pattern Classification using Neural Networks, IEEE Communications Magazine, 47–50.Google Scholar
  2. 2.
    Reid M.B. (1989) Rapid Training of Higher-order Neural Networks for Invariant Pattern Recognition, Processing of IJCNN Washington D.C, 1:689–692.Google Scholar
  3. 3.
    Ivakhnenko A.G. (1971) Polynomial theory of complex systems, IEEE transactions on Systems, Man, and Cybernetics, 1(4):364–378.Google Scholar
  4. 4.
    Giles C.L., Maxwell T. (1987) Learning, invariance, and generalization in high-order neural networks, 26:4972–4978.Google Scholar
  5. 5.
    Mats B. (1990) Higher order artificial neural networks. DIANE Publishing Company, Darby PA, USA, ISBN 0941375927.Google Scholar
  6. 6.
    Lippmann. R. P (1987) An introduction to computing with neural nets. IEEE ASSP Magazine, 4–22.Google Scholar
  7. 7.
    Shin Y., Ghosh, J. (1991) Realization of boolean functions using binary pi-sigma networks. In: Dagli, C.H., Kumara, S.R.T., Shin, Y.C. (eds.) Intelligent engineering systems through artificial neural networks, ASME Press, New York, 205–210.Google Scholar
  8. 8.
    Shin Y., Ghosh J. (1991) The Pi-Sigma Networks: An Efficient Higher-Order Neural Network for Pattern Classification and Function Approximation, In: Proceedings of International Joint Conference on Neural Networks, Seattle, WA, USA, 13–18.Google Scholar
  9. 9.
    Shin Y., Ghosh J. (1992) Approximation of multivariate functions using ridge polynomial networks, In Neural Networks, IJCNN., International Joint Conference on, 380–385.Google Scholar
  10. 10.
    Shin Y. (1992) Efficient higher-order feed forward networks for function approximation and classification (Doctoral dissertation, University of Texas at Austin).Google Scholar
  11. 11.
    Ghosh J., Shin Y. (1992) Efficient higher-order neural networks for classification and function approximation, Int J Neural Syst, 4(3):323–350.Google Scholar
  12. 12.
    Shin Y., Ghosh J., Samani D. (1992) Computationally efficient invariant pattern recognition with higher order Pi-Sigma Networks, The University of Texas at Austin.Google Scholar
  13. 13.
    Shin Y., Ghosh J., Samani D. (1992) computationally efficient invariant pattern classification with higher order pi-sigma networks, in Intelligent Engineering Systems through Artificial Neural Networks, C. H. Dagli, L. I. Burke, and Y. C. Shin, Eds. New York: ASME Press, 2: 379–384.Google Scholar
  14. 14.
    Shin Y., Ghosh I. (1995) Ridge Polynomial Networks, IEEE Transactions on Neural Networks, 6(3): 610–622.Google Scholar
  15. 15.
    Shin Y., Jin K-S. Yoon Byung-Moon. (1997) A complex pi-sigma network and its application to equalization of nonlinear satellite channels Neural Networks, International Conference on, 1:148–152.Google Scholar
  16. 16.
    Medsker L., Jain L.C. (1999) Recurrent neural networks: design and applications. CRC press.Google Scholar
  17. 17.
    Hussain A.J., Liatsis P. (2002) Recurrent pi-sigma networks for DPCM image coding, Neurocomputing, 55:363–382.Google Scholar
  18. 18.
    Voutriaridis C., Boutalis Y.S., Mertzios G. (2003) Ridge Polynomial Networks in pattern recognition, EC-VIP-MC 2003, 4th EURASIP Conference focused on Video/Image Processing and Multimedia Communications, Croatia, 519–524.Google Scholar
  19. 19.
    Ghazali R., Hussain A., El-Dereby W. (2006) Application of Ridge Polynomial Neural Networks to Financial Time Series Prediction, International Joint Conference on Neural Networks (IJCNN), 913–920.Google Scholar
  20. 20.
    Xiu J., Chang-Liang Xia. (2007) Modeling of Switched Reluctance Motor Based on Pi-sigma Neural Network, IEEE International Symposium on Industrial Electronics, 1258–1263.Google Scholar
  21. 21.
    Ge S., Peng C., Miao X. (2008) Visual Cryptography Scheme Using Pi-sigma Neural Networks, International Symposium on Information Science and Engineering, 2: 679–682.Google Scholar
  22. 22.
    Nie Y., Deng W. (2008) A Hybrid Genetic Learning Algorithm for Pi-Sigma Neural Network and the Analysis of Its Convergence, Fourth International Conference on Natural Computation, 3:19–23.Google Scholar
  23. 23.
    Husssain A.J., Jameel A.J., Al-Jumeily D., Ghazali R. (2009) Speech prediction using higher order neural networks, International Conference on Innovations in Information Technology (IIT), 294–298.Google Scholar
  24. 24.
    Ghazali R., Hussain A. J., Al-Jumeily D., & Merabti M. (2007, April) Dynamic ridge polynomial neural networks in exchange rates time series forecasting. In International Conference on Adaptive and Natural Computing Algorithms, Springer, Berlin, Heidelberg, 123–132.Google Scholar
  25. 25.
    Ghazali R., Hussain A.J., Al-Jumeily D., Lisboa P. (2009)Time series prediction using dynamic ridge polynomial neural networks, In Developments in eSystems Engineering (DESE), Second International Conference on, 354–363.Google Scholar
  26. 26.
    Ghazali R., Hussain A.J., Nawi N.M., Mohamad B. (2009) Non-stationary and stationary prediction of financial time series using dynamic ridge polynomial neural network, Neurocomputing, 72(10): 2359–2367.Google Scholar
  27. 27.
    Ghazali R., Jumeily D. (2009) Application of Pi-Sigma Neural Networks and Ridge Polynomial Neural networks to Financial Time Series Prediction, In: Zhang, M. (ed.) Artificial Higher order Neural Networks for Economics and Business, Information Science Reference, 271–293.Google Scholar
  28. 28.
    Ghazali R., Hussain A., Nawi N.M. (2010) Dynamic ridge polynomial higher order neural network, Artificial Higher Order Neural Networks for Computer Science and Engineering, 255–268.Google Scholar
  29. 29.
    Husaini N.A., Ghazali R., Nawi N.M. and Ismail L.H. (2011) Jordan Pi-sigma neural network for temperature prediction, in Ubiquitous Computing and Multimedia Applications. Springer, Berlin, Heidelberg, 547–558.Google Scholar
  30. 30.
    Husaini N.A., Ghazali R., Nawi N.M., Ismail L.H. (2011) Pi-Sigma Neural Network for Temperature Forecasting in Batu Pahat, In: Zain, J.M., Wan Mohd, W.M.b., El-Qawasmeh, E. (eds.) ICSECS, Part II. CCIS Springer, Heidelberg, 180: 530–541.Google Scholar
  31. 31.
    Ghazali R., Husaini N.A., Ismail L.H., Samsuddin N.A. (2012) An application of Jordan Pi-sigma neural network for the prediction of temperature time series signal, Recurrent Neural Networks and Soft Computing, 13(4):275–290.Google Scholar
  32. 32.
    Yu X., FengChen Q. (2012) Convergence of gradient method with penalty for Ridge Polynomial neural network, Neurocomputing, (97): 405–409.Google Scholar
  33. 33.
    Cao Q., Ewing B.T., Thompson M.A. (2012) Forecasting wind speed with recurrent neural networks. Eur. J. Oper. Res, 221(1): 148–154.Google Scholar
  34. 34.
    Panigrahi S., Pandey S., Singh R. (2013) A Novel Evolutionary Higher Order Neural Network for Pattern Classification, International Journal of Engineering Research and Technology, 2(9).Google Scholar
  35. 35.
    Karali Y., Panigrahi S., Behera H.S. (2013) A novel Differential evolution based algorithm for higher order neural network training, Journal of Theoretical & Applied Information Technology, 56(3).Google Scholar
  36. 36.
    Sahu K.K., Panigrahi S., Behera H.S. (2013) A Novel Chemical Reaction Optimization Algorithm For Higher Order Neural Network Training, Journal of Theoretical & Applied Information Technology, 53(3).Google Scholar
  37. 37.
    Panigrahi S., Behera H.S. (2013) Effect of Normalization Techniques on Univariate Time Series Forecasting using Evolutionary Higher Order Neural Network, Int. J. Eng. Adv. Technol 3(2): 280–285.Google Scholar
  38. 38.
    Nayak J., Naik B., Behera H.S. (2014) A hybrid PSO-GA based Pi sigma neural network (PSNN) with standard back propagation gradient descent learning for classification. Control, Instrumentation, Communication and Computational Technologies (ICCICCT), International Conference on. IEEE (2014).Google Scholar
  39. 39.
    Behera N.K.S., Behera H.S. (2014) Firefly based ridge polynomial neural network for classification, IEEE International Conference on Advanced Communications, Control and Computing Technologies, 1110–1113.Google Scholar
  40. 40.
    Nayak J., Kanungo D.P., Naik B., Behera H.S. (2014) A higher order evolutionary Jordan Pi-Sigma Neural Network with gradient descent learning for classification, International Conference on High Performance Computing and Applications (ICHPCA), 1–6.Google Scholar
  41. 41.
    Nayak S. C., Misra B.B., Behera H.S. (2015) A Pi-Sigma Higher Order Neural Network for Stock Index Forecasting. Computational Intelligence in Data Mining, Springer India, 2: 311–319.Google Scholar
  42. 42.
    Nayak J., Naik B., Behera H.S. (2016) A novel nature inspired firefly algorithm with higher order neural network: Performance analysis, Engineering Science and Technology, an International Journal, 19(1):197–211.Google Scholar
  43. 43.
    Nayak J., Naik B., Behera H.S. (2016) Optimizing a higher order neural network through teaching learning based optimization algorithm, Computational Intelligence in Data Mining, Springer India, (1):57–71.Google Scholar
  44. 44.
    Nayak J., Naik B., Behera H.S. (2016) Solving nonlinear Classification problems with black hole optimization and higher order Jordan Pi-Sigma neural network: a novel approach, 2(4): 236–251.Google Scholar
  45. 45.
    Panigrahi S. (2017) A Novel Hybrid Chemical Reaction Optimization Algorithm with Adaptive Differential Evolution Mutation Strategies for Higher Order Neural Network Training, International Arab Journal of Information Technology (IAJIT), 14(1).Google Scholar
  46. 46.
    Waheeb W., Ghazali R., Herawan T. (2017) Time Series Forecasting Using Ridge Polynomial Neural Network with Error Feedback, In: Herawan T., Ghazali R., Nawi N., Deris M. (eds) Recent Advances on Soft Computing and Data Mining. SCDM 2016. Advances in Intelligent Systems and Computing Springer, Cham, 549.Google Scholar
  47. 47.
    Schmitt M. (2002) On the complexity of computing and learning with multiplicative neurons, Neural Computation, 14(2):241–301.Google Scholar
  48. 48.
    Jordan M.I. (1986) Attractor Dynamics and Parallelism in a Connectionist Sequential Machine, Proceedings of the Eighth Conference of the Cognitive Science Society, New Jersey, USA.Google Scholar
  49. 49.
    Husaini N.A., Ghazali R., Ismail L.H., Herawan T. (2014) A Jordan Pi-Sigma Neural Network for Temperature Forecasting in Batu Pahat Region. In: Herawan T., Ghazali R., Deris M. (eds) Recent Advances on Soft Computing and Data Mining, Advances in Intelligent Systems and Computing Springer, Cham, 287.Google Scholar
  50. 50.
    Ghazali R. (2007) Higher order neural networks for financial time series prediction, PhD diss., Liverpool John Moores University.Google Scholar
  51. 51.
    Liu Y., Yang J., Yang D., Wu W. (2014) A modified gradient-based neuro-fuzzy learning algorithm for pi-sigma network based on first-order takagi-sugeno system. J Math Res Appl, 34(1): 114–126.Google Scholar
  52. 52.
    Ghazali R., Nazri M.N., Mohd N.M.S. (2011) Dynamic Ridge Polynomial Neural Network with a Real Time Recurrent Learning Algorithm: Forecasting the S&P 500 (<Special Issue> SOFT COMPUTING METHODOLOGIES AND ITS APPLICATIONS), Biomedical fuzzy and human sciences: the official journal of the Biomedical Fuzzy Systems Association, 16(2): 97–103.Google Scholar
  53. 53.
    Ghazali R., Hussain A.J., Liatsis P. (2011) Dynamic Ridge Polynomial Neural Network: Forecasting the univariate non-stationary and stationary trading signals, Expert Systems with Applications, 38(4): 3765–3776.Google Scholar
  54. 54.
    Barbounis T., Theocharis J. (2007) A locally recurrent fuzzy neural network with application to the wind speed prediction using spatial correlation. Neurocomputing 70: 1525–1452.Google Scholar
  55. 55.
    Rao R.V., Savsani V.J., Vakharia D.P. (2012) Teaching–learning-based optimization: an optimization method for continuous non-linear large scale problems, Information sciences 183(1):1–15.Google Scholar
  56. 56.
    L. Spirkovska and M.B. Reid, (1990) Connectivity Strategies for Higher-order Neural Networks applied to Pattern Recognition, Proceedings of IJCNN, San Diego, 1:21–26.Google Scholar
  57. 57.
    Hara Y. (1994) Application of neural networks to radar image classification, IEEE Transactions on Geoscience and Remote Sensing, 32(1): 100–109.Google Scholar
  58. 58.
    Hsieh T.J., Hsiao H.F., Yeh W.C. (2011) Forecasting stock markets using wavelet transforms and recurrent neural networks: an integrated system based on artificial bee colony algorithm. Appl. Soft. Comput, 11(2): 2510–2525.Google Scholar
  59. 59.
    D.S. Huang, H.H.S. lp, K.C.K. Law and Z. Chi (2005) Zeroing polynomials using modified constrained neural network approach. IEEE Transactions on neural networks, 16(3):721–732.Google Scholar
  60. 60.
    S. Perantonis, N. Ampazis, S. Varoufakis and G. Antoniou (1998) Constrained learning in neural networks, Application to factorization of 2-d polynomials, Neural Processing Letter, 7(1):5–14.Google Scholar
  61. 61.
    Tawfik H., Liatsis P. (1997) Prediction of Non-linear Time series using Higher Order Neural Network, Proceeding IWSSIP’97 Conference, Poznan, Poland.Google Scholar
  62. 62.
    Voutriaridis C., Boutalis Y.S, Mertzios G. (2003) Ridge Polynomial Networks in Pattern Recognition.EC-VIP-MC 2003, 4th EURASIP Conference focused on Video/Image Processing and Multimedia Communications, Croatia, 519–524.Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Department of Computer Science and EngineeringGodavari Institute of Technology (Auto.)RajahmundryIndia
  2. 2.Department of Computer Science and Engineering & Information TechnologyVeer Surendra Sai University of TechnologyBurlaIndia

Personalised recommendations