Advertisement

Neural Processing Letters

, Volume 45, Issue 1, pp 29–58 | Cite as

Hyperconic Multilayer Perceptron

  • Juan Pablo Serrano-Rubio
  • Arturo Hernández-Aguirre
  • Rafael Herrera-Guzmán
Article

Abstract

This paper introduces the design of the hyperconic multilayer perceptron (HC-MLP). Complex non-linear decision regions for classification purposes are generated by quadratic hyper-surfaces spawned by the hyperconic neurons in the hidden layer (for instance, spheres, ellipsoids, paraboloids, hyperboloids and degenerate conics). In order to generate quadratic hyper-surfaces, the hyperconic neurons’ transfer function includes the estimation of a quadratic polynomial. The proper assignment of decision regions to classes is achieved in the output layer by using spheres to determine whether a point is inside or outside the spherical region. The particle swarm optimization algorithm is used for training the HC-MLP. The learning of the HC-MLP selects the best conic surface that separates the data set vectors. For illustration purposes, two experiments are conducted using two distributions of synthetic data in order to show the advantages of HC-MLP when the patterns between classes are contiguous. Furthermore a comparison to the traditional multilayer perceptron is carried out to evaluate the complexity (in terms of the number of estimated patterns) and classification accuracy. HC-MLP is the principal component to implement a diagnosis system to detect faults in an induction motor and to implement an image segmentation system. The performance of HC-MLP is compared to other leading algorithms by using 4 databases commonly used in related literature.

Keywords

Artificial neural networks Non-linear decision regions  Induction motor fault diagnostic Image segmentation 

Notes

Acknowledgments

The first author would like to thank to Antonio Zamarron for technical advice in induction motors. The third author would like to thank the International Centre for Theoretical Physics (ICTP) and the Institut Des Hautes Etudes Scientifiques (IHES) for its hospitality and support.

References

  1. 1.
    Arena P, Fortuna L, Occhipinti L, Xibilia M (1994) Neural networks for quaternion-valued function approximation. In: 1994 IEEE international symposium on circuits and systems, 1994. ISCAS ’94, vol 6, pp 307–310Google Scholar
  2. 2.
    Astorino A, Fuduli A, Gaudioso M (2010) Dc models for spherical separation. J Glob Optim 48(4):657–669MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Astorino A, Fuduli A, Gaudioso M (2012) Margin maximization in spherical separation. Comput Optim Appl 53(2):301–322MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Bache K, Lichman M (2013) Uci machine learning repository. http://archive.ics.uci.edu/ml
  5. 5.
    Bishop CM (2006) Pattern recognition and machine learning (information science and statistics). Springer, Secaucus, NJMATHGoogle Scholar
  6. 6.
    Blake A, Rother C, Brown M, Perez P, Torr P (2004) Interactive image segmentation using an adaptive gmmrf model. In: Computer vision-ECCV 2004, pp 428–441. Springer, BerlinGoogle Scholar
  7. 7.
    Bookstein FL (1979) Fitting conic sections to scattered data. Comput Gr Image Process 9(1):56–71CrossRefGoogle Scholar
  8. 8.
    Buchholz S, Sommer G (2000) A hyperbolic multilayer perceptron. In: Proceedings of the IEEE-INNS-ENNS international joint conference on neural networks, 2000. IJCNN 2000, vol 2, pp 129–133Google Scholar
  9. 9.
    Cantu-Paz E, Kamath C (2005) An empirical comparison of combinations of evolutionary algorithms and neural networks for classification problems. IEEE Trans Syst Man Cybern B Cyber 35(5):915–927CrossRefGoogle Scholar
  10. 10.
    Carvalho M, Ludermir T (2007) Particle swarm optimization of neural network architectures and weights. In: 7th international conference on hybrid intelligent systems, 2007. HIS 2007. pp 336–339Google Scholar
  11. 11.
    Casasent DP, Barnard E (1990) Adaptive-clustering optical neural net. Appl Opt 29(17):2603–2615CrossRefGoogle Scholar
  12. 12.
    Casasent D, Natarajan S (1995) A classifier neural net with complex-valued weights and square-law nonlinearities. Neural Netw 8(6):989–998CrossRefGoogle Scholar
  13. 13.
    Cheng H, Jiang X, Sun Y, Wang J (2001) Color image segmentation: advances and prospects. Pattern Recognit 34(12):2259–2281CrossRefMATHGoogle Scholar
  14. 14.
    Eberhart R, Kennedy J (1995) A new optimizer using particle swarm theory. In: Proceedings of the sixth international symposium on micro machine and human science, 1995. MHS ’95. pp 39–43Google Scholar
  15. 15.
    Engelbrecht AP (2006) Fundamentals of computational swarm intelligence. Wiley, New YorkGoogle Scholar
  16. 16.
    Garro B, Sossa H, Vzquez R (2011) Back-propagation vs particle swarm optimization algorithm: which algorithm is better to adjust the synaptic weights of a feed-forward ann? Int J Artif Intel 7(11A):208–218Google Scholar
  17. 17.
    Georgiou G, Koutsougeras C (1992) Complex domain backpropagation. IEEE Trans Circuits Syst II Analog Digit Sig Process 39(5):330–334CrossRefMATHGoogle Scholar
  18. 18.
    Hernandez-Lopez FJ, Rivera M (2014) Change detection by probabilistic segmentation from monocular view. Mach Vis Appl 25(5):1175–1195CrossRefGoogle Scholar
  19. 19.
    Hong ZQ, Yang JY (1991) Optimal discriminant plane for a small number of samples and design method of classifier on the plane. Pattern Recognit 24(4):317–324MathSciNetCrossRefGoogle Scholar
  20. 20.
    Kiranyaz S, Ince T, Yildirim A, Gabbouj M (2009) Evolutionary artificial neural networks by multi-dimensional particle swarm optimization. Neural Netw 22(10):1448–1462CrossRefGoogle Scholar
  21. 21.
    Lin S, Zeng J, Xu Z (2015) Error estimate for spherical neural networks interpolation. Neural Process Lett 42(2):369–379CrossRefGoogle Scholar
  22. 22.
    Lipson H, Siegelmann HT (2000) Clustering irregular shapes using high-order neurons. Neural Comput 12(10):2331–2353CrossRefGoogle Scholar
  23. 23.
    Liu R, Sun X, Jiao L (2010) Particle swarm optimization based clustering: a comparison of different cluster validity indices. In: Li K, Li X, Ma S, Irwin G (eds) Life system modeling and intelligent computing, communications in computer and information science, vol 98. Springer, Berlin, pp 66–72Google Scholar
  24. 24.
    Lowe D, Broomhead D (1988) Multivariable functional interpolation and adaptive networks. Complex Syst 2:321–355MathSciNetMATHGoogle Scholar
  25. 25.
    Mabu S, Obayashi M, Kuremoto T (2015) Ensemble learning of rule-based evolutionary algorithm using multi-layer perceptron for supporting decisions in stock trading problems. Appl Soft Comput 36:357–367CrossRefGoogle Scholar
  26. 26.
    Minsky M (1969) Perceptrons. MIT Press, Cambridge, MAMATHGoogle Scholar
  27. 27.
    Misra BB, Satapathy SC, Biswal B, Dash P, Panda G (2006) Pattern classification using polynomial neural network. In: 2006 IEEE conference on cybernetics and intelligent systems, pp 1–6Google Scholar
  28. 28.
    Misra B, Satapathy S, Biswal B, Dash P, Panda G (2006) Pattern classification using polynomial neural network. In: 2006 IEEE conference on cybernetics and intelligent systems, pp 1–6. IEEEGoogle Scholar
  29. 29.
    Natarajan SS, Casasent DP (1993) Piecewise quadratic optical neural network. In: San Diego’92, pp 142–147. International Society for Optics and PhotonicsGoogle Scholar
  30. 30.
    Nejjari H, Benbouzid M (1999) Monitoring and diagnosis of induction motors electrical faults using a current park’s vector pattern learning approach. In: International conference IEMD ’99 electric machines and drives, 1999, pp 275–277Google Scholar
  31. 31.
    Nitta T, Buchholz S (2008) On the decision boundaries of hyperbolic neurons. In: IEEE international joint conference on neural networks, 2008. IJCNN 2008. (IEEE World Congress on Computational Intelligence), pp 2974–2980Google Scholar
  32. 32.
    Oh SK, Pedrycz W (2002) The design of self-organizing polynomial neural networks. Inf Sci 141(34):237–258CrossRefMATHGoogle Scholar
  33. 33.
    Park H, Ozeki T, Amari Si (2005) Geometric approach to multilayer perceptrons. In: Handbook of geometric computing, pp 69–96. Springer, BerlinGoogle Scholar
  34. 34.
    Perwass C, Banarer V, Sommer G (2003) Spherical decision surfacesusing conformal modelling. In: Michaelis B, Krell G (eds) Pattern recognition. Lecture notes in computer science, vol 2781, pp 9–16. Springer, BerlinGoogle Scholar
  35. 35.
    Pillay P, Xu Z (1996) Motor current signature analysis. In: Industry applications conference, 1996. Thirty-first IAS annual meeting, IAS ’96., Conference record of the 1996 IEEE, vol 1, pp 587–594Google Scholar
  36. 36.
    Prechelt L et al (1994) Proben1: a set of neural network benchmarkproblems and benchmarking rules. Fakultät für Informatik, Univ. Karlsruhe, Karlsruhe, Germany, Tech Rep 21:94Google Scholar
  37. 37.
  38. 38.
    Salinas-Gutiérrez R, Hernández-Aguirre A, Rivera-Meraz MJ, Villa-Diharce ER (2010) Supervised probabilistic classification based on gaussian copulas. In: Sidorov G, Hernández Aguirre A, Reyes García CA (eds) Advances in soft computing. Lecture notes in computer science, vol 6438, pp 104–115. Springer, BerlinGoogle Scholar
  39. 39.
    Serrano-Rubio J Color image segmentation. https://www.youtube.com/watch?v=sAVDPCed1qM
  40. 40.
    Serrano-Rubio J Conic sections produced by hc-mlp. https://www.youtube.com/watch?v=A1Y2rJTSZbQ
  41. 41.
    Serrano-Rubio J Linear surfaces produced by hp-mlp. https://www.youtube.com/watch?v=CfTL-AGmpzo
  42. 42.
    Sexton RS, Dorsey RE (2000) Reliable classification using neural networks: a genetic algorithm and backpropagation comparison. Decis Support Syst 30(1):11–22CrossRefGoogle Scholar
  43. 43.
    Sossa, H., Garro, B., Villegas, J., Olague, G., Avils, C.: Evolutionary computation applied to the automatic design of artificial neural networks and associative memories. In: Schtze O, Coello Coello CA, Tantar AA, Tantar E, Bouvry P, Del Moral P, Legrand P (eds) Evolve—a bridge between probability, set oriented numerics, and evolutionary computation II. Advances in intelligent systems and computing, vol 175, pp 285–297. Springer, BerlinGoogle Scholar
  44. 44.
    Tablada L, Valena M (2012) Self-organizing polynomial neuralnetworks based on matrix inversion and differential evolution. In: Yin H, Costa J, Barreto G (eds) Intelligent data engineering andautomated learning—IDEAL 2012. Lecture notes in computer science, vol 7435, pp 399–406. Springer, BerlinGoogle Scholar
  45. 45.
    Taylor BJ (2005) Methods and procedures for the verification and validation of artificial neural networks. Springer, Secaucus, NJGoogle Scholar
  46. 46.
    Thomson W, Fenger M (2001) Current signature analysis to detect induction motor faults. IEEE Ind Appl Mag 7(4):26–34CrossRefGoogle Scholar
  47. 47.
    Tufféry S (2011) Data mining and statistics for decision making. Wiley, New YokCrossRefMATHGoogle Scholar
  48. 48.
    UCI: Breast cancer wisconsin (diagnostic) data set—machine learning repository (1995). http://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+%28Diagnostic%29
  49. 49.
    UCI: Diabetes data set—machine learning repository. http://archive.ics.uci.edu/ml/datasets/Diabetes
  50. 50.
    UCI: Heart disease data set—machine learning repository (1988). http://archive.ics.uci.edu/ml/datasets/Heart+Disease
  51. 51.
    UCI: Lung cancer data set—machine learning repository (1992). https://archive.ics.uci.edu/ml/datasets/Lung+Cancer
  52. 52.
    van den Bergh F, Engelbrecht A (2006) A study of particle swarm optimization particle trajectories. Inf Sci 176(8):937–971MathSciNetCrossRefMATHGoogle Scholar
  53. 53.
    Weber DM, Casasent DP (1998) The extended piecewise quadratic neural network. Neural Netw 11(5):837–850CrossRefGoogle Scholar
  54. 54.
    Yao X, Liu Y (1997) A new evolutionary system for evolving artificial neural networks. IEEE Trans Neural Netw 8(3):694–713CrossRefGoogle Scholar
  55. 55.
    Yu J, Xi L, Wang S (2007) An improved particle swarm optimization for evolving feedforward artificial neural networks. Neural Process Lett 26(3):217–231CrossRefGoogle Scholar
  56. 56.
    Zhang C, Shao H (2000) An ann’s evolved by a new evolutionary system and its application. In: Proceedings of the 39th IEEE conference on decision and control, 2000, vol 4, pp 3562–3563Google Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Juan Pablo Serrano-Rubio
    • 1
  • Arturo Hernández-Aguirre
    • 2
  • Rafael Herrera-Guzmán
    • 3
  1. 1.Information Technologies LaboratoryTechnological Institute of Irapuato (ITESI)IrapuatoMexico
  2. 2.Computer Science DepartmentCenter for Research in Mathematics (CIMAT)GuanajuatoMexico
  3. 3.Mathematics DepartmentCenter for Research in Mathematics (CIMAT)GuanajuatoMexico

Personalised recommendations