Neural PCA and Maximum Likelihood Hebbian Learning on the GPU

  • Pavel Krömer
  • Emilio Corchado
  • Václav Snášel
  • Jan Platoš
  • Laura García-Hernández
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7553)


This study introduces a novel fine-grained parallel implementation of a neural principal component analysis (neural PCA) variant and the maximum Likelihood Hebbian Learning (MLHL) network designed for modern many-core graphics processing units (GPUs). The parallel implementation as well as the computational experiments conducted in order to evaluate the speedup achieved by the GPU are presented and discussed. The evaluation was done on a well-known artificial data set, the 2D bars data set.


neural PCA Maximum Likelihood Hebbian Learning Exploratory Projection Pursuit GPU CUDA performance 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Brandstetter, A., Artusi, A.: Radial basis function networks gpu-based implementation. IEEE Transactions on Neural Networks 19(12), 2150–2154 (2008)CrossRefGoogle Scholar
  2. 2.
    Corchado, E., Fyfe, C.: Orientation selection using maximum likelihood hebbian learning. Int. Journal of Knowledge-Based Intelligent Engineering 2(7) (2003)Google Scholar
  3. 3.
    Corchado, E., Han, Y., Fyfe, C.: Structuring global responses of local filters using lateral connections. J. Exp. Theor. Artif. Intell. 15(4), 473–487 (2003)CrossRefzbMATHGoogle Scholar
  4. 4.
    Corchado, E., MacDonald, D., Fyfe, C.: Maximum and minimum likelihood hebbian learning for exploratory projection pursuit. Data Mining and Knowledge Discovery 8, 203–225 (2004)CrossRefMathSciNetGoogle Scholar
  5. 5.
    Corchado, E., Perez, J.C.: A three-step unsupervised neural model for visualizing high complex dimensional spectroscopic data sets. Pattern Anal. Appl. 14(2), 207–218 (2011)CrossRefMathSciNetGoogle Scholar
  6. 6.
    De, P., Veronese, L., Krohling, R.A.: Swarm’s flight: accelerating the particles using c-cuda. In: Proceedings of the Eleventh conference on Congress on Evolutionary Computation, CEC 2009, pp. 3264–3270. IEEE Press, Piscataway (2009)Google Scholar
  7. 7.
    Friedman, J., Tukey, J.: A projection pursuit algorithm for exploratory data analysis. IEEE Transactions on Computers C- 23(9), 881–890 (1974)CrossRefzbMATHGoogle Scholar
  8. 8.
    Fyfe, C., Corchado, E.: Maximum likelihood Hebbian rules. In: Verleysen, M. (ed.) ESANN 2002, Proceedings of the 10th European Symposium on Artificial Neural Networks, Bruges, Belgium, April 24-26, pp. 143–148 (2002)Google Scholar
  9. 9.
    Fyfe, C.: A neural network for pca and beyond. Neur. Proc. Letters 6, 33–41 (1997)CrossRefGoogle Scholar
  10. 10.
    Guzhva, A., Dolenko, S., Persiantsev, I.: Multifold Acceleration of Neural Network Computations Using GPU. In: Alippi, C., Polycarpou, M., Panayiotou, C., Ellinas, G. (eds.) ICANN 2009, Part I. LNCS, vol. 5768, pp. 373–380. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  11. 11.
    Ho, T.Y., Lam, P.M., Leung, C.S.: Parallelization of cellular neural networks on gpu. Pattern Recogn. 41(8), 2684–2692 (2008)CrossRefzbMATHGoogle Scholar
  12. 12.
    Krömer, P., Platoš, J., Snášel, V., Abraham, A.: An Implementation of Differential Evolution for Independent Tasks Scheduling on GPU. In: Corchado, E., Kurzyński, M., Woźniak, M. (eds.) HAIS 2011, Part I. LNCS, vol. 6678, pp. 372–379. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  13. 13.
    Langdon, W.B., Banzhaf, W.: A SIMD Interpreter for Genetic Programming on GPU Graphics Cards. In: O’Neill, M., Vanneschi, L., Gustafson, S., Esparcia Alcázar, A.I., De Falco, I., Della Cioppa, A., Tarantino, E. (eds.) EuroGP 2008. LNCS, vol. 4971, pp. 73–85. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  14. 14.
    Lopes, N., Ribeiro, B.: GPU Implementation of the Multiple Back-Propagation Algorithm. In: Corchado, E., Yin, H. (eds.) IDEAL 2009. LNCS, vol. 5788, pp. 449–456. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  15. 15.
    Martínez-Zarzuela, M., Díaz-Pernas, F., Antón-Rodríguez, M., Díez-Higuera, J., González-Ortega, D., Boto-Giralda, D., López-González, F., De La Torre, I.: Multi-scale neural texture classification using the gpu as a stream processing engine. Machine Vision and Applications 22, 947–966 (2011)CrossRefGoogle Scholar
  16. 16.
    Martínez-Zarzuela, M., Pernas, F., de Pablos, A., Rodríguez, M., Higuera, J., Giralda, D., Ortega, D.: Adaptative Resonance Theory Fuzzy Networks Parallel Computation Using CUDA. In: Cabestany, J., Sandoval, F., Prieto, A., Corchado, J.M. (eds.) IWANN 2009, Part I. LNCS, vol. 5517, pp. 149–156. Springer, Heidelberg (2009)Google Scholar
  17. 17.
    Martínez-Zarzuela, M., Díaz Pernas, F., Díez Higuera, J., Rodríguez, M.: Fuzzy ART Neural Network Parallel Computing on the GPU. In: Sandoval, F., Prieto, A.G., Cabestany, J., Graña, M. (eds.) IWANN 2007. LNCS, vol. 4507, pp. 463–470. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  18. 18.
    Oh, K.S., Jung, K.: GPU implementation of neural networks. Pattern Recognition 37(6), 1311–1314 (2004)CrossRefzbMATHGoogle Scholar
  19. 19.
    Oja, E.: Neural networks, principal components, and subspaces. International Journal of Neural Systms 1(1), 61–68 (1989)CrossRefMathSciNetGoogle Scholar
  20. 20.
    Patulea, C., Peace, R., Green, J.: Cuda-accelerated genetic feedforward-ann training for data mining. J. of Physics: Conference Series 256(1), 012014 (2010)Google Scholar
  21. 21.
    Sanders, J., Kandrot, E.: CUDA by Example: An Introduction to General-Purpose GPU Programming, 1st edn. Addison-Wesley Professional (July 2010)Google Scholar
  22. 22.
    Scanzio, S., Cumani, S., Gemello, R., Mana, F., Laface, P.: Parallel implementation of artificial neural network training for speech recognition. Pattern Recognition Letters 31(11), 1302–1309 (2010)CrossRefGoogle Scholar
  23. 23.
    Shitara, A., Nishikawa, Y., Yoshimi, M., Amano, H.: Implementation and evaluation of self-organizing map algorithm on a graphic processor. In: Parallel and Distributed Computing and Systems 2009 (2009)Google Scholar
  24. 24.
    Sierra-Canto, X., Madera-Ramirez, F., Uc-Cetina, V.: Parallel training of a back-propagation neural network using cuda. In: Proceedings of the 2010 Ninth International Conference on Machine Learning and Applications, ICMLA 2010, pp. 307–312. IEEE Computer Society, Washington, DC (2010)CrossRefGoogle Scholar
  25. 25.
    Zhang, K., Li, Y., Scarf, P., Ball, A.: Feature selection for high-dimensional machinery fault diagnosis data using multiple models and radial basis function networks. Neurocomputing 74(17), 2941–2952 (2011)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Pavel Krömer
    • 1
    • 2
  • Emilio Corchado
    • 2
    • 3
  • Václav Snášel
    • 1
    • 2
  • Jan Platoš
    • 1
    • 2
  • Laura García-Hernández
    • 4
  1. 1.Department of Computer ScienceVŠB-Technical University of OstravaOstrava-PorubaCzech Republic
  2. 2.IT4InnovationsOstrava-PorubaCzech Republic
  3. 3.Departamento de Informática y AutomáticaUniversidad de SalamancaSpain
  4. 4.Area of Project EngineeringUniversity of CordobaSpain

Personalised recommendations