Advertisement

Hardware/Software Co-design for a Neural Network Trained by Particle Swarm Optimization Algorithm

  • Tuan Linh Dang
  • Yukinobu Hoshino
Article
  • 66 Downloads

Abstract

A co-design architecture for the field programmable gate array (FPGA)-based classification system is proposed. In this architecture, a neural network (NN) is trained by the particle swarm optimization algorithm (PSO). The NN is implemented in hardware, and the PSO training is executed by a processor. The ARM processor and the NIOS II processor are used. This architecture maintains the operating speed of the NN. This approach also has the flexibility to change the parameters or even the PSO algorithm without affecting the FPGA part. Three publicly recognized databases (Iris, Balance-scale, Credit approval) are used to evaluate the system. Experimental results showed that the co-design was successfully implemented. In addition, compared to the NIOS II approach, the ARM approach had the advantages regarding the operating speed and the logic utilization.

Keywords

Neural network Particle swarm optimization Field programmable gate array ARM processor 

Notes

Acknowledgements

This work was supported by Japan Society for the Promotion of Science (JSPS) KAKENHI Grant Number 25330240.

References

  1. 1.
    Haykin S (2008) Neural networks and learning machines, 3rd edn. Prentice Hall, Upper Saddle RiverGoogle Scholar
  2. 2.
    Mingguang L, Gaoyang L (2009) Artificial neural network co-optimization algorithm based on differential evolution. In: Proceedings of 2009 second international symposium on computational intelligence and design, pp 256–259Google Scholar
  3. 3.
    Nawi1 NM, Khan1 A, Rehman MZ (2013) A new back-propagation neural network optimized with cuckoo search algorithm. In: Computational science and its applications ICCSA 2013 (lecture notes in computer science), Springer, Berlin, pp 413–426Google Scholar
  4. 4.
    Gudise VG, Venayagamoorthy GK (2003) Comparison of particle swarm optimization and backpropagation as training algorithms for neural networks. In: Proceedings of the 2003 IEEE swarm intelligence symposium, pp 110–117Google Scholar
  5. 5.
    Hsiung JT, Suewatanaku W, Himmelblau DM (1991) Should backpropagation be replaced by more effective optimization algorithms? Proc Int Jt Conf Neural Netw 1:353–356Google Scholar
  6. 6.
    Lee Y, Oh SH, Kim MW (1993) An analysis of premature saturation in back propagation learning. Neural Netw 6(5):719–728CrossRefGoogle Scholar
  7. 7.
    Leung FHF, Lam HK, Ling SH (2003) Tuning of the structure and parameters of a neural network using an improved genetic algorithm. IEEE Trans Neural Netw 14(1):79–88CrossRefGoogle Scholar
  8. 8.
    Sexton R, Dorsey R, Johnson JD (1999) Optimization of neural networks: a comparative analysis of the genetic algorithm and simulated annealing. Eur J Oper Res 114(3):589–601CrossRefzbMATHGoogle Scholar
  9. 9.
    Ilonen J, Kamarainen JK, Lampinen J (2003) Differential evolution training algorithm for feed-forward neural networks. Neural Process Lett 17(1):93–105CrossRefGoogle Scholar
  10. 10.
    Sexton R, Alidaee B, Dorsey R, Johnson JD (1998) Global optimization for artificial neural networks: a tabu search application. Eur J Oper Res 106(2–3):570–584CrossRefzbMATHGoogle Scholar
  11. 11.
    Socha K, Blum C (2007) An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training. Neural Comput Appl 16(3):235–247CrossRefGoogle Scholar
  12. 12.
    Yang XS, Deb S (2014) Cuckoo search: recent advances and applications. Neural Comput Appl 24(1):169–174CrossRefGoogle Scholar
  13. 13.
    Kennedy J, Eberhart R (1995) Particle swarm optimization. Proc IEEE Int Conf Neural Netw 4:1942–1948CrossRefGoogle Scholar
  14. 14.
    Jin N, Rahmat-Samii Y (2007) Advances in particle swarm optimization for antenna designs: real-Number, binary, single-objective and multiobjective implementations. IEEE T Antenna Propag 55(3):556–567CrossRefGoogle Scholar
  15. 15.
    Wang W, Lu Y, Fu J, Xiong YZ (2005) Particle swarm optimization and finite-element based approach for microwave filter design. IEEE Trans Magn 41(5):1800–1803CrossRefGoogle Scholar
  16. 16.
    Gaing ZL (2004) A particle swarm optimization approach for optimum design of PID controller in AVR system. IEEE Trans Energy Convers 19(2):384–391CrossRefGoogle Scholar
  17. 17.
    Chatterjee A, Pulasinghe K, Watanabe K, Izumi K (2005) A particle-swarm-optimized fuzzy-neural network for voice-controlled robot systems. IEEE Trans Ind Electron 52(6):1478–1489CrossRefGoogle Scholar
  18. 18.
    Vlachogiannis J, Lee K (2006) A comparative study on particle swarm optimization for optimal steady-state performance of power systems. IEEE Trans Power Syst 21(4):1718–1728CrossRefGoogle Scholar
  19. 19.
    Yuan Y, He Z, Chen M (2006) Virtual MIMO-based crosslayer design for wireless sensor networks. IEEE Trans Veh Technol 55(3):856–864CrossRefGoogle Scholar
  20. 20.
    Venayagamoorthy GK, Zha W (2007) Comparison of nonuniform optimal quantizer designs for speech coding with adaptive critics and particle swarm. IEEE Trans Ind Appl 43(1):238–244CrossRefGoogle Scholar
  21. 21.
    Mendes R, Cortez P, Rocha M, Neves J (2002) Particle swarms for feedforward neural network training. Proc IEEE Int Jt Conf Neural Netw 2:1895–1899Google Scholar
  22. 22.
    Bashir ZA, El-Hawary ME (2009) Applying wavelets to short-term load forecasting using PSO-based neural networks. IEEE Trans Power Syst 24(1):20–27CrossRefGoogle Scholar
  23. 23.
    Chau KW (2007) Application of a PSO-based neural network in analysis of outcomes of construction claims. Autom Constr 16(5):642–646CrossRefGoogle Scholar
  24. 24.
    Yalcin N, Tezel G, Karakuzu C (2015) Epilepsy diagnosis using artificial neural network learned by PSO. Turk J Electr Comput 23(2):421–432CrossRefGoogle Scholar
  25. 25.
    Khoury P, Gorse D (2015) Investing in emerging markets using neural networks and particle swarm optimisation. In: Proceedings of the 2015 international joint conference on neural networks, pp 1–7Google Scholar
  26. 26.
    Monmasson E, Cirstea NCM (2007) FPGA design methodology for industrial control systems—a review. IEEE Trans Ind Electron 54(4):1824–1842CrossRefGoogle Scholar
  27. 27.
    Cavuslu MA, Karakuzu C, Karakaya F (2012) Neural identification of dynamic systems on FPGA with improved PSO learning. Appl Soft Comput 12(9):2707–2718CrossRefGoogle Scholar
  28. 28.
    Bezborah A (2012) A hardware architecture for training of artificial neural networks using particle swarm optimization. In: Proceedings of the third international conference on intelligent systems, modelling and simulation (ISMS), pp 67–70Google Scholar
  29. 29.
    Lin CJ, Tsai HM (2008) FPGA implementation of a wavelet neural network with particle swarm optimization learning. Math Comput Model 47:982–996MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Dang TL, Hoshino Y (2015) A hardware implementation of particle swarm optimization with a control of velocity for training neural network. In: Proceedings of the 2015 IEEE international conference on systems, man, and cybernetics, pp 1980–1985Google Scholar
  31. 31.
    Hoshino Y, Takimoto H (2012) PSO training of the neural network application for a controller of the line tracing car. In: Proceedings of the IEEE international conference on fuzzy systems (FUZZ-IEEE), pp 1–8Google Scholar
  32. 32.
    Altera Company (2015) NIOS II processor reference handbook. https://www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/hb/nios2/n2cpu_nii5v1.pdf. Accessed 17 Oct 2016
  33. 33.
    Altera Company (2015) Avalon interface specifications. https://www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/manual/mnl_avalon_spec.pdf. Accessed 17 Oct 2016
  34. 34.
    Altera Company (2015) Cyclone V hard processor system technical reference manual. https://www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/hb/cyclone-v/cv_5v4.pdf. Accessed 17 Oct 2016
  35. 35.
    Bates GL, Nooshabadi S (1997) FPGA implementation of a median filter. In: Proceedings of IEEE region 10 annual conference on speech and image technologies for computing and telecommunications, vol 2, pp 437–440Google Scholar
  36. 36.
    Javadi MHS, Mahdiani HR (2015) Efficient utilization of imprecise blocks for hardware implementation of a Gaussian filter. In: Proceedings of 2015 IEEE computer society annual symposium on VLSI, pp 33–37Google Scholar
  37. 37.
    Neoh H, Hazanchuk A (2004) Adaptive edge detection for real-time video processing using FPGA. In: Proceedings of global signal processing, pp 1–6Google Scholar
  38. 38.
    OpenCV (2016) The OpenCV Tutorials. http://docs.opencv.org/2.4/opencv_tutorials.pdf. Accessed 17 Oct 2016
  39. 39.
    Altera Company (2015) Floating-point IP cores user guide. https://www.altera.com/en_US/pdfs/literature/ug/ug_altfp_mfug.pdf. Accessed 17 Oct 2016
  40. 40.
    Terasic Company (2015) DE1-SoC user manual. http://de1-soc.terasic.com. Accessed 17 Oct 2016
  41. 41.
    Altera Company (2016) SoC product brochure. https://www.altera.com/products/soc/overview.html. Accessed 17 Oct 2016
  42. 42.
    Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml. Accessed 17 Oct 2016

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.School of Systems EngineeringKochi University of TechnologyTosayamada, KamiJapan

Personalised recommendations