Skip to main content
SpringerLink
Log in

Hardware/Software Co-design for a Neural Network Trained by Particle Swarm Optimization Algorithm

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

A co-design architecture for the field programmable gate array (FPGA)-based classification system is proposed. In this architecture, a neural network (NN) is trained by the particle swarm optimization algorithm (PSO). The NN is implemented in hardware, and the PSO training is executed by a processor. The ARM processor and the NIOS II processor are used. This architecture maintains the operating speed of the NN. This approach also has the flexibility to change the parameters or even the PSO algorithm without affecting the FPGA part. Three publicly recognized databases (Iris, Balance-scale, Credit approval) are used to evaluate the system. Experimental results showed that the co-design was successfully implemented. In addition, compared to the NIOS II approach, the ARM approach had the advantages regarding the operating speed and the logic utilization.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

References

  1. Haykin S (2008) Neural networks and learning machines, 3rd edn. Prentice Hall, Upper Saddle River

    Google Scholar 

  2. Mingguang L, Gaoyang L (2009) Artificial neural network co-optimization algorithm based on differential evolution. In: Proceedings of 2009 second international symposium on computational intelligence and design, pp 256–259

  3. Nawi1 NM, Khan1 A, Rehman MZ (2013) A new back-propagation neural network optimized with cuckoo search algorithm. In: Computational science and its applications ICCSA 2013 (lecture notes in computer science), Springer, Berlin, pp 413–426

  4. Gudise VG, Venayagamoorthy GK (2003) Comparison of particle swarm optimization and backpropagation as training algorithms for neural networks. In: Proceedings of the 2003 IEEE swarm intelligence symposium, pp 110–117

  5. Hsiung JT, Suewatanaku W, Himmelblau DM (1991) Should backpropagation be replaced by more effective optimization algorithms? Proc Int Jt Conf Neural Netw 1:353–356

    Google Scholar 

  6. Lee Y, Oh SH, Kim MW (1993) An analysis of premature saturation in back propagation learning. Neural Netw 6(5):719–728

    Article  Google Scholar 

  7. Leung FHF, Lam HK, Ling SH (2003) Tuning of the structure and parameters of a neural network using an improved genetic algorithm. IEEE Trans Neural Netw 14(1):79–88

    Article  Google Scholar 

  8. Sexton R, Dorsey R, Johnson JD (1999) Optimization of neural networks: a comparative analysis of the genetic algorithm and simulated annealing. Eur J Oper Res 114(3):589–601

    Article  MATH  Google Scholar 

  9. Ilonen J, Kamarainen JK, Lampinen J (2003) Differential evolution training algorithm for feed-forward neural networks. Neural Process Lett 17(1):93–105

    Article  Google Scholar 

  10. Sexton R, Alidaee B, Dorsey R, Johnson JD (1998) Global optimization for artificial neural networks: a tabu search application. Eur J Oper Res 106(2–3):570–584

    Article  MATH  Google Scholar 

  11. Socha K, Blum C (2007) An ant colony optimization algorithm for continuous optimization: application to feed-forward neural network training. Neural Comput Appl 16(3):235–247

    Article  Google Scholar 

  12. Yang XS, Deb S (2014) Cuckoo search: recent advances and applications. Neural Comput Appl 24(1):169–174

    Article  Google Scholar 

  13. Kennedy J, Eberhart R (1995) Particle swarm optimization. Proc IEEE Int Conf Neural Netw 4:1942–1948

    Article  Google Scholar 

  14. Jin N, Rahmat-Samii Y (2007) Advances in particle swarm optimization for antenna designs: real-Number, binary, single-objective and multiobjective implementations. IEEE T Antenna Propag 55(3):556–567

    Article  Google Scholar 

  15. Wang W, Lu Y, Fu J, Xiong YZ (2005) Particle swarm optimization and finite-element based approach for microwave filter design. IEEE Trans Magn 41(5):1800–1803

    Article  Google Scholar 

  16. Gaing ZL (2004) A particle swarm optimization approach for optimum design of PID controller in AVR system. IEEE Trans Energy Convers 19(2):384–391

    Article  Google Scholar 

  17. Chatterjee A, Pulasinghe K, Watanabe K, Izumi K (2005) A particle-swarm-optimized fuzzy-neural network for voice-controlled robot systems. IEEE Trans Ind Electron 52(6):1478–1489

    Article  Google Scholar 

  18. Vlachogiannis J, Lee K (2006) A comparative study on particle swarm optimization for optimal steady-state performance of power systems. IEEE Trans Power Syst 21(4):1718–1728

    Article  Google Scholar 

  19. Yuan Y, He Z, Chen M (2006) Virtual MIMO-based crosslayer design for wireless sensor networks. IEEE Trans Veh Technol 55(3):856–864

    Article  Google Scholar 

  20. Venayagamoorthy GK, Zha W (2007) Comparison of nonuniform optimal quantizer designs for speech coding with adaptive critics and particle swarm. IEEE Trans Ind Appl 43(1):238–244

    Article  Google Scholar 

  21. Mendes R, Cortez P, Rocha M, Neves J (2002) Particle swarms for feedforward neural network training. Proc IEEE Int Jt Conf Neural Netw 2:1895–1899

    Google Scholar 

  22. Bashir ZA, El-Hawary ME (2009) Applying wavelets to short-term load forecasting using PSO-based neural networks. IEEE Trans Power Syst 24(1):20–27

    Article  Google Scholar 

  23. Chau KW (2007) Application of a PSO-based neural network in analysis of outcomes of construction claims. Autom Constr 16(5):642–646

    Article  Google Scholar 

  24. Yalcin N, Tezel G, Karakuzu C (2015) Epilepsy diagnosis using artificial neural network learned by PSO. Turk J Electr Comput 23(2):421–432

    Article  Google Scholar 

  25. Khoury P, Gorse D (2015) Investing in emerging markets using neural networks and particle swarm optimisation. In: Proceedings of the 2015 international joint conference on neural networks, pp 1–7

  26. Monmasson E, Cirstea NCM (2007) FPGA design methodology for industrial control systems—a review. IEEE Trans Ind Electron 54(4):1824–1842

    Article  Google Scholar 

  27. Cavuslu MA, Karakuzu C, Karakaya F (2012) Neural identification of dynamic systems on FPGA with improved PSO learning. Appl Soft Comput 12(9):2707–2718

    Article  Google Scholar 

  28. Bezborah A (2012) A hardware architecture for training of artificial neural networks using particle swarm optimization. In: Proceedings of the third international conference on intelligent systems, modelling and simulation (ISMS), pp 67–70

  29. Lin CJ, Tsai HM (2008) FPGA implementation of a wavelet neural network with particle swarm optimization learning. Math Comput Model 47:982–996

    Article  MathSciNet  MATH  Google Scholar 

  30. Dang TL, Hoshino Y (2015) A hardware implementation of particle swarm optimization with a control of velocity for training neural network. In: Proceedings of the 2015 IEEE international conference on systems, man, and cybernetics, pp 1980–1985

  31. Hoshino Y, Takimoto H (2012) PSO training of the neural network application for a controller of the line tracing car. In: Proceedings of the IEEE international conference on fuzzy systems (FUZZ-IEEE), pp 1–8

  32. Altera Company (2015) NIOS II processor reference handbook. https://www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/hb/nios2/n2cpu_nii5v1.pdf. Accessed 17 Oct 2016

  33. Altera Company (2015) Avalon interface specifications. https://www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/manual/mnl_avalon_spec.pdf. Accessed 17 Oct 2016

  34. Altera Company (2015) Cyclone V hard processor system technical reference manual. https://www.altera.com/content/dam/altera-www/global/en_US/pdfs/literature/hb/cyclone-v/cv_5v4.pdf. Accessed 17 Oct 2016

  35. Bates GL, Nooshabadi S (1997) FPGA implementation of a median filter. In: Proceedings of IEEE region 10 annual conference on speech and image technologies for computing and telecommunications, vol 2, pp 437–440

  36. Javadi MHS, Mahdiani HR (2015) Efficient utilization of imprecise blocks for hardware implementation of a Gaussian filter. In: Proceedings of 2015 IEEE computer society annual symposium on VLSI, pp 33–37

  37. Neoh H, Hazanchuk A (2004) Adaptive edge detection for real-time video processing using FPGA. In: Proceedings of global signal processing, pp 1–6

  38. OpenCV (2016) The OpenCV Tutorials. http://docs.opencv.org/2.4/opencv_tutorials.pdf. Accessed 17 Oct 2016

  39. Altera Company (2015) Floating-point IP cores user guide. https://www.altera.com/en_US/pdfs/literature/ug/ug_altfp_mfug.pdf. Accessed 17 Oct 2016

  40. Terasic Company (2015) DE1-SoC user manual. http://de1-soc.terasic.com. Accessed 17 Oct 2016

  41. Altera Company (2016) SoC product brochure. https://www.altera.com/products/soc/overview.html. Accessed 17 Oct 2016

  42. Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml. Accessed 17 Oct 2016

Download references

Acknowledgements

This work was supported by Japan Society for the Promotion of Science (JSPS) KAKENHI Grant Number 25330240.

Author information

Authors and Affiliations

  1. School of Systems Engineering, Kochi University of Technology, Tosayamada, Kami, Kochi, 782-8502, Japan

    Tuan Linh Dang & Yukinobu Hoshino

Authors
  1. Tuan Linh Dang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yukinobu Hoshino
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tuan Linh Dang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dang, T.L., Hoshino, Y. Hardware/Software Co-design for a Neural Network Trained by Particle Swarm Optimization Algorithm. Neural Process Lett 49, 481–505 (2019). https://doi.org/10.1007/s11063-018-9826-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-018-9826-4

Keywords

Search

Navigation