Applied Intelligence

, Volume 49, Issue 11, pp 4022–4032 | Cite as

Using convolutional neural networks for character verification on integrated circuit components of printed circuit boards

  • Chun-Hui Lin
  • Shyh-Hau Wang
  • Cheng-Jian LinEmail author


Manufacturers of printed circuit boards (PCBs) typically use automated optical inspection (AOI) machines to test their PCBs. However, AOI machines employ conventional image-processing methods. If the integrated circuit (IC) components are not identical to the golden samples, then the AOI machine registers those IC components as flaws. Conventional image-processing methods cause misjudgments and increase the cost of manual reviews. Character-verification and image-classification systems are proposed in this paper for detecting misplaced, missing, and reversed-polarity parts. The regions of IC components can be identified on PCBs by using the contour border-detection method. Through the proposed convolutional neural network (CNN) structure and refinement mechanism, the characters can be successfully recognized. The image-classification system was applied only to images with blurry characters. Different CNN learning structures were used in both systems, and the refinement mechanism was used in both systems to improve the results. The proposed character-verification and image-classification methods achieved 98.84% and 99.48% passing rates, and the amount of required training time was less than that of other methods, demonstrating the proposed methods’ greater effectiveness.


Printed circuit board Convolutional neural networks Component testing Contour detection Deep learning 



  1. 1.
    Cho HJ, Park TH (2008) Template matching method for SMD inspection using discrete wavelet transform. Proc SICE Annual Conference: 3198–3201Google Scholar
  2. 2.
    Crispin AJ, Rankov V (2007) Automated inspection of PCB components using a genetic algorithm template-matching approach. Int J Adv Manuf Technol 35(3–4):293–300CrossRefGoogle Scholar
  3. 3.
    Lee JJ, Lee PH, Lee SW, Yuille A, Koch C (2011) AdaBoost for text detection in natural scene. International Conference on Document Analysis and Recognition: 429–434Google Scholar
  4. 4.
    Epshtein B, Ofek E, Wexler Y (2010) Detecting text in natural scenes with stroke width transform. IEEE Computer Society Conference on Computer Vision and Pattern Recognition: 2963–2970Google Scholar
  5. 5.
    Nava CF, Gonzalez FF (2015) OCR for unreadable damaged characters on PCBs using principal component analysis and bayesian discriminant functions. Proceedings international conference on computational science and computational intelligence (CSCI 2015): 535–538Google Scholar
  6. 6.
    Li W, Neullens S, Breier M, Bosling M, Pretz T, Merhof D (2014) Text recognition for information retrieval in images of printed circuit boards. Annual conference of the IEEE industrial electronics society (IECON 2014): 3487–3493Google Scholar
  7. 7.
    Zhang S, Liu Y, Jin L, Luo C (2017) Feature enhancement network: a refined scene text detector. Association for the Advancement of artificial intelligence (AAAI 2018): 2612–2619Google Scholar
  8. 8.
    Shi B, Bai X, Yao C (2017) An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans Pattern Anal Mach Intell 39(11):2298–2304CrossRefGoogle Scholar
  9. 9.
    Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016) SSD: single shot multibox detector. European conference on computer vision (ECCV2016): 21–37Google Scholar
  10. 10.
    Liao M, Shi B, Bai X, Wang X, Liu W (2017) Textboxes: a fast text detector with a single deep neural network. Association for the Advancement of artificial intelligence (AAAI 2017): 4161–4167Google Scholar
  11. 11.
    Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern 9(1):62–66CrossRefGoogle Scholar
  12. 12.
    Suzuki S, Be K (1985) Topological structural analysis of digitized binary images by border following. Comput Vision, Graph Image Process 30(1):32–46CrossRefGoogle Scholar
  13. 13.
    Kingma DP, Ba J (2014) Adam:a method for stochastic optimization. arXiv preprint arXiv:1412.6980Google Scholar
  14. 14.
    Redmon J, Divvala S, Girshick R, Farhadi A (2016) You only look once: unified, real-time object detection. Proc IEEE Conf Comput Vision and Pattern Recogn, Las Vegas, Nevada, USA: 779–788Google Scholar
  15. 15.
    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556Google Scholar
  16. 16.
    Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition: 7–12Google Scholar
  17. 17.
    Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Proc 25th Int Conf Neural Inform Process Syst 1:1097–1105Google Scholar
  18. 18.
    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, Las Vegas, Nevada, USA: 770–778Google Scholar
  19. 19.
    Liu L, Shen C, Hengel AVD (2017) Cross-convolutional-layer pooling for image recognition. IEEE Trans Pattern Anal Mach Intell 39(11):2305–2313CrossRefGoogle Scholar
  20. 20.
    Zhang J, Shao K, Luo X (2018) Small sample image recognition using improved convolutional neural network. J Vis Commun Image Represent 55:640–647CrossRefGoogle Scholar
  21. 21.
    Chen B, Li J, Wei G, Ma B (2018) A novel localized and second order feature coding network for image recognition. Pattern Recogn 76:339–348CrossRefGoogle Scholar
  22. 22.
    Chen PH, Si S, Kumar S, Li Y, Hsieh C (2018) Learning to screen for fast Softmax inference on large vocabulary neural networks. arXiv preprint arXiv: 1810.12406Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer Science & Information EngineeringNational Cheng Kung UniversityTainanTaiwan
  2. 2.Intelligent Manufacturing Research CenterNational Cheng Kung UniversityTainanTaiwan
  3. 3.Department of Computer Science & Information EngineeringNational Chin-Yi University of TechnologyTaichungTaiwan

Personalised recommendations