Advertisement

Pattern Analysis and Applications

, Volume 21, Issue 1, pp 249–260 | Cite as

Efficient visual code localization with neural networks

  • Péter Bodnár
  • Tamás Grósz
  • László Tóth
  • László G. Nyúl
Industrial and Commercial Application
  • 237 Downloads

Abstract

The use of computer-readable visual codes became common in our everyday life both in industrial environments and for private use. The reading process of visual codes consists of two steps, namely, localization and data decoding. In this paper we examine the localization step of visual codes using conventional and deep rectifier neural networks. They are also evaluated in the discrete cosine transform domain and shown to be efficient, which makes full decompression unnecessary for setups involving JPEG images. This approach is also efficient from a storage viewpoint and computation cost viewpoint, since camera hardware can provide a JPEG stream as output in many cases. The use of neural networks implemented on graphics processing unit allows real-time automatic code object localization. In our earlier studies, the proposed approach was evaluated on the most popular code type, quick response code, and some other 2D codes as well. Here, we also prove that deep rectifier networks are also suitable for 1D barcode localization and present extensive evaluation and comparison to state-of-the-art approaches.

Keywords

QR code Barcode DCT Pattern recognition Neural networks Machine learning Deep learning DRN Deep rectifier networks Feature extraction Localization 

Abbreviations

ANN

Conventional (nondeep) neural network

AUC

Area under curve

DCT

Discrete cosine transform

DFT

Discrete Fourier transform

DNN

Deep neural network

DRN

Deep rectifier network

FIP

Finder pattern (of QR code)

FPS

Frames per second

GPU

Graphics processing unit

HSV

Hue–saturation–value (color space)

JPEG

Joint photographic experts group (format)

MSE

Mean squared error

MSER

Maximum stable extremal region

NN

Neural network

RLE

Runlength encoding

QR code

Quick response code

Notes

Acknowledgements

Tamás Grósz was supported by the ÚNKP-16-3 New National Excellence Program of the Ministry of Human Capacities.

References

  1. 1.
    Belussi LFF, Hirata NST (2011) Fast QR code detection in arbitrarily acquired images. In: 2011 24th SIBGRAPI conference on graphics, patterns and images (Sibgrapi), pp 281–288Google Scholar
  2. 2.
    Bodnár P, Grósz T, Tóth L, Nyúl LG (2014) Localization of visual codes in the dct domain using deep rectifier neural networks. In: International workshop on artificial neural networks and intelligent information processing: proceedings of ANNIIP, pp 37–44Google Scholar
  3. 3.
    Bodnár P, Nyúl LG (2012) Improving barcode detection with combination of simple detectors. In: The 8th international conference on signal image technology (SITIS 2012), pp 300–306Google Scholar
  4. 4.
    Bodnár P, Nyúl LG (2013) A novel method for barcode localization in image domain. In: Image analysis and recognition, vol 7950. Lecture notes in computer science. Springer, Berlin, pp 189–196Google Scholar
  5. 5.
    Chu CH, Yang DN, Pan YL, Chen MS (2011) Stabilization and extraction of 2D barcodes for camera phones. Multimedia Syst 17:113–133CrossRefGoogle Scholar
  6. 6.
    Ciresan D, Giusti A, Gambardella LM, Schmidhuber J (2012) Deep neural networks segment neuronal membranes in electron microscopy images. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems 25. Curran Associates Inc., Red Hook, pp 2843–2851Google Scholar
  7. 7.
    Creusot C, Munawar A (2015) Real-time barcode detection in the wild. In: 2015 IEEE winter conference on applications of computer vision, pp 239–245. doi: 10.1109/WACV.2015.39
  8. 8.
    Dubská M, Herout A, Havel J (2013) Real-time precise detection of regular grids and matrix codes. J Real Time Image Process. doi: 10.1007/s11554-013-0325-6 MATHGoogle Scholar
  9. 9.
    Gallo O, Manduchi R (2011) Image-based barcode reader. WO Patent App. PCT/US2010/002,023Google Scholar
  10. 10.
    Gallo O, Manduchi R (2011) Reading 1D barcodes with mobile phones using deformable templates. IEEE Trans Pattern Anal Mach Intell 33(9):1834–1843CrossRefGoogle Scholar
  11. 11.
    Glorot X, Bengio Y (2012) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of AISTATS, pp 249–256Google Scholar
  12. 12.
    Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier networks. In: Proceedings of artificial intelligence and statistics (AISTATS), pp 315–323Google Scholar
  13. 13.
    Grósz T, Bodnár P, Tóth L, Nyúl LG (2014) QR code localization using deep neural networks. In: 2014 IEEE international workshop on machine learning for signal processing (MLSP). IEEE, pp 1–6Google Scholar
  14. 14.
    Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554MathSciNetCrossRefMATHGoogle Scholar
  15. 15.
    Katona M, Nyúl LG (2012) A novel method for accurate and efficient barcode detection with morphological operations. In: Proceedings of the international conference on signal image technology & internet systems (SITIS), pp 307–314Google Scholar
  16. 16.
    Kurtz C, Desjardins GE, Sanchez SJ (2007) Self checkout system with automated transportation conveyor. US Patent 7,204,346Google Scholar
  17. 17.
    LeCun Y, Bottou L, Orr GB, Müller K-R (1998) Efficient backprop. In: Orr GB, Müller K-R (eds) Neural networks: tricks of the trade. Springer, Berlin HeidelbergGoogle Scholar
  18. 18.
    Lin DT, Lin CL (2013) Automatic location for multi-symbology and multiple 1D and 2D barcodes. J Mar Sci Technol 21(6):663–668Google Scholar
  19. 19.
    Matas J, Chum O, Urban M, Pajdla T (2004) Robust wide-baseline stereo from maximally stable extremal regions. Image Vis Comput 22(10):761–767CrossRefGoogle Scholar
  20. 20.
    Ohbuchi E, Hanaizumi H, Hock LA (2004) Barcode readers using the camera device in mobile phones. In: 2004 international conference on cyberworlds, pp 260–265Google Scholar
  21. 21.
    Seide F, Li G, Chen X, Yu D (2011) Feature engineering in context-dependent deep neural networks for conversational speech transcription. In: Proceedings of automatic speech recognition and understanding (ASRU), pp 24–29Google Scholar
  22. 22.
    Sörös G, Flörkemeier C (2013) Blur-resistant joint 1D and 2D barcode localization for smartphones. In: Proceedings of the 12th international conference on mobile and ubiquitous multimedia, MUM ’13. ACM, New York, pp 11:1–11:8Google Scholar
  23. 23.
    Sörös G, Semmler S, Humair L, Hilliges O (2015) Fast blur removal for wearable QR code scanners. In: Proceedings of the 2015 ACM international symposium on wearable computers, ISWC ’15. ACM, New York, pp 117–124Google Scholar
  24. 24.
    Szentandrási I, Herout A, Dubská M (2013) Fast detection and recognition of QR codes in high-resolution images. In: Proceedings of the 28th spring conference on computer graphics, SCCG ’12. ACM, New York, pp 129–136Google Scholar
  25. 25.
    Tekin E, Coughlan J (2009) A Bayesian algorithm for reading 1D barcodes. In: Proceedings of the 2009 Canadian conference on computer and robot vision, CRV ’09. IEEE Computer Society, Washington, pp 61–67Google Scholar
  26. 26.
    Tekin E, Coughlan J (2012) Blade: Barcode localization and decoding engine. Tech. rep., Technical Report 2012-RERCGoogle Scholar
  27. 27.
    Thomes BJ, Emil BF, Sanville WW (1970) Automatic car identification system. US Patent 3,543,007Google Scholar
  28. 28.
    Tóth L, Grósz T (2013) A comparison of deep neural network training methods for large vocabulary speech recognition. In: Proceedings of text, speech and dialogue (TSD), pp 36–43Google Scholar
  29. 29.
    Wallace GK (1992) The JPEG still picture compression standard. IEEE Trans Consumer Electron 38(1):xviii–xxxivGoogle Scholar
  30. 30.
    Wang K, Zou Y, Wang H (2005) Bar code reading from images captured by camera phones. In: 2005 2nd international conference on mobile technology, applications and systems, p 6Google Scholar
  31. 31.
    Wu Y (2016) Embedded QR code intelligent recognition platform based on team progress algorithm. iJOE 12(2):46–50Google Scholar
  32. 32.
    Zamberletti A, Gallo I, Carullo M, Binaghi E (2010) Neural image restoration for decoding 1-d barcodes using common camera phones. In: Computer vision, imaging and computer graphics. Theory and applications, pp 5–11Google Scholar

Copyright information

© Springer-Verlag London 2017

Authors and Affiliations

  • Péter Bodnár
    • 1
  • Tamás Grósz
    • 2
  • László Tóth
    • 2
  • László G. Nyúl
    • 1
  1. 1.Department of Image Processing and Computer GraphicsUniversity of SzegedSzegedHungary
  2. 2.MTA-SZTE Research Group on Artificial IntelligenceHungarian Academy of Sciences and University of SzegedSzegedHungary

Personalised recommendations