Skip to main content
SpringerLink
Log in

Efficient visual code localization with neural networks

  • Industrial and Commercial Application
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

The use of computer-readable visual codes became common in our everyday life both in industrial environments and for private use. The reading process of visual codes consists of two steps, namely, localization and data decoding. In this paper we examine the localization step of visual codes using conventional and deep rectifier neural networks. They are also evaluated in the discrete cosine transform domain and shown to be efficient, which makes full decompression unnecessary for setups involving JPEG images. This approach is also efficient from a storage viewpoint and computation cost viewpoint, since camera hardware can provide a JPEG stream as output in many cases. The use of neural networks implemented on graphics processing unit allows real-time automatic code object localization. In our earlier studies, the proposed approach was evaluated on the most popular code type, quick response code, and some other 2D codes as well. Here, we also prove that deep rectifier networks are also suitable for 1D barcode localization and present extensive evaluation and comparison to state-of-the-art approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. http://www.inf.u-szeged.hu/~bodnaar/barcode_database/.

  2. http://cvpr.uni-muenster.de/research/barcode.

  3. http://artelab.dista.uninsubria.it/downloads/datasets/barcode/.

  4. http://medusa.fit.vutbr.cz/pclines/?p=86.

  5. http://people.inf.ethz.ch/soeroesg/.

Abbreviations

ANN:

Conventional (nondeep) neural network

AUC:

Area under curve

DCT:

Discrete cosine transform

DFT:

Discrete Fourier transform

DNN:

Deep neural network

DRN:

Deep rectifier network

FIP:

Finder pattern (of QR code)

FPS:

Frames per second

GPU:

Graphics processing unit

HSV:

Hue–saturation–value (color space)

JPEG:

Joint photographic experts group (format)

MSE:

Mean squared error

MSER:

Maximum stable extremal region

NN:

Neural network

RLE:

Runlength encoding

QR code:

Quick response code

References

  1. Belussi LFF, Hirata NST (2011) Fast QR code detection in arbitrarily acquired images. In: 2011 24th SIBGRAPI conference on graphics, patterns and images (Sibgrapi), pp 281–288

  2. Bodnár P, Grósz T, Tóth L, Nyúl LG (2014) Localization of visual codes in the dct domain using deep rectifier neural networks. In: International workshop on artificial neural networks and intelligent information processing: proceedings of ANNIIP, pp 37–44

  3. Bodnár P, Nyúl LG (2012) Improving barcode detection with combination of simple detectors. In: The 8th international conference on signal image technology (SITIS 2012), pp 300–306

  4. Bodnár P, Nyúl LG (2013) A novel method for barcode localization in image domain. In: Image analysis and recognition, vol 7950. Lecture notes in computer science. Springer, Berlin, pp 189–196

  5. Chu CH, Yang DN, Pan YL, Chen MS (2011) Stabilization and extraction of 2D barcodes for camera phones. Multimedia Syst 17:113–133

    Article  Google Scholar 

  6. Ciresan D, Giusti A, Gambardella LM, Schmidhuber J (2012) Deep neural networks segment neuronal membranes in electron microscopy images. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems 25. Curran Associates Inc., Red Hook, pp 2843–2851

    Google Scholar 

  7. Creusot C, Munawar A (2015) Real-time barcode detection in the wild. In: 2015 IEEE winter conference on applications of computer vision, pp 239–245. doi:10.1109/WACV.2015.39

  8. Dubská M, Herout A, Havel J (2013) Real-time precise detection of regular grids and matrix codes. J Real Time Image Process. doi:10.1007/s11554-013-0325-6

    MATH  Google Scholar 

  9. Gallo O, Manduchi R (2011) Image-based barcode reader. WO Patent App. PCT/US2010/002,023

  10. Gallo O, Manduchi R (2011) Reading 1D barcodes with mobile phones using deformable templates. IEEE Trans Pattern Anal Mach Intell 33(9):1834–1843

    Article  Google Scholar 

  11. Glorot X, Bengio Y (2012) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of AISTATS, pp 249–256

  12. Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier networks. In: Proceedings of artificial intelligence and statistics (AISTATS), pp 315–323

  13. Grósz T, Bodnár P, Tóth L, Nyúl LG (2014) QR code localization using deep neural networks. In: 2014 IEEE international workshop on machine learning for signal processing (MLSP). IEEE, pp 1–6

  14. Hinton GE, Osindero S, Teh YW (2006) A fast learning algorithm for deep belief nets. Neural Comput 18(7):1527–1554

    Article  MathSciNet  MATH  Google Scholar 

  15. Katona M, Nyúl LG (2012) A novel method for accurate and efficient barcode detection with morphological operations. In: Proceedings of the international conference on signal image technology & internet systems (SITIS), pp 307–314

  16. Kurtz C, Desjardins GE, Sanchez SJ (2007) Self checkout system with automated transportation conveyor. US Patent 7,204,346

  17. LeCun Y, Bottou L, Orr GB, Müller K-R (1998) Efficient backprop. In: Orr GB, Müller K-R (eds) Neural networks: tricks of the trade. Springer, Berlin Heidelberg

    Google Scholar 

  18. Lin DT, Lin CL (2013) Automatic location for multi-symbology and multiple 1D and 2D barcodes. J Mar Sci Technol 21(6):663–668

    Google Scholar 

  19. Matas J, Chum O, Urban M, Pajdla T (2004) Robust wide-baseline stereo from maximally stable extremal regions. Image Vis Comput 22(10):761–767

    Article  Google Scholar 

  20. Ohbuchi E, Hanaizumi H, Hock LA (2004) Barcode readers using the camera device in mobile phones. In: 2004 international conference on cyberworlds, pp 260–265

  21. Seide F, Li G, Chen X, Yu D (2011) Feature engineering in context-dependent deep neural networks for conversational speech transcription. In: Proceedings of automatic speech recognition and understanding (ASRU), pp 24–29

  22. Sörös G, Flörkemeier C (2013) Blur-resistant joint 1D and 2D barcode localization for smartphones. In: Proceedings of the 12th international conference on mobile and ubiquitous multimedia, MUM ’13. ACM, New York, pp 11:1–11:8

  23. Sörös G, Semmler S, Humair L, Hilliges O (2015) Fast blur removal for wearable QR code scanners. In: Proceedings of the 2015 ACM international symposium on wearable computers, ISWC ’15. ACM, New York, pp 117–124

  24. Szentandrási I, Herout A, Dubská M (2013) Fast detection and recognition of QR codes in high-resolution images. In: Proceedings of the 28th spring conference on computer graphics, SCCG ’12. ACM, New York, pp 129–136

  25. Tekin E, Coughlan J (2009) A Bayesian algorithm for reading 1D barcodes. In: Proceedings of the 2009 Canadian conference on computer and robot vision, CRV ’09. IEEE Computer Society, Washington, pp 61–67

  26. Tekin E, Coughlan J (2012) Blade: Barcode localization and decoding engine. Tech. rep., Technical Report 2012-RERC

  27. Thomes BJ, Emil BF, Sanville WW (1970) Automatic car identification system. US Patent 3,543,007

  28. Tóth L, Grósz T (2013) A comparison of deep neural network training methods for large vocabulary speech recognition. In: Proceedings of text, speech and dialogue (TSD), pp 36–43

  29. Wallace GK (1992) The JPEG still picture compression standard. IEEE Trans Consumer Electron 38(1):xviii–xxxiv

  30. Wang K, Zou Y, Wang H (2005) Bar code reading from images captured by camera phones. In: 2005 2nd international conference on mobile technology, applications and systems, p 6

  31. Wu Y (2016) Embedded QR code intelligent recognition platform based on team progress algorithm. iJOE 12(2):46–50

    Google Scholar 

  32. Zamberletti A, Gallo I, Carullo M, Binaghi E (2010) Neural image restoration for decoding 1-d barcodes using common camera phones. In: Computer vision, imaging and computer graphics. Theory and applications, pp 5–11

Download references

Acknowledgements

Tamás Grósz was supported by the ÚNKP-16-3 New National Excellence Program of the Ministry of Human Capacities.

Author information

Authors and Affiliations

  1. Department of Image Processing and Computer Graphics, University of Szeged, Szeged, Hungary

    Péter Bodnár & László G. Nyúl

  2. MTA-SZTE Research Group on Artificial Intelligence, Hungarian Academy of Sciences and University of Szeged, Szeged, Hungary

    Tamás Grósz & László Tóth

Authors
  1. Péter Bodnár
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tamás Grósz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. László Tóth
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. László G. Nyúl
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to László G. Nyúl.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bodnár, P., Grósz, T., Tóth, L. et al. Efficient visual code localization with neural networks. Pattern Anal Applic 21, 249–260 (2018). https://doi.org/10.1007/s10044-017-0619-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-017-0619-6

Keywords

Search

Navigation