Abstract
With the rapid growth in retail e-commerce industry, most of the traditional in-store retailers are focusing more on online and mobile channels. To stay competitive, retailers need quality metadata and powerful search platforms that entice customers make effective buy decisions. Many retailers have incomplete and inaccurate product information on their Web sites, and they use multiple manual-intensive methods for acquiring product information from suppliers and third-party sources. There is no one proven channel through which retailers can achieve high-quality metadata. Our study proposes an automation method to improve the extraction of unstructured product metadata from food product label images using computer vision (CV), machine learning (ML), optical character recognition (OCR), and natural language processing (NLP). We propose an automatic image quality classification system to identify images that give a high degree of metadata extraction accuracy, and we propose a technique to improve the quality of images using traditional computer vision algorithms to improve text detection and OCR- and NLP-based metadata extraction accuracy. Our results show 95% accuracy for attribute extraction from high-quality product images with machine-printed characters having contrasting backgrounds.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
“Cracking the Consumer Code: Product Content Drives Conversion” by Salsify, 2016.
“Five Predictions For Retail In 2017” by Forbes, Jan 4, 2017.
Lidong Bing, Tak-Lam Wong, and Wai Lam.“Unsupervised extraction of popular product attributes from E-commerce Web sites by considering customer reviews”. ACM Trans. Internet Technol. 16, 2, Article 12, (April 2016).
Petrovski, Petar, and Christian Bizer. “Extracting Attribute-Value Pairs from Product Specifications on the Web.” Web Intelligence (WI’17), August 2017, Leipzig, Germany.
George, M., & Floerkemeier, C. (2014, September). Recognizing products: A per-exemplar multi-label image classification approach. In European Conference on Computer Vision (pp. 440–455). Springer, Cham.
Singh, Amarjot, Ketan Bacchuwar, and Akshay Bhasin. “A survey of OCR Applications.” International Journal of Machine Learning and Computing 2.3 (2012): 314.
Smith, Ray. “An overview of the Tesseract OCR engine.” Document Analysis and Recognition, 2007. ICDAR 2007. Ninth International Conference on. Vol. 2. IEEE, 2007.
Singh, Sukhpreet. “Optical character recognition techniques: a survey.” Journal of emerging Trends in Computing and information Sciences (2013): 545–550.
Bieniecki, Wojciech, Szymon Grabowski, and Wojciech Rozenberg. “Image preprocessing for improving ocr accuracy.” Perspective Technologies and Methods in MEMS Design, 2007. MEMSTECH 2007. International Conference on. IEEE, 2007.
Wang, Kai, Boris Babenko, and Serge Belongie. “End-to-end scene text recognition.” Computer Vision” 2011 IEEE International Conference on. IEEE, 2011.
Huang, Xiaodong, and Huadong Ma. “Automatic detection and localization of natural scene text in video.” Pattern Recognition (ICPR), 2010 20th International Conference on. IEEE, 2010.
Xi Wang, Zhenfeng Sun, Wenqiang Zhang, Yu Zhou, and Yu-Gang Jiang, “Matching User Photos to Online Products with Robust Deep Features”. In Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval (ICMR ’16). ACM, New York, NY, USA, 7–14. 2016.
Hossain, M. Zahid, M. Ashraful Amin, and Hong Yan. “Rapid feature extraction for optical character recognition.” arXiv preprint (2012).
Chen, Huizhong, et al. “Robust Text Detection in Natural Images with Edge-Enhanced Maximally Stable Extremal Regions.” Image Processing (ICIP), 2011 18th IEEE International Conference on. IEEE, 2011.
Epshtein, Boris, Eyal Ofek, and Yonatan Wexler. “Detecting text in natural scenes with stroke width transform.” Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on. IEEE, 2010.
Khurshid, Khurram, et al. “Comparison of Niblack inspired binarization methods for ancient documents.” DRR 7247 (2009): 1–10.
Gupta, Anshul, et al. “Automatic Assessment of OCR Quality in Historical Documents.” AAAI. 2015.
Zornitsa Kozareva. “Everyone Likes Shopping! Multi-class Product Categorization for e-Commerce”. In The Annual Conference of the North Americal Chapter for the ACL. 1329–1333, 2015.
Yang, Shuo, and Jingzhi Guo. “A novel approach for cross-context document reasoning in e-commerce.” Software Engineering and Service Science (ICSESS), 2015 6th IEEE International Conference on. IEEE, 2015.
Gabor Melli. 2014. Shallow Semantic Parsing of Product Offering Titles (for better automatic hyperlink insertion). In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 1670– 1678.
Lienhart, Rainer, and Axel Wernicke. “Localizing and segmenting text in images and videos.” IEEE Transactions on circuits and systems for video technology 12.4 (2002): 256–268.
Nadeau, David, and Satoshi Sekine. “A survey of named entity recognition and classification.” Lingvisticae Investigationes 30.1 (2007): 3–26.
Sridhar R. Avula, Jinshan Tang and Scott T. Acton, “An object-based image retrieval system for digital libraries”, Multimedia Systems, Springer Berlin, March 2006, vol 11 No. 3 pp: 260–270.
Andre Folkers and Hanan Samet, “Content based image retrieval using Fourier descriptors on a Logo database”, IEEE Proceedings of 16th International Conference of Pattern Recognition, 2002, vol 3 pp. 521–524.
Romberg, Stefan, et al. “Scalable logo recognition in real-world images.” Proceedings of the 1st ACM International Conference on Multimedia Retrieval. ACM, 2011.
Zhu, Guangyu, and David Doermann. “Automatic document logo detection.” Document Analysis and Recognition, 2007. ICDAR 2007. Ninth International Conference on. Vol. 2. IEEE, 2007.
Dalal, Navneet, and Bill Triggs. “Histograms of oriented gradients for human detection.” Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on. Vol. 1. IEEE, 2005.
Dalal, N., Triggs, B., & Schmid, C. (2006, May). Human detection using oriented histograms of flow and appearance. In European conference on computer vision (pp. 428–441). Springer, Berlin, Heidelberg.
Li, Teng, et al. “Contextual bag-of-words for visual categorization.” IEEE Transactions on Circuits and Systems for Video Technology 21.4 (2011): 381–392.
Tanwir Zaman, “Vision Based Extraction of Nutrition Information from Skewed Nutrition Labels”, Thesis submitted to Utah State University, 2016.
Pouladzadeh, Parisa, Shervin Shirmohammadi, and Rana Al-Maghrabi. “Measuring calorie and nutrition from food image.” IEEE Transactions on Instrumentation and Measurement 63.8 (2014): 1947–1956.
Walter H. Delashmit and Michael T. Manry, “Recent Developments in Multilayer Perceptron Neural Networks”, Proceedings of the 7th Annual Memphis Area Engineering and Science Conference, MAESC 2005, pp. 1–3.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Gundimeda, V., Murali, R.S., Joseph, R., Naresh Babu, N.T. (2019). An Automated Computer Vision System for Extraction of Retail Food Product Metadata. In: Bapi, R., Rao, K., Prasad, M. (eds) First International Conference on Artificial Intelligence and Cognitive Computing . Advances in Intelligent Systems and Computing, vol 815. Springer, Singapore. https://doi.org/10.1007/978-981-13-1580-0_20
Download citation
DOI: https://doi.org/10.1007/978-981-13-1580-0_20
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-13-1579-4
Online ISBN: 978-981-13-1580-0
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)