Skip to main content
SpringerLink
Log in

Detection and Recognition of Food in Photo Galleries for Analysis of User Preferences

  • Conference paper
  • First Online:
Book cover Image Analysis and Recognition (ICIAR 2020)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12131))

Included in the following conference series:

Abstract

Food analysis is one of the most important parts of user preference prediction engines for recommendation systems in the travel domain. In this paper, we describe and study the neural network method that allows you to recognize food in a gallery of photos taken with mobile devices. The described method consists of three main stages, including the classification of scenes, food detection, and subsequent classification. An essential feature of the developed method is the use of lightweight neural network models, which allows its usage on mobile devices. The development of the method was carried out using both known open data and a proprietary data set.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Savchenko, A.V., Demochkin, K.V., Grechikhin, I.S.: User preference prediction in visual data on mobile devices. arXiv preprint 1907.04519 (2019)

    Google Scholar 

  2. Matsuda, Y., Yanai, K.: Multiple-food recognition considering co-occurrence employing manifold ranking. In: Proceedings of the 21st International Conference on Pattern Recognition, ICPR 2012, pp. 2017–2020, November 2012

    Google Scholar 

  3. Kitamura, K., Yamasaki, T., Aizawa, K.: FoodLog: capture, analysis and retrieval of personal food images via web. In: Proceedings of the ACM Multimedia 2009 Workshop on Multimedia for Cooking and Eating Activities, CEA 2009, pp. 23–30. Association for Computing Machinery, New York (2009)

    Google Scholar 

  4. Farinella, G.M., Allegra, D., Stanco, F., Battiato, S.: On the exploitation of one class classification to distinguish food vs non-food images. In: Murino, V., Puppo, E., Sona, D., Cristani, M., Sansone, C. (eds.) ICIAP 2015. LNCS, vol. 9281, pp. 375–383. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23222-5_46

    Chapter  Google Scholar 

  5. Ragusa, F., Tomaselli, V., Furnari, A., Battiato, S., Farinella, G.M.: Food vs non-food classification. In: Proceedings of the International Workshop on Multimedia Assisted Dietary Management (MADiMa), pp. 77–81. ACM (2016)

    Google Scholar 

  6. Myers, A., et al.: Im2Calories: towards an automated mobile vision food diary. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1233–1241, December 2015

    Google Scholar 

  7. Anzawa, M., Amano, S., Yamakata, Y., Motonaga, K., Kamei, A., Aizawa, K.: Recognition of multiple food items in a single photo for use in a buffet-style restaurant. IEICE Trans. Inf. Syst. E102.D(2), 410–414 (2019)

    Google Scholar 

  8. Kagaya, H., Aizawa, K., Ogawa, M.: Food detection and recognition using convolutional neural network. In: Proceedings of the 22nd ACM International Conference on Multimedia, MM 2014, pp. 1085–1088. Association for Computing Machinery, New York (2014)

    Google Scholar 

  9. Singla, A., Yuan, L., Ebrahimi, T.: Food/non-food image classification and food categorization using pre-trained GoogLeNet model. In: Proceedings of the 2nd International Workshop on Multimedia Assisted Dietary Management. MADiMa 2016. Association for Computing Machinery, New York (2016)

    Google Scholar 

  10. Aguilar, E., Bolaños, M., Radeva, P.: Exploring food detection using CNNs. In: Moreno-Díaz, R., Pichler, F., Quesada-Arencibia, A. (eds.) EUROCAST 2017. LNCS, vol. 10672, pp. 339–347. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-74727-9_40

    Chapter  Google Scholar 

  11. Oliveira, L., Costa, V., Neves, G., Oliveira, T., Jorge, E., Lizarraga, M.: A mobile, lightweight, poll-based food identification system. Pattern Recogn. 47(5), 1941–1952 (2014)

    Article  Google Scholar 

  12. Martinel, N., Piciarelli, C., Micheloni, C., Foresti, G.L.: A structured committee for food recognition. In: 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), pp. 484–492 (2015)

    Google Scholar 

  13. Zheng, J., Wang, Z., Zhu, C.: Food image recognition via superpixel based low-level and mid-level distance coding for smart home applications. Sustainability 9(5), 856 (2017)

    Article  Google Scholar 

  14. Bettadapura, V., Thomaz, E., Parnami, A., Abowd, G.D., Essa, I.: Leveraging context to support automated food recognition in restaurants. In: Proceedings of the Winter Conference on Applications of Computer Vision (WACV), pp. 580–587. IEEE (2015)

    Google Scholar 

  15. Bolanos, M., P., R.: Simultaneous food localization and recognition. In: International Conference on Pattern Recognition, pp. 3140–3145 (2017)

    Google Scholar 

  16. Wu, H., Merler, M., Uceda-Sosa, R., Smith, J.R.: Learning to make better mistakes: semantics-aware visual food recognition. In: Proceedings of the 24th International Conference on Multimedia (MM), pp. 172–176. ACM (2016)

    Google Scholar 

  17. Ciocca, G., Napoletano, P., Schettini, R.: Food recognition: a new dataset, experiments, and results. IEEE J. Biomed. Health Inform. 21(3), 588–598 (2016)

    Article  Google Scholar 

  18. Kaur, P., Sikka, K., Wang, W., Belongie, S., Divakaran, A.: FoodX-251: a dataset for fine-grained food classification. arXiv preprint 1907.06167 (2019)

    Google Scholar 

  19. Ming, Z.-Y., Chen, J., Cao, Y., Forde, C., Ngo, C.-W., Chua, T.S.: Food photo recognition for dietary tracking: system and experiment. In: Schoeffmann, K., et al. (eds.) MMM 2018. LNCS, vol. 10705, pp. 129–141. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-73600-6_12

    Chapter  Google Scholar 

  20. Aguilar, E., Bolaños, M., Radeva, P.: Food recognition using fusion of classifiers based on CNNs. In: Battiato, S., Gallo, G., Schettini, R., Stanco, F. (eds.) ICIAP 2017. LNCS, vol. 10485, pp. 213–224. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68548-9_20

    Chapter  Google Scholar 

  21. Xin Wang, Kumar, D., Thome, N., Cord, M., Precioso, F.: Recipe recognition with large multimodal food dataset. In: Proceedings of the International Conference on Multimedia Expo Workshops (ICMEW), pp. 1–6. IEEE (2015)

    Google Scholar 

  22. Liu, C., Cao, Y., Luo, Y., Chen, G., Vokkarane, V., Ma, Y.: DeepFood: deep learning-based food image recognition for computer-aided dietary assessment. In: Chang, C.K., Chiari, L., Cao, Y., Jin, H., Mokhtari, M., Aloulou, H. (eds.) ICOST 2016. LNCS, vol. 9677, pp. 37–48. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-39601-9_4

    Chapter  Google Scholar 

  23. Farinella, G.M., Allegra, D., Stanco, F.: A benchmark dataset to study the representation of food images. In: Agapito, L., Bronstein, M.M., Rother, C. (eds.) ECCV 2014. LNCS, vol. 8927, pp. 584–599. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16199-0_41

    Chapter  Google Scholar 

  24. Bossard, L., Guillaumin, M., Van Gool, L.: Food-101 – mining discriminative components with random forests. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 446–461. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10599-4_29

    Chapter  Google Scholar 

  25. Zhang, H., Cissé, M., Dauphin, Y.N., Lopez-Paz, D.: Mixup: beyond empirical risk minimization. CoRR abs/1710.09412 (2017)

    Google Scholar 

  26. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. CoRR abs/1709.01507 (2017)

    Google Scholar 

Download references

Acknowledgements

This research is based on the work supported by Samsung Research, Samsung Electronics.

Author information

Authors and Affiliations

  1. St. Petersburg Department of Steklov Institute of Mathematics, St. Petersburg, Russia

    Evgeniy Miasnikov & Andrey Savchenko

  2. Laboratory of Algorithms and Technologies for Network Analysis, National Research University Higher School of Economics, Nizhny Novgorod, Russia

    Andrey Savchenko

Authors
  1. Evgeniy Miasnikov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrey Savchenko
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Evgeniy Miasnikov .

Editor information

Editors and Affiliations

  1. University of Porto, Porto, Portugal

    Aurélio Campilho

  2. University of Waterloo, Waterloo, ON, Canada

    Fakhri Karray

  3. University of Waterloo, Waterloo, ON, Canada

    Zhou Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Miasnikov, E., Savchenko, A. (2020). Detection and Recognition of Food in Photo Galleries for Analysis of User Preferences. In: Campilho, A., Karray, F., Wang, Z. (eds) Image Analysis and Recognition. ICIAR 2020. Lecture Notes in Computer Science(), vol 12131. Springer, Cham. https://doi.org/10.1007/978-3-030-50347-5_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-50347-5_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-50346-8

  • Online ISBN: 978-3-030-50347-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation