Transforming User Experience of Nutrition Facts Label - An Exploratory Service Innovation Study

  • Prateek JainEmail author
  • Soussan Djamasbi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11588)


Nutrition facts label is an important tool for consumers to get information regarding servings, calories and nutrients in a packaged food product. Previous research shows that nutrition labels are generally confusing and difficult to use. Nutrition information in the label can be transformed into dynamic feedback to make nutrition facts labels easy to use and helpful in making healthy decisions. In this research, we created a decision support system using a smartphone application that scans the label using OCR, then apply the FDA’s 5-20 rule to determine if a particular nutrient is in healthy amount and visualizes this feedback in either an augmented reality or a static popup format using color-coded thumbs up and thumbs down signs. Our results show that the app significantly helped consumers in making healthy decisions and improved the overall experience of using nutrition facts labels. While our results did not show a significant difference between the impact of augmented reality and static popup feedback on user behavior, they indicated a slightly more favorable reaction toward feedback that used augmented reality.


Nutrition facts label Augmented reality Decision making Percent daily value 


  1. 1.
    McMahon, D.D., Cihak, D.F., Gibbons, M.M., Fussell, L., Mathison, S.: Using a mobile app to teach individuals with intellectual disabilities to identify potential food allergens. J. Spec. Educ. Technol. 28(3), 21–32 (2013)CrossRefGoogle Scholar
  2. 2.
    Bayu, M.Z., Arshad, H., Ali, N.M.: Nutritional information visualization using mobile augmented reality technology. Procedia Technol. 11, 396–402 (2013)CrossRefGoogle Scholar
  3. 3.
    Waltner, G., et al.: MANGO - mobile augmented reality with functional eating guidance and food awareness. In: Murino, V., Puppo, E., Sona, D., Cristani, M., Sansone, C. (eds.) ICIAP 2015. LNCS, vol. 9281, pp. 425–432. Springer, Cham (2015). Scholar
  4. 4.
    Ahn, J., Williamson, J., Gartrell, M., Han, R., Lv, Q., Mishra, S.: Supporting healthy grocery shopping via mobile augmented reality. ACM Trans. Multimedia Comput. Commun. Appl. (TOMM) 12(1s), 16 (2015)Google Scholar
  5. 5.
    Kulyukin, V., Vanka, A., Wang, H.: Skip trie matching: a greedy algorithm for real-time OCR error correction on smartphones. Int. J. Digit. Inf. Wirel. Commun. (IJDIWC) 3(3), 261–270 (2013)Google Scholar
  6. 6.
    Kulyukin, V., Kutiyanawala, A., Zaman, T., Clyde, S.: Vision-based localization and text chunking of nutrition fact tables on android smartphones. In: Proceedings of International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV 2013), pp. 314–320 (2013)Google Scholar
  7. 7.
    Zaman, T., Kulyukin, V.: Text skew angle detection in vision-based scanning of nutrition labels. In: Proceedings of the International Conference on Image Processing, Computer Vision, and Pattern Recognition (IPCV), p. 139. The Steering Committee of the World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp) (2015)Google Scholar
  8. 8.
    Kulyukin, V., Blay, C.: An algorithm for mobile vision-based localization of skewed nutrition labels that maximizes specificity. In: Emerging Trends in Image Processing, Computer Vision and Pattern Recognition, pp. 277–293 (2015)CrossRefGoogle Scholar
  9. 9.
    Neuhouser, M.L., Kristal, A.R., Patterson, R.E.: Use of food nutrition labels is associated with lower fat intake. J. Am. Diet. Assoc. 99(1), 45–53 (1999)CrossRefGoogle Scholar
  10. 10.
    Rothman, R.L., et al.: Patient understanding of food labels: the role of literacy and numeracy. Am. J. Prev. Med. 31(5), 391–398 (2006)CrossRefGoogle Scholar
  11. 11.
    Temple, N.J., Fraser, J.: Food labels: a critical assessment. Nutrition 30(3), 257–260 (2014)CrossRefGoogle Scholar
  12. 12.
    Levy, L., Patterson, R.E., Kristal, A.R., Li, S.S.: How well do consumers understand percentage daily value on food labels? Am. J. Health Promot. 14(3), 157–160 (2000)CrossRefGoogle Scholar
  13. 13.
    Ollberding, N.J., Wolf, R.L., Contento, I.: Food label use and its relation to dietary intake among US adults. J. Am. Diet. Assoc. 111(5), S47–S51 (2011)CrossRefGoogle Scholar
  14. 14.
    Graham, D.J., Orquin, J.L., Visschers, V.H.: Eye tracking and nutrition label use: a review of the literature and recommendations for label enhancement. Food Policy 37(4), 378–382 (2012)CrossRefGoogle Scholar
  15. 15.
    Kreuter, M.W., Brennan, L.K., Scharff, D.P., Lukwago, S.N.: Do nutrition label readers eat healthier diets? Behavioral correlates of adults’ use of food labels. Am. J. Prev. Med. 13(4), 277–283 (1997)CrossRefGoogle Scholar
  16. 16.
    Cowburn, G., Stockley, L.: Consumer understanding and use of nutrition labelling: a systematic review. Public Health Nutr. 8(1), 21–28 (2005)CrossRefGoogle Scholar
  17. 17.
    Li, F., Miniard, P.W., Barone, M.J.: The facilitating influence of consumer knowledge on the effectiveness of daily value reference information. J. Acad. Mark. Sci. 28(3), 425–436 (2000)CrossRefGoogle Scholar
  18. 18.
    Nayga Jr., R.M., Lipinski, D., Savur, N.: Consumers’ use of nutritional labels while food shopping and at home. J. Consum. Aff. 32(1), 106–120 (1998)CrossRefGoogle Scholar
  19. 19.
    Nayga Jr., R.M.: Nutrition knowledge, gender, and food label use. J. Consum. Aff. 34(1), 97–112 (2000)CrossRefGoogle Scholar
  20. 20.
    Campos, S., Doxey, J., Hammond, D.: Nutrition labels on pre-packaged foods: a systematic review. Public Health Nutr. 14(8), 1496–1506 (2011)CrossRefGoogle Scholar
  21. 21.
    Taylor, C.L., Wilkening, V.L.: How the nutrition food label was developed, Part 1: the Nutrition Facts panel. J. Am. Diet. Assoc. 108(3), 437–442 (2008)CrossRefGoogle Scholar
  22. 22.
    Graham, D.J., Roberto, C.A.: Evaluating the impact of US Food and Drug Administration–proposed nutrition facts label changes on young adults’ visual attention and purchase intentions. Health Educ. Behav. 43(4), 389–398 (2016)CrossRefGoogle Scholar
  23. 23.
    Graham, D.J., Jeffery, R.W.: Location, location, location: eye-tracking evidence that consumers preferentially view prominently positioned nutrition information. J. Am. Diet. Assoc. 111(11), 1704–1711 (2011)CrossRefGoogle Scholar
  24. 24.
    Azuma, R.T.: A survey of augmented reality. Presence Teleoperators Virtual Environ. 6(4), 355–385 (1997)CrossRefGoogle Scholar
  25. 25.
    Olsson, T., Salo, M.: Online user survey on current mobile augmented reality applications. In: 2011 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 75–84. IEEE(2011)Google Scholar
  26. 26.
    Akçayır, M., Akçayır, G.: Advantages and challenges associated with augmented reality for education: a systematic review of the literature. Educ. Res. Rev. 20, 1–11 (2017)CrossRefGoogle Scholar
  27. 27.
    Jain, P., Hall-Phillips, A., Djamasbi, S.: Effect of social media product reviews on buying decision when presented in augmented reality. In: Nah, F.F.-H., Xiao, B.S. (eds.) HCIBGO 2018. LNCS, vol. 10923, pp. 313–326. Springer, Cham (2018). Scholar
  28. 28.
    US Department of Health and Human Services and U.S. Department of Agriculture. 2015–2020 Dietary Guidelines for Americans, 8th edn., December 2015.
  29. 29.
    Center for Food Safety and Applied Nutrition: Labeling & Nutrition - How to Understand and Use the Nutrition Facts Label (n.d.).
  30. 30.
    Levine, A.S., Kotz, C.M., Gosnell, B.A.: Sugars and fats: the neurobiology of preference. J. Nutr. 133(3), 831S–834S (2003)CrossRefGoogle Scholar
  31. 31.
    Tran, D.M., Westbrook, R.F.: Rats fed a diet rich in fats and sugars are impaired in the use of spatial geometry. Psychol. Sci. 26(12), 1947–1957 (2015)CrossRefGoogle Scholar
  32. 32.
    Drewnowski, A.: The real contribution of added sugars and fats to obesity. Epidemiol. Rev. 29(1), 160–171 (2007)MathSciNetCrossRefGoogle Scholar
  33. 33.
    Johnston, C.S., Tjonn, S.L., Swan, P.D.: High-protein, low-fat diets are effective for weight loss and favorably alter biomarkers in healthy adults. J. Nutr. 134(3), 586–591 (2004)CrossRefGoogle Scholar
  34. 34.
    Westerterp-Plantenga, M.S., Lejeune, M.P.G.M., Nijs, I., Van Ooijen, M., Kovacs, E.M.R.: High protein intake sustains weight maintenance after body weight loss in humans. Int. J. Obes. 28(1), 57 (2004)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Worcester Polytechnic InstituteWorcesterUSA

Personalised recommendations