Multisensory Consumer-Packaging Interaction (CPI): The Role of New Technologies

  • Olivia Petit
  • Carlos Velasco
  • Charles Spence


The recent development of various sensory-enabling technologies (SETs) has attracted the interest of those marketers wishing to enhance the online and in-store multisensory experiences that they offer to customers. Such technologies have also proven relevant to the delivery of more engaging multisensory human-food interactions. However, to date, little work has been conducted on their potential role in the interaction between consumers and product packaging, a key element of branding. In this chapter, we present an overview of how the latest SETs can be (and in some cases already are being) incorporated into the packaging of various different products in order to deliver novel multisensory product experiences. We predict that these technologies will increasingly come to enhance the scope of packaging as a marketing communication tool. They might, for instance, be used to project people into consumption experiences, promote brand engagement, as well as improve product evaluation, by means of, say, augmented reality applications. Such technologies will become an increasingly important element in the consumer experience. They may even be able to enhance the perceived sensory properties of products, help in personalization, and/or help regulating our eating behaviour.


Sensory marketing Digital marketing Packaging Consumer behaviour Augmented reality Interactive technologies 


  1. Ampuero, O., & Vila, N. (2006). Consumer perceptions of product packaging. Journal of Consumer Marketing, 23(2), 100–112.CrossRefGoogle Scholar
  2. Animesh, A., Pinsonneault, A., Yang, S. B., & Oh, W. (2011). An odyssey into virtual worlds: Exploring the impacts of technological and spatial environments on intention to purchase virtual products. MIS Quarterly, 35(3), 789–810.CrossRefGoogle Scholar
  3. Barsalou, L. W. (2008). Grounded cognition. Annual Review of Psychology, 59, 617–645.CrossRefGoogle Scholar
  4. Batra, R., & Homer, P. M. (2004). The situational impact of brand image beliefs. Journal of Consumer Psychology, 14(3), 318–330.CrossRefGoogle Scholar
  5. Biggs, L., Juravle, G., & Spence, C. (2016). Haptic exploration of plateware alters the perceived texture and taste of food. Food Quality & Preference, 50, 129–134.CrossRefGoogle Scholar
  6. Billeter, D., Zhu, M., & Inman, J. J. (2012). Transparent packaging and consumer purchase decisions. In J. Sevilla (Ed.), When it’s what’s outside that matters: Recent findings on product and packaging design. Paper presented at Association for Consumer Research 2012 Conference. Vancouver, Canada (pp. 308–312).Google Scholar
  7. Bonetti, F., Warnaby, G., & Quinn, L. (2018). Augmented reality and virtual reality in physical and online retailing: A review, synthesis and research agenda. In Augmented reality and virtual reality (pp. 119–132). Cham, Switzerland: Springer.CrossRefGoogle Scholar
  8. Brakus, J. J., Schmitt, B. H., & Zarantonello, L. (2009). Brand experience: What is it? How is it measured? Does it affect loyalty. Journal of Marketing, 73(3), 52–68.CrossRefGoogle Scholar
  9. Chandran, S., Batra, R. K., & Lawrence, B. (2009). Is seeing believing? Consumer responses to opacity of product packaging. In A. L. McGill & S. Shavitt (Eds.), Advances in consumer research (Vol. 36, p. 970). Duluth, MN: Association for Consumer Research.Google Scholar
  10. Choi, J. H. J., Foth, M., & Hearn, G. (Eds.). (2014). Eat, cook, grow: Mixing human-computer interactions with human-food interactions. Cambridge, MA: MIT Press.Google Scholar
  11. Choi, Y. K., & Taylor, C. R. (2014). How do 3-dimensional images promote products on the internet. Journal of Business Research, 67(10), 2164–2170.CrossRefGoogle Scholar
  12. Chu, S., & Downes, J. J. (2000). Odour-evoked autobiographical memories: Psychological investigations of Proustian phenomena. Chemical Senses, 25(1), 111–116.CrossRefGoogle Scholar
  13. Comber, R., Choi, J. H. J., Hoonhout, J., & O’Hara, K. (2014). Designing for human–food interaction: An introduction to the special issue on ‘food and interaction design. International Journal of Human-Computer Studies, 72(2), 181–184.CrossRefGoogle Scholar
  14. Connell, P. M., Brucks, M., & Nielsen, J. H. (2014). How childhood advertising exposure can create biased product evaluations that persist into adulthood. Journal of Consumer Research, 41(1), 119–134.CrossRefGoogle Scholar
  15. Cowburn, G., & Stockley, L. (2005). Consumer understanding and use of nutrition labelling: A systematic review. Public Health Nutrition, 8(1), 21–28.CrossRefGoogle Scholar
  16. Creusen, M. E., & Schoormans, J. P. (2005). The different roles of product appearance in consumer choice. Journal of Product Innovation Management, 22(1), 63–81.CrossRefGoogle Scholar
  17. Dawar, N., & Parker, P. (1994). Marketing universals: Consumers’ use of brand name, price, physical appearance, and retailer reputation as signals of product quality. Journal of Marketing, 58(2), 81–95.Google Scholar
  18. Elder, R., & Krishna, A. (2012). The visual depiction effect: Inducing embodied mental simulation that evokes motor responses. Journal of Consumer Research, 38(6), 988–1003.CrossRefGoogle Scholar
  19. Elder, R. S., Schlosser, A. E., Poor, M., & Xu, L. (2017). So close I can almost sense it: The interplay between sensory imagery and psychological distance. Journal of Consumer Research, 44(1), 877–894.CrossRefGoogle Scholar
  20. Geier, A., Wansink, B., & Rozin, P. (2012). Red potato chips: Segmentation cues can substantially decrease food intake. Health Psychology, 31(3), 521–525.CrossRefGoogle Scholar
  21. Glenday, J. (2017). Guinness tantalises Tesco shoppers with VR tasting experience. Retrieved from
  22. Goel, L., & Prokopec, S. (2009). If you build it will they come?—An empirical investigation of consumer perceptions and strategy in virtual worlds. Electronic Commerce Research, 9(1–2), 115–134.CrossRefGoogle Scholar
  23. Grunert, K. G., Wills, J. M., & Fernández-Celemín, L. (2010). Nutrition knowledge, and use and understanding of nutrition information on food labels among consumers in the UK. Appetite, 55(2), 177–189.CrossRefGoogle Scholar
  24. Iglesias, O., Singh, J. J., & Batista-Foguet, J. M. (2011). The role of brand experience and affective commitment in determining brand loyalty. Journal of Brand Management, 18(8), 570–582.CrossRefGoogle Scholar
  25. Javornik, A. (2016). Augmented reality: Research agenda for studying the impact of its media characteristics on consumer behaviour. Journal of Retailing and Consumer Services, 30, 252–261.CrossRefGoogle Scholar
  26. Karnal, N., Machiels, C. J., Orth, U. R., & Mai, R. (2016). Healthy by design, but only when in focus: Communicating non-verbal health cues through symbolic meaning in packaging. Food Quality and Preference, 52, 106–119.CrossRefGoogle Scholar
  27. Kent, A., Dennis, C., Cano, M. B., Helberger, E., & Brakus, J. (2018). Branding, marketing, and design: Experiential in-store digital environments. In Fashion and Textiles: Breakthroughs in Research and Practice (pp. 275–298). Hershey, PA: IGI Global.Google Scholar
  28. Kim, J., & Forsythe, S. (2008a). Sensory enabling technology acceptance model (SE-TAM): A multiple-group structural model comparison. Psychology & Marketing, 25(9), 901–922.CrossRefGoogle Scholar
  29. Kim, J., & Forsythe, S. (2008b). Adoption of virtual try-on technology for online apparel shopping. Journal of Interactive Marketing, 22(2), 45–59.CrossRefGoogle Scholar
  30. Klein, L. R. (2003). Creating virtual product experiences: The role of telepresence. Journal of Interactive Marketing, 17(1), 41–55.CrossRefGoogle Scholar
  31. Knoeferle, K., Knoeferle, P., Velasco, C., & Spence, C. (2016). Multisensory brand search: How the meaning of sounds guides consumers’ visual attention. Journal of Experimental Psychology: Applied, 22, 196–210.Google Scholar
  32. Krishna, A., Cian, L., & Aydınoğlu, N. Z. (2017). Sensory aspects of package design. Journal of Retailing, 93(1), 43–54.CrossRefGoogle Scholar
  33. Krishna, A., Lwin, M. O., & Morrin, M. (2010). Product scent and memory. Journal of Consumer Research, 37(1), 57–67.CrossRefGoogle Scholar
  34. Lam, M. C., Nizam, S. S. M., Arshad, H., A’isyah Ahmad Shukri, S., Hashim, N. C., Putra, H. M., & Abidin, R. Z. (2017, October). A usability evaluation of an interactive application for halal products using optical character recognition and augmented reality technologies. In AIP Conference Proceedings (Vol. 1891, No. 1, p. 020084). AIP Publishing.Google Scholar
  35. Lee, K. C., & Chung, N. (2008). Empirical analysis of consumer reaction to the virtual reality shopping mall. Computers in Human Behavior, 24(1), 88–104.CrossRefGoogle Scholar
  36. Li, H., Daugherty, T., & Biocca, F. (2002). Impact of 3-D advertising on product knowledge, brand attitude, and purchase intention: The mediating role of presence. Journal of Advertising, 31(3), 43–57.CrossRefGoogle Scholar
  37. McCarthy, E. J. (1960). Basic marketing: A managerial approach. Homewood, IL: Richard D. Irwin.Google Scholar
  38. Milosavljevic, M., Navalpakkam, V., Koch, C., & Rangel, A. (2012). Relative visual saliency differences induce sizable bias in consumer choice. Journal of Consumer Psychology, 22, 67–74.CrossRefGoogle Scholar
  39. Morewedge, C. K., Huh, Y. E., & Vosgerau, J. (2010). Thought for food: Imagined consumption reduces actual consumption. Science, 330(6010), 1530–1533.CrossRefGoogle Scholar
  40. Mumani, A., & Stone, R. (2018). State of the art of user packaging interaction (UPI). Packaging Technology and Science, 31(6), 401-419.CrossRefGoogle Scholar
  41. Nah, F. F. H., Eschenbrenner, B., & DeWester, D. (2011). Enhancing brand equity through flow and telepresence: A comparison of 2D and 3D virtual worlds. MIS Quarterly, 35(3), 731–747.CrossRefGoogle Scholar
  42. Narumi, T., Ban, Y., Kajinami, T., Tanikawa, T., & Hirose, M. (2012, May). Augmented perception of satiety: Controlling food consumption by changing apparent size of food with augmented reality. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 109–118). ACM.Google Scholar
  43. Narumi, T., Nishizaka, S., Kajinami, T., Tanikawa, T., & Hirose, M. (2011, May). Augmented reality flavors: Gustatory display based on edible marker and cross-modal interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 93–102). ACM.Google Scholar
  44. Nishizawa, M., Jiang, W., & Okajima, K. (2016, November). Projective-AR system for customizing the appearance and taste of food. In Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality (p. 6). ACM.Google Scholar
  45. Obrist, M., Gatti, E., Maggioni, E., Vi, C. T., & Velasco, C. (2017). Multisensory experiences in HCI. IEEE MultiMedia, 24(2), 9–13.CrossRefGoogle Scholar
  46. Obrist, M., Velasco, C., Vi, C. T., Ranasinghe, N., Israr, A., Cheok, A. D., Spence, C., & Gopalakrishnakone, P. (2016, May). Touch, taste, & smell user interfaces: The future of multisensory HCI. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (pp. 3285–3292). ACM.Google Scholar
  47. Okajima, K., & Spence, C. (2011). Effects of visual food texture on taste perception. i-Perception, 2(8), 966–966.CrossRefGoogle Scholar
  48. Pantano, E., & Naccarato, G. (2010). Entertainment in retailing: The influences of advanced technologies. Journal of Retailing and Consumer Services, 17(3), 200–204.CrossRefGoogle Scholar
  49. Parise, C. V., & Spence, C. (2012). Assessing the associations between brand packaging and brand attributes using an indirect performance measure. Food Quality and Preference, 24(1), 17–23.CrossRefGoogle Scholar
  50. Petit, O., Basso, F., Merunka, D., Spence, C., Cheok, A. D., & Oullier, O. (2016). Pleasure and the control of food intake: An embodied cognition approach to consumer self-regulation. Psychology & Marketing, 33(8), 608–619.CrossRefGoogle Scholar
  51. Petit, O., Cheok, A. D., & Oullier, O. (2016). Can food porn make us slim? How brains of consumers react to food in digital environments. Integrative food. Nutrition and Metabolism, 3(1), 251–255.Google Scholar
  52. Petit, O., Cheok, A. D., Spence, C., Velasco, C., & Karunanayaka, K. T. (2015, November). Sensory marketing in light of new technologies. Workshop in the 12th International Conference on Advances in Computer Entertainment Technology (p. 53). ACM.Google Scholar
  53. Petit, O., Merunka, D., Anton, J. L., Nazarian, B., Spence, C., Cheok, A. D., … Oullier, O. (2016). Health and pleasure in consumers’ dietary food choices: Individual differences in the brain’s value system. PLoS ONE, 11(7), e0156333.CrossRefGoogle Scholar
  54. Petit, O., & Spence, C. (2017). Using food insecurity in health prevention to promote consumer’s embodied self-regulation. Behavioral and Brain Sciences, 40, e126.CrossRefGoogle Scholar
  55. Petit, O., Spence, C., Velasco, C., Woods, A. T., & Cheok, A. D. (2017). Changing the influence of portion size on consumer behavior via imagined consumption. Journal of Business Research, 75, 240–248.CrossRefGoogle Scholar
  56. Petit, O., Velasco, C., & Spence, C. (in press). Digital sensory marketing: Integrating new technologies into multisensory online experience. Journal of Interactive Marketing.Google Scholar
  57. Pilditch, J. G. C. (1961). The silent salesman. How to develop packaging that sells. London, UK: Business Publications.Google Scholar
  58. Pine, B. J., & Gilmore, J. H. (1998). Welcome to the experience economy. Harvard Business Review, 76, 97–105.Google Scholar
  59. Piqueras-Fiszman, B., & Spence, C. (2012). The influence of the feel of product packaging on the perception of the oral-somatosensory texture of food. Food Quality and Preference, 26(1), 67–73.CrossRefGoogle Scholar
  60. Piqueras-Fiszman, B., Velasco, C., Salgado-Montejo, A., & Spence, C. (2013). Using combined eye tracking and word association in order to assess novel packaging solutions: A case study involving jam jars. Food Quality and Preference, 28(1), 328–338.CrossRefGoogle Scholar
  61. Reimann, M., Zaichkowsky, J., Neuhaus, C., Bender, T., & Weber, B. (2010). Aesthetic package design: A behavioral, neural, and psychological investigation. Journal of Consumer Psychology, 20(4), 431–441.CrossRefGoogle Scholar
  62. Rollo, M. E., Bucher, T., Smith, S. P., & Collins, C. E. (2017). ServAR: An augmented reality tool to guide the serving of food. International Journal of Behavioral Nutrition and Physical Activity, 14, 65.CrossRefGoogle Scholar
  63. Rolls, B. J., Morris, E. L., & Roe, L. S. (2002). Portion size of food affects energy intake in normal-weight and overweight men and women. The American Journal of Clinical Nutrition, 76(6), 1207–1213.CrossRefGoogle Scholar
  64. Salgado-Montejo, A., Velasco, C., Ariza, L., Salgado, R., & Moreno, A. M. (2017). The four moments of experience: Streamlining the process of packaging development. ESOMAR WORLD CONGRESS 2017. ISBN: 92-831-0293-2.Google Scholar
  65. Scholz, J., & Smith, A. N. (2016). Augmented reality: Designing immersive experiences that maximize consumer engagement. Business Horizons, 59(2), 149–161.CrossRefGoogle Scholar
  66. Silayoi, P., & Speece, M. (2004). Packaging and purchase decisions: An exploratory study on the impact of involvement level and time pressure. British Food Journal, 106(8), 607–628.CrossRefGoogle Scholar
  67. Silayoi, P., & Speece, M. (2007). The importance of packaging attributes: A conjoint analysis approach. European Journal of Marketing, 41(11/12), 1495–1517.CrossRefGoogle Scholar
  68. Simmonds, G., & Spence, C. (2017). Thinking inside the box: How seeing products on, or through, the packaging influences consumer perceptions and purchase behaviour. Food Quality and Preference, 62, 340–351.CrossRefGoogle Scholar
  69. Spence, C. (2012). Managing sensory expectations concerning products and brands: Capitalizing on the potential of sound and shape symbolism. Journal of Consumer Psychology, 22(1), 37–54.CrossRefGoogle Scholar
  70. Spence, C. (2015). Leading the consumer by the nose: On the commercialization of olfactory design for the food and beverage sector. Flavour, 4(1), 31.CrossRefGoogle Scholar
  71. Spence, C. (2016). Multisensory packaging design: Color, shape, texture, sound, and smell. In M. Chen & P. Burgess (Eds.), Integrating the packaging and product experience: A road-map to consumer satisfaction (pp. 1–22). Oxford, UK: Elsevier.Google Scholar
  72. Spence, C., Obrist, M., Velasco, C., & Ranasinghe, N. (2017). Digitizing the chemical senses: Possibilities & pitfalls. International Journal of Human-Computer Studies, 107, 62–74.CrossRefGoogle Scholar
  73. Spence, C., Okajima, K., Cheok, A. D., Petit, O., & Michel, C. (2016). Eating with our eyes: From visual hunger to digital satiation. Brain and Cognition, 110, 53–63.CrossRefGoogle Scholar
  74. Spence, C., Puccinelli, N., Grewal, D., & Roggeveen, A. L. (2014). Store atmospherics: A multisensory perspective. Psychology & Marketing, 31, 472–488.CrossRefGoogle Scholar
  75. Spence, C., & Wang, Q. J. (2015a). Wine and music (II): Can you taste the music? Modulating the experience of wine through music and sound. Flavour, 4(1), 33.CrossRefGoogle Scholar
  76. Spence, C., & Wang, Q. J. (2015b). Sensory expectations elicited by the sounds of opening the packaging and pouring a beverage. Flavour, 4(1), 35.CrossRefGoogle Scholar
  77. Spence, C., & Wang, Q. J. (2017). Assessing the impact of closure type on wine ratings and mood. Beverages, 3(4): 52.201Google Scholar
  78. Sundar, A., & Noseworthy, T. J. (2014). Place the logo high or low? Using conceptual metaphors of power in packaging design. Journal of Marketing, 78(5), 138–151.CrossRefGoogle Scholar
  79. Thaler, R. H., & Sunstein, C. (2008). Nudge. Improving decisions about health, wealth and happiness. London, UK: Penguin Books.Google Scholar
  80. Tormala, Z. L., & Petty, R. E. (2004). Source credibility and attitude certainty: A metacognitive analysis of resistance to persuasion. Journal of Consumer Psychology, 14(4), 427–442.CrossRefGoogle Scholar
  81. Underwood, R. L. (2003). The communicative power of product packaging: Creating brand identity via lived and mediated experience. Journal of Marketing Theory and Practice, 11(1), 62–76.CrossRefGoogle Scholar
  82. Underwood, R. L., & Klein, N. M. (2002). Packaging as brand communication: Effects of product pictures on consumer responses to the package and brand. Journal of Marketing Theory and Practice, 10(4), 58–68.CrossRefGoogle Scholar
  83. Underwood, R. L., Klein, N. M., & Burke, R. R. (2001). Packaging communication: Attentional effects of product imagery. Journal of Product & Brand Management, 10(7), 403–422.CrossRefGoogle Scholar
  84. Välkkynen, P., Boyer, A., Urhemaa, T., & Nieminen, R. (2011). Mobile augmented reality for retail environments. In Mobile HCI 2011 Workshop on Mobile Interaction in Retail Environments. ACM.Google Scholar
  85. Van Rijswijk, W., & Frewer, L. J. (2008). Consumer perceptions of food quality and safety and their relation to traceability. British Food Journal, 110(10), 1034–1046.CrossRefGoogle Scholar
  86. Van Rompay, T. J. L., De Vries, P. W., Bontekoe, F., & Tanja-Dijkstra, K. (2012). Embodied product perception: Effects of verticality cues in advertising and packaging design on consumer impressions and price expectations. Psychology & Marketing, 29(12), 919–928.CrossRefGoogle Scholar
  87. Van Rompay, T. J. L., Finger, F., Saakes, D., & Fenko, A. (2017). “See me, feel me”: Effects of 3D-printed surface patterns on beverage evaluation. Food Quality & Preference, 62, 332–339.CrossRefGoogle Scholar
  88. Van Rompay, T. J. L., Kramer, L.-M., & Saakes, D. (2018). The sweetest punch: Effects of 3D-printed surface textures and graphic design on ice-cream evaluation. Food Quality and Preference, 68, 198–204.CrossRefGoogle Scholar
  89. Van Waterschoot, W., & Van den Bulte, C. (1992). The 4P classification of the marketing mix revisited. The Journal of Marketing, 56(4), 83–93.CrossRefGoogle Scholar
  90. Vazquez, E., Gevers, T., Lucassen, M., van de Weijer, J., & Baldrich, R. (2010). Saliency of color image derivatives: A comparison between computational models and human perception. Journal of the Optical Society of America. A, Optics, Image Science and Vision, 27(3), 613–621.CrossRefGoogle Scholar
  91. Velasco, C., Carvalho, F. R., Petit, O., & Nijholt, A. (2016). A multisensory approach for the design of food and drink enhancing sonic systems. In Proceedings of the 1st Workshop on Multi-sensorial Approaches to Human-Food Interaction (p. 7). ACM.Google Scholar
  92. Velasco, C., Jones, R., King, S., & Spence, C. (2013). The sound of temperature: What information do pouring sounds convey concerning the temperature of a beverage. Journal of Sensory Studies, 28(5), 335–345.CrossRefGoogle Scholar
  93. Velasco, C., Karunanayaka, K., & Nijholt, A. (Eds.). (2018). Multisensory human-food interaction. Lausanne: Frontiers Media.Google Scholar
  94. Velasco, C., Obrist, M., Petit, O., & Spence, C. (2018). Multisensory technology for flavor augmentation: A mini review. Frontiers in Psychology, 9, 26.CrossRefGoogle Scholar
  95. Velasco, C., Woods, A. T., Petit, O., Cheok, A. D., & Spence, C. (2016). Crossmodal correspondences between taste and shape, and their implications for product packaging: A review. Food Quality and Preference, 52, 17–26.CrossRefGoogle Scholar
  96. Victor, A. (2015, January 8). Is this the future of food? Virtual reality experiment lets you eat anything you want without worrying about calories or allergies. DailyMail Online. Retrieved May 7, 2015, from
  97. Wansink, B., & Chandon, P. (2006). Meal size, not body size, explains errors in estimating the calorie content of meals. Annals of Internal Medicine, 145(5), 326–332.CrossRefGoogle Scholar
  98. Zampini, M., & Spence, C. (2004). The role of auditory cues in modulating the perceived crispness and staleness of potato chips. Journal of Sensory Studies, 19(5), 347–363.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2019

Authors and Affiliations

  • Olivia Petit
    • 1
  • Carlos Velasco
    • 2
  • Charles Spence
    • 3
  1. 1.Department of MarketingKEDGE Business SchoolMarseilleFrance
  2. 2.Department of MarketingBI Norwegian Business SchoolOsloNorway
  3. 3.Crossmodal Research Laboratory, Department of Experimental PsychologyUniversity of OxfordOxfordUK

Personalised recommendations