Skip to main content
SpringerLink
Log in

Effect of Image Captioning with Description on the Working Memory

  • Conference paper
  • First Online:
HCI International 2021 - Late Breaking Papers: Cognition, Inclusion, Learning, and Culture (HCII 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 13096))

Included in the following conference series:

  • 1253 Accesses

Abstract

Working memory plays an important role in human activities across academic, professional, and social settings. Working memory is defined as the memory extensively involved in goal-directed behaviors in which information must be retained and manipulated to ensure successful task execution. The aim of this research is to understand the effect of image captioning with image description on an individual’s working memory. A study was conducted with eight neutral images comprising situations relatable to daily life such that each image could have a positive or negative description associated with the outcome of the situation in the image. The study consisted of three rounds where the first and second round involved two parts and the third round consisted of one part. The image was captioned a total of five times across the entire study. The findings highlighted that only 25% of participants were able to recall the captions which they captioned for an image after a span of 9–15 days; when comparing the recall rate of the captions, 50% of participants were able to recall the image caption from the previous round in the present round; and out of the positive and negative description associated with the image, 65% of participants recalled the former description rather than the latter.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Allen, R., Baddeley, A., Hitch, G.: Is the binding of visual features in working memory resource-demanding? J. Exp. Psychol. Gen. 135, 298–313 (2006). https://doi.org/10.1037/0096-3445.135.2.298

    Article  Google Scholar 

  2. Atkinson, R.C., Shiffrin, R.M.: Human memory: A proposed system and its control processes. In: Psychology of Learning and Motivation, vol. 2, pp. 89–195. Elsevier (1968)

    Google Scholar 

  3. Baddeley, A.: Working memory: looking back and looking forward. Nat. Rev. Neurosci. 4, 829–839 (2003). https://doi.org/10.1038/nrn1201

    Article  Google Scholar 

  4. Baddeley, A.: Working memory. Curr. Biol. 20(4), R136–R140 (2010)

    Article  Google Scholar 

  5. Baddeley, A.: Working memory: theories, models, and controversies. Ann. Rev. Psychol. 63(1), 1–29 (2012). https://doi.org/10.1146/annurev-psych-120710-100422. pMID: 21961947

    Article  MathSciNet  Google Scholar 

  6. Blalock, L., Clegg, B.: Encoding and representation of simultaneous and sequential arrays in visuospatial working memory. Q. J. Exp. Psychol. 2006(63), 856–62 (2010). https://doi.org/10.1080/17470211003690680

    Article  Google Scholar 

  7. Broadbent, D.E.: Percept. Commun. Pergamon Press, New York (1958)

    Book  Google Scholar 

  8. Buchanan, T., Adolphs, R.: The role of the human amygdala in emotional modulation of long-term declarative memory. Adv. Cons. Res. 44, 9–34 (2002). https://doi.org/10.1075/aicr.44.02buc

    Article  Google Scholar 

  9. Chai, W.J., Abd Hamid, A.I., Abdullah, J.M.: Working memory from the psychological and neurosciences perspectives: a review. Front. Psychol. 9, 401 (2018)

    Article  Google Scholar 

  10. Cheng, P., Holyoak, K.: Pragmatic reasoning schemas. Cogn. Psychol. 17, 391–416 (1985). https://doi.org/10.1016/0010-0285(85)90014-3

    Article  Google Scholar 

  11. Cowan, N.: What are the differences between long-term, short-term, and working memory? Prog. Brain Res. 169, 323–38 (2008)

    Article  Google Scholar 

  12. Cowan, N.: The magical number 4 in short-term memory: a reconsideration of mental storage capacity. Behav. Brain Sci. 24(1), 87–114 (2001). https://doi.org/10.1017/S0140525X01003922

    Article  Google Scholar 

  13. Daneman, M., Carpenter, P.A.: Individual differences in working memory and reading. J. Verbal Learn. Verbal Behav. 19(4), 450–466 (1980)

    Article  Google Scholar 

  14. Daneman, M., Merikle, P.: Working memory and language comprehension: a meta-analysis. Psychon. Bull. Rev. 3, 422–433 (1996). https://doi.org/10.3758/BF03214546

    Article  Google Scholar 

  15. D’Esposito, M., Postle, B.R.: The cognitive neuroscience of working memory. Ann. Rev. Psychol. 66(1), 115–142 (2015). https://doi.org/10.1146/annurev-psych-010814-015031. pMID: 25251486

    Article  Google Scholar 

  16. Doerksen, S., Shimamura, A.: Source memory enhancement for emotional words. Emotion 1, 5–11 (2001). https://doi.org/10.1037//1528-3542.1.1.5. (Washington, D.C.)

    Article  Google Scholar 

  17. Dolan, R.: Emotion, cognition, and behavior. Science 298, 1191–1194 (2002). https://doi.org/10.1126/science.1076358. (New York, NY)

  18. Elliman, N., Green, M., Rogers, P., Finch, G.: Processing-efficiency theory and the working-memory system: Impairments associated with sub-clinical anxiety. Pers. Individ. Differ. 23, 31–35 (1997). https://doi.org/10.1016/S0191-8869(97)00016-0

    Article  Google Scholar 

  19. Engle, R., Tuholski, S.W., Laughlin, J., Conway, A.: Working memory, short-term memory, and general fluid intelligence: a latent-variable approach. J. Exp. Psychol. Gen. 128(3), 309–331 (1999)

    Article  Google Scholar 

  20. Engle, R., Tuholski, S., Laughlin, J., Conway, A.: Working memory, short-term memory and general fluid intelligence: a latent variable approach. J. Exp. Psychol. Gen. 130, 169–183 (1999). https://doi.org/10.1037/0096-3445.128.3.309

    Article  Google Scholar 

  21. Eysenck, M., Calvo, M.: Anxiety and performance: the processing efficiency theory. Cogn. Emot. 6, 409–434 (1992). https://doi.org/10.1080/02699939208409696

    Article  Google Scholar 

  22. Fukuda, K., Vogel, E., Mayr, U., Awh, E.: Quantity, not quality: the relationship between fluid intelligence and working memory capacity. Psychon. Bull. Rev. 17, 673–679 (2010). https://doi.org/10.3758/17.5.673

    Article  Google Scholar 

  23. Gray, J.: Emotional modulation of cognitive control: approach-withdrawal states double dissociate spatial from verbal 2-back task performance. J. Exp. Psychol. Gen. 130, 436–52 (2001). https://doi.org/10.1037/0096-3445.130.3.436

    Article  Google Scholar 

  24. Hamann, S.: Cognitive and neural mechanisms of emotional memory. Trends Cogn. Sci. 5, 394–400 (2001). https://doi.org/10.1016/S1364-6613(00)01707-1

    Article  Google Scholar 

  25. Harden, L.: A review of research on working memory and its importance in education of the deaf. Ph.D. thesis, Program in Audiology and Communication Sciences, Washington University (2011). http://digitalcommons.wustl.edu/pacs_capstones/627

  26. Heuer, F., Reisberg, D.: Vivid memories of emotional events: the accuracy of remembered minutiae. Mem. Cogn. 18, 496–506 (1990). https://doi.org/10.3758/BF03198482

    Article  Google Scholar 

  27. Johnson, A., Miles, C.: Serial position effects in 2-alternative forced choice recognition: functional equivalence across visual and auditory modalities. Memory 17, 84–91 (2008). https://doi.org/10.1080/09658210802557711. (Hove, England)

    Article  Google Scholar 

  28. Kyllonen, P.C., Christal, R.E.: Reasoning ability is (little more than) working-memory capacity?! Intelligence 14(4), 389–433 (1990)

    Article  Google Scholar 

  29. Lecerf, T., de Ribaupierre, A.: Recognition in a visuospatial memory task: the effect of presentation. Eur. J. Cogn. Psychol. 17, 47–75 (2005). https://doi.org/10.1080/09541440340000420

    Article  Google Scholar 

  30. Luck, S., Vogel, E.: The capacity of visual working memory for features and conjunctions. Nature 390, 279–81 (1997). https://doi.org/10.1038/36846

    Article  Google Scholar 

  31. MacDonald, M.: Your Brain: The Missing Manual: The Missing Manual. O’Reilly Media, Sebastopol (2008)

    Google Scholar 

  32. Miller, G., Galanter, E., Pribram, K.: Plans and the structure of behavior. Am. J. Psychol. 75 (1960). https://doi.org/10.2307/1419559

  33. Oberauer, K., Cowan, N.: Working memory capacity: (2005). Exp. Psychol. 54, 245–246 (2007). https://doi.org/10.1027/1618-3169.54.3.245

    Article  Google Scholar 

  34. Pashler, H.: Familiarity and visual change detection. Percept. Psychophys. 44, 369–78 (1988)

    Article  Google Scholar 

  35. Perlstein, W., Elbert, T., Stenger, V.: Dissociation in human prefrontal cortex of affective influences on working memory-related activity. In: Proceedings of the National Academy of Sciences of the United States of America, vol. 99, pp. 1736–41, March 2002. https://doi.org/10.1073/pnas.241650598

  36. Seibert, P., Ellis, H.: Irrelevant thoughts, emotional mood states, and cognitive task performance. Mem. Cogn. 19, 507–13 (1991). https://doi.org/10.3758/BF03199574

    Article  Google Scholar 

  37. Smyth, M., Hay, D., Hitch, G., Horton, N.: Serial position memory in the visual-spatial domain: reconstructing sequences of unfamiliar faces. Q. J. Exp. Psychol. Hum. Exp. Psychol. 58, 909–30 (2005). https://doi.org/10.1080/02724980443000412

    Article  Google Scholar 

  38. Spies, K., Hesse, F., Hummitzsch, C.: Mood and capacity in baddeley’s model of human memory. Zeitschrift für Psychologie mit Zeitschrift für angewandte Psychologie 204, 367–381 (1996)

    Google Scholar 

  39. Vogel, E., Woodman, G., Luck, S.: Storage of features, conjunctions, and objects in visual working memory. J. Exp. Psychol. Hum. Percept. Perform. 27, 92–114 (2001). https://doi.org/10.1037//0096-1523.27.1.92

    Article  Google Scholar 

  40. William, P., Phillips, W.A.: on the distinction between sensory storage and short-term visual memory. Percept. Psychophys. 16, 283–290 (1974). https://doi.org/10.3758/BF03203943

    Article  Google Scholar 

  41. William, P., Christie, D.: Components of visual memory. Q. J. Exp. Psychol. 29, 117–133 (1977). https://doi.org/10.1080/00335557743000080

    Article  Google Scholar 

  42. Wolf, T., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38–45. Association for Computational Linguistics, Online, October 2020. https://www.aclweb.org/anthology/2020.emnlp-demos.6

Download references

Author information

Authors and Affiliations

  1. School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, USA

    Nithiya Shree Uppara & Hemanth Venkateswara

  2. The Polytechnic School, Arizona State University, Mesa, AZ, USA

    Troy McDaniel

Authors
  1. Nithiya Shree Uppara
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Troy McDaniel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hemanth Venkateswara
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nithiya Shree Uppara .

Editor information

Editors and Affiliations

  1. University of Crete and Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Constantine Stephanidis

  2. Coventry University, Coventry, UK

    Don Harris

  3. Safety and Accident Investigation Centre, Cranfield University, Cranfield, UK

    Wen-Chin Li

  4. Soar Technology Inc., Orlando, FL, USA

    Dylan D. Schmorrow

  5. Design Interactive, Inc., Orlando, FL, USA

    Cali M. Fidopiastis

  6. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Margherita Antona

  7. Tsinghua University, Beijing, China

    Qin Gao

  8. Chongqing University, Chongqing, China

    Jia Zhou

  9. Department of Multimedia and Graphic Arts, Cyprus University of Technology, Limassol, Cyprus

    Panayiotis Zaphiris

  10. Department of Multimedia and Graphic Arts, Cyprus University of Technology, Limassol, Cyprus

    Andri Ioannou

  11. Soar Technology Inc., Orlando, FL, USA

    Robert A. Sottilare

  12. MMS, Fraunhofer FKIE, Wachtberg, Nordrhein-Westfalen, Germany

    Jessica Schwarz

  13. Industrial Design, Eindhoven University of Technology, Eindhoven, Noord-Brabant, The Netherlands

    Matthias Rauterberg

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Uppara, N.S., McDaniel, T., Venkateswara, H. (2021). Effect of Image Captioning with Description on the Working Memory. In: Stephanidis, C., et al. HCI International 2021 - Late Breaking Papers: Cognition, Inclusion, Learning, and Culture. HCII 2021. Lecture Notes in Computer Science(), vol 13096. Springer, Cham. https://doi.org/10.1007/978-3-030-90328-2_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-90328-2_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-90327-5

  • Online ISBN: 978-3-030-90328-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation