The detail is in the difficulty: Challenging search facilitates rich incidental object encoding

Abstract

When searching for objects in the environment, observers necessarily encounter other, nontarget, objects. Despite their irrelevance for search, observers often incidentally encode the details of these objects, an effect that is exaggerated as the search task becomes more challenging. Although it is well established that searchers create incidental memories for targets, less is known about the fidelity with which nontargets are remembered. Do observers store richly detailed representations of nontargets, or are these memories characterized by gist-level detail, containing only the information necessary to reject the item as a nontarget? We addressed this question across two experiments in which observers completed multiple-target (one to four potential targets) searches, followed by surprise alternative forced-choice (AFC) recognition tests for all encountered objects. To assess the detail of incidentally stored memories, we used similarity rankings derived from multidimensional scaling to manipulate the perceptual similarity across objects in 4-AFC (Experiment 1a) and 16-AFC (Experiments 1b and 2) tests. Replicating prior work, observers recognized more nontarget objects encountered during challenging, relative to easier, searches. More importantly, AFC results revealed that observers stored more than gist-level detail: When search objects were not recognized, observers systematically chose lures with higher perceptual similarity, reflecting partial encoding of the search object’s perceptual features. Further, similarity effects increased with search difficulty, revealing that incidental memories for visual search objects are sharpened when the search task requires greater attentional processing.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Notes

  1. 1.

    Previous work has demonstrated that multiple-target search incurs performance costs relative to single-target search (e.g., Menneer et al., 2007; Menneer, Cave, & Donnelly, 2009). However, it is possible that these costs derive from memory demands rather than from task parameters such as search difficulty.

  2. 2.

    Color serves as a basic visual feature, which would make most search processes relatively easy, regardless of other manipulations (see Wolfe & Horowitz, 2017).

  3. 3.

    https://osf.io/p7yk2/

  4. 4.

    We thank an anonymous reviewer for this suggestion.

References

  1. Alexander, R. G., & Zelinsky, G. J. (2011). Visual similarity effects in categorical search. Journal of Vision, 11, 1–15.

    Article  Google Scholar 

  2. Alexander, R. G., & Zelinsky, G. J. (2012). Effects of part-based similarity on visual search: The Frankenbear experiment. Vision Research, 54, 20-30.

    PubMed  PubMed Central  Article  Google Scholar 

  3. Antonelli, K. B., & Williams, C. C. (2017). Task-relevant perceptual features can define categories in visual memory too. Memory & Cognition, 45, 1295-1305.

    Article  Google Scholar 

  4. Balaban, H. Assaf, D., Arad Meir, M., & Luria, R. (2019). Different features of real-world objects are represented in a dependent manner in long-term memory. Journal of Experimental Psychology: General. https://doi.org/10.1037/xge0000716

  5. Berman, M. G., Hout, M. C., Kardan, O., Hunter, M., Yourganov, G., Henderson, J. M., Hanayik, T., Karimi, H., & Jonides, J. (2014). The perception of naturalness correlates with low-level visual features of environmental scenes. PLoS ONE, 9: e114572. doi: https://doi.org/10.1371/journal.pone.0114572.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Brady, T. F., Konkle, T., Alvarez, G. A., & Oliva, A. (2008). Visual long-term memory has a massive storage capacity for object details. Proceedings of the National Academy of Science, USA, 105, 14325-14329.

    Article  Google Scholar 

  7. Brady, T. F., Konkle, T., Alvarez, G. A., & Oliva, A. (2013a). Real-world objects are not represented as bound units: Independent forgetting of different object details from visual memory. Journal of Experimental Psychology: General, 142, 791-808.

    Article  Google Scholar 

  8. Brady, T. F., Konkle, T., Gill, J., Oliva, A., & Alvarez, G. A. (2013b). Visual long-term memory has the same limit on fidelity as visual working memory. Psychological Science, 24(6), 981-990.

    Article  Google Scholar 

  9. Castelhano, M. S., & Henderson, J. M. (2005). Incidental visual memory for objects in scenes. Visual Cognition, 12(6), 1017-1040.

    Article  Google Scholar 

  10. Coburn, A., Kardan, O., Kotabe, H., Steinberg, J., Hout, M. C., Robbins, A., MacDonald, J., Hayn-Leichsenring, G., & Berman, M. (2019). Psychological responses to natural patterns in architecture. Journal of Environmental Psychology, 62, 133-145. doi: https://doi.org/10.1016/j.jenvp.2019.02.007.

    Article  Google Scholar 

  11. Cunningham, C. A., Yassa, M. A., & Egeth, H. E. (2015). Massive memory revisited: Limitations on storage capacity for object details in visual long-term memory. Learning & Memory, 22(11), 563-566.

    Article  Google Scholar 

  12. Draschkow, D., Wolfe, J. M., & Võ, M. L. H. (2014). Seek and you shall remember: Scene semantics interact with visual search to build better memories. Journal of Vision, 14(8):10, 1-18.

  13. Draschkow, D., Reinecke, S., Cunningham, C. A., & Võ, M. L. H. (2018). The lower bounds of massive memory: Investigating memory for object details after incidental encoding. Quarterly Journal of Experimental Psychology. doi: https://doi.org/10.1177/1747021818783722

  14. Duncan, J., & Humphreys, G. W. (1989). Visual search and stimulus similarity. Psychological Review, 96(3), 433–458.

    PubMed  Article  PubMed Central  Google Scholar 

  15. Godwin, H., Hout, M. C., & Menneer, T. (2014). Visual similarity is stronger than semantic similarity in guiding visual search for numbers. Psychonomic Bulletin & Review, 21, 689-695. doi: https://doi.org/10.3758/s13423-013-0547-4

    Article  Google Scholar 

  16. Goldinger, S. D. (1998). Echoes or echoes? An episodic theory of lexical access. Psychological Review, 105, 251-279.

    PubMed  Article  PubMed Central  Google Scholar 

  17. Goldstone, R. (1994). An efficient method for obtaining similarity data. Behavior Research Methods, Instruments, & Computers, 26(4), 381-386.

    Article  Google Scholar 

  18. Guerin, S. A., Robbins, C. A., Gilmore, A. W., & Schacter, D. L. (2012). Retrieval failure contributes to gist-based false recognition. Journal of Memory and Language, 66, 68-78. doi: https://doi.org/10.1016/j.jml.2011.07.002

    Article  PubMed  PubMed Central  Google Scholar 

  19. Guevara Pinto, J. D., & Papesh, M. H. (2019). Incidental memory following rapid object processing: The role of attention allocation strategies. Journal of Experimental Psychology: Human Perception and Performance, 45(9), 1174-1190.

    PubMed  Google Scholar 

  20. Hicks, J. L., Marsh, R. L., & Cook, G. I. (2005). Task interference in time-based, event-based, and dual intention prospective memory conditions. Journal of Memory and Language, 53, 430-444.

    Article  Google Scholar 

  21. Hollingworth, A. (2004). Constructing visual representations of natural scenes: The roles of short- and long-term visual memory. Journal of Experimental Psychology: Human Perception & Performance, 30, 519-537.

    Google Scholar 

  22. Hollingworth, A. (2006). Visual memory for natural scenes: Evidence from change detection and visual search. Visual Cognition, 14, 781-807.

    Article  Google Scholar 

  23. Hollingworth, A., & Henderson, J. M. (2002). Accurate visual memory for previously attended objects in natural scenes. Journal of Experimental Psychology: Human Perception and Performance, 28(1), 113-136.

    Google Scholar 

  24. Homa, D., Blair, M., McClure, S. M., Medema, J., & Stone, G. (2018). Learning concepts when instances never repeat. Memory & Cognition, 47(3), 395-411.

    Article  Google Scholar 

  25. Horst, J. S., & Hout, M. C. (2015). The Novel Object and Unusual Name (NOUN) Database: A collection of novel images for use in experimental research. Behavior Research Methods, 48, 1393-1409. doi: https://doi.org/10.3758/s13428-015-0647-3.

    Article  Google Scholar 

  26. Hout, M. C., & Goldinger, S. D. (2010). Learning in repeated visual search. Attention, Perception, & Psychophysics, 72, 1267–1282.

    Article  Google Scholar 

  27. Hout, M. C., & Goldinger, S. D. (2012). Incidental learning speeds visual search by lowering response thresholds, not by improving efficiency: Evidence from eye movements. Journal of Experimental Psychology: Human Perception and Performance, 38(1), 90-112.

    PubMed  PubMed Central  Google Scholar 

  28. Hout, M. C., & Goldinger, S. D. (2015). Target templates: The precision of mental representations affects attentional guidance and decision-making in visual search. Attention, Perception & Psychophysics, 77, 128-149. doi: https://doi.org/10.3758/s13414-014-0764-6.

    Article  Google Scholar 

  29. Hout, M. C., & Goldinger, S. D. (2016). SpAM is convenient, but also satisfying: Reply to Verheyen et al. (2016). Journal of Experimental Psychology: General, 3, 383-387. doi: https://doi.org/10.1037/xge000017.

    Article  Google Scholar 

  30. Hout, M. C., Papesh, M. H., & Goldinger, S. D. (2012). Multidimensional scaling. Wiley Interdisciplinary Reviews (WIREs): Cognitive Science, 4, 93-103. doi: https://doi.org/10.1002/wcs.1203

  31. Hout, M. C., Goldinger, S. D., & Ferguson, R. W. (2013). The versatility of SpAM: A fast, efficient spatial method of data collection for multidimensional scaling. Journal of Experimental Psychology: General, 142, 256-281. doi: https://doi.org/10.1037/a0028860.

    Article  Google Scholar 

  32. Hout, M. C., Goldinger, S. D., & Brady, K. J. (2014). MM-MDS: A multidimensional scaling database with similarity ratings for 240 object categories from the Massive Memory picture database. PloS one, 9(11), e112644

    PubMed  PubMed Central  Article  Google Scholar 

  33. Hout, M. C., Godwin, H. J., Fitzsimmons, G., Robbins, A., Menneer, T., & Goldinger, S. D. (2015). Using multidimensional scaling to quantify similarity in visual search and beyond. Attention, Perception, & Psychophysics, 78, 3-20. doi: https://doi.org/10.3758/s13414-015-1010-6.

    Article  Google Scholar 

  34. JASP Team (2018). JASP (Version 0.8.5)

  35. Josephs, E. L., Draschkow, D., Wolfe, J. M., & Võ, M. L. H. (2016). Gist in time: Scene semantics and structure enhance recall of searched objects. Acta Psychologica, 169, 100-108.

    PubMed  PubMed Central  Article  Google Scholar 

  36. Konkle, T., Brady, T. F., Alvarez, G. A., & Oliva, A. (2010a). Conceptual distinctiveness supports detailed visual long-term memory for real-world objects. Journal of Experimental Psychology: General, 139(3), 558-578.

    Article  Google Scholar 

  37. Konkle, T., Brady, T. F., Alvarez, G. A., & Oliva, A. (2010b). Scene memory is more detailed than you think: The role of categories in visual long-term memory. Psychological Science, 21(11), 1551-1556.

    PubMed  PubMed Central  Article  Google Scholar 

  38. Lourenço, J. S., Hill, J. H., & Maylor, E. A. (2015). Too easy? The influence of task demands conveyed tacitly on prospective memory. Frontiers in Human Neuroscience, 9, 1-6.

    Article  Google Scholar 

  39. Madrid, J., & Hout, M. C. (2019). Examining passive and active strategies on search behavior during hybrid visual memory search: Evidence from eye tracking. Cognitive Research: Principles and Applications, 4(1), 39.

    Google Scholar 

  40. Marsh, R. L., Hicks, J. L., & Cook, G. I. (2005). On the relationship between effort toward an ongoing task and cue detection in event-based prospective memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(1), 68-75.

    PubMed  PubMed Central  Google Scholar 

  41. Marsh, R. L., Cook, G. I., & Hicks, J. L. (2006). Task interference from event-based intentions can be material specific. Memory & Cognition, 34(8), 1636-1643.

    Article  Google Scholar 

  42. Menneer, T., Barrett, D. J. K., Phillips, L., Donnelly, N., & Cave, K. R. (2007). Costs in searching for two targets: Dividing search across target types could improve airport security screening. Applied Cognitive Psychology, 21, 915-932.

    Article  Google Scholar 

  43. Menneer, T., Cave, K. R., & Donnelly, N. (2009). The cost of search for multiple targets: Effects of practice and target similarity. Journal of Experimental Psychology: Applied, 15, 125-139.

    PubMed  Google Scholar 

  44. Nosofsky, R. M. (1984). Choice, similarity, and the context theory of classificiation. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10(1). 104-114

    Google Scholar 

  45. Nosofsky, R. M. (1992). Similarity scaling and cognitive process models. Annual Review of Psychology, 43(1), 25-53.

    Article  Google Scholar 

  46. Papesh, M. H., & Goldinger, S. D. (2010). A multidimensional scaling analysis of own- and cross-race face spaces. Cognition, 116(2), 283-288.

    PubMed  PubMed Central  Article  Google Scholar 

  47. Pedelty, L., Cohen-Levine, S., & Shevell, S. K. (1985). Developmental changes in face processing: Results from multidimensional scaling. Journal of Experimental Child Psychology, 39, 421-236.

    PubMed  Article  Google Scholar 

  48. Rouder, J. N., Morey, R. D., Speckman, P. L., & Province, J. M. (2012). Default bayes factors for ANOVA designs. Journal of Mathematical Psychology, 56(5), 356-374.

    Article  Google Scholar 

  49. Schmidt, J., & Zelinsky, G. J. (2009). Search guidance is proportional to the categorical specificity of a target cue. The Quarterly Journal of Experimental Psychology, 62, 1904–1914.

    Article  Google Scholar 

  50. Shepard, R. N. (1967). Recognition memory for words, sentences, and pictures. Journal of Verbal Learning and Verbal Behavior, 6, 156-163.

    Article  Google Scholar 

  51. Shepard, R. N. (1987). Toward a universal law of generalization for psychological science. Science, 237, 1317—1323

    Article  Google Scholar 

  52. Smilek, D., Enns, J. T., Eastwood, J. D. & Merikle, P. M. (2006) Relax! Cognitive strategy influences visual search. Visual Cognition, 14, 543–64.

    Article  Google Scholar 

  53. Standing, L. (1973). Learning 10,000 pictures. Quarterly Journal of Experimental Psychology, 25, 207-222.

    PubMed  Article  PubMed Central  Google Scholar 

  54. Standing, L., Conezio, J., & Haber, R. N. (1970). Perception and memory for pictures: Single-trial learning of 2,500 visual stimuli. Psychonomic Science, 19, 73-74.

    Article  Google Scholar 

  55. Theeuwes, J. (1994a). Stimulus-driven capture and attention set: Selective search for color and visual abrupt onsets. Journal of Experimental Psychology: Human perception and performance, 20(4), 799-806.

    PubMed  PubMed Central  Google Scholar 

  56. Theeuwes, J. (1994b). Endogenous and exogenous control of visual selection. Perception, 23(4), 429-440.

    PubMed  Article  PubMed Central  Google Scholar 

  57. Thomas, M. D., & Williams, C. C. (2014). The target effect: Visual memory for unnamed search targets. Quarterly Journal of Experimental Psychology, 67(11), 2090-2104.

    Article  Google Scholar 

  58. Torgerson, W. S. (1952). Multidimensional scaling: I. Theory and method. Psychometrika, 17, 401-419.

    Article  Google Scholar 

  59. Utochkin, I. S., & Brady, T. F. (2019). Independent storage of different features of real-world objects in long-term memory. Journal of Experimental Psychology: General. https://doi.org/10.1037/xge0000664

  60. Valentine, T. (1991). A unified account of the effects of distinctiveness, inversion, and race in face recognition. The Quarterly Journal of Experimental Psychology, 43(2), 161-204.

    PubMed  Article  PubMed Central  Google Scholar 

  61. Vogt, S., & Magnussen, S. (2007). Long-term memory for 400 pictures on a common theme. Experimental Psychology, 54, 298-303.

    PubMed  Article  PubMed Central  Google Scholar 

  62. Wagenmakers, E. J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., Selker, R., Gronau, Q. F., Šmíra, M., Epskamp, S., Matzke, D., Rouder, J. N., & Morey, R. D. (2018). Bayesian inference for psychology: Part I: Theoretical advantages and practical ramifications. Psychonomic Bulletin & Review, 25, 35-57.

    Article  Google Scholar 

  63. Whitney, D., & Levi, D. M. (2011). Visual crowding: A fundamental limit on conscious perception and object recognition. Trends in Cognitive Science, 15(4), 160-168.

    Article  Google Scholar 

  64. Williams, C. C. (2010). Incidental and intentional visual memory: What memories are and are not affected by encoding task? Visual Cognition, 18, 1348-1367.

    Article  Google Scholar 

  65. Williams, C. C., Henderson, J. M., & Zacks, R. T. (2005). Incidental visual memory for targets and nontargets in visual search. Perception & Psychophysics, 67, 816-827.

    Article  Google Scholar 

  66. Wolfe, J. M., & Horowitz, T. S. (2017). Five factors that guide attention in visual search. Nature Human Behaviour, 1(3), 1-8.

    Article  Google Scholar 

  67. Wolfe, J. M., Cave, K. R., & Franzel, S. L. (1989). Guided search: An alternative to the feature integration model for visual search. Journal of Experimental Psychology. Human Perception and Performance, 15(3), 419–433.

    PubMed  Article  PubMed Central  Google Scholar 

  68. Yang, H., & Zelinsky, G. J. (2009). Visual search is guided to categorically-defined targets. Vision Research, 49(16), 2095–2103.

    PubMed  PubMed Central  Article  Google Scholar 

  69. Yonelinas, A. P. (2002). The nature of recollection and familiarity: a review of 30 years of research. Journal of Memory and Language, 46, 441-517.

    Article  Google Scholar 

Download references

Acknowledgements

We thank Emily Green for her efforts at helping conceptualize this project, and for overseeing all data collection at NMSU as part of her NMSU Honors College Thesis project. We also thank Jessica Budd and Cameron Grimball for their assistance with data collection.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Michael C. Hout.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendices

Appendix A

Table 4 Object categories used across experiments with average multidimensional vector inter-item distances

Appendix B. Visual search performance

Experiments 1a and 1b

Search accuracy was examined in 3 (Working-Memory Load: High, Medium, Low) × 2 (Number of Targets) repeated-measures ANOVA on search hit rates. The main effect of WM Load in Experiment 1a, F (1.82, 109.06) = 58.13, p < .001, ηp2 = .49, BF10 =5.272e+18, revealed higher hits rate on Low-Load trials (M = .99, SE = .001), followed by Medium-Load (M = .97, SE = .004) and High-Load (M = .96, SE = .005) trials. The main effect of Number of Targets, F (1, 60) = 76.56, p < .001, ηp2 = .56, BF10 =3.614e+7, is interpreted in the context of the reliable interaction, F (2, 120) = 11.58 p < .001, ηp2 = .16, BF10 =247.5: Search hits for one target (M = .98, SE = .002) were higher than two targets (M = .97, SE = .004), but only when searching under High or Medium WM Loads, both ps < .05. This was replicated in Experiment 1b, where a main effect of Number of Targets, F (1, 63) = 45.06, p < .001, ηp2 = .42, BF10 =840,223, and a reliable interaction was observed, F (1.82, 114.39) = 7.86, p < .001, ηp2 = .11, BF10 =21.22. The main effect of WM Load in Experiment, F (2, 126) = 48.05, p < .001, ηp2 = .43, BF10=1.276e+15, again revealed higher hits rate on Low Load trials (M = .99 SE = .002), followed by Medium Load (M = .98 SE = .003) and High Load (M = .97 SE = .004) trials.

Experiment 2

A paired-sample t-test was conducted on the search hit rate across Load conditions in Experiment 2. The effect of Load is reliable, t(62) = 10.76, p < .001 , Cohen’s d = 1.36, BF10= 7.79e+12, indicating that more targets were detected in the Low- (M = 97.1, SE = .01), relative to High- (M = 88.7, SE = .01), Load condition. A similar effect was observed in false alarms, t(62) = 2.82 p = .007, Cohen’s d = 0.36, BF10= 4.98, with lower false alarms in the Low- (M = .02, SE = .004) than in the High-Load condition (M = .04, SE = .01).

Appendix C. Target recognition memory

Experiments 1a and 1b

The proportion of correct recognition choices was examined in one-way repeated-measures ANOVAs, comparing recognition for targets to nontargets encountered during Low-, Medium-, and High-Load searches. A reliable effect was observed in Experiment 1a, F (3, 180) = 437.5, p < .001, ηp2 = .88, BF10= 1.475e+81, in which recognition differed across all items, with target objects recognized the best (M = .94, SE = .01). More importantly, nontargets objects encountered during High-Load search (M = .63, SE = .02) were recognized better than nontargets encountered during Medium- (M = .58, SE = .02) and Low-Load search (M = .48, SE = .01). Similarly, a reliable effect was observed in Experiment 1b, F (2.56, 161.24) = 660.88, p < .001, ηp2 = .91, BF10= 1.574e+101, with target objects recognized the best (M = .80, SE = .02), while nontargets encountered during High-Load search (M = .31, SE = .02) were recognized more often than nontargets encountered during Medium- (M = .26, SE = .01) and Low-Load search (M = .22, SE = .01), both ps < .001. Nontargets from Medium-Load search were recognized better than nontargets from Low-Load search, p = .035.

Appendix D. Target recognition errors

Experiments 1a and 1b

In Experiment 1a, each 4-AFC comparison included lures designated as relatively similar, moderately similar, and relatively dissimilar to the studied exemplar. Similarly, in Experiment 1b, lures were rank-ordered, from 1 (highest similarity) to 15 (lowest similarity). False alarms could thus be examined as a function of the perceptual overlap between studied exemplars and categorically related lures in both experiments. Separate one-way repeated-measures ANOVAs analyzed the impact of exemplar-lure similarity on the proportion of false alarms in Experiments 1a and 1b. Both experiments revealed reliable effects of lure similarity (Experiment 1a: F (1.5, 90.02) = 25.79, p < .001, ηp2 = .30, BF10 = 7.141e+9; Experiment 1b: F (9.92, 624.92) = 13.45, p < .001, ηp2 = .18, BF10 = 4.035e+27), indicating that when recognition of target objects failed, observers were more likely to commit false alarms to lures of high similarity with the target.

Appendix E

Table 5 Statistics from omnibus analyses on raw data values for proportion of false alarms as a function of search difficult and exemplar-lure similarity in Experiments 1a, 1b, and 2
Table 6 Statistics from simple effect analyses on raw data values for the effect of exemplar-lure similarity across levels of search difficult in Experiments 1a, 1b, and 2

Appendix F

Table 7 Transformed false-alarm data value means (± 1 SEM in parentheses) across conditions and exemplar-lure similarity in Experiments 1a, 1b, and 2

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Guevara Pinto, J.D., Papesh, M.H. & Hout, M.C. The detail is in the difficulty: Challenging search facilitates rich incidental object encoding. Mem Cogn 48, 1214–1233 (2020). https://doi.org/10.3758/s13421-020-01051-3

Download citation

Keywords

  • Incidental memory
  • Visual search
  • Search difficulty