Electrophysiological correlates of encoding processes in a full-report visual working memory paradigm

  • Kyle W. Killebrew
  • Gennadiy Gurariy
  • Candace E. Peacock
  • Marian E. Berryhill
  • Gideon P. Caplovitz
Article
  • 11 Downloads

Abstract

Why are some visual stimuli remembered, whereas others are forgotten? A limitation of recognition paradigms is that they measure aggregate behavioral performance and/or neural responses to all stimuli presented in a visual working memory (VWM) array. To address this limitation, we paired an electroencephalography (EEG) frequency-tagging technique with two full-report VWM paradigms. This permitted the tracking of individual stimuli as well as the aggregate response. We recorded high-density EEG (256 channel) while participants viewed four shape stimuli, each flickering at a different frequency. At retrieval, participants either recalled the location of all stimuli in any order (simultaneous full report) or were cued to report the item in a particular location over multiple screen displays (sequential full report). The individual frequency tag amplitudes evoked for correctly recalled items were significantly larger than the amplitudes of subsequently forgotten stimuli, regardless of retrieval task. An induced-power analysis examined the aggregate neural correlates of VWM encoding as a function of items correctly recalled. We found increased induced power across a large number of electrodes in the theta, alpha, and beta frequency bands when more items were successfully recalled. This effect was more robust for sequential full report, suggesting that retrieval demands can influence encoding processes. These data are consistent with a model in which encoding-related resources are directed to a subset of items, rather than a model in which resources are allocated evenly across the array. These data extend previous work using recognition paradigms and stress the importance of encoding in determining later VWM retrieval success.

Keywords

EEG Encephalography Attention 

Notes

Author note

This research was funded by grants awarded to GPC and MEB from the National Institutes of Health: R15EY022775, 1P20GM103650 and the National Science Foundation: NSF OIA 1632849 and NSF OIA 1632738.

References

  1. Adam, K. C. S., Mance, I., Fukuda, K., & Vogel, E. K. (2015). The contribution of attentional lapses to individual differences in visual working memory capacity. Journal of Cognitive Neuroscience, 27, 1601–1616. doi: https://doi.org/10.1162/jocn_a_00811 CrossRefPubMedPubMedCentralGoogle Scholar
  2. Appelbaum, L. G., Wade, A. R., Pettet, M. W., Vildavski, V. Y., & Norcia, A. M. (2006). Cue-invariant networks for figure background processing in human visual cortex. Journal of Neuroscience, 26, 11695–11708. doi: https://doi.org/10.1523/JNEUROSCI.2741-06.2006 CrossRefPubMedPubMedCentralGoogle Scholar
  3. Appelbaum, L., Wade, A., Pettet, M., Vildavski, V., & Norcia, A. (2008). Figure–ground interaction in the human visual cortex. Journal of Vision, 8(9), 8. doi: https://doi.org/10.1167/8.9.8 CrossRefPubMedPubMedCentralGoogle Scholar
  4. Bastiaansen, M., & Hagoort, P. (2003). Event-induced theta responses as a window on the dynamics of memory. Cortex, 39, 967–992. doi: https://doi.org/10.1016/s0010-9452(08)70873-6 CrossRefPubMedGoogle Scholar
  5. Berryhill, M. E., Chein, J. M., & Olson, I. R. (2011). At the intersection of attention and memory: The mechanistic role of the posterior parietal lobe in working memory. Neuropsychologia, 49, 1306–1315.CrossRefPubMedPubMedCentralGoogle Scholar
  6. Brainard, D. H. (1997). The Psychophysics Toolbox. Spatial Vision, 10, 433–436. doi: https://doi.org/10.1163/156856897X00357 CrossRefPubMedGoogle Scholar
  7. Cabeza, R., Kapur, S., Craik, F. I. M., McIntosh, A. R., Houle, S., & Tulving, E. (1997). Functional neuroanatomy of recall and recognition: A PET study of episodic memory. Journal of Cognitive Neuroscience, 9, 254–265.CrossRefPubMedGoogle Scholar
  8. Canolty, R. T., & Knight, R. T. (2010). The functional role of cross-frequency coupling. Trends in Cognitive Sciences, 14, 506–515. doi: https://doi.org/10.1016/j.tics.2010.09.001 CrossRefPubMedPubMedCentralGoogle Scholar
  9. Caplan, J. B., Madsen, J. R., Raghavachari, S., & Kahana, M. (2001). Distinct patterns of brain oscillations underlie two basic parameters of human maze learning. Journal of Neurophysiology, 86, 368–380.CrossRefPubMedGoogle Scholar
  10. Cohen, J. R., Sreenivasan, K. K., & D’Esposito, M. (2012). Correspondence between stimulus encoding- and maintenance-related neural processes underlies successful working memory. Cerebral Cortex, 24, 593–599. doi: https://doi.org/10.1093/cercor/bhs339 CrossRefPubMedPubMedCentralGoogle Scholar
  11. Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24, 87–114, disc. 114–185. doi: https://doi.org/10.1017/S0140525X01003922 CrossRefPubMedGoogle Scholar
  12. de Vries, I., van Driel, J., & Olivers, C. (2016). Posterior alpha EEG dynamics dissociate visual search template from accessory memory items. Journal of Vision, 16(12), 761. doi: https://doi.org/10.1167/16.12.761 CrossRefGoogle Scholar
  13. Deiber, M. P., Missonnier, P., Bertrand, O., Gold, G., Fazio-Costa, L., Ibañez, V., & Giannakopoulos, P. (2007). Distinction between perceptual and attentional processing in working memory tasks: A study of phase-locked and induced oscillatory brain dynamics. Journal of Cognitive Neuroscience, 19, 158–172. doi: https://doi.org/10.1162/jocn.2007.19.1.158 CrossRefPubMedGoogle Scholar
  14. Duncan, J., Schramm, M., Thompson, R., & Dumontheil, I. (2012). Task rules, working memory, and fluid intelligence. Psychonomic Bulletin & Review, 19, 864–870. doi: https://doi.org/10.3758/s13423-012-0225-y CrossRefGoogle Scholar
  15. Emrich, S. M., Riggall, A. C., LaRocque, J. J., & Postle, B. R. (2013). Distributed patterns of activity in sensory cortex reflect the precision of multiple items maintained in visual short-term memory. Journal of Neuroscience, 33, 6516–6523. doi: https://doi.org/10.1523/JNEUROSCI.5732-12.2013 CrossRefPubMedPubMedCentralGoogle Scholar
  16. Erickson, M. A., Albrecht, M. A., Robinson, B., Luck, S. J., & Gold, J. M. (2017). Impaired suppression of delay-period alpha and beta is associated with impaired working memory in schizophrenia. Biological Psychiatry, 2, 272–279. doi: https://doi.org/10.1016/j.bpsc.2016.09.003 PubMedGoogle Scholar
  17. Ester, E. F., Serences, J. T., & Awh, E. (2009). Spatially global representations in human primary visual cortex during working memory maintenance. Journal of Neuroscience, 29, 15258–15265. doi: https://doi.org/10.1523/JNEUROSCI.4388-09.2009 CrossRefPubMedPubMedCentralGoogle Scholar
  18. Fukuda, K., & Woodman, G. F. (2017). Visual working memory buffers information retrieved from visual long-term memory. Proceedings of the National Academy of Sciences, 114, 5306–5311. doi: https://doi.org/10.1073/pnas.1617874114 CrossRefGoogle Scholar
  19. Fukuda, K., Mance, I., & Vogel, E. K. (2015). Alpha power modulation and event-related slow wave provide dissociable correlates of visual working memory. Journal of Neuroscience, 35, 14009–14016. doi: https://doi.org/10.1523/JNEUROSCI.5003-14.2015 CrossRefPubMedPubMedCentralGoogle Scholar
  20. Fukuda, K., Kang, M. S., & Woodman, G. F. (2016). Distinct neural mechanisms for spatially lateralized and spatially global visual working memory representations. Journal of Neurophysiology, 116, 1715–1727. doi: https://doi.org/10.1152/jn.00991.2015 CrossRefPubMedPubMedCentralGoogle Scholar
  21. Gevins, A., Smith, M. E., McEvoy, L., & Yu, D. (1997). High-resolution EEG mapping of cortical activation related to working memory: Effects of task difficulty, type of processing, and practice. Cerebral Cortex, 7, 374–385. doi: https://doi.org/10.1093/cercor.7.4.374 CrossRefPubMedGoogle Scholar
  22. Gurariy, G., Killebrew, K. W., Berryhill, M. E., & Caplovitz, G. P. (2016). Induced and evoked human electrophysiological correlates of visual working memory set-size effects at encoding. PLoS ONE, 11, e0167022. doi: https://doi.org/10.1371/journal.pone.0167022 CrossRefPubMedPubMedCentralGoogle Scholar
  23. Harrison, S. A., & Tong, F. (2009). Decoding reveals the contents of visual working memory in early visual areas. Nature, 458, 632–635. doi: https://doi.org/10.1038/nature07832 CrossRefPubMedPubMedCentralGoogle Scholar
  24. Hillyard, S. A., & Anllo-Vento, L. (1998). Event-related brain potentials in the study of visual selective attention. Proceedings of the National Academy of Sciences, 95, 781–787. doi: https://doi.org/10.1073/pnas.95.3.781 CrossRefGoogle Scholar
  25. Hsieh, L. T., & Ranganath, C. (2014). Frontal midline theta oscillations during working memory maintenance and episodic encoding and retrieval. NeuroImage, 85, 721–729. doi: https://doi.org/10.1016/j.neuroimage.2013.08.003 CrossRefPubMedGoogle Scholar
  26. Jensen, O., & Tesche, C. D. (2002). Frontal theta activity in humans increases with memory load in a working memory task. European Journal of Neuroscience, 15, 1395–1399. doi: https://doi.org/10.1046/j.1460-9568.2002.01957.x CrossRefPubMedGoogle Scholar
  27. Jensen, O., Gelfand, J., Kounios, J., & Lisman, J. E. (2002). Oscillations in the alpha band (9–12 Hz) increase with memory load during retention in a short-term memory task. Cerebral Cortex, 12, 877–882. doi: https://doi.org/10.1093/cercor/12.8.877 CrossRefPubMedGoogle Scholar
  28. Kahana, M. J., Sekuler, R., Caplan, J. B., Kirschen, M., & Madsen, J. R. (1999) Human theta oscillations exhibit task dependence during virtual maze navigation. Nature, 399, 781–784. doi: https://doi.org/10.1038/21645 CrossRefPubMedGoogle Scholar
  29. Kahana, M. J., Seelig, D., & Madsen, J. R. (2001). Theta returns. Current Opinion in Neurobiology, 11, 739–744. doi: https://doi.org/10.1016/S0959-4388(01)00278-1 CrossRefPubMedGoogle Scholar
  30. Klein, K. A., Addis, K. M., & Kahana, M. J. (2005). A comparative analysis of serial and free recall. Memory & Cognition, 33, 833–839. doi: https://doi.org/10.3758/BF03193078 CrossRefGoogle Scholar
  31. Klimesch, W. (1999). EEG alpha and theta oscillations reflect cognitive and memory performance: A review and analysis. Brain Research Reviews, 29, 169–195. doi: https://doi.org/10.1016/S0165-0173 CrossRefPubMedGoogle Scholar
  32. Klimesch, W., Doppelmayr, M., Schimke, H., & Ripper, B. (1997). Theta synchronization and alpha desynchornization in a memory task. Psychophysiology, 34, 169–176.CrossRefPubMedGoogle Scholar
  33. Klimesch, W., Doppelmayr, M., Schwaiger, J., Auinger, P., & Winkler, T. (1999). “Paradoxical” alpha synchronization in a memory task. Cognitive Brain Research, 7, 493–501. doi: https://doi.org/10.1016/S0926-6410(98)00056-1 CrossRefPubMedGoogle Scholar
  34. Krause, C. M., Lang, A. H., Laine, M., Kuusisto, M., & Pörn, B. (1996). Event-related EEG desynchronization and synchronization during an auditory task. Electroencephalography and Clinical Neurophysiology, 98, 319–326.CrossRefPubMedGoogle Scholar
  35. Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390, 279–281. doi: https://doi.org/10.1038/36846 CrossRefPubMedGoogle Scholar
  36. Luck, S. J., & Vogel, E. K. (2013). Visual working memory capacity: From psychophysics and neurobiology to individual differences. Trends in Cognitive Sciences, 17, 391–400. doi: https://doi.org/10.1016/j.tics.2013.06.006 CrossRefPubMedPubMedCentralGoogle Scholar
  37. Mance, I., Adam, K. C. S., Fukuda, K., & Vogel, E. K. (2014). The contribution of attentional lapses to estimates of individual differences in working memory capacity. Journal of Vision, 14(10), 40. doi: https://doi.org/10.1167/14.10.40 CrossRefGoogle Scholar
  38. Morgan, S. T., Hansen, J. C., & Hillyard, S. A. (1996). Selective attention to stimulus location modulates the steady-state visual evoked potential. Proceedings of the National Academy of Sciences, 93, 4770–4774. doi: https://doi.org/10.1073/pnas.93.10.4770 CrossRefGoogle Scholar
  39. Müller, M. M., Teder-Sälejärvi, W., & Hillyard, S. A. (1998). The time course of cortical facilitation during cued shifts of spatial attention. Nature Neuroscience, 1, 631–634. doi: https://doi.org/10.1038/2865 CrossRefPubMedGoogle Scholar
  40. Muthu, K., Suzuki, S., Joon Kim, Y., Grabowecky, M., & Paller, K. A. (2007). Attention induces synchronization-based response gain in steady-state visual evoked potentials. Nature Neuroscience, 10, 117–125. doi: https://doi.org/10.1038/nn1821 CrossRefPubMedGoogle Scholar
  41. Norcia, A. M., Appelbaum, L. G., Ales, J. M., Cottereau, B. R., & Rossion, B. (2015). The steady-state visual evoked potential in vision research: A review. Journal of Vision, 15(6), 4. doi: https://doi.org/10.1167/15.6.4 CrossRefPubMedPubMedCentralGoogle Scholar
  42. Pelli, D. G. (1997). The VideoToolbox software for visual psychophysics: Transforming numbers into movies. Spatial Vision, 10, 437–442. doi: https://doi.org/10.1163/156856897X00366 CrossRefPubMedGoogle Scholar
  43. Peterson, D. J., Gurariy, G., Dimotsantos, G. G., Arciniega, H., Berryhill, M. E., & Caplovitz, G. P. (2014). The steady-state visual evoked potential reveals neural correlates of the items encoded into visual working memory. Neuropsychologia, 63, 145–153. doi: https://doi.org/10.1016/j.neuropsychologia.2014.08.020 CrossRefPubMedPubMedCentralGoogle Scholar
  44. Postle, B. R. (2006). Working memory as an emergent property of the mind and brain. Neuroscience, 139, 23–28. doi: https://doi.org/10.1016/j.neuroscience.2005.06.005 CrossRefPubMedGoogle Scholar
  45. Rouder, J. N., Morey, R. D., Cowan, N., Zwilling, C. E., Morey, C. C., & Pratte, M. S. (2008). An assessment of fixed-capacity models of visual working memory. Proceedings of the National Academy of Sciences, 105, 5975–5979. doi: https://doi.org/10.1073/pnas.0711295105 CrossRefGoogle Scholar
  46. Stokes, M. G. (2015). “Activity-silent” working memory in prefrontal cortex: A dynamic coding framework. Trends in Cognitive Science, 19, 394–405. doi: https://doi.org/10.1016/j.tics.2015.05.004 CrossRefGoogle Scholar
  47. Tadel, F., Baillet, S., Mosher, J. C., Pantazis, D., & Leahy, R. M. (2011). Brainstorm: A user-friendly application for MEG/EEG analysis. Computational Intelligence and Neuroscience, 2011, 879716:8–9. doi: https://doi.org/10.1155/2011/879716 Google Scholar
  48. Todd, J. J., & Marois, R. (2004). Capacity limit of visual short-term memory in human posterior parietal cortex. Nature, 428, 751–754. doi: https://doi.org/10.1038/nature02466 CrossRefPubMedGoogle Scholar
  49. Vogel, E. K., & Machizawa, M. G. (2004). Neural activity predicts individual differences in visual working memory capacity. Nature, 428, 748–751. doi: https://doi.org/10.1038/nature02447 CrossRefPubMedGoogle Scholar
  50. Ward, L. M. (2003). Synchronous neural oscillations and cognitive processes. Trends in Cognitive Sciences, 7, 553–559. doi: https://doi.org/10.1016/j.tics.2003.10.012 CrossRefPubMedGoogle Scholar
  51. Zhang, P., Jamison, K., Engel, S., He, B., & He, S. (2011). Binocular rivalry requires visual attention. Neuron, 71, 362–369. doi: https://doi.org/10.1016/j.neuron.2011.05.035 CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Psychonomic Society, Inc. 2018

Authors and Affiliations

  • Kyle W. Killebrew
    • 1
  • Gennadiy Gurariy
    • 1
  • Candace E. Peacock
    • 2
  • Marian E. Berryhill
    • 1
  • Gideon P. Caplovitz
    • 1
  1. 1.Department of Psychology, Program in Cognitive and Brain SciencesUniversity of NevadaRenoUSA
  2. 2.Department of PsychologyUniversity of CaliforniaDavisUSA

Personalised recommendations