Cognitive-behavioral and electrophysiological evidence of the affective consequences of ignoring stimulus representations in working memory
Ignoring visual stimuli in the external environment leads to decreased liking of those items, a phenomenon attributed to the affective consequences of attentional inhibition. Here we investigated the generality of this “distractor devaluation” phenomenon by asking whether ignoring stimuli represented internally within visual working memory has the same affective consequences. In two experiments we presented participants with two or three visual stimuli and then, after the stimuli were no longer visible, provided an attentional cue indicating which item in memory was the target they would have to later recall, and which were task-irrelevant distractors. Participants subsequently judged how much they liked these stimuli. Previously-ignored distractors were consistently rated less favorably than targets, replicating prior findings of distractor devaluation. To gain converging evidence, in Experiment 2, we also examined the electrophysiological processes associated with devaluation by measuring individual differences in attention (N2pc) and working memory (CDA) event-related potentials following the attention cue. Larger amplitude of an N2pc-like component was associated with greater devaluation, suggesting that individuals displaying more effective selection of memory targets—an act aided by distractor inhibition—displayed greater levels of distractor devaluation. Individuals showing a larger post-cue CDA amplitude (but not pre-cue CDA amplitude) also showed greater distractor devaluation, supporting prior evidence that visual working-memory resources have a functional role in effecting devaluation. Together, these findings demonstrate that ignoring working-memory representations has affective consequences, and adds to the growing evidence that the contribution of selective-attention mechanisms to a wide range of human thoughts and behaviors leads to devaluation.
KeywordsWorking memory Emotion Attention ERP
When selectively ignoring distracting stimuli in our surrounding environment, these distractors are subsequently affectively devalued compared to the targets of our attention and previously-unseen (novel) stimuli (see Fenske & Raymond, 2006 for review). This distractor devaluation effect seems to be a widespread phenomenon in tasks involving attentional inhibition, and perhaps may even be an intrinsic consequence of inhibition. For example, the effect has been replicated in many different types of search tasks (Fenske, Raymond, & Kunar, 2004; Raymond, Fenske, & Westoby, 2005) using many different types of external sensory stimuli (Ferrey, Frischen, & Fenske, 2012; Goolsby, Shapiro, & Raymond, 2009a; Veling, Holland, & van Knippenberg, 2007). Moreover, we (De Vito & Fenske, 2017) and others (Vivas, Marful, Panagiotidou, & Bajo, 2016) have recently demonstrated that devaluation does not only occur after ignoring stimuli in the environment, but is also a consequence of stopping the retrieval of representations in long-term memory. Thus, devaluation may be a consequence of inhibition through both internal and external attentional mechanisms (Chun, Golomb, & Turk-Browne, 2011). In the present study, we further test the pervasiveness of distractor devaluation by assessing the affective consequences of ignoring another type of internal memory representation; namely, representations in visual working memory.
The brain has a limited capacity for processing stimuli, so the completion of goal-directed behaviors requires that we focus our processing resources on task-relevant items (see Marois & Ivanoff, 2005, for review). By reducing the ability of task-irrelevant stimuli to interfere with the processing of the current focus of our thoughts and actions, inhibition aids in the completion of these goal-directed behaviors and tasks (see Houghton & Tipper, 1994; Munakata et al., 2011 for reviews). Inhibition acts by stopping a mental or motor process and results in the reduced activation of neural signals that are associated with task-irrelevant items (see Bari & Robbins, 2013; Munakata et al., 2011, for reviews). Along with resulting in reduced levels of activation, inhibitory processes also have affective consequences (see Fenske & Raymond, 2006, for review).
Raymond et al. (2003) were the first to demonstrate the devaluation of ignored stimuli after pairing a visual search task with an affective evaluation task. Participants located a target stimulus in the presence of one distractor and then subsequently evaluated the cheerfulness or dreariness of the target, distractor, or a novel stimulus. Distractors were significantly devalued compared to targets and novel stimuli, suggesting a direct link between attentional processing and emotional consequences. Distractor devaluation has now been replicated using various behavioral search tasks including those with multiple distractors (Fenske et al., 2004; Raymond et al., 2005; Veling et al., 2007).
Kiss et al. (2007) extended the investigation of distractor devaluation by adding concurrent electroencephalography (EEG) to the search-then-evaluate method used by Raymond et al. (2003). Kiss et al. were particularly interested in measuring the N2pc component, an event-related potential (ERP) associated with the focusing of visual spatial attention (Luck & Hillyard, 1994; Woodman & Luck, 1999): A larger, earlier N2pc component is associated with more strongly focused attention. They found that the N2pc component emerged earlier on trials containing distractors that were later rated more negatively than on trials containing distractors that were later rated more positively. This finding suggests that focusing attention more strongly and more efficiently on a target results in more successful inhibition of distractors, which thereby triggers stronger affective devaluation of these distractors. Their results converge with the extensive behavioral evidence (see Fenske & Raymond, 2006; Gollwitzer, Martiny-Huenger, & Oettingen, 2014; Raymond, 2009, for reviews) that suggests that attentional inhibition results in the devaluation of distracting stimuli in our surrounding environment.
To investigate whether devaluation also occurs following the stopping of retrieval of internal long-term memory representations, we (De Vito & Fenske, 2017) used a somewhat different approach involving the Think/No-think paradigm. In this paradigm, participants first learn word pairs or word-image pairs. They are then shown a cue word from each pair—one at a time—and are asked to recall (i.e., think) or inhibit (i.e., no-think) the memory of the associated response word or image. Despite the differences between the Think/No-think paradigm and the visual search paradigms used in previous studies, and despite having subjects ignore internal memory representations rather than external stimuli, our findings were strikingly similar to those of past distractor devaluation studies: We found that the ignored no-think items were disliked compared to baseline items. Vivas et al. (2016) also observed a similar long-term memory effect using a directed-forgetting task to elicit inhibition of stimulus memories. The similarity of these findings in long-term memory tasks to past distractor devaluation effects in visual search raises a question: Is affective devaluation a fundamental property of inhibition, occurring regardless of what we ignore and how?
To further assess the proposal that devaluation is an intrinsic aspect of inhibition, in the present study we tested whether ignoring items represented in visual working memory (Cowan, 2001; Luck & Vogel, 1997; Phillips, 1974) causes those items to be subsequently devalued. On the one hand, working memory would seem to be a good candidate system for finding devaluation, as inhibition is thought to play a critical role in preventing interference from irrelevant information stored in working memory (Bao, Li, Chen, & Zhang, 2006; Blair, Vadaka, Schuchat, & Li, 2011; Hasher, Lustig, & Zacks, 2007). On the other hand, there is reason to believe that inhibition in working memory will not lead to devaluation. Specifically, Goolsby et al. (2009a) previously demonstrated that loading visual working memory interferes with distractor devaluation. Based on these findings, Goolsby et al. concluded that visual working memory plays a functional role in distractor devaluation by facilitating the association of ignored stimuli with an inhibition-altered affective response. Thus, it remains unclear whether visual working memory can simultaneously play this functional role while also maintaining the representations of any to-be-attended and to-be-ignored items.
Across two experiments, we tested the prediction that ignoring items represented in visual working memory will cause those items to be subsequently devalued. For Experiment 1 we modified Raymond et al.'s (2003) search-then-evaluate method so that participants did not know which stimuli to attend, and which to ignore, until after the search items had been removed from the display. Consequently, inhibition could only be applied to the internal memory representations of those stimuli. In Experiment 2, we sought converging evidence by adapting this task for concurrent recording of EEG, and testing for relationships between individual differences in the magnitude of devaluation and both attention and working memory-related ERP components: the N2pc and contralateral delay activity (CDA) components, respectively. To preview the results, across both experiments we observed consistent evidence that ignoring items in visual working memory results in distractor devaluation.
In Experiment 1, our primary innovation was to manipulate whether a cue informing participants about the type of image to localize was presented prior to the appearance of the visual array (pre-cue) or only after the array had disappeared (post-cue). Whereas the pre-cue encouraged a visual search of the external display for the target, the post-cue constrained the target search to occur solely within the contents of working memory. Because Raymond et al. (2003) and others (e.g., Griffiths & Mitchell, 2008; Kiss et al., 2007; Martiny-Huenger et al., 2014; Veling et al., 2007) have repeatedly shown that the changes in subsequent stimulus ratings in such studies are due to increases in negative affect for prior distractors rather than positive affect for prior targets, we opted to omit the novel baseline condition often used to establish the effect as being about distractor devaluation. Simplifying our experimental design in this way allowed us to incorporate both the pre-cue and post-cue versions of the localization task in a fully within-subjects design. We expected previously ignored distractors to receive more affectively negative ratings than prior targets, regardless of whether the items were maintained solely in working memory or were part of the external visual display when attended or ignored.
All materials and procedures were approved by the Research Ethics Board at the University of Guelph. Seventy-nine participants (mean age 20.2 years, SD = 3.4, 32 females, 69 right-handed) were recruited from the University of Guelph undergraduate participant pool in exchange for course credit or $10. One participant performed at chance on both search tasks (M = 49%), so their data were excluded from the analyses reported below, resulting in a final sample of 78 participants. This sample size was above the 45 participants that G*Power (Faul, Erdfelder, Lang, & Buchner, 2007) indicated was necessary to detect the standard effect size found in previous investigations of devaluation following inhibition (d z = .5; Raymond et al., 2003, 2005; Veling et al., 2007) with a power of 0.95 at an alpha of .05. Participants were unique from those tested in Experiment 2 and were naïve to the aims of the study. Every participant had normal or corrected-to-normal vision and provided informed consent.
Apparatus and stimuli
The experimental methods were based on the paradigm devised by Raymond et al. (2003) combining a two-item search task with an affective-evaluation task. Participants performed one block of 100 pre-cue visual-search trials that encouraged a search of the external visual display for the to-be-localized target, and one block of 100 post-cue working-memory search trials that constrained the target search to occur solely within the contents of working memory (Fig. 1). A short break occurred at the mid-point of each block and between blocks to reduce fatigue. The order of the blocks was counterbalanced across participants. Each participant completed a set of ten practice trials at the beginning of each block of trials, and for the first five practice trials stimulus durations were doubled to make it easier for participants to become familiar with the procedure. The entire experimental session lasted approximately 30 min.
Each of the pre-cue visual search trials began with a 500-ms fixation cross. The pre-cue, which was either the word “Circle” or “Square,” was then presented for 500 ms at the display center to indicate which type of stimulus the participant should search for and localize in the upcoming stimulus array. After the pre-cue disappeared a pair of stimuli then appeared for 150 ms, one circle-type image and one square-type image, each centered 5.06° to the left or right of a central fixation cross (relative location of each stimulus type counterbalanced across trials). This was followed by the presentation of the fixation cross alone for 50 ms and then a response probe—the word “Circle” or “Square”—prompting the participant to report the location of the target item by pressing the letter “k” (left) or “l” (right). The pre-cue and response probe were always the same word (i.e., the pre-cue was 100% valid). The response probe remained on the screen until the participant made a response. The response for each target-localization trial was immediately followed by an affective-evaluation trial that began with a 50-ms fixation cross followed by a 1,000-ms presentation of the rating scale along with the words “Get Ready.” The screen was then cleared and the to-be-rated item—either the previous target or previous distractor—was then presented for 350 ms at the display center. The to-be-rated item then disappeared and a question mark (?) appeared in the center of the screen. The question mark remained until participants had reported their subjective emotional response to the image in terms of how much they liked it. Participants did this by pressing the numeric key on a standard keyboard that corresponded to their rating using a 4-point scale ranging from “1 – Dislike very much” to “4 – Like very much.” The post-cue working memory trials and affective evaluations were identical to the pre-cue visual search trials with the exception that the pre-cue was replaced by the words “Get Ready”, and thus participants did not know which stimulus was their target and which to ignore, until the stimuli were no longer visible and solely held within memory.
Results and discussion
Target-localization accuracy was very high for both the pre-cue visual search trials (M = 95.3%, SD = .06) and post-cue working-memory search trials (M = 94.8%, SD = .05). Because errors are also known to have negative affective consequences for associated items (e.g., Chetverikov, Jóhannesson, & Kristjánsson, 2015; Chetverikov, 2014), only ratings of images associated with a correct target-localization response were included in our analyses. This eliminated 4.7% of the ratings from the pre-cue visual search trials and 5.2% of the ratings from the post-cue working-memory search trials.
The degree to which affective evaluations of distractors were more negative than of targets in our pre-cue external visual search condition closely resembles that originally reported by Raymond et al. (2003) and others in subsequent replications and extensions (Goolsby et al., 2009a; Griffiths & Mitchell, 2008; Kiss et al., 2007; Raymond et al., 2005). While the replication of important findings is a worthwhile endeavour per se, the innovative aspect of Experiment 1 was our extension of Raymond et al.’s basic paradigm to the domain of working memory; specifically, by requiring participants to first maintain two abstract visual patterns and their relative locations in working memory before receiving a cue indicating which of the two was the to-be-localized target. Correct performance in this post-cue working-memory search task depends on the extent to which participants can successfully ignore what has suddenly become a distractor within the contents of working memory in order to select the target. The results from the memory-search condition of the present study support the conclusion that this process of ignoring a distractor in visual working memory leads to the subsequent devaluation of that item.
To further investigate the devaluation of working memory distractors, in Experiment 2 we again paired a working memory task with an affective evaluation task; however, here participants also underwent concurrent EEG. In Experiment 2 we used a similar method to that of our post-cue working memory trials from Experiment 1. While our post-cue working memory trials in Experiment 1 included a memory array followed by a text cue to indicate the identity of the target stimulus, in Experiment 2 we replaced the text cue with a spatial post-cue. The purpose of each type of cue, presented in their respective Experiment following the memory array, was to alert participants to the identity of the target stimulus for that particular trial. We employed EEG to make use of two ERP components in particular: the N2pc associated with visual-spatial attention, and the contralateral delay activity (CDA) associated with visual working memory activity. Both components are lateralized negativities, and can be measured at posterior electrode sites over the left or right hemisphere when participants attend to (N2pc; Luck & Hillyard, 1994; Woodman & Luck, 1999) or memorize (CDA; Vogel & Machizawa, 2004) visual information in the contralateral visual field. The latency and magnitude of each component provides information about the timing and extent to which selective attention and working memory contribute to the task, respectively.
Our motivation for measuring these ERP components was two-fold. First, the CDA component provides a useful measure for clarifying which memory system participants employ on memory-search trials. Given that search stimuli in Experiment 1 were visual and changed on each trial, it is likely that participants employed visual working memory in this experiment. However, it is also possible to accomplish Experiment 1 memory-search trials without visual working memory. For example, participants could have used verbal working memory to encode stimuli (“Circle” vs. “Square”) and their locations (“left” vs. “right”). Moreover, because these verbal labels changed minimally from trial to trial, such a strategy could also eventually be supported by long-term memory (Carlisle, Arita, Pardo, & Woodman, 2011). Accordingly, our first goal for Experiment 2 was to assess if the devaluation effect found in Experiment 1 can be replicated in a task where verbal labels are less effective, and where memory delays are thereby associated with the CDA.
Second, measurement of ERP components also allows us to examine the neurophysiological effects associated with post-cues (i.e., the cue that triggers inhibition in memory), and how these effects relate to subsequent devaluation. If participants are using post-cues to prioritize one item in working memory, and to ignore any other working memory representations, we can expect to see both cue-related N2pc and CDA components, and individual differences in both should correlate with individual differences in subsequent affective devaluation. The N2pc occurs following both shifts of attention amongst external stimuli and amongst internal visual working memory representations (Kuo, Rao, Lepsien, & Nobre, 2009). Thus, just as Kiss et al. (2007) observed a correlation between the N2pc and devaluation in their external attention task, we can expect individual differences in the N2pc in our working-memory task to correlate with differences in affective devaluation (i.e., individuals showing a larger amplitude and/or earlier latency N2pc, reflecting more effective attentional selection of the target and inhibition of working-memory distractors, should subsequently show greater distractor devaluation).
For the CDA component, there are two competing predictions about the potential relation between this index of working-memory and distractor devaluation. On one hand, to the extent that inhibition of distractors in working memory causes those distractors to be suppressed or removed from memory (Duarte et al., 2013; Kuo, Stokes, & Nobre, 2012; but see Tsubomi, Fukuda, Kikumoto, & Vogel, 2015), successful inhibition may reduce CDA amplitude (i.e., reflecting the reduced involvement of working memory in the task) leading to a negative correlation with devaluation. On the other hand, the CDA has been established as a reliable index of individual differences in working memory capacity (Vogel & Machizawa, 2004), and there is evidence that the availability of visual working memory resources plays a critical role in supporting distractor devaluation (Goolsby et al., 2009a). Thus, to the extent that individuals showing a larger CDA tend to have greater levels of available working-memory resources, they may also show greater levels of distractor devaluation than individuals with fewer available resources. Being able to directly test these competing predictions—increased devaluation for individuals more effective at inhibition-related reductions in working-memory activity versus increased devaluation for individuals with more available working memory resources—underscores the potential value of our electrophysiological measures in Experiment 2.
All of the materials and procedures used in this experiment were approved by the Research Ethics Board at the University of Guelph. Twenty participants (mean age 18.7 years, SD = 1.7, 15 females, 19 right-handed) were recruited for this experiment through flyers and through advertisements on social media websites (i.e., Facebook), and they received $20 as compensation for participating. Two participants performed at chance on the working memory task, both scoring 54% accuracy (all other participants scored at least 65%), and their data were excluded from the analyses reported below. Two other participants were excluded from the analysis below because they had high rates of artifact rejection (i.e., 54.7% and 46.6%), while no other participant’s artifact rejection rate exceeded 34.4%. The inclusion of these two participants in the analyses did not alter the observed pattern of statistically significant effects reported below. Their removal resulted in a final sample size of 16 participants. The sample size of 16 participants used here was above the 13 participants indicated by G*Power (Faul et al., 2007) to be required to detect a similar effect size as in previous investigations of the N2pc and CDA components (d z = 1.0; Duarte et al., 2013; Kiss et al., 2007) with a power of 0.95 at an alpha of .05. This sample was similar in size to that used by Kiss et al. (2007; n = 16), who used EEG to investigate the underlying mechanism guiding the devaluation of external sensory stimuli. All participants provided informed consent and had normal or corrected-to-normal vision.
Apparatus and stimuli
Stimuli for this experiment were generated using an E-Prime script (Psychology Software Tools, Pittsburgh, PA, USA). The program created Mondrian stimuli (1.45° × 1.45°) by randomly generating 200 overlapping squares within a square border. The width of these squares was between 10% and 30% the width of the full stimulus. These stimuli were therefore very similar to the square Mondrians used in Experiment 1; however, in Experiment 2, for each stimulus, all squares were drawn with the same color randomly selected from Blue, Green, Orange, Pink, Purple, Red, and Yellow. Later Adobe Photoshop was used to re-color the squares within each Mondrian to be varying degrees of the initial Mondrian color. 500 stimuli were created for each color, resulting in 3,500 total stimuli. The stimuli were presented on a grey background. All stimulus presentation and behavioral response collection for this experiment were controlled using PsychoPy software (Peirce, 2007) running on a personal computer with a 50.8 cm LCD monitor (resolution: 1,920 × 1,080 pixels). Participants sat approximately 71 cm from the monitor.
In each trial of this experiment, a working memory task was followed by an affective evaluation task. The experiment consisted of 32 blocks of 12 trials each, resulting in 384 trials (sample trial depicted in Fig. 3). Participants were given a break to rest following each block of trials.
Participants were instructed to maintain fixation throughout the experiment. Each trial began with the working memory task. The working memory task began by presenting an eight-armed fixation cross (see Fig. 3) measuring 1.78° × 1.61° in the center of the screen, and 500 ms following the onset of the fixation cross an arrow measuring 1.29° × 0.65° was added to the display above fixation. This arrow is unrelated to distractor devaluation and, rather, is necessary for measuring the CDA. The arrow appeared for 150 ms and directed participants to the side of the upcoming memory array that they would be required to memorize (i.e., memorize all items on the left or right side of the display). The arrow then disappeared, and the fixation point was presented alone for 150–250 ms (randomly jittered) until the memory array was added to the display. The memory array was presented for 150 ms and consisted of six differently colored randomly-selected stimuli. Three stimuli in the memory array were presented to the left of fixation and three were presented to the right of fixation. There were eight possible locations at which a stimulus could appear in the memory array (i.e., four on the left side of fixation and four to the right). The eight locations were organized in a circular orientation equidistant (6.05°) from fixation. Following the disappearance of the memory array, the fixation cross was presented alone for 900 ms, constituting the first retention interval. Following the first retention interval, the post-cue was presented for 200 ms, consisting of the doubling in width of the arm of the fixation cross that pointed to the location of a previously-memorized stimulus. This now-cued stimulus (i.e., that which was pointed to by the post-cue) was classified as the target for that particular trial. On 50% of trials, no post-cue was presented (Uncued trials are explained in full detail below). These uncued trials provided a baseline against which any ERP changes elicited by the post-cue (e.g., in N2pc or CDA components) on the 50% of cued trials could be measured. After the post-cue disappeared, the second retention interval consisted of the fixation cross presented alone for 800 ms. The fixation cross then disappeared and a memory probe was presented in the center of the screen above the text “Same or Different” for 1,000 ms. This memory probe was either the previously-cued stimulus (50% of trials) or one of the previously-memorized but uncued stimuli (50% of trials). Participants were required to indicate using the up or down key on the keyboard (counterbalanced across participants) whether the presented probe exactly matched the cued stimulus. On 50% of trials no memory probe was presented (Unprobed trials explained in full detail below). There is extensive literature showing that affective evaluations can change as a result of repeated stimulus exposure (i.e., the mere exposure effect; e.g., Zajonc, 2001), suggesting that evaluations in our task could change as a result of presenting the target or distractor stimulus again as a memory probe. Using unprobed trials created a condition where stimulus exposures for targets and distractors were always equal as they occurred solely during the presentation of the memory array. A 1,000-ms blank screen followed the memory probe during which participants were given more time to make their “same or different” response.
Uncued Trials (see Fig. 4)
Unprobed Trials (see Fig. 4)
To facilitate the interpretation of participants’ affective evaluations (described next), 50% of both cued and uncued trials were also unprobed (i.e., no memory probe was presented). The rationale for including unprobed trials is that merely exposing participants to stimuli can affect their evaluations of those stimuli (Zajonc, 2001), potentially confounding the affective consequences of post-cues. Accordingly, to assess the impact of post-cues on affective evaluations, we only examined participants’ evaluations on cued-unprobed trials. In particular, our critical behavioral comparison was between target and distractor items on cued-unprobed trials, where participants were asked to focus their attention in working memory on a single item, but the number of exposures between the target and distractors was equal because no memory probe was presented. For unprobed trials, the memory probe was replaced by the presentation of the words “no probe” (measuring 2.10° × 0.65°) on the screen for 1,000 ms. Participants did not make any keyboard response on these trials while maintaining central fixation.
Every single working memory trial was followed immediately by an affective evaluation trial. This includes trials of all four types as depicted in Fig. 4. Affective evaluations began with the presentation of the fixation point for 500 ms. The to-be-evaluated stimulus was then presented in the center of the screen for 1,000 ms while the text “1 – 2 – 3 – 4” (measuring 5.24° × 0.65°) was displayed below it. On cued trials, the presented stimulus was either the cued stimulus (i.e., the attended target) or one of the previously-memorized stimuli that had not been cued (i.e., the inhibited distractors). On uncued trials, the evaluation stimulus was one of the previously-memorized stimuli. Participants were asked to affectively evaluate the cheerfulness of the presented stimulus on a 4-point scale using the numbered keys on the keyboard (1-Not at all Cheerful, 4-Very Cheerful). A 1,500-ms blank screen followed the presentation of the to-be evaluated stimulus. Participants were able to make their evaluation response at any time while the stimulus was presented or during the 1,500-ms blank screen that followed. A 1,000-ms inter-trial interval followed each affective evaluation.
EEG recording and analysis
All EEG data analyses were conducted using EEGLab Toolbox (Delorme & Makeig, 2004) and ERPLab Toolbox (Lopez-Calderon & Luck, 2014). A BioSemi ActiveTwo system (Ag/AgCl electrodes) with 64 scalp electrodes in standard 10–20 placement was used to record the EEG. Electrodes were also placed at each mastoid, at the outer canthus of each eye, as well as above and below the right eye. The data sets were re-referenced according to the average of the mastoids and resampled to 250 Hz. Averaged ERP waveforms were calculated using epochs of 2,500 ms that were time-locked to the presentation of the memory array, and baselined to the 150-ms time-interval preceding the appearance of the memory array.
The horizontal electrooculogram (HEOG) and the vertical electrooculogram (VEOG) were low-pass (30-Hz) filtered and were both used to remove any eye-movement or blink artifacts from the data sets. The VEOG represents the difference in voltage between the electrodes placed above and below the right eye. The HEOG represents the difference in voltage between the electrodes that were placed on the participants’ canthi. We identified artifacts using the moving window peak-to-peak threshold and step-like function rejection algorithms within ERPLAB, and removed any trials from analysis in which the VEOG exceeded -80 or 80 μV or where HEOG exceeded 32 μV, respectively, along with visual inspection.
All analyzed ERP components (i.e., the CDA and N2pc components) were calculated by creating an averaged waveform of the voltage differences across five pairs of electrodes: P5/P6, P7/P8, PO3/PO4, PO7/PO8 and O1/O2. These match electrodes used previously in investigations involving the CDA component (e.g., Emrich, Al-Aidroos, Pratt, & Ferber, 2009). The voltage ipsilateral to the memorized stimuli was subtracted from the voltage contralateral to the memorized stimuli, and this voltage difference was then used to measure the size of each respective component.
Results and discussion
A 2 (Hemisphere: Contralateral vs. Ipsilateral) × 2 (Cue Status: Cued vs. Uncued) repeated-measures ANOVA conducted on voltages during the pre-cue time-window (i.e., 400–900 ms following the presentation of the memory array) revealed a statistically significant main effect of hemisphere, F(1,15) = 34.894, p < .001, η2partial = .699; neither the main effect of cue status nor the two-way interaction reached statistical significance, both F-values < 1. The main finding here is that the presentation of the memory array was associated with a significant CDA, suggesting that participants were indeed using visual working memory to encode task items. It is not surprising that this CDA was similar across cued and uncued trials, given that prior to the post cue, the cued and uncued trial sequences were identical.
A 2 (Hemisphere: Contralateral vs. Ipsilateral) × 2 (Cue Status: Cued vs. Uncued) repeated-measures ANOVA conducted on voltages during the post-cue time-window revealed a marginally non-significant interaction, F(1,15) = 4.232, p = .058, η2partial = .220, as well as statistically significant main effects of hemisphere, F(1,15) = 32.762, p < .001, η2partial = .686, and cue status, F(1,15) = 24.288, p < .001, η2partial = .618. As is evident in Fig. 7, following the presentation of the post-cue, a CDA was present on both cued and uncued trials, and the marginal interaction reflects the trend for a larger (more negative) CDA on cued trials.
N2pc component in working memory
While the N2pc component has been frequently studied in external attention studies, it has only been measured in a small number of working memory studies. Kuo et al. (2009) observed an N2pc in memory using a time window of 240–300 ms following a post-cue. We used the same time-window in the present study; i.e., 240–300 ms following the post-cue, or 1,290–1,350 ms following the presentation of the original memory array. Consistent with the presence of an N2pc on cued trials, the mean amplitude of voltage during this time range contralateral to the memorized stimuli was significantly more negative than the mean amplitude ipsilateral, t(15) = 3.71, p = .002. This negativity did not differ across cued and uncued trials, t(15) = 0.07, p = .943, likely reflecting the persistence of the CDA on uncued trials. Looking at cued trials in Fig. 7, the usual peak of activity associated with the N2pc component is not visible during Kuo et al.’s time range, and instead appears at an earlier time point (i.e., the large negative voltage at about 150 ms following the post cue). Indeed, the mean amplitude of the contralateral-ipsilateral difference waveform during this earlier time range—1,170-1,230 ms following the original memory array—was also statistically significantly different from zero, t(15) = -5.17, p < .001. Because it is unclear which of these time ranges provides the most appropriate measure of the N2pc, or indeed whether either provides an appropriate measure, we elected to examine both time ranges, and characterize these components as N2pc-like activity.
N2pc and devaluation
In the present study we asked whether ignoring items represented in visual working memory causes those items to be affectively devalued. In Experiment 1, we presented an attentional post-cue during the retention interval of a working-memory task that indicated to participants which memory item was their target, and which was a distractor they could ignore. When later asked to judge how much they liked each item, participants consistently rated the ignored distractors more negatively than prior target items. Comparison of these stimulus ratings with those following a standard visual-search condition revealed that the magnitude of devaluation observed after ignoring working-memory distractors was similar to that after ignoring distractors present in the external visual environment. Thus, ignoring stimuli represented in visual working memory leads to affective devaluation of those items.
In Experiment 2, we replicated and extended this working-memory distractor devaluation effect by measuring event-related potentials to specifically examine how differences in the working-memory CDA and selective-attention N2pc components may be linked to the devaluation of working-memory distractors. As in Experiment 1, participants consistently rated previously-ignored working-memory distractors more negatively than prior targets. Our observation of a strong CDA component throughout the memory-retention interval confirmed the involvement of visual working memory resources throughout the target-selection task. Moreover, individual differences in the magnitude of the CDA amplitude following the post-cue positively correlated with the magnitude of working-memory distractor devaluation. An N2pc-like deflection was also observed time-locked to the post-cue, with individual differences in the amplitude of this attentional-selection component likewise correlated with the subsequent magnitude of working-memory distractor devaluation. Thus, individual differences in the capacity to select target information while resisting distraction from task-irrelevant information within visual working memory, and in the availability/allocation of visual working-memory resources, were both predictive of the extent to which the ignored stimulus-representations became affectively devalued. Together, these behavioral and electrophysiological findings support the conclusions that the affective devaluation of stimuli whose representations are ignored in visual working memory is linked to both working memory and selective attention, and that there is variability in individuals’ capacity to display such cognition-emotion interactions.
Many prior studies of distractor devaluation have included a novel baseline condition in which ratings were obtained for previously-unseen items with no history of being either attended or ignored in prior trials (De Vito, Al-Aidroos, & Fenske, 2017; Duff & Faber, 2011; Goolsby et al., 2009a, b; Griffiths & Mitchell, 2008; Kihara, Yagi, Takeda, & Kawahara, 2011; Martiny-Huenger et al., 2014; Raymond et al., 2003; Veling et al., 2007). The typical finding from these studies is that distractors are rated more negatively than novel-baseline items, while ratings of targets do not differ from those of novel-baseline items (i.e., distractor devaluation, not target valuation). The finding of devaluation compared to novels is not exclusive to external sensory stimuli, as our investigation using a Think/No-think task showed that suppressed no-think items held solely in long-term memory are devalued compared to baseline items. Our observation in the current investigation that participants liked working-memory distractors less than working-memory targets is consistent with this collective group of prior findings using external sensory stimuli and representations held in long-term memory. However, the lack of a novel-baseline condition in our studies means that we can not unequivocally conclude that the differences we observed reflect increases in negative affect for stimuli whose representations were ignored in working memory, rather than increases in positive affect for stimuli whose representations were attended in working memory.
One of the questions we asked in Experiment 2 was whether individual differences in the N2pc component elicited by post-cues would correlate with devaluation. While we did observe a brief negative-going potential at posterior electrodes contralateral to the cued memory target that resembled an N2pc (i.e., the peak in Fig. 7 referred to as the Early N2pc), the latency of this component was shorter than expected. Indeed, the peak latency of this component (about 150 ms) falls just outside of the N2pc time range typically examined for both shifts in attention within external space (~180–280 ms; Jolicœur, Brisson, & Robitaille, 2008) and shifts in attention within working memory (240–300 ms, Kuo et al., 2009). Despite the unexpected latency, there is reason to believe this negative potential reflects N2pc-like activity. This potential is a difference between contralateral and ipsilateral electrodes. Such a lateralized difference could be driven by sensory events that differ across the left and right visual hemifields; however, the only consistent lateralized sensory difference in our task was the post-cue stimulus, and this stimulus was designed to produce minimal sensory stimulation and to be similar to cues used in previous working memory tasks where no such component was observed (e.g., Kuo et al., 2012). If this lateralized potential is not driven by sensory events, then it can only be a consequence of spatially-specific cognitive operations, such as spatial attention and working memory. Given that modulations of working memory related activity (i.e., the CDA) tend to emerge even later than the N2pc (e.g., Jolicœur et al., 2008), it is unclear what else could be driving this early potential other than a spatial-attentional mechanism. To date, there has been relatively little investigation of the N2pc component following attentional shifts within working memory. Thus, further study will likely be needed within this area to definitely interpret our observed potentials. That said, our observations that individual differences in N2pc-like activity —both within the typical N2pc time range and within the time range in our study showing the most prominent deflection—predict the subsequent magnitude of working-memory distractor devaluation nicely converges with the link between N2pc magnitude and distractor devaluation observed by Kiss et al. (2007) in their external attention study.
Another open question we had at the outset of this study was whether our post-cues would be associated with an increase or decrease in CDA amplitude. Given that two previous studies have shown a decrease in CDA amplitude following a spatial post-cue (Duarte et al., 2013; Kuo et al., 2012)—ostensibly reflecting the removal of uncued items from memory—one might have expected our results to mirror this effect. However, recent unpublished findings by Tsubomi et al. (2015) are inconsistent with these past results. They found a decrease in CDA amplitude when participants were instructed to remove all items from working memory, but not when participants were cued to selectively remove only some. In our study, rather than decreasing or remaining constant, the CDA component exhibited a marginally non-significant increase in amplitude following the post cue. Moreover, individuals showing larger CDA amplitudes at this point also subsequently showed greater amounts of working-memory distractor devaluation. While it is possible that this relationship is due to target-liking as a result of a strengthened target representation, this pattern seems to be best explained by the findings of Goolsby et al. (2009a). In their study, the stimuli and basic procedure closely resembled that of both Raymond et al. (2003) and our Experiment 1 pre-cue visual search trials, with the exception that they manipulated the availability of visual working memory resources by requiring participants to maintain different amounts of unrelated information in memory throughout the visual-search task. They found that the relative devaluation of prior distractors, which was otherwise robust, was eliminated when visual search was performed under conditions of high working-memory load. Based on these findings, Goolsby et al. concluded that visual working memory is a key stage of processing within which inhibition may impact how stimulus value is coded and stored, and their findings have since been replicated in a sample of individuals with schizophrenia (Strauss et al., 2012). Because the CDA has been established as a reliable index of individual differences in working memory capacity (Vogel & Machizawa, 2004), our observation that individuals with greater post-cue CDA amplitude show greater working-memory distractor devaluation provides converging evidence that the availability of visual working memory resources does indeed play a critical role in supporting distractor devaluation (Goolsby et al., 2009a). Whereas Goolsby et al. showed that working memory resources must be available for the distractor devaluation effect to take place, here, using an ERP-marker of working memory processing—the CDA component—we have shown at an individual differences level that working memory processing clearly corresponds to the level of distractor devaluation. Therefore, these findings support those of Goolsby et al. by showing that working memory resources play a critical role in the distractor devaluation effect. More broadly, while past research has demonstrated a link between trial-by-trial fluctuations in inhibition and subsequent distractor devaluation (e.g., Kiss et al., 2007), our findings linking the post-cue N2pc and CDA to devaluation are the first to suggest that individual differences in attention- and memory-related processes are linked to individual differences in the magnitude of distractor devaluation.
There has been some disagreement regarding whether distractor devaluation is indeed caused by inhibition. Dittrich and Klauer (2012) used Eder and Rothermund's (2008) evaluative coding principle to develop an alternative to the devaluation-by-inhibition explanation, whereby distracting stimuli are rated more negatively than targets because of evaluative codes associated with each stimulus type through experimental instructions. They propose that simply instructing participants to select, or attend to, certain items (i.e., targets) and to avoid, or ignore, other items (i.e., distractors) is enough to alter stimulus ratings as a result of many prior associations that have linked approach behaviors to positive items and avoidance behaviors to negative items (Chen & Bargh, 1999). While such an account may be feasible for the results of Experiment 1, it is impossible for it to explain the clear links we observed in Experiment 2 between specific attention- and working-memory neurocognitive operations and subsequent ratings. Indeed, the association we observed between individual differences in the CDA and N2pc components and in the devaluation of working-memory distractors speaks more clearly in support of a critical role in how efficiently targets can be attended within visual working memory— an act aided by inhibition of working-memory distractors— and the availability of working-memory resources in determining subsequent affective responses to prior distractors. Moreover, we recently conducted a direct test of opposing predictions arising from the evaluative-coding and devaluation-by-inhibition hypotheses using an electrophysiological marker of attentional inhibition—the Pd component— in a task that requires participants to avoid interference from abstract-shape distractors presented while maintaining a uniquely-colored stimulus in memory (De Vito et al., 2017). Importantly, the results of this investigation confirmed all of the devaluation-by-inhibition predictions, including the finding that fluctuations in distractor inhibition, as indicated by the magnitude of the Pd component, are closely associated with subsequent affective evaluations of the prior distractors. In contrast, none of the evaluative-coding-based predictions were confirmed (see also Gollwitzer et al., 2014; Martiny-Huenger et al., 2014). Thus, while there are still questions to be addressed about how inhibition might lead to stimulus devaluation, the bulk of the evidence to date is consistent the hypothesis that attentional suppression of visual distractors leads to their affective devaluation.
The findings that we report in this manuscript extend an ongoing discussion that is centered on the notion that inhibitory processes can be separated into active and passive inhibition (see Baddeley, 1998; Hofmann, Schmeichel, & Baddeley, 2012; Munakata et al., 2011 for reviews). Active inhibition refers to the effortful inhibition that is applied through the use of top-down cognitive control such as the inhibition used in directed forgetting or response inhibition (e.g., Go/No-go) tasks. Passive inhibition, on the other hand, refers to situations where active target representations are amplified, and as a result alternative competing representations are inhibited through diffuse lateral inhibitory connectivity. In a recent investigation we demonstrated that using active inhibition processes to prevent the recall of long-term memory representations (i.e., during a Think/No-think task) results in their affective devaluation (De Vito & Fenske, 2017). This result aligns with findings by Vivas et al. (2016) who also showed devaluation following active inhibition that was applied during a directed forgetting paradigm. Collectively, these results suggest that active inhibitory processes result in affective devaluation. In previous investigations we have discussed the possibility that active inhibition may be the only type of inhibitory processes that lead to devaluation—a suggestion that was based on a finding of no devaluation following a retrieval-induced forgetting paradigm, a task thought to possibly involve passive inhibition (Janczyk & Wühr, 2012). However, the collective results reported in this manuscript add to this discussion. Our two experiments showing devaluation of distracting representations held in working memory seem to suggest that devaluation can also result when passive inhibition is used to resolve interference from competing representations and therefore facilitate focusing on a task-relevant target stimulus. Therefore, the results reported here not only suggest that the relationship between inhibition and devaluation extends to the domain of working memory, but also provide evidence in support of the hypothesis that various types of inhibitory processes can lead to devaluation.
Our main finding, that ignored items in visual working memory are liked less than working-memory targets is consistent with the view that inhibition-related stimulus devaluation is a widespread phenomenon. Stimulus devaluation has been found following many types of tasks involving cognitive inhibition to avoid distraction, including preview-search, two-item localization tasks, and large-array visual search (see Fenske & Raymond, 2006; Raymond, 2009 for reviews). The devaluation of inhibited items has also been observed following tasks involving response inhibition (Kiss, Raymond, Westoby, Nobre, & Eimer, 2008), as well as the suppression of representations in longer-term memory (De Vito & Fenske, 2017; Vivas et al., 2016). Might it be the case that inhibition always leads to devaluation? There are two studies that suggest the answer to this question is not completely clearcut. Janczyk and Wühr (2012), for example, paired a retrieval-induced forgetting paradigm with stimulus evaluations and did not show affective consequences following this task; however, it has been questioned if retrieval-induced forgetting indeed employs inhibitory processes (Murayama, Miyatsu, Buchli, & Storm, 2014). As well, the fact that Goolsby et al. (2009a) and Strauss et al. (2012) found that distractor devaluation is eliminated when working memory is loaded with stimuli that are irrelevant to the current attention task suggests that certain factors must be in place in order for inhibition to lead to devaluation (i.e., the availability of working memory resources). Our findings add to the growing evidence that distractor devaluation is a broad phenomenon that generalizes across many tasks, and the inhibition of many types of representations.
This research was supported by the Natural Science and Engineering Research Council of Canada, and the Canadian Foundation for Innovation. We are grateful for Katherine McArthur’s data collection efforts for Experiment 1. Our thanks also to James Intriligator who provided the E-prime script used to create the stimuli in Experiment 2.
- Bao, M., Li, Z. H., Chen, X. C., & Zhang, D. R. (2006). Backward inhibition in a task of switching attention within verbal working memory. Brain Research Bulletin, 69(2), 214–221. https://doi.org/10.1016/j.brainresbull.2005.12.007 CrossRefPubMedGoogle Scholar
- Carlisle, N. B., Arita, J. T., Pardo, D., & Woodman, G. F. (2011). Attentional Templates in Visual Working Memory. Journal of Neuroscience, 31(25), 9315–9322. https://doi.org/10.1523/JNEUROSCI.1097-11.2011.Attentional CrossRefPubMedPubMedCentralGoogle Scholar
- Chun, M. M., Golomb, J. D., & Turk-Browne, N. B. (2011). A taxonomy of external and internal attention. Annual Review of Psychology, 62, 73–101. https://doi.org/10.1146/annurev.psych.093008.100427 CrossRefPubMedGoogle Scholar
- De Vito, D., & Fenske, M. J. (2017). Suppressing memories of words and familiar objects results in their affective devaluation: Evidence from Think/No-think tasks. Cognition, COGNIT-D-1, 1–37. https://doi.org/10.1016/j.cognition.2017.01.020
- De Vito, D., Al-Aidroos, N., & Fenske, M. J. (2017). Neural evidence that inhibition is linked to the affective devaluation of distractors that match the contents of working memory. Neuropsychologia, 99(October 2016), 259–269. https://doi.org/10.1016/j.neuropsychologia.2017.03.022 CrossRefPubMedGoogle Scholar
- Duarte, A., Hearons, P., Jiang, Y., Delvin, M. C., Newsome, R. N., & Verhaeghen, P. (2013). Retrospective attention enhances visual working memory in the young but not the old: An ERP study. Psychophysiology, 50(5), 465–476. https://doi.org/10.1111/psyp.12034 CrossRefPubMedPubMedCentralGoogle Scholar
- Hasher, L., Lustig, C., & Zacks, R. T. (2007). On the control of attention. Canadian Journal of Experimental Psychology, 63(3), 240–252.Google Scholar
- Houghton, G., & Tipper, S. P. (1994). A model of inhibitory mechanisms in selective attention. In D. Dagenbach & T. H. Carr (Eds.), Inhibitory Processes in Attention, Memory, and Language. San Diego, CA: Academic Press, Inc.Google Scholar
- Kiss, M., Goolsby, B. A, Raymond, J. E., Shapiro, K. L., Silvert, L., Nobre, A. C., … Eimer, M. (2007). Efficient attentional selection predicts distractor devaluation : ERP evidence for a direct link between attention and emotion. Journal of Cognitive Neuroscience, 19(8), 1316–1322. https://doi.org/10.1162/jocn.2007.19.8.1316.Efficient CrossRefPubMedPubMedCentralGoogle Scholar
- Luck, S. J., & Hillyard, S. A. (1994). Electrophysiological correlates of feature analysis during visual search. Psychophysiology https://doi.org/10.1111/j.1469-8986.1994.tb02218.x
- Martiny-Huenger, T., Gollwitzer, P., & Oettingen, G. (2014). Distractor Devaluation in a Flanker Task: Object-Specific Effects Without Distractor Recognition Memory. Journal of Experimental Psychology: Human Perception and Performance, 40(2), 613–625. Retrieved from http://psycnet.apa.org.subzero.lib.uoguelph.ca/journals/xhp/40/2/613.pdf
- Raymond, J. E., Fenske, M. J., & Tavassoli, N. T. (2003). Selective attention determines emotional responses to novel visual stimuli. Psychological Science, 14(6), 537–42. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/14629683
- Raymond, J. E., Fenske, M. J., & Westoby, N. (2005). Emotional devaluation of distracting patterns and faces: A consequence of attentional inhibition during visual search? Journal of Experimental Psychology. Human Perception and Performance, 31(6), 1404–1415. https://doi.org/10.1037/0096-15220.127.116.114 CrossRefPubMedGoogle Scholar
- Strauss, G. P., Lee, B. G., Waltz, J. A., Robinson, B. M., Brown, J. K., & Gold, J. M. (2012). Cognition-emotion interactions are modulated by working memory capacity in individuals with schizophrenia. Schizophrenia Research, 141(2–3), 257–261. https://doi.org/10.1016/j.schres.2012.08.010 CrossRefPubMedPubMedCentralGoogle Scholar
- Tsubomi, H., Fukuda, K., Kikumoto, A., & Vogel, E. K. (2015). Forgetting no-longer necessary items from visual working memory. In Poster presented at the annual meeting of the Psychonomic Society. Chicago, Illinois.Google Scholar