Keywords

1 Introduction

According to Mackworth, “Vigilance is defined as a state of readiness to detect and respond to small changes occurring at random time intervals in the environment” [1]. In other words, vigilance or sustained attention is an act of careful observation of critical or rare events whose negligence may lead to catastrophe [2]. In today’s world, where emphasis is laid on reducing risks and errors, and mitigating the chances of accidents, it seems rational to assess the operator vigilance in real time to avoid human errors. Air traffic control, drowsiness detection in drivers, inspection and quality control, automated navigation, military and border surveillance, life-guarding, cyber operations, space exploration, etc., [3], are some major domains where operators are involved in monotonous tasks for prolonged intervals of time and remaining vigilant is an utmost requirement. However, in [4, 5], it has been pointed that sleep deprivation, work overload, stress, time pressure, drowsiness and prolonged working hours are the major factors that lead to low vigilance, thereby, human errors.

Till date several concerted efforts have been made in the literature to propose and design new techniques of vigilance detection with the help of features like, heart rate variability [6], galvanic skin response [6], pupil diameter, eye blink frequency [3] and brain activity measurement [7,8,9] (namely, EEG (Electroencephalogram), MEG (Magnetoencephalogram), fNIRS (functional near infrared spectroscopy), ECoG (electrocorticogram), fMRI (functional magnetic resonance imaging), etc.). Although, the techniques mentioned above are good contender for vigilance detection, yet, they have several serious limitations associated with them. For instance, eye related features show strong inter-personal and intra-personal variability, EEG suffers from poor spatial resolution, MEG requires special operating environment for its functioning, ECoG involves implantation of electrodes in an invasive manner, fMRI is associated with high equipment overhead and fNIRS suffers from low spatial resolution.

Amongst the above-mentioned methods, designed for vigilance detection, EEG is the most commonly studied physiological measure despite of its poor spatial resolution. The prime reasons behind its tremendous popularity amongst researchers are: (1) its high time resolution, (2) its non-invasive nature and simplicity of operation and (3) relatively cheap cost compared to other devices. Furthermore, as vigilance deteriorates with time it seems plausible to study the brain signals in time bound fashion to assess the vigilance status in real-time. In this regard, the Event Related Potentials (ERPs) present in the EEG signals have successfully been utilized to study the changes occurring in the human brain with passing time [10]. For instance, ERP features namely P100-N200 have been utilized for studying emotional information processing in [11]; frontal midline theta and N200 ERP have been shown to reflect complementary information about expectancy and outcome evaluation in [12]; in [13] authors utilized N200 ERP for word recognition; in [14], N100, P200, N200 and P300 ERP components have been used to study the impact of depression on attention. Further, ERPs have also been used for understanding reaction times in response to pictures of people depicting pain [15]; in [16] ERPs have been utilized to understand the state of brain in schizophrenia patients; in [17] authors demonstrated the association of mMMN, P200 and P500 ERP components with artificial grammar learning in the primate brain; in [18], N400 and the P200 components have been utilized in the investigation of semantic and phonological processing in skilled and less-skilled comprehenders; besides, ERPs have also found utility in studying multisensory integration (MSI) ability of the brain in school-aged children [19].

From the above literature, we observe that P200 and N200 ERPs (see Fig. 1) have been instrumental in studying cognitive behaviour of humans and is prospective for real-time assessment of vigilance. Here, concisely P200 ERP refers to a positive spike in EEG signals which is generally observed within 150 to 250 ms after the exhibition of a target stimulus (auditory or visual event) [20], while N200 is a negative potential usually evoked between 180 to 325 ms after the presentation of a specific visual or auditory stimulus following a string of standard (non-target) stimuli [21, 22]. In general, P200 latency is a measure of stimulus classification speed and its amplitude represents the amount of attentional resources devoted to the task along with the required degree of information processing, whereas N200 ERP, which is usually evoked only during conscious stimulus attention before the motor response, is helpful in stimulus identification and distinction, thereby suggesting its link to the cognitive processes.

Fig. 1.
figure 1

P200 and N200 components in ERP signal of EEG data

Full size image

In this work we propose (a) to use N200 and P200 ERPs for studying vigilance, (b) observe the correlation of N200 and P200 ERPs with behavioural accuracy obtained, (c) observe the variation in the amplitude of both N200 and P200 ERPs under the presence of target and non-target stimuli, (d) observe the variation in the active areas of the brain before, during and after the experiment and check whether the hotspots are present in the areas from which P200 and N200 evoke.

2 Proposed Methodology

In the following, we present our proposed research methodology and steps for extracting ERPs (P200 and N200) from the EEG signals.

2.1 Experimental Setup

Subjects: Ten healthy, right handed participants with normal or corrected-to-normal vision, aged between 26 to 33 years volunteered for the experiment (see Table 1). To carefully monitor the vigilance of each volunteer, a proper schedule was maintained. It was ensured that the participants: (a) were not sleep deprived, (b) were under no medication and (c) had no history of mental illness. We also took written consent from each participant, which was approved by the institution’s ethical committee, before conducting the experiment. Further, we asked each volunteer to do not consume tea or coffee 3 to 4 h prior to the experiment. Keeping in mind the usual circadian cycle of activeness of each participant, the experiment was conducted in the morning, that is between 7 am and 10 am.

Table 1. Participant details
Full size table

Vigilance Task: To study the variation of vigilance over a long period of time, we utilized the computerized version of the Mackworth Clock Test as the experimentation tool, wherein the small circular pointer moves in a circle like the seconds’ hand of an analog clock. It changes its position approximately after one second. However, at infrequent and irregular intervals, the pointer can make a double jump. Here, the task of each participant is to detect and respond to the double jump of the pointer, indicating the presence of the target event, by pressing the space bar key of the keyboard.

Fig. 2.
figure 2

The overall experimental procedure

Full size image

2.2 Protocol

The participants were comfortably seated in a quiet and isolated room (devoid of any Wi-Fi connections), wherein a constant room temperature was also maintained. Before conducting the actual experiment, each participant was given proper demonstrations and instructions about the experiment and were asked to relax for ten minutes. Further, a practice session of five minutes was also arranged for each participant to make them accustomed to the task. We utilized a large 20 in. monitor kept at a distance of 65 cm from the user for presenting the visual stimuli to the participant. The beginning of the experiment was marked by an EEG recording of an idle session of five minutes followed by the clock test of 20 min. There were a total of 1200 trials in the experiment. After completion of the clock test, we again recorded the EEG signals for an idle session of five minutes. Besides, to keep track of a participant’s responses and to ensure true marking of the target events, we also recorded the hardware interrupt from the keyboard. The entire experimental procedure has been pictorially shown in Fig. 2.

2.3 Data Acquisition

The experiment was designed to be completed in 30 min. Further, all EEG data recordings were carried out with the help of portable, user friendly and cost effective Emotiv Epoc+ device which follows the well-known 10–20 international system. This device comprises of 14 electrodes positioned at AF3, F7, F3, FC5, T7, P7, O1, O2, P8, T8, FC6, F4, F8 and AF4 locations and has a sampling rate of 128 Hz. We collected 12000 trials for our experiment with the help of ten voluntary participants.

2.4 Detection of ERPs

  1. 1.

    Pre-processing: Usually while recording EEG data, due to various external environmental disturbances, the data gets contaminated with various kinds of artifacts. The extraction of desired/useful features from the EEG signal becomes very difficult under the presence of artifacts. Hence, to minimize the effect of artifacts, it is mandatory to pre-process the recorded raw EEG signals. For this purpose, filters in the standard frequency range of (0.1–30 Hz) are used. Thus, filters help in extracting the desired brain activity by rejecting the other undesired brain signals within a frequency range of (<0.1 Hz and >30 Hz). In the present work, we have used the Chebyshev’s high pass filter (having cut off frequency of 0.1 Hz) to remove all disturbing components emerging from breathing and voltage changes in neuronal and non-neuronal artifacts. Besides, we used Chebyshev’s low pass filter (having cut off frequency of 30 Hz) to eliminate the noise arising from muscle movements. Further, to ensure perfect rejection of the strong 50 Hz power supply interference, impedance fluctuation, cable defects, electrical noise, and unbalanced impedances of the electrodes, we utilized (at the recording time) a notch filter with null frequency of 50 Hz.

  2. 2.

    Feature Extraction: It is known from the literature that P200 and N200 ERPs are dominant over parietal, occipital and frontal regions, respectively, of the brain. Thus to locate these features, we have used the AF3, AF4, F3, F4, P7, P8, O1 and O2 electrodes. Now, for extracting the features from the EEG signals, first, the pre-processed EEG data is marked to identify the type of event (that is, correctly identified event, falsely identified event and missed event). Next, baseline removal process is carried over this marked data, followed by epoch averaging (500 ms pre-stimulus and 1000 ms post-stimulus) to generate the ERP waveforms. Furthermore, to verify the presence of P200 and N200 ERPs we performed ensemble averaging of the target event epochs and plotted the average waveform.

3 Results and Discussions

The recorded EEG data, of 20 min, has been divided into 10 equal observation periods of two minutes each to carefully observe the pattern of vigilance changes. Next, we observed the amplitude and latency variation of P200 and N200 component of ERPs for those instances where the user responded correctly to an occurrence of the target event. Further, we compared the accuracy attained by each individual, while focusing on the pointer of the clock test and trying to correctly detect the target, with the amplitude of P200 and N200 ERPs to establish a correlation amongst them. The variation of amplitude and latency of P200 and N200 has been reported in Tables 2 and 3, respectively. The amplitude ranges for P200 and N200 ERP are heuristically defined as follows:

$$\begin{aligned} P200 \ (amplitude)= {\left\{ \begin{array}{ll} very \ low,&{} \text {for }value \geqslant 0.1 \ \upmu \mathrm{V} \ and<1 \ \upmu \mathrm{V} \\ low,&{} \text {for }value \geqslant 1 \ \upmu \mathrm{V} \ and<3 \ \upmu \mathrm{V}\\ moderate,&{} \text {for }value \geqslant 3 \ \upmu \mathrm{V} \ and <7 \ \upmu \mathrm{V} \\ high,&{} \text {for }value \geqslant 7 \ \upmu \mathrm{V} \end{array}\right. } \end{aligned}$$
(1)
$$\begin{aligned} N200 \ (amplitude)= {\left\{ \begin{array}{ll} very \ low,&{} \text {for }value \geqslant -0.01 \ \upmu \mathrm{V} \ and<-1 \ \upmu \mathrm{V} \\ low,&{} \text {for }value \geqslant -1 \ \upmu \mathrm{V} \ and<-3 \ \upmu \mathrm{V}\\ moderate,&{} \text {for }value \geqslant -3 \ \upmu \mathrm{V} \ and <-6 \ \upmu \mathrm{V} \\ high,&{} \text {for }value \geqslant -6 \ \upmu \mathrm{V} \end{array}\right. } \end{aligned}$$
(2)
Table 2. Variation of amplitude and latency of P200 ERP
Full size table
Table 3. Variation of amplitude and latency of N200 ERP
Full size table

To evaluate the performance of the participants in terms of accuracy of detection, we sub-divided the recorded EEG data into four categories defined as true alarm (TA), true skip (TS), false alarm (FA) and false skip (FS). In terms of Mackworth Clock test experiment, true alarm represents correct identification of target stimuli, true skip represents correct identification of non-target stimuli, false alarm represents incorrect key pressed at non-targets and false skip represents non-identification of the target stimuli. Based on these data, the accuracy is calculated by using Eq. 3. The accuracy of detection of each individual who participated in the experiment has been tabulated in Table 4. The latencies of P200 and N200 ERPs were observed to be within the already known ranges; however, no particular trend with respect to amplitude has been observed for the obtained latencies.

$$\begin{aligned} Accuracy=\frac{TA+TS}{TA+TS+FA+FS} \end{aligned}$$
(3)

The accuracy (in %) obtained is divided into four classes which is defined as follows:

$$\begin{aligned} Accuracy \ (\%)= {\left\{ \begin{array}{ll} very \ low,&{} \text {for }value \,{\geqslant }\, 0 \% \ and \,{<}\, 30 \%\\ low,&{} \text {for }value \,{\geqslant }\, 30 \% \ and \,{<}\, 50 \%\\ moderate,&{} \text {for }value \,{\geqslant }\, 50 \% \ and \,{<}\, 80 \%\\ high,&{} \text {for }value \,{\geqslant }\, 80 \% \ and \,{\leqslant }\, 100 \% \end{array}\right. } \end{aligned}$$
(4)
Table 4. Variation of accuracy (in %) of each participant
Full size table

From Tables 2, 3 and 4, and using Eqs. 1, 2 and 4, we can see that for participant P2, during an observation interval between (4–6) min the amplitude of P200 is high (12.67 \(\upmu \)V) while the amplitude of N200 is low (\(-3.04\,\upmu \)V), thereby resulting in 93.48% accuracy. In case of participant P6, during an observation interval between (16–18) min the amplitude of both P200 and N200 is low and is 2.31 \(\upmu \)V and \(-1.25\,\upmu \)V, respectively, thereby resulting in 71.41% accuracy. In case of participant P1, during an observation interval between (6–8) min the amplitude of P200 is low (1.71 \(\upmu \)V) while the amplitude of N200 is high (\(-7.19\,\upmu \)V), thereby resulting in 81.48% accuracy. Similarly, in case of participant P1, during observation interval between (0–2) min the amplitude of both P200 and N200 is high and is 10.48 \(\upmu \)V and \(-6.109\,\upmu \)V, respectively, thereby resulting in 100% accuracy. Other values may be verified from the tables in the similar manner to conclude that both accuracy and ERPs (P200 and N200) are correlated to each other, such that, whenever the accuracy of detection is high there is a high amplitude of P200 and N200. In other words, we can say that whenever an individual successfully distinguishes the target stimuli amongst all other presented stimuli, the two ERPs, viz., N200 and P200 are elicited with high amplitude. Besides, we also show the variation of accuracy and amplitude of P200 and N200 with time for participant P1 in Fig. 3. It can be easily observed from Fig. 3 that accuracy of target detection and amplitude of both ERPs are correlated.

Fig. 3.
figure 3

Plot showing variation of accuracy and amplitude of P200 and N200 with time

Full size image
Fig. 4.
figure 4

P200 and N200 peaks during true alarm and false alarm conditions

Full size image

To study the variations in the ERPs under the presence of true alarm (when the user correctly identifies the target events) and false alarm (when the user incorrectly identifies the events), we plotted the P200 and N200 ERPs on a single graph with common origin (see Fig. 4). We observed that there is a considerable difference in the amplitude of both ERPs under true and false alarm conditions.

Fig. 5.
figure 5

Variation of ERPs in target and non-target conditions

Full size image

Further, Fig. 5 shows the variation in P200 and N200 ERPs under the presence of target and non-target stimuli.

Fig. 6.
figure 6

Variation of scalp plot before, during and after experiment (Color figure online)

Full size image

Figure 6 depicts the variation in the active regions of the brain before, during and after the completion of the experiment. Here, blue spots visible in pre and post experiment scalp images show low brain activity, while the red spots visible during experimentation show an increase in the brain activity of the associated regions. Besides, from instance 1 we observed that during the experiment - parietal, frontal and some parts of occipital region were highly energized and these regions showed the presence of P200 and N200 ERPs. Further, from instance 2 we observed that during the experiment frontal region was highly energized and the region showed the presence of N200 ERP. Through this, we verified that our experiment successfully evokes the two ERPs from the designated regions. Hence, we can apply the selected ERPs for vigilance detection.

4 Conclusion

In the literature different features of EEG signals have been utilized to study vigilance level of human beings. In this work, First, we successfully demonstrated that both P200 and N200 ERPs are suitable candidates for studying vigilance. Second, we observed the variation in P200 and N200 amplitude with true alarm and false alarm. Third, we observed the variation in P200 and N200 amplitude under the presence of target and non-target stimuli. Fourth, with the help of scalp plot of Fig. 6 we verified the hot-spots/active regions of the brain from where the studied ERPs originate.

This work may be applied in real-time analysis of vigilance. Besides, in future we plan to extend this work to quantize the level of vigilance instead of indicating its mere presence and absence.