Keywords

1 Introduction

Human-computer interaction is the interaction between people and machine, in essence refers to the interaction between people and computer, or can be understood as people and machine that contains a computer. The ultimate aim of human-computer interaction research is to explore how to design a computer that can help people accomplish a mission more safely and more efficiently.

With the advent of the era of big data, advanced interactive technology shows the characteristics of “human-centered” and “natural interaction”, and researches on human nature perception channels came into being such as voice, touch, somatosensory, augmented reality, eye control and brain control, in which brain control technology is one of the most cutting-edge interactive technology.

Brain control is based on brain-machine interface technology, speculate the brain’s thinking activities by extracting EEG signal generated by human or animal brain, and translated into the corresponding command to control the external computer or other equipment. At present, brain control has infiltrated into military weapons and equipment, virtual reality, household equipment, games and medical fields.

The application of brain-computer interaction in digital interface gives the user an unprecedented experience. Researchers at home and abroad have carried out relevant research in the field of brain-computer interaction, mainly using brain waves to control computers and electromechanical devices. In 1991, Wolpaw et al. researched and published result that control the movement of the cursor by changing the amplitude of mu rhythm in the EEG signal, proposed the concept of brain drive control technology [1]. In 1999, Nature first published an article about the brain-computer interface which used slow potential to achieve spelling [2]. In 2003 Nature reported experiments made by Duke University which used monkey brain to control robot [3]. Professor Miller from Northwestern University achieved functional electrical stimulation control paralyzed muscles [4]. Pittsburgh University achieved human brain ECoG signal control robot [5]. Cheng M using the steady-state visual evoked potential experiment paradigm BCI system realized a four-degree-of-freedom artificial limb pouring process based on SSVEP control [6]. In 2010, Graz University of Technology achieved brain-computer interface paradigm which used event-related synchronization potential duration as the classification [7].

Research has found that the EEG signal will show a corresponding and regular pattern of change when people are in thinking activities or in the outside world induced by a stimulus [8]. At present, brain electrical signals used in brain-computer interface are: Slow cortical potential (SCP), P300, Motor imagery (MI) and Steady state visual evoked potential (SSVEP) [9]. In 2012, Zhang et al. proposed a way to improve the accuracy of brain-computer interaction recognition by using inverted face to induce vertex positive potential (VPP) and N170, combined with P300, experimental results showed that, compared to highlight icon and object stimulation, inverted face stimuli can induce stronger N170, VPP and P300, resulting in more discriminatory features [10].

The application of digital interface elements in brain-computer interaction, mainly using the experimental tasks to complete the visual movement, collecting tested brain waves at the same time, then match the EEG signal library in the computer microprocessors according to the processed EEG components, the command can be activated when the signal matches the data base, achieving the interaction without operation between digital interface elements EEG signal and computer.

In the field of digital interface BCI, Zhu et al. [11] proposed a smart typing method using multi-mode EEG control, which realized the remote control process of computer typing without body-typed motion. Guan et al. [12] proposed a virtual Chinese-English keyboard design using brain waves, which realized the direct input of Chinese and English information using brain waves; Liu et al. [13] proposed a extraction method of motor imagery EEG feature which can be used to control the cursor.

In the field of brain-computer interaction of ERP components, Wu et al. [14] proposed a novel BCI Chinese input virtual keyboard system based on the P300 potential; Hong et al. [15] using N2 potential, proposed a brain-computer interaction method which used visual movement-related neural signals as the carrier. Li et al. [16] developed an experimental system for measuring brain stress by using EEG.

As stated above, in the field of digital interface brain-computer interaction, scholars mostly do text input research that based on brainwave, or realize the control of cursor through the movement imagination, practicality and application range are both very limited. In the brain-computer interaction field of ERP components, P300 and N200 potentials have already been used in brain-computer interaction. There are so many ERPs related to cognition but still lack of digital interface brain-computer interaction researches that using digital interface element cognition.

2 The Implement Method of Icon Brain-Computer Interaction

Icon is the most important element and component of the digital interface, and it will be one of the most important input ways of digital interface brain-computer interaction in the future. Integrated predecessors’ research, this paper uses icon elements as cognitive objects, presents digital interface brain-computer interactive method based on icon control from the perspective of ERP application and then explores the interface elements on digital interface BCI control. The method in the paper mainly includes following steps: icon collecting and processing; acquiring user’s computer function control commands of the icon pictures; time and frequency feature extraction of target icon image’s EEG signals. Similarity calculation and activate control instruction of target icon pictures.

2.1 Icon Collection and Processing

Collect i function icon pictures from a variety of digital interface, which \( {\text{i}} = 1, 2\ldots 10 \), common function icon pictures are collected from the digital interface by using screenshot tool, function icon picture refer to close icon picture, save icon picture, undo icon picture, forward icon picture, enlarge icon picture, zoom icon picture, select icon picture, cut icon picture, maximize icon picture, minimize icon picture and other common icons. To avoid the interference of the difference of perspective and clarity, icons need to be processed, using Photoshop, Illustrator or Coreldraw etc. image processing softwares to process icons, get the png format icon picture, both area and pixels are 48px × 48px, then put the icon picture into the center of an area of 1024px × 768px white background image, generate the bmp format icon picture. As shown in Fig. 1, use the “magnifying glass” as an example, collect “magnifying glass” function icon picture from the Visio software, picture preview software, Photoshop CC and AutoCAD software etc. 10 digital interfaces, then generate images meet the test requirements by using graphics softwares.

Fig. 1.
figure 1

Icon collection and processing schematic diagram

Full size image

2.2 The User Computer Control Commands for Obtaining Icon Pictures

The user computer function control commands for obtaining icon pictures includes the following stages: communication between the EEG and the test computer, the testing process, the off-line analysis of the original signal, the extraction of the group average EEG signal in time and frequency features and the microprocessor processing.

The Communications Between EEG Equipment and Test Computer.

Connect ERP device with the test computer loaded with the E-prime software to realize the communication between the ERP device and the computer. E-Prime is a set of computerized experimental design, generation and operation software for mental and behavioral experiments. ERP device is a Neuroscan event-related potential system. The communication between ERP device and test computer is by selecting the corresponding data transmission com interface in E-Prime, and inserting inline statement in E-Prime to achieve ERP device’s triggering, recording, visual stimulation and EEG synchronization and marking.

ERP Experimental Test Process.

20 subjects were selected and each participant was tested 10 times for each icon image. Each icon image obtained 200 test samples, and 200 test samples formed a group. The testing process was as follows: A test image was presented to the subject, using the electrode cap worn on the head of the subject which is configured by ERP devices, the Scan software captured the primitive EEG signals of the subject during the stimulation period of the test image, sampling rate is 500 Hz and all the single test images were traversed to obtain subjects’ original EEG signals of all the test images during the stimulus period. The presentation of the test image included the prompt stage, the visual stimulation presentation stage of icon images and the blank stage. At the prompt stage, there was a 32px × 32px black cross in the center of the screen, then disappeared after 1000 ms, at this stage, subjects need to concentrate. In the visual stimulus presentation stage of icon pictures, the center of the screen would present any single icon image in an area of 48px × 48px, subjects need to observe it carefully, the icon image would disappear after 1000 ms. In the blank stage, the screen showed a white blank screen, lasting for 1000 ms to eliminate the visual residue of subjects.

The number of subjects was 20, including 10 males and 10 females, all with university education background, aged between 20–30 years old, all right-handed, without history of mental illness or brain trauma, with normal vision or normal vision correction. The Neuroscan ERP system includes the Synamp2 signal amplifier, the Scan EEG recording analysis system and the 64-channels Ag/AgCl electrode cap, the electrodes are placed in accordance with the international 10–20 system. Before the EEG was recorded, Scan parameter settings include: the reference electrode was placed in bilateral mastoid connection. The ground electrode is placed in the connection midpoint between the FPZ electrode and the FZ electrode, both horizontal and vertical ocular electricity were recorded simultaneously, and the bandpass is 0.05–100 Hz, the sampling frequency is 500 Hz, electrode and scalp contact resistance are less than 5 kΩ. The visual stimulus program is running on the E-Prime, presented through the display, all the pictures’ background is white in the test process, and all stimuli are located in the center of the screen, the test process diagram is shown in Fig. 2.

Fig. 2.
figure 2

Schematic diagram of ERP experiment testing process

Full size image

Raw Signals Offline Analysis.

This research use Scan software to offline analyze the raw EEG signals, which includes each EEG signal pre-processing and each group superposed average, aiming to obtain the group average EEG electrical signals \( \overline{{y_{i} (t)}} \) of each icon. \( \overline{{y_{i} (t)}} \) is the general expression of the group average EEG signals of each icon, denoting the group average EEG signals of the icon i.

Pre-processing is to respectively preview each raw EEG signal and to eliminate ocular artifacts, and then extract raw EEG signals segmentally, to obtain the icon i visual stimuli raw signal \( y_{i} (t) \), t are the sampling time points and \( t = 1000 + 2m \), null point of t is the time when visual guidance center appear, m are the sampling amounts of icon visual stimuli raw EEG signals and m is integer on [1, 500], finally, correct baseline of the icon visual stimuli raw signals \( y_{i} (t) \) and eliminate artifacts. The icon visual stimuli raw signals \( y_{i} (t) \) that the artifacts have been removed from 10 tests of 20 subjects respectively were processing by group superposition average, then get \( \overline{{y_{i} (t)}} \).

Offline analysis is to re-analyze and re-process the recorded raw physiological signals. All processes of the offline analysis can be accomplished by Scan software. The process of offline analysis includes EEG preview, ocular artifacts elimination, raw EEG signals segmental extraction, baseline correction, artifacts elimination and group superposition average. EEG preview is to remove the obvious offset EEG data; Ocular artifacts elimination can be accomplished by the independent component analysis (ICA) refer to the plus or minus of EOG amplitude, the direction of EOG and the direction of EEG; Raw EEG segmental extraction is to segment the continuous recorded raw EEG data, according to the time point when each icon shows and the 1000 ms from start to end of the chosen icons that marker recorded. This time means a 1000 ms time period that we extract each icon from showing to disappearing; Baseline correction is to eliminate the offset that EEG relative to baseline, the baseline of this process is according to the signals when icons display; Artifacts elimination is to remove the higher amplitude artifacts of the EEG data which exist in the segmental period. The amplitude defaults of the artifacts elimination are between ±50–±100 μv; Group superposition average obtains the relevant group average EEG signals by superpose and average all EEG data of all subjects. The group average EEG signals of each icon are the EEG signals after 200 times superposition average.

The Time Domain and Frequency Domain Features Extraction of Group Average EEG Signals.

In this stage, we extract the time domain and frequency domain feature for group average EEG signals \( \overline{{y_{i} (t)}} \) of each icon. The time domain feature \( \overline{{Y_{i} }} \) contains average \( \overline{{Y_{i1} }} \), absolute average amplitude \( \overline{{Y_{i2} }} \), variance \( \overline{{Y_{i3} }} \), average root value \( \overline{{Y_{i4} }} \), peak-peak value \( \overline{{Y_{i5} }} \), waveform factor, \( \overline{{Y_{i6} }} \) kurtosis factor \( \overline{{Y_{i7} }} \) and skewness factor.\( \overline{{Y_{i8} }} \) The frequency domain feature \( \overline{{S_{i} }} \) contains spectrum average amplitude \( \overline{{S_{i1} }} \), spectrum variance \( \overline{{S_{i2} }} \), the first spectrum feature frequency \( \overline{{S_{i3} }} \) and the second spectrum feature frequency \( \overline{{S_{i4} }} \).

  1. (1)

    The following method shows how to extract the time domain feature of the icon i group average EEG signal \( \overline{{y_{i} (t)}} \):

The average \( \overline{{Y_{i1} }} \) of the time domain feature \( \overline{{Y_{i} }} \) in the icon i group average EEG signal \( \overline{{y_{i} (t)}} \) is:

$$ \overline{{Y_{i1} }} = \frac{1}{500}\sum\limits_{1002}^{2000} {\overline{{y_{i} (t)}} } = \frac{1}{500}\sum\limits_{m = 1}^{500} {\overline{{y_{i} (1000 + 2m)}} } $$
(1)

Average amplitude \( \overline{{Y_{i2} }} \):

$$ \overline{{Y_{i2} }} = \frac{1}{500}\sum\limits_{1002}^{2000} {\overline{{\left| {y_{i} (t)} \right|}} } = \frac{1}{500}\sum\limits_{m = 1}^{500} {\left| {\overline{{y_{i} (1000 + 2m)}} } \right|} $$
(2)

Variance \( \overline{{Y_{i3} }} \):

$$ \overline{{Y_{i3} }} = \frac{1}{499}\sum\limits_{1002}^{2000} {\left( {\overline{{y_{i} (t)}} - \overline{{Y_{i1} }} } \right)^{2} } = \frac{1}{499}\sum\limits_{m = 1}^{500} {\left( {\overline{{y_{i} (1000 + 2m)}} - \overline{{Y_{i1} }} } \right)^{2} } $$
(3)

Average root value \( \overline{{Y_{i4} }} \):

$$ \overline{{Y_{i4} }} = \sqrt {\frac{1}{500}\sum\limits_{1002}^{2000} {\left( {\overline{{y_{i} (t)}} } \right)^{2} } } = \sqrt {\frac{1}{500}\sum\limits_{m = 1}^{500} {\left( {\overline{{y_{i} (1000 + 2m)}} } \right)^{2} } } $$
(4)

Peak-peak value \( \overline{{Y_{i5} }} \):

$$ \overline{{Y_{i5} }} = \hbox{max} \left( {\overline{{y_{i} (t)}} } \right) - \hbox{min} \left( {\overline{{y_{i} (t)}} } \right) = \hbox{max} \left( {\overline{{y_{i} (1000 + 2m)}} } \right) - \hbox{min} \left( {\overline{{y_{i} (1000 + 2m)}} } \right) $$
(5)

Waveform factor \( \overline{{Y_{i6} }} \):

$$ \overline{{Y_{i6} }} = \frac{{\sqrt {\frac{1}{500}\sum\limits_{1000}^{2000} {\left( {\overline{{y_{i} (t)}} } \right)^{2} } } }}{{\frac{1}{500}\sum\limits_{1000}^{2000} {\overline{{\left| {y_{i} (t)} \right|}} } }} = \frac{{\sqrt {\frac{1}{500}\sum\limits_{m = 1}^{500} {\left( {\overline{{y_{i} (1000 + 2m)}} } \right)^{2} } } }}{{\frac{1}{500}\sum\limits_{m = 1}^{500} {\left| {\overline{{y_{i} (1000 + 2m)}} } \right|} }} = \frac{{\overline{{Y_{i4} }} }}{{\overline{{Y_{i2} }} }} $$
(6)

Kurtosis factor \( \overline{{Y_{i7} }} \):

$$ \overline{{Y_{i7} }} = \frac{{\frac{1}{500}\sum\limits_{1002}^{2000} {\left( {\overline{{y_{i} (t)}} } \right)^{4} } }}{{\left( {\frac{1}{500}\sum\limits_{1002}^{2000} {\left( {\overline{{y_{i} (t)}} } \right)^{2} } } \right)^{2} }} = \frac{{\frac{1}{500}\sum\limits_{m = 1}^{500} {\left( {\overline{{y_{i} (1000 + 2m)}} } \right)^{4} } }}{{\left( {\frac{1}{500}\sum\limits_{m = 1}^{500} {\left( {\overline{{y_{i} (1000 + 2m)}} } \right)^{2} } } \right)^{2} }} $$
(7)

Skewness factor \( \overline{{Y_{i8} }} \):

$$ \overline{{Y_{i8} }} = \frac{{\frac{1}{500}\sum\limits_{1002}^{2000} {\left( {\overline{{y_{i} (t)}} } \right)^{3} } }}{{\left( {\sqrt {\left( {\frac{1}{500}\sum\limits_{1002}^{2000} {\left( {\overline{{y_{i} (t)}} } \right)^{3} } } \right)} } \right)^{3} }} = \frac{{\frac{1}{500}\sum\limits_{m = 1}^{500} {\left( {\overline{{y_{i} (1000 + 2m)}} } \right)^{3} } }}{{\left( {\sqrt {\left( {\frac{1}{500}\sum\limits_{m = 1}^{500} {\left( {\overline{{y_{i} (1000 + 2m)}} } \right)^{3} } } \right)} } \right)^{3} }} $$
(8)

\( y_{i} (t) \) are the icon i visual stimuli raw EEG signal and \( \overline{{y_{i} (t)}} \) are the icon i group average EEG signal, t are the sampling time points and \( t = 1000 + 2m \), m are the sampling amounts of icon visual stimuli raw EEG signals and m is integer on [1, 500].

  1. (2)

    The following method shows how to extract the frequency domain feature of the icon i group average EEG signals \( \overline{{y_{i} (t)}} \):

First, group EEG signals \( \overline{{y_{i} (t)}} \) of the icons are analyzed based on Fourier Transform respectively, then spectrum function \( \overline{{S_{i} \left( k \right)}} \) of \( \overline{{y_{i} (t)}} \) is obtained as follows:

$$ \overline{{S_{i} \left( k \right)}} = \sum\limits_{1002}^{2000} {e^{{ - i\frac{2\pi }{500}tk}} } \overline{{y_{i} (t)}} = \sum\limits_{m = 1}^{500} {e^{{ - i\frac{2\pi }{500}(1000 + 2m)k}} } \overline{{y_{i} (1000 + 2m)}} $$
(9)

−i is the plural unit, t are sampling time points and \( t = 1000 + 2m \), m are the sampling amounts of icon visual stimuli raw EEG signals and m is integer on [1, 500],k represent spectrum points and k is integer [1, 500].

Then, The following method shows how to extract the frequency domain feature \( \overline{{S_{i} }} \) of the icon i group average EEG signal.

The spectrum average amplitude \( \overline{{S_{i1} }} \) of the frequency domain feature \( \overline{{S_{i} }} \) in the icon i group average EEG signal is:

$$ \overline{{S_{i1} }} = \frac{{\sum\limits_{k = 1}^{500} {\overline{{S_{i} \left( k \right)}} } }}{500} $$
(10)

Spectrum variance \( \overline{{S_{i2} }} \):

$$ \overline{{S_{i2} }} = \frac{{\sum\limits_{k = 1}^{500} {\left( {\overline{{S_{i} \left( k \right)}} - \overline{{S_{i1} }} } \right)^{2} } }}{499} $$
(11)

The first spectrum feature frequency \( \overline{{S_{i3} }} \):

$$ \overline{{S_{i3} }} = \frac{{\sum\limits_{k = 1}^{500} {\overline{{S_{i} \left( k \right)}} } f_{k} }}{{\sum\limits_{k = 1}^{500} {\overline{{S_{i} \left( k \right)}} } }} $$
(12)

The second spectrum feature frequency \( \overline{{S_{i4} }} \):

$$ \overline{{S_{i4} }} = \sqrt {\frac{{\sum\limits_{k = 1}^{500} {\overline{{S_{i} \left( k \right)}} \left( {f_{k} - \overline{{S_{i3} }} } \right)^{2} } }}{500}} $$
(13)

\( \overline{{S_{i} \left( k \right)}} \) are obtained by Fourier Transform to the icon group average EEG signal \( \overline{{y_{i} (t)}} \), and denote the spectrum of the icon i group average EEG signal \( \overline{{y_{i} (t)}} \). \( f_{k} \) are the frequency value of the kth spectrum points, and the calculation formula of \( f_{k} \) is \( f_{k} = \frac{k.Fs}{500} \). In this formula, Fs is the sampling frequency (500 Hz), k denote the spectrum points and k is integer on [1, 500].

Microprocessor Processing.

In this process, the time domain feature and the frequency domain feature which extracted from the icon group average EEG signal are sent to the microprocessor, and have been translated into digital signals that users can recognize, they are used to the user computer function control command of the corresponding icons through the USB or other general input sent to the user computer, which are stored into the icon activation command module of the user computer. This process is aimed to activate the close icon, save icon, cancel icon, forward icon, zoom in icon, zoom out icon, select icon, cut icon, maximize icon or minimize icon in the user computer.

2.3 The Extraction of the Time Domain and Frequency Domain Features in EEG Signals of the Target Icons

This process connects the ERP devices to the user computer. Users should wear an electrode cap and observe certain target icons, then the certain target icon will have a 1000 ms stimuli to the users. We use Scan software in ERP devices, and collect the raw EEG signals when users are stimulated. The sampling frequency is 500 Hz. We do EEG preview, ocular artifacts elimination, baseline correction and artifacts elimination to the collected raw EEG signals, and then extract the time domain feature \( Y_{i} \) and the frequency domain feature \( {\text{S}}_{i} \) of the EEG signals \( y_{i}^{{\prime }} (t^{{\prime }} ) \) that artifacts have been eliminated.

\( y_{i}^{{\prime }} (t^{{\prime }} ) \) are the general expression of the EEG signals in the stimulation of certain target icon after artifacts elimination, representing the EEG signals of the target icon that correspond to the icon i. \( t^{{\prime }} \) are the sampling time points when observing the certain target icon, time null point is the time when certain target icon displays and \( t^{{\prime }} = 2n \), n are the sampling amounts and is integer on [1, 500].

The time domain feature \( \overline{{Y_{i} }} \) contains average \( \overline{{Y_{i1} }} \), absolute average amplitude \( \overline{{Y_{i2} }} \), variance \( \overline{{Y_{i3} }} \), average root value \( \overline{{Y_{i4} }} \), peak-peak value \( \overline{{Y_{i5} }} \), waveform factor \( \overline{{Y_{i6} }} \), kurtosis factor \( \overline{{Y_{i7} }} \) and skewness factor \( \overline{{Y_{i8} }} \). The following method shows how to extract the time domain feature of the EEG signals \( y_{i}^{{\prime }} (t^{{\prime }} ) \) in certain target icon which correspond to the icon i.

The average \( Y_{i} \) of the time domain feature \( Y_{i1} \) of the EEG signal \( y_{i}^{{\prime }} (t^{{\prime }} ) \) in certain target icon which correspond to the icon i is:

$$ Y_{i1} = \frac{1}{500}\sum\limits_{2}^{1000} {y_{i}^{{\prime }} (t^{{\prime }} )} = \frac{1}{500}\sum\limits_{n = 1}^{500} {y_{i}^{{\prime }} (2n)} $$
(14)

Average amplitude \( Y_{i2} \):

$$ Y_{i2} = \frac{1}{500}\sum\limits_{2}^{1000} {\left| {y_{i}^{{\prime }} (t^{{\prime }} )} \right|} = \frac{1}{500}\sum\limits_{n = 1}^{500} {\left| {y_{i}^{{\prime }} (2n)} \right|} $$
(15)

Variance \( Y_{i3} \):

$$ Y_{i3} = \frac{1}{499}\sum\limits_{2}^{1000} {\left( {y_{i}^{{\prime }} (t^{{\prime }} ) - Y_{i1} } \right)}^{2} = \frac{1}{499}\sum\limits_{n = 1}^{500} {\left( {y_{i}^{{\prime }} (2n) - Y_{i1} } \right)^{2} } $$
(16)

Average root value \( Y_{i4} \):

$$ Y_{i4} = \sqrt {\frac{1}{500}\sum\limits_{2}^{1000} {(y_{i}^{{\prime }} (t^{{\prime }} ))^{2} } } = \sqrt {\frac{1}{500}\sum\limits_{n = 1}^{500} {(y_{i}^{{\prime }} (2n))^{2} } } $$
(17)

Peak-peak value \( Y_{i5} \):

$$ Y_{i5} = \hbox{max} (y_{i}^{{\prime }} (t^{{\prime }} )) - \hbox{min} (y_{i}^{{\prime }} (t^{{\prime }} )) = \hbox{max} (y_{i}^{{\prime }} (2n)) - \hbox{min} (y_{i}^{{\prime }} (2n)) $$
(18)

Waveform factor \( Y_{i6} \):

$$ Y_{i6} = \frac{{\sqrt {\frac{1}{500}\sum\limits_{2}^{1000} {(y_{i}^{{\prime }} (t^{{\prime }} ))^{2} } } }}{{\frac{1}{500}\sum\limits_{2}^{1000} {\left| {y_{i}^{{\prime }} (t^{{\prime }} )} \right|} }} = \frac{{\sqrt {\frac{1}{500}\sum\limits_{n = 1}^{500} {(y_{i}^{{\prime }} (2n))^{2} } } }}{{\frac{1}{500}\sum\limits_{n = 1}^{500} {\left| {y_{i}^{{\prime }} (2n)} \right|} }} = \frac{{Y_{i4} }}{{Y_{i2} }} $$
(19)

Kurtosis factor \( Y_{i7} \):

$$ Y_{i7} = \frac{{\frac{1}{500}\sum\limits_{2}^{1000} {(y_{i}^{{\prime }} (t^{{\prime }} ))^{4} } }}{{\left( {\frac{1}{500}\sum\limits_{2}^{1000} {(y_{i}^{{\prime }} (t^{{\prime }} ))^{2} } } \right)^{2} }} = \frac{{\frac{1}{500}\sum\limits_{n = 1}^{500} {(y_{i}^{{\prime }} (2n))^{4} } }}{{\left( {\frac{1}{500}\sum\limits_{n = 1}^{500} {(y_{i}^{{\prime }} (2n))^{2} } } \right)^{2} }} $$
(20)

Skewness factor \( Y_{i8} \):

$$ Y_{i8} = \frac{{\frac{1}{500}\sum\limits_{2}^{1000} {(y_{i}^{{\prime }} (t^{{\prime }} ))^{3} } }}{{\left( {\sqrt {\left( {\frac{1}{500}\sum\limits_{2}^{1000} {(y_{i}^{{\prime }} (t^{{\prime }} ))^{3} } } \right)} } \right)^{3} }} = \frac{{\frac{1}{500}\sum\limits_{n = 1}^{500} {(y_{i}^{{\prime }} (2n))^{3} } }}{{\left( {\sqrt {\left( {\frac{1}{500}\sum\limits_{n = 1}^{500} {(y_{i}^{{\prime }} (2n))^{3} } } \right)} } \right)^{3} }} $$
(21)

\( y_{i}^{{\prime }} (t^{{\prime }} ) \) are the EEG signals of the certain target icon that corresponds to the icon i, \( t^{{\prime }} \) are the sampling time points and \( t^{{\prime }} = 2n \), n are the sampling amounts and n is integer on [1, 500].

The frequency domain feature \( \overline{{S_{i} }} \) contains spectrum average amplitude \( \overline{{S_{i1} }} \), spectrum variance \( \overline{{S_{i2} }} \), the first spectrum feature frequency \( \overline{{S_{i3} }} \) and the second spectrum feature frequency \( \overline{{S_{i4} }} \). The following method shows how to extract the frequency domain feature of the EEG signals \( y_{i}^{{\prime }} (t^{{\prime }} ) \) in certain target icon which correspond to the icon i.

First, EEG signals of certain target icon \( y_{i}^{{\prime }} (t^{{\prime }} ) \) are analyzed based on Fourier Transform respectively, then spectrum function \( S_{i} (k^{{\prime }} ) \) of \( y_{i}^{{\prime }} (t^{{\prime }} ) \) is obtained as follows:

$$ S_{i} (k^{{\prime }} ) = \sum\limits_{2}^{1000} {e^{{ - {\text{i}}\frac{2\pi }{500}t^{{\prime }} k^{{\prime }} }} } y_{i}^{{\prime }} (t^{{\prime }} ) = \sum\limits_{n = 1}^{500} {e^{{ - {\text{i}}\frac{2\pi }{500}2nk^{{\prime }} }} } y_{i}^{{\prime }} (2n) $$
(22)

In which, −i is the plural unit, \( t^{{\prime }} \) are EEG sampling time points and \( t^{{\prime }} = 2n \), n are the sampling amounts and n is integer on [1, 500], k’ represent spectrum points and k is integer to [1, 500].

Then, extracting process of the frequency domain feature \( S_{i} \) of the group average EEG signal are as follows:

The spectrum average amplitude \( S_{i1} \) of the frequency domain feature \( S_{i} \) in the certain target icon EEG signal \( y_{i}^{{\prime }} (t^{{\prime }} ) \) that correspond to the icon i is:

$$ S_{i1} = \frac{{\sum\limits_{{k^{{\prime }} = 1}}^{500} {S_{i} (k^{{\prime }} )} }}{500} $$
(23)

Spectrum variance \( S_{i2} \):

$$ S_{i2} = \frac{{\sum\limits_{{k^{{\prime }} = 1}}^{500} {\left( {S_{i} (k^{{\prime }} ) - S_{i1} } \right)^{2} } }}{499} $$
(24)

The first spectrum feature frequency \( S_{i3} \):

$$ S_{i3} = \frac{{\sum\limits_{{k^{{\prime }} = 1}}^{500} {S_{i} (k^{{\prime }} )f_{{k^{{\prime }} }}^{{\prime }} } }}{{\sum\limits_{{k^{{\prime }} = 1}}^{500} {S_{i} (k^{{\prime }} )} }} $$
(25)

The second spectrum feature frequency \( S_{i4} \):

$$ S_{i4} = \sqrt {\frac{{\sum\limits_{{k^{{\prime }} = 1}}^{500} {S_{i} (k^{{\prime }} )(f_{{k^{{\prime }} }}^{{\prime }} - S_{i3} )^{2} } }}{500}} $$
(26)

\( S_{i} (k^{{\prime }} ) \) are obtained by Fourier Transform to the certain target icon EEG signal \( y_{i}^{{\prime }} (t^{{\prime }} ) \) which correspond to the icon i, and denote the spectrum of the certain target icon EEG signal \( y_{i}^{{\prime }} (t^{{\prime }} ) \). \( f_{{k^{{\prime }} }}^{{\prime }} \) are the frequency value of the kth spectrum points, and \( f_{{k^{{\prime }} }}^{{\prime }} = \frac{{k^{{\prime }} \cdot Fs}}{500} \). In which, Fs is the sampling frequency (500 Hz), k’ denote the spectrum points and k’ is integer on [1, 500].

2.4 Similarity Calculation and Control Command of Activation Target Icons

This stage is aiming to calculate the time domain similarity A between the time domain feature \( Y_{i} \) of certain target icon EEG signals after artifacts elimination and the time domain feature \( \overline{{Y_{i} }} \) of the group average EEG signals which correspond to their icons. Then, calculate the frequency domain similarity B between target icon and the corresponding icon. If both of time domain similarity A and frequency domain similarity B > 90%,the control command of certain target icons will be triggered. As a result, EEG signals that are triggered by icons can control the interface.

Figure 3 depicts the process of activating control command of target icons in magnifying glass. In this figure, 1 denotes electrode cap, 2 denotes ERP devices, 3 denotes SCAN EEG signal processing module, 4 denotes microprocessor, 5 is the similarity of calculating on the EEG signal time domain and frequency domain features, 6 shows the circumstance that both of the time domain and frequency domain similarity is above 0.9, 7 trigger user computer “magnifying glass” command, 8 denotes the display device, 9 shows the target icon “magnifying glass” command has been activated, thus it can accomplish page zoom in.

Fig. 3.
figure 3

The target icon control command activation process

Full size image

The following calculation process shows the time domain similarity A between the certain target EEG signal of the icon i and the corresponding icon average EEG signal.

$$ A = \sum\limits_{j = 1}^{8} {\frac{1}{8}} A_{ij} ,\quad A_{ij} = \left\{ {\begin{array}{*{20}l} {\frac{{\overline{{Y_{ij} }} }}{{Y_{ij} }},\overline{{Y_{ij} }} < Y_{ij} } \hfill \\ {\frac{{Y_{ij} }}{{\overline{{Y_{ij} }} }},\overline{{Y_{ij} }} \ge Y_{ij} } \hfill \\ \end{array} } \right. $$
(27)

\( A_{ij} \) is the jth time domain feature similarity of the icon i, \( \overline{{Y_{ij} }} \) is the jth time domain feature value of the icon i group average EEG signal, \( Y_{ij} \) is the jth time domain feature value of the icon i’s certain target icon EEG signal. j is integer on [1, 8], and there are 8 indexes in time domain feature and each index weights \( \frac{1}{8} \).

The calculation of frequency domain similarity B is as follows:

$$ B = \sum\limits_{l = 1}^{4} {\frac{1}{4}} B_{il} ,\quad B_{il} = \left\{ {\begin{array}{*{20}l} {\frac{{\overline{{S_{il} }} }}{{S_{il} }},\overline{{S_{il} }} < S_{il} } \hfill \\ {\frac{{S_{il} }}{{\overline{{S_{il} }} }},\overline{{S_{il} }} \ge S_{il} } \hfill \\ \end{array} } \right. $$
(28)

\( B_{il} \) is the lth frequency domain feature similarity of the icon i, \( \overline{{S_{il} }} \) is the lth frequency domain feature value of the icon i group average EEG signal, \( S_{il} \) is the lth frequency domain feature value of the icon i’s certain target icon EEG signal. l is integer on [1, 4], there are 4 indexes in frequency domain feature and each index weights \( \frac{1}{4} \).

3 Discussion

This paper realizes the EEG control of the digital interface through using icons. In the future, we can try to use the selective attention mechanism of the navigation bar, the brain mechanism of color difference, in order to realize the brain-computer interaction of digital interface by using other elements.

In the early stage, brain-computer interaction needs to be carried out EEG experiment design according to EEG experimental standards and norms. In the using of navigation bar for brain-computer interaction, we can try to use the serial mismatch pattern to complete; in the using of color for brain-computer interaction, we can try to use Go-Nogo experimental paradigm. EEG features include time domain and frequency domain, wherein time domain and frequency domain extraction, classification and identification need to be achieved through the algorithm, and different mathematical algorithms will produce different effects. And other elements of brain-computer interaction, we can try to use wavelet algorithm and other mathematical methods. To achieve a high degree of matching between the EEG signal and the target command signal, various mathematical algorithms can be tried to reduce the error and improve the accuracy.

4 Conclusion

This paper first introduces the specific concept of brain-computer interaction, and analyzes the role of event-related potentials in brain-computer interaction. At the same time, the application of EEG assessment method in interface design, brain-computer interaction was proposed. Finally, integrating ERP experiments, signal feature extraction and classification technology, a digital interface BCI method of icon control was proposed, which is a new attempt and exploration in brain-computer interaction field. Finally, the realization of the brain-computer interaction method of other elements in the interface was discussed.