Keywords

1 Introduction

There is a significant increase in the use of Computer-Based Neuropsychological Assessment (CBNA), in clinical diagnosis practice in different specialties, such as neuroscience and cognition dyslexia diagnosis [1, 2], sports-related concussion [3, 4], human computer biological signals interaction [5], neurologic patients [4]. Since CBNA offers several advantages when applied to different clinical disorders in neuroscience, it is possible to accurately control variables for measuring cognitive functions, such as reaction time, correct and incorrect responses, error types, and direct stimuli administration. CBNA brings a great advantage for clinicians to dynamically manage assessment sessions, reducing the risk of errors related to sequence of tasks during an evaluation. Furthermore, it significantly simplifies quantitative analysis of the outcome results, by automatically calculating the desired data with high definition and data grouping according to neuropsychological clinical models previously established. Administration of assessment procedures in neuropsychology in general requires at least one measure which needs to be computed, scored or counted [6].

Other important advantages are identified when using CBNA instruments, such as reliability and efficient use of available resources. When assessing executive functions there are specific considerations that a clinician must take care about, minimal deflections of procedures during administration of the tests, could lead to a complete discard of results. For example, an assessment instrument called Wisconsin Card Sorting Test, has shown good reliability when measuring executive functions. A study by Tien et al. [7] which compared the performance between paper-based (typical cards variant of the test and a computer-based version in psychiatric participants, showed more reliable administration and accuracy in data acquisition and scoring, when using the computer version.

Adolescence is behavioral and physiological, including hormonal changes age and stage of life that requires specific assessment methods. Puberty supposes a cascade of changes over the endocrinal system that involves brain and psychological processes related to affect regulation [8]. The mentioned changes often require accurate clinical assessment procedures for taking decisions about improving development. Likewise, cognitive competence in adolescence includes the ability to reason effectively, problem solve, think abstractly and reflect, and plan for the future. Despite their rapidly developing capacity for higher-level thinking, most adolescents still need guidance from adults to develop their potential for rational decision making [9]. According to Giedd [10] in a longitudinal study review related to teenager brain assisted by neuroimaging, investigations report that at ages from 7–29 years, adolescents respond to reward particularly in regions corresponding to the nucleus accumbens, which was equivalent to that found in adults, but adolescent orbitofrontal activity was similar to the equivalent area in children, fact that may be associated to a lack of maturity of rational decision making functioning. Similarly, maturation of prefrontal cortex that regulates judgment, caution, and appropriate behavior is a relatively late in adolescence, early adult [11]. Other studies indicate that the level of maturation of intelligence activity is related to the trajectory of cortical thickening during its development through childhood and adolescence, primarily in frontal regions [12]. Development of frontal cortex through adolescence period, has relevant changes that bias maturity of executive functions. Executive function developmental process, ends between the second and third decades of life, specifically in a critical period between 12–15 years old where cognitive functions improvement such as, mental flexibility and sequencing, visuospatial and sequential planning, verbal and visuospatial working memory, and risk and benefit mental processing [13], functions that have an important role in self-control, coordination of thoughts and adaptive behavior to diverse circumstances of lifespan [14]. According to research results [15] assessment of executive functions during adolescence, is critical as a dominant factor among other non-cognitive skills related to educational performance, health behavior and delinquency or substance abuse.

There are instruments such as the Self-descriptive Adolescents Inventory [16], dedicated to evaluate personality and social skills in adolescents, the procedures of application of those instruments are complex. Limitations of neuropsychological executive functions instruments often are related to a lack of well-standardized, developmental assessment techniques [17]. When geographical conditions represent a challenge to resolve, as often occurs in rural areas, as well as vulnerable conditions where clinical services are difficult to approach, remote administration of neuropsychological assessment tests by minor requirements or internet facilitates these clinical procedures [3]. Previous built CBNA as the NeuroCogFX [4], had included executive function tasks for assessing short term memory, working memory, psychomotor speed, selective attention, verbal and figural memory and verbal fluency, which resulted in a brief 25 min structured battery supported on statistical reliability and standards.

The study of the impact of computer-based assessment tools has been extended even to academic and educational areas [18]. Results in the field of academic assessment, had demonstrated that after familiarization period, computer-based assessment performance, and acceptance is an alternative to pen-and-paper theoretical practical examinations.

It has been reported that computer familiarity is related to performance on computerized neurocognitive assessment [19]. Particularly adolescents had recently increased their screen-time spent, which is reported as a factor of behavior modification [20], but can also be taken as an advantage for improving clinical assessment methods. The aim of the present study is to describe the validation procedures of a CBNA tool in a sample of control and clinical adolescent participants.

2 Method

This study starts from previously validated and elsewhere published clinical assessment tasks [21, 22], which are commonly used by neuropsychologists for executive functions and emotion interaction diagnosis, for different cognitive disorders and syndromes.

2.1 Instruments and Procedures

Montreal Cognitive Assessment (MoCA) [23] was used as a screening test to measure global cognitive state. The CBNA software was developed by requirements from a neuroscience and cognition laboratory. It is constituted by an Emotionally Interfered Working Memory task (EIWM), composed by 36 emotionally loaded BMP format images, divided in three groups (pleasant, unpleasant, and neutral). And a modified Iowa Gambling Task (mIGT) characterized by 4 card decks configured according to the neuropsychological assessment model, in which the participant can lose or gain virtual money.

Required modules were developed using JAVA, and were integrated in a unique interface from which can be evocated (Fig. 1). All logic and flow rules are embedded in the code, as well as the acquired results of the task, which are exported to a file. Options for the identification of patient options were included in order to keep a record of the assessed participants. The results of the application of the CBNA record particular indicators of the neuropsychological task features, including type of visual stimuli (pleasant, unpleasant, neutral, or facial emotional expression). For the mIGT, the software is able to record the cumulative score obtained by the participant, by considering the losses or gains. Visual stimuli can be presented in any size monitor, however, for clinical purposes a 19” screen is recommended to ensure the emotional effect of the visual stimuli. Gambling task responses can be obtained by means of the mouse, or a touch screen. EIWM require the use of an external numeric keyboard, marked with a red spot over the number 4, and a green spot over the number 6 to acquire accuracy of response and reaction time. Indicators within reaction times are exported automatically to a predefined folder in the system, in CSV (comma separated values) format, and it is labeled by the task name, participant’s name, and application date. Visual stimuli images are presented in 1024 × 768 pixels (Fig. 1).

Fig. 1.
figure 1

CBNA system screen sample illustrating initial display for participant registration. (Color figure online)

Full size image

EIWM is constituted of three blocks of 21 equally divided pleasant, unpleasant and neutral visual stimuli from the International Affective Picture System. The configuration of EIWM task was measured using previous designed and validated method [22, 24] facial images, objects, and scenes having one of three different emotional valences retrieved from the International Affective Picture System (IAPS) [25]. From this database, 42 pictures were selected, and divided into three groups: 14 associated with positive emotional states, 14 associated with negative emotional states, and 14 considered neutral. The images were presented in pairs with 4 s delay between the first and second pictures, the pairs of images could be the same, meaning that the second picture was not different from the first one, or different, meaning that the picture could have one or more features that are different from the first picture (50% of the pairs were the same and 50% were different). Each of the pairs of pictures differed from other pairs in the emotional valence being shown, in addition to being the same or different, and were presented in a pseudorandom order. The participants were asked to maintain in memory the first image of the pair, and compare the second image with the stored image. Participants were asked to press a green button on a keyboard if the pictures were the same and a red button if the pictures were different, a lag of 10 s was given to respond. If no response was given during that time period, the next pair of pictures was presented. Accuracy of response (AR) as measured by the number of correct responses made and reaction times (RT) were measured in milliseconds for each participant. The assessments were individually conducted in a quiet room where the visual stimuli were shown on a 19 inch color monitor at a distance of 40 cm from participant’s face.

mIGT was a 50 card sequence applied, each card was previously configured and controlled by the person who applied the protocol. The configuration was based on previous developed test procedures [26]. Each participant received a $2000 (virtual Mexican Peso) credit, and had to choose a card at a time from four possible decks A, B, C, and D. Each card could sum or rest money to the participant’s credit. In this variant there was no time limit, and the participant was allowed to change from card deck as many times as he or she wanted, and that the goal of the task was to acquire the largest possible money amount. A research assistant remained close to the participant to give support and resolve doubts during the instructions part of the study, the participant was free to ask questions to the assistant.

2.2 Participants

The sample was a total of 35 control participants, 14.1 years old mean age (SD = 1.6), 53% male, 47% female, 22.6 MoCA Score, and 33 clinically referred volunteer participants, 13.47 years old mean age (SD = 1.23), 63% male, 37% female, 17.9 MoCA Score, whose consultation reason were mainly impulsive behavior. All right handed, volunteer participants signed an informed consent. The study was considered as non-risk, considered all human rights and Helsinki Accord criteria, and was properly approved by the Bioethics Committee of the Medicine and Psychology Faculty of Autonomous University of Baja California.

2.3 Data Analysis

Acquired data was analyzed by two methods, (a) gold-standard comparison with conventional paper based task, for the mIGT, and (b) a two independent samples t-student test between control and experimental participants for both applied tasks: mIGT and EIWM.

2.4 Results

Paper-based mIGT showed statistical difference between groups (t = −11.14, DF = 308, p > .001) means are shown in Fig. 2. A similar result was found for EIWM reaction time for pleasant stimuli (t = 23.1, DF = 308, p > .001), as well as for unpleasant type of stimuli (t = 19.4, DF = 308, p > .001) between groups. An effect of longer reaction times was observed in means of response for each type of stimuli (Fig. 3).

Fig. 2.
figure 2

Means of mIGT scores obtained between control and experimental groups.

Full size image
Fig. 3.
figure 3

Means of reaction times in milliseconds observed between control and experimental participants resolving the EIWM task.

Full size image

Otherwise, accuracy of response also presents differences between types of stimuli (Fig. 4) observed in the mean of number of correct responses. This suggests a lower performance of the experimental group compared to their control pairs, and an effect of reduction of accuracy in contrast with reaction time.

Fig. 4.
figure 4

Mean of number of correct responses observed between control and experimental participants resolving the EIWM task.

Full size image

3 Discussion and Conclusions

CBNA showed the same accurate results when applied in clinical conditions as paper-based instruments, particularly mIGT in this study. Significant differences are found between groups, mainly due to the clinical conditions of participants. Furthermore, CBNA is highly sensitive when measuring reaction times, which is highly difficult to practice with paper-based procedures.

Studies have shown that, gathering of clinical data through digital tests is relevant to detect neuropsychological impairment. It has been observed [27], that scanning of the cognitive function test (CogState) offers, a clear evidence of the neuropsychological alterations related to HIV. In the same way, they reported that the Cogstate instrument has clinical sensitivity and specificity to detect cognitive impairment in these patients. Likewise, with similar procedures [28] another finding was that the use of CBNA in patients with concussions as a result of sports practice, can be useful to detect subtle negative changes in the cognitive process, allowing the neuropsychologist to determine how long patients should wait before starting their training, and therefore, continue participating in sports competitions. Other studies had reported significant differences in the cognitive performance of three groups, two of them, were patients with Parkinson’s disease, with and without mild cognitive impairment, in contrast to a control group, using a technique virtual evaluation and a classic pencil and paper method [29]. Researchers observed that patients with Parkinson’s disease with normal cognitive performance recorded by classical neuropsychological technique and the virtual executive test, show differences, even though in the classical evaluation this group score as a normal; therefore, these results show that use of virtual neuropsychological techniques had shown sensitivity to detect cognitive changes as a predictor of major neurocognitive impairment.

The advantages of using CBNA, reside in the efficacy reached through diverse aspects, like time reduced procedures, application and precision evaluation of tests, integrating some clinical relevant performance characteristics that participants exhibit during test application as the test’s stimuli control presentation, which could derive in an increase of test reliability, thus CBNA allows the clinician, focalize his/her attention in treatment; results from other studies [3] suggests the use of CBNA decreases the influence of examiners. Another benefit as also mentioned by other researchers [28] CBNA could be applied at the same time to certain amount of people, without requiring that more than one neuropsychologist to check the test procedures.

One strength of our study was to leave a precedent, about the relevance of CBNA and the study of digital assessment in a Hispanic speaker population with the objective to create in the future new tools that help and enhance assessment neuropsychology techniques that allow neuropsychologists to make early diagnosis, in addition to the repercussion in the creation of an intervention plan based in virtual environments.

In regards to study limitations they were as follows, the narrowed use of our study sample to observe difference between performance in a Paper-based mIGT group and CBNA groups in order to generalize our results over a population, which allows to accomplish an external validity as a psychometric criteria. Another point is despite the control of variables, the lack of familiarity with technology in our sample could interfere with task performance, according to previous studies [19, 30] individuals with greater computer experience perform better on computer-based assessments than those with less computer experience.

Concluding, the use of CBNA have to take into account psychometric criteria, mainly focusing on two kinds of validity, an ecological one and other of localization data. The first related to prediction of a particular construct with regard to real life abilities and execution of activities of daily living, and the other, associated to test accurately focal lesions. Another relevant point is the characteristic of CBNA referred to clinical aspects, such as sensibility test property to detect subtle neuropsychological abnormalities and specificity test characteristic, or the fact to differentiate neuropsychological deficits between patients. Furthermore, it is important to mention the use of normative data, related to demographic patient’s provenience. It would help to avoid diagnosis errors in individuals who do not have impairment [31]. Another related idea is the future of CBNA as regards the standardization procedures which have to be updated according to emerging technology, like gadgets, taking into account the use of apps through portable devices it has become an advantage nowadays [32]. It should be noted, the study of new virtual environments and how they would improve the detection of subtle changes in cognitive abilities until now, they are not precisely detected with paper-pencil neuropsychological tests [6].

Recent research suggest that new skills are arising from activity performance of actual life, such as information and communication technologies [33], or writing [34]. Such skills may no longer be assessed by conventional manual or paper methods, because of technical and accuracy limitations.

This particular CBNA battery offers an alternative for cognition-emotion interaction in human cognitive processing, well known as hot executive function [35]. Special attention to hot executive functions assessing in adolescents, by using the Cambridge gambling task offers an important predictor of developmental outcomes related to emotional problems during this age [36], but may be related to other psychological disorders like obsessive-compulsive [37], and autism spectrum disorder [38].

Future work may suggest ethical considerations about using CBNA instruments. Digital availability of clinical instruments could lead to inappropriate use of these type of tools, which at the same time may lead to misunderstanding of results, by non-specialists users. For this, and other reasons, previous studies [7] indicate that computer versions of CBNA tests, should not be the substitute for the human clinicians, but a reliable tool that could carry out mechanistic processes of tests administrations, such as presentation of stimuli or scoring.

Considering these ideas, we thought that neuropsychologists have to adapt their clinical procedures, according to the available technology up to now, as a response of the constant technology advances related to the digital era.