1 Introduction

This paper shows the development an assessment tool to diagnose Kanner syndrome by the measurement of eyeball movements.

Kanner syndrome is a developmental disability of communication and the social nature. The number of patients have been increasing, but the basic mechanism of the onset is not yet elucidated and effective cures or precautionary measures are not established [1–4]. In Japan, it is reported that approximately 10 % of boys and 4–5 % of girls in primary schools are considered to have some developmental disabilities. Act on Support for Persons with Developmental Disabilities is passed in 2003, and the special support education to children with disabilities is started in 2006. [5, 6] Also, the body of laws and administrative frameworks are maintained. Based on such background, an assessment tool to evaluate not only intellectual abilities and the individual developmental characteristic but also the adaptive behavior or the ability that is necessary to live in society are demanded with medical and welfare institutions to adopt to a real daily life. In addition, it is necessary to support children with developmental disabilities on a long-term basis from infants to adults. To detect and respond to disabilities in early stage, the assessment tool which everyone can give diagnoses is needed.

2 Purpose of the Study

In past study, we developed communication applications “Eye Talk” and “Eye Tell” for people who have difficulty in conversation and writing such as children with physical disability, ALS patients or elderlies using the blink determination system. These applications receive strong supports from not only special support schools in Japan but also some researchers in foreign countries [7–9].

Then we developed the image processing technique by afterimage used in the blink determination. First the eye area is captured by a front camera of laptop PC.

Even when a person gazes at one point and his/her eyeballs always stay fixed at that point, their eyes perform subtle fixating movements (i.e. tremors, drifting, microsaccades) to keep the retinal image clear. Particularly, the microsaccades link with nerves and reflect the mechanism that process sight in a brain. We converted the differences between these movements into numbers.

The process of the conversion is as followed:

  1. (1)

    Select the pixel indicating the subject’s pupil from images of captured frames.

  2. (2)

    Set up a reference image, known as an afterimage, from the pixel indicating the subject’s pupil.

  3. (3)

    Divide the pupil of the subject into four from the center in the acquired frame image.

  4. (4)

    Select the pixel in each divided part and count the number of the pixels of the overlapping part with the present pixel based on the afterimage.

  5. (5)

    Process the images with precision in 30–40 fps from a camera and convert the amount of change in the pixels of the movements of the right and left eyeballs in to numbers.

Then the pixels of pupils were extracted with 30–40 fps of accuracy and eyeball movements were digitized.

We converted the sequence of eyeball movements into a graph and define it in multidimensional measure. Then we set the identification border with density function of the distribution, cumulative frequency function, and ROC curve. With this, we established the objective index to determine Kanner syndrome, normal, false positive, and false negative. With this method, we developed the objective evaluation indicator to judge non-autistic and autistic people more clearly in early stage.

By this new assessment, the degree of disabilities and the needed supports become clear. In addition, more accurate and suitable supports can be offered to each children who has Kanner syndrome by considering the intelligence level or the degree of development. The additional evaluations including the possibility to jobs are enabled by grasping a merger of the mental disabilities and the psychoneurotic merger (Fig. 1).

Fig. 1.
figure 1

Acquisition of the blurring of gaze of the eyes of children who has Kanner syndrome

Full size image

3 Structure of the System

3.1 Definition of Afterimage and Digitizing Oculomotor Change

The difference in the area of the amount of change occurs by measuring the difference between the afterimage in consecutive frames and the present frame. We set the amount of change to the quantity of the eyeball movements. This method made it possible to detect a change of the eyeball movements in numerical value. We compared the difference in these movements between non-autistc and autistic people and analyzed the result.

The definition of the afterimage and the detection process are as follows.

When the area (C, Fig. 2) that is overlapping part of the afterimage of iris (A, Fig. 2) and the current iris situation (B, Fig. 2) is diminishing rapidly comparing from a few frames before, we determine it as closing of the eyes. It is possible to exclude a slow change and capture a rapid change. By using the afterimage, the number of past frames to compare increases and we can capture changes with high accuracy. As a result, we can quantify the changes of series of movements “eye-opening → eye-closing → eye-opening”. And it is possible to determine the blinks as short blinks, long blinks, or closing the eyes continuously, except the malfunction due to fine movement of eyes. Figure 2 shows determination by afterimages visually. A is the afterimage to be compared, B is the iris in current frame, and C is the overlapped area of the afterimage and current frame.

Fig. 2.
figure 2

Detection of Unconcious Blink by Afterimages, Visually

Full size image

Figure 3 is an illustration indicating the states to judge opening and closing of eyes. Since it is difficult for a subject with weak muscular strength to close his/her eyes perfectly, it may be with a halfshut blink even if it were to be a conscious blink, as shown in the middle and lower section of Fig. 3.

Fig. 3.
figure 3

Diagram of Opening and Closing of Eye

Full size image

Figure 4 is the illustration indicating of the relations with the number of frames and the blink judgment. Even if a subject has weak muscle strength, an unconscious blink (a blink carried out naturally) is faster than a conscious blink.

Fig. 4.
figure 4

Relationship Between Number of Frames and Blink Determination

Full size image

The number of the first frames K1 is set from a few frames to more than 10 and less than 20 frames to remove such a fast unconsciousness blink. In this way, we can remove the unconsciousness blink from the closing and opening of eyes that continued in a few frames (Fig. 4(a)). On the other hand, the number of the second frames K2 is set from in dozens of frame degree (ex; 50 frames) to remove the blink that is slower than a normal, consciousness blink. It makes it possible to cover a case where a subject with weak muscle strength continues closing his/her eyes because he/she becomes tired to watch a screen from conscious blinking (Fig. 4(b)). In this way, we remove an unconsciousness blink and the case that a subject continues closing his/her eyes and determine it as a conscious blink when K (the number of the frame acquisition) is in the range between K1 (the number of the first frame) and K2 (the number of the second frame) to judge a conscious blink appropriately. In addition, if the sensitivity in blink judgment sensitivity setting part 44 becomes higher (if it gets closer to value 100), K1 is set in a small value and K2 is set in a big value, and the appointed number range will increase. Then, the fast blink that is almost an unconsciousness blink is judged as a consciousness blink. For this issue, we can remove an unconsciousness blink and the continued closing of eyes properly by adjusting it to an appropriate set point.

The application “Eye Talk” which the selected letters are pronounced with is developed by using this highly precise blink judgment. We focused on making it possible to choose a letter less.

The definitions of the afterimage generation are as followed: After judging white or black, the part that is being recognized as black is determined to be an afterimage.

When it continues being recognized, it means that it is judged as black more than about 13f in the case of 24 fps and it is determined to be an afterimage.

Images are always judged consecutively. All pixels have Persistence Value and subtract 1 frame from every value.

In addition, Persistence Value does not become less than 0 and is always an integer. When it is determined that a pixel is black, 5 values are added.

We define the black part as an afterimage based on value of image persistences. The value becomes - 1 + 5 when the value of image persistences is judged to be black, and we make the program considering it as an afterimage when the value is over 50. Over 50 means the part which is judged as a black part during approximately 0.45 s is determined as an afterimage because 24–30 frames are processed per a second (Fig. 5).

Fig. 5.
figure 5

The Change of Persistance Value

Full size image

In the eye movement measurement with a tablet, the position sensing of the eyes part using the OpenCV HAAR-like assorter burdened the image processing (Fig. 6).

Fig. 6.
figure 6

Gaze Direction Detecting

Full size image

Therefore we decided to use the image from the front camera of the laptop PC this time instead of a tablet. With parallel computation by PC multitasking, we can always demand ocular centers. We divided the iris of the eye into 4 parts from the center point and set the image persistence value. When the value is beyond “50 level”, it assumes as an afterimage. We compare the number of the current pixels with the afterimage and determine it as eye movement in right and left when each condition is met. Figure 12 shows the condition of each judgment.

This blink determination and gaze detraction detecting make it possible to choose letters remarkably fast (Fig. 7).

Fig. 7.
figure 7

Number of the pixels of afterimage divided into four parts

Full size image

We measured the amount of change of the eyeball movements of right and left eyes with precision of 14 fps–18 fps by the comparison with the afterimage. The right part [Fig. 7] summarized the number of the pixels of right and left eyes divided in 4 in every 1 flame. We divide the right eye into number 1 to 4, and the left eye into number 5 to 8 and digitize the number of each pixel.

4 Clinical Application of Using the Blurring Gaze of Eyes Judgement for Handicapped Children

The team of Osaka University discovered abnormality of the gaze activity of children with Attention Deficit Hyperactivity Disorder (ADHD) by measuring fast oculomotor movement (saccade ocular movement) using eye-tracking (Tobii) in “Research trends in eye-hand coordination of mentally retarded persons” [10–13].

Diagnosis are made based on a comparison of “how long we look into the subject’s eyes”, that is the time of gazing an object in the conventional autistic study [14–16], but the results do not match.

In this study, the subjects of experiments are 54 children in the special support school who had diagnosed as Kanner syndrome. We showed them the animation combined factor of the fixation and pursuit which a ball moves and stops for 15 s and measured the blurring of gaze of the eyes.

We measured the amount of the degree that the eyes of the subject run off the track based on the afterimage, then added up the amount of change of right and left eyes and showed the total. Figure 8 is the distribution map which showed the oculomotor dispersion of each group.

Fig. 8.
figure 8

Difference of eyes dispersion between normal development group and Kanner syndrome group

Full size image

While the dots were concentrated in the center line in the normal development group, they were scattered around the area in the Kanner syndrome group. It indicate that subjects in the normal development group can follow pursuit the moving target with eyes, but gaze of subjects in the Kanner syndrome group run off the track easily. By comparing the distance from the center line of this distribution in each group, it is proved that the distance of the Kanner syndrome group is significantly farther than the normal development group in both children and adults.

We compare the data of 30 normal developing people with the data which we acquired by this experiment in density function and calculate each identification border.

Figure 9 shows the relations of the normal development group and the Kanner syndrome group by the function of density of probability.

Fig. 9.
figure 9

Relations of normal development group and the Kanner syndrome group with the density of probability

Full size image

We also compare the group that cannot to follow the instructions, the autistic group and normal developing group in density function and calculate each identification border.

The red line is the group not following the instructions, the blue line is normal developing coup and the green line is the Kanner syndrome group (Fig. 9).

From the results, the identification border between the group not following the instructions and the normal developing group is 240. The border between the Kanner syndrome group and normal developing group is 460.

The true negative rate that could judge the normal development group is 99 % and the false positive rate is 1 % (Table 1).

Table 1. Boolean table indication whether positive or negative
Full size table

The truth positive rate to judge the Kanner syndrome group is 96 % and the false negative rate that judges an Kanner syndrome person as normal (Table 1).

Then we measure the reliability of the results using ROC curve [17, 18]. The reliability that could distinguish a normal developing person to an autistic one is 98.77 % (Fig. 10). On ROC, the shape shifts to the top left when there is little heap of two density functions, and shows that the performance of the decision surface line is high. AUC(Area under the curve) surrounded by the ROC curves is AUC = 0.9877. Since the area is big, it is considered that the reliability of this method is high.

Fig. 10.
figure 10

The reliability measurement using ROC curve

Full size image

From this result, the identification method of this study considered quite useful for the judgment of people who has Kanner syndrome.

5 Conclusion

In this study, it was intended to establish the objective index using the blurring of gaze of the eyes to be used for the diagnosis of children who has Kanner syndrome based on our past researches. We succeeded to develop the method of the gaze of the eyes measurement with highly precise. By using the front camera of the popular laptop PC, we established the objective index to judge the Kanner syndrome group, the normal development group, the truth negative and the false negative without using an expensive devise.

Children who has Kanner syndrome tend not to make eye contacts. In the study of Osaka University, they used Tobii and showed that their gaze moves and are not fix. From this, it is considered that this measurement is the technique that is epoch making as a diagnosis of the Kanner syndrome by the eyes detection. Even if the final diagnosis is entrusted to a child psychiatrist, utility value and the versatility of this measurement as a supporting assessment are relatively high. Particularly, it is thought that it has a huge meaning just to understand possibility of the Kanner syndrome for the parent of the child in the gray zone. This measurement is stress free because it does not require to restrict a subject and takes only 15 s. The measurement is also unique in the point of showing the possibility of ASD without doing action observation.

Children who has disabilities change their behaviors through trainings or experiences and get used to the society. Special support schools aim to independency in both daily and social lives and occupational self reliance. Self reliance means they can get some supports depending on their needs.

This method can carry out important tasks as the supporting assessment tool of the judgements of psychiatrists to discover disabilities in children and consider appropriate correspondence. In addition, as well as a judgment, it is considered that this method leads to evaluations of education and curative effects in educational fields.