Advertisement

Fusion of Static and Dynamic Parameters at Decision Level in Human Gait Recognition

  • Marcin DerlatkaEmail author
  • Mariusz Bogdan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9124)

Abstract

This paper presents the bimodal biometric system based on human gait data of different type: dynamic - ground reaction forces and static - some anthropometric data of human body derived by means of Kinect. The innovation of this work is the use of unprecedented hitherto in the literature set of signals. The study was conducted on a group of 31 people (606 gait cycles). Kistlers force plates and Kinect device as well as the authors software were used to measure and process data. The following anthropometric parameters were used here: torso, hip width, length of left thigh, length of right thigh and body height. These signals have been combined at decision level of the biometric system. Our biometric system in gait recognition process involves both k-nearest neighbour classifier as well as majority voting system. In case of users the False Rejected Rate (FRR) reaches the level of 4.55 % and False Accepted Rate (FAR) is equal to 0.85 %. In the case of impostors it has been possible to reject 26 cases previously classified by 5NN. The presented biometric system fills the gaps in the existing studies and confirms the superiority of systems based fusion over typical methods of human gait recognition.

Keywords

Gait recognition Data fusion Ground reaction forces Kinect 

1 Introduction

The accuracy of biometric systems is extremely important. Therefore, there are currently many approaches, in which the biggest emphasis is put on reducing the classification error as much as possible. As the result, more and more popular are becoming the methods which use fusion of two or more sensors, traits or classifiers. A number of studies show that in this manner better results are obtained than with conventional biometrics [5, 12, 14, 19, 21]. Among a range of biometric methods human gait deserves a special attention [2, 3]. Walking is a natural human activity, which is the result of consistent interaction between nervous, skeletal and muscular systems. It is a unique phenomenon that allows identification of people. Compared to most other biometric methods, depending on the used measurement method, human gait can be measured at a distance and it does not require the cooperation of the person undergoing this type of procedure. In work [4] all attempts to recognize human gait have been divided dependently on the origin of the data. Gafurov has indicated the methods based on the data obtained from: video [8], floor sensor [3] and portable sensors [4]. In the first approach, the measurement data derive from the equipment recording image sequences. There are also many works [1, 10, 11, 13, 17] in which, on the basis of human movement and its anthropometric characteristics obtained by means of Kinect sensor, a person is identified. At least a few approaches and problems associated with the correct recognition can be distinguished here with:
  • the choice of the type of neural network (classifier) and with the reduction of features in the input vector [11, 13],

  • specifying the number of static (anthropometric features) and dynamic (changing the angles between selected pairs of kinematic) recognition parameters [10, 11, 13, 17],

  • identifying the quality of recognition and the parameters describing this quality (factors of evaluation) [10, 17],

  • type of movement (walking, running, etc.) [10, 11, 16].

Recognition of persons based on the data derived from the floor sensor assumes that during walking, the person tested will walk through the measurement path, in which a hidden device registers such parameters as ground reaction forces or the distribution of pressure while placing foot on the ground. Obviously, these methods record signals generated in the gait supporting phase only. Systems of this type can be used everywhere where examined people are forced to pass through hidden (e.g. in floor) force plates, for example through different kinds of gates at the airport. The last group of methods involves the measurement of parameters by devices (most often accelerometers) placed directly on the examined person. Studies of this type require full cooperation of the person that is going to be recognized. In the literature we can find works that, to identify a person by the their gait, use methods of join fusion based methods. In [15] people were recognized based on data derived from eleven cameras that were positioned at a different angle (\(0^{\circ }\), \(18^{\circ }\), \(36^{\circ }\), ... , \(180^{\circ }\)) in relation to the person examined. In [18] dynamic and static characteristics derived from the images of people in motion were combined to improve the quality of the recognition. Whereas in [7], after dividing the gait cycle into a number of parts, submits each of them to identification using the nearest neighbor classifiers (NN). The global decision is the result of majority voting based on the classification results of the NN.

2 Selected Signals Describing Human Movement

Human gait is a multidimensional and highly nonlinear phenomenon described by multiple data collection containing kinematic, dynamic, anthropomorphic, electromyographic and metabolic variables. Thus, the need to use a variety of signals for its description and analysis is something desirable and natural.

2.1 Ground Reaction Forces (GRF)

In human walking, during the support phase of gait, forces are generated between the foot and the ground. These forces are called ground reaction forces. There are three components of the GRF (Fig. 1): vertical, anterior-posterior and lateral. Ground reaction forces depend on the subject’s weight, but also on the cadence, the person velocity and the footwear. Ground reaction forces in the framework of this study were measured by the use of two KISTLER platforms operating at the frequency of 1000 Hz.
Fig. 1.

Components GRF in: (a) anterior/posterior, (b) vertical, (c) medial/lateral direction, during the support phase of the left lower limb

2.2 Anthropometric Measures

Human gait recognition based on anthropometric data began to experience a renaissance with the spread of video cameras and the ability to process a sequence of images on-line. Therefore, a number of works have been published, such as [1], in which the so-called static data have been used for the purpose of identifying a person. It is important to note that, according to the results presented in [3], taking account of the height of a person’s body in human gait recognition based on GRF should reduce the number of false recognitions by half. In our work we chose anthropometric parameters which have the greatest values. Thanks to this, possible measurement error associated with the non-perpendicular setting of Kinect system will have a relatively smaller influence on the results obtained. Hip width parameter has been added to ensure an odd number of parameters, and so, the clear result of the operation of majority voting method, and to facilitate the distinction between the sexes of the subjects. Here we have used the following parameters averaged over a gait cycle: torso, hip width, length of left thigh, length of right thigh and body height (Fig. 2).
Fig. 2.

Body joints in SDK Kinect

3 Movement Tracking by Means of Kinect Device

The SDK Software (version 1.8) contains a library of NUI Skeleton, which allows to obtain information about the locations of 20 body parts (joints) towards the sensor - determining their coordinates (Fig. 2). In order to examine the factors influencing the quality and sensitivity of both detection and parametric description of the captured figure movements (based on the SDK), the GUI (Fig. 3) application has been built. This application allows for:
  • simultaneous capture of image data stream from the RGB camera and depth camera of the Kinect controller,

  • skeletal tracking,

  • the choice of image resolution from RGB camera and a depth camera,

  • a description of the figure movement - calculating and displaying the static parameters of the registered figure (the length of body segments, body height), the angles between the parts of the body (kinematic pairs), the coordinates of points describing the parts of the skeleton,

  • displaying graphs from earlier collected data - modification of the static parameters in time and the angles between the parts of the body,

  • to record specific (significant) parameters to an Excel file or unformatted text file.

Fig. 3.

Main window Kinect application: (a) preview of the tracked skeleton and sliders to change the filter parameters, (b) displaying and change of the image resolution from RGB camera, (c) displaying and change of the image resolution from depth camera, (d) change of the controllers inclination angle, (e) panel with tabs

First, the value of the filtration parameters of raw image data were established. In order to stabilize the joint position - minimize occurring vibrations, the information about tracking skeletal connections can be adjusted in each frame. Kinect for Windows SDK guarantees a mechanism that smoothens the joint position in the frame. What is more, the implemented algorithm can adjust the position of the joints in each frame depending on the fixed action - a filtering method. The smoothing filter used is based on the Holt Double Exponential Smoothing method [6]. The advantage of this filter is operating with low delay while ensuring proper smoothing. The filter can be controlled by five smoothing parameters (Fig. 3):
  • smoothing (range of changes 0.0–1.0, selected value 0.5) - along with the increased value of the parameter the smoother frame is obtained,

  • correction (range of changes 0.0–1.0, selected value 0.5) - lower values are slower to correct towards the raw data and appear smoother, while higher values will correct toward the raw data more quickly,

  • prediction (range of changes \(\ge \)0, selected value 0.5) - the values should be chosen wisely because exceeding the value of 0.5 can cause overshooting effect - too rapid changes. Small values of Max Deviation Radius reductively affect large values of this parameter,

  • jitter radius (range of changes \(\ge \)0, selected value 0.05) - this parameter is responsible for the reduction of jitter. It is expressed in the unit of meters,

  • max deviation radius (range of changes \(\ge \)0, selected value 0.05) - This parameter is expressed in meters. The designated position after filtration may differ from original position.

For the purposes of this article, it was decided to choose only the static parameters (selected anthropometric characteristics). It should be noted that static data are fixed i.e. it is not dependent on the type of human gait (it is often of non-constant speed and non-constant frequency), on its characteristics (speed of locomotion, stride length, etc.). The test of the correctness of the calculation of static parameters depended on the completion of measurements of the selected parts of the body, e.g. body height (BH) and comparing them with the results obtained from the application. The test was performed repeatedly, once with the controller located at the height equal to half of the measured persons height, and the second time at the height equal to 75 % of BH; the examined person moved in accordance with the optical axis of the controller. The measurements commenced at the time when the user was within 2 m from the controller. Then the person had to slowly regress until the sensor lost the sight of its skeleton, and then again returned to the starting location (2 m from the sensor). On the basis of data collected during the test, it can be concluded that the values obtained during the measurement differ little from the real ones. Measurement errors of individual limbs are as follows: left arm - 0.35 %, right forearm - 1.6 %, the left thigh - 2 %, right calf - 2.5 %. The results obtained are very good and the errors are small. In the case of body height, in most frames, the obtained results were lowered by a few centimetres and the error is 3 %. For the calculation of height the sum of individual body sections (among others the length of feet) was used. The sections of the body were connected by points tracked by Kinect. During the movement of the user the sensor could incorrectly identify the position of the feet, which probably influenced the outcome. Calculations of the static parameters are affected by a very small error, in most cases it is in the range of 0.5–2 %, regardless of the height at which the controller is located. It was noted that the measurement error increased when an examined person moved in direction incompatible with the position of the optical axis of the camera sensor. Therefore, in the rest of our study we adopted possible acceptable measurement error equal to \(\pm \) 5 %.

4 Gait Recognition Based on GRF Data and Kinect

Natural differences in the dynamics of gait influence the support phase duration. As a result, the time series describing the ground reaction forces have different lengths. Comparing these time series is possible thanks to DTW (dynamic time warping). DTW is a well-known algorithm that determines the minimum cost of the ‘fit’ of one time series to the other [20]. This cost is the smaller, the more similar to each other the time series are. In this study, DTW was used to determine the similarity of the individual components of the GRF. The investigation takes into consideration all (L = 6) components of ground reaction forces. The total distance was calculated as:
$$\begin{aligned} \rho =\sum _{l=1}^L DTW_{l} \end{aligned}$$
(1)
where: \(DTW_{l}\) is the DTW distance between the same components of GRF of the same lower limb. From some data patterns were created - biometric signatures characteristic to people who were to be recognized (so called users). In the case of GRF these were time series describing the individual components, which at the time of identification were used to calculate the distance according to the formula (1). In case of anthropometric measurements the average value over gait cycle of measured parameters was taken into account. Thanks to this it was possible to eliminate accidental measurement errors.
Fig. 4.

The scheme of the human gait recognition by means of the typical 5NN classifier based on GRF and body parameters derived from Kinect device

Identification of persons was held in two stages (Fig. 4). In the first phase \(\rho \) distances between the measured components of GRF and models included in the database of GRFs were calculated. Next, based on the 5NN classifier, the candidate User ID was selected. Of course, there is a number of more sophisticated methods than the one used here, in terms of both classification [9] and resistance to falsification [3]. However, the main aim was to assess the discriminatory abilities of selected features and the impact of phase II on the obtained results. Next step was followed by the verification of the choice made by the 5NN classifier, based on anthropometric measurements provided by Kinect. It should be emphasized that in the second phase it was not possible to change the decision, only to accept or deny it. The measured values were compared with the models of those parameters that were stored in the database for the candidate user. If the value of the parameter measured was within \(\pm \) 5 % of the value stored in the database, it was treated as a vote for acceptance of the selection. Otherwise, it was treated as a voice for the negation of the 5NN choice. The decision was taken in accordance with the choice of the majority.

5 Result and Discussion

The measurements were made in the Bialystok University of Technology on a group of 31 subjects (14 men and 17 women) with the use of the two Kistler force plates and a Kinect device at the same time. The persons taking part in the research were at ages 21.93 \(\pm \) 0.92, BW: 69.54 \(\pm \) 15.10 kg and BH 170.95 \(\pm \) 9.56 cm. It should be noted that the subjects were a rather homogeneous group, characterized by similar physical form and the lack of serious illnesses or injuries which could permanently affect the change in the motion pattern. During the research subjects walked freely in their own sports shoes along the measurement path in which two force plates were hidden. In the distance of 2 m from the measurement path Kinect device was hidden which captured and registered the movement of the subject in the sagittal plane. The gait sequences were repeated several times so a little more than 600 gait cycles were recorded. 224 strides from 28 users (8 strides per each user) were stored in the database and used as so called prototype points. Three subjects (30 strides) were treated as impostors who wanted to falsify the biometric system and obtain an access to protected resources. Their data were used for testing jointly with the rest of users data. The whole set for testing contained of 382 strides (gait cycles). As a result of the operation of 5NN classifier in 5 cases (1.20 %) incorrect recognition (False Acceptance Rate - FAR) was obtained in users group. In the case of these mistakes the classifier most frequently confuses people of different sexes, however, these are often people of similar body weight (BW). As many as three cases concern the confusion between the same people. Of course 5NN classifier is 100 % wrong (FAR) in the case of persons whose patterns are not present in the database (impostors). Here as well, most people confused with each other have a similar BW. This observation is not surprising because, as stated earlier, the mass of the subject affects the values of the GRF. We should also mention here the work [22] which demonstrated, that the normalization of the GRF to BW significantly reduces the quality of recognition.
Fig. 5.

Histogram with voting results for: (a) users, (b) impostors

Voting results at the second stage of identification are shown in Fig. 5. It is easy to notice that, with the case of users, in general, the biometric system does not have any problems with the confirmation of the selection it has made earlier (at the first stage). In more than half cases, all measured anthropometric parameters fit the set limit of 5 % (Fig. 5a). In the case of impostors (Fig. 5b), the presented biometric system captures differences in selected anthropometric parameters, without any major problems. However, only in a few cases (3) all the parameters differ by more than 5 % of the data stored in the database. There was not such a case, in which the number of votes was 5. Most often the above mentioned differences appear here for data 3 or 4. To sum up, this reflects a well-chosen value of tolerance with respect to the accuracy of the Kinect system. In relation to the group of users, the second stage reduces the FAR rejecting 2 of incorrectly diagnosed cases (FAR =0.85 %). Now, all incorrectly diagnosed gait cycles belong to the same person. The mistake occurring here refers to the persons of similar weight (64.4 kg and 63.2 kg) and body height (167.1 cm and 164.7 cm). Unfortunately, as a result of the operation of comparator a few earlier correctly diagnosed cases have been rejected (exactly 14). Essentially, the False Rejected Rate (FRR) reaches the level of 4.55 % (Table 1). In the case of impostors it has been possible to reject 26 cases previously classified by 5NN. The remaining falsely accepted strides belong to one person (108 kg and 181.6 cm) and are mistaken for another person with a mass of 90.1 kg and body height 188.2 cm. A deeper analysis of these studies clearly show that the misclassification of these people at the first stage is not a result of a close resemblance between them, but rather the fact that people have the highest body weights amongst all of the examined. Hence, it has been difficult to find more similar person for 5NN classifier (when using 5NN classifier). Unfortunately, also in the case of anthropometric parameters it is possible to notice a certain resemblance between these people. This resemblance along with the measuring error of the Kinect system causes the incorrect acceptance of a few strides.
Table 1.

FAR, FRR for users and impostors

6 Conclusions

The proposed bimodal biometric system works on the basis of the two kinds of signals: dynamic GRF and static the length parameters of the subjects bodies. The performed experiment shows the potential of human gait as biometrics and on the other hand the superiority of systems based on fusion over standard methods of human gait recognition. In work [11], the proposed method based on full-body motion and anthropometric biometrics has an average ROC EER (Receiver Operating Characteristic Equal Error Rate) of 13 % and an average Cumulative Match Curve Rank-1 identification rate of 90 % ROC. In this work we have achieved comparable results (with Impostors). The authors of presented article are aware of some deficiencies of the presented research. It is necessary to extend the number of investigated persons which could increase the potential application capabilities of the system. In addition, more sophisticated classification algorithms should be used to reduce the level of recognition errors at the first stage of the operation. The use of the second Kinect device set in the frontal plane of a walking subject would be worth considering, as it would improve the accuracy of the used anthropometric parameters.

References

  1. 1.
    Bobick, A.F., Johnson, A.Y.: Gait recognition using static, activity-specific parameters. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, vol. 1, p. I-423 (2001)Google Scholar
  2. 2.
    Boulgouris, N.V., Hatzinakos, D., Plataniotis, K.N.: Gait recognition: a challenging signal processing technology for biometric identification. IEEE Signal Process. Mag. 22(6), 78–90 (2005)CrossRefGoogle Scholar
  3. 3.
    Derlatka, M.: Modified kNN algorithm for improved recognition accuracy of biometrics system based on gait. In: Saeed, K., Chaki, R., Cortesi, A., Wierzchoń, S. (eds.) CISIM 2013. LNCS, vol. 8104, pp. 59–66. Springer, Heidelberg (2013) CrossRefGoogle Scholar
  4. 4.
    Gafurov, D., Bours, P., Snekkenes, E.: User authentication based on foot motion. SIViP 5(4), 457–467 (2011)CrossRefGoogle Scholar
  5. 5.
    Gudavalli, M., Babu, A.V, Raju, S.V, Kumar, D.S.: Multimodal biometrics-sources, architecture and fusion techniques: an overview. In: IEEE International Symposium on Biometrics and Security Technologies (ISBAST), pp. 27–34 (2012)Google Scholar
  6. 6.
    Goodwin, P.: The holt-winters approach to exponential smoothing: 50 years old and going strong. In: FORESIGHT Fall, p. 3034 (2010)Google Scholar
  7. 7.
    Lee, B., Hong, S., Lee, H., Kim, E.: Gait recognition system using decision-level fusion. In: 2010 5th IEEE Conference on Industrial Electronics and Applications (ICIEA), pp. 313–316 (2010)Google Scholar
  8. 8.
    Lee, L., Grimson, W.E.L.: Gait analysis for recognition and classification. In: Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 148–155 (2002)Google Scholar
  9. 9.
    Lu, J., Zhang, E.: Gait recognition for human identification based on ICA and fuzzy SVM through multiple views fusion. Pattern Recognit. Lett. 28(16), 2401–2411 (2007)CrossRefGoogle Scholar
  10. 10.
    Milovanovic, M., Minovic, M., Starcevic, D.: Walking in colors: human gait recognition using Kinect and CBIR. IEEE Multimedia 20(4), 28–36 (2013)CrossRefGoogle Scholar
  11. 11.
    Munsell, B.C., Temlyakov, A., Qu, C., Wang, S.: Person identification using full-body motion and anthropometric biometrics from kinect videos. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012 Ws/Demos, Part III. LNCS, vol. 7585, pp. 91–100. Springer, Heidelberg (2012) Google Scholar
  12. 12.
    Poh, N., Ross, A., Lee, W., Kittler, J.: A user-specific and selective multimodal biometric fusion strategy by ranking subjects. Pattern Recognit. 46(12), 3341–3357 (2013)CrossRefGoogle Scholar
  13. 13.
    Preis, J., Kessel, M., Werner, M., Linnhoff-Popien, C.: Gait recognition with kinect. In: 1st International Workshop on Kinect in Pervasive Computing (2012)Google Scholar
  14. 14.
    Proenca, H., Alexandre, L.A.: Toward covert iris biometric recognition: experimental results from the nice contests. IEEE Trans. Inf. Forensics Secur. 7(2), 798–808 (2012)CrossRefGoogle Scholar
  15. 15.
    Sharma, S., Shukla, A., Tiwari, R., Singh, V.: View variations effect in gait recognition and performance improvement using fusion. In: 1st IEEE International Conference on Recent Advances in Information Technology (RAIT), pp. 892–896 (2012)Google Scholar
  16. 16.
    Shum, H.P.H., Ho, E.S.L., Jiang, Y., Takagi, S.: Real-time posture reconstruction for microsoft kinect. IEEE Trans. Cybern. 43(5), 1357–1369 (2013)CrossRefGoogle Scholar
  17. 17.
    Sinha, A., Chakravarty, K., Bhowmick, B.: Person identification using skeleton information from Kinect. In: The Sixth International Conference on Advances in Computer-Human Interactions ACHI 2013, Nice, France, pp. 101–108 (2013)Google Scholar
  18. 18.
    Veres, G.V., Nixon, M.S., Middleton, L., Carter, J.N.: Fusion of dynamic and static features for gait recognition over time. In: 2005 8th IEEE International Conference on Information Fusion, vol. 2, p. 7 (2005)Google Scholar
  19. 19.
    Wang, L., Ning, H., Tan, T., Hu, W.: Fusion of static and dynamic body biometrics for gait recognition. IEEE Trans. Circuits Syst. Video Technol. 14(2), 149–158 (2004)CrossRefGoogle Scholar
  20. 20.
    Wang, X., Mueen, A., Ding, H., Trajcevski, G., Scheuermann, P., Keogh, E.: Experimental comparison of representation methods and distance measures for time series data. Data Min. Knowl. Disc. 26(2), 275–309 (2013)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Yao, Y.F., Jing, X.Y., Wong, H.S.: Face and palmprint feature level fusion for single sample biometrics recognition. Neurocomputing 70(7), 1582–1586 (2007)CrossRefGoogle Scholar
  22. 22.
    Yao, Z.M, Zhou, X., Lin, E.D, Xu, S., Sun, Y.N.: A novel biometric recognition system based on ground reaction force measurements of continuous gait. In: 2010 3rd IEEE Conference on Human System Interactions (HSI), pp. 452–458 (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Faculty of Mechanical EngineeringBialystok University of TechnologyBialystokPoland

Personalised recommendations