Geometry and Gesture-Based Features from Saccadic Eye-Movement as a Biometric in Radiology

  • Folami T. AlamudunEmail author
  • Tracy Hammond
  • Hong-Jun Yoon
  • Georgia D. Tourassi
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10284)


In this study, we present a novel application of sketch gesture recognition on eye-movement for biometric identification and estimating task expertise. The study was performed for the task of mammographic screening with simultaneous viewing of four coordinated breast views as typically done in clinical practice. Eye-tracking data and diagnostic decisions collected for 100 mammographic cases (25 normal, 25 benign, 50 malignant) and 10 readers (three board certified radiologists and seven radiology residents), formed the corpus for this study. Sketch gesture recognition techniques were employed to extract geometric and gesture-based features from saccadic eye-movements. Our results show that saccadic eye-movement, characterized using sketch-based features, result in more accurate models for predicting individual identity and level of expertise than more traditional eye-tracking features.


Eye-tracking Biometrics Sketch recognition Mammography 



This material is based upon work supported by the U.S. Department of Energy and the Office of Science under contract number DE-AC05-00OR22725. The authors also thank Kathleen B. Hudson, MD, and Garnetta Morin-Ducote, MD for contributions during data collection.


  1. 1.
    Alamudun, F., Yoon, H.J., Hudson, K.B., Morin-Ducote, G., Hammond, T., Tourassi, G.D.: Fractal analysis of visual search activity for mass detection during mammographic screening. Med. Phys. (2017).
  2. 2.
    Bartley, J., Forsyth, J., Pendse, P., Xin, D., Brown, G., Hagseth, P., Agrawal, A., Goldberg, D.W., Hammond, T.: World of workout: a contextual mobile RPG to encourage long term fitness. In: Proceedings of the 2nd ACM SIGSPATIAL International Workshop on the Use of GIS in Public Health, pp. 60–67. ACM (2013)Google Scholar
  3. 3.
    Bednarik, R., Kinnunen, T., Mihaila, A., Fränti, P.: Eye-movements as a biometric. In: Kalviainen, H., Parkkinen, J., Kaarna, A. (eds.) SCIA 2005. LNCS, vol. 3540, pp. 780–789. Springer, Heidelberg (2005). doi: 10.1007/11499145_79 CrossRefGoogle Scholar
  4. 4.
    Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)CrossRefzbMATHGoogle Scholar
  5. 5.
    Chi, M.T., Glaser, R., Farr, M.J.: The Nature of Expertise. Psychology Press, Park Drive (2014)zbMATHGoogle Scholar
  6. 6.
    Çığ, Ç., Sezgin, T.M.: Gaze-based prediction of pen-based virtual interaction tasks. Int. J. Hum.-Comput. Stud. 73, 91–106 (2015). CrossRefGoogle Scholar
  7. 7.
    Corey, P., Hammond, T.: Gladder: combining gesture and geometric sketch recognition. In: Proceedings of the 23rd National Conference on Artificial Intelligence, vol. 3, pp. 1788–1789. AAAI Press (2008)Google Scholar
  8. 8.
    Deravi, F., Guness, S.P.: Gaze trajectory as a biometric modality. In: BIOSIGNALS, pp. 335–341 (2011)Google Scholar
  9. 9.
    Dixon, D., Prasad, M., Hammond, T.: iCanDraw: using sketch recognition and corrective feedback to assist a user in drawing human faces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 897–906. ACM (2010)Google Scholar
  10. 10.
    Eoff, B.D., Hammond, T.: Who dotted that‘i’?: context free user differentiation through pressure and tilt pen data. In: Proceedings of Graphics Interface 2009, pp. 149–156. Canadian Information Processing Society (2009)Google Scholar
  11. 11.
    Galdi, C., Nappi, M., Riccio, D., Cantoni, V., Porta, M.: A new gaze analysis based soft-biometric. In: Carrasco-Ochoa, J.A., Martínez-Trinidad, J.F., Rodríguez, J.S., Baja, G.S. (eds.) MCPR 2013. LNCS, vol. 7914, pp. 136–144. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-38989-4_14 CrossRefGoogle Scholar
  12. 12.
    Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The weka data mining software: an update. ACM SIGKDD Explor. Newsl. 11(1), 10–18 (2009)CrossRefGoogle Scholar
  13. 13.
    Hammond, T., Gajos, K., Davis, R., Shrobe, H.E.: An agent-based system for capturing and indexing software design meetings. In: Proceedings of the International Workshop on Agents in Design (WAID 2002). Citeseer (2002)Google Scholar
  14. 14.
    Hammond, T., O’Sullivan, B.: Recognizing freeform hand-sketched constraint network diagrams by combining geometry and context. In: Proceedings of the Eurographics Ireland (2007)Google Scholar
  15. 15.
    Hammond, T., Prasad, M., Dixon, D.: Art 101: learning to draw through sketch recognition. In: Taylor, R., Boulanger, P., Krüger, A., Olivier, P. (eds.) SG 2010. LNCS, vol. 6133, pp. 277–280. Springer, Heidelberg (2010). doi: 10.1007/978-3-642-13544-6_30 CrossRefGoogle Scholar
  16. 16.
    Hammond, T.A.: Ladder: A perceptually-based language to simplify sketch recognition user interface development. Ph.D. thesis, Massachusetts Institute of Technology (2007)Google Scholar
  17. 17.
    Heath, M., Bowyer, K., Kopans, D., Kegelmeyer Jr., P., Moore, R., Chang, K., Munishkumaran, S.: Current status of the digital database for screening mammography. In: Karssemeijer, N., Thijssen, M., Hendriks, J., van Erning, L. (eds.) Digital Mammography, vol. 13, pp. 457–460. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  18. 18.
    Holland, C., Komogortsev, O.: Biometric identification via eye movement scanpaths in reading. In: 2011 International Joint Conference on Biometrics (IJCB), pp. 1–8, October 2011Google Scholar
  19. 19.
    Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., Van de Weijer, J.: Eye Tracking: A Comprehensive Guide to Methods and Measures. Oxford University Press, Oxford (2011)Google Scholar
  20. 20.
    Kara, L.B., Stahovich, T.F.: An image-based, trainable symbol recognizer for hand-drawn sketches. Comput. Graph. 29(4), 501–517 (2005). CrossRefGoogle Scholar
  21. 21.
    Kasprowski, P., Ober, J.: Eye movements in biometrics. In: Maltoni, D., Jain, A.K. (eds.) BioAW 2004. LNCS, vol. 3087, pp. 248–258. Springer, Heidelberg (2004). doi: 10.1007/978-3-540-25976-3_23 CrossRefGoogle Scholar
  22. 22.
    Krupinski, E.A.: Influence of experience on scanning strategies in mammography. Proc. SPIE Med. Imaging 2712, 95–101 (1996). International Society for Optics and PhotonicsCrossRefGoogle Scholar
  23. 23.
    Krupinski, E.A., Graham, A.R., Weinstein, R.S.: Characterizing the development of visual search expertise in pathology residents viewing whole slide images. Hum. Pathol. 44(3), 357–364 (2013)CrossRefGoogle Scholar
  24. 24.
    Krupinski, E.A., Tillack, A.A., Richter, L., Henderson, J.T., Bhattacharyya, A.K., Scott, K.M., Graham, A.R., Descour, M.R., Davis, J.R., Weinstein, R.S.: Eye-movement study and human performance using telepathology virtual slides. Implications for medical education and differences with experience. Hum. Pathol. 37(12), 1543–1556 (2006). CrossRefGoogle Scholar
  25. 25.
    Kundel, H.L., Paul, S., La Follette, J.: Visual search patterns and experience with radiological images. Radiology 103(3), 523–528 (1972). pMID: 5022947. CrossRefGoogle Scholar
  26. 26.
    Li, W., Hammond, T.: Recognizing text through sound alone. In: Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2011, pp. 1481–1486. AAAI Press (2011).
  27. 27.
    Long Jr., A.C., Landay, J.A., Rowe, L.A., Michiels, J.: Visual similarity of pen gestures. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2000, pp. 360–367. ACM, New York (2000).
  28. 28.
    Lynch, P.J.: Eye movements abductors, December 2006.
  29. 29.
    Miller, J., Hammond, T.: Wiiolin: a virtual instrument using the wii remote. In: Proceedings of the 2010 Conference on New Interfaces for Musical Expression (NIME), pp. 497–500 (2010)Google Scholar
  30. 30.
    Nodine, C.F., Kundel, H.L., Mello-Thoms, C., Weinstein, S.P., Orel, S.G., Sullivan, D.C., Conant, E.F.: How experience and training influence mammography expertise. Acad. Radiol. 6(10), 575–585 (1999). CrossRefGoogle Scholar
  31. 31.
    Noton, D., Stark, L.: Scanpaths in eye movements during pattern perception. Science 171(3968), 308–311 (1971). CrossRefGoogle Scholar
  32. 32.
    Ouyang, T.Y., Davis, R.: A visual approach to sketched symbol recognition. In: Proceedings of the 21st International Joint Conference on Artificial Intelligence, IJCAI 2009, pp. 1463–1468. Morgan Kaufmann Publishers Inc., San Francisco (2009).
  33. 33.
    Paulson, B., Cummings, D., Hammond, T.: Object interaction detection using hand posture cues in an office setting. Int. J. Hum.-Comput. Stud. 69(1), 19–29 (2011)CrossRefGoogle Scholar
  34. 34.
    Paulson, B., Hammond, T.: Paleosketch: accurate primitive sketch recognition and beautification. In: Proceedings of the 13th International Conference on Intelligent User Interfaces, IUI 2008, pp. 1–10. ACM, New York (2008).
  35. 35.
    Paulson, B., Rajan, P., Davalos, P., Gutierrez-Osuna, R., Hammond, T.: What!?! no rubine features?: using geometric-based features to produce normalized confidence values for sketch recognition. In: HCC Workshop: Sketch Tools for Diagramming (VL/HCC), pp. 57–63 (2008)Google Scholar
  36. 36.
    Pramanik, S., Bhattacharjee, D.: Geometric feature based face-sketch recognition. In: 2012 International Conference on Pattern Recognition, Informatics and Medical Engineering (PRIME), pp. 409–415, March 2012Google Scholar
  37. 37.
    American College of Radiology: BI-RADS Committee. Breast imaging reporting and data system. American College of Radiology (1998)Google Scholar
  38. 38.
    Rajan, P., Hammond, T.: From paper to machine: extracting strokes from images for use in sketch recognition. In: Proceedings of the 5th Eurographics Conference on Sketch-Based Interfaces and Modeling, pp. 41–48. Eurographics Association (2008)Google Scholar
  39. 39.
    Rayner, K.: Eye movements in reading and information processing: 20 years of research. Psychol. Bull. 124(3), 372 (1998)CrossRefGoogle Scholar
  40. 40.
    Rubine, D.: Specifying gestures by example. SIGGRAPH Comput. Graph. 25(4), 329–337 (1991). CrossRefGoogle Scholar
  41. 41.
    Schnitzer, B.S., Kowler, E.: Eye movements during multiple readings of the same text. Vis. Res. 46(10), 1611–1632 (2006). CrossRefGoogle Scholar
  42. 42.
    Smith, R.A., Cokkinides, V., von Eschenbach, A.C., Levin, B., Cohen, C., Runowicz, C.D., Sener, S., Saslow, D., Eyre, H.J.: American cancer society guidelines for the early detection of cancer. CA Cancer J. Clin. 52(1), 8–22 (2002)CrossRefGoogle Scholar
  43. 43.
    Taele, P., Peschel, J., Hammond, T.: A sketch interactive approach to computer-assisted biology instruction. In: Proceedings of IUI 2009 Workshop on Sketch Recognition (2009)Google Scholar
  44. 44.
    Tweed, D., Vilis, T.: Geometric relations of eye position and velocity vectors during saccades. Vis. Res. 30(1), 111–127 (1990). CrossRefGoogle Scholar
  45. 45.
    Valentine, S., Vides, F., Lucchese, G., Turner, D., Kim, H.H., Li, W., Linsey, J., Hammond, T.: Mechanix: a sketch-based tutoring system for statics courses. In: Proceedings of the 24th Innovative Applications of Artificial Intelligence Conference on Artificial Intelligence (IAAI), pp. 2253–2260. AAAI (2012)Google Scholar
  46. 46.
    Yoon, H.J., Carmichael, T.R., Tourassi, G.: Gaze as a biometric. In: Proceedings of SPIE, pp. 903707–903707 (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Folami T. Alamudun
    • 1
    • 2
    Email author
  • Tracy Hammond
    • 1
    • 2
  • Hong-Jun Yoon
    • 1
  • Georgia D. Tourassi
    • 1
  1. 1.Biomedical Sciences, Engineering, and Computing GroupOak Ridge National LaboratoryOak RidgeUSA
  2. 2.Sketch Recognition Lab, Department of Computer Science and EngineeringTexas A&M UniversityCollege StationUSA

Personalised recommendations