Interactive Gestures for Liver Angiography Operation

  • Dina A. ElmanakhlyEmail author
  • Ayman Atia
  • Essam A. Rashed
  • Mostafa-Samy M. Mostafa
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9745)


The main challenge of creating large interactive displays in the operating rooms (ORs) is in the definition of ways that are efficient and easy to learn for the physician. Apart from traditional input methods such as mouse and keyboard, we have developed a multimodal system with two different vision based human-computer interaction (HCI) systems that can simplify the way surgeons interact with the medical images shown on the LCD display. The purpose of this work is to construct a gesture recognition system with a fast, accurate, and easily attainable method. The first system is a laser pointer interaction framework that supports a 2D stroke gesture interface. The recorded laser gestures are recognized using two different algorithms: dynamic time warping (DTW) and one dollar (1$) recognizer. Our experimental results showed that the DTW algorithm performs better with an overall accuracy of 90 %. The second prototype presents an intuitive HCI to manipulate images using freehand gestures. In order to strengthen the gesture recognition process, the system incorporates contextual information to determine the intent of the user of interacting with the large display. Two cameras are used to observe the surgeon’s hand movements to continuously determine and monitor what the surgeon intends to perform. Experimental results showed that the system accuracy is 95 % for recognition with the effect of contextual integration.


Gesture recognition Laser pointers Hand gestures 


  1. 1.
    Jafari, R., Ziou, D.: Eye-gaze estimation under various head positions and iris states. Expert Syst. Appl. 42(1), 510–518 (2015)CrossRefGoogle Scholar
  2. 2.
    Munteanu, C., Jones, M., Whittaker, S., Oviatt, S., Aylett, M., Penn, G., Brewster, S., Alessandro, N.: Designing speech and language interactions. In: CHI 2014 Extended Abstracts on Human Factors in Computing Systems, Toronto, Canada (2014)Google Scholar
  3. 3.
    Gallo, L., Placitelli, A. P., Ciampi, M.: Controller-free exploration of medical image data: experiencing the kinect. In: Computer-Based Medical Systems (CBMS), Bristol, United Kingdom (2011)Google Scholar
  4. 4.
    Alapetite, A., Andersen, H.B., Hertzum, M.: Acceptance of speech recognition by physicians: a survey of expectations, experiences, and social influence. Inter. J. Hum. Comput. Stud. 67(1), 36–49 (2009)CrossRefGoogle Scholar
  5. 5.
    Yoshimitsu, K., Muragaki, Y., Maruyama, T., Yamato, M., Iseki, H.: Development and initial clinical testing of opect: an innovative device for fully intangible control of the intraoperative image-displaying monitor by the surgeon. Oper. Neurosurg. 10(1), 46–50 (2014)CrossRefGoogle Scholar
  6. 6.
    Myers, C., Rabiner, L.: A level building dynamic time warping algorithm for connected word recognition. IEEE Trans. Acoust. Speech Sig. Proc. 29(2), 284–297 (1981)CrossRefzbMATHGoogle Scholar
  7. 7.
    Wobbrock, J. O., Wilson, A. D., Li, Y.: Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In: Proceedings of the 20th Annual Symposium on User Interface Software and Technology, New York, USA (2007)Google Scholar
  8. 8.
    Bradski, G.R.: Real time face and object tracking as a component of a perceptual user interface. In: Proceedings of IEEE Workshop on Applications of Computer Vision, Princeton, NJ (1998)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Dina A. Elmanakhly
    • 2
    Email author
  • Ayman Atia
    • 1
  • Essam A. Rashed
    • 2
  • Mostafa-Samy M. Mostafa
    • 1
  1. 1.HCI-LAB, Department of CS, Faculty of Computers and InformationHelwan UniversityHelwanEgypt
  2. 2.Image Science-LAB, Department of Mathematics, Faculty of ScienceSuez Canal UniversityIsmailiaEgypt

Personalised recommendations