Personal and Ubiquitous Computing

, Volume 23, Issue 5–6, pp 739–748 | Cite as

Investigating large curved interaction devices

  • Andreas BraunEmail author
  • Sebastian Zander-Walz
  • Martin Majewski
  • Arjan Kuijper
Original Article


Large interactive surfaces enable novel forms of interaction for their users, particularly in terms of collaborative interaction. During longer interactions, the ergonomic factors of interaction systems have to be taken into consideration. Using the full interaction space may require considerable motion of the arms and upper body over a prolonged period of time, potentially causing fatigue. In this work, we present Curved, a large-surface interaction device, whose shape is designed based on the natural movement of an outstretched arm. It is able to track one or two hands above or on its surface by using 32 capacitive proximity sensors. Supporting both touch and mid-air interaction can enable more versatile modes of use. We use image processing methods for tracking the user’s hands and classify gestures based on their motion. Virtual reality is a potential use case for such interaction systems and was chosen for our demonstration application. We conducted a study with ten users to test the gesture tracking performance, as well as user experience and user preference for the adjustable system parameters.


Curved surfaces Capacitive sensing Gestural interaction Virtual reality 



We would like to thank Stefan Krepp, Steeven Zeiss, Joachim Loge, and Maxim Djakow for their input to the hardware and software development, as well as our study participants for their valuable comments.

Funding information

This work was partially supported by EC Grant Agreement No. 610840.


  1. 1.
    Annett M, Grossman T, Wigdor D, Fitzmaurice G (2011) Medusa: a proximity-aware multi-touch tabletop. In: Proceedings UIST. ACM, pp 337–346Google Scholar
  2. 2.
    Bachynskyi M, Oulasvirta A, Palmas G, Weinkauf T (2014) Is motion capture-based biomechanical simulation valid for HCI studies?: study and implications. In: Proceedings of the 32nd annual ACM conference on human factors in computing systems. ACM, pp 3215–3224Google Scholar
  3. 3.
    Bachynskyi M, Palmas G, Oulasvirta A, Steimle J, Weinkauf T (2015) Performance and ergonomics of touch surfaces: a comparative study using biomechanical simulationGoogle Scholar
  4. 4.
    Benko H, Jota R, Wilson A (2012) Miragetable: freehand interaction on a projected augmented reality tabletop. In: Proceedings CHI. ACM, pp 199–208Google Scholar
  5. 5.
    Braun A, Hamisu P (2011) Designing a multi-purpose capacitive proximity sensing input device. In: Proceedings PETRA, p. Article No. 15.
  6. 6.
    Braun A, Krepp S, Kuijper A (2015) Acoustic tracking of hand activities on surfaces. In: Proceedings iWOAR, WOAR ’15. ACM, New York, pp 9:1–9:5.
  7. 7.
    Braun A, Wichert R, Kuijper A, Fellner DW (2015) Capacitive proximity sensing in smart environments. J Ambient Intell Smart Environ 7(4):1–28. CrossRefGoogle Scholar
  8. 8.
    Braun A, Zander-Walz S, Krepp S, Rus S, Wichert R, Kuijper A (2016) CapTap: combining capacitive gesture recognition and acoustic touch detection. In: Proceedings of the 3rd international workshop on sensor-based activity recognition and interaction. ACM, p 6Google Scholar
  9. 9.
    Braun A, Zander-Walz S, Majewski M, Kuijper A (2017) Curved—free-form interaction using capacitive proximity sensors. Procedia Comput Sci 109:59–66. CrossRefGoogle Scholar
  10. 10.
    Earnshaw RA (2014) Virtual reality systems. Academic, New YorkGoogle Scholar
  11. 11.
    Faulkner L (2003) Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods 35(3):379–383CrossRefGoogle Scholar
  12. 12.
    Grosse-Puppendahl T, Berghoefer Y, Braun A, Wimmer R, Kuijper A (2013) OpenCapSense: a rapid prototyping toolkit for pervasive interaction using capacitive sensing. In: Proceedings PerCom, pp 152–159.
  13. 13.
    Grosse-Puppendahl T, Braun A, Kamieth F, Kuijper A (2013) Swiss-cheese extended: an object recognition method for ubiquitous interfaces based on capacitive proximity sensing. In: Proceedings of the 2013 ACM annual conference on human factors in computing systems. ACM, pp 1401–1410Google Scholar
  14. 14.
    Grossman T, Wigdor D, Balakrishnan R (2004) Multi-finger gestural interaction with 3D volumetric displays. In: Proceedings UIST. ACM, pp 61–70.
  15. 15.
    Hu MK (1962) Visual pattern recognition by moment invariants. IRE Trans Inf Theory 8(2):179–187CrossRefGoogle Scholar
  16. 16.
    Le Goc M, Taylor S, Izadi S, Keskin C et al (2014) A low-cost transparent electric field sensor for 3D interaction on mobile devices. In: Proceedings CHI, pp 3167–3170Google Scholar
  17. 17.
    Ma D, Fan X, Gausemeier J, Grafe M (2011) Virtual reality & augmented reality in industry, Springer, Berlin.
  18. 18.
    Rekimoto J (2002) SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. In: Proceedings CHI. ACM, pp 113–120Google Scholar
  19. 19.
    Ren Z, Meng J, Yuan J, Zhang Z (2011) Robust hand gesture recognition with kinect sensor. In: Multimedia. ACM Press, New York, p 759.
  20. 20.
    Roudaut A, Pohl H, Baudisch P (2011) Touch input on curved surfaces. In: Proceedings CHI. ACM, pp 1011–1020Google Scholar
  21. 21.
    Samsung Inc. (2014) CES 2014: Samsung unveils first curved ultra high definition (UHD) TVsGoogle Scholar
  22. 22.
    Schlömer T, Poppinga B, Henze N, Boll S (2008) Gesture recognition with a Wii controller. In: Proceedings of the 2nd international conference on tangible and embedded interaction. ACM, pp 11–14Google Scholar
  23. 23.
    Smith J, White T, Dodge C, Paradiso J, Gershenfeld N, Allport D (1998) Electric field sensing for graphical interfaces. IEEE Comput Graph Appl 18(3):54–60CrossRefGoogle Scholar
  24. 24.
    University of Groningen (2011) Reality touch theatre.
  25. 25.
    Weichert F, Bachmann D, Rudak B, Fisseler D (2013) Analysis of the accuracy and robustness of the leap motion controller. Sensors 13(5):6380–6393CrossRefGoogle Scholar
  26. 26.
    Wimmer R, Baudisch P (2011) Modular and deformable touch-sensitive surfaces based on time domain reflectometry. In: Proceedings UIST. ACM, pp 517–526Google Scholar
  27. 27.
    Wimmer R, Kranz M, Boring S, Schmidt A (2007) Captable and capshelf-unobtrusive activity recognition using networked capacitive sensors. In: INSS. IEEE, pp 85–88Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  1. 1.Fraunhofer Institute for Computer Graphics Research IGDDarmstadtGermany

Personalised recommendations