Advertisement

InvisiShapes: A Recognition System for Sketched 3D Primitives in Continuous Interaction Spaces

  • Paul TaeleEmail author
  • Tracy Hammond
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9317)

Abstract

Continued improvements and rising ubiquity in touchscreen and motion-sensing technologies enable users to leverage mid-air input modalities for intelligent surface sketching into the third dimension. However, existing approaches largely either focus on constrained 3D gesture sets, require specialized hardware setups, or do not deviate beyond surface sketching assumptions. We present InvisiShapes, a recognition system for users to sketch 3D geometric primitives in continuous interaction spaces that explore surfaces and mid-air environments. Our system leverages a collection of sketch and gesture recognition techniques and heuristics and takes advantage of easily accessible computing hardware for users to incorporate depth to their sketches. From our interaction study and user evaluations, we observed that our system successfully accomplishes strong recognition and intuitive interaction capabilities on collected sketch+motion data and interactive sketching scenarios, respectively.

Keywords

Sketch recognition Gesture recognition Continuous interaction spaces 3d drawing Mid-air interaction 

References

  1. 1.
    Anthony, L., Wobbrock, J.O.: $N-protractor: a fast and accurate multistroke recognizer. In: Proceedings of Graphics Interface 2012. Canadian Information Processing Society, Toronto (2012)Google Scholar
  2. 2.
    Cheema, S., Gulwani, S., LaViola, J.: QuickDraw: improving drawing experience for geometric diagrams. In: 2012 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York (2012)Google Scholar
  3. 3.
    De Araùjo, B.R., Casiez, G., Jorge, J.A.: Mockup builder: direct 3d modeling on and above the surface in a continuous interaction space. In: Proceedings of Graphics Interface 2012, pp. 173–180. Canadian Information Processing Society, Toronto (2012)Google Scholar
  4. 4.
    Hammond, T., Davis, R.: LADDER, a sketching language for user interface developers. Comput. Graph. 29, 518–532 (2005)CrossRefGoogle Scholar
  5. 5.
    Hoffman, M., Varcholik, P., LaViola Jr., J.J.: Breaking the status quo: improving 3D gesture recognition with spatially convenient input devices. In: Proceedings of the 2010 IEEE Virtual Reality Conference. IEEE Computer Society, Washington (2010)Google Scholar
  6. 6.
    Kratz, S., Rohs, M.: A $3 gesture recognizer: simple gesture recognition for devices equipped with 3D acceleration sensors In: Proceedings of the 15th International Conference on Intelligent User Interfaces. ACM, New York (2010)Google Scholar
  7. 7.
    Kratz, S., Rohs, M.: Protractor3D: a closed-form solution to rotation-invariant 3D gestures In: Proceedings of the 16th International Conference on Intelligent User Interfaces. ACM, New York (2011)Google Scholar
  8. 8.
    Kratz, S., Rohs, M., Essl, G.: Combining acceleration and gyroscope data for motion gesture recognition using classifiers with dimensionality constraints. In: Proceedings of the 18th International Conference on Intelligent User Interfaces. ACM, New York (2013)Google Scholar
  9. 9.
    Marquardt, N., Jota, R., Greenberg, S., Jorge, J.A.: The continuous interaction space: interaction techniques unifying touch and gesture on and above a digital surface. In: Campos, P., Graham, N., Jorge, J., Nunes, N., Palanque, P., Winckler, M. (eds.) INTERACT 2011. LNCS, vol. 6948, pp. 461–476. Springer, Heidelberg (2011). doi: 10.1007/978-3-642-23765-2_32 CrossRefGoogle Scholar
  10. 10.
    Paulson, B., Hammond, T.: PaleoSketch: accurate primitive sketch recognition and beautification. In: Proceedings of the 13th International Conference on Intelligent User Interfaces, pp. 1–10. ACM, New York (2008)Google Scholar
  11. 11.
    Spindler, M., Martsch, M., Dachselt, R.: Going beyond the surface: studying multi-layer interaction above the tabletop. In: 2012 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1277–1286. ACM, New York (2012)Google Scholar
  12. 12.
    Steins, C., Gustafson, S., Holz, C., Baudisch, P.: Imaginary devices: gesture-based interaction mimicking traditional input devices. In: Proceedings of the 15th International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 123–126. ACM, New York (2013)Google Scholar
  13. 13.
    Subramanian, S., Aliakseyeu, D., Lucero, A.: Multi-layer interaction for digital tables. In: Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology. ACM, New York (2006)Google Scholar
  14. 14.
    Taele, P., Hammond, T.: Initial approaches for extending sketch recognition to beyond-surface environments. In: Proceedings of the 2012 ACM Annual Conference Extended Abstracts on Human Factors in Computing Systems Extended Abstracts, pp. 2039–2044. ACM, New York (2012)Google Scholar
  15. 15.
    Taele, P., Hammond, T.: Developing sketch recognition and interaction techniques for intelligent surfaceless sketching user interfaces. In: Proceedings of the Companion Publication of the 19th International Conference on Intelligent User Interfaces, pp. 53–56. ACM, New York (2014)Google Scholar
  16. 16.
    van Sommers, P.: Drawing and Cognition: Descriptive and Experimental Studies of Graphic Production Processes. Cambridge University Press, Cambridge (1984)CrossRefGoogle Scholar
  17. 17.
    Wolin, A., Eoff, B., Hammond, T.: ShortStraw: a simple and eective corner finder for polylines. In: Proceedings of the Fifth Eurographics Conference on Sketch-Based Interfaces and Modeling, pp. 33–40. ACM, New York (2008)Google Scholar
  18. 18.
    Xiong, Y., LaViola Jr., J.J.: A ShortStraw-based algorithm for corner finding in sketch-based interfaces. Comput. Graph. 34(5), 513–527 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Sketch Recognition LabTexas A&M UniversityCollege StationUSA

Personalised recommendations