Learning to Assemble Building Blocks with a Leap Motion Controller

  • Juncong Lin
  • Wei Yang
  • Xing GaoEmail author
  • Minghong Liao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9412)


Playing toy blocks can greatly help children develop various abilities such as spatial, mathematical, creative problem solving etc. Developing a computer-aided block building system would benefit a lot in recreational and educational applications. Providing a natural user interface to increase the user’s sense of immersion is very important for the success of such kind system. In this paper, we develop a virtual toy block assembly system targeting on children’s preschool education. With the help of leap motion controller, child only need to make various hand gestures to play toy blocks similar to how they do in real scenario. A block snapping interface is designed to help quickly place block at the appropriate position. Our system demonstrate great potential in education area.


Toy block assembly Leap motion Hand gesture Snapping interface 



This work was supported by the National Natural Science Foundation of China (No. 61202142), the National Key Technology R&D Program Foundation of China (No.2015BAH34F00/F01), Joint Funds of the Ministry of Education of China and China Mobile (No.MCM20130221), and Key Technology R&D Program of Xiamen (Nos.3502Z20140059,3502Z20141009)


  1. 1.
    Bröckl-Fox, U.: Real-Time 3D interaction with up to 16 degrees of freedom from monocular image flows. In: Proceedings of International Workshop on Automatic Face and Gesture Recognition, pp. 172–178 (1995)Google Scholar
  2. 2.
    Datcu, D., Lukosch, S.: Free-hands interaction in augmented reality. In: Proceedings of the 1st Symposium on Spatial User Interaction, pp. 33–40 (2013)Google Scholar
  3. 3.
    Mitani, J., Suzuki, H.: Making papercraft toys from meshes using strip-based approximate unfolding. ACM Trans. Graph. 23(3), 259–263 (2004)CrossRefGoogle Scholar
  4. 4.
    Kim, D., Hilliges, O., Izadi, S., Butler, A., Chen, J., Oikonomidis, I., Olivier, P.: Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor. In: Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology, pp. 167–176 (2012)Google Scholar
  5. 5.
    Kim, H., Albuquerque, G., Havemann, S., Fellner, D.W.: Tangible 3D: hand gesture interaction for immersive 3D modeling. In: Proceedings of the 11th Eurographics Conference on Virtual Environments, pp. 191–199 (2005)Google Scholar
  6. 6.
    Lin, J., Sun, Q., Li, G., He, Y.: SnapBlocks: a snapping interface for assembling toy blocks with XBOX Kinect. Multimedia tools and applications 73(3), 2009–2032 (2014)CrossRefGoogle Scholar
  7. 7.
    Patwardhan, K.S., Roy, S.D.: Hand gesture modelling and recognition involving changing shapes and trajectories, using a predictive eigentracker. Pattern Recogn. Lett. 28(3), 329–334 (2007)CrossRefGoogle Scholar
  8. 8.
    Penelle, B., Debeir, O.: Multi-sensor data fusion for hand tracking using Kinect and Leap Motion. In: Proceedings of the 2014 Virtual Reality International Conference, p. Article No. 22 (2014)Google Scholar
  9. 9.
    Xin, S., Lai, C.F., Fu, C.W., Wong, T.T., He, Y., Cohen-Or, D.: Making burr puzzles from 3D models. ACM Trans. Graph. 30(4), 97 (2011)CrossRefGoogle Scholar
  10. 10.
    Sturman, D.J., Zeltzer, D.: A survey of glove-based input. IEEE Comput. Graph. Appl. 14, 30–39 (1994)CrossRefGoogle Scholar
  11. 11.
    Triescha, J., von der Malsburg, C.: Classification of hand postures against complex backgrounds using elastic graph matching. Image Vis. Comput. 20(13–14), 937–943 (2002)CrossRefGoogle Scholar
  12. 12.
    Li, X.Y., Shen, C.H., Huang, S.S., Ju, T., Hu, S.M.: Popup: Automatic Paper Architectures from 3D Models. ACM Transactions on Graphics 29(4), 1–9 (2011). Article No: 111Google Scholar
  13. 13.
    Mori, Y., Igarashi, T.: Plushie: an interactive design system for plush toys. ACM Transactions on Graphics 26(3), 45 (2007)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Juncong Lin
    • 1
  • Wei Yang
    • 1
  • Xing Gao
    • 1
    Email author
  • Minghong Liao
    • 1
  1. 1.Software SchoolXiamen UniversityXiamenPeople’s Republic of China

Personalised recommendations