Skip to main content
SpringerLink
Log in
Book cover

Design Thinking Research pp 185–200Cite as

Investigating Active Tangibles and Augmented Reality for Creativity Support in Remote Collaboration

  • Chapter
  • First Online:

Part of the book series: Understanding Innovation ((UNDINNO))

Abstract

Physical manipulation is a key part of externalizing representations of knowledge and the creative process. However, contemporary tools for remote collaboration ignore physical manipulation and the haptic modality. We are interested in exploring remote physical manipulation in the context of ideation and brainstorming. Augmented Reality provides much of the benefits of spatial representation of remote participants, yet AR does not allow for rich physical manipulation and haptic feedback. Thus, we propose to use pairs of multi-robot system to provide synchronized haptic proxies in conjunction with the AR system. These small, tangible robots can be used directly as handles for digital models. We share insights gathered during experimentation to help design platforms combining AR and actuated tangibles, and present several application scenarios to illustrate their potential for remote collaboration.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  • Africano, D., Berg, S., Lindbergh, K., Lundholm, P., Nilbrink, F., & Persson, A. (2004). Designing tangible interfaces for children’s collaboration. In CHI ’04 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’04 (pp. 853–868). ACM, New York, NY, USA. https://doi.org/10.1145/985921.985945.

  • Amano, K., & Yamamoto, A. (2012). Tangible interactions on a flat panel display using actuated paper sheets. In Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces, ITS ’12 (pp. 351–354). ACM, New York, NY, USA. https://doi.org/10.1145/2396636.2396698.

  • Antle, A. N. (2007). The cti framework: Informing the design of tangible systems for children. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction, TEI ’07 (pp. 195–202). ACM, New York, NY, USA. https://doi.org/10.1145/1226969.1227010.

  • Billinghurst, M., Kato, H., & Poupyrev, I. (2001). The magicbook–Moving seamlessly between reality and virtuality. IEEE Computer Graphics and Applications, 21(3), 6–8. https://doi.org/10.1109/38.920621.

    Article  Google Scholar 

  • Brave, S., Ishii, H., & Dahley, A. (1998). Tangible interfaces for remote collaboration and communication. In Proceedings of the 1998 ACM Conference on Computer Supported Cooperative Work, CSCW ’98 (pp. 169–178). ACM, New York, NY, USA. https://doi.org/10.1145/289444.289491.

  • Brewer, J., Williams, A., & Dourish, P. (2007). A handle on what’s going on: Combining tangible interfaces and ambient displays for collaborative groups. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction, TEI ’07 (pp. 3–10). ACM, New York, NY, USA. https://doi.org/10.1145/1226969.1226971.

  • Buxton, B. (2009). Mediaspace–meaningspace–meetingspace. In Media space 20+ years of mediated life (pp. 217–231). Springer.

    Google Scholar 

  • Clark, H. H., Brennan, S. E., et al. (1991). Grounding in communication. Perspectives on Socially Shared Cognition, 13, 127–149.

    Article  Google Scholar 

  • Elvezio, C., Sukan, M., Oda, O., Feiner, S., & Tversky, B. (2017). Remote collaboration in ar and vr using virtual replicas. In ACM SIGGRAPH 2017 VR Village (p. 13). ACM.

    Google Scholar 

  • Everitt, K. M., Klemmer, S. R., Lee, R., & Landay, J. A. (2003). Two worlds apart: Bridging the gap between physical and virtual media for distributed design collaboration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 553–560). ACM

    Google Scholar 

  • Furió, D., Fleck, S., Bousquet, B., Guillet, J. P., Canioni, L., & Hachet, M. (2017). Hobit: Hybrid optical bench for innovative teaching. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17 (pp. 949–959). ACM, New York, NY, USA. https://doi.org/10.1145/3025453.3025789.

  • He, Z., Zhu, F., & Perlin, K. (2017) Physhare: Sharing physical interaction in virtual reality. In Adjunct Publication of the 30th Annual ACM Symposium on User Interface Software and Technology (pp. 17–19). ACM

    Google Scholar 

  • Henderson, S., & Feiner, S. (2011). Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Transactions on Visualization and Computer Graphics, 17(10), 1355–1368.

    Article  Google Scholar 

  • Holmquist, L. E., Falk, J., & Wigström, J. (1999). Supporting group collaboration with interpersonal awareness devices. Personal Technologies, 3(1), 13–21. https://doi.org/10.1007/BF01305316.

    Article  Google Scholar 

  • Hornecker, E., & Buur, J. (2006). Getting a grip on tangible interaction: A framework on physical space and social interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’06 (pp. 437–446). ACM, New York, NY, USA. https://doi.org/10.1145/1124772.1124838.

  • Ishii, H., & Kobayashi, M. (1992). Clearboard: A seamless medium for shared drawing and conversation with eye contact. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 525–532). ACM.

    Google Scholar 

  • Ishii, H., Lakatos, D., Bonanni, L., & Labrune, J. B. (2012). Radical atoms: Beyond tangible bits, toward transformable materials. Interactions, 19(1), 38–51. https://doi.org/10.1145/2065327.2065337.

    Article  Google Scholar 

  • Ishii, H., & Miyake, N. (1991). Toward an open shared workspace: Computer and video fusion approach of teamworkstation. Communications of the ACM, 34(12), 37–50.

    Article  Google Scholar 

  • Jordà, S., Geiger, G., Alonso, M., & Kaltenbrunner, M. (2007). The reactable: Exploring the synergy between live music performance and tabletop tangible interfaces. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction, TEI ’07 (pp. 139–146). ACM, New York, NY, USA. https://doi.org/10.1145/1226969.1226998.

  • Kasahara, S., Niiyama, R., Heun, V., & Ishii, H. (2013). Extouch: Spatially-aware embodied manipulation of actuated objects mediated by augmented reality. In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction, TEI ’13 (pp. 223–228). ACM, New York, NY, USA. https://doi.org/10.1145/2460625.2460661.

  • Kato, H., Billinghurst, M., Poupyrev, I., Tetsutani, N., & Tachibana, K. (2001). Tangible augmented reality for human computer interaction. In Proceedings of Nicograph 2001.

    Google Scholar 

  • Kojima, M., Sugimoto, M., Nakamura, A., Tomita, M., Nii, H., & Inami, M. (2006). Augmented coliseum: An augmented game environment with small vehicles. In First IEEE International Workshop on Horizontal Interactive Human-Computer Systems (TABLETOP ’06) (pp. 6 pp). https://doi.org/10.1109/TABLETOP.2006.3.

  • Le Goc, M., Kim, L. H., Parsaei, A., Fekete, J. D., Dragicevic, P., & Follmer, S. (2016). Zooids: Building blocks for swarm user interfaces. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology, UIST ’16 (pp. 97–109). ACM, New York, NY, USA (2016). https://doi.org/10.1145/2984511.2984547.

  • Leap Motion Inc. (2018). Leap motion. Online. https://www.leapmotion.com/.

  • Lee, G. A., Nelles, C., Billinghurst, M., & Kim, G. J. (2004). Immersive authoring of tangible augmented reality applications. In Proceedings of the 3rd IEEE/ACM International Symposium on Mixed and Augmented Reality (pp. 172–181). IEEE Computer Society.

    Google Scholar 

  • Lee, M., Norouzi, N., Bruder, G., Wisniewski, P. J., & Welch, G. F. (2018). The physical-virtual table: exploring the effects of a virtual human’s physical influence on social interaction. In Proceedings of the 24th ACM Symposium on Virtual Reality Software and Technology (p. 25). ACM.

    Google Scholar 

  • Leithinger, D., Follmer, S., Olwal, A., & Ishii, H. (2014). Physical telepresence: Shape capture and display for embodied, computer-mediated remote collaboration. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, UIST ’14 (pp. 461–470). ACM, New York, NY, USA. https://doi.org/10.1145/2642918.2647377.

  • Mi, H., & Sugimoto, M. (2011). Hats: Interact using height-adjustable tangibles in tabletop interfaces. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, ITS ’11 (pp. 71–74). ACM, New York, NY, USA. https://doi.org/10.1145/2076354.2076368.

  • Microsoft Corporation. (2019). Microsoft hololens | the leader in mixed reality technology. Online. https://www.microsoft.com/en-us/hololens.

  • Mueller, F. F., Cole, L., O’Brien, S., & Walmink, W. (2006). Airhockey over a distance: A networked physical game to support social interactions. In Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, ACE ’06. ACM, New York, NY, USA. https://doi.org/10.1145/1178823.1178906.

  • Nowacka, D., Ladha, K., Hammerla, N. Y., Jackson, D., Ladha, C., Rukzio, E., & Olivier, P. (2013). Touchbugs: Actuated tangibles on multi-touch tables. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13 (pp. 759–762). ACM, New York, NY, USA. https://doi.org/10.1145/2470654.2470761.

  • Oda, O., Elvezio, C., Sukan, M., Feiner, S., & Tversky, B. (2015). Virtual replicas for remote assistance in virtual and augmented reality. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (pp. 405–415). ACM.

    Google Scholar 

  • Özgür, A., Johal, W., Mondada, F., & Dillenbourg, P. (2017). Windfield: Demonstrating wind meteorology with handheld haptic robots. In Proceedings of the Companion of the 2017 ACM/IEEE International Conference on Human-Robot Interaction (pp. 48–49). ACM.

    Google Scholar 

  • Pangaro, G., Maynes-Aminzade, D., & Ishii, H. (2002). The actuated workbench: Computer-controlled actuation in tabletop tangible interfaces. In Proceedings of the 15th Annual ACM Symposium on User Interface Software and Technology, UIST ’02 (pp. 181–190). ACM, New York, NY, USA. https://doi.org/10.1145/571985.572011.

  • Patten, J., & Ishii, H. (2007). Mechanical constraints as computational constraints in tabletop tangible interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’07 (pp. 809–818). ACM, New York, NY, USA. https://doi.org/10.1145/1240624.1240746.

  • Patten Studio. (2014). Thumbles: Expanding the vocabulary of design. Online. http://www.pattenstudio.com/works/thumbles/.

  • Pedersen, E. W., & Hornbæk, K. (2011). Tangible bots: Interaction with active tangibles in tabletop interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’11 (pp. 2975–2984). ACM, New York, NY, USA. https://doi.org/10.1145/1978942.1979384.

  • Poupyrev, I., Nashida, T., & Okabe, M. (2007). Actuation and tangible user interfaces: The vaucanson duck, robots, and shape displays. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction, TEI ’07 (pp. 205–212). ACM, New York, NY, USA. https://doi.org/10.1145/1226969.1227012.

  • Richter, J., Thomas, B. H., Sugimoto, M., & Inami, M. (2007). Remote active tangible interactions. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction, TEI ’07 (pp. 39–42). ACM, New York, NY, USA. https://doi.org/10.1145/1226969.1226977.

  • Riedenklau, E., Hermann, T., & Ritter, H. (2012). An integrated multi-modal actuated tangible user interface for distributed collaborative planning. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction, TEI ’12 (pp. 169–174). ACM, New York, NY, USA. https://doi.org/10.1145/2148131.2148167.

  • Roo, J. S., Gervais, R., Frey, J., & Hachet, M. (2017). Inner garden: Connecting inner states to a mixed reality sandbox for mindfulness. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17 (pp. 1459–1470). ACM, New York, NY, USA. https://doi.org/10.1145/3025453.3025743.

  • Roo, J. S., & Hachet, M. (2017). One reality: Augmenting how the physical world is experienced by combining multiple mixed reality modalities. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology, UIST ’17 (pp. 787–795). ACM, New York, NY, USA. https://doi.org/10.1145/3126594.3126638.

  • Rosenfeld, D., Zawadzki, M., Sudol, J., & Perlin, K. (2004). Physical objects as bidirectional user interface elements. IEEE Computer Graphics and Applications, 24(1), 44–49. https://doi.org/10.1109/MCG.2004.1255808.

    Article  Google Scholar 

  • Schneider, B., Sharma, K., Cuendet, S., Zufferey, G., Dillenbourg, P., & Pea, R. (2016). Using mobile eye-trackers to unpack the perceptual benefits of a tangible user interface for collaborative learning. ACM Transactions on Computer-Human Interaction (TOCHI), 23(6), 39.

    Article  Google Scholar 

  • Tang, J. C., & Minneman, S. (1991). Videowhiteboard: Video shadows to support remote collaboration. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 315–322). ACM.

    Google Scholar 

  • Underkoffler, J., & Ishii, H. (1999). Urp: A luminous-tangible workbench for urban planning and design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’99 (pp. 386–393). ACM, New York, NY, USA. https://doi.org/10.1145/302979.303114.

  • Weiss, M., Schwarz, F., Jakubowski, S., & Borchers, J. (2010). Madgets: Actuating widgets on interactive tabletops. In Proceedings of the 23Nd Annual ACM Symposium on User Interface Software and Technology, UIST ’10 (pp. 293–302). ACM, New York, NY, USA. https://doi.org/10.1145/1866029.1866075.

  • Wilson, A. D., & Robbins, D. C. (2007). Playtogether: Playing games across multiple interactive tabletops. In IUI Workshop on Tangible Play: Research and Design for Tangible and Tabletop Games.

    Google Scholar 

  • Xie, L., Antle, A. N., & Motamedi, N. (2008). Are tangibles more fun? Comparing children’s enjoyment and engagement using physical, graphical and tangible user interfaces. In Proceedings of the 2nd International Conference on Tangible and Embedded Interaction (pp. 191–198). ACM.

    Google Scholar 

  • Yarosh, S., Tang, A., Mokashi, S., & Abowd, G. D. (2013). Almost touching: parent-child remote communication using the sharetable system. In Proceedings of the 2013 Conference on Computer Supported Cooperative Work (pp. 181–192). ACM.

    Google Scholar 

  • Zhao, Y., Kim, L.H., Wang, Y., Le Goc, M., & Follmer, S. (2017). Robotic assembly of haptic proxy objects for tangible interaction and virtual reality. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (pp. 82–91). ACM.

    Google Scholar 

  • Zhou, Z., Cheok, A. D., Chan, T., Pan, J. H., & Li, Y. (2004). Interactive entertainment systems using tangible cubes. In IE2004 Proceedings of the First Australian Workshop on Interactive Entertainment (p. 19).

    Google Scholar 

Download references

Acknowledgements

This work is supported in part by the Hasso Plattner Design Thinking Research Program.

Author information

Authors and Affiliations

  1. SHAPE Lab, Stanford University, 524 Duena St., Stanford, CA, 94305, USA

    Mathieu Le Goc, Allen Zhao, Ye Wang, Griffin Dietz, Rob Semmens & Sean Follmer

  2. Naval Postgraduate School, Monterey, CA, USA

    Rob Semmens

Authors
  1. Mathieu Le Goc
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Allen Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ye Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Griffin Dietz
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rob Semmens
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Sean Follmer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mathieu Le Goc .

Editor information

Editors and Affiliations

  1. Hasso Plattner Institute for Digital Engineering, University of Potsdam, Potsdam, Germany

    Christoph Meinel

  2. Stanford University, Stanford, CA, USA

    Larry Leifer

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Le Goc, M., Zhao, A., Wang, Y., Dietz, G., Semmens, R., Follmer, S. (2020). Investigating Active Tangibles and Augmented Reality for Creativity Support in Remote Collaboration. In: Meinel, C., Leifer, L. (eds) Design Thinking Research . Understanding Innovation. Springer, Cham. https://doi.org/10.1007/978-3-030-28960-7_12

Download citation

Publish with us

Policies and ethics

Search

Navigation