Design for Assistive Augmentation—Mind, Might and Magic

  • Ellen Yi-Luen DoEmail author
Part of the Cognitive Science and Technology book series (CSAT)


In this preface I argue that the Design for Assistive Augmentation should take 3M’s into consideration: Mind, to observe before acting, to be thoughtful, and open-minded; Might, to consider the capacity, and competency of people and the technology; and, Magic, to have technology wonderfully blended in everyday life activities. I discuss the idea of building computational tools as a way of working, the idea of Creating Unique Technology for Everyone, and the asipiration to create new types of physical and computational dumbbells to help people to build muscles, running shoes to help people run faster, and skis to enable people to have new kind of experience enabled by the tools and to augment human capabilities.



This material is based upon research work supported by the US National Science Foundation under Grant SHB 1117665, DUE 0127579, IIS 96-19856, Emory ADRC/ACTSI Pilot Grant, Korea Institute for Advancement of Technology (KIAT)’s Global Industry-Academia Cooperation Program, Georgia Institute of Technology’s Health System Institute, Strategic Energy Initiative, and Center for Music Technology, Atlanta’s VA Center, Emory Center for Neurodegenerative Disease and Alzheimer’s Disease Center, Pennsylvania Infrastructure Technology Alliance (PITA), People’s Association (Singapore), and the National Research Foundation, Prime Minister’s Office, Singapore under its International Research Centre in Singapore Funding Initiative. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the funding agencies.


  1. 1.
    Maslow AH (1943) A theory of human motivation. Psychol Rev 50(4):370–396.
  2. 2.
  3. 3.
    Wikipedia. Arthur Clarke’s three laws.
  4. 4.
    Do EY-L (1998) The right tool at the right time—investigation of freehand drawing as an interface to knowledge based design tools. Ph.D dissertation, Georgia Institute of Technology, GVU Center Technical Reports [536].
  5. 5.
    Do EY-L (1995) What’s in a diagram that a computer should understand? In: Tan M, Teh R (eds) The global design studio, proceedings of the sixth international conference on computer aided architectural design futures (CAAD Futures 95), National University of Singapore, pp 469–482. ISBN:9971-62-423-0.
  6. 6.
    Do EY-L, Gross MD (2001) Thinking with diagrams in architectural design. In: Artificial intelligence review, vol 15. Kluwer Academic Publishers, Dordrecht, The Netherlands, pp 135–149.
  7. 7.
    Do EY-L (2005) Design sketches and sketch design tools. In: Nakakoji K, Gross MD, Candy L, Edmonds E (eds) KBS—knowledge based systems, vol 18. Elsevier Publisher, pp 383–405. Accessed 11 Aug 2005.
  8. 8.
    Do EY-L (2001) VR sketchpad: creating instant 3D worlds by sketching on a transparent window. In: de Vries B, van Leeuwen JP, Achten HH (eds) CAAD futures 2001, July 2001. Kluwer Academic Publishers, Eindhoven, The Netherlands, pp 161–172.
  9. 9.
    Oh Y, Gross MD, Do EY-L (2004) Critiquing freehand sketches: a computational tool for design evaluation. In: Gero J, Knight T (eds) Visual and spatial reasoning in design III [VR’04], 22–23 July 2004. MIT, pp 105–120Google Scholar
  10. 10.
    Jung T, Gross MD, Do EY-L (2001) Space pen, annotation and sketching on 3D models on the internet. In: de Vries B, van Leeuwen JP, Achten HH (eds) CAAD futures 2001, July 2001. Kluwer Academic Publishers, Eindhoven, The Netherlands, pp 257–270Google Scholar
  11. 11.
    Jung T, Gross, MD, Do EY (1999) Immersive redlining and annotation of 3D design models on the web. In: Augenbroe G, Eastman C (eds) Computers in building (June, ‘99) proceedings of the CAAD futures ‘99 conference, pp 81–98Google Scholar
  12. 12.
    Jung T, Gross MD, Do EY-L (2002) Annotating and sketching on 3D web models. In: International conference on intelligent user interfaces (IUI), 13–16 Jan 2002. ACM Press, San Francisco, pp 95–102Google Scholar
  13. 13.
    Do EY-L, Gross MD (2004) Let there be light! Knowledge-based 3-D sketching design tools. In: Jabi W, Pinet C (eds) IJAC Int J Archit Comput (Multi-Science Publishing Co Ltd) 2(2):211–227(17).
  14. 14.
    Lee M-C, Do EY-L (2003) Space maker: creating space by sketching it. In: Annual conference of ACADIA, association of computer aided design in architecture, 23–26 Oct 2003. Ball State University, pp 311–323Google Scholar
  15. 15.
    Oh Y, Johnson G, Gross MD, Do EY-L (2006) The designosaur and the furniture factory. In: International conference on design computing and cognition (DCC 06), 10–12 July 2006, Eindhoven, NetherlandsGoogle Scholar
  16. 16.
    Johnson G, Gross MD, Do EY-L, Hong J (2012) Sketch it, make it: sketching precise drawings for laser cutting. In: CHI EA 2012, pp 1079–1082.
  17. 17.
    Do EY-L (2011) Sketch that scene for me and meet me in cyberspace. In: Wang X, Tsai JJ-H (eds) Collaborative design in virtual environments (chapter 11). Springer, pp 121–130. ISBN: 978-94-007-0604-0Google Scholar
  18. 18.
    Gross MD, Do EY-L (1995) Diagram query and image retrieval in design. In: 2nd IEEE international conference on image processing, vol 2. IEEE Computer Society Press, Washington, DC, pp 308–311Google Scholar
  19. 19.
    Do EY-L (2001) Graphics interpreter of design actions: the GIDA system of diagram sorting and analysis. In: de Vries B, van Leeuwen JP, Achten HH (eds) Computer aided architectural design futures (CAAD Futures 2001), July 2001. Kluwer Academic Publishers, Eindhoven, The Netherlands, pp 271–284Google Scholar
  20. 20.
    Zimring C, Bafna S, Do EY-L (1996) Structuring cases in a case-based design aid. In: Vanegas J, Chinowsky P (eds) Third congress on design computing, Anaheim, A/E/C’96. American Society of Civil Engineers (ASCE), pp 308–313Google Scholar
  21. 21.
    Gross MD, Do EY-L (1996) Ambiguous intentions—a paper-like interface for creative design. In: Proceedings of ninth annual symposium for user interface software and technology (UIST 96), pp 183–192Google Scholar
  22. 22.
    Cai H, Do EY-L, Zimring CM (2010) Extended linkography and distance graph in design evaluation: an empirical study of the dual effects of inspiration sources in creative design. J Design Stud 31(2):146–168.
  23. 23.
    Williams J, Do EY-L (2009) Locomotion storytelling: a study of the relationship between kinesthetic intelligence and tangible objects in facilitating preschoolers creativity in storytelling. In: ACM creativity and cognition, Oct 27–30, Berkeley, pp 415–416.
  24. 24.
    Lee S, Ezer N, Sanford J, Do EY-L (2009) Designing together while apart: the role of computer-mediated communication and collaborative virtual environments on design collaboration. In: IEEE systems, mans and cybernetics, San Antonio, Oct 11–14, pp 3693–3698Google Scholar
  25. 25.
    Do EY-L, Gross MD (2007) Environments for creativity—a lab for making things. In: Shneiderman B, Fischer G, Giaccardi E, Eisenberg M (eds) Creativity and cognition. ACM Press, New York, pp 27–36.
  26. 26.
    Camarata K, Do EY-L, Johnson BR, Gross MD (2002) Navigating information space with tangible media. In: International conference on intelligent user interfaces (IUI), 13–16 Jan 2002. ACM Press, San Francisco, pp 31–38.
  27. 27.
    Eng M, Camarata K, Do EY-L, Gross MD (2006) FlexM: designing a physical construction kit for 3D modeling. In: Cheng N, Pinet C (eds) IJAC Int J Archit Comput (Multi-Science Publishing Co Ltd) 4(2):27–47.
  28. 28.
    Wu A, Joyner D, Do EY-L (2010) Move, beam, and check! Imagineering tangible optical chess on an interactive tabletop display. In: ACM Computers in Entertainment (CIE), vol 8 (3), Article 20 (15 pp.).
  29. 29.
    Camarata K, Do EY-L, Gross MD (2006) Energy cube and energy magnets. In: Cheng N, Pinet C (eds) IJAC—Int J Archit Comput 4(2):49–66.
  30. 30.
    Weller MP, Do EY-L, Gross MD (2008) Posey: instrumenting a poseable hub and strut construction toy. In: Tangible and embedded interaction (TEI’08), Feb 18–20, Bonn, Germany, pp 39–46.
  31. 31.
    Oh Y, Camarata K, Weller MP, Gross MD, Do EY-L (2008) TeleTables and window seat: bilocative furniture interfaces. In: Theng Y-L, Duh HBL (eds) Ubiquitous computing: design, implementation, and usability. Hershey: Information Science Reference, 2008 (chapter 11), pp 160–171. ISBN-13: 978-1-59904-693-8 (hardcover); ISBN-13: 978-1-59904-695-2 (e-book)Google Scholar
  32. 32.
    Camarata K, Gross MD, Do EY-L (2003) A physical computing studio: exploring computational artifacts and environments. In: Cheng NY-W, Do EY-L (eds) Int J Archit Comput (Multi-Science Publisher, UK) 1(2):169–190.
  33. 33.
    Do EY-L (2009) Technological interventions for hand hygiene adherence—research and intervention for smart patient room. In: Tidalfi T, Dorta T (eds) CAAD futures, 17–19 June 2009, Montreal, Canada, pp 303–313Google Scholar
  34. 34.
    Huang K, Starner T, Do E, Weinberg G, Kohlsdorf D, Ahlrichs C, Leibrandt R (2010) Mobile music touch: mobile tactile stimulation for passive learning. In: CHI’10 proceedings of the 28th international conference on human factors in computing systems, pp 791–800.
  35. 35.
    Lawson MA, Do EY-L, Marston JR, Ross DA (2011) Helping hands versus ERSP vision: evaluating the effectiveness of two wearable object recognition technologies. In: Stephanidis C (ed) Human-computer interaction thematic area, 9–14 July 2011, Orlando, in HCI International 2011: Posters, Part I, HCII 2011, CCIS 173, pp 383–388, 2011. Springer Berlin, Heidelberg 2011.
  36. 36.
    Lu S-C, Blackwell N, Do EY-L (2011) mediRobbi: an interactive companion for pediatric patients during hospital visit. In: Health and well-being applications, Tuesday, 12 July 2011: 08:00–10:00, Orlando, in HCI International 2011, vol 2, LNCS_6762, 2011. Springer, Berlin, Heidelberg, pp 547–556Google Scholar
  37. 37.
    Kim H, Swarts M, Lee S, Do EYL (2010) HealthQuest: technology that encourages physical activity in the workplace (2010). In: ICOST—international conference on smart homes and health telemetics, Seoul, June 22–24, pp 263–266.
  38. 38.
    Kim H, Cho YS, Guha A, Do EY-L (2010) ClockReader: investigating senior computer interaction through pen-based computing. In: CHI workshop on senior-friendly technologies: interaction design for the elderly, 10 Apr 2010, Atlanta, GA, USA, pp 30–33Google Scholar
  39. 39.
    Agrell B, Ove D (1998) in Age and Ageing (1998), The clock-drawing test. 27:399–403 (Republished Age Ageing. 41(Suppl 3):iii41–5 (2012).
  40. 40.
    Kim H, Hsiao C-P, Do EY-L (2012) Home-based computerized cognitive assessment tool for dementia screening. J Ambient Intell Smart Environ (JAISE IOS Press) 429–442.
  41. 41.
    Stroke Statistics, Internet Stroke Center. US statistics from U.S. Centers for Disease Control and Prevention, retrieved July 2nd, 2017
  42. 42.
    Zhao C, Hsiao C-P, Davis N, Do EY-L (2013) Tangible games for stroke rehabilitation with digital box and blocks test. In: CHI EA’13 CHI’13 extended abstracts on human factors in computing systems. ACM, New York, NY, USA, pp 523–528.
  43. 43.
    Huang K, Do EY-L, Starner T (2008) PianoTouch: a wearable haptic piano instruction system for passive learning of piano skills. In: 12th IEEE international symposium on wearable computers ISWC 2008, pp 41–44, Sep 28–Oct 1, Pittsburgh.,
  44. 44.
    Markow T, Ramakrishnan N, Huang K, Starner T, Eicholtz M, Garrett S, Profita H, Scarlata A, Schooler C, Tarun A, Backus D (2010) Mobile music touch: vibration stimulus as a possible hand rehabilitation method. In: Proceedings of the 4th international pervasive health conference, March 2010, Munich, GermanyGoogle Scholar
  45. 45.
    Li R, Wang Y, Hsiao C-P, Davis N, Hallam J, Do E (2016) Tactile teacher: enhancing traditional piano lessons with tactile instructions. In: Proceedings of CSCW (computer supported collaborative work), Feb 27–March 2, 2016. ACM Press, NY, pp 329–332.
  46. 46.
    Do EY-L (2005) Creating unique technology for everyone. In: ASEAN CHI symposium’15 proceedings of the ASEAN CHI symposium’15, Seoul, Republic of Korea, 18–23 April 2015. ACM, New York, NY, pp 42–45.
  47. 47.
    Tan A (2014) New app to help seniors live more independently, 13 April 2014, pp 16, home section, Sunday Times, The Strait Times, Singapore.
  48. 48.
    Lu W, Sun CC, Bleeker T, You Y, Kitazawa S, Do EY-L (2014) Sensorendipity, a real time web-enabled smartphone sensor platform for idea generation and sensor platform. In: International symposium of Chinese CHI (Chinese CHI’14), Toronto, 26–27 Apr 2014, pp 11–18.
  49. 49.
    Katsumoto Y, Tokuhisa S, Inakage M (2013) Ninja track: design of electronic toy variable in shape and flexibility. In: International conference on tangible, embedded and embodied interaction, 10–13 Feb 2013. ACM, Barcelona, Spain, pp 17–24.
  50. 50.
    Feit D (2015) This is the sword-slash-fishing reel you’ve been waiting for. In: Culture, Wired, 09.18.15, #INDIE GAMES #REEL BLADE #TOKYO GAME SHOW 2015. or,
  51. 51.
    (2015) Reel blade: battle of the high seas.
  52. 52.
  53. 53.
    NUS News (2014) Electronic spoon helps the elderly age tastefully, Published: 16 May 2014, Category: Highlights.,
  54. 54.
    Ranasinghe N, Lee K-Y, Suthokumar G, Do EY-L (2014) The sensation of taste in the future of immersive media. In: ImmersiveMe’14 proceedings of the 2nd ACM international workshop on immersive media experiences 07–07 November 2014, Orlando, Florida, USA. ACM, New York, NY, USA, pp 7–12.
  55. 55.
    Ranasinghe N, Jain P, Karwita S, Do EY-L (2017) Virtual lemonade: let’s teleport your lemonade! In: Proceedings of the 11th international conference on tangible, embedded and embodied interactions, (TEI 2017), 20–23 March 2017, Yokohama, Japan, pp 183–190 (41/151 = 27%).
  56. 56.
    Ranasinghe N, Jain P, Karwita S, Tolley D, Do EY-L (2017) Ambiotherm: enhancing sense of presence in virtual reality by simulating real-world environmental conditions. In ACM human factors in computing conference—CHI’17 papers and notes, 06–11 May 2017, Denver, CO, USA, 2017. ACM, pp 1731–1742. ISBN: 978-1-4503-4655-9/17/05.
  57. 57.
    Nakakoji K (2006) Meanings of tools, support, and uses for creative design processes. In: The proceedings of international design research symposium’06, Nov 2006. CREDITS Research Center, Seoul, Korea, pp 156–165. ISBN: 89-950046-4-9Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Keio-NUS CUTE CenterNational University of SingaporeSingaporeSingapore
  2. 2.ATLAS InstituteUniversity of Colorado BoulderBoulderUSA

Personalised recommendations