Creative Sketching Apprentice: Supporting Conceptual Shifts in Sketch Ideation

  • Pegah KarimiEmail author
  • Kazjon Grace
  • Nicholas Davis
  • Mary Lou Maher
Conference paper


Sketching in design is typically a part of the ideation process. A common occurrence in sketching creativity is the conceptual shift, or when a drawn object is reinterpreted as belonging to a different object category.


  1. 1.
    Goel V (1991) Sketches of thought: a study of the role of sketching in design problem-solving and its implications for the computational theory of the mind. Ph.D. Dissertation, University of California, BerkeleyGoogle Scholar
  2. 2.
    Johnson G, Gross MD, Hong J, Do EYL (2009) Computational support for sketching in design: a review. Found Trends® Hum Comput Interact 2(1):1–93CrossRefGoogle Scholar
  3. 3.
    Biles JA (1994, September) GenJam: a genetic algorithm for generating jazz solos. In: ICMC, vol 94, pp 131–137Google Scholar
  4. 4.
    Hoffman G, Weinberg G (2010, April) Shimon: an interactive improvisational robotic marimba player. In: CHI’10 extended abstracts on human factors in computing systems, pp 3097–3102 (ACM)Google Scholar
  5. 5.
    Jacob M, Zook A, Magerko B (2013) Viewpoints AI: procedurally representing and reasoning about gestures. In: DiGRA conferenceGoogle Scholar
  6. 6.
    Kantosalo A, Toivanen JM, Xiao P, Toivonen H (2014, June) From isolation to involvement: adapting machine creativity software to support human-computer co-creation. In: ICCC, pp 1–7Google Scholar
  7. 7.
    Davis N, Hsiao CP, Singh KY, Li L, Moningi S, Magerko B (2015, June) Drawing apprentice: an enactive co-creative agent for artistic collaboration. In: Proceedings of the 2015 ACM SIGCHI conference on creativity and cognition, pp 185–186 (ACM)Google Scholar
  8. 8.
    Davis NM, Popova Y, Sysoev I, Hsiao CP, Zhang D, Magerko B (2014) Building artistic computer colleagues with an enactive model of creativity. In: ICCC, pp 38–45Google Scholar
  9. 9.
    Boden M (1990) The creative mind: myths and mechanisms. Weidenfeld and Nicoison, London, UKGoogle Scholar
  10. 10.
    Colton S, Wiggins GA (2012, August) Computational creativity: the final frontier? In: Proceedings of the 20th European conference on artificial intelligence, pp 21–26 (IOS Press)Google Scholar
  11. 11.
    Wiggins GA (2006) Searching for computational creativity. New Gener Comput 24(3):209–222CrossRefGoogle Scholar
  12. 12.
    Wiggins GA (2006) A preliminary framework for description, analysis and comparison of creative systems. Knowl-Based Syst 19(7):449–458CrossRefGoogle Scholar
  13. 13.
    Edmonds EA, Candy L (2005) Computer support for creativity. Int J Hum Comput Stud 63(4–5):363–364CrossRefGoogle Scholar
  14. 14.
    Shneiderman B (2007) Creativity support tools: accelerating discovery and innovation. Commun ACM 50(12):20–32CrossRefGoogle Scholar
  15. 15.
    Shneiderman B, Fischer G, Czerwinski M, Resnick M, Myers B, Candy L, Edmonds E, Eisenberg M, Giaccardi E, Hewett T, Jennings P, Kules B, Nakakoji K, Nunamaker J, Pausch R, Selker T, Sylvan E, Terry M (2006) Creativity support tools: report from a US national science foundation sponsored workshop. Int J Hum Comput Interact 20(2):61–77CrossRefGoogle Scholar
  16. 16.
    Voigt M, Niehaves B, Becker J (2012) Towards a unified design theory for creativity support systems. In: Design science research in information systems. Advances in theory and practice, pp 152–173Google Scholar
  17. 17.
    Lubart T (2005) How can computers be partners in the creative process: classification and commentary on the special issue. Int J Hum Comput Stud 63(4):365–369CrossRefGoogle Scholar
  18. 18.
    Davis N, Hsiao CP, Singh KY, Magerko B (2016, September) Co-creative drawing agent with object recognition. In: Twelfth artificial intelligence and interactive digital entertainment conferenceGoogle Scholar
  19. 19.
    Getzels JW, Csikszentmihalyi M (1977) The creative vision: a longitudinal study of problem finding in art. J Aesthetics Art Criticism 36(1):96–98CrossRefGoogle Scholar
  20. 20.
    Grace K, Maher ML (2015) Surprise and reformulation as meta-cognitive processes in creative design. In: Proceedings of the third annual conference on advances in cognitive systems ACS, p 8Google Scholar
  21. 21.
    Poon J, Maher ML (1997) Co-evolution and emergence in design. Artif Intell Eng 11(3):319–327CrossRefGoogle Scholar
  22. 22.
    Schon DA (1984) The reflective practitioner: how professionals think in action, vol 5126 (Basic books)Google Scholar
  23. 23.
    Schön D (1983) The reflective practitioner: how practitioners think in action. London: Temple SmithGoogle Scholar
  24. 24.
    Schon DA, Wiggins G (1992) Kinds of seeing and their functions in designing. Des Stud 13(2):135–156CrossRefGoogle Scholar
  25. 25.
    Grace K, Maher ML, Fisher D, Brady K (2015) Modeling expectation for evaluating surprise in design creativity. In: Design computing and cognition’14. Springer, Cham, pp 189–206Google Scholar
  26. 26.
    Suwa M, Gero J, Purcell T (2000) Unexpected discoveries and S-invention of design requirements: important vehicles for a design process. Des Stud 21(6):539–567CrossRefGoogle Scholar
  27. 27.
    Prats M, Garner S (2006). Observations on ambiguity in design sketches. Tracey Online J Contemp Drawing Res, pp 1–7Google Scholar
  28. 28.
    Gero JS (1998) Conceptual designing as a sequence of situated acts. In: Artificial intelligence in structural engineering. Springer, Berlin, Heidelberg, pp 165–177CrossRefGoogle Scholar
  29. 29.
    Eitz M, Hays J, Alexa M (2012) How do humans sketch objects? ACM Trans Graph 44:1–44:10Google Scholar
  30. 30.
    Yu Q, Yang Y, Song YZ, Xiang T, Hospedales T (2015) Sketch-a-net that beats humans. arXiv preprint arXiv:1501.07873Google Scholar
  31. 31.
    LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444CrossRefGoogle Scholar
  32. 32.
    Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556Google Scholar
  33. 33.
    Maaten LVD, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(Nov):2579–2605zbMATHGoogle Scholar
  34. 34.
    Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009, June) Imagenet: a large-scale hierarchical image database. In: IEEE conference on computer vision and pattern recognition, CVPR 2009, IEEE, pp 248–255Google Scholar
  35. 35.
    Ha D, Eck D (2017) A neural representation of sketch drawings. arXiv preprint arXiv:1704.03477Google Scholar
  36. 36.
    Shneiderman B (2007) Creativity support tools: accelerating discovery and innovation. Commun ACM 50(12):20–32CrossRefGoogle Scholar
  37. 37.
    Karimi P, Davis N, Grace K, Maher ML (2018) Deep learning for identifying potential conceptual shifts for Co-creative drawing. arXiv prepreint arXiv: 1801.00723Google Scholar
  38. 38.
    Torrance EP (1962) Guiding creative talent. Prentice-Hall, Inc., Englewood Cliffs, NJ. Scholar
  39. 39.
    Gentner D (1983) Structure-mapping: a theoretical framework for analogy. Cogn Sci 7(2):155–170CrossRefGoogle Scholar
  40. 40.
    Zeiler MD, Fergus R (2014, September) Visualizing and understanding convolutional networks. In: European conference on computer vision. Springer, Cham, pp 818–833Google Scholar
  41. 41.
    Davies J, Goel AK, Nersessian NJ (2009) A computational model of visual analogies in design. Cogn Syst Res 10(3):204–215CrossRefGoogle Scholar
  42. 42.
    Goldschmidt G (2001) Visual analogy: a strategy for design reasoning and learning. In: Design knowing and learning: cognition in design education, pp 199–220CrossRefGoogle Scholar
  43. 43.
    Casakin H (2004) Visual analogy as a cognitive strategy in the design process: expert versus novice performance. J Des Res 4(2):1–18CrossRefGoogle Scholar
  44. 44.
    Jaderberg M, Simonyan K, Zisserman A (2015) Spatial transformer networks. In: Advances in neural information processing systems, pp 2017–2025Google Scholar
  45. 45.
    Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434Google Scholar
  46. 46.
    Suwa M, Tversky B (1997) What do architects and students perceive in their design sketches? A protocol analysis. Des Stud 18(4):385–403CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Pegah Karimi
    • 1
  • Kazjon Grace
    • 2
  • Nicholas Davis
    • 1
  • Mary Lou Maher
    • 1
  1. 1.University of North Carolina at CharlotteCharlotteUSA
  2. 2.The University of SydneySydneyAustralia

Personalised recommendations