Abstract
The School of Creative Arts & Technologies at Ulster University (Magee) has brought together the subject of computing with creative technologies, cinematic arts (film), drama, dance, music and design in terms of research and education. We propose here the development of a flagship computer software platform, SceneMaker, acting as a digital laboratory workbench for integrating and experimenting with the computer processing of new theories and methods in these multidisciplinary fields. We discuss the architecture of SceneMaker and relevant technologies for processing within its component modules. SceneMaker will enable the automated production of multimodal animated scenes from film and drama scripts or screenplays. SceneMaker will highlight affective or emotional content in digital storytelling with particular focus on character body posture, facial expressions, speech, non-speech audio, scene composition, timing, lighting, music and cinematography. Applications of SceneMaker include automated simulation of productions and education and training of actors, screenwriters and directors.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Hanser, E., Mc Kevitt, P., Lunney, T., Condell, J.: SceneMaker: intelligent multimodal visualisation of natural language scripts. In: Coyle, L., Freyne, J. (eds.) AICS 2009. LNCS (LNAI), vol. 6206, pp. 144–153. Springer, Heidelberg (2010). doi:10.1007/978-3-642-17080-5_17
Hanser, E., Mc Kevitt, P., Lunney, T., Condell, J.: SceneMaker: automatic visualisation of screenplays. In: Mertsching, B., Hund, M., Aziz, Z. (eds.) KI 2009. LNCS (LNAI), vol. 5803, pp. 265–272. Springer, Heidelberg (2009). doi:10.1007/978-3-642-04617-9_34
Ma, M., McKevitt, P.: Virtual human animation in natural language visualisation. Artif. Intell. Rev. 25(1–2), 37–53 (2006). Special Issue on the 16th Artificial Intelligence and Cognitive Science Conference (AICS 2005)
Muñoz, K., Mc Kevitt, P., Lunney, T., Noguez, J., Neri, L.: Designing and evaluating emotional student models for game-based learning. In: Ma, M., Oikonomou, A., Jain, L.C. (eds.) Serious Games and Edutainment Applications, London, England, pp. 245–272. Springer, Heidelberg (2011). Also presented at the First International Workshop on Serious Games Development and Applications (SGDA 2010), School of Computing, University of Derby, England, 8 July 2011
Mulholland, E., Mc Kevitt, P., Lunney, T., Farren, J., Wilson, J.: 360-MAM-affect: sentiment analysis with the google prediction API and EmoSenticNet. In: Proceedings of the 7th International Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN-2015), Politecnico di Torino, Turin (Torino), Italy, 10–12, 1–5 June 2015. EUDL EAI Endorsed Transactions on Scalable Information Systems vol. 2, no. 6, p. e5, November 2015
Ekman, P., Rosenberg, E.L.: What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System. Oxford University Press, England (1997)
Mehrabian, A.: Framework for a comprehensive description and measurement of emotional states. Gen. Soc. Gen. Psychol. Monogr. 121(3), 339–361 (1995). Heldref Publishing
Ortony, A., Clore, G.L., Collins, A.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge (1988)
Francisco V., Hervás R., Gervás P.: Two different approaches to automated mark up of emotions in text. In: Bramer M., Coenen F., Tuson A. (eds.) Research and Development in Intelligent Systems XXIII. Springer, London (2007)
Strapparava, C., Mihalcea, R.: Learning to identify emotions in text. In: Proceedings of the 2008 ACM Symposium on Applied Computing, SAC 2008, pp. 1556–1560. ACM, New York (2008)
Su, W.-P., Pham, B., Wardhani, A.: Personality and emotion-based high-level control of affective story characters. IEEE Trans. Vis. Comput. Graph. 13(2), 281–293 (2007)
Liu, H., Lieberman, H., Selker, T.: A model of textual affect sensing using real-world knowledge. In: Proceedings of the 8th International Conference on Intelligent User Interfaces, IUI 2003, pp. 125–132. ACM, New York (2003)
Liu, H., Singh, P.: ConceptNet: a practical commonsense reasoning toolkit. BT Technol. J. 22(4), 211–226 (2004). Springer, The Netherlands
Shaikh, M.A.M., Prendinger, H., Ishizuka, M.: A linguistic interpretation of the OCC emotion model for affect sensing from text. In: Tao, J., Tan, T. (eds.) Affective Information Processing, pp. 45–73. Springer, London (2009)
Prendinger, H., Ishizuka, M.: SCREAM: scripting emotion-based agent minds. In: Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems: Part 1, AAMAS 2002, pp. 350–351. ACM, New York (2002)
Gebhard, P.: ALMA - layered model of affect. In: Proceedings of the 4th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2005), Utrecht University, The Netherlands, pp. 29–36. ACM, New York (2005)
De Raad, B.: The big five personality factors. In: The Psycholexical Approach to Personality. Hogrefe & Huber, Cambridge, MA, USA (2000)
Mancini, M., Pelachaud, C.: Implementing distinctive behavior for conversational agents. In: Sales Dias, M., Gibet, S., Wanderley, M.M., Bastos, R. (eds.) GW 2007. LNCS, vol. 5085, pp. 163–174. Springer, Heidelberg (2009). doi:10.1007/978-3-540-92865-2_17
Coyne, B., Sproat, R.: WordsEye: an automatic text-to-scene conversion system. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 487–496. ACM Press, Los Angeles (2001)
De Melo, C., Paiva, A.: Multimodal expression in virtual humans. Comput. Animation Virtual Worlds 17(3–4), 239–348 (2006)
Liu, Z., Leung, K.: Script visualization (ScriptViz): a smart system that makes writing fun. Soft Comput. 10(1), 34–40 (2006)
Hanser, E., Mc Kevitt, P., Lunney, T., Condell, J.: NewsViz: emotional visualization of news stories. In: Inkpen, D., Strapparava, C. (eds.) Proceedings of the NAACL-HLT Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, Millennium Biltmore Hotel, Los Angeles, CA, USA, pp. 125–130, 5 June 2010
Rasheed, Z., Sheikh, Y., Shah, M.: On the use of computable features for film classification. IEEE Trans. Circ. Syst. Video Technol. 15(1), 52–64 (2005). IEEE Circuits and Systems Society
Shim, H., Kang, B.G.: CAMEO - camera, audio and motion with emotion orchestration for immersive cinematography. In: Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology, ACE 2008, vol. 352, pp. 115–118. ACM, New York (2008)
Kuo, F., Chiang, M., Shan, M., Lee, S.: Emotion-based music recommendation by association discovery from film music. In: Proceedings of the 13th Annual ACM International Conference on Multimedia, MULTIMEDIA 2005, pp. 507–510. ACM, New York (2005)
Wahlster, W.: Smartkom: Foundations of Multimodal Dialogue Systems. Springer, Berlin/Heidelberg (2006)
Final Draft (2016). http://www.finaldraft.com. Accessed 11 Apr 2016
Maguire, T.: Performing Story on the Contemporary Stage. Palgrave MacMillan, Basingstoke (2015)
Fitzpatrick, L.: Gender and affect in testimonial performance. Irish Univ. Rev. 45(1), 126–140 (2015)
Solon, A.J., Mc Kevitt, P., Curran, K.: TeleMorph: a fuzzy logic approach to network-aware transmoding in mobile intelligent multimedia presentation systems. In: Dumitras, A., Radha, H., Apostolopoulos, J., Altunbasak, Y. (eds.) Special issue on Network-Aware Multimedia Processing and Communications (2007). IEEE Journal of Selected Topics in Signal Processing vol. 1, no. 2, pp. 254–263, August 2007
Bridges, B., Graham, R.: Electroacoustic music as embodied cognitive praxis: Denis Smalley’s theory of spectromorphology as an implicit theory of embodied cognition. In: Proceedings of the Electroacoustic Music Studies Network Conference, The Art of Electroaoustic Music (EMS 2015), University of Sheffield, England
Moore, P., Lyons, F., O’Hanlon, G.: The river still sings. In: Digital Arts and Humanities Conference, Ulster University, Magee, Derry/Londonderry, Northern Ireland, 10–13 September. Digital Humanities Ireland, 20 pp. (2013)
Melvin, A.: Sonic motifs, structure and identity in Steve McQueen’s hunger. Soundtrack 4(1), 23–32 (2011)
Cunningham, H.: General Architecture for Text Engineering (GATE) (2016). http://www.gate.ac.uk. Accessed 11 Apr 2016
Connexor (2016). http://www.connexor.com/nlplib. Accessed 11 Apr 2016
Tesniere, L.: Elements de syntaxe structurale. Klincksieck, Paris (1959)
Fellbaum, C.: WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998)
Strapparava, C., Valitutti, A.: WordNet-affect: an affective extension of WordNet. In: Proceedings of the 4th International Conference on Language Resources and Evaluation (LREC 2004), Lisbon, Portugal, pp. 1083–1086, 26–28 May 2004
Lexical Conceptual Structure (LCS) Database (2016). http://www.umiacs.umd.edu/~bonnie/LCS_Database_Documentation.html. Accessed 11 Apr 2016
Gelbukh, A.: EmoSenticNet (2014). http://www.gelbukh.com/emosenticnet/. Accessed 20 Feb 2014
RapidMiner: RapidMiner (2015). https://rapidminer.com/. Accessed 4 May 2015
Humanoid Animation Working Group (2016). http://www.h-anim.org. Accessed 11 Apr 2016
FreeTTS 1.2 - A speech synthesizer written entirely in the JavaTM programming language (2016). http://freetts.sourceforge.net/docs/index.php. Accessed 11 Apr 2016
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Akser, M. et al. (2017). SceneMaker: Creative Technology for Digital StoryTelling. In: Brooks, A., Brooks, E. (eds) Interactivity, Game Creation, Design, Learning, and Innovation. ArtsIT DLI 2016 2016. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 196. Springer, Cham. https://doi.org/10.1007/978-3-319-55834-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-55834-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-55833-2
Online ISBN: 978-3-319-55834-9
eBook Packages: Computer ScienceComputer Science (R0)