Skip to main content
SpringerLink
Log in

Towards affective camera control in games

  • Original Paper
  • Published:
User Modeling and User-Adapted Interaction Aims and scope Submit manuscript

Abstract

Information about interactive virtual environments, such as games, is perceived by users through a virtual camera. While most interactive applications let users control the camera, in complex navigation tasks within 3D environments users often get frustrated with the interaction. In this paper, we propose inclusion of camera control as a vital component of affective adaptive interaction in games. We investigate the impact of camera viewpoints on psychophysiology of players through preference surveys collected from a test game. Data is collected from players of a 3D prey/predator game in which player experience is directly linked to camera settings. Computational models of discrete affective states of fun, challenge, boredom, frustration, excitement, anxiety and relaxation are built on biosignal (heart rate, blood volume pulse and skin conductance) features to predict the pairwise self-reported emotional preferences of the players. For this purpose, automatic feature selection and neuro-evolutionary preference learning are combined providing highly accurate affective models. The performance of the artificial neural network models on unseen data reveals accuracies of above 80% for the majority of discrete affective states examined. The generality of the obtained models is tested in different test-bed game environments and the use of the generated models for creating adaptive affect-driven camera control in games is discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Amerson, D., Kime, S., Young, R.M.: Real-time cinematic camera control for interactive narratives. In: Advances in Computer Entertainment Technology, p. 369, 2005

  • Aylett, R., Dias, J., Paiva, A.: An affectively driven planner for synthetic characters. In: Proceedings of ICAPS, pp. 2–10. AAAI, 2006a

  • Aylett R., Louchart S., Dias J., Paiva A., Vala M., Woods S., Hall L.: Unscripted narrative for affectively driven characters. IEEE Comput. Graph. Appl. 26(3), 42–52 (2006b)

    Article  Google Scholar 

  • Bares, W., Lester, J.: Cinematographic user models for automated realtime camera control in dynamic 3d environments. In: User Modeling: In Proceedings of the Sixth International Conference, UM97, 1997

  • Bares, W.H., McDermott, S., Boudreaux, C., Thainimit, S.: Virtual 3d camera composition from frame constraints. In: ACM Multimedia, pp. 177–186, 2000

  • Blinn J.: Where am I? What am I looking at?. IEEE Comput. Graph. Appl. 8(4), 76–81 (1988)

    Article  Google Scholar 

  • Bourne O., Sattar A., Goodwin S.D.: A constraint-based autonomous 3d camera system. Constraints 13(1–2), 180–205 (2008)

    Article  MATH  Google Scholar 

  • Brosschot J.F., Thayer J.F.: Anger inhibition, cardiovascular recovery, and vagal function: A model of the link between hostility and cardiovascular disease. J. Ann. Behav. Med. 20(4), 326–332 (1998)

    Article  Google Scholar 

  • Burelli, P., Yannakakis, G.N.: Combining local and global optimisation for virtual camera control. In: Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games, pp. 403–401, Copenhagen, Denmark, August 2010a. IEEE

  • Burelli, P., Yannakakis, G.N.: Global search for occlusion minimization in virtual camera control. In: Proceedings of the 2010 IEEE World Congress on Computational Intelligence, pp. 2718–2725, Barcelona, Spain, July, 2010b. IEEE

  • Burelli, P., Di Gaspero, L., Ermetici, A., Ranon, R.: Virtual camera composition with particle swarm optimization. In: Smart Graphics, pp. 130–141. Springer-Verlag (2008)

  • Christianson, D.B., Anderson, S.E., He, L., Salesin, D., Weld, D.S., Cohen, M.F.: Declarative camera control for automatic cinematography. In: AAAI/IAAI, vol. 1, pp. 148–155, 1996

  • Christie M., Normand J.-M.: A semantic space partitioning approach to virtual camera composition. Comput. Graph. Forum 24(3), 247–256 (2005)

    Article  Google Scholar 

  • Christie, M., Olivier, P.: Camera Control in Computer Graphics. pp. 89–113. Eurographics Association, 2006

  • Christie, M., Machap, R., Normand, J.-M., Olivier, P., Pickering, J.: Virtual camera planning: a survey. In: Smart Graphics, pp. 40–52, 2005a

  • Christie, M., Machap, R., Normand, J.-M., Olivier, P., Pickering, J.: Virtual camera planning: a survey. In: Smart Graphics, pp. 40–52, 2005b

  • Conati, C., Chabbal, R., Maclaren, H.: A study on using biometric sensors for detecting user emotions in educational games. In: Proceedings of the Workshop “Assessing and Adapting to User Attitude and Affects: Why, When and How?”. In Conjunction with UMć603, 9th International Conference on User Modeling, pp. 60–65, Pittsburgh, PA, USA, 2003

  • Fairclough S.H.: Fundamentals of physiological computing. Interact. Comput. 21(1–2), 133–145 (2009)

    Article  Google Scholar 

  • Fürnkranz J., Hüllermeier E.: Preference learning. Künstliche Intelligenz 19(1), 60–61 (2005)

    Google Scholar 

  • Gleicher, M., Witkin, A.: Through-the-lens camera control. In: Computer Graphics, pp. 331–340, 1992

  • Goldberger J.J., Challapalli S., Tung R., Parker M.A., Kadish A.H.: Relationship of heart rate variability to parasympathetic effect. Circulation 103, 1977–1983 (2001)

    Google Scholar 

  • Hazlett, R.L.: Measuring emotional valence during interactive experiences: boys at video game play. In: CHI ’06: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1023–1026, New York, NY, USA, 2006. ACM Press

  • Hudlicka, E.: Affective game engines: motivation and requirements. In: Proceedings of the 4th International Conference on the Foundations of Digital Games, pp. 299–306. ACM, 2009

  • IOM Biofeedback Equipment. Wild divine. http://www.wilddivine.com/

  • Jhala, A., Young, R.M.: A discourse planning approach to cinematic camera control for narratives in virtual environments. In: AAAI, pp. 307–312, 2005

  • Jhala, A., Young, R.M.: Representational requirements for a plan based approach to virtual cinematography. In: Artificial Intelligence in Interactive Digital Entertainment (AIIDE), Marina Del Rey, CA, 2006. AAAI, AAAI Press

  • Mandryk R.L., Atkins M.S.: A fuzzy physiological approach for continuously modeling emotion during interaction with play environments. Int. J. Hum. Comput. Stud. 65, 329–347 (2007)

    Article  Google Scholar 

  • Martinez, H.P., Jhala, A., Yannakakis, G.N.: Analyzing the impact of camera viewpoint on player psychophysiology. In: Proceedings of the International Conference on Affective Computing and Intelligent Interaction, pp. 394–399, Amsterdam, The Netherlands, September 2009. IEEE

  • Massina M.M., Derkenne B., von Bernuth G.: Correlations between indices of heart rate variability in healthy children and children with congenital heart disease. Cardiology 91(2), 109–113 (1999)

    Article  Google Scholar 

  • McQuiggan, S., Lee, S., Lester, J.: Predicting user physiological response for interactive environments: an inductive approach. In: Proceedings of the 2nd Artificial Intelligence for Interactive Digital Entertainment Conference, pp. 60–65, 2006

  • Mejia-Lavalle, M., Arroyo-Figueroa, G.: Power system database feature selection using a relaxed perceptron paradigm. In: Proceedings of 5th Mexican international conference on artificial intelligence, LNCS, pp. 522–531. Springer, Berlin/Heidelberg, 2006

  • Nicholas, H., Ralf, H., Thomas, S.: A camera engine for computer games: managing the trade-off between constraint satisfaction and frame coherence. In: Proceedings Eurographics, vol. 20, pp. 174–183, 2001

  • Paiva A., Andersson G., Höök K., Mourão D., Costa M., Martinho C.: SenToy in FantasyA: designing an affective sympathetic interface to a computer game. Pers. Ubiquitous Comput. 15(4), 378–389 (2002)

    Article  Google Scholar 

  • Pedersen, C., Togelius, J., Yannakakis, G.N.: Modeling player experience in Super Mario Bros. In: Proceedings of the IEEE Symposium on Computational Intelligence and Games, pp. 132–139, Milan, Italy, September 2009. IEEE

  • Pedersen C., Togelius J., Yannakakis G.N.: Modeling player experience for content creation. IEEE Trans. Comput. Intell. AI Games 2(1), 54–67 (2010)

    Article  Google Scholar 

  • Picard R.W., Vyzas E., Healey J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)

    Article  Google Scholar 

  • Pickering, J.: Intelligent camera planning for computer graphics. PhD thesis, University of York (2002)

  • Rani, P., Sarkar, N., Liu, C.: Maintaining optimal challenge in computer games through real-time physiological feedback. In: Proceedings of the 11th International Conference on Human Computer Interaction, 2005

  • Ravaja N., Saari T., Turpeinen M., Laarni J., Salminen M., Kivikangas M.: Spatial presence and emotions during video game playing: does it matter with whom you play?. Presence Teleop. Virt. Environ. 15(4), 381–392 (2006)

    Article  Google Scholar 

  • Schwartz, M., Martinez, H.P., Yannakakis, G.N., Jhala, A.: Investigating the interplay between camera viewpoints, game information, and challenge. In: Proceedings of Artificial Intelligence and Interactive Digital Entertainment (AIIDE’09), Palo Alto, CA, October 2009. AAAI Press

  • Ware C., Osborne S.: Exploration and virtual camera control in virtual three dimensional environments. SIGGRAPH 24(2), 175–183 (1990)

    Article  Google Scholar 

  • Yannakakis, G.N. Preference learning for affective modeling. In: Proceedings of the International Conference on Affective Computing and Intelligent Interaction, pp. 126–131, Amsterdam, The Netherlands, September 2009. IEEE

  • Yannakakis G.N., Hallam J.: Entertainment modeling through physiology in physical play. Int. J. Hum. Comput. Stud. 66, 741–755 (2008)

    Article  Google Scholar 

  • Yannakakis G.N., Hallam J.: Real-time game adaptation for optimizing player satisfaction. IEEE Trans. Comput. Intell. AI Games 1(2), 121–133 (2009)

    Article  Google Scholar 

  • Yannakakis G.N., Hallam J., Lund H.H.: Entertainment capture through heart rate activity in physical interactive playgrounds. User Model. User-Adapt. Interact. Special Issue: Affective Modeling and Adaptation 18(1–2), 207–243 (2008)

    Google Scholar 

  • Yannakakis G.N., Maragoudakis M., Hallam J.: Preference learning for cognitive modeling: a case study on entertainment preferences. IEEE Syst. Man Cybernet. A: Syst Hum. 39(6), 1165–1175 (2009)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Computer Games Research, IT University of Copenhagen, Rued Langgaards Vej 7 (room 4B05), 2300, Copenhagen S, Denmark

    Georgios N. Yannakakis

  2. Center for Computer Games Research, IT University of Copenhagen, Rued Langgaards Vej 7 (room 4B09), 2300, Copenhagen S, Denmark

    Héctor P. Martínez

  3. University of California, Santa Cruz, CA, USA

    Arnav Jhala

Authors
  1. Georgios N. Yannakakis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Héctor P. Martínez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Arnav Jhala
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Georgios N. Yannakakis.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yannakakis, G.N., Martínez, H.P. & Jhala, A. Towards affective camera control in games. User Model User-Adap Inter 20, 313–340 (2010). https://doi.org/10.1007/s11257-010-9078-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11257-010-9078-0

Keywords

Search

Navigation