Abstract
Decisions in designing artificial social interactants to reproduce culturally-specific forms of human sociality evince a range of conceptions of the norms and cognitive processes involved in the human social interactions themselves. Regarding the use of machine learning (ML) in such systems, decisions whether or not to use this approach implicitly presents questions on the nature of the interpersonal adaptation that takes place and indicate a range of conceptions of the values which structure these interactions. In the design of virtual performers of musical free improvisation, several designers assume that the experience of equal partnership between improvisers can only be afforded through deployment of ML in such systems. By contrast, tests of agents not based in ML reveal that human beings experience illusions of “adaptation” in interactions with systems which lack any adaptive capacity. Such results suggest that HCI research with artificial social interactants may be used to raise new questions about the nature of human interaction and interpersonal adaptation in the formation of relationships over time.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Again, in the case of a virtual free improviser, the designer does retain control over the system’s behavior. But unlike the control exerted over a system built to function as an “instrument” (Rowe 1992), a “player” system cannot be directly controlled on a moment to moment basis.
- 2.
While peer criticism is not seen in this way by participants of other egalitarian social projects (Chaudron 1984; Snyder and Fessler 2014), improvisers view criticism as a kind of speech act that instantly nullifies equality by placing the speaker in a position of authority with regard to the actions of the addressee.
- 3.
Similarity is judged by an analytical comparison of incoming and stores phrases on the basis of their loudness, pitch content, spectral centroid (or “brightness”), noisiness (or ratio of tone to noise) as well as the mean and range for these values in a given phrase.
- 4.
As Eitan Wilf notes (2013b), this is a very specific notion of the term “style” which essentially predetermines what can and cannot even count as style.
- 5.
These criteria were (1) the degree to which the system inspired you to respond to its playing, (2) satisfaction with unexpected or surprising responses from the system, (3) the overall sense that the interaction was meaningful, and (4) whether the system’s responses seemed relevant or random.
- 6.
The main question for this experiment focused on the issue of whether or not the active listening of another improviser increases or decreases an improviser’s level of aesthetic or social-interactional satisfaction of the experience of playing music. In order to investigate this question, in a random selection of the 10 takes, the system was set to listen to a prerecorded track (and therefore, not listen to the sonic events of the current take) whereas in the remainder of takes the system listened to the combination of itself and the human performer, this being the way the system was originally designed to receive input in a performance setting. (For further discussion, see Banerji 2012.)
- 7.
To be clear, the quantitative data from the experiment does not necessarily suggest a clear sense of evolution in the player’s experience across the set of 10 takes. However, the quantitatively-graded criteria do not directly correspond with positive or negative sentiments about the system’s interactivity as an experience. With the exception of one criterion (“meaningfulness”), the criteria evaluated refer to the player’s observations about the interaction overall and do not inherently convey judgments about the aesthetic value of the experience.
- 8.
These were mainly minor tweaks in order to enable the system to start and stop at the push of a single button. Such changes had no effect on how the system would begin to play, behave during the improvisation itself, or how it would “end” pieces.
- 9.
When improvisers meet in private, it is rare for them to “rehearse” materials. Instead, it is far more common to play for a duration similar to that of an actual concert (ranging from 20 minutes to an hour) without break. Afterwards, some discussion may take place about the music. However, given the fact that recalling specific details of such a long duration of improvisation, in which temporal coordination (i.e., pulse) is often absent and each player is engaged in significantly independent lines of action, it is doubtful that one will have a clear recollection of specific events. Therefore, it is unlikely as well that one will have the epistemological certainty required to make a comment about what has happened and how it should have been done differently (Corbett 1994).
References
Amershi S, Cakmak M, Knox WB, Kulesza T (2014) Power to the people: the role of humans in interactive machine learning. AI Mag 35(4):105–120
Andersen K, Brooks R (2003) Special guest: Rodney brooks. Studio 360: WNYC. National Public Radio, Washington, DC
Assayag G, Dubnov S (2004) Using factor oracles for machine improvisation. Soft Comput 8(9):604–610
Assayag G, Bloch G, Chemillier M, Cont A, Dubnov S (2006) Omax brothers: a dynamic topology of agents for improvization learning. In: Proceedings of the 1st ACM workshop on audio and music computing multimedia, pp 125–132
Assayag G, Bloch G, Cont A, Dubnov S (2010) Interaction with machine improvisation. In: Argamon S, Burns K, Dubnov S (eds) The structure of style. Springer, Berlin and Heidelberg, pp 219–245
Backstrom MJ (2013). The field of cultural production and the limits of freedom in improvisation. Crit Stud Improv/Études critiques en improvisation 9(1)
Bailey D (1980/1993) Improvisation: its nature and practice in music. Da Capo Press, New York
Banerji R (2010) Maxine Banerji: the mutually beneficial practices of youth development and interactive systems development. eContact! J Can Electroacoust Commun 12(3). https://econtact.ca/12_3/banerji_maxine.html
Banerji R (2012) Maxine’s Turing test: a player-program as co-ethnographer of socio-aesthetic interaction in improvised music. In: Proceedings of the Artificial Intelligence and Interactive Digital Entertainment Conference (AIIDE’12)
Banerji R (2016) Balancing defiance and cooperation: the design and human critique of a virtual free improviser. In: Proceedings of the international computer music conference, pp 49–54
Banerji R, Maxine, the Astromusicologist (2014) Palmer square. pfMentum, San Diego
Bardzell J, Bardzell S (2011) Pleasure is your birthright: digitally enabled designer sex toys as a case of third-wave HCI. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 257–266
Bardzell J, Bardzell S (2016) Humanistic HCI. Interactions 23(2):20–29
Baumer EP, Silberman M (2011) When the implication is not to design (technology). In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 2271–2274
Beins B (2011) Entwurf und Ereignis (Scheme and Event). In: Beins B, Kesten C, Nauck G, Neumann A (eds) echtzeitmusik berlin: selbstbestimmung einer szene (self-defining a scene). Wolke Verlag, Hofheim
Bernieri F, Gillis JS (1995) The judgment of rapport: a cross-cultural comparison between Americans and Greeks. J Nonverbal Behav 19(2):115–130
Bernieri FJ, Gillis JS, Davis JM, Grahe JE (1996) Dyad rapport and the accuracy of its judgment across situations: a lens model analysis. J Pers Soc Psychol 71(1):110
Blackwell T, Bentley P (2002) Improvised music with swarms. In: Proceedings of the World conference on computational intelligence, pp 1462–1467
Blackwell T, Young M (2004) Self-organised music. Organised Sound 9(2):123–136
Bødker S (2006) When second wave HCI meets third wave challenges. In: Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles. ACM, New York, pp 1–8
Boehm C (1993) Egalitarian behavior and reverse dominance hierarchy. Curr Anthropol 34(3):227–254
Boehner K, DePaula R, Dourish P, Sengers P (2007) How emotion is made and measured. Int J Hum Comput Stud 65(4):275–291
Borgo D (2002) Synergy and Surrealestate: the orderly disorder of free improvisation. Pac Rev Ethnomusicol 10:1–24
Bornstein RF (1989) Exposure and affect: overview and meta-analysis of research, 1968–1987. Psychol Bull 106(2):265–289
Bown O (2011) Experiments in modular design for the creative composition of live algorithms. Comput Music J 35(3):73–85
Bown O (2015) Player responses to a live algorithm: conceptualising computational creativity without recourse to human comparisons? In: Proceedings of the sixth international conference on computational creativity
Brinner B (1995) Knowing music, making music: Javanese gamelan and the theory of musical competence and interaction. University of Chicago Press, Chicago
Burrell J (2016) How the machine ‘thinks’: understanding opacity in machine learning algorithms. Big Data Soc 3(1):1–12
Canonne C (2013) Focal points in collective free improvisation. Perspect New Music 51(1):40–55
Canonne C, Garnier N (2015) Individual decisions and perceived form in collective free improvisation. J New Music Res 44(2):145–167
Cappella JN (1990) On defining conversational coordination and rapport. Psychol Inq 1(4):303–305
Carey B (2012) Designing for cumulative interactivity: the _derivations system. In: Proceedings of the international conference on new interfaces for musical expression
Carles P, Comolli J-L (2015 [1971]) Free jazz/Black power. University Press of Mississippi, Mississippi
Casal DP (2008) Time after time: short-circuiting the emotional distance between algorithm and human improvisors. In: Proceedings of the international computer music conference
Casal DP, Morelli D (2007). Remembering the future: applications of genetic co-evolution in music improvisation. In: Proceedings of the European conference on artificial life
Cassell J, Gill AJ, Tepper PA (2007) Coordination in conversation and rapport. In: Proceedings of the workshop on embodied language processing. Association for Computational Linguistics, pp 41–50
Chadabe J (1997) Interaction. In: Electric sound: the past and promise of electronic music. Prentice-Hall, Upper Saddle River, pp 286–323
Chaudron C (1984) The effects of feedback on students’ composition revisions. RELC J 15(2):1–14
Collins N (2008) Reinforcement learning for live musical agents. In: Proceedings of the International Computer Music Conference (ICMC), Belfast
Cont A, Dubnov S, Assayag G (2006) A framework for anticipatory machine improvisation and style imitation. In: Proceedings of the conference on Anticipatory Behavior in Adaptive Learning Systems (ABiALS)
Corbett J (1994) Evan Parker: saxophone botany. In: Extended play: sounding off from John Cage to Dr. Funkenstein. Duke University Press, Duke, pp 201–208
Corbett J (2016) A Listener’s guide to free improvisation. University of Chicago Press, Chicago
Crabtree A, Rodden T, Tolmie P, Button G (2009) Ethnography considered harmful. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 879–888
Dourish P (2006) Implications for design. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 541–550
Edwards PN (1997) The closed world: computers and the politics of discourse in Cold War America. MIT Press, Cambridge, MA
Eiben AE, Smith JE (2003) Introduction to evolutionary computing. Springer, Berlin/Heidelberg
Eklund R (2002) Ingressive speech as an indication that humans are talking to humans (and not to machines). In: Proceedings of the 7th International Conference on Spoken Language Processing (ICSLP)
Fails JA, Olsen DR Jr (2003) Interactive machine learning. In: Proceedings of the 8th international conference on Intelligent user interfaces. ACM, New York, pp 39–45
Fiebrink R, Cook PR, Trueman D (2011) Human model evaluation in interactive supervised learning. In: Proceedings of the SIGCHI conference on human factors in computing systems, pp 147–156
Fischlin D, Heble A, Lipsitz G (2013) Take two: sounds of surprise; rights, risks and responsibilities in improvised music. In: The fierce urgency of now: improvisation, rights, and the ethics of co-creation. Duke University Press, Duke, pp 203–219
Forsythe D (2002) Studying those who study us: an anthropologist in the world of artificial intelligence. Stanford University Press, Stanford
Friedman B, Nissenbaum H (1996) Bias in computer systems. ACM Trans Inf Syst (TOIS) 14(3):330–347
Fuchsberger V, Moser C, Tscheligi M (2012) Values in action (ViA): combining usability, user experience and user acceptance. In: CHI’12 extended abstracts on human factors in computing systems. ACM, New York, pp 1793–1798
Giles H, Coupland N, Coupland I (eds) (1991) Contexts of accommodation: developments in applied sociolinguistics. Cambridge University Press, Cambridge
Goffman E (1955) On face-work: an analysis of ritual elements in social interaction. Psychiatry Interpersonal Biol Process 18(3):213–231
Goffman E (1967) Interaction ritual: essays in face-to-face behavior. Anchor Books, New York
Grahe JE, Bernieri FJ (1999) The importance of nonverbal cues in judging rapport. J Nonverbal Behav 23(4):253–269
Gratch J et al (2006) Virtual Rapport. In: Gratch J, Young M, Aylett R, Ballin D, Olivier P (eds) Intelligent Virtual Agents. IVA. Springer, Berlin/Heidelberg, pp 14–27
Gratch J, Wang N, Gerten, J., Fast, E., & Duffy, R. (2007). Creating rapport with virtual agents. In: Pelachaud C, Martin J-C, André E, Chollet G, Karpouzis K, Pelé D (eds) International Workshop on Intelligent Virtual Agents. Springer, Berlin, pp 125–138
Harrison S, Tatar D, Sengers P (2007) The three paradigms of HCI. alt. chi
Helmreich S (2001) After culture: reflections on the apparition of anthropology in artificial life, a science of simulation. Cult Anthropol 16(4):612–627
Hofstadter DR (1995) The ineradicable Eliza effect and its dangers. In: Hofstadter DR (ed) Fluid concepts and creative analogies: computer models of the fundamental mechanisms of thought. Basic books, New York, pp 155–168
Hsu W (2005) Using timbre in a computer-based improvisation system. In: Proceedings of the international computer music conference
Hsu W (2010) Strategies for managing timbre and interaction in automatic improvisation systems. Leonardo Music J 20:33–39
Hsu W, Sosnick M (2009) Evaluating interactive music systems: an HCI approach. In: Proceedings of the international conference on New Interfaces in Musical Expression (NIME)
Huang L, Morency L-P, Gratch J (2011) Virtual rapport 2.0. In: Vilhjálmsson HH, Kopp S, Marsella S, Thórisson KR (eds) Proceedings of the 10th International Conference on Intelligent Virtual Agents. Springer, Berlin/Heidelberg, pp 68–79
Huron DB (2006) Sweet anticipation: music and the psychology of expectation. MIT Press, Cambridge, MA
Husserl E (1913/2012) Ideas: general introduction to pure phenomenology (WRB Gibson, Trans.). Routledge
Kendon A (1990) Conducting interaction: patterns of behavior in focused encounters. Cambridge University Press, Cambridge
Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of the IEEE conference on neural networks, pp 1942–1948
Kiousis S (2002) Interactivity: a concept explication. New Media Soc 4(3):355–383. https://doi.org/10.1177/146144480200400303
Kofsky F (1970) Black nationalism and the revolution in music. Pathfinder Press, New York
Lakin JL, Chartrand TL (2003) Using nonconscious behavioral mimicry to create affiliation and rapport. Psychol Sci 14(4):334–339
Lange BR (2011) Teaching the ethics of free improvisation. Crit Stud Improvis/Études critiques en improvisation 7(2)
Langton CG (1997) Artificial life: an overview. MIT Press, Cambridge, MA
Lewis GE (1993). Voyager (G Lewis, R Mitchell, V. I. V. I. System, Trans.). Avant, Japan
Lewis GE (1999) Interacting with latter-day musical automata. Contemp Music Rev 18(3):99–112
Lewis GE (2000) Too many notes: computers, complexity and culture inVoyager. Leonardo Music J 10:33–39
Lewis GE (2007) Mobilitas animi: improvising technologies, intending chance. Parallax 13(4):108–122
Linson A (2014) Investigating the cognitive foundations of collaborative musical free improvisation: experimental case studies using a novel application of the subsumption architecture. Doctoral thesis, The Open University
Linson A, Dobbyn C, Lewis GE, Laney R (2015) A Subsumption agent for collaborative free improvisation. Comput Music J 39(4):96–115
Maatman R, Gratch J, Marsella S (2005) Natural behavior of a listening agent. In: Intelligent virtual agents. Springer, Heidelberg, pp 25–36
Monson I (1996) Saying something: jazz improvisation and interaction. University of Chicago Press, Chicago
Moreland RL, Zajonc RB (1982) Exposure effects in person perception: familiarity, similarity, and attraction. J Exp Soc Psychol 18(5):395–415
Morrissey K, Kirakowski J (2013) ‘Realness’ in Chatbots: establishing quantifiable criteria. In: Kurosu M (ed) Proceedings of the 15th international conference on human-computer interaction. Springer, Berlin/Heidelberg, pp 87–96
Nissenbaum H (2001) How computer systems embody values. Computer 34(3):120–119
Pachet F (2003) The continuator: musical interaction with style. J New Music Res 32(3):333–341. https://doi.org/10.1076/jnmr.32.3.333.16861
Pras A, Schober MF, Spiro N (2017) What about their performance do free jazz improvisers agree upon? A case study [Original Research]. Front Psychol 8(966). https://doi.org/10.3389/fpsyg.2017.00966
Reis HT, Maniaci MR, Caprariello PA, Eastwick PW, Finkel EJ (2011) Familiarity does indeed promote attraction in live interaction. J Pers Soc Psychol 101(3):557
Rodriguez AW (2016) Harmolodic pedagogy and the challenge of omni-musicality. Jazz Perspect 9(2):173–192
Rowe R (1992) Interactive music systems: machine listening and composing. MIT Press, Cambridge, MA
Seaver N (2012) Algorithmic recommendations and synaptic functions. Limn 1(2)
Sengers P, Kaye J, Boehner K, Fairbank J, Gay G, Medynskiy Y et al (2004) Culturally embedded computing. IEEE Pervasive Comput 3(1):14–21
Serenko A, Bontis N (2011) What’s familiar is excellent: the impact of exposure effect on perceived journal quality. J Informet 5(1):219–223
Seyfert R, Roberge J (2016) Algorithmic cultures: essays on meaning, performance and new technologies. Routledge, New York
Snyder JK, Fessler DM (2014) Narcotics anonymous: anonymity, admiration, and prestige in an egalitarian community. Ethos 42(4):440–459
Spellman AB (1966) Four lives in the bebop business. Pantheon Books, New York
Stanyek J (1999) Articulating intercultural free improvisation: Evan Parker’s synergetics project. Resonance 7(2):44–47
Steinbeck P (2010) “Patience, sincerity, and consistency”: Fred Anderson’s musical and social practices. Crit Stud Improvis/Études critiques en improvisation 6(2)
Suchman L (2006) Human-machine reconfigurations: plans and situated actions. Cambridge University Press, Cambridge
Sunardi C (2011) Negotiating authority and articulating gender: performer interaction in Malang, East Java. Ethnomusicology 55(1):32–54
Tickle-Degnen L, Rosenthal R (1990) The nature of rapport and its nonverbal correlates. Psychol Inq 1(4):285–293
Waterman E (2008) Naked intimacy: eroticism, improvisation, and gender. Crit Stud Improvis/Études critiques en improvisation 4(2)
Weizenbaum J (1976) Computer power and human reason. W. H. Freeman and Company, San Francisco
Wilf E (2013a) Sociable robots, jazz music, and divination: contingency as a cultural resource for negotiating problems of intentionality. Am Ethnol 40(4):605–618
Wilf E (2013b) Toward an anthropology of computer-mediated, algorithmic forms of sociality. Curr Anthropol 54(6):716–739
Wilson GB, MacDonald RA (2015) Musical choices during group free improvisation: a qualitative psychological investigation. Psychol Music:1–15
Woodburn J (1982) Egalitarian societies. Man 17(3):431–451
Yee-King MJ (2007) An automated music improviser using a genetic algorithm driven synthesis engine. In: Giacobini M (ed) Applications of evolutionary computing (EvoWorkshops 2007) proceedings. Springer, Berlin/Heidelberg, pp 567–576
Young M (2008) NN music: improvising with a ‘living’ computer. In: Kronland-Martinet R, Ystad S, Jensen K (eds) Computer music modeling and retrieval: sense of sounds. Springer, Berlin, pp 337–350
Young M (2010) Identity and intimacy in human-computer improvisation. Leonardo Music J 20(online supplement)
Zajonc RB (1968) Attitudinal effects of mere exposure. J Pers Soc Psychol 9(2, part 2):1–27
Acknowledgements
Earlier versions of this chapter benefitted from the insightful critiques of Nick Seaver, Zachary Chase Lipton, as well as editors and reviewers of this volume. Financial support for this project came from the Fulbright U.S. Young Journalist’s Fellowship in Germany, the Berlin Program for Advanced German and European Studies, and the Berkeley-Mellon Fellowship.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Banerji, R. (2018). De-instrumentalizing HCI: Social Psychology, Rapport Formation, and Interactions with Artificial Social Agents. In: Filimowicz, M., Tzankova, V. (eds) New Directions in Third Wave Human-Computer Interaction: Volume 1 - Technologies. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-319-73356-2_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-73356-2_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73355-5
Online ISBN: 978-3-319-73356-2
eBook Packages: Computer ScienceComputer Science (R0)