Skip to main content
SpringerLink
Log in

Anticipatory Behavior of Software Agents in Self-organizing Negotiations

  • Chapter
  • First Online:
Anticipation Across Disciplines

Part of the book series: Cognitive Systems Monographs ((COSMOS,volume 29))

Abstract

Software agents are a well-established approach for modeling autonomous entities in distributed artificial intelligence. Iterated negotiations allow for coordinating the activities of multiple autonomous agents by means of repeated interactions. However, if several agents interact concurrently, the participants’ activities can mutually influence each other. This leads to poor coordination results. In this paper, we discuss these interrelations and propose a self-organization approach to cope with that problem. To that end, we apply distributed reinforcement learning as a feedback mechanism to the agents’ decision-making process. This enables the agents to use their experiences from previous activities to anticipate the results of potential future actions. They mutually adapt their behaviors to each other which results in the emergence of social order within the multiagent system. We empirically evaluate the dynamics of that process in a multiagent resource allocation scenario. The results show that the agents successfully anticipate the reactions to their activities in that dynamic and partially observable negotiation environment. This enables them to maximize their payoffs and to drastically outperform non-anticipating agents.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    A famous example for this is the prisoner’s dilemma in which the equilibrium point is the only strategy combination not belonging to the Pareto frontier.

  2. 2.

    In this state of double contingency, both participants are unable to act because each of their activities depends on the other’s previous actions and they lack any existing expectations for selecting them. However, Luhmann notes that this is a highly unstable fixpoint of the interaction’s dynamics which never actually occurs in real encounters [15, 17]. Instead, every slight action allows for generating initial expectations which facilitate the emergence of social order.

  3. 3.

    All deviations are half-widths of the 99 % confidence interval.

References

  1. Berndt, J.O.: Self-organizing supply networks: autonomous agent coordination based on expectations. In: Filipe, J., Fred, A. (eds.) ICAART 2011, vol. 2, pp. 104–113. SciTePress, Rome (2011)

    Google Scholar 

  2. Berndt, J.O.: Self-organizing logistics process control: an agent-based approach. In: Filipe, J., Fred, A. (eds.) Agents and Artificial Intelligence, pp. 397–412. Springer, Berlin (2013)

    Chapter  Google Scholar 

  3. Berndt, J.O., Herzog, O.: Efficient multiagent coordination in dynamic environments. In: Boissier, O., Bradshaw, J., Cao, L., Fischer, K., Hacid, M.S. (eds.) WI-IAT 2011, pp. 188–195. IEEE Computer Society, Lyon (2011)

    Google Scholar 

  4. Berndt, J.O., Herzog, O.: Distributed learning of best response behaviors in concurrent iterated many-object negotiations. In: Timm, I.J., Guttmann, C. (eds.) MATES 2012, pp. 15–29. Springer, Berlin (2012)

    Google Scholar 

  5. Berndt, J.O., Herzog, O.: Distributed reinforcement learning for optimizing resource allocation in autonomous logistics processes. In: Kreowski, H.J., Scholz-Reiter, B., Thoben, K.D. (eds.) LDIC 2012, pp. 429–439. Springer, Berlin (2013)

    Google Scholar 

  6. Buşoniu, L., Babuška, R., De Schutter, B.: Multi-agent reinforcement learning: an overview. In: Srinivasan, D., Jain, L. (eds.) Innovations in Multi-Agent Systems and Applications—1, pp. 183–221. Springer, Heidelberg (2010)

    Google Scholar 

  7. Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multiagent systems. In: AAAI 1998. pp. 746–752. Madison, USA (1998)

    Google Scholar 

  8. Cramton, P., Shoham, Y., Steinberg, R. (eds.): Combinatorial Auctions. The MIT Press, Cambridge (2006)

    MATH  Google Scholar 

  9. Endriss, U., Maudet, N., Sadri, F., Toni, F.: Negotiating socially optimal allocations of resources. J. Artif. Intell. Res. 25, 315–348 (2006)

    MathSciNet  Google Scholar 

  10. Faratin, P., Sierra, C., Jennings, N.R.: Negotiation decision functions for autonomous agents. Robot. Auton. Syst. 24(3–4), 159–182 (1998)

    Article  Google Scholar 

  11. Foundation for Intelligent Physical Agents: FIPA Iterated Contract Net Interaction Protocol Specification. Standard (2002), document No. SC00030H

    Google Scholar 

  12. Gjerstad, S., Dickhaut, J.: Price formation in double auctions. Game. Econ. Behav. 22(1), 1–29 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  13. Jennings, N.R., Faratin, P., Lomuscio, A.R., Parsons, S., Wooldridge, M.J., Sierra, C.: Automated negotiation: prospects. Methods Chall. Group Decis. Negoti. 10, 199–215 (2001)

    Article  Google Scholar 

  14. Luckhart, C., Irani, K.B.: An algorithmic solution of N-person games. In: AAAI 1986. vol. 1, pp. 158–162. Morgan Kaufmann, Philadelphia, USA (1986)

    Google Scholar 

  15. Luhmann, N.: Soziale Systeme. Grundriß einer allgemeinen Theorie. Suhrkamp, Frankfurt (1984)

    Google Scholar 

  16. Luhmann, N.: Probleme mit operativer Schließung. In: Luhmann, N. (ed.) Die Soziologie und der Mensch, Soziologische Aufklärung, vol. 6, pp. 12–24. Westdeutscher Verlag, Opladen (1995)

    Google Scholar 

  17. Luhmann, N.: Social Systems. Stanford University Press, Stanford (1995)

    Google Scholar 

  18. Mazur, D.R.: Combinatorics. A guided tour. MAA Textbooks, The Mathematical Association of America, Washington (2010)

    MATH  Google Scholar 

  19. Nash, J.: Non-cooperative Games. Ann. Math. 54(2), 286–295 (1950)

    Article  MathSciNet  Google Scholar 

  20. Porter, R., Nudelman, E., Shoham, Y.: Simple search methods for finding a Nash equilibrium. Game. Econ. Behav. 63(2), 642–662 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  21. Ramezani, S., Endriss, U.: Nash social welfare in multiagent resource allocation. In: David, E., Gerding, E., Sarne, D., Shehory, O. (eds.) Agent-Mediated Electronic Commerce, pp. 117–131. Springer, Heidelberg (2010)

    Google Scholar 

  22. Schuldt, A.: Multiagent coordination enabling autonomous logistics. Springer, Heidelberg (2011)

    Book  MATH  Google Scholar 

  23. Schuldt, A., Berndt, J.O., Herzog, O.: The interaction effort in autonomous logistics processes: potential and limitations for cooperation. In: Hülsmann, M., Scholz-Reiter, B., Windt, K. (eds.) Autonomous Cooperation and Control in Logistics, pp. 77–90. Springer, Berlin (2011)

    Chapter  Google Scholar 

  24. Schuldt, A., Gehrke, J.D., Werner, S.: Designing a simulation middleware for FIPA multiagent systems. In: Jain, L., Gini, M., Faltings, B.B., Terano, T., Zhang, C., Cercone, N., Cao, L. (eds.) WI-IAT 2008, pp. 109–113. IEEE Computer Society Press, Sydney (2008)

    Google Scholar 

  25. Schuldt, A., Werner, S.: Distributed Clustering of Autonomous Shipping Containers by Concept, Location, and Time. In: Müller, J.P., Petta, P., Klusch, M., Georgeff, M. (eds.) MATES 2007, pp. 121–132. Springer, Berlin (2007)

    Google Scholar 

  26. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. The MIT Press, Cambridge (1998)

    Google Scholar 

  27. van Bragt, D.D.B., La Poutré, J.A.: Why Agents for Automated Negotiations Should Be Adaptive. Netnomics 5(2), 101–118 (2003)

    Article  Google Scholar 

  28. von Neumann, J.: Zur Theorie der Gesellschaftsspiele. Math. Ann. 100, 295–320 (1928)

    Article  MATH  MathSciNet  Google Scholar 

  29. von Neumann, J., Morgenstern, O.: Theory of Games and Economic Behavior. Princeton University Press, Princeton (1944)

    MATH  Google Scholar 

  30. Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)

    MATH  Google Scholar 

  31. Wooldridge, M., Jennings, N.R.: Intelligent agents: theory and practice. Knowl. Eng. Rev. 10(2), 115–152 (1995)

    Article  Google Scholar 

  32. Wooldridge, M., Jennings, N.R.: The cooperative problem-solving process. J. Logic Comput. 9(4), 563–592 (1999)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Center for Computing and Communication Technologies (TZI), Universität Bremen, Bremen, Germany

    Jan Ole Berndt & Otthein Herzog

Authors
  1. Jan Ole Berndt
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Otthein Herzog
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jan Ole Berndt .

Editor information

Editors and Affiliations

  1. University of Texas at Dallas, antÉ—Institute for Research in Anticipatory Systems, Richardson, Texas, USA

    Mihai Nadin

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Berndt, J.O., Herzog, O. (2016). Anticipatory Behavior of Software Agents in Self-organizing Negotiations. In: Nadin, M. (eds) Anticipation Across Disciplines. Cognitive Systems Monographs, vol 29. Springer, Cham. https://doi.org/10.1007/978-3-319-22599-9_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-22599-9_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-22598-2

  • Online ISBN: 978-3-319-22599-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation