Abstract
In cooperative MASs, the interests of individual agents are usually consistent with that of the overall system. In the cases of conflicting interest, each agent is assumed to have the willingness to cooperate toward a common goal of the system even at the cost of sacrificing its own benefits. Therefore, the behaviors of each agent in cooperative environments can be determined by the designer(s) of the system, which thus allows for intricate coordination strategies to be implemented beforehand. To achieve effective coordinations among agents, one traditional approach is to employ a superagent to determine the behaviors for all other agents in the system. However, there exist a number of disadvantages by adopting this approach. First, the scalability problem will become serious when the number of agents is significantly increased, since the computational space of the superagent increases exponentially to the number of agents. Second, it explicitly requires the superagent to be able to communicate with all agents in the system and has the global information, which may not be possible in distributed environments, where the communication cost can be very high. Lastly, it makes the system very vulnerable since the malfunction of the superagent would lead to the failure of the whole system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
ε is assumed to be a positive real number.
- 2.
The implicit assumption here is that each agent knows the potentially maximum payoff by taking each action. We claim that this assumption is reasonable since the agents can have access to this information in advance in practical application such as resource allocation problem with conflicting interests, which can be naturally modeled by conflicting-interest games.
- 3.
Here we assume that the agents are always truthfully revealing their payoff information. There are various mechanisms (e.g., adopt a truthful third party to supervise this process) to guarantee this, which is beyond the scope of the book.
- 4.
The value of m reflects the tolerance degree of the agents to malicious agents and thus can be set to different values accordingly.
- 5.
The following analysis assumes that both agents in the system adopt this adaptive strategy and thus this mechanism is not activated.
- 6.
The reason behind is stated as follows: since agent 1 chooses action R2 at the end of period t, attr R1 < attr R2 is satisfied at this time, and the difference between attr R1 and attr R2 can be expressed as \(\text{diff} =\mathrm{ attr}_{R2} -\mathrm{ attr}_{R1}\). After agent 1 increases both of its attitudes by \(\Delta U_{+}\), the difference between attr R1 and attr R2 is changed to \(\text{diff}^{{\prime}} =\mathrm{ attr}_{R2} -\mathrm{ attr}_{R1} + \Delta U_{+}(h_{R1} - h_{R2}) + \Delta U_{+}(u_{R2} - u_{R1})\). Since h R2 = 1 at this time, h R1 − h R2 ≤ 0, and also u R2 − u R1 < 0, we can easily see that \(\text{diff}^{{\prime}} <\text{diff}\).
- 7.
If agent 1 chooses its action depending on its non-conflict ratio only, the number of time steps that it sticks to action R1 is no larger than the length of history h l .
- 8.
Since strict fairness (ε = 0) can be achieved in this game, we will use the term fairness only in the following section.
- 9.
The payoffs specified in Examples 1 and 2 represent the agents’ material payoffs.
- 10.
Note that it is possible to perform higher-level modeling here. However, as we will show, modeling at the second level is already enough and also can keep the model analysis tractable.
- 11.
When u j h(b j ) = u j min(b j ), agent j would always receive the same payoff no matter which action agent i chooses, and thus there is no issue of kindness, and \(K_{i}(a_{i},b_{j},\alpha _{i}^{{\prime}},\beta _{i}^{{\prime}}) = 0\).
- 12.
Note that we only discuss the case of mixed strategy here, since a mixed strategy is also a behavioral strategy in this simple example, and vice versa.
- 13.
Here we simply say that fairness is achieved if the players’ payoffs are equal since we focus on symmetric games only. More general definition of ε-fairness [2] may be adopted in general-sum games as future work.
- 14.
Note that here the ranges of α i t and β i t cover the negative ranges since there exist some people who prefer to see other people suffer (have less payoff than himself) or be altruistic (give other people more payoff).
- 15.
Note that the notation here is a little different from the example PD game in Fig. 3.16b, since under this general constraints, the Nash equilibrium in the PD game is (C, C) instead of (D, D).
- 16.
It is based on the known constraint of 2a < c + d.
- 17.
The readers can verify it based on the known constraints of c + d > 2a and n ≥ 2. The reason of n ≥ 2 is that it needs at least two steps to return to his fairness strategy for player 2 before deviating again (from state (C, C) to state (C, D) and then to state (D, C)).
References
Shoham Y, Powers R, Grenager T (2007) If multi-agent learning is the answer, what is the question? Artif Intell 171:365–377
Hao JY, Leung HF (2010) Strategy and fairness in repeated two-agent interaction. In: Proceedings of ICTAI’10, Arras, pp 3–6
Hao JY, Leung HF (2012) Incorporating fairness into infinitely repeated games with conflicting interests for conflicts elimination. In: Proceedings of ICTAI’12, Athens, pp 314–321
Hao JY, Leung HF (2012) Incorporating fairness into agent interactions modeled as two-player normal-form games. In: Proceedings of PRICAI’12, Kuching
Watkins CJCH, Dayan PD (1992) Q-learning. Mach Learn 3:279–292
Claus C, Boutilier C (1998) The dynamics of reinforcement learning in cooperative multiagent systems. In: Proceedings of AAAI’98, Madison, pp 746–752
Littman M (1994) Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of ICML’94, New Brunswick, pp 322–328
Nowé A, Parent J, Verbeeck K (2001) Social agents playing a periodical policy. In: Proceedings of ECML’01, vol 2176, pp 382–393. Springer, Berlin/New York
Verbeeck K, Nowé A, Parent J, Tuyls K (2006) Exploring selfish reinforcement learning in repeated games with stochastic rewards. Auton Agents Multi-agent Syst 14:239–269
Fehr E, Schmidt KM (1999) A theory of fairness, competition and cooperation. Q J Econ 114:817–868
Kahneman D, Tversky A (1979) Prospect theory: an analysis of decison under risk. Econometrica 47(2):263–291
Gagne R (1985) The conditions of learning, 4th edn. Holt, Rinehart and Winston, New York
Chevaleyre Y, Dunne PE et al (2006) Issues in multiagent resource allocation. Informatica 30:3–31
Simon H (1972) Theories of bounded rationality. Decis Organ 1:161–176
Chevaleyre Y, Endriss U, Lang J, Maudet N (2007) A short introduction to computational social choice. SOFSEM 4362:51–69
Rabin M (1993) Incorporating fairness into game theory and economics. Am Econ Rev 83:1281–1302
Dawes RM, Thaleri RH (1988) Anomalies: cooperation. J Econ Perspect 2:187–198
Thaler RH (1985) Mental accounting and consumer choice. Mark Sci 4:199–214
Kahneman D, Knetsch JL, Thaler RH (1986) Fairness as a constraint on profit seeking: entitlements in the market. Am Econ Rev 76:728–741
Camerer C, Thaler RH (1995) Ultimatums, dictators, and manners. J Econ Perspect 9:209–219
Agell J, Lundberg P (1995) Theories of pay and unemployment: survery evidence from swedish manufacturing firms. Scand J Econ XCVII:295–308
Bewley T (1995) A depressed labor market as explained by participants. Am Econ Rev Pap Proc LXXXV:250–254
Leventhal G, Anderson D (1970) Self-interest and the maintenance of equity. J Personal Soc Psychol 15:57–62
Geanakoplos J, Pearce D, Stacchetti E (1989) Psychological games and sequential rationality. Games Econ Behav 1:60–79
Bolton GE, Ockenfels A (2000) Erc-a theory of equity, reciprocity and competition. Am Econ Rev 90:166–193
Falka A, Fischbache U (2006) A theory of reciprocity. Games Econ Behav 54:293–315
Shoham Y, Power WR, Grenager T (2007) If multi-agent learning is the answer, what is the question? Artif Intell 171(7):365–377
Blount S (1995) When social outcomes aren’t fair: the effects of causual attributions on preferences. Organ Behav Hum Decis Process LXIII:131–144
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2016 Higher Education Press, Beijing and Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Hao, J., Leung, Hf. (2016). Fairness in Cooperative Multiagent Systems. In: Interactions in Multiagent Systems: Fairness, Social Optimality and Individual Rationality. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-49470-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-662-49470-7_3
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-49468-4
Online ISBN: 978-3-662-49470-7
eBook Packages: EngineeringEngineering (R0)