An Adaptive Witness Selection Method for Reputation-Based Trust Models

  • Han Yu
  • Zhiqi Shen
  • Bo An
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7455)


In distributed multi-agent systems where agents need to cooperate with each other in order to thrive, accurately estimating a potential partner’s trustworthiness is vital to an agent’s wellbeing. Many distributed reputation-based agent trust models have been proposed. It has been agreed that testimony sharing is a useful way for agents to gain knowledge about the reputation of potential interaction partners without having to expose themselves to the risk of actually interacting with them. However, the presence of unfair testimonies adversely affects an agent’s long term wellbeing and has been an important problem in agent trust research. Many testimony filtering methods have been proposed, but they often rely on assumptions about the characteristics of the witness agent population or supporting facilities in the environment. In addition, some methods involve highly iterative approaches that consume excessive amount of computational resources.

In this paper, we propose a witness selection method, based on the principles of reinforcement learning, for distributed reputation-based agent trust models. The proposed method innovatively formulates the witness selection problem to allow reinforcement learning technique to be leveraged to solve it without making limiting assumptions on the characteristics of the witness agent population or the existence of supporting facilities. Extensive simulations have shown that it significantly outperforms existing approaches in mitigating the adverse effect of unfair testimonies, especially under conditions where trustee agents and witness agents strategically collude to boost their chances of being selected for interaction by unsuspecting truster agents.


trust reputation testimony reinforcement learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Tesauro, G.: Temporal Difference Learning and TD-Gammon. Communications of the ACM 38(3), 58–68 (1995)CrossRefGoogle Scholar
  2. 2.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 342 p. MIT Press (1998)Google Scholar
  3. 3.
    Jøsang, A., Ismail, R.: The Beta Reputation System. In: Proceedings of the 15th Bled Conference on Electronic Commerce (2002)Google Scholar
  4. 4.
    Sabater-Mir, J., Sierra, C.: Reputation and Social Network Analysis in Multi-agent Systems. In: Proceedings of the 1st International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), pp. 475–482 (2002)Google Scholar
  5. 5.
    Konda, V.R., Tsitsiklis, J.N.: On Actor-Critic Algorithms. SIAM Journal on Control and Optimization 42(4), 1143–1166 (2002)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Ba, S., Pavlou, P.A.: Evidence of the Effect of Trust Building Technology in Electronic Markets: Price Premiums and Buyer Behavior. MIS Quarterly 26(3), 243–268 (2002)CrossRefGoogle Scholar
  7. 7.
    Yu, B., Singh, M.P.: Detecting Deception in Reputation Management. In: Proceedings of 2nd International Joint Conference on Autonomous Agents and Multi Agent Systems (AAMAS), pp. 73–80 (2003)Google Scholar
  8. 8.
    Jurca, R., Faltings, B.V.: Towards Incentive-Compatible Reputation Management. In: Falcone, R., Barber, S.K., Korba, L., Singh, M.P. (eds.) AAMAS 2002. LNCS (LNAI), vol. 2631, pp. 138–147. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  9. 9.
    Whitby, A., Jøsang, A., Indulska, J.: Filtering out Unfair Ratings in Bayesian Reputation Systems. In: Proceedings of the Workshop on Trust in Agent Societies at the 3rd International Joint Conference on Autonomous Agents and Multi-Agent Systems, AAMAS (2004)Google Scholar
  10. 10.
    Miller, N., Resnick, P., Zeckhauser, R.: Eliciting Informative Feedback: The Peer-prediction Method. Management Science 51(9), 1359–1373 (2005)CrossRefGoogle Scholar
  11. 11.
    Weng, J., Miao, C., Goh, A.: An Entropy-based Approach to Protecting Rating Systems from Unfair Testimonies. IEICE Transactions 89-D(9), 2502–2511 (2006)Google Scholar
  12. 12.
    Jøsang, A., Ismail, R., Boyd, C.: A Survey of Trust and Reputation Systems for Online Service Provision. Decision Support System 43(2), 618–644 (2007)CrossRefGoogle Scholar
  13. 13.
    Fullam, K.K., Barber, K.S.: Dynamically Learning Sources of Trust Information: Experience vs. Reputation. In: Proceedings of 6th International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), pp. 1055–1060 (2007)Google Scholar
  14. 14.
    Zhang, J., Cohen, R., Larson, K.: A Trust-based Incentive Mechanism for E-Marketplaces. In: Falcone, R., Barber, S.K., Sabater-Mir, J., Singh, M.P. (eds.) Trust 2008. LNCS (LNAI), vol. 5396, pp. 135–161. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  15. 15.
    Marsh, S., Briggs, P.: Examining Trust, Forgiveness and Regret as Computational Concepts. In: Computing with Social Trust. Human-computer Interaction Series, Part I, pp. 9–43 (2009)Google Scholar
  16. 16.
    Yu, H., Shen, Z., Miao, C., Leung, C., Niyato, D.: A Survey of Trust and Reputation Management Systems in Wireless Communications. Proceedings of the IEEE 98(10), 1755–1772 (2010)CrossRefGoogle Scholar
  17. 17.
    Weng, J., Shen, Z., Miao, C., Goh, A., Leung, C.: Credibility: How Agents Can Handle Unfair Third-party Testimonies in Computational Trust Models. IEEE Transactions on Knowledge and Data Engineering (TKDE) 22(9), 1286–1298 (2010)CrossRefGoogle Scholar
  18. 18.
    Noorian, Z., Ulieru, M.: The State of the Art in Trust and Reputation Systems: A Framework for Comparison. Journal of Theoretical and Applied Electronic Commerce Research 5(2), 97–117 (2010)CrossRefGoogle Scholar
  19. 19.
    An, B., Lesser, V., Irwin, D., Zink, M.: Automated negotiation with decommitment for dynamic resource allocation in cloud computing. In: Proceedings of the 9th International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), pp. 981–988 (2010)Google Scholar
  20. 20.
    Liu, S., Zhang, J., Miao, C., Theng, Y.-L., Kot, A.C.: iCLUB: An Integrated Clustering-based Approach to Improve the Robustness of Reputation Systems. In: Proceedings of 10th International Joint Conference on Autonomous Agents and Multi-Agent Systems (AAMAS), pp. 1151–1152 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Han Yu
    • 1
  • Zhiqi Shen
    • 1
  • Bo An
    • 2
  1. 1.School of Computer EngineeringNanyang Technological UniversitySingapore
  2. 2.Computer Science DepartmentUniversity of Southern CaliforniaUSA

Personalised recommendations