Skip to main content
SpringerLink
Log in

Learning Improved Entertainment Trading Strategies for the TAC Travel Game

  • Conference paper
Agent-Mediated Electronic Commerce. Designing Trading Strategies and Mechanisms for Electronic Markets (AMEC 2009, TADA 2009)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 59))

Abstract

For almost five years we continually operated a simulation testbed exploring strategies for the TAC Travel game. Building on techniques developed in our recent study of continuous double auctions, we performed an equilibrium analysis of our testbed data, and employed reinforcement learning in the equilibrium environment to derive a new entertainment strategy for this domain. A second iteration of this process led to further improvements. We thus demonstrate that interleaving empirical game-theoretic analysis with reinforcement learning in an effective method for generating stronger trading strategies in this domain.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aurell, E., Boman, M., Carlsson, M., Eriksson, J., Finne, N., Janson, S., Kreuger, P., Rasmusson, L.: A trading agent built on constraint programming. In: Eighth International Conference of the Society for Computational Economics: Computing in Economics and Finance, Aix-en-Provence (2002)

    Google Scholar 

  2. Boadway, J., Precup, D.: Reinforcement learning applied to a multiagent system. Presentation at TAC Workshop (2001)

    Google Scholar 

  3. Boyan, J., Greenwald, A.: Bid determination in simultaneous auctions: An agent architecture. In: Third ACM Conference on Electronic Commerce, Tampa, pp. 210–212 (2001)

    Google Scholar 

  4. Cliff, D.: Evolving parameter sets for adaptive trading agents in continuous double-auction markets. In: Agents-98 Workshop on Artificial Societies and Computational Markets, Minneapolis, MN, May 1998, pp. 38–47 (1998)

    Google Scholar 

  5. Fasli, M., Poursanidis, N.: Thalis: A flexible trading agent. Technical Report CSM-388, University of Essex, Department of Computer Science (2003)

    Google Scholar 

  6. Fourer, R., Gay, D.M., Kernighan, B.W.: AMPL: A Modeling Language for Mathematical Programming. Boyd & Fraser (1993)

    Google Scholar 

  7. Friedman, D.: The double auction market institution: A survey. In: Friedman, D., Rust, J. (eds.) The Double Auction Market: Institutions, Theories, and Evidence, pp. 3–25. Addison-Wesley, Reading (1993)

    Google Scholar 

  8. Fritschi, C., Dorer, K.: Agent-oriented software engineering for successful TAC participation. In: First International Joint Conference on Autonomous Agents and Multi-Agent Systems, Bologna (2002)

    Google Scholar 

  9. Greenwald, A., Lee, S.J., Naroditskiy, V.: RoxyBot-06: Stochastic prediction and optimization in TAC travel. Journal of Artificial Intelligence Research 36, 513–546 (2009)

    MATH  Google Scholar 

  10. He, M., Jennings, N.R.: SouthamptonTAC: An adaptive autonomous trading agent. ACM Transactions on Internet Technology 3, 218–235 (2003)

    Article  Google Scholar 

  11. Jordan, P.R., Kiekintveld, C., Wellman, M.P.: Empirical game-theoretic analysis of the TAC supply chain game. In: Sixth International Joint Conference on Autonomous Agents and Multi-Agent Systems, Honolulu, pp. 1188–1195 (2007)

    Google Scholar 

  12. Jordan, P.R., Schvartzman, L.J., Wellman, M.P.: Strategy exploration in empirical games. In: Ninth International Conference on Autonomous Agents and Multi-Agent Systems, Toronto (2010)

    Google Scholar 

  13. Kehagias, D., Toulis, P., Mitkas, P.: A long-term profit seeking strategy for continuous double auctions in a trading agent competition. In: Fourth Hellenic Conference on Artificial Intelligence, Heraklion, Greece (2006)

    Google Scholar 

  14. L’Ecuyer, P.: Efficiency improvement and variance reduction. In: Twenty-Sixth Winter Simulation Conference, Orlando, FL, pp. 122–132 (1994)

    Google Scholar 

  15. Onodera, M., Kawamura, H., Yamamoto, M., Kurumatani, K., Ohuchi, A.: Design of adaptive trading strategy for trading agent competition. In: International Technical Conference on Circuits/Systems, Computers and Communications, pp. 337–340 (2003)

    Google Scholar 

  16. Rust, J., Miller, J.H., Palmer, R.: Characterizing effective trading strategies: Insights from a computerized double auction tournament. Journal of Economic Dynamics and Control 18, 61–96 (1994)

    Article  Google Scholar 

  17. Sardinha, J.A.R.P., Milidiú, R.L., Paranhos, P.M., Cunha, P.M., Lucena, C.J.P.: An agent based architecture for highly competitive electronic markets. In: Eighteenth International FLAIRS Conference, Clearwater Beach, FL, pp. 326–331 (2005)

    Google Scholar 

  18. Schvartzman, L.J., Wellman, M.P.: Stronger CDA strategies through empirical game-theoretic analysis and reinforcement learning. In: Eighth International Conference on Autonomous Agents and Multi-Agent Systems, Budapest, pp. 249–256 (2009)

    Google Scholar 

  19. Stone, P., Littman, M.L., Singh, S., Kearns, M.: ATTac-2000: An adaptive autonomous bidding agent. Journal of Artificial Intelligence Research 15, 189–206 (2001)

    MATH  Google Scholar 

  20. Stone, P., Schapire, R.E., Littman, M.L., Csirik, J.A., McAllester, D.: Decision-theoretic bidding based on learned density models in simultaneous, interacting auctions. Journal of Artificial Intelligence Research 19, 209–242 (2003)

    MATH  MathSciNet  Google Scholar 

  21. Sutton, R.S., Barto, A.G.: Reinforcement Learning. MIT Press, Cambridge (1998)

    Google Scholar 

  22. Tesauro, G., Bredin, J.L.: Strategic sequential bidding in auctions using dynamic programming. In: First International Joint Conference on Autonomous Agents and Multi-Agent Systems, Bologna, pp. 591–598 (2002)

    Google Scholar 

  23. Vetsikas, I.A., Selman, B.: A principled study of the design tradeoffs for autonomous trading agents. In: Second International Joint Conference on Autonomous Agents and Multi-Agent Systems, Melbourne, pp. 473–480 (2003)

    Google Scholar 

  24. Vytelingum, P., Cliff, D., Jennings, N.R.: Strategic bidding in continuous double auctions. Artificial Intelligence 172, 1700–1729 (2008)

    Article  MATH  Google Scholar 

  25. Wellman, M.P.: Methods for empirical game-theoretic analysis (extended abstract). In: Twenty-First National Conference on Artificial Intelligence, Boston, pp. 1552–1555 (2006)

    Google Scholar 

  26. Wellman, M.P., Greenwald, A., Stone, P.: Autonomous Bidding Agents: Strategies and Lessons from the Trading Agent Competition. MIT Press, Cambridge (2007)

    Google Scholar 

  27. Wellman, M.P., Reeves, D.M., Lochner, K.M., Cheng, S.-F., Suri, R.: Approximate strategic reasoning through hierarchical reduction of large symmetric games. In: Twentieth National Conference on Artificial Intelligence, Pittsburgh, pp. 502–508 (2005)

    Google Scholar 

  28. Wellman, M.P., Reeves, D.M., Lochner, K.M., Suri, R.: Searching for Walverine 2005. In: La Poutré, H., Sadeh, N.M., Janson, S. (eds.) AMEC 2005 and TADA 2005. LNCS (LNAI), vol. 3937, pp. 157–170. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  29. Wellman, M.P., Reeves, D.M., Lochner, K.M., Vorobeychik, Y.: Price prediction in a trading agent competition. Journal of Artificial Intelligence Research 21, 19–36 (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science & Engineering, University of Michigan, 2260 Hayward St, Ann Arbor, MI, 48109-2121, USA

    L. Julian Schvartzman & Michael P. Wellman

Authors
  1. L. Julian Schvartzman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Michael P. Wellman
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science, Ashkelon Academic College, 78461, Ashkelon, Israel

    Esther David

  2. Highfield, University of Southampton, SO17 1BJ, Southampton, UK

    Enrico Gerding

  3. Department of Computer Science, Bar Ilan University, 52900, Ramat-Gan, Israel

    David Sarne

  4. IBM Haifa Research Lab, Haifa University Campus, 31905, Haifa, Mount Carmel, Israel

    Onn Shehory

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Schvartzman, L.J., Wellman, M.P. (2010). Learning Improved Entertainment Trading Strategies for the TAC Travel Game. In: David, E., Gerding, E., Sarne, D., Shehory, O. (eds) Agent-Mediated Electronic Commerce. Designing Trading Strategies and Mechanisms for Electronic Markets. AMEC TADA 2009 2009. Lecture Notes in Business Information Processing, vol 59. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15117-0_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-15117-0_14

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-15116-3

  • Online ISBN: 978-3-642-15117-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation