Pack Light on the Move: Exploitation and Exploration in a Dynamic Environment

  • Marco LiCalziEmail author
  • Davide Marchiori
Part of the Lecture Notes in Economics and Mathematical Systems book series (LNE, volume 669)


This paper revisits a recent study by Posen and Levinthal (Manag Sci 58:587–601, 2012) on the exploration/exploitation tradeoff for a multi-armed bandit problem, where the reward probabilities undergo random shocks. We show that their analysis suffers two shortcomings: it assumes that learning is based on stale evidence, and it overlooks the steady state. We let the learning rule endogenously discard stale evidence, and we perform the long run analyses. The comparative study demonstrates that some of their conclusions must be qualified.


Dynamic Environment Learning Model Turbulence Level Search Intensity Bandit Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Berry D, Fristedt B (1985) Bandit problems. Chapman and Hall, LondonCrossRefGoogle Scholar
  2. 2.
    Cesa-Bianchi N, Lugosi G (2006) Prediction, learning, and games. Cambridge University Press, New YorkCrossRefGoogle Scholar
  3. 3.
    Duffy J (2006) Agent-based models and human subject experiments. In: Tesfatsion L, Judd KL (eds) Handbook of computational economics, vol 2. North-Holland, Amsterdam/New York, pp 949–1011Google Scholar
  4. 4.
    LiCalzi M, Marchiori D (2013) Pack light on the move: exploitation and exploration in a dynamic environment. Working Paper 4/2013, Department of Management, Università Ca’ Foscari Venezia,Google Scholar
  5. 5.
    March JG (1991) Exploration and exploitation in organizational learning. Organ Sci 1:71–87CrossRefGoogle Scholar
  6. 6.
    Posen HE, Levinthal DA (2012) Chasing a moving target: exploitation and exploration in dynamic environments. Manag Sci 58:587–601CrossRefGoogle Scholar
  7. 7.
    Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. The MIT University Press, CambridgeGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  1. 1.Department of ManagementUniversità Ca’ Foscari VeneziaVeniceItaly

Personalised recommendations