Using Landscape Theory to Measure Learning Difficulty for Adaptive Agents

Abstract

In many real-world settings, particularly economic settings, an adaptive agent is interested in maximizing its cumulative reward. This may require a choice between different problems to learn, where the agent must trade optimal reward against learning difficulty. A landscape is one way of representing a learning problem, where highly rugged landscapes represent difficult problems. However, ruggedness is not directly measurable. Instead, a proxy is needed.

We compare the usefulness of three different metrics for estimating ruggedness on learning problems in an information economy domain. We empirically evaluate the ability of each metric to predict ruggedness and use these metrics to explain past results showing that problems that yield equal reward when completely learned yield different profits to an adaptive learning agent.