Flexibility of Emulation Learning from Pioneers in Nonstationary Environments

Abstract

This is an extension from a selected paper from JSAI2019. Social learning is crucial in acquisition of the intelligent behaviors of humans and many kinds of animals, as it makes behavior learning far more efficient than pure trial-and-error. In imitation learning, a representative form of social learning, the agent observes specific action-state pair sequences produced by another agent (expert) and reflect them into its own action. One of its implementations in reinforcement learning is the inverse reinforcement learning. We propose another form of social learning, emulation learning, which requires much less information from another agent (pioneer). In emulation learning, the agent is given only a certain level of achievement by another agent, or a record. In this study, we implement emulation learning in the reinforcement learning setting by applying a model of satisficing action policy. We show that the emulation learning algorithm works well both in stationary and non-stationary reinforcement learning tasks, breaking the often observed trade-off like relationship between efficiency and flexibility.