The Invariance Approach

Abstract

This chapter considers a particular family of lifelong learning problems. The lifelong supervised learning framework applies the idea of lifelong learning in a concrete (and restrictive) context: the learner is assumed to face supervised learning problems of the same type and, moreover, these learning problems must be related by some domain-specific properties (casted as invariances) that are unknown in the beginning of lifelong learning but can be learned. Central to the learning approach taken here is the domain theory. It consists of a single network, called the invariance network, which represents invariances that exist for all target functions. EBNN analyzes training examples using the invariance network, in order to guide generalization when learning a new function. As will be illustrated, knowing the invariances of the domain can be most instrumental for successful learning if training data is scarce.