Image representations for visual learning

Abstract

The last few years have seen new successful approaches to object recognition and to computer graphics based directly on images without the use of intermediate 3D models. I will argue that most of these techniques depend on a representation of images that induces a linear vector space structure and in principle requires dense correspondence. This image representation allows the use of learning techniques for the analysis and for the synthesis of images, that is for both computer vision and computer graphics.

The key assumption hidden in most view-based approaches to object recognition and object detection is that the relevant images are vectors. This is not true unless they are set in correspondence. The correspondence step associates a shape vector and a texture vector to each image. I will focus on the domain of face images and review how this representation can be used to learn

• to estimate parameters such as expression and pose from images

• to interpolate in multiple dimensions new images and images sequences from examples

• to extrapolate from single images and generate virtual examples

In particular, I will describe a new example-based correspondence technique based on a flexible template represented as the linear combination of prototypes [1]–[4]. The approach is hierarchical and may allow to exploit context in vision tasks. It has interesting biological implications and may applied to other domains beyond vision and graphics.