Towards Meta-learning of Deep Architectures for Efficient Domain Adaptation

Abstract

This paper proposes an efficient domain adaption approach using deep learning along with transfer and meta-level learning. The objective is to identify how many blocks (i.e. groups of consecutive layers) of a pre-trained image classification network need to be fine-tuned based on the characteristics of the new task. In order to investigate it, a number of experiments have been conducted using different pre-trained networks and image datasets. The networks were fine-tuned, starting from the blocks containing the output layers and progressively moving towards the input layer, on various tasks with characteristics different from the original task. The amount of fine-tuning of a pre-trained network (i.e. the number of top layers requiring adaptation) is usually dependent on the complexity, size, and domain similarity of the original and new tasks. Considering these characteristics, a question arises of how many blocks of the network need to be fine-tuned to get maximum possible accuracy? Which of a number of available pre-trained networks require fine-tuning of the minimum number of blocks to achieve this accuracy? The experiments, that involve three network architectures each divided into 10 blocks on average and five datasets, empirically confirm the intuition that there exists a relationship between the similarity of the original and new tasks and the depth of network needed to fine-tune in order to achieve accuracy comparable with that of a model trained from scratch. Further analysis shows that the fine-tuning of the final top blocks of the network, which represent the high-level features, is sufficient in most of the cases. Moreover, we have empirically verified that less similar tasks require fine-tuning of deeper portions of the network, which however is still better than training a network from scratch.