Advertisement

Deep domain adaptation with manifold aligned label transfer

  • Breton MinnehanEmail author
  • Andreas Savakis
Original Paper
  • 7 Downloads

Abstract

We propose a novel deep learning domain adaptation method that performs transductive learning from the source domain to the target domain based on cluster matching between the source and target features. The proposed method combines Adaptive Batch Normalization and Locality Preserving Projection-based subspace alignment on deep features to produce a common feature space for label transfer. Adaptive Batch Normalization automatically conditions the features from the source/target domain by normalizing the activations in each layer of our network. Following Manifold Subspace Alignment, we cluster the data in each domain using Gaussian Mixture Model clustering in feature space. The clusters are matched between domains to transfer labels from the closest source cluster to each target cluster. The transfer labels are compared to the network prediction, and the samples with consistent labels are used to adapt the network on the target domain. The proposed manifold-guided label transfer method produces state-of-the-art results for deep adaptation on digit recognition datasets. Furthermore, we perform domain adaptation on remote sensing datasets.

Keywords

Domain adaptation Label transfer Deep learning 

Notes

Acknowledgements

This research was supported in part by a grant from the AFOSR Dynamic Data Driven Applications Systems (DDDAS) program.

References

  1. 1.
    Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 898–916 (2011)CrossRefGoogle Scholar
  2. 2.
    Bousmalis, K., Silberman, N., Dohan, D., Erhan, D., Krishnan, D.: Unsupervised pixel-level domain adaptation with generative adversarial networks. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, p. 7 (2017)Google Scholar
  3. 3.
    Dai, W., Xue, G.R., Yang, Q., Yu, Y.: Transferring naive Bayes classifiers for text classification. AAAI 7, 540–545 (2007)Google Scholar
  4. 4.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005 (CVPR 2005), vol. 1, pp. 886–893. IEEE (2005)Google Scholar
  5. 5.
    Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B (Methodol.) 39(1), 1–38 (1977)MathSciNetzbMATHGoogle Scholar
  6. 6.
    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, 2009 (CVPR 2009), pp. 248–255. IEEE (2009)Google Scholar
  7. 7.
    Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., Darrell, T.: Decaf: a deep convolutional activation feature for generic visual recognition. In: International Conference on Machine Learning, pp. 647–655 (2014)Google Scholar
  8. 8.
    Elshamli, A., Taylor, G.W., Berg, A., Areibi, S.: Domain adaptation using representation learning for the classification of remote sensing images. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10(9), 4198–4209 (2017).  https://doi.org/10.1109/JSTARS.2017.2711360 CrossRefGoogle Scholar
  9. 9.
    Fernando, B., Habrard, A., Sebban, M., Tuytelaars, T.: Unsupervised visual domain adaptation using subspace alignment. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 2960–2967. IEEE (2013)Google Scholar
  10. 10.
    French, G., Mackiewicz, M., Fisher, M.: Self-Ensembling for Domain Adaptation. arXiv preprint arXiv:1706.05208v3 (2017)
  11. 11.
    Ganin, Y., Lempitsky, V.: Unsupervised Domain Adaptation by Backpropagation. arXiv preprint arXiv:1409.7495 (2014)
  12. 12.
    Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., Lempitsky, V.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(1), 2030–2096 (2016)MathSciNetzbMATHGoogle Scholar
  13. 13.
    Ghifary, M., Kleijn, W.B., Zhang, M., Balduzzi, D., Li, W.: Deep reconstruction-classification networks for unsupervised domain adaptation. In: European Conference on Computer Vision, pp. 597–613. Springer, Berlin (2016)Google Scholar
  14. 14.
    Gong, B., Grauman, K., Sha, F.: Connecting the dots with landmarks: discriminatively learning domain-invariant features for unsupervised domain adaptation. In: International Conference on Machine Learning, pp. 222–230 (2013)Google Scholar
  15. 15.
    Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. In: Advances in Neural Information Processing Systems, pp. 2672–2680 (2014)Google Scholar
  16. 16.
    Gopalan, R., Li, R., Chellappa, R.: Domain adaptation for object recognition: an unsupervised approach. In: 2011 IEEE International Conference on Computer Vision (ICCV), pp. 999–1006. IEEE (2011)Google Scholar
  17. 17.
    Haeusser, P., Frerix, T., Mordvintsev, A., Cremers, D.: Associative domain adaptation. In: International Conference on Computer Vision (ICCV), vol. 2, p. 6 (2017)Google Scholar
  18. 18.
    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)Google Scholar
  19. 19.
    Hoffman, J., Rodner, E., Donahue, J., Darrell, T., Saenko, K.: Efficient Learning of Domain-invariant Image Representations. arXiv preprint arXiv:1301.3224 (2013)
  20. 20.
    Huang, G., Liu, Z., Weinberger, K.Q., van der Maaten, L.: Densely connected convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, p. 3 (2017)Google Scholar
  21. 21.
    Hull, J.J.: A database for handwritten text recognition research. IEEE Trans. Pattern Anal. Mach. Intell. 16(5), 550–554 (1994)CrossRefGoogle Scholar
  22. 22.
    Krizhevsky, A., Hinton, G.: Learning Multiple Layers of Features from Tiny Images (2009)Google Scholar
  23. 23.
    Kulis, B., Saenko, K., Darrell, T.: What you saw is not what you get: domain adaptation using asymmetric kernel transforms. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1785–1792. IEEE (2011)Google Scholar
  24. 24.
    Kullback, S., Leibler, R.A.: On information and sufficiency. Ann. Math. Stat. 22(1), 79–86 (1951)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)CrossRefGoogle Scholar
  26. 26.
    Lee, D.H.: Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In: Workshop on Challenges in Representation Learning, ICML, vol. 3, p. 2 (2013)Google Scholar
  27. 27.
    Li, H., Xu, Z., Taylor, G., Goldstein, T.: Visualizing the Loss Landscape of Neural Nets. arXiv preprint arXiv:1712.09913 (2017)
  28. 28.
    Li, Y., Wang, N., Shi, J., Liu, J., Hou, X.: Revisiting Batch Normalization for Practical Domain Adaptation. arXiv preprint arXiv:1603.04779 (2016)
  29. 29.
    Long, M., Cao, Y., Wang, J., Jordan, M.I.: Learning Transferable Features with Deep Adaptation Networks. arXiv preprint arXiv:1502.02791 (2015)
  30. 30.
    Lowe, D.G.: Object recognition from local scale-invariant features. In: The Proceedings of the Seventh IEEE International Conference on Computer vision, 1999, vol. 2, pp. 1150–1157. IEEE (1999)Google Scholar
  31. 31.
    Maaten, Lvd, Hinton, G.: Visualizing data using t-sne. J. Mach. Learn. Res. 9, 2579–2605 (2008)zbMATHGoogle Scholar
  32. 32.
    Minnehan, B., Savakis, A.: Manifold guided label transfer for deep domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 65–73 (2017)Google Scholar
  33. 33.
    Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning. In: NIPS Workshop on Deep Learning and Unsupervised Feature Learning, vol. 2011, pp. 1–9 (2011)Google Scholar
  34. 34.
    Ranjan, V., Harit, G., Jawahar, C.: Domain adaptation by aligning locality preserving subspaces. In: 2015 Eighth International Conference on Advances in Pattern Recognition (ICAPR), pp. 1–6. IEEE (2015)Google Scholar
  35. 35.
    Russo, P., Carlucci, F.M., Tommasi, T., Caputo, B.: From Source to Target and Back: Symmetric Bi-directional Adaptive Gan. arXiv preprint arXiv:1705.08824 (2017)
  36. 36.
    Saenko, K., Kulis, B., Fritz, M., Darrell, T.: Adapting visual category models to new domains. In: European Conference on Computer Vision, pp. 213–226. Springer, Berlin (2010)Google Scholar
  37. 37.
    Saito, K., Ushiku, Y., Harada, T.: Asymmetric Tri-training for Unsupervised Domain Adaptation. arXiv preprint arXiv:1702.08400 (2017)
  38. 38.
    Samat, A., Persello, C., Gamba, P., Liu, S., Abuduwaili, J., Li, E.: Supervised and semi-supervised multi-view canonical correlation analysis ensemble for heterogeneous domain adaptation in remote sensing image classification. Remote Sens. 9(337) (2017).  https://doi.org/10.3390/rs9040337. http://www.mdpi.com/2072-4292/9/4/337
  39. 39.
    Sankaranarayanan, S., Balaji, Y., Castillo, C.D., Chellappa, R.: Generate to Adapt: Aligning Domains Using Generative Adversarial Networks. arXiv preprint arXiv:1704.01705 (2017)
  40. 40.
    Sener, O., Song, H.O., Saxena, A., Savarese, S.: Learning transferrable representations for unsupervised domain adaptation. In: Advances in Neural Information Processing Systems, pp. 2110–2118 (2016)Google Scholar
  41. 41.
    Sener, O., Song, H.O., Saxena, A., Savarese, S.: Unsupervised Transductive Domain Adaptation. arXiv preprint arXiv:1602.03534 (2016)
  42. 42.
    Shekhar, S., Patel, V.M., Nguyen, H.V., Chellappa, R.: Generalized domain-adaptive dictionaries. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 361–368. IEEE (2013)Google Scholar
  43. 43.
    Shi, Q., Du, B., Zhang, L.: Domain adaptation for remote sensing image classification: a low-rank reconstruction and instance weighting label propagation inspired algorithm. IEEE Trans. Geosci. Remote Sens. 53(10), 5677–5689 (2015).  https://doi.org/10.1109/TGRS.2015.2427791 CrossRefGoogle Scholar
  44. 44.
    Shi, Y., Sha, F.: Information-Theoretical Learning of Discriminative Clusters for Unsupervised Domain Adaptation. arXiv preprint arXiv:1206.6438 (2012)
  45. 45.
    Sun, B., Feng, J., Saenko, K.: Return of frustratingly easy domain adaptation. AAAI 6, 2058–2065 (2016)Google Scholar
  46. 46.
    Sun, B., Saenko, K.: Deep coral: Correlation alignment for deep domain adaptation. In: European Conference on Computer Vision, pp. 443–450. Springer, Berlin (2016)Google Scholar
  47. 47.
    Torralba, A., Efros, A.A.: Unbiased look at dataset bias. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1521–1528. IEEE (2011)Google Scholar
  48. 48.
    Tzeng, E., Hoffman, J., Saenko, K., Darrell, T.: Adversarial discriminative domain adaptation. Comput. Vis. Pattern Recognit. 1, 2962–2971 (2017)Google Scholar
  49. 49.
    Tzeng, E., Hoffman, J., Zhang, N., Saenko, K., Darrell, T.: Deep Domain Confusion: Maximizing for Domain Invariance. arXiv preprint arXiv:1412.3474 (2014)
  50. 50.
    Wen, Y., Zhang, K., Li, Z., Qiao, Y.: A discriminative feature learning approach for deep face recognition. In: European Conference on Computer Vision (ECCV) (2016)Google Scholar
  51. 51.
    Xia, G.S., Hu, J., Hu, F., Shi, B., Bai, X., Zhong, Y., Zhang, L., Lu, X.: Aid: a benchmark data set for performance evaluation of aerial scene classification. IEEE Trans. Geosci. Remote Sens. 55(7), 3965–3981 (2017)CrossRefGoogle Scholar
  52. 52.
    Yang, Y., Newsam, S.: Bag-of-visual-words and spatial extensions for land-use classification. In: Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems, pp. 270–279. ACM (2010)Google Scholar
  53. 53.
    Zhang, X., Yu, F.X., Chang, S.F., Wang, S.: Deep Transfer Network: Unsupervised Domain Adaptation. arXiv preprint arXiv:1503.00591 (2015)
  54. 54.
    Zheng, X., Cai, D., He, X., Ma, W.Y., Lin, X.: Locality preserving clustering for image database. In: Proceedings of the 12th Annual ACM International Conference on Multimedia, pp. 885–891. ACM (2004)Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Rochester Institute of TechnologyRochesterUSA

Personalised recommendations