An innovative linear unsupervised space adjustment by keeping low-level spatial data structure

  • Samad Nejatian
  • Vahideh Rezaie
  • Hamid Parvin
  • Mohamadamin Pirbonyeh
  • Karamolah Bagherifard
  • Sharifah Kamilah Syed Yusof
Regular Paper
  • 4 Downloads

Abstract

A novel objective function has been introduced for solving the problem of space adjustment when supervisor is unavailable. In the introduced objective function, it has been tried to minimize the difference between distributions of the transformed original and test-data spaces. The local structural information presented in the original space is preserved by optimizing the mentioned objective function. We have proposed two techniques to preserve the structural information of original space: (a) identifying those pairs of examples that are as close as possible in original space and minimizing the distance between these pairs of examples after transformation and (b) preserving the naturally occurring clusters that are presented in original space during transformation. This cost function together with its constraints has resulted in a nonlinear objective function, used to estimate the weight matrix. An iterative framework has been employed to solve the problem of optimizing the objective function, providing a suboptimal solution. Next, using orthogonality constraint, the optimization task has been reformulated into the Stiefel manifold. Empirical examination using real-world datasets indicates that the proposed method performs better than the recently published state-of-the-art methods.

Keywords

Space adjustment Optimization Nonparametric clustering Structure maintenance Classification 

References

  1. 1.
    Pan SJ, Tsang I, Kwok J, Yang Q (2011) Domain adaptation via transfer component analysis. IEEE Trans Neural Netw 22:199–210CrossRefGoogle Scholar
  2. 2.
    Bache K, Lichman M (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml
  3. 3.
    Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: European conference on computer vision, pp 213–226Google Scholar
  4. 4.
    Pan SJ, Yang Q (2010) A survey on transfer learning. IEEE Trans Knowl Data Eng 22:1345–1359CrossRefGoogle Scholar
  5. 5.
    Beijbom O (2012) Domain adaptation for computer vision applications. Technical report, University of California, San DiegoGoogle Scholar
  6. 6.
    Sugiyama M, Nakajima S, Kashima H, von Bünau P, Kawanabe M (2007) Direct importance estimation with model selection and its application to covariate shift adaptation. In: Proceedings of neural information processing systems, pp 1962–1965Google Scholar
  7. 7.
    Dai W, Yang Q, Xue GR, Yu Y (2007) Boosting for transfer learning. In: International conference on machine learning, pp 193–200Google Scholar
  8. 8.
    Wan C, Pan R, Li J (2011) Bi-weighting domain adaptation for cross-language text classification. In: International joint conference on artificial intelligence, pp 1535–1540Google Scholar
  9. 9.
    Gopalan R, Li R, Chellappa R (2011) Domain adaptation for object recognition: an unsupervised approach. In: International conference in computer vision, pp 999–1006Google Scholar
  10. 10.
    Kulis B, Saenko K, Darrell T (2011) What you saw is not what you get: domain adaptation using asymmetric kernel transforms. In: IEEE conference on computer vision and pattern recognition, pp 1785–1792Google Scholar
  11. 11.
    Jhuo IH, Liu D, Lee DT, Chang SF (2012) Robust visual domain adaptation with low-rank reconstruction. In: IEEE conference on computer vision and pattern recognition, pp 2168–2175Google Scholar
  12. 12.
    Chattopadhyay R, Krishnan NC, Panchanathan S (2011) Topology preserving domain adaptation for addressing subject based variability in SEMG signal. In: AAAI spring symposium: computational physiology, pp 4–9Google Scholar
  13. 13.
    Howard A, Jebara T (2009) Transformation learning via kernel alignment. In: International conference on machine learning and applications, pp 301–308Google Scholar
  14. 14.
    Jiang W, Zavesky E, Fu Chang S, Loui A (2008) Cross-domain learning methods for high-level visual concept classification. In: International conference on image processing, pp 161–164Google Scholar
  15. 15.
    Yang J, Yan R, Hauptmann AG (2007) Cross-domain video concept detection using adaptive SVMs. In: International conference on multimedia, pp 188–197Google Scholar
  16. 16.
    Shi X, Fan W, Ren J (2008) Actively transfer domain knowledge. In: European conference on machine learning, pp 342–357Google Scholar
  17. 17.
    Baktashmotlagh M, Harandi M, Lovell B, Salzmann M (2013) Unsupervised domain adaptation by domain invariant projection. In: International conference on computer vision, pp 769–776Google Scholar
  18. 18.
    Duan L, Xu D, Tsang IW, Luo J (2012) Visual event recognition in videos by learning from web data. IEEE Trans Pattern Anal Mach Intell 34:1667–1680CrossRefGoogle Scholar
  19. 19.
    Fernando B, Habrard A, Sebban M, Tuytelaars T (2013) Unsupervised visual domain adaptation using subspace alignment. In: International conference in computer vision, pp 2960–2967Google Scholar
  20. 20.
    Gong B, Shi Y, Sha F, Grauman K (2012) Geodesic flow kernel for unsupervised domain adaptation. In: IEEE conference on computer vision and pattern recognition, pp 2066–2073Google Scholar
  21. 21.
    Samanta S, Das S (2013) Domain adaptation based on eigen-analysis and clustering, for object categorization. In: International conference on computer analysis of images and patterns, LNCS, pp 245–253Google Scholar
  22. 22.
    Hoffmann H (2007) Kernel PCA for novelty detection. In: Pattern recognition, pp 863–874Google Scholar
  23. 23.
    Pezeshki A, Scharf LL, Chong EK (2010) The geometry of linearly and quadratically constrained optimization problems for signal processing and communications. J Frankl Inst 347:818–835MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Boyd S, Vandenberghe L (2006) Convex optimization. Cambridge University Press, New YorkMATHGoogle Scholar
  25. 25.
    Absil PA, Mahony R, Sepulchre R (2008) Optimization algorithms on matrix manifolds. Princeton University Press, PrincetonCrossRefMATHGoogle Scholar
  26. 26.
    Tagare HD (2011) Notes on optimization on Stiefel manifolds. Technical report, Department of Diagnostic Radiology, Department of Biomedical Engineering, Yale UniversityGoogle Scholar
  27. 27.
    Wen Z, Yin W (2013) A feasible method for optimization with orthogonality constraints. Math Prog 142:397–434MathSciNetCrossRefMATHGoogle Scholar
  28. 28.
    Löfberg J (2004) YALMIP: a Toolbox for modeling and optimization in MATLAB. In: Proceedings of the CACSD conference, Taiwan, TaipeiGoogle Scholar
  29. 29.
    Chopra S, Balakrishnan S, Gopalan R (2013) Dlid: Deep learning for domain adaptation by interpolating between domains. In: ICML workshop on challenges in representation learningGoogle Scholar
  30. 30.
    Tzeng E, Hoffman J, Zhang N, Saenko K, Darrell T (2014) Deep domain confusion: maximizing for domain invariance. CoRR, abs/1412.3474Google Scholar
  31. 31.
    Long M, Wang J (2015) Learning transferable features with deep adaptation networks. CoRR, abs/1502.02791Google Scholar
  32. 32.
    Duan L, Xu D, Tsang IWH (2012) Domain adaptation from multiple sources: a domain-dependent regularization approach. IEEE Trans Neural Netw Learn Syst 23:504–518CrossRefGoogle Scholar
  33. 33.
    Bay H, Ess A, Tuytelaars T, Gool LV (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110:346–359CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2018

Authors and Affiliations

  • Samad Nejatian
    • 1
    • 2
  • Vahideh Rezaie
    • 2
    • 3
  • Hamid Parvin
    • 4
    • 5
  • Mohamadamin Pirbonyeh
    • 4
    • 5
  • Karamolah Bagherifard
    • 2
    • 6
  • Sharifah Kamilah Syed Yusof
    • 7
  1. 1.Department of Electrical Engineering, Yasooj BranchIslamic Azad UniversityYasoojIran
  2. 2.Young Researchers and Elite Club, Yasooj BranchIslamic Azad UniversityYasoojIran
  3. 3.Department of Mathematics, Yasooj BranchIslamic Azad UniversityYasoojIran
  4. 4.Department of Computer Engineering, Nourabad Mamasani BranchIslamic Azad UniversityNourabad MamasaniIran
  5. 5.Young Researchers and Elite Club, Nourabad Mamasani BranchIslamic Azad UniversityNourabad MamasaniIran
  6. 6.Department of Computer Engineering, Yasooj BranchIslamic Azad UniversityYasoojIran
  7. 7.UTM-MIMOS Centre of Excellence, Faculty of Electrical EngineeringUniversiti Teknologi Malaysia (UTM)Johor BahruMalaysia

Personalised recommendations