Multimedia Tools and Applications

, Volume 74, Issue 3, pp 671–688 | Cite as

Nonlinear distance function learning using neural network: an iterative framework

  • Junying Chen
  • Haoyu Zeng
  • Na Fan


In this paper, we extend several existing methods that apply distance function learning to regression problems. We discover that these methods may be viewed as approximating a matrix consisting of desired distances among all training samples. Based on this understanding, we propose an iterative framework where outlier samples are corrected by their neighbors via asymptotically increasing the correlation coefficients between the desired distances and the distances of sample labels. Moreover, using this framework, we find that most existing methods iterate only once. As another extension, we adopt a nonlinear distance function and approximate it with neural network. For a fair comparison, we conduct an experiment on age estimation from face images as a regression problem, and the results are comparable to the state of the art.


Metric learning Multimodal learning Nonlinearity Regression models 


  1. 1.
    Balasubramanian VN, Ye J, Panchanathan S (2007) Biased manifold embedding: A framework for person-independent head pose estimation, Proc. CVPR, pp.1–7Google Scholar
  2. 2.
    Bar-Hillel AD (2007) Weinshall, Learning distance function by coding similarity, Proc. ICML, pp.65–72Google Scholar
  3. 3.
    Castillo E, Berdinas BG, Romero OF, Betanzos AA (2006) A very fast learning method for neural networks based on sensitivity analysis. J Mach Learn Res 7:1159–1182MATHMathSciNetGoogle Scholar
  4. 4.
    Cherkassky V, Shao X, Mulier FM, Vapnik VN (1999) Model complexity control for regression using VC generalization bounds. IEEE Trans Neural Netw 10(5):1075–1089CrossRefGoogle Scholar
  5. 5.
    Chopra S, Hadsell R, LeCun Y (2005) Learning a similarity metric discriminatively, with application to face verification, Proc. CVPR, pp.539–546Google Scholar
  6. 6.
    Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. IEEE Trans on PAMI 23(6):681–685CrossRefGoogle Scholar
  7. 7.
    Davis JV, Kulis B, Jain P, Sra V, Dhillon IS (2007) Information-theoretic metric learning, Proc. ICML, pp.209–216Google Scholar
  8. 8.
    Fan N (2011) Learning nonlinear distance functions using neural network for regression with application to robust human age estimation, Proc. ICCV, pp.249–254Google Scholar
  9. 9.
    FG-NET Aging Database,
  10. 10.
    Geng X, Miles KS, Zhou ZZ (2008) Facial age estimation by nonlinear aging pattern subspace, Proc. ACM Multimedia, pp.721–724Google Scholar
  11. 11.
    Geng X, Zhou ZH, Miles KS (2007) Automatic age estimation based on facial aging patterns. IEEE Trans PAMI 29(12)):2234–2240CrossRefGoogle Scholar
  12. 12.
    Goldberger J, Roweis S, Hinton G, Salakhutdinov R (2005) Neighbourhood components analysis, Proc. NIPS, pp.513–520Google Scholar
  13. 13.
    Guo GD, Mu G, Fu Y, Dyer C, Huang TS (2009) A study on automatic age estimation using a large database, Proc. ICCV, pp.1–8Google Scholar
  14. 14.
    Guo GD, Mu G, Fu Y, Huang (2009) Human age estimation using bio-inspired features, Proc. CVPR, pp.1–8Google Scholar
  15. 15.
    He X, Ma WY, Zhang HJ (2004) Learning an image manifold for retrieval, Proc. ACM Multimedia, pp.17–23Google Scholar
  16. 16.
    Huang YZ, Long YJ (2008) Demosaicking recognition with applications in digital photo authentication based on a quadratic pixel correlation model, Proceedings of CVPR, pp.1–8Google Scholar
  17. 17.
    Jin C, Long YJ (2010) On label information incorporated metric learning for regressions. Int J Comput Intell Appl 9(4):339–351CrossRefMATHGoogle Scholar
  18. 18.
    Lanitis A, Draganova C, Christodoulou C (2004) Comparing different classifiers for automatic age estimation. IEEE Trans SMC-B 34(1):621–628Google Scholar
  19. 19.
    Long YJ, Huang YZ (2006) Image based source camera identification using demosaicking, Proceedings of the 8th International conference on Workshop Multimedia Signal Processing, pp. 419–424Google Scholar
  20. 20.
    Macskassy SA, Hirsh H, Banerjee A, Dayanik AA (2003) Converting numerical classification into text classification. Artif Intell 143(1):51–77CrossRefMATHMathSciNetGoogle Scholar
  21. 21.
    McCullagh P (1980) Regression models for ordinal data. J R Stat Soc Ser B 42(2):109–142MATHMathSciNetGoogle Scholar
  22. 22.
    Min R, van der Maaten LJP, Yuan Z, Bonner A, Zhang Z (2010) Deep supervised t-distributed embedding, Proc. ICML, pp.791–798Google Scholar
  23. 23.
    Moller AF (1993) A scaled conjugate gradient algorithm for fast supervised learning. Neural Netw 6(4):525–533CrossRefGoogle Scholar
  24. 24.
    Pan (2010) Human age estimation by metric learning for regression problems. Proc. EMM CVPR, pp. 455–465Google Scholar
  25. 25.
    Ramanathan N, Chellappa R, Biswas S (2009) Age progression in human faces: a survey. J Vis Lang Comput 20:131–144CrossRefGoogle Scholar
  26. 26.
    Salakhutdinov R, Hinton G (2007) Learning a nonlinear embedding by preserving class neighbourhood structure, Proc. AI and Statistics, pp. 412–419Google Scholar
  27. 27.
    Shalev-Shwartz S, Singer Y, Ng AY (2004) Online and batch learning of pseudo-metrics, Proc. ICML, pp.743–750Google Scholar
  28. 28.
    Shental N, Hertz T, Weinshall D, Pavel M (2002) Adjustment learning and relevant component analysis, Proc. ECCV, pp.776–792Google Scholar
  29. 29.
    Smith L (2002) A tutorial on Principal Components Analysis
  30. 30.
    Stanley KO (2007) Compositional pattern producing networks: a novel abstraction of development. Genet Program Evolvable Mach 8(2):131–162CrossRefGoogle Scholar
  31. 31.
    Tan X, Chen S, Li J, Zhou Z (2006) Learning non-metric partial similarity based on maximal margin criterion, Proc. CVPR, pp.138–145Google Scholar
  32. 32.
    Taylor G, Fergus R, Williams G, Spiro I, Bregler C (2010) Pose-sensitive embedding by nonlinear NCA regression. Proc, NIPSGoogle Scholar
  33. 33.
    Weinberger K, Blitzer J, Saul L (2006) Distance metric learning for large margin nearest neighbor classification, Proc. NIPS, pp.1475–1482Google Scholar
  34. 34.
    Xing E, Ng A, Jordan MI, Russell S (2002) Distance metric learning with application to clustering with side-information, Proc. NIPS, pp.505–512Google Scholar
  35. 35.
    Yan S, Wang H, Huang TS, Tang X (2007) Ranking with uncertain labels, Proc. ICME, pp.96–99Google Scholar
  36. 36.
    Yan S, Wang H, Tang X, Huang T (2007) Learning auto-structured regressor from uncertain nonnegative labels. Proc. ICCV, pp.1–8Google Scholar
  37. 37.
    Yan S, Zhou X, M. Liu, M. H. Johnson, T. Huang (2008) Regression from patch-kernel, Proc. CVPR, pp.1–8Google Scholar
  38. 38.
    Yang L, Jin R (2006) Distance metric learning: a comprehensive survey, Technical report, Michigan State University.
  39. 39.
    Yeung DY, Chang H (2007) A kernel approach for semi-supervised metric learning. IEEE Trans on Neural Net 18(1):141–149CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  1. 1.College of SciencesAgricultural University of HebeiBaodingChina
  2. 2.Department of Electronic EngineeringEast China Normal UniversityShanghaiPeople’s Republic of China

Personalised recommendations