Supervised Earth Mover’s Distance Learning and Its Computer Vision Applications

  • Fan Wang
  • Leonidas J. Guibas
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7572)


The Earth Mover’s Distance (EMD) is an intuitive and natural distance metric for comparing two histograms or probability distributions. It provides a distance value as well as a flow-network indicating how the probability mass is optimally transported between the bins. In traditional EMD, the ground distance between the bins is pre-defined. Instead, we propose to jointly optimize the ground distance matrix and the EMD flow-network based on a partial ordering of histogram distances in an optimization framework. Our method is further extended to accept information from general labeled pairs. The trained ground distance better reflects the cross-bin relationships, hence produces more accurate EMD values and flow-networks. Two computer vision applications are used to demonstrate the effectiveness of the algorithm: first, we apply the optimized EMD value to face verification, and achieve state-of-the-art performance on the PubFig and the LFW data sets; second, the learned EMD flow-network is used to analyze face attribute changes, obtaining consistent paths that demonstrate intuitive transitions on certain facial attributes.


Face Recognition Face Image Image Retrieval Reference Identity Computer Vision Application 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60(2), 91–110 (2004)CrossRefGoogle Scholar
  2. 2.
    Belongie, S., Malik, J., Puzicha, J.: Shape Context: a new descriptor for shape matching and object recognition. In: NIPS (2001)Google Scholar
  3. 3.
    Fei-Fei, L., Perona, P.: A Bayesian hierarchical model for learning natural scene categories. In: CVPR (2005)Google Scholar
  4. 4.
    Rubner, Y., Tomasi, C., Guibas, L.J.: The Earth Mover’s Distance as a Metric for Image Retrieval. International Journal of Computer Vision 40(2), 99–121 (2000)zbMATHCrossRefGoogle Scholar
  5. 5.
    Xu, D., Yan, S., Luo, J.: Face recognition using spatially constrained Earth Mover’s Distance. IEEE Transactions on Image Processing 17(11), 2256–2260 (2008)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Zhao, Q., Yang, Z., Tao, H.: Differential Earth Mover’s Distance with its applications to visual tracking. IEEE TPAMI 32, 274–287 (2008)CrossRefGoogle Scholar
  7. 7.
    Grauman, K., Darrell, T.: Fast contour matching using approximate Earth Mover’s Distance. In: CVPR, pp. 220–227 (2004)Google Scholar
  8. 8.
    Xing, E.P., Ng, A.Y., Jordan, M.I., Russell, S.: Distance metric learning with application to clustering with side-information. In: NIPS (2003)Google Scholar
  9. 9.
    Bar-Hillel, A., Hertz, T., Shental, N., Weinshall, D.: Learning distance functions using equivalence relations. In: ICML (2003)Google Scholar
  10. 10.
    Frome, A., Singer, Y., Malik, J.: Image retrieval and classification using local distance functions. In: NIPS (2006)Google Scholar
  11. 11.
    Domeniconi, C., Gunopulos, D.: Adaptive nearest neighbor classification using Support Vector Machines. In: NIPS (2002)Google Scholar
  12. 12.
    Cuturi, M., Avis, D.: Ground Metric Learning. arXiv:1110.2306v1 (2011)Google Scholar
  13. 13.
    Wang, X.L., Liu, Y., Zha, H.: Learning robust cross-bin similarities for the bag-of-features model. Technical report, Key Labs of Machine Perception, Peking University, China (2009)Google Scholar
  14. 14.
    Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: CVPR, pp. 539–546 (2005)Google Scholar
  15. 15.
    Guillaumin, M., Verbeek, J., Schmid, C.: Is that you? metric learning approaches for face identification. In: ICCV, pp. 498–505 (2009)Google Scholar
  16. 16.
    Nguyen, H.V., Bai, L.: Cosine Similarity Metric Learning for Face Verification. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part II. LNCS, vol. 6493, pp. 709–720. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  17. 17.
    Kumar, N., Berg, A.C., Belhumeur, P.N., Nayar, S.K.: Attribute and simile classifiers for face verification. In: ICCV (2009)Google Scholar
  18. 18.
    Parikh, D., Grauman, K.: Relative attributes. In: ICCV (2011)Google Scholar
  19. 19.
    Levina, E., Bickel, P.: The Earth Mover’s Distance is the Mallows distance: some insights from statistics. In: ICCV, pp. 251–256. IEEE Computer Society (2001)Google Scholar
  20. 20.
    Villani, C.: Topics in Optimal Transportation. American Mathematical Society (2003)Google Scholar
  21. 21.
    Villani, C.: Optimal transport, Old and New. Grundlehren der Mathematischen Wissenschaften, vol. 338. Springer (2009)Google Scholar
  22. 22.
    Pele, O., Werman, M.: Fast and robust Earth Mover’s Distances. In: ICCV, pp. 460–467 (2009)Google Scholar
  23. 23.
    Pele, O., Werman, M.: A Linear Time Histogram Metric for Improved SIFT Matching. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 495–508. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  24. 24.
    Andoni, A., Ba, K.D., Indyk, P., Woodruff, D.: Efficient sketches for Earth-Mover Distance, with applications. In: IEEE FOCS, pp. 324–330 (October 2009)Google Scholar
  25. 25.
    Chvátal, V.: Linear Programming. W. H. Freeman (1983)Google Scholar
  26. 26.
    Grant, M., Boyd, S.: CVX: Matlab software for disciplined convex programming, version 1.21 (April 2011),
  27. 27.
    Zhang, L., Yang, M., Feng, X.: Sparse representation or collaborative representation: Which helps face recognition? In: ICCV (2011)Google Scholar
  28. 28.
    Alon, N., Cosares, S., Hochbaum, D.S., Shamir, R.: An algorithm for the detection and construction of Monge sequences. Linear Algebra and Its Application, 669–680 (1989)Google Scholar
  29. 29.
    Huang, G.B., Ramesh, M., Berg, T., Learned-Miller, E.: Labeled Faces in the Wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst (October 2007)Google Scholar
  30. 30.
    Sivic, J., Everingham, M., Zisserman, A.: “who are you?” - learning person specific classifiers from video. In: CVPR (2009)Google Scholar
  31. 31.
    Turk, M.A., Pentland, A.P.: Eigenfaces for recognition. Journal of Cognitive Neuroscience 3(1), 71–86 (1991)CrossRefGoogle Scholar
  32. 32.
    Ying, Y., Li, P.: Distance metric learning with eigenvalue optimization. JMLR (Special Topics on Kernel and Metric Learning) 13, 1–26 (2012)Google Scholar
  33. 33.
    Taigman, Y., Wolf, L., Hassner, T.: Multiple one-shots for utilizing class label information. In: BMVC (2009)Google Scholar
  34. 34.
    Cao, Z., Yin, Q., Tang, X., Sun, J.: Face recognition with learning-based descriptor. In: CVPR (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Fan Wang
    • 1
  • Leonidas J. Guibas
    • 1
  1. 1.Stanford UniversityUnited States

Personalised recommendations