Descriptor Learning Using Convex Optimisation

  • Karen Simonyan
  • Andrea Vedaldi
  • Andrew Zisserman
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7572)


The objective of this work is to learn descriptors suitable for the sparse feature detectors used in viewpoint invariant matching. We make a number of novel contributions towards this goal: first, it is shown that learning the pooling regions for the descriptor can be formulated as a convex optimisation problem selecting the regions using sparsity; second, it is shown that dimensionality reduction can also be formulated as a convex optimisation problem, using the nuclear norm to reduce dimensionality. Both of these problems use large margin discriminative learning methods. The third contribution is a new method of obtaining the positive and negative training data in a weakly supervised manner. And, finally, we employ a state-of-the-art stochastic optimizer that is efficient and well matched to the non-smooth cost functions proposed here. It is demonstrated that the new learning methods improve over the state of the art in descriptor learning for large scale matching, Brown et al. [2], and large scale object retrieval, Philbin et al. [10].


Image Retrieval Convex Optimisation Convex Optimisation Problem Pooling Region Feature Pair 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Arandjelović, R., Zisserman, A.: Three things everyone should know to improve object retrieval. In: IEEE Int. Conf. on Comp. Vis. and Pat. Rec. IEEE Press, New York (2012)Google Scholar
  2. 2.
    Brown, M., Hua, G., Winder, S.: Discriminative learning of local image descriptors. IEEE Trans. on Patt. Anal. and Mach. Intell. 33(1), 43–57 (2011)CrossRefGoogle Scholar
  3. 3.
    Fazel, M., Hindi, H., Boyd, S.P.: A rank minimization heuristic with application to minimum order system approximation. In: IEEE Amer. Control Conf., pp. 4734–4739. IEEE Press, New York (2001)Google Scholar
  4. 4.
    Joachims, T.: Optimizing search engines using clickthrough data. In: ACM SIGKDD Int. Conf. on Knowl. Disc. and Data Mining, pp. 133–142. ACM Press, New York (2002)Google Scholar
  5. 5.
    Kumar, M.P., Packer, B., Koller, D.: Self-paced learning for latent variable models. In: Lafferty, J., Williams, C.K.I., Shawe-Taylor, J., Zemel, R., Culotta, A. (eds.) Adv. Neur. Inf. Proc. Sys. 23, pp. 1189–1197. Curran Associates, Inc, Red Hook (2010)Google Scholar
  6. 6.
    Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comp. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  7. 7.
    Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L.: A comparison of affine region detectors. Int. J. Comp. Vis. 65(1/2), 43–72 (2005)CrossRefGoogle Scholar
  8. 8.
    Nesterov, Y.: Primal-dual subgradient methods for convex problems. J. Math. Prog. 120(1), 221–259 (2009)MathSciNetzbMATHCrossRefGoogle Scholar
  9. 9.
    Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: IEEE Int. Conf. on Comp. Vis. and Pat. Rec. IEEE Press, New York (2007)Google Scholar
  10. 10.
    Philbin, J., Isard, M., Sivic, J., Zisserman, A.: Descriptor learning for efficient retrieval. In: Daniilidis, K. (ed.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 677–691. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  11. 11.
    Rennie, J.D.M., Srebro, N.: Fast maximum margin matrix factorization for collaborative prediction. In: De Raedt, L., Wrobel, S. (eds.) Int. Conf. Mach. Learn., pp. 713–719. ACM Press, New York (2005)Google Scholar
  12. 12.
    Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring photo collections in 3D. ACM Trans. on Graph. 25(3), 835–846 (2006)CrossRefGoogle Scholar
  13. 13.
    Tola, E., Lepetit, V., Fua, P.: A fast local descriptor for dense matching. In: IEEE Int. Conf. on Comp. Vis. and Pat. Rec. IEEE Press, New York (2008)Google Scholar
  14. 14.
    Torresani, L., Lee, K.: Large margin component analysis. In: Schölkopf, B., Platt, J., Hoffman, T. (eds.) Adv. Neur. Inf. Proc. Sys. 19, pp. 1385–1392. MIT Press, Cambridge (2007)Google Scholar
  15. 15.
    Weinberger, K., Blitzer, J., Saul, L.: Distance metric learning for large margin nearest neighbor classification. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Adv. Neur. Inf. Proc. Sys. 18, pp. 1473–1480. MIT Press, Cambridge (2006)Google Scholar
  16. 16.
    Xiao, L.: Dual averaging methods for regularized stochastic learning and online optimization. J. Mach. Learn. Res. 11, 2543–2596 (2010)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Karen Simonyan
    • 1
  • Andrea Vedaldi
    • 1
  • Andrew Zisserman
    • 1
  1. 1.Visual Geometry GroupUniversity of OxfordUK

Personalised recommendations