Learning Using Privileged Information in Prototype Based Models

  • Shereen Fouad
  • Peter Tino
  • Somak Raychaudhury
  • Petra Schneider
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7553)


In some pattern analysis problems, there exists expert knowledge, in addition to the original data involved in the classification process. Most of existing approaches ignore such auxiliary (privileged) knowledge. Recently a new learning paradigm - Learning Using Hidden Information - was introduced in the SVM+ framework. This approach is formulated for binary classification and, as typical for many kernel based methods, can scale unfavorably with the number of training examples. In this contribution we present a more direct novel methodology, based on a prototype metric learning model, for incorporation of valuable privileged knowledge. This is done by changing the global metric in the input space, based on distance relations revealed by the privileged information. Our method achieves competitive performance against the SVM+ formulations. We also present a successful application of our method to a large scale multi-class real world problem of galaxy morphology classification.


Learning Using Hidden Information (LUHI) Generalized Matrix Learning Vector Quantization (GMLVQ) Information Theoretic Metric Learning (ITML) 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Vapnik, V., Vashist, A.: A New Learning Paradigm: Learning Using Privileged Information. In: Neural Networks (NNs), vol. 22(5-6), pp. 544–555. Elsevier Ltd. (2009)Google Scholar
  2. 2.
    Cervantes, J., Li, X., Yu, W.: Multi-Class SVM for Large Data Sets Considering Models of Classes Distribution. In: International Conference on Data Mining (DMIN), pp. 30–35. CSREA Press, Las Vegas (2008)Google Scholar
  3. 3.
    Cervantes, J., Li, X., Yu, W., Li, K.: Support Vector Machine Classification For Large Data Sets Via Minimum Enclosing Ball Clustering. In: Neural Networks (NNs): Algorithms and Applications, 4th International Symposium on Neural Networks 2008, vol. 71(4-6), pp. 611–619. Elsevier Science Publishers, Amsterdam (2008)Google Scholar
  4. 4.
    Biehl, M., Hammer, B., Schneider, P., Villmann, T.: Metric Learning for Prototype-Based Classification. In: Innovations in Neural Information Paradigms and Applications, vol. 247, pp. 183–199. Springer (2009)Google Scholar
  5. 5.
    Kohonen, T.: Learning Vector Quantization for Pattern Recognition. Technical report, No (TKK-F-A601), Helsinki University of Technology. Espoo, Finland (1986)Google Scholar
  6. 6.
    Sato, A.S., Yamada, K.: Generalized Learning Vector Quantization. In: Touretzky, D., Leen, T. (eds.) Advances in Neural Information Processing Systems (ANIPS), vol. 7, pp. 423–429. MIT Press (1995)Google Scholar
  7. 7.
    Xing, E.P., Ng, A.Y., Jordan, M.I., Russell, S.: Distance Metric Learning with Application to Clustering with Side-Information. In: Neural Information Processing Systems, vol. 15, pp. 505–512. MIT Press (2002)Google Scholar
  8. 8.
    Davis, J.V., Kulis, B., Jain, P., Sra, S., Dhillon, I.S.: Information-Theoretic Metric Learning. In: Proceedings of the 24th International Conference on Machine Learning, ICML 2007, pp. 209–216. ACM Press, New York (2007)Google Scholar
  9. 9.
    Wijesinghe, D.B., Hopkins, A.M., Kelly, B.C., Welikala, N., Connolly, A.J.: Morphological Classification of Galaxies and Its Relation to Physical Properties. Monthly Notices of the Royal Astronomical Society (MNRAS) 404(4), 2077–2086 (2010)Google Scholar
  10. 10.
    Lintott, C., Schawinski, K., Slosar, A., Land, K., Bamford, S., Thomas, D., Raddick, M., Nichol, B., Szalay, A., Andreescu, D.: Galaxy Zoo: Morphologies Derived From Visual Inspection of Galaxies From The Sloan Digital Sky Survey. Monthly Notices of the Royal Astronomical Society 389(3), 1179–1189 (2008) ISSN 0035-8711Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Shereen Fouad
    • 1
  • Peter Tino
    • 1
  • Somak Raychaudhury
    • 1
  • Petra Schneider
    • 1
  1. 1.The University of BirminghamBirminghamUnited Kingdom

Personalised recommendations