Skip to main content
SpringerLink
Log in

Learning with Intelligent Teacher: Similarity Control and Knowledge Transfer

In memory of Alexey Chervonenkis

  • Conference paper
  • First Online:
Statistical Learning and Data Sciences (SLDS 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9047))

Included in the following conference series:

Abstract

This paper introduces an advanced setting of machine learning problem in which an Intelligent Teacher is involved. During training stage, Intelligent Teacher provides Student with information that contains, along with classification of each example, additional privileged information (explanation) of this example. The paper describes two mechanisms that can be used for significantly accelerating the speed of Student’s training: (1) correction of Student’s concepts of similarity between examples, and (2) direct Teacher-Student knowledge transfer.

This material is based upon work partially supported by AFRL and DARPA under contract FA8750-14-C-0008. Any opinions, findings and / or conclusions in this material are those of the authors and do not necessarily reflect the views of AFRL and DARPA.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Brachman, R., Levesque, H.: Knowledge Representation and Reasoning. Morgan Kaufmann, San Francisco (2004)

    Google Scholar 

  2. Berman, H., Westbrook, J., Feng, Z., Gillil, G., Bhat, T., Weissig, H., et al.: The protein data bank. Nucleic Acids Research 28, 235–242 (2000)

    Article  Google Scholar 

  3. Burges, C.: Simplified support vector decision rules. In: 13th International Conference on Machine Learning, pp. 71–77 (1996)

    Google Scholar 

  4. Casdagli, M.: Nonlinear prediction of chaotic time series. Physica D 35, 335–356 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  5. Chervonenkis, A.: Computer Data Analysis (in Russian). Yandex, Moscow (2013)

    Google Scholar 

  6. Izmailov, R., Vapnik, V., Vashist, A.: Multidimensional Splines with Infinite Number of Knots as SVM Kernels. In: International Joint Conference on Neural Networks, pp. 1096–1102. IEEE Press, New York (2013)

    Google Scholar 

  7. Kimeldorf, G., Wahba, G.: Some Results on Tchebycheffian Spline Functions. Journal of Mathematical Analysis and Applications 33, 82–95 (1971)

    Article  MATH  MathSciNet  Google Scholar 

  8. Kuang, R., Le, E., Wang, K., Wang, K., Siddiqi, M., Freund, Y., et al.: Profile-Based String Kernels for Remote Homology Detection and Motif Extraction. Journal of Bioinformatics and Computational Biology 3, 527–550 (2005)

    Article  Google Scholar 

  9. Liao, L., Noble, W.: Combining Pairwise Sequence Similarity and Support Vector Machines for Remote Protein Homology Detection. Journal of Computational Biology 10, 857–868 (2003)

    Article  Google Scholar 

  10. Mukherjee, S., Osuna, E., Girosi, F.: Nonlinear Prediction of Chaotic Time Series Using Support Vector Machines. Neural Networks for Signal Processing, 511–520 (1997)

    Google Scholar 

  11. Murzin, A., Brenner, S., Hubbard, T., Chothia, C.: SCOP: A Structural Classification of Proteins Database for Investigation of Sequences and Structures. Journal of Molecular Biology 247, 536–540 (1995)

    Google Scholar 

  12. NEC Labs America. http://ml.nec-labs.com/download/data/svm+/mnist.priviledged

  13. Ortiz, A., Strauss, C., Olmea, O.: MAMMOTH (Matching Molecular Models Obtained from Theory): An Automated Method for Model Comparison. Protein Science 11, 2606–2621 (2002)

    Article  Google Scholar 

  14. Pechyony, D., Izmailov, R., Vashist, A., Vapnik, V.: SMO-style Algorithms for Learning Using Privileged Information. In: 2010 International Conference on Data Mining, pp. 235–241 (2010)

    Google Scholar 

  15. Tsybakov, A.: Optimal Aggregation of Classifiers in Statistical Learning. Annals of Statistcs 31, 135–166 (2004)

    MathSciNet  Google Scholar 

  16. Schölkopf, B., Herbrich, R., Smola, A.J.: A Generalized Representer Theorem. In: Helmbold, D.P., Williamson, B. (eds.) COLT 2001 and EuroCOLT 2001. LNCS (LNAI), vol. 2111, p. 416. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  17. Steinwart, I., Scovel, C. When do support machines learn fast?. In: 16th International Symposium on Mathematical Theory of Networks and Systems (2004)

    Google Scholar 

  18. Vapnik, V.: The Nature of Statistical Learning Theory. Springer-Verlag, New York (1995)

    Book  MATH  Google Scholar 

  19. Vapnik, V.: Statistical Learning Theory. John Wiley & Sons, New York (1998)

    MATH  Google Scholar 

  20. Vapnik, V.: Estimation of Dependencies Based on Empirical. Data (2nd Edition). Springer, New York (2006)

    Google Scholar 

  21. Vapnik, V., Chervonenkis, A.: Theory of Pattern Recognition (in Russian). Nauka, Moscow (1974). German translation: Wapnik W., Tscherwonenkis, A.: Theorie des Zeichenerkennung. Akademie-Verlag, Berlin (1974)

    Google Scholar 

  22. Vapnik, V., Vashist, A.: A New Learning Paradigm: Learning Using Privileged Information. Neural Networks 22, 546–557 (2009)

    Article  Google Scholar 

  23. Vapnik, V., Izmailov, R.: Statistical inference problems and their rigorous solutions. In: Gammerman, A., Vovk, V., Papadopoulos, H. (eds.) Statistical Learning and Data Sciences. SLDS 2015, LNCS (LNAI), vol. 9047, pp. 33–71. Springer-Verlag, London (2015)

    Google Scholar 

  24. Vapnik, V., Braga, I., Izmailov, R.: A Constructive Setting for the Problem of Density Ratio Estimation. In: SIAM International Conference on Data Mining, pp. 434–442 (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Columbia University, New York, NY, USA

    Vladimir Vapnik

  2. AI Research Lab, Facebook, New York, NY, USA

    Vladimir Vapnik

  3. Applied Communication Sciences, Basking Ridge, NJ, USA

    Rauf Izmailov

Authors
  1. Vladimir Vapnik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rauf Izmailov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vladimir Vapnik .

Editor information

Editors and Affiliations

  1. University of London, Egham, Surrey, United Kingdom

    Alexander Gammerman

  2. University of London, Egham, Surrey, United Kingdom

    Vladimir Vovk

  3. Frederick University, Nicosia, Cyprus

    Harris Papadopoulos

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Vapnik, V., Izmailov, R. (2015). Learning with Intelligent Teacher: Similarity Control and Knowledge Transfer. In: Gammerman, A., Vovk, V., Papadopoulos, H. (eds) Statistical Learning and Data Sciences. SLDS 2015. Lecture Notes in Computer Science(), vol 9047. Springer, Cham. https://doi.org/10.1007/978-3-319-17091-6_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-17091-6_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-17090-9

  • Online ISBN: 978-3-319-17091-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation