Predicting Structured Outputs k-Nearest Neighbours Method

  • Mitja Pugelj
  • Sašo Džeroski
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6926)


In this work, we address several tasks of structured prediction and propose a new method for handling such tasks. Structured prediction is becoming important as data mining is dealing with increasingly complex data (images, videos, sound, graphs, text,...). Our method, k-NN for structured prediction (kNN-SP), is an extension of the well known k-nearest neighbours method and can handle three different structured prediction problems: multi-target prediction, hierarchical multi-label classification, and prediction of short time-series. We evaluate the performance of kNN-SP on several datasets for each task and compare it to the performance of other structured prediction methods (predictive clustering trees and rules). We show that, despite it’s simplicity, the kNN-SP method performs satisfactory on all tested problems.


Near Neighbor Dynamic Time Warping Structure Output Descriptive Attribute Short Time Series 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Bakir, G.H., Hofmann, T., Schölkopf, B., Smola, A.J., Taskar, B., Vishwanathan, S.V.N.: Predicting Structured Data. The MIT Press, Cambridge (2007)Google Scholar
  2. 2.
    Blockeel, H., De Raedt, L., Ramon, J.: Top-down induction of clustering trees. In: Proc. 15th International Conference on Machine Learning, pp. 55–63. Morgan Kaufmann, San Francisco (1998)Google Scholar
  3. 3.
    Blockeel, H., Struyf, J.: Efficient algorithms for decision tree cross-validation. Journal of Machine Learning Research 3, 621–650 (2002)zbMATHGoogle Scholar
  4. 4.
    Bratko, I.: Prolog Programming for Artificial Intelligence, 3rd edn. Addison Wesley, Reading (2000)zbMATHGoogle Scholar
  5. 5.
    Dasarathy, B.V. (ed.): Nearest Neighbor (NN) Norms: NN Pattern Classification Techniques. IEEE Computer Society, Los Alamitos (1990)Google Scholar
  6. 6.
    Demšar, J.: Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, 1–30 (2006)MathSciNetzbMATHGoogle Scholar
  7. 7.
    Džeroski, S.: Towards a general framework for data mining. In: Džeroski, S., Struyf, J. (eds.) KDID 2006. LNCS, vol. 4747, pp. 259–300. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  8. 8.
    Kocev, D.: Ensembles for Predicting Structured Outputs. PhD Thesis, Jozef Stefan International Postgraduate School, Ljubljana, Slovenia (2011)Google Scholar
  9. 9.
    Silla, C., Freitas, A.: A survey of hierarchical classification across different application domains. Data Mining and Knowledge Discovery 22(1-2), 31–72 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  10. 10.
    Slavkov, I., Gjorgjioski, V., Struyf, J., Dzeroski, S.: Finding explained groups of time-course gene expression profiles with predictive clustering trees. Molecular BioSystems 6, 729–740 (2010)CrossRefGoogle Scholar
  11. 11.
    Vens, C., Struyf, J., Schietgat, L., Džeroski, S., Blockeel, H.: Decision trees for hierarchical multi-label classification. Machine Learning 73(2), 185–214 (2008)CrossRefGoogle Scholar
  12. 12.
    Ženko, B., Džeroski, S.: Learning classification rules for multiple target attributes. In: Washio, T., Suzuki, E., Ting, K.M., Inokuchi, A. (eds.) PAKDD 2008. LNCS (LNAI), vol. 5012, pp. 454–465. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  13. 13.
    Yianilos, P.N.: Data structures and algorithms for nearest neighbor search in general metric spaces. In: Proc. 4th ACM-SIAM Symposium on Discrete Algorithms, pp. 311–321. SIAM, Philadelphia (1993)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Mitja Pugelj
    • 1
  • Sašo Džeroski
    • 1
  1. 1.Department of Knowledge TechnologiesJožef Stefan InstituteLjubljanaSlovenia

Personalised recommendations