Skip to main content
SpringerLink
Log in
Book cover

Multiple Instance Learning pp 67–98Cite as

Instance-Based Classification Methods

  • Chapter
  • First Online:

  • 1472 Accesses

Abstract

Instance-based classification algorithms perform their main learning process at the instance level. They try to approximate a function that assigns class labels to instances. The instance classifier is combined with an underlying MI assumption, which links the class label of instances inside a bag with the bag class label. Many strategies have been devised to construct the instance classifier. We discuss the most prominent of them: wrapper methods (Sect. 4.2), maximum likelihood methods (Sect. 4.3), decision trees and rules methods (Sect. 4.4), maximum margin methods (Sect. 4.5), connectionist methods (Sect. 4.6), and evolutionary methods (Sect. 4.7). An experimental analysis on the performance of representative instance-based classifiers is presented in Sect. 4.8. Summarizing remarks are given in Sect. 4.9.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Amar, R.A., Dooly, D.R., Goldman, S.A., Zhang, Q.: Multiple-instance learning of real-valued data. In: Brodley, C., Danyluk, A. (eds.) Proceedings of the 18th International Conference on Machine Learning (ICML 2001), pp. 3–10. Morgan Kaufmann Publishers, San Francisco (2001)

    Google Scholar 

  2. Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing Systems, pp. 561–568. MIT Press, Cambridge (2002)

    Google Scholar 

  3. Babenko, B., Belongie, S., Yang, M.H.: Visual tracking with online multiple instance learning. In: Flynn, P., Mortensen, E. (eds.) Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2009), pp. 983–990. IEEE, Los Alamitos (2009)

    Google Scholar 

  4. Bascom, J.: Darwin’s theory of the origin of species. Am. Theol. Rev. 3, 349–379 (1871)

    Google Scholar 

  5. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)

    MATH  Google Scholar 

  6. Bjerring, L., Frank, E.: Beyond trees: adopting MITI to learn rules and ensemble classifiers for multi-instance data. In: Wang, D., Reynolds, M. (eds.) Lecture Notes in Artificial Intelligence, pp. 41–50. Springer, Berlin (2011)

    Google Scholar 

  7. Blockeel, H., Page, D., Srinivasan, A.: Multi-instance tree learning. In: De Raedt, L., Wrobel, S. (eds.) Proceedings of the 22nd International Conference on Machine Learning (ICML 2005), pp. 57–64. ACM, New York (2005)

    Google Scholar 

  8. Bouchachia, A.: Multiple instance learning with radial basis function neural networks. In: Pal, N., Kasabov, N., Mudi, R., Pal, S., Parui, S. (eds.) Advances in Neural Information Processing Systems (NIPS Conference), pp. 440–445. Springer, Berlin (2004)

    Google Scholar 

  9. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  10. Bunescu, R., Mooney, R.: Multiple instance learning for sparse positive bags. In: Proceedings of the 24th International Conference on Machine Learning (ICML 2007), pp. 105–112. ACM, New York (2007)

    Google Scholar 

  11. Cano, A., Zafra, A., Ventura, S.: Speeding up multiple instance learning classification rules on GPUs. Knowl. Inf. Syst. 44(1), 127–145 (2015)

    Article  Google Scholar 

  12. Chen, Y., Wang, J.: Image categorization by learning and reasoning with regions. J. Mach. Learn. Res. 5, 913–939 (2004)

    MathSciNet  Google Scholar 

  13. Cheung, P., Kwok, J.: A regularization framework for multiple-instance learning. In: Ghahramani, Z. (ed.) Proceedings of the 23rd International Conference on Machine learning, pp. 193–200 (2006)

    Google Scholar 

  14. Chevaleyre, Y., Zucker, J.: Solving multiple-instance and multiple-part learning problems with decision trees and rule sets. Application to the mutagenesis problem. In: Stroulia, E., Matwin, S. (eds.) Lecture Notes in Artificial Intelligence, pp. 204–214. Springer, Berlin (2001)

    Google Scholar 

  15. Chevaleyre, Y., Bredeche, N., Zucker, J.: Learning rules from multiple instance data: issues and algorithms. In: Proceedings of the 9th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU 2002), pp. 455–459. Esia, Annecy (2002)

    Google Scholar 

  16. Chien, B.C., Lin, J.Y., Hong, T.P.: Learning discriminant functions with fuzzy attributes for classification using genetic programming. Expert Syst. Appl. 23(1), 31–37 (2002)

    Article  Google Scholar 

  17. Chuang, S.C., Xu, Y.Y., Fu, H.C.: Neural network based image retrieval with multiple instance leaning techniques. In: Khosla, R., Howlett, R., Jain, L. (eds.) Lecture Notes in Artificial Intelligence, pp. 1210–1216. Springer, Berlin (2005)

    Google Scholar 

  18. Cohen, W.: Fast effective rule induction. In: Prieditis, A., Russell, S. (eds.) Proceedings of the 12th International Conference on Machine Learning, pp. 115–123. Morgan Kaufmann, San Francisco (1995)

    Google Scholar 

  19. Davis, R.A., Charlton, A.J., Oehlschlager, S., Wilson, J.C.: Novel feature selection method for genetic programming using metabolomic 1H NMR data. Chemom. Intell. Lab. 81(1), 50–59 (2006)

    Article  Google Scholar 

  20. Ded, K., Pratap, A., Agarwal, S., Meyarivan, T.: A fast and elitist multi-objective genetic algorithm: NSGA-II. IEEE Trans. Evol. Comput. 6(2), 149–172 (2002)

    Google Scholar 

  21. Deza, M.M., Deza, E.: Dictionary of Distances. Elsevier, Amsterdam (2006)

    MATH  Google Scholar 

  22. Dong, L.: A comparison of multi-instance learning algorithms. Master thesis, University of Waikato, New Zealand (2006)

    Google Scholar 

  23. Feng, S., Xu, D.: Transductive multi-instance multi-label learning algorithm with application to automatic image annotation. Expert Syst. Appl. 37(1), 661–670 (2010)

    Article  Google Scholar 

  24. Frank, E., Xu, X.: Applying propositional learning algorithms to multi-instance data. Technical report 06/03, Department of Computer Science, University of Waikato, Hamilton, New Zealand (2003)

    Google Scholar 

  25. Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors). Ann. Stat. 28(2), 337–407 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  26. Fu, Z., Robles-Kelly, A.: Fast multiple instance learning via L1,2 logistic regression. In: Proceedings of the 19th International Conference on Pattern Recognition (ICPR 2008), pp. 3815–3818. IEEE, Los Alamitos (2008)

    Google Scholar 

  27. Garcez, A., Zaverucha, G.: Multi-instance learning using recurrent neural networks. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE, Los Alamitos (2012)

    Google Scholar 

  28. Geyer-Schulz, A.: Fuzzy Rule-Based Expert Systems and Genetic Machine Learning, vol. 3. Physica Verlag, Heidelberg (1997)

    MATH  Google Scholar 

  29. Goldberg, D.E.: Zen and the art of genetic algorithms. In: Proceedings of the 3rd International Conference on Genetic Algorithms, pp. 80–85. Morgan Kaufmann Publishers, San Francisco (1989)

    Google Scholar 

  30. Gondra, I., Xu, T.: A multiple instance learning based framework for semantic image segmentation. Multimed. Tools Appl. 48(2), 339–365 (2010)

    Article  Google Scholar 

  31. Grabner, H., Grabner, M., Bischof, H.: Real-time tracking via on-line boosting. In: Chantler, M., Trucco, E., Fisher, B. (eds.) Proceedings of the British Machine Vision Conference, pp. 47–56. British Machine Vision Association, Durham (2006)

    Google Scholar 

  32. Holland, J.H.: Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. MIT Press, Cambridge (1992)

    Google Scholar 

  33. Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E., Saltz, J.H.: Efficient multiple instance convolutional neural networks for gigapixel resolution image classification (2015). arXiv:1504.07947

  34. Jaszkiewicz, A.: Genetic local search for multi-objective combinatorial optimization. Eur. J. Oper. Res. 137(1), 50–71 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  35. Jolliffe, I.: Principal Component Analysis. Springer, New York (2002)

    MATH  Google Scholar 

  36. Kattan, A., Agapitos, A., Ong, Y.S., Alghamedi, A., O’Neill, M.: GP made faster with semantic surrogate modelling. Inf. Sci. 355–356, 169–185 (2016)

    Article  Google Scholar 

  37. Kishore, J.K., Patnaik, L.M., Mani, V., Agrawal, V.: Application of genetic programming for multicategory pattern classification. IEEE Trans. Evol. Comput. 4(3), 242–258 (2000)

    Article  Google Scholar 

  38. Knuth, D.E.: Backus normal form vs. Backus Naur form. Commun. ACM 7(12), 735–736 (1964)

    Article  Google Scholar 

  39. Kouchakpour, P., Zaknich, A., Bräunl, T.: Dynamic population variation in genetic programming. Inf. Sci. 179(8), 1078–1091 (2009)

    Article  Google Scholar 

  40. Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)

    MATH  Google Scholar 

  41. Leistner, C., Saffari, A., Bischof, H.: MIForests: multiple-instance learning with randomized trees. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) Computer Vision - ECCV 2010, pp. 29–42. Springer, Berlin (2010)

    Chapter  Google Scholar 

  42. Lensberg, T., Eilifsen, A., McKee, T.E.: Bankruptcy theory development and classification via genetic programming. Eur. J. Oper. Res. 169(2), 677–697 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  43. Li, C.H., Gondra, I.: A novel neural network-based approach for multiple instance learning. In: Proceedings of the 2010 IEEE 10th International Conference on Computer and Information Technology (CIT), pp. 451–456. IEEE, Los Alamitos (2010)

    Google Scholar 

  44. Li, C.H., Gondra, I., Liu, L.: An efficient parallel neural network-based multi-instance learning algorithm. J. Supercomput. 62(2), 724–740 (2012)

    Article  Google Scholar 

  45. Maron, O.: Learning from ambiguity. Ph.D. thesis, Massachusetts Institute of Technology, United States of America (1998)

    Google Scholar 

  46. Maron, O., Lozano-Pérez, T.: A framework for multiple-instance learning. In: Jordan, M., Kearns, M., Solla, S. (eds.) Advances in Neural Information Processing Systems, pp. 570–576. MIT Press, Cambridge (1998)

    Google Scholar 

  47. Mason, L., Baxter, J., Bartlett, P.L., Frean, M.R.: Boosting algorithms as gradient descent. In: Solla, S., Leen, T., Müller, K. (eds.) Advances in Neural Information Processing Systems, pp. 512–518. MIT Press, Cambridge (2000)

    Google Scholar 

  48. McGovern, A., Jensen, D.: Identifying predictive structures in relational data using multiple instance learning. In: Fawcett, T., Mishra, N. (eds.) Proceedings of the 20th International Conference on Machine Learning (ICML 2003), pp. 528–535. The AAAI Press, Menlo Park (2003)

    Google Scholar 

  49. Mckay, R.I., Hoai, N.X., Whigham, P.A., Shan, Y., O‘Neill, M.: Grammar-based genetic programming: a survey. Genet. Program. Evol. M 11(3–4), 365–396 (2010)

    Google Scholar 

  50. Muharram, M., Smith, G.D.: Evolutionary constructive induction. IEEE Trans. Knowl. Data Eng. 17(11), 1518–1528 (2005)

    Article  Google Scholar 

  51. Oza, N.C.: Online ensemble learning. Ph.D. thesis, University of California, Berkeley, United States of America (2001)

    Google Scholar 

  52. Pao, H., Chuang, S., Xu, Y., Fu, H.: An EM based multiple instance learning method for image classification. Expert Syst. Appl. 35(3), 1468–1472 (2008)

    Article  Google Scholar 

  53. Qi, Z., Xu, Y., Wang, L., Song, Y.: Online multiple instance boosting for object detection. Neurocomputing 74(10), 1769–1775 (2011)

    Article  Google Scholar 

  54. Quinlan, J.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)

    Google Scholar 

  55. Ramon, J., De Raedt, L.: Multi instance neural networks. In: Proceedings of the ICML-2000 Workshop on Attribute-Value and Relational Learning, pp. 53–60. Morgan Kaufmann Publishers, San Francisco (2000)

    Google Scholar 

  56. Ray, S., Craven, M.: Supervised versus multiple instance learning: an empirical comparison. In: De Raedt, L., Wrobel, S. (eds.) Proceedings of the 22nd International Conference on Machine Learning (ICML 2005), pp. 697–704. ACM, New York (2005)

    Google Scholar 

  57. Rose, K., Gurewitz, E., Fox, G.: Deterministic annealing, constrained clustering, and optimization. In: Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN), pp. 2515–2520. IEEE, Los Alamitos (1991)

    Google Scholar 

  58. Ruffo, G.: Learning single and multiple instance decision trees for computer security applications. Ph.D. thesis, University of Turin, Italy (2000)

    Google Scholar 

  59. Saul, L.K., Rahim, M.G., Allen, J.B.: A statistical model for robust integration of narrowband cues in speech. Comput. Speech Lang. 15, 175–194 (2001)

    Article  Google Scholar 

  60. Schapire, R.E., Singer, Y.: Improved boosting algorithms using confidence-rated predictions. Mach. Learn. 37(3), 297–336 (1999)

    Article  MATH  Google Scholar 

  61. Smola, A.J., Vishwanathan, S., Hofmann, T.: Kernel methods for missing variables. In: Cowell, R., Ghahramani, Z. (eds.) Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics (AISTATS), pp. 325–332. The Society for Artificial Intelligence and Statistics (2005)

    Google Scholar 

  62. Song, Y., Li, Q.: Visual tracking based on multiple instance learning particle filter. In: Proceedings of the 2011 IEEE International Conference on Mechatronics and Automation, pp. 1063–1067. IEEE, Los Alamitos (2011)

    Google Scholar 

  63. Sternig, S., Roth, P., Bischof, H.: Inverse multiple instance learning for classifier grids. In: Proceedings of the 20th International Conference on Pattern Recognition (ICPR), pp. 770–773. IEEE, Los Alamitos (2010)

    Google Scholar 

  64. Tsakonas, A.: A comparison of classification accuracy of four genetic programming-evolved intelligent structures. Inf. Sci. 176(6), 691–724 (2006)

    Article  Google Scholar 

  65. Uwents, W., Blockeel, H.: Classifying relational data with neural networks. In: Kramer, S., Pfahringer, B. (eds.) Lecture Notes in Artificial Intelligence, pp. 384–396. Springer, Berlin (2005)

    Google Scholar 

  66. Uwents, W., Blockeel, H.: A comparison between neural network methods for learning aggregate functions. In: Jean-Fran, J., Berthold, M., Horváth, T. (eds.) Lecture Notes in Artificial Intelligence, pp. 88–99. Springer, Berlin (2008)

    Google Scholar 

  67. Viola, P., Platt, J., Zhang, C.: Multiple instance boosting for object detection. In: Weiss, Y., Schölkopf, B., Platt, J. (eds.) Advances in Neural Information Processing Systems, pp. 1417–1424. MIT Press, Cambridge (2005)

    Google Scholar 

  68. Whigham, P.A.: Grammatically-based genetic programming. In: Rosca, J.P. (ed.) Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications, pp. 33–41. University of Rochester, Rochester (1995)

    Google Scholar 

  69. Wu, X., Kumar, V., Quinlan, J.R., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., Ng, A., Liu, B., Philip, S.Y., Zhou, Z., Steinbach, M., Hand, D., Steinberg, D.: Top 10 algorithms in data mining. Knowl. Inf. Syst. 14(1), 1–37 (2008)

    Article  Google Scholar 

  70. Xie, Y., Qu, Y., Li, C., Zhang, W.: Online multiple instance gradient feature selection for robust visual tracking. Pattern Recognit. Lett. 33(9), 1075–1082 (2012)

    Article  Google Scholar 

  71. Xu, X.: Statistical learning in multiple instance problems. Master thesis, University of Waikato, New Zealand (2003)

    Google Scholar 

  72. Xu, X., Frank, E.: Logistic regression and boosting for labeled bags of instances. In: Dai, H., Srikant, R., Zhang, C. (eds.) Lecture Notes in Artificial Intelligence, pp. 272–281. Springer, Berlin (2004)

    Google Scholar 

  73. Xu, Y.Y., Shih, C.H.: Multiple-instance learning via decision-based neural networks. In: Watada, J., Philips-Wren, G., Jain, L., Howlett, R. (eds.) Intelligent Decision Technologies, pp. 885–895. Springer, Berlin (2011)

    Chapter  Google Scholar 

  74. Yang, C.Y.C., Dong, M.D.M., Hua, J.H.J.: Region-based image annotation using asymmetrical support vector machine-based multiple-instance learning. In: Fitzgibbon, A., Taylor, C., LeCun, Y. (eds.) Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), pp. 2057–2063. IEEE, Los Alamitos (2006)

    Google Scholar 

  75. Zafra, A., Ventura, S.: Predicting student grades in learning management systems with multiple instance learning genetic programming. In: Barnes, T., Desmarais, M., Romero, C., Ventura, S. (eds.) Proceedings of the 2nd International Conference on Educational Data Mining, pp. 309–318 (2009)

    Google Scholar 

  76. Zafra, A., Ventura, S.: G3p-MI: a genetic programming algorithm for multiple instance learning. Inf. Sci. 180(23), 4496–4513 (2010)

    Article  Google Scholar 

  77. Zafra, A., Ventura, S.: Multi-objective approach based on grammar-guided genetic programming for solving multiple instance problems. Soft Comput. 16(6), 955–977 (2012)

    Article  Google Scholar 

  78. Zafra, A., Romero, C., Ventura, S., Herrera-Viedma, E.: Multi-instance genetic programming for web index recommendation. Expert Syst. Appl. 36(9), 11470–11479 (2009)

    Article  Google Scholar 

  79. Zeisl, B., Leistner, C., Saffari, A., Bischof, H.: On-line semi-supervised multiple-instance boosting. In: Boykov, Y., Schmidt, F.R., Kahl, F., Lemptisky, V. (eds.) Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2010), pp. 1879–1879. IEEE, Los Alamitos (2010)

    Google Scholar 

  80. Zhang, Q., Goldman, S.A.: EM-DD: an improved multiple-instance learning technique. In: Dietterich, T., Becker, S., Ghahramani (eds.) Advances in Neural Information Processing Systems, pp. 1073–1080. MIT Press, Cambridge (2001)

    Google Scholar 

  81. Zhang, M., Zhou, Z.: Adapting RBF neural networks to multi-instance learning. Neural Process. Lett. 23(1), 1–26 (2006)

    Article  Google Scholar 

  82. Zhang, M.L., Zhou, Z.H.: A multi-instance regression algorithm based on neural network. J. Softw. 14(7), 1238–1242 (2003)

    MATH  Google Scholar 

  83. Zhang, M.L., Zhou, Z.H.: Improve multi-instance neural networks through feature selection. Neural Process. Lett. 10(1), 1–10 (2004)

    Google Scholar 

  84. Zhou, Z., Xu, J.: On the relation between multi-instance learning and semi-supervised learning. In: Ghahramani, Z. (ed.) Proceedings of the 24th International Conference on Machine Learning (ICML 2007), pp. 1167–1174. ACM, New York (2007)

    Google Scholar 

  85. Zhou, Z., Zhang, M.: Neural networks for multi-instance learning. Technical report, Department of Computer Science and Technology, Nanjing University, Nanjing, China (2002)

    Google Scholar 

  86. Zhou, Z., Jiang, K., Li, M.: Multi-instance learning based web mining. Appl. Intell. 22(2), 135–147 (2005)

    Article  Google Scholar 

  87. Zitzler, E., Laumanns, M., Thiele, L.: SPEA2: improving the strength Pareto evolutionary algorithm. Eurogen 3242(103), 95–100 (2001)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Artificial Intelligence, University of Granada, Granada, Spain

    Francisco Herrera

  2. Department of Computer Science, University of Córdoba, Córdoba, Spain

    Sebastián Ventura

  3. Center of Information Studies, Central University “Marta Abreu” of Las Villas, Santa Clara, Villa Clara, Cuba

    Rafael Bello

  4. Department of Applied Mathematics, Computer Science and Statistics, Ghent University, Ghent, Belgium

    Chris Cornelis & Sarah Vluymans

  5. Department of Computer Science and Numerical Analysis, University of Córdoba, Córdoba, Spain

    Amelia Zafra

  6. Central University “Marta Abreu” of Las Villas, Santa Clara, Villa Clara, Cuba

    Dánel Sánchez-Tarragó

Authors
  1. Francisco Herrera
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sebastián Ventura
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rafael Bello
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chris Cornelis
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Amelia Zafra
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Dánel Sánchez-Tarragó
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Sarah Vluymans
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Francisco Herrera .

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this chapter

Cite this chapter

Herrera, F. et al. (2016). Instance-Based Classification Methods. In: Multiple Instance Learning. Springer, Cham. https://doi.org/10.1007/978-3-319-47759-6_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-47759-6_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-47758-9

  • Online ISBN: 978-3-319-47759-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation