Advertisement

Neural Processing Letters

, Volume 23, Issue 1, pp 1–26 | Cite as

Adapting RBF Neural Networks to Multi-Instance Learning

  • Min-Ling Zhang
  • Zhi-Hua Zhou
Article

Abstract

In multi-instance learning, the training examples are bags composed of instances without labels, and the task is to predict the labels of unseen bags through analyzing the training bags with known labels. A bag is positive if it contains at least one positive instance, while it is negative if it contains no positive instance. In this paper, a neural network based multi-instance learning algorithm named RBF-MIP is presented, which is derived from the popular radial basis function (RBF) methods. Briefly, the first layer of an RBF-MIP neural network is composed of clusters of bags formed by merging training bags agglomeratively, where Hausdorff metric is utilized to measure distances between bags and between clusters. Weights of second layer of the RBF-MIP neural network are optimized by minimizing a sum-of-squares error function and worked out through singular value decomposition (SVD). Experiments on real-world multi-instance benchmark data, artificial multi-instance benchmark data and natural scene image database retrieval are carried out. The experimental results show that RBF-MIP is among the several best learning algorithms on multi-instance problems.

Keywords

content-based image retrieval Hausdorff distance machine learning multi-instance learning neural networks principle component analysis radial basis function singular value decomposition 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Alphonse, E., Matwin, S. 2004Filtering multi-instance problems to reduce dimensionality in relational learningJournal of Intelligent Information Systems222340CrossRefGoogle Scholar
  2. 2.
    Amar, R. A., Dooly, D. R., Goldman, S. A. and Zhang, Q.: Multiple-instance learning of real-valued data, In: Proceedings of the 18th International Conference on Machine Learning, pp. 3–10, Williamstown, MA, 2001. [http://www.cs.wustl.edu/~sg/multi-inst-data]Google Scholar
  3. 3.
    Andrews, S., Tsochantaridis, I., Hofmann, T. 2003Support vector machines for multiple-instance learningBecker, S.Thrun, S.Obermayer, K. eds. Advances in Neural Information Processing Systems 15MIT PressCambridge, MA561568Google Scholar
  4. 4.
    Auer, P.: On learning from multi-instance examples: empirical evaluation of a theoretical approach, In: Proceedings of the 14th International Conference on Machine Learning, pp. 21–29, Nashville, TN, 1997.Google Scholar
  5. 5.
    Auer, P., Long, P. M., Srinivasan, A. 1998Approximating hyper-rectangles: learning and pseudo-random setsJournal of Computer and System Sciences57376388CrossRefMathSciNetGoogle Scholar
  6. 6.
    Bishop, C. M. 1995Neural Networks for Pattern RecognitionOxford University PressNew YorkGoogle Scholar
  7. 7.
    Blake, C., Keogh, E. and Merz, C. J.: UCI repository of machine learning databases. Department of Information and Computer Science, University of California, Irvine, CA, 1998. [http://www.ics.uci.edu/~mlearn/MLRepository.html]Google Scholar
  8. 8.
    Blum, A., Kalai, A. 1998A note on learning from multiple-instance examplesMachine Learning302329CrossRefGoogle Scholar
  9. 9.
    Chevaleyre, Y., Zucker, J.-D. 2001Solving multiple-instance and multiple-part learning problems with decision trees and decision rules. Application to the mutagenesis problemStroulia, E.Matwin, S. eds. Lecture Notes in Artificial Intelligence 2056SpringerBerlin204214Google Scholar
  10. 10.
    Dempster, A. P., Laird, N. M., Rubin, D. B. 1977Maximum likelihood from incomplete data via the EM algorithmJournal of the Royal Statistics Society, Series B39138MathSciNetGoogle Scholar
  11. 11.
    Raedt, L. 1998Attribute-value learning versus inductive logic programming: the missing linksPage, D. eds. Lecture Notes in Artificial Intelligence 1446SpringerBerlin18Google Scholar
  12. 12.
    Dietterich, T. G., Lathrop, R. H., Lozano-Pérez, T. 1997Solving the multiple-instance problem with axis-parallel rectanglesArtificial Intelligence893171Google Scholar
  13. 13.
    Dooly, D. R., Goldman, S. A., Kwek, S. S. 2001Real-valued multiple-instance learning with queriesAbe, N.Khardon, R.Zeugmann, T. eds. Lecture Notes in Artificial Intelligence 2225SpringerBerlin167180Google Scholar
  14. 14.
    Edgar, G. A. 1995Measure, Topology, and Fractal GeometrySpringer-VerlagBerlin3rd printGoogle Scholar
  15. 15.
    Gärtner, T., Flach, P. A., Kowalczyk, A. and Smola, A. J.: Multi-instance kernels, In: Proceedings of the 19th International Conference on Machine Learning, pp. 179–186, Sydney, Australia, 2002.Google Scholar
  16. 16.
    Goldman, S. A., Kwek, S. S., Scott, S. D. 2001Agnostic learning of geometric patternsJournal of Computer and System Sciences62123151CrossRefMathSciNetGoogle Scholar
  17. 17.
    Goldman, S. A., Scott, S. D. 2003Multiple-instance learning of real-valued geometric patternsAnnals of Mathematics and Artificial Intelligence39259290CrossRefMathSciNetGoogle Scholar
  18. 18.
    Jollife, I. T. 1986Principle Component AnalysisSpringer-VerlagNew YorkGoogle Scholar
  19. 19.
    Kearns, M. J.: Efficient noise-tolerant learning from statistical queries, In: Proceedings of the 25th Annual ACM Symposium on Theory of Computing, pp. 392–401, San Diego, CA, 1993.Google Scholar
  20. 20.
    Kearns, M. J., Schapire, R. E. 1994Efficient distribution-free learning of probabilistic conceptsJournal of Computer and System Sciences48464497CrossRefMathSciNetGoogle Scholar
  21. 21.
    Lindsay, R., Buchanan, B., Feigenbaum, E., Lederberg, J. 1980Applications of Artificial Intelligence to Organic Chemistry: The DENDRAL ProjectMcGraw-HillNew YorkGoogle Scholar
  22. 22.
    Long, P. M., Tan, L. 1998PAC learning axis-aligned rectangles with respect to product distribution from multiple-instance examplesMachine Learning30721CrossRefGoogle Scholar
  23. 23.
    Maron, O. 1998Learning from AmbiguityDepartment of Electronical Engineering and Computer Science, MITCambridge, MAJun. 1998, PhD dissertationGoogle Scholar
  24. 24.
    Maron, O., Lozano-Pérez, T. 1998A framework for multiple-instance learningJordan, M. I.Kearns, M. J.Solla, S. A. eds. Advances in Neural Information Processing Systems 10MIT PressCambridge, MA570576Google Scholar
  25. 25.
    Maron, O. and Ratan, A. L.: Multiple-instance learning for natural scene classification, In: Proceedings of the 15th International Conference on Machine Learning, pp. 341–349, Madison, WI, 1998.Google Scholar
  26. 26.
    Press, W. H., Teukolsky, S. A., Vetterling, W. T., Flannery, B. P. 1992Numerical Recipes in C: The Art of Scientific Computing2Cambridge University PressNew YorkGoogle Scholar
  27. 27.
    Ray, S. and Page, D.: Multiple instance regression, In: Proceedings of the 18th International Conference on Machine Learning, pp. 425–432, Williamstown, MA, 2001.Google Scholar
  28. 28.
    Ruffo, G. 2000Learning single and multiple decision trees for security applicationsDepartment of Computer Science, University of TurinItalyPhD dissertationGoogle Scholar
  29. 29.
    Rumelhart, D. E., Hinton, G. E., Williams, R. J. 1986Learning internal representations by error propagationRumelhart, D. E.McClelland, J. L. eds. Parallel Distributed Processing: Explorations in the Microstructure of CognitionMIT PressCambridge, MA318362Google Scholar
  30. 30.
    Sebag, M. and Rouveirol, C.: Tractable induction and classification in first order logic, In: Proceedings of the 15th International Joint Conference on Artificial Intelligence, pp. 888–893, Nagoya, Japan, 1997.Google Scholar
  31. 31.
    Wang, J. and Zucker, J.-D.: Solving the multiple-instance problem: a lazy learning approach, In: Proceedings of the 17th International Conference on Machine Learning, pp. 1119–1125, San Francisco, CA, 2000.Google Scholar
  32. 32.
    Yang, C. and Lozano- Pérez, T.: Image database retrieval with multiple-instance learning techniques, In: Proceedings of the 16th International Conference on Data Engineering, pp. 233–243, San Diego, CA, 2000.Google Scholar
  33. 33.
    Zhang, Q., Goldman, S. A. 2002EM-DD: an improved multiple-instance learning techniqueDietterich, T. G.Becker, S.Ghahramani, Z. eds. Advances in Neural Information Processing Systems 14MIT PressCambridge, MA10731080Google Scholar
  34. 34.
    Zhang, Q., Yu, W., Goldman, S. A. and Fritts, J. E.: Content-based image retrieval using multiple-instance learning, In: Proceedings of the 19th International Conference on Machine Learning, pp. 682–689, Sydney, Australia, 2002.Google Scholar
  35. 35.
    Zhang, M.-L., Zhou, Z.-H. 2004Improve multi-instance neural network through feature selectionNeural Processing Letters19110CrossRefGoogle Scholar
  36. 36.
    Zhou, Z.-H., Wu, J., Tang, W. 2002Ensembling neural networks: many could be better than allArtificial Intelligence137239263MathSciNetGoogle Scholar
  37. 37.
    Zhou, Z.-H., Zhang, M.-L. 2002Neural networks for multi-instance learningAI Lab, Computer Science & Technology Department, Nanjing UniversityChinaTechnical Report, Aug.Google Scholar
  38. 38.
    Zhou, Z.-H., Zhang, M.-L. 2003Ensembles of multi-instance learnersLavrač, N.Gamberger, D.Blockeel, H.Todorovski, L. eds. Lecture Notes in Artificial Intelligence 2837Springer-VerlagBerlin492502Google Scholar
  39. 39.
    Zucker, J.-D. and Ganascia, J.-G.: Changes of representation for efficient learning in structural domains, In: Proceedings of the 13th International Conference on Machine Learning, pp. 543–551, Bary, Italy, 1996.Google Scholar
  40. 40.
    Zucker, J.-D., Ganascia, J.-G. 1998Learning structurally indeterminate clausesPage, D. eds. Lecture Notes in Artificial Intelligence 1446SpringerBerlin235244Google Scholar

Copyright information

© Springer 2006

Authors and Affiliations

  1. 1.National Laboratory for Novel Software TechnologyNanjing UniversityNanjingChina

Personalised recommendations