Skip to main content
Log in

Filtering Multi-Instance Problems to Reduce Dimensionality in Relational Learning

  • Published:
Journal of Intelligent Information Systems Aims and scope Submit manuscript

Abstract

Attribute-value based representations, standard in today's data mining systems, have a limited expressiveness. Inductive Logic Programming provides an interesting alternative, particularly for learning from structured examples whose parts, each with its own attributes, are related to each other by means of first-order predicates. Several subsets of first-order logic (FOL) with different expressive power have been proposed in Inductive Logic Programming (ILP). The challenge lies in the fact that the more expressive the subset of FOL the learner works with, the more critical the dimensionality of the learning task. The Datalog language is expressive enough to represent realistic learning problems when data is given directly in a relational database, making it a suitable tool for data mining. Consequently, it is important to elaborate techniques that will dynamically decrease the dimensionality of learning tasks expressed in Datalog, just as Feature Subset Selection (FSS) techniques do it in attribute-value learning. The idea of re-using these techniques in ILP runs immediately into a problem as ILP examples have variable size and do not share the same set of literals. We propose here the first paradigm that brings Feature Subset Selection to the level of ILP, in languages at least as expressive as Datalog. The main idea is to first perform a change of representation, which approximates the original relational problem by a multi-instance problem. The representation obtained as the result is suitable for FSS techniques which we adapted from attribute-value learning by taking into account some of the characteristics of the data due to the change of representation. We present the simple FSS proposed for the task, the requisite change of representation, and the entire method combining those two algorithms. The method acts as a filter, preprocessing the relational data, prior to the model building, which outputs relational examples with empirically relevant literals. We discuss experiments in which the method was successfully applied to two real-world domains.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Almuallim, H. and Dietterich, T.G. (1994). Learning Boolean Concepts in the Presence of Many Irrelevant Features. Artificial Intelligence, 69(1/2), 279-305.

    Google Scholar 

  • Alphonse, E. and Rouveirol, C. (2000). Lazy Propositionalization for Relational Learning. In W. Horn (Ed.), Proc. of the 14th European Conference on Artificial Intelligence (ECAI'2000) (pp. 256-260). IOS Press.

  • Bisson, G. (1991). KBG: A Generator of Knowledge Bases. In Y. Kodratoff (Ed.), Proceedings of the European Working Session on Learning: Machine Learning (EWSL-91), Vol. 482 of LNAI (pp. 137-137). Porto, Portugal: Springer Verlag. ISBN 3-540-53816-X.

    Google Scholar 

  • Blum, A. and Kalai, A. (1998). A Note on Learning from Multiple-Instance Examples. Machine Learning, 30, 23-29.

    Google Scholar 

  • Chevaleyre, Y. and Zucker, J. (2000). Noise-Tolerant Rule Induction for Multi-Instance Data. In ICML 2000, Workshop on Attribute-Value and Relational Learning.

  • Cohen, W.W. (1993). Learnability of Restricted Logic Programs. In S. Muggleton (Ed.), Proceedings of the 3rd International Workshop on Inductive Logic Programming (pp. 41-72). J. Stefan Institute.

  • Cohen, W.W. (1995). Learning to Classify English Text with ILP Methods. In L.D. Raedt (Ed.), Advances in Inductive Logic Programming (pp. 124-143). Amsterdam, NL: IOS Press.

    Google Scholar 

  • Conklin, D. (1995). Machine Discovery of Protein Motifs. Machine Learning, 21, 125-150.

    Google Scholar 

  • de Raedt, L. (1997). Logical Settings for Concept-Learning. Artificial Intelligence, 95(1), 187-201.

    Google Scholar 

  • Dietterich, T.G., Lathrop, R.H., and Lozano-Pérez, T. (1997). Solving the Multiple Instance Problem with Axis-Parallel Rectangles. Artificial Intelligence, 89(1/2), 31-71.

    Google Scholar 

  • Esposito, F., Malerba, D., and Semeraro, G. (1994). Multistrategy Learning for Document Recognition. Applied Artificial Intelligence, 8, 33-84.

    Google Scholar 

  • Feo, T.A. and Resende, M.G.C. (1995). Greedy Randomized Adaptive Search Procedures. Journal of Global Optimization, 6, 109-133.

    Google Scholar 

  • Fürnkranz, J. (1997). Dimensionality Reduction in ILP: A Call to Arms. In L. de Raedt and S. Muggleton (Eds.), Proceedings of the IJCAI-97 Workshop on Frontiers of Inductive Logic Programming (pp. 81-86).

  • Gottlob, G. (1987). Subsumption and Implication. Information Processing Letters, 24(2), 109-111.

    Google Scholar 

  • Jagota, A. (1993). Constraint Satisfaction and Maximum Clique. In Working Notes, AAAI Spring Symposium on AI and NP-Hard Problems (pp. 92-97). Stanford University.

  • Kietz, J.-U., Reimer, U., and Staudt, M. (1997). Mining Insurance Data at Swiss Life. The VLDB Journal, 562-566.

  • Kira, K. and Rendell, L.A. (1992). A Practical Approach to Feature Selection. In Proc. of the Ninth Int. Conference on Machine Learning (pp. 249-256). MK.

  • Kohavi, R. and John, G.H. (1997). Wrappers for Feature Subset Selection. Artificial Intelligence, 97(1/2), 273-323.

    Google Scholar 

  • Lavra?, N. and Džeroski, S. (1994). Inductive Logic Programming: Techniques and Applications. Ellis Horwood.

  • Lee, Y., Buchanan, B.G., and Aronis, J.M. (1998). Knowledge-Based Learning in Exploratory Science: Learning Rules to Predict Rodent Carcinogenicity. Machine Learning, 30, 217-240.

    Google Scholar 

  • Liu, H. and Motoda, H. (1998). Feature Extraction, Construction and Selection: A Data Mining Perspective. Kluwer Academic Publisher.

  • Nédellec, C., Rouveirol, C., Ade, H., Bergadano, F., and Tausend, B. (1996). Declarative Bias in Inductive Logic Programming. In L. de Raedt (Ed.), Advances in Inductive Logic Programming (pp. 82-103). IOS Press.

  • Plotkin, G. (1970). A Note on Inductive Generalization. In Machine Intelligence, Vol. 5. Edinburgh University Press.

  • Pompe, U. and Kononenko, I. (1995). Linear Space Induction in First Order Logic with RELIEFF. In R. Kruse, R. Viertl, and G. Della Ricci (Eds.), Mathematical and Statistical Methods in Artificial Intelligence, CISM Course and Lecture Notes 363 (pp. 185-220). Springer-Verlag.

  • Quinlan, J.R. (1990). Learning Logical Definitions from Relations. Machine Learning, 5(3), 239-266.

    Google Scholar 

  • Rouveirol, C. (2000). Expressiveness/Efficiency: The Dilemma of the Inductive Logic Programming. LRI, Université Paris. HDR, in French.

  • Sebag, M. (1998). Resource Bounded Induction and Deduction in FOL. In Proceedings of 4th International Workshop on Multistrategy Learning (pp. 95-105).

  • Sebag, M. and Rouveirol, C. (1996). Constraint Inductive Logic Programming. In L. De Raedt (Ed.), Advances in Inductive Logic Programming (pp. 277-294). IOS Press.

  • Sebag, M. and Rouveirol, C. (1997). Tractable Induction and Classification in First Order Logic via Stochastic Matching. In 15th Int. Join Conf. on Artificial Intelligence (IJCAI'97) (pp. 888-893). Morgan Kaufmann.

  • Silverstein, G. and Pazzani, M.J. (1991). Relational Cliches: Constraining Constructive Induction During Relational Learning. In L. Birnbaum and G. Collins (Eds.), Proceedings of the 8th International Workshop on Machine Learning (pp. 203-207). Morgan Kaufmann.

  • Srinivasan, A., Muggleton, S., King, R., and Sternberg, M. (1994). Mutagenesis: ILP Experiments in a Non-Determinate Biological Domain. In S. Wrobel (Ed.), Proceedings of the 4th International Workshop on Inductive Logic Programming, Vol. 237 of GMD-Studien (pp. 217-232). Gesellschaft für Mathematik und Datenverarbeitung MBH.

  • Zucker, J.-D. and Ganascia, J.-G. (1996). Changes of Representation for Efficient Learning in Structural Domains. In Proc. of 13th International Conference on Machine Learning. Morgan Kaufmann.

  • Zucker, J.-D. and Ganascia, J.-G. (1998). Learning Structural Indeterminate Clauses. In D. Page (Ed.), Proc. of the 8th International Workshop on Inductive Logic Programming (pp. 235-244). Springer Verlag.

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Alphonse, É., Matwin, S. Filtering Multi-Instance Problems to Reduce Dimensionality in Relational Learning. Journal of Intelligent Information Systems 22, 23–40 (2004). https://doi.org/10.1023/A:1025876613117

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1025876613117

Navigation