Abstract
This paper introduces Deft, a new multitask learning approach for rule learning algorithms. Like other multitask learning systems, the one proposed here is able to improve learning performance on a primary task through the use of a bias learnt from similar secondary tasks. What distinguishes Deft from other approaches is its use of rule descriptions as a basis for task similarity. By translating a rule into a feature vector or “description”, the performance of similarly described rules on the secondary tasks can be used to modify the evaluation of the rule for the primary task. This explicitly addresses difficulties with accurately evaluating, and therefore finding, good rules from small datasets. Deft is implemented on top of an existing ILP system and the approach is tested on a variety of relational learning tasks. Given appropriate secondary tasks, the results show that Deft is able to compensate for insufficient training examples.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Srinivasan, A., King, R.D.: Feature construction with inductive logic programming: a study of quantative predictions of biological activity aided by structural attributes. Data Mining and Knowledge Discovery 3, 37–57 (1999)
Caruana, R.: Multitask learning. Machine Learning 28, 41–75 (1997)
Mitchell, T.M.: The need for biases in learning generalizations. Technical Report CBM-TR-117, Rutgers University, New Brunswick, New Jersey (1980)
Nédellec, C., Rouveirol, C., Adé, H., Bergadano, F., Tausend, B.: Declarative bias in ILP. In: Advances in ILP. Frontiers in AI and Applications, vol. 32, pp. 82–103. IOS Press, Amsterdam (1996)
Fürnkranz, J.: Separate-and-conquer rule learning. Artificial Intelligence Review 13, 3–54 (1999)
Datta, P., Kibler, D.F.: Concept sharing: A means to improve multi-concept learning. In: Proc. of the 10th ICML, pp. 89–96 (1993)
Khan, K., Muggleton, S., Parson, R.: Repeat learning using predicate invention. In: Proc. of the 8th ILP, pp. 165–174. Springer, Heidelberg (1998)
Morin, J.: Learning Relational Clichés with Contextual Generalization. PhD thesis, School of Information Technology and Engineering, University of Ottawa, Canada (1999)
Srinivasan, A.: ALEPH: A learning engine for proposing hypotheses. Prolog code (2001), http://www.comlab.ox.ac.uk/oucl/research/areas/machlearn/Aleph/
Silver, D.: Selective Transfer of Neural network Task Knowledge. PhD thesis, Graduate Program in Computer Science, University of Western Ontario, London, Ontario, Canada (2000)
Good, I.J.: The Estimation of Probabilities: An Essay on Modern Bayesian Methods. MIT Press, Cambridge (1965)
Fürnkranz, J., Flach, P.A.: An analysis of rule evaluation metrics. In: Proc. of the 19th ICML, pp. 202–209. AAAI Press, Menlo Park (2003)
Muggleton, S.H.: Inverse entailment and progol. New Generation Computing 13, 245–286 (1995)
Srinivasan, A.: A study of two sampling methods for analysing large datasets with ILP. Data Mining and Knowledge Discovery 3, 95–123 (1999)
De Raedt, L., Bruynhooghe, M.: Interactive concept-learning and constructive induction by analogy. Machine Learning 8, 107–150 (1992)
Srinivasan, A., Muggleton, S., King, R.D., Sternberg, M.J.E.: Mutagenesis: ILP experiments in a non-determinate biological domain. In: Proc. of the 4th ILP (1994)
Srinivasan, A., King, R.D., Muggleton, S., Sternberg, M.J.E.: Carcinogenesis predictions using ILP. In: Proc. of the 7th ILP, pp. 273–287 (1997)
Holte, R.C., Acker, L.E., Porter, B.W.: Concept learning and the problem of small disjuncts. In: Proc. of the 11th IJCAI, pp. 813–818 (1989)
Srinivasan, A.: Extracting context-sensitive models in inductive logic programming. Machine Learning 44, 301–324 (2001)
Srinivasan, A.: Personal communication. Email regarding chemical data (2002)
Lavrač, N., Flach, P., Zupan, B.: Rule evaluation measures: A unifying view. In: Proc. of the 9th ILP, pp. 174–185. Springer, Heidelberg (1999)
Vilalta, R., Oblinger, D.: A quantification of distance-bias between evaluation metrics in classification. In: Proc. of the 17th ICML, pp. 1087–1094 (2000)
Cestnik, B.: Estimating probabilities: A crucial task in machine learning. In: Proc. of the 9th European Conference on AI, Pitman, pp. 147–149 (1990)
Baxter, J.: A model of inductive bias learning. Journal of Artificial Intelligence Research 12, 149–198 (2000)
Heskes, T.: Empirical bayes for learning to learn. In: Proc. of the 17th ICML, pp. 367–374. Morgan Kaufmann, San Francisco (2000)
Cussens, J.: Using prior probabilities and density estimation for relational classification. In: Proc. of the 8th ILP, pp. 106–115. Springer, Heidelberg (1998)
McCreath, E., Sharma, A.: LIME: A system for learning relations. In: Richter, M.M., Smith, C.H., Wiehagen, R., Zeugmann, T. (eds.) ALT 1998. LNCS (LNAI), vol. 1501, pp. 336–374. Springer, Heidelberg (1998)
Lavrač, N., Flach, P.A.: An extended transformation approach to inductive logic programming. ACM Trans. on Computational Logic (TOCL) 2, 458–494 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Reid, M.D. (2004). Improving Rule Evaluation Using Multitask Learning. In: Camacho, R., King, R., Srinivasan, A. (eds) Inductive Logic Programming. ILP 2004. Lecture Notes in Computer Science(), vol 3194. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30109-7_20
Download citation
DOI: https://doi.org/10.1007/978-3-540-30109-7_20
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-22941-4
Online ISBN: 978-3-540-30109-7
eBook Packages: Springer Book Archive