Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Machine Learning and Data Mining in Pattern Recognition

MLDM 2013: Machine Learning and Data Mining in Pattern Recognition pp 42–56Cite as

Multi Model Transfer Learning with RULES Family

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 7988))

Abstract

Due to the rapid growth of computer technologies and the extensive changes in human needs, expertise and digital information were used to induce general conclusions. Such conclusions can be used to deal with future activates and make the life of humans easier. One active filed of machine learning that was developed for this purpose is inductive learning, and several families have emerged from this field. Specifically, RULES family was discovered as covering algorithm that directly induces good and general conclusions in the shape of simple rules. However, it was found that RULES suffer from two major deficiencies. It needs to tradeoff between time and accuracy when inducing the best rule and it did not appropriately handle incomplete data. As a result, this paper will present a new RULES algorithm, which takes advantage of previous versions of RULES family in addition to other advance methods of machine learning, specifically Transfer learning. Moreover, multi-modeling is also merged to transfer the knowledge of a different classification model and further improve the original algorithm. At the end, an empirical test is applied to compare the proposed algorithm with different single-model algorithms to prove that using the past knowledge of other agents in different domains improves specialization accuracy, whether the data is complete or incomplete.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)

    MATH  Google Scholar 

  2. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann (1993)

    Google Scholar 

  3. Stahl, F., Bramer, M.: Computationally efficient induction of classification rules with the PMCRI and J-PMCRI frameworks. Knowledge-Based Systems (2012)

    Google Scholar 

  4. Kurgan, L.A., Cios, K.J., Dick, S.: Highly Scalable and Robust Rule Learner: Performance Evaluation and Comparison. IEEE Systems, Man, and Cybernetics—Part B: Cybernetics 36, 32–53 (2006)

    Article  Google Scholar 

  5. Aksoy, M.: A review of rules family of algorithms. Mathematical and Computational Applications 13, 51–60 (2008)

    MathSciNet  MATH  Google Scholar 

  6. Theron, H.: An Empirical Evaluation of Beam Search and Pruning in BEXA. In: Fifth International Conference on Tools with Artificial Intelligence (TAI 1993), Boston, MA, pp. 132–139 (1993)

    Google Scholar 

  7. Kotsiantis, S.B.: Supervised Machine Learning: A Review of Classification Techniques. Presented at the Proceedings of the 2007 Conference on Emerging Artificial Intelligence Applications in Computer Engineering: Real Word AI Systems with Applications in eHealth, HCI, Information Retrieval and Pervasive Technologies (2007)

    Google Scholar 

  8. Maclin, R., Opitz, D.W.: Popular Ensemble Methods: An Empirical Study. Journal of Artificial Intelligence Research 11, 169–198 (1999)

    MATH  Google Scholar 

  9. Pham, D., Afify, A.: Machine-learning techniques and their applications in manufacturing. Proceedings of the I MECH E Part B Journal of Engineering Manufacture 219, 395–412 (2005)

    Google Scholar 

  10. ElGibreen, H., Aksoy, M.: RULES – TL: A simple and Improved RULES Algorithm for Incomplete and Large Data. Journal of Theoretical and Applied Information Technology 47 (2013)

    Google Scholar 

  11. Sinno Jialin, P., Qiang, Y.: A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering 22, 1345–1359 (2010)

    Article  Google Scholar 

  12. Pham, D., Bigot, S., Dimov, S.: RULES-5: a rule induction algorithm for classification problems involving continuous attributes. In: Institution of Mechanical Engineers, pp. 1273–1286 (2003)

    Google Scholar 

  13. Ramon, J., Driessens, K., Croonenborghs, T.: Transfer Learning in Reinforcement Learning Problems Through Partial Policy Recycling. In: Kok, J.N., Koronacki, J., Lopez de Mantaras, R., Matwin, S., Mladenič, D., Skowron, A. (eds.) ECML 2007. LNCS (LNAI), vol. 4701, pp. 699–707. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  14. Taylor, M., Suay, H.B., Chernova, S.: Integrating Reinforcement Learning with Human Demonstrations of Varying Ability. In: International Conferance of Autonomous Agents and Multiagent Systems (AAMAS 2011), Taipei, Taiwan (2011)

    Google Scholar 

  15. Mahmud, M.M.H.: On Universal Transfer Learning. In: Hutter, M., Servedio, R.A., Takimoto, E. (eds.) ALT 2007. LNCS (LNAI), vol. 4754, pp. 135–149. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  16. Yongqiang, L.: A Review About Transfer Learning Methods and Applications. In: International Conference on Information and Network Technology IPCSIT, Singapore, pp. 7–11 (2011)

    Google Scholar 

  17. Pham, D.T., Aksoy, M.S.: RULES: A simple rule extraction system. Expert Systems with Applications 8, 59–65 (1995)

    Article  Google Scholar 

  18. Pham, D.T., Aksoy, M.S.: An algorithm for automatic rule induction. Artificial Intelligence in Engineering 8, 277–282 (1993)

    Article  Google Scholar 

  19. Pham, D.T., Aksoy, M.S.: A new algorithm for inductive learning. Journal of Systems Engenering 5, 115–122 (1995)

    Google Scholar 

  20. Pham, D.T., Dimov, S.S.: The RULES-3 Plus inductive learning algorithm. In: Proceedings of the Third World Congress on Expert Systems, Seoul, Korea, pp. 917–924 (1996)

    Google Scholar 

  21. Aksoy, M.S., Mathkour, H.: CSLS: Connectionist Symbolic Learning System. Mathematical and Computational Applications 14, 177–186 (2009)

    MATH  Google Scholar 

  22. Pham, D.T., Dimov, S.S.: An algorithm for incremental inductive learning. Journal of Engineering Manufacture 211, 239–249 (1997)

    Article  Google Scholar 

  23. Pham, D.T., Soroka, A.J.: An Immune-network inspired rule generation algorithm (RULES-IS). In: Third Virtual International Conference on Innovative Production Machines and Systems. Whittles, Dunbeath (2007)

    Google Scholar 

  24. Pham, D.T., Bigot, S., Dimov, S.S.: RULES-F: A fuzzy inductive learning algorithm. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 220, 1433–1447 (2006)

    Article  Google Scholar 

  25. Bigot, S.: A new rule space representation scheme for rule induction in classification and control applications. Proceedings of the Institution of Mechanical Engineers, Part I: Journal of Systems and Control Engineering (2011)

    Google Scholar 

  26. Pham, D.T., Afify, A.A.: RULES-6: A Simple Rule Induction Algorithm for Supporting Decision Making. Presented at the 31st Annual Conference of IEEE Industrial Electronics Society (IECON 2005) (2005)

    Google Scholar 

  27. Shehzad, K.: EDISC: A Class-tailored Discretization Technique for Rule-based Classification. IEEE Transactions on Knowledge and Data Engineering 24, 1435–1447 (2012)

    Article  Google Scholar 

  28. Afify, A.A., Pham, D.T.: SRI: A Scalable Rule Induction Algorithm. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 220, 537–552 (2006)

    Google Scholar 

  29. Pan, W., Zhong, E., Yang, Q.: Transfer Learning for Text Mining, pp. 223–257 (2012)

    Google Scholar 

  30. Xie, Y.-F., Su, S.-Z., Li, S.-Z.: A Pedestrian Classification Method Based on Transfer Learning. Presented at the International Conference on Image Analysis and Signal Processing, IASP, Zhejiang (2010)

    Google Scholar 

  31. Estévez, J.I., Toledo, P.A., Alayón, S.: Using an Induced Relational Decision Tree for Rule Injection in a Learning Classifier System. Presented at the IEEE Congress on Evolutionary Computation New Orleans, LA (2011)

    Google Scholar 

  32. Boström, H.: Induction of Recursive Transfer Rules. In: Cussens, J., Džeroski, S. (eds.) LLL 1999. LNCS (LNAI), vol. 1925, pp. 237–450. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  33. Reid, M.D.: DEFT Guessing: Using Inductive Transfer to Improve Rule Evaluation from Limited Data. In: Doctor of Philosophy, School of Computer Science and Engineering. University of New South Wales, Sydney (2007)

    Google Scholar 

  34. Ganchev, P., Malehorn, D., Bigbee, W.L., Gopalakrishnan, V.: Transfer learning of classification rules for biomarker discovery and verification from molecular profiling studies. Journal of Biomedical Informatics 44(suppl. 1), S17–S23 (2011)

    Article  Google Scholar 

  35. Bigot, S.: A study of specialisation and classification heuristics used in covering algorithms. Presented at the IPROM2009 Innovative Production Machines and Systems Fifth I*PROMS Virtual Conference, Cardiff, UK (2009)

    Google Scholar 

  36. Alcalá-Fdez, J., Sánchez, L., García, S., del Jesus, M.J., Ventura, S., Garrell, J.M., Otero, J., Romero, C., Bacardit, J., Rivas, V.M., Fernández, J.C., Herrera, F.: KEEL: A Software Tool to Assess Evolutionary Algorithms to Data Mining Problems. Soft Computing 13, 307–318 (2009)

    Article  Google Scholar 

  37. Alcalá-Fdez, J., Fernandez, A., Luengo, J., Derrac, J., García, S., Sánchez, L., Herrera, F.: KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework. Journal of Multiple-Valued Logic and Soft Computing 17, 255–287 (2011)

    Google Scholar 

  38. Efron, B., Tibshirani, R.: An Introduction to the Bootstrap. Chapman & Hall, USA (1993)

    MATH  Google Scholar 

  39. Fayyad, U.M., Irani, K.B.: Multi-interval discretization of continuousvalued attributes for classification learning. Presented at the 13th International Joint Conference of Artificial Intelligence (1993)

    Google Scholar 

  40. Cai, Z.: Technical Aspects of Data Mining. PhD, University of Wales Cardiff, Cardiff, UK (2001)

    Google Scholar 

  41. Pham, D.T., Afify, A.A.: Online Discretization of Continuous-Valued Attributes in Rule Induction. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science 219, 829–842 (2005)

    Article  Google Scholar 

  42. Luengo, J., García, S., Herrera, F.: On the choice of the best imputation methods for missing values considering three groups of classification methods. Knowledge and Information Systems 32, 77–108 (2012)

    Article  Google Scholar 

  43. Li, D., Deogun, J., Spaulding, W., Shuart, B.: Towards Missing Data Imputation: A Study of Fuzzy K-means Clustering Method. In: Tsumoto, S., Słowiński, R., Komorowski, J., Grzymała-Busse, J.W. (eds.) RSCTC 2004. LNCS (LNAI), vol. 3066, pp. 573–579. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  44. Cohen, W.W.: Fast Effective Rule Induction. In: Twelfth International Conference on Machine Learning, pp. 115–123 (1995)

    Google Scholar 

  45. Stahl, F., Bramer, M., Adda, M.: P-Prism: A Computationally Efficient Approach to Scaling up Classification Rule Induction. In: Bramer, M. (ed.) Artificial Intelligence in Theory and Practice II. IFIP, vol. 276, pp. 77–86. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  46. Stahl, F., Bramer, M.: Induction of Modular Classification Rules: Using Jmax-pruning, pp. 79–92 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IT Department, College of Computer and Information Sciences, King Saud University, Saudi Arabia

    Hebah ElGibreen

  2. IS Department, College of Computer and Information Sciences, King Saud University, Saudi Arabia

    Mehmet Sabih Aksoy

Authors
  1. Hebah ElGibreen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mehmet Sabih Aksoy
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institute of Computer Vision and Applied Computer Sciences, IBaI, Leipzig, Germany

    Petra Perner

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

ElGibreen, H., Aksoy, M.S. (2013). Multi Model Transfer Learning with RULES Family. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2013. Lecture Notes in Computer Science(), vol 7988. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-39712-7_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-39712-7_4

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-39711-0

  • Online ISBN: 978-3-642-39712-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation