Abstract
The chapter presents a convenient algorithm of classification-rule learning that uses the Multiple Objective Particle Swarm Optimization technique to obtain a set of the best classification rules as an alternative way to cover algorithms, avoiding the loss of quality of the rules. The best rules are memorized in an unordered classifier, based on the concept of Pareto dominance, which permits the selection of rules with several characteristics that can be defined by the user. The rules are selected from the classifier at the same time that they are discovered and, differently from the cover algorithms, the selection does not remove examples from the dataset. We study classification-rule learning, describing some paradigms, different rule’s representations and different particle initialization procedures. Then, we describe the way the particles move in the search space and the fitness functions. After that, we show some aspects of the required global repositories and describe aspects of different stopping criteria. Then, we present the proposed algorithm MOPSO-RL, showing its complexity and restrictions. Comparisons with other algorithms from the literature show that the proposed algorithm is competitive related to the area under ROC (Receiver Output Curve) and regarding the number of rules in the classifier. Besides, the selected rules have high support and weighted relative accuracy (Wracc), which denotes that the rules are important even when considered in isolation. In this way, it produces a set that is good for classification, with very good rules that can bring knowledge even if they are analyzed in isolation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Russel, S., Norvig, P.: Artificial Intelligence: A modern approach, 946 pp. Prentice-Hall, New Jersey (2003)
Fawcett, T.: Using Rule Sets to Maximize ROC Performance. In: IEEE International Conference on Data Mining (ICDM 2001), pp. 131–138 (2001)
Provost, F., Fawcett, T.: The case against accuracy estimation for comparing induction algorithms. In: Proceedings of the 15th International Conference on Machine Learning, pp. 445–453. Morgan Kaufmann, San Francisco (1998)
Westin, L.K.: Receiver operating characteristics (ROC) analysis: Evaluating discriminance effects among decision support systems. Umea University, Umea (2001)
Coello, C.A., Lechuga, M.S.: MOPSO: A proposal for Multiple Objective Particle Swarm Optimization. In: IEEE World Congress on Computational Intelligence, 2002. Proceedings of the 2002 Congress on Evolutionary Computation, pp. 1051–1056. IEEE Press, Hawaii (2002)
Sousa, T.F., Silva, A.P., Silva, A.F.: Particle Swarm Based Data Mining Algorithms for Classification Tasks. Parallel Computing 30, 767–783 (2004)
Jin, Y. (ed.): Multi-Objective Machine Learning. Springer, Berlin (2006)
Ishibuchi, H., Nojima, Y.: Accuracy-Complexity Tradeoff Analysis by Multiobjective Rule Selection. In: Proc. of ICDM 2005 Workshop on Computational Intelligence in Data Mining, pp. 39–48 (2005)
de la Iglesia, B., Reynolds, A., Rayward-Smith, V.J.: Developments on a Multi-Objective Metaheuristic (MOMH) Algorithm for Finding Interesting Sets of Classification Rules. In: Coello Coello, C.A., Hernández Aguirre, A., Zitzler, E. (eds.) EMO 2005. LNCS, vol. 3410, pp. 826–840. Springer, Heidelberg (2005)
Ishibuchi, H., Kuwajima, I., Nojima, Y.: Multiobjective association rule mining. In: Proc. of PPSN Workshop on Multiobjective Problem Solving from Nature, Reykjavik, Iceland, September 9, 2006, 12 pages (2006)
de la Iglesia, B., Philpott, M.S., Bagnall, A.J., Rayward-Smith, V.J.: Data Mining Rules using Multi-Objective Evolutionary Algorithms. In: Proc. of 2003 Congress on Evolutionary Computation, pp. 1552–1559 (2003)
Clark, P., Boswell, R.: Rule Induction with CN2: Some Recent Improvements. In: Machine Learning - Proceedings of the European Conference, pp. 151–163. Springer, Berlin (1991)
Pham, T.H., Clemente, J.C., Satou, K., Ho, T.B.: Computational Discovery of transcriptional regulatory rules. Bioinformatics 21(1), 101–107 (2005)
Prati, C.R., Flash, P.: Roccer: A ROC convex hull rule learning algorithm. In: ECML/PKDD 2004 Workshop on Advances in Inductive Rule Learning, Pisa (Italy), pp. 144–153 (2004)
Niblett, T., Clark, P.: The CN2 induction algorithm. Machine Learning 3(4), 261–283 (1989)
Prati, C.R., Flash, P.A.: ROCCER: An Algorithm for Rule Learning Based on ROC Analysis. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence (IJCAI 2005), August 2005, pp. 823–828 (2005)
Quinlan, J.R.: C4.5 programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
Monard, M.C., Baranauskas, J.A.: Inducão de Regras e Árvores de Decisão. In: Rezende, S. (ed.) Sistemas Inteligentes: Fundamentos e Aplicacões, 525 p. Editora Manole, Barueri (2003) (in Portuguese)
Liu, B., Hsu, W., Ma, Y.: Integrating classification and association rule mining. In: Proceedings of the 4th Int. Conf. on Knowledge Discovery and Data Mining, New York, pp. 80–86 (1998)
Batista, G.E.A., Prati, R.C., Monard, M.C., Giusti, R., Milaré, C.R.: Classificacão Associativa Utilizando Selecão e Construcão de Regras: um Estudo Comparativo. In: Encontro Nacional de Inteligência Artificial (ENIA). Rio de Janeiro. Anais do Congresso da Sociedade Brasileira de Computacão, pp. 1321–1330 (2007) (in Portuguese)
Prati, R.C., Batista, G.E.A.P.A., Monard, M.C.: A Study with Class Imbalance and Random Sampling for a Decision Tree Learning System. In: Bramer, M. (ed.) Artificial Intelligence in Theory and Practice II, IFIP 20th World Computer Congress, TC 12: IFIP AI 2008, Stream, Milano, Italy, September 7-10, 2008, pp. 131–140 (2008)
Kennedy, J., Eberhart, R.C.: Particle Swarm Optimization. In: Proceedings of the 1995 IEEE International Conference on Neural Networks, pp. 1492–1948. IEEE Press, Los Alamitos (1995)
Pareto, V.: Manuel D’Économie Politique. Marcel Giard, Paris (1927)
Freitas, A.A.: Data Mining and Knowledge Discovery with Evolutionary Algorithms, 264 p. Springer, Berlin (2002)
Fieldsend, J.E.: Multi-Objective particle Swarm optimization methods (March 2004)
Asuncion, A., Newman, D.J.: UCI Machine Learning Repository. Irvine, University of California (2007), http://www.ics.uci.edu/mlearn/MLRepository.html
Zitzler, E.: Evolutionary algorithms for multiobjective optimization: methods and applications. PhD Thesis Swiss Federal Institute of Technology. Zürich, Switzerland
The R Project for Statistical Computing (Accessed: February 2008), http://www.r-project.org
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
de Almeida Prado G. Torácio, A. (2009). Multiobjective Particle Swarm Optimization in Classification-Rule Learning. In: Coello, C.A.C., Dehuri, S., Ghosh, S. (eds) Swarm Intelligence for Multi-objective Problems in Data Mining. Studies in Computational Intelligence, vol 242. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03625-5_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-03625-5_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-03624-8
Online ISBN: 978-3-642-03625-5
eBook Packages: EngineeringEngineering (R0)