Abstract
The Fuzzy Integral (FI) is a non-linear aggregation operator which enables the fusion of information from multiple sources in respect to a Fuzzy Measure (FM) which captures the worth of both the individual sources and all their possible combinations. Based on the expected potential of non-linear aggregation offered by the FI, its application to decision-level fusion in ensemble classifiers, i.e. to fuse multiple classifiers outputs towards one superior decision level output, has recently been explored. A key example of such a FI-FM ensemble classification method is the Decision-level Fuzzy Integral Multiple Kernel Learning (DeFIMKL) algorithm, which aggregates the outputs of kernel based classifiers through the use of the Choquet FI with respect to a FM learned through a regularised quadratic programming approach. While the approach has been validated against a number of classifiers based on multiple kernel learning, it has thus far not been compared to the state-of-the-art in ensemble classification. Thus, this paper puts forward a detailed comparison of FI-FM based ensemble methods, specifically the DeFIMKL algorithm, with state-of-the art ensemble methods including Adaboost, Bagging, Random Forest and Majority Voting over 20 public datasets from the UCI machine learning repository. The results on the selected datasets suggest that the FI based ensemble classifier performs both well and efficiently, indicating that it is a viable alternative when selecting ensemble classifiers and indicating that the non-linear fusion of decision level outputs offered by the FI provides expected potential and warrants further study.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Krawczyk, B., Minku, L.L., Gama, J., Stefanowski, J., Woźniak, M.: Ensemble learning for data stream analysis: a survey. Inf. Fusion 37, 132–156 (2017)
Domingos, P.: A few useful things to know about machine learning. Commun. ACM 55(10), 78 (2012)
Islam, M.A., Anderson, D.T., Pinar, A.J., Havens, T.C.: Data-driven compression and efficient learning of the Choquet integral. IEEE Trans. Fuzzy Syst. (2017). https://doi.org/10.1109/TFUZZ.2017.2755002
Pinar, A.J., Rice, J., Hu, L., Anderson, D.T., Havens, T.C.: Efficient multiple kernel classification using feature and decision level fusion. IEEE Trans. Fuzzy Syst. 25(6), 1403–1416 (2016)
Anderson, D.T., Price, S.R., Havens, T.C.: Regularization-based learning of the Choquet integral. In: IEEE International Conference on Fuzzy Systems, pp. 2519–2526. IEEE, July 2014
Pinar, A.J., Havens, T.C., Islam, M.A., Anderson, D.T.: Visualization and learning of the Choquet integral with limited training data. In: 2017 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–6, July 2017
Pinar, A.J., Anderson, D.T., Havens, T.C., Zare, A., Adeyeba, T.: Measures of the Shapley index for learning lower complexity fuzzy integrals. Granular Comput. 2(4), 303–319 (2017)
Pinar, A., Havens, T.C., Anderson, D.T., Hu, L.: Feature and decision level fusion using multiple kernel learning and fuzzy integrals. In: 2015 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–7, August 2015
Hu, L., Anderson, D.T., Havens, T.C.: Multiple kernel aggregation using fuzzy integrals. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE) (2013)
Li, K., Chen, C., Liu, W., Fang, X., Lu, Q.: Software defect prediction using fuzzy integral fusion based on GA-FM. Wuhan Univ. J. Nat. Sci. 19(5), 405–408 (2014)
Zhang, L., Zhou, D.Q., Zhou, P., Chen, Q.T.: Modelling policy decision of sustainable energy strategies for Nanjing city: a fuzzy integral approach. Renew. Energy 62, 197–203 (2014)
Cavrini, F., Bianchi, L., Quitadamo, L.R., Saggio, G.: A fuzzy integral ensemble method in visual P300 brain-computer interface. Comput. Intell. Neurosci. 2016, 1–9 (2016)
Karczmarek, P., Pedrycz, W., Reformat, M., Akhoundi, E.: A study in facial regions saliency: a fuzzy measure approach. Soft. Comput. 18, 379–391 (2014)
Wang, Z., Xiao, N.: Fuzzy integral-based neural network ensemble for facial expression recognition. In: Proceedings of the International Conference on Computer Information Systems and Industrial Applications (2015)
Anderson, D.T., Havens, T.C., Wagner, C., Keller, J.M., Anderson, M.F., Wescott, D.J.: Extension of the fuzzy integral for general fuzzy set-valued information. IEEE Trans. Fuzzy Syst. 22(6), 1625–1639 (2014)
Pinar, A.J., Rice, J., Havens, T.C., Masarik, M., Burns, J., Anderson, D.T.: Explosive hazard detection with feature and decision level fusion, multiple kernel learning, and fuzzy integrals. In: 2016 IEEE Symposium Series on Computational Intelligence, SSCI 2016, pp. 1–8, December 2017
Wagner, C., Havens, T.C., Anderson, D.T.: The arithmetic recursive average as an instance of the recursive weighted power mean. In: IEEE International Conference on Fuzzy Systems (2017)
Fakhar, K., El Aroussi, M., Saidi, M.N., Aboutajdine, D.: Applying the upper integral to the biometric score fusion problem in the identification model. In: International Conference on Electrical and Information Technologies (ICEIT) (2015)
Wang, Q., Zheng, C., Yu, H., Deng, D.: Integration of heterogeneous classifiers based on choquet fuzzy integral. In: 7th International Conference on Intelligent Human-Machine Systems and Cybernetics, IHMSC 2015, vol. 1, pp. 543–547 (2015)
Murofushi, T., Sugeno, M.: A learning model using fuzzy measures and the Choquet integral. In: Proceedings of the 5th Fuzzy System Symposium, vol. 29, pp. 213–218 (1989)
Grabisch, M.: The application of fuzzy integrals in multicriteria decision making. Eur. J. Oper. Res. 89(3), 445–456 (1996)
Ko, Y.C., Fujita, H., Tzeng, G.H.: An extended fuzzy measure on competitiveness correlation based on WCY 2011. Knowl.-Based Syst. 37, 86–93 (2013)
Anderson, D.T., Keller, J.M., Havens, T.C.: Learning fuzzy-valued fuzzy measures for the fuzzy-valued Sugeno fuzzy integral. In: Hüllermeier, E., Kruse, R., Hoffmann, F. (eds.) IPMU 2010. LNCS (LNAI), vol. 6178, pp. 502–511. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14049-5_52
Freund, Y., Schapire, R.E.: A desicion-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119–139 (1997)
Galar, M., Fernandez, A., Barrenechea, E., Bustince, H., Herrera, F.: A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans. Syst. Man Cybern. Part C: Appl. Rev. 42(4), 463–484 (2012)
Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)
Sun, Q., Pfahringer, B.: Bagging ensemble selection. In: Wang, D., Reynolds, M. (eds.) AI 2011. LNCS (LNAI), vol. 7106, pp. 251–260. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-25832-9_26
Rodriguez-Galiano, V.F., Ghimire, B., Rogan, J., Chica-Olmo, M., Rigol-Sanchez, J.P.: An assessment of the effectiveness of a random forest classifier for land-cover classification. ISPRS J. Photogramm. Remote Sens. 67(1), 93–104 (2012)
Lichman, M.: UCI Machine Learning Repository. University of California, Irvine, School of Information and Computer Sciences (2013)
Fernández-Delgado, M., Cernadas, E., Barro, S., Amorim, D., Amorim Fernández-Delgado, D.: Do we need hundreds of classifiers to solve real world classification problems? J. Mach. Learn. Res. 15, 3133–3181 (2014)
Patro, S.G.K., Sahu, K.K.: Normalization: a preprocessing stage. arXiv preprint arXiv:1503.06462 (2015)
Acknowledgements
The research was supported by The University of Nottingham Vice-Chancellor’s Scholarship for Research Excellence (International).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Agrawal, U., Pinar, A.J., Wagner, C., Havens, T.C., Soria, D., Garibaldi, J.M. (2018). Comparison of Fuzzy Integral-Fuzzy Measure Based Ensemble Algorithms with the State-of-the-Art Ensemble Algorithms. In: Medina, J., et al. Information Processing and Management of Uncertainty in Knowledge-Based Systems. Theory and Foundations. IPMU 2018. Communications in Computer and Information Science, vol 853. Springer, Cham. https://doi.org/10.1007/978-3-319-91473-2_29
Download citation
DOI: https://doi.org/10.1007/978-3-319-91473-2_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-91472-5
Online ISBN: 978-3-319-91473-2
eBook Packages: Computer ScienceComputer Science (R0)