Skip to main content
SpringerLink
Log in

Transparent Deep Rule-Based Classifiers

  • Chapter
  • First Online:
Empirical Approach to Machine Learning

Part of the book series: Studies in Computational Intelligence ((SCI,volume 800))

  • 1404 Accesses

Abstract

In this chapter, a new type of deep rule-based (DRB) classifier with a multi-layer architecture is presented for image classification, which combines the computer vision techniques with a massively parallel set of zero-order fuzzy rules as its learning engine. With its prototype-based nature, the DRB classifiers are able to identify a transparent and human-understandable fuzzy rule-based (FRB) system structure from the data through an autonomous, non-iterative, non-parametric and highly parallel online learning process, and offer extremely high classification accuracy. The DRB classifier can start “from scratch”, and conduct classification from the very first image of each class in the same way as humans do. The DRB classifier can also learn in a semi-supervised mode initialized with only a small proportion of the labelled data and continue in a fully unsupervised mode after that. The ability of semi-supervised learning further allows the DRB classifier to learn new classes actively without human experts’ involvement. Thanks to the prototype-based nature of the DRB classifier, it is free from prior assumptions about the type of the data distribution, their random or deterministic nature, and there are no requirements to make ad hoc decisions. Its supervised and semi-supervised learning processes are fully transparent and human-interpretable. The semi-supervised DRB classifiers can perform classification on out-of-sample images and also support recursive online training on a sample-by-sample basis or a batch-by-batch basis.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. L. Kuncheva, Combining Pattern Classifiers: Methods and Algorithms (Wiley, Hoboken, New Jersey, 2004)

    Book  Google Scholar 

  2. P. Angelov, X. Zhou, Evolving fuzzy-rule based classifiers from data streams. IEEE Trans. Fuzzy Syst. 16(6), 1462–1474 (2008)

    Article  Google Scholar 

  3. P.P. Angelov, X. Gu, Autonomous learning multi-model classifier of 0-order (ALMMo-0), in IEEE International Conference on Evolving and Autonomous Intelligent Systems (2017), pp. 1–7

    Google Scholar 

  4. P.P. Angelov, X. Gu, Towards anthropomorphic machine learning. IEEE Comput., (2018)

    Google Scholar 

  5. T. Cover, P. Hart, Nearest neighbor pattern classification. IEEE Trans. Inf. Theory 13(1), 21–27 (1967)

    Article  Google Scholar 

  6. E. Pȩkalska, R.P.W. Duin, P. Paclík, Prototype selection for dissimilarity-based classifiers. Pattern Recognit. 39(2), 189–208 (2006)

    Article  Google Scholar 

  7. N. Cristianini, J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods (Cambridge University Press, Cambridge, 2000)

    Book  Google Scholar 

  8. R.D. Baruah, P.P. Angelov, J. Andreu, Simpl_eClass : simplified potential-free evolving fuzzy rule-based classifiers, in IEEE International Conference on Systems, Man, and Cybernetics (SMC) (2011), pp. 2249–2254

    Google Scholar 

  9. D. Kangin, P. Angelov, J.A. Iglesias, Autonomously evolving classifier TEDAClass. Inf. Sci. (Ny) 366, 1–11 (2016)

    Article  MathSciNet  Google Scholar 

  10. T. Kohonen, Self-organizing Maps (Springer, Berlin, 1997)

    Book  Google Scholar 

  11. P. Perner, Prototype-based classification. Appl. Intell. 28(3), 238–246 (2008)

    Article  Google Scholar 

  12. P.P. Angelov, X. Gu, Deep rule-based classifier with human-level performance and characteristics. Inf. Sci. (Ny) 463–464, 196–213 (2018)

    Article  Google Scholar 

  13. P.P. Angelov, X. Gu, MICE: Multi-layer multi-model images classifier ensemble, in IEEE International Conference on Cybernetics (2017), pp. 436–443

    Google Scholar 

  14. P. Angelov, X. Gu, A cascade of deep learning fuzzy rule-based image classifier and SVM, in International Conference on Systems, Man and Cybernetics (2017), pp. 1–8

    Google Scholar 

  15. X. Gu, P. Angelov, C. Zhang, P. Atkinson, A massively parallel deep rule-based ensemble classifier for remote sensing scenes. IEEE Geosci. Remote Sens. Lett. 15(3), 345–349 (2018)

    Article  Google Scholar 

  16. J. Zhang, X. Kong, P.S. Yu, Predicting social links for new users across aligned heterogeneous social networks, in IEEE International Conference on Data Mining (2013), pp. 1289–1294

    Google Scholar 

  17. D. Ciresan, U. Meier, J. Schmidhuber, Multi-column deep neural networks for image classification, in Conference on Computer Vision and Pattern Recognition (2012), pp. 3642–3649

    Google Scholar 

  18. P. Angelov, Machine learning (collaborative systems), 8250004 (2006)

    Google Scholar 

  19. D.C. Cireşan, U. Meier, L.M. Gambardella, J. Schmidhuber, Convolutional neural network committees for handwritten character classification, in International Conference on Document Analysis and Recognition, vol. 10 (2011), pp. 1135–1139

    Google Scholar 

  20. K. Fukushima, Neocognitron for handwritten digit recognition. Neurocomputing 51, 161–180 (2003)

    Article  Google Scholar 

  21. A. Oliva, A. Torralba, Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42(3), 145–175 (2001)

    Article  Google Scholar 

  22. G.-S. Xia, J. Hu, F. Hu, B. Shi, X. Bai, Y. Zhong, L. Zhang, AID: a benchmark dataset for performance evaluation of aerial scene classification. IEEE Trans. Geosci. Remote Sens. 55(7), 3965–3981 (2017)

    Article  Google Scholar 

  23. N. Dalal, B. Triggs, Histograms of oriented gradients for human detection, in IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2005), pp. 886–893

    Google Scholar 

  24. D.G. Lowe, Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  MathSciNet  Google Scholar 

  25. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, C. Hill, A. Arbor, Going deeper with convolutions, in IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 1–9

    Google Scholar 

  26. A. Krizhevsky, I. Sutskever, G.E. Hinton, ImageNet classification with deep convolutional neural networks, in Advances in Neural Information Processing Systems (2012), pp. 1097–1105

    Google Scholar 

  27. P. Angelov, R. Yager, A new type of simplified fuzzy rule-based system. Int. J. Gen Syst 41(2), 163–185 (2011)

    Article  MathSciNet  Google Scholar 

  28. P.P. Angelov, X. Gu, J. Principe, A generalized methodology for data analysis. IEEE Trans. Cybern. 48(10), 2987–2993 (2018).

    Article  Google Scholar 

  29. http://www.vision.caltech.edu/Image_Datasets/Caltech101/

  30. A. Okabe, B. Boots, K. Sugihara, S.N. Chiu, Spatial Tessellations: Concepts and Applications of Voronoi Diagrams, 2nd edn. (Wiley, Chichester, England, 1999)

    MATH  Google Scholar 

  31. X. Gu, P.P. Angelov, Self-organising fuzzy logic classifier. Inf. Sci. (Ny) 447, 36–51 (2018)

    Article  Google Scholar 

  32. P. Angelov, X. Gu, D. Kangin, Empirical data analytics. Int. J. Intell. Syst. 32(12), 1261–1284 (2017)

    Article  Google Scholar 

  33. P.P. Angelov, X. Gu, J. Principe, D. Kangin, Empirical data analysis—a new tool for data analytics, in IEEE International Conference on Systems, Man, and Cybernetics (2016), pp. 53–59

    Google Scholar 

  34. P. Angelov, Autonomous Learning Systems: From Data Streams to Knowledge in Real Time. Wiley, New York (2012)

    Book  Google Scholar 

  35. X. Zhu, Z. Ghahraman, J.D. Lafferty, Semi-supervised learning using gaussian fields and harmonic functions, in International Conference on Machine Learning (2003), pp. 912–919

    Google Scholar 

  36. D. Zhou, O. Bousquet, T.N. Lal, J. Weston, B. Schölkopf, Learning with local and global consistency. Adv. Neural. Inform. Process Syst., pp. 321–328 (2004)

    Google Scholar 

  37. V. Sindhwani, P. Niyogi, M. Belkin, Beyond the point cloud: from transductive to semi-supervised learning, in International Conference on Machine Learning, vol. 1 (2005), pp. 824–831

    Google Scholar 

  38. F. Noorbehbahani, A. Fanian, R. Mousavi, H. Hasannejad, An incremental intrusion detection system using a new semi-supervised stream classification method. Int. J. Commun Syst 30(4), 1–26 (2017)

    Article  Google Scholar 

  39. O. Chapelle, A. Zien, Semi-supervised classification by low density separation, in AISTATS (2005), pp. 57–64

    Google Scholar 

  40. M. Guillaumin, J.J. Verbeek, C. Schmid, Multimodal semi-supervised learning for image classification, in IEEE Conference on Computer Vision & Pattern Recognition (2010), pp. 902–909

    Google Scholar 

  41. J. Wang, T. Jebara, S.F. Chang, Semi-supervised learning using greedy Max-Cut. J. Mach. Learn. Res. 14, 771–800 (2013)

    MathSciNet  MATH  Google Scholar 

  42. F. Wang, C. Zhang, H. C. Shen, J. Wang, Semi-supervised classification using linear neighborhood propagation, in IEEE Conference on Computer Vision & Pattern Recognition (2006), pp. 160–167

    Google Scholar 

  43. S. Xiang, F. Nie, C. Zhang, Semi-supervised classification via local spline regression. IEEE Trans. Pattern Anal. Mach. Intell. 32(11), 2039–2053 (2010)

    Article  Google Scholar 

  44. B. Jiang, H. Chen, B. Yuan, X. Yao, Scalable graph-based semi-supervised learning through sparse bayesian model. IEEE Trans. Knowl. Data Eng. (2017). https://doi.org/10.1109/TKDE.2017.2749574

    Article  Google Scholar 

  45. J. Thorsten, Transductive inference for text classification using support vector machines. Int. Conf. Mach. Learn. 9, 200–209 (1999)

    Google Scholar 

  46. O. Chapelle, V. Sindhwani, S. Keerthi, Optimization techniques for semi-supervised support vector machines. J. Mach. Learn. Res. 9, 203–233 (2008)

    MATH  Google Scholar 

  47. K. Wu, K.-H. Yap, Fuzzy SVM for content-based image retrieval: a pseudo-label support vector machine framework. IEEE Comput. Intell. Mag. 1(2), 10–16 (2006)

    Article  Google Scholar 

  48. D.-H. Lee, Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks, in ICML 2013 Workshop: Challenges in Representation Learning (2013), pp. 1–6

    Google Scholar 

  49. X. Gu, P.P. Angelov, Semi-supervised deep rule-based approach for image classification. Appl. Soft Comput. 68, 53–68 (2018)

    Article  Google Scholar 

  50. K. Simonyan, A. Zisserman, Very deep convolutional networks for large-scale image recognition, in International Conference on Learning Representations (2015), pp. 1–14

    Google Scholar 

  51. Y. Yang, S. Newsam, Bag-of-visual-words and spatial extensions for land-use classification, in International Conference on Advances in Geographic Information Systems (2010), pp. 270–279

    Google Scholar 

  52. http://weegee.vision.ucmerced.edu/datasets/landuse.html

  53. P.T. Daniels, W. Bright (eds.) The World’s Writing Systems. Oxford University Press on Demand (1996)

    Google Scholar 

  54. D. Kangin, P. Angelov, J.A. Iglesias, A. Sanchis, Evolving classifier TEDAClass for big data. Procedia Comput. Sci. 53(1), 9–18 (2015)

    Article  Google Scholar 

  55. P. Angelov, Machine learning (collaborative systems), US 8250004, 2012

    Google Scholar 

  56. X. Gu, P.P. Angelov, G. Gutierrez, J. A. Iglesias, A. Sanchis, Parallel computing TEDA for high frequency streaming data clustering, in INNS Conference on Big Data (2016), pp. 238–253

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing and Communications, Lancaster University, Lancaster, UK

    Plamen P. Angelov & Xiaowei Gu

Authors
  1. Plamen P. Angelov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaowei Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Plamen P. Angelov .

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Angelov, P.P., Gu, X. (2019). Transparent Deep Rule-Based Classifiers. In: Empirical Approach to Machine Learning. Studies in Computational Intelligence, vol 800. Springer, Cham. https://doi.org/10.1007/978-3-030-02384-3_9

Download citation

Publish with us

Policies and ethics

Search

Navigation