New Learning Paradigms in Soft Computing pp 55-96 | Cite as

# Learning and Decision-Making in the Framework of Fuzzy Lattices

## Abstract

A novel theoretical framework is delineated for supervised and unsupervised learning. It is called framework of fuzzy lattices, or FL-framework for short, and it suggests mathematically sound tools for dealing separately and/or jointly with disparate types of data including vectors of numbers, fuzzy sets, symbols, etc. Specific schemes are proposed for clustering and classification having the capacity to deal with both missing and don’t care data values; the schemes in question can be implemented as neural networks. The proposed learning schemes are employed here for pattern recognition on seven data sets including benchmark data sets, and the results are compared with those ones by various learning techniques from the literature. Finally, aiming at a mutual cross-fertilization, the FL-framework is associated with established theories for learning and/or decision-making including probability theory, fuzzy set theory, Bayesian decision-making, theory of evidence, and adaptive resonance theory.

## Keywords

Fuzzy Number Complete Lattice Adaptive Resonance Theory Inclusion Measure Lattice Interval## Preview

Unable to display preview. Download preview PDF.

## References

- [1]Aha, D.W. (1998), “The omnipresence of case-based reasoning in science and application,”
*Knowledge-Based Systems*, vol. 11, no. 5–6, pp. 261–273.CrossRefGoogle Scholar - [2]Amari, S. and Wu, S. (1999), “Improving Support Vector Machine Classifiers by Modifying Kernel Functions,”
*Neural Networks*, vol. 12, no. 6, pp. 783–789.CrossRefGoogle Scholar - [3]Baker, D., Brett, P.N., Griffiths, M.V., and Reyes, L. (1996), “A Mechatronic Tool for Ear Surgery: A Case Study of Some Design Characteristics,”
*Mechatronics*, vol. 6, no. 4, pp. 461–477.CrossRefGoogle Scholar - [4]Baralis, E., Ceri, S., and Paraboschi, S. (1998), “Compile-Time and Runtime Analysis of Active Behaviors,”
*IEEE Trans. Knowledge Data Engineering*, vol. 10, no. 3, pp. 353–370.CrossRefGoogle Scholar - [5]Bianchini, M., Frasconi, P., and Gori, M. (1995), “Learning Without Local Minima in Radial Basis Function Networks,”
*IEEE Trans. Neural Networks*, vol. 6, no. 3, pp. 749–756.CrossRefGoogle Scholar - [6]Birkhoff, G. (1967),
*Lattice Theory*, American Mathematical Society, Colloquium Publications, vol. 25, Providence, RI.MATHGoogle Scholar - [7]Bishop, C. (1991), “Improving the generalization properties of radial basis function neural networks,”
*Neural Computation*, vol. 3, no. 4, pp. 579–588.CrossRefGoogle Scholar - [8]Blumer, A., Ehrenfeucht, A., Haussier, D. and Warmuth, M.K. (1989), “Learnability and the Vapnik-Chervonenkis dimension,”
*Journal of the ACM*, vol. 36, no. 4, pp. 929–965.MATHCrossRefGoogle Scholar - [9]Bogler, P.L. (1987) “Shafer-Dempster reasoning with applications to multisensor target identification systems,”
*IEEE Trans. Systems*,*Man*,*and Cybernetics*, vol. 17, no. 6, pp. 968–977.Google Scholar - [10]Bryson, N. and Mobolurin, A. (1998), “A qualitative discriminant approach for generating quantitative belief functions,”
*IEEE Trans. Knowledge Data Engineering*, vol. 10, no. 2, pp. 345–348.CrossRefGoogle Scholar - [11]Buede, D.M. and Girardi, P. (1997), “A target identification comparison of Bayesian and Dempster-Shafer multisensor fusion,”
*IEEE Trans. Sys*.,*Man*,*and Cybernetics–A*, vol. 27, no. 5, pp. 569–577.CrossRefGoogle Scholar - [12]Card, H.C., Rosendahl, G.K., McNeill, D.K., and McLeod, R.D. (1998), “Competitive learning algorithms and neurocomputer architecture,”
*IEEE Trans. Computers*, vol. 47, no. 8, pp. 847–858.CrossRefGoogle Scholar - [13]Carpenter, G.A. and Grossberg, S. (1987), “A massively parallel architecture for self-organizing neural pattern recognition machine,”
*Computer Vision*,*Graphics and Image Understanding*, vol. 37, pp. 54–115.MATHCrossRefGoogle Scholar - [14]Carpenter, G.A., Grossberg, S., and Rosen, D.B. (1991), “Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system,”
*Neural Networks*, vol. 4, pp. 759–771.CrossRefGoogle Scholar - [15]Cercone, N., An, A., and Chan, C. (1999), “Rule-induction and case-based reasoning: hybrid architectures appear advantageous,”
*IEEE Trans. Knowledge and Data Engineering*, vol. 11, no. 1, pp. 166–174.CrossRefGoogle Scholar - [16]Chen, J-Q. and Xi, Y-G. (1998), “Nonlinear system modeling by competitive learning and adaptive fuzzy inference system,”
*IEEE Trans. Systems*,*Man*,*and Cybernetics–C*, vol. 28, no. 2, pp. 231–238.CrossRefGoogle Scholar - [17]Chen, S., Cowan, C.F.N., and Grant, P.M. (1991), “Orthogonal least squares learning for radial basis function networks,”
*IEEE Trans. Neural Networks*, vol. 2, no. 2, pp. 302–309.CrossRefGoogle Scholar - [18]Davey, B.A. and Priestley, H.A. (1990),
*Introduction to Lattices and Order*, Cambridge University Press, Cambridge, Great Britain.Google Scholar - [19]Dempster, A.P. (1967), “Upper and lower probabilities induced by multivalued mappings,”
*Annals of Math. Stat*., vol. 38, pp. 325–329.MathSciNetMATHCrossRefGoogle Scholar - [20]Dunyak, J., Saad, I.W., and Wunsch, D. (1999), “A theory of independent fuzzy probability for system reliability,”
*IEEE Trans. Fuzzy Systems*, vol. 7, no. 2, pp. 286–294.CrossRefGoogle Scholar - [21]Evans, B. and Fisher, D. (1994), “Overcoming process delays with decision tree induction,”
*IEEE Expert*, pp. 60–66, February.Google Scholar - [22]Fahlman, S.E. and White, M.,
*Carnegie Mellon University (CMU) repository of neural net benchmarks*. Available at http://www.boltz.cs.cmu.edu/. - [23]Feller, W. (1968),
*An Introduction to Probability Theory and Its Applications*, third edition, John Wiley and Sons, New York, NY.MATHGoogle Scholar - [24]Fixsen, D. and Mahler, R.P.S. (1997), “The modified Dempster-Shafer approach to classification,”
*IEEE Trans. Systems*,*Man*,*and Cybernetics–A*, vol. 27, no. 1, pp. 96–104.CrossRefGoogle Scholar - [25]Fritzke, B. (1994), “Growing cell structures–A self-organizing network for unsupervised learning,”
*Neural Networks*,*vol*. 7, no. 9, pp. 1441–1460.CrossRefGoogle Scholar - [26]Gaines, B.R. (1978), “Fuzzy and probability uncertainty logics,”
*Information and Control*, vol. 38, pp. 154–169.MathSciNetMATHCrossRefGoogle Scholar - [27]Georgiopoulos, M., Dagher, I., Heileman, G.L., and Bebis, G. (1999), “Properties of learning of a fuzzy ART variant,”
*Neural Networks*, vol. 12, no. 6, pp. 837–850.CrossRefGoogle Scholar - [28]Giarratano, J. and Riley G. (1994),
*Expert Systems - Principles and Programming*, second edition, PWS Publishing Company, Boston, MA.Google Scholar - [29]Giles, C.L., Miller, C.B., Chen, D., Chen, H.H., Sun, G.Z., and Lee. Y.C. (1992), “Learning and extracting finite state automata with second-order recurrent neural networks,”
*Neural Computation*, vol. 4, no. 3, pp. 393–405.CrossRefGoogle Scholar - [30]Grätzer, G. (1971),
*Lattice theory*, W.H. Freeman, San Francisco, CA.MATHGoogle Scholar - [31]Healy, M.J. (1999), “A topological semantics for rule extraction with neural networks,”
*Connection Science*, vol. 11, no. 1, pp. 91–113.CrossRefGoogle Scholar - [32]Healy, M.J. and Caudell, T.P. (1997), “Acquiring rule sets as a product of learning in a logical neural architecture,”
*IEEE Trans. Neural Networks*, vol. 8, no. 3, pp. 461–474.CrossRefGoogle Scholar - [33]Hong, L. and Jain, A. (1998), “Integrating faces and fingerprints for personal identification,”
*IEEE Trans. Pattern Analysis and Machine Intelligence*, vol. 20, no. 12, pp. 1295–1307.CrossRefGoogle Scholar - [34]Horiuchi, T. (1998), “Decision rule for pattern classification by integrating interval feature values,”
*IEEE Trans. Pattern Analysis and Machine Intelligence*, vol. 20, no. 4, pp. 440–448.MathSciNetCrossRefGoogle Scholar - [35]Hummel, R. and Manevitz, L. (1996), “A statistical approach to the representation of uncertainty in beliefs using spread of opinion,”
*IEEE Trans. Sys*.,*Man*,*and Cybernetics–A*, vol. 26, no. 3, pp. 378–384.CrossRefGoogle Scholar - [36]Ishibuchi, H., Fujioka, R., and Tanaka, H. (1993), “Neural networks that learn from fuzzy if-then rules,”
*IEEE Trans. on Fuzzy Systems*, vol. 1, no. 2, pp. 8597.CrossRefGoogle Scholar - [37]John, M.F.St. and McClelland, J.L. (1990), “Learning and applying contextual constraints in sentence comprehension,”
*Artificial Intelligence*, vol. 46, pp. 5–46.CrossRefGoogle Scholar - [38]Joshi, A., Ramakrishman, N., Houstis, E.N., and Rice, J.R. (1997), “On neurobiological, neuro-fuzzy, machine learning, and statistical pattern recognition techniques,”
*IEEE Trans. Neural Networks*, vol. 8, no. 1, pp. 18–31.CrossRefGoogle Scholar - [39]Kaburlasos, V.G. (1992),
*Adaptive Resonance Theory with Supervised Learning and Large Database Applications*, Ph.D. Thesis, Dept. Electrical Engineering, University of Nevada, Reno.Google Scholar - [40]Kaburlasos, V.G. and Petridis, V. (1997), “Fuzzy lattice neurocomputing (FLN): A novel connectionist scheme for versatile learning and decision making by clustering,”
*International Journal of Computers and Their Applications*, vol. 4, no. 2, pp. 31–43.Google Scholar - [41]Kaburlasos, V.G. and Petridis, V. (1998), “A unifying framework for hybrid information processing,”
*Proceedings of the ISCA 7th Intl. Conf. on Intelligent Systems (ICIS’98)*, Paris, France, pp. 68–71.Google Scholar - [42]Kaburlasos, V.G., Petridis, V., Brett, P., and Baker, D. (1998), “Learning a linear association of drilling profiles in stapedotomy surgery,”
*Proceedings of the IEEE 1998 Intl. Conf. on Robotics and Automation (ICRA’98)*, Leuven, Belgium, pp. 705–710.Google Scholar - [43]Kearns, M.J. and Vazirani, U.V. (1994),
*An Introduction to Computational Learning Theory*, The MIT Press, Cambridge, MA.Google Scholar - [44]Kim, I. and Vachtsevanos, G. (1998), “Overlapping object recognition: a paradigm for multiple sensor fusion,”
*IEEE Robotics and Automation Magazine*, pp. 37–44, September.Google Scholar - [45]Klir, G.J. and Folger, T.A. (1988),
*Fuzzy Sets*,*Uncertainty*,*and Information*, Prentice-Hall International Editions, London, UK.Google Scholar - [46]Klir, G.J. and Yuan, B. (1997),
*Fuzzy Sets*,*and Fuzzy Logic*, Prentice-Hall of India, New Delhi, India.Google Scholar - [47]Kohonen, T. (1995),
*Self-Organizing Maps*, Springer-Verlag, Berl in Heidelberg.CrossRefGoogle Scholar - [48]Kosko, B. (1998), “Global stability of generalized additive fuzzy systems,”
*IEEE Trans. Systems*,*Man*,*and Cybernetics–C*, vol. 28, no. 3, pp. 441–452.CrossRefGoogle Scholar - [49]Kuo, Y-H., Hsu, J-P., and Wang, C-W. (1998), “A parallel fuzzy inference model with distributed prediction scheme for reinforcement learning,”
*IEEE Trans. Systems*,*Man*,*and Cybernetics–B*, vol. 28, no. 2, pp. 160–172.CrossRefGoogle Scholar - [50]Lin, C-T., Lin, C-J., and Lee, C.S.G. (1995), “Fuzzy adaptive learning control network with on-line neural learning,”
*Fuzzy Sets and Systems*, vol. 71, pp. 2545.MathSciNetCrossRefGoogle Scholar - [51]Lucas, C. and Araabi, B.N. (1999), “Generalization of the Dempster-Shafer theory: A fuzzy-valued measure,”
*IEEE Trans. Fuzzy Systems*, vol. 7, no. 3, pp. 255–270.MathSciNetCrossRefGoogle Scholar - [52]Luo, R.C. and Kay, M.G. (1989), “Multisensor integration and fusion in intelligent systems,”
*IEEE Trans. Systems*,*Man*,*and Cybernetics*, vol. 19, no. 5, pp. 901–931.CrossRefGoogle Scholar - [53]Mamdani, E.H. and Assilian, S. (1975), “An experiment in linguistic synthesis with a fuzzy logic controller,”
*Intl. J. of Man-Machine Studies*, vol. 7, pp. 1–13.MATHCrossRefGoogle Scholar - [54]Mardia, K.V., Kent, J.T., and Bibby, J.M. (1979),
*Multivariate Analysis*, Academic Press, London, UK.Google Scholar - [55]Margaris, A. (1990),
*First Order Mathematical Logic*, Dover Publications, Inc., New York, NY.Google Scholar - [56]McLachlan, G.J. (1992),
*Discriminant Analysis and Statistical Pattern Recognition*, John Wiley and Sons, New York, NY.CrossRefGoogle Scholar - [57]Merz, C.J. and Murphy, P.M. (1996), UCI Repository of Machine Learning Databases, Dept. Inform. Comput. Sci., University of California, Irvine.Google Scholar
- [58]Murphy, R.R. (1998), “Dempster-Shafer theory for sensor fusion in autonomous mobile robots,”
*IEEE Trans. Robotics and Automation*, vol. 14, no. 2, pp. 197206.Google Scholar - [59]Nguyen, H.T. and Walker, E.A. (1997),
*Fuzzy Logic*, CRC Press Inc., New York, NY.Google Scholar - [60]Papoulis, A. (1991),
*Probability*,*Random Variables*,*and Stochastic Processes*, third edition, McGraw-Hill, Inc., New York, NY.Google Scholar - [61]Park, J.S., Chen, M-S., and Yu, P.S. (1997), “Using a hash-based method with transaction trimming for mining association rules,”
*IEEE Trans. Knowledge and Data Engring*, vol. 9, no. 5, pp. 813–825.CrossRefGoogle Scholar - [62]Petridis, V., Kaburlasos, V.G., Brett, P., Parker, T., and Day, J.C.C. (1996), “Two level fuzzy lattice (2L-FL) supervised clustering: A new method for soft tissue identification in surgery,”
*Proceedings of the CESA’96 IMACS Multiconference*, Lille, France, pp. 232–237.Google Scholar - [63]Petridis, V. and Kaburlasos, V.G. (1998), “Fuzzy lattice neural network (FLNN): A hybrid model for learning,”
*IEEE Trans. Neural Networks*, vol. 9, no. 5, pp. 877–890.CrossRefGoogle Scholar - [64]Petridis, V. and Kaburlasos, V.G. (1999), “Learning in the framework of fuzzy lattices,”
*IEEE Trans. Fuzzy Systems*, vol. 7, no. 4, pp. 422–440.CrossRefGoogle Scholar - [65]Pontil, M. and Verri, A. (1998), “Support vector machines for 3D object recognition,”
*IEEE Trans. Pattern Analysis and Machine Intelligence*, vol. 20, no. 6, pp. 637–646.CrossRefGoogle Scholar - [66]Pindyck, R.S. and Rubinfeld, D.L. (1991),
*Econometric Models and Economic Forecasts*, McGraw-Hill, Inc., New York, NY.Google Scholar - [67]Rutherford, D.E. (1965),
*Introduction to Lattice Theory*, Oliver and Boyd Ltd., Edinburgh, Great Britain.Google Scholar - [68]Saridis, G.N. (1983), “Intelligent robotic control,”
*IEEE Trans. Automatic Control*, vol. 28, no. 5, pp. 547–557.MathSciNetMATHCrossRefGoogle Scholar - [69]Sarkar, S., Chakrabarti, P.P., and Ghose, S. (1998), “A framework for learning in search-based systems,”
*IEEE Trans. Knowledge and Data Engineering*, vol. 10, no. 4, pp. 563–575.CrossRefGoogle Scholar - [70]Setnes, M., Babuska, R., Kaymak, U., and van Nauta Lemke, H.R. (1998), “Similarity measures in fuzzy rule base simplification,”
*IEEE Trans. Systems*,*Man*,*and Cybernetics–B*, vol. 28, no. 3, pp. 376–386.CrossRefGoogle Scholar - [71]Shafer, G. (1976), A
*Mathematical Theory of Evidence*, Princeton University Press.Google Scholar - Simpson, P.K. (1992), “Fuzzy min-max neural networks - partl: classification
^{-},*IEEE Trans. Neural Networks*,vol. 3, no. 5, pp. 776–786.Google Scholar - [73]Simpson, P.K. (1993), “Fuzzy min-max neural networks–part2: clustering,”
*IEEE Trans. Fuzzy Systems*, vol. 1, no. 1, pp. 32–45.CrossRefGoogle Scholar - [74]Smets, P. (1990), “The combination of evidence in the transferable belief model,”
*IEEE Trans. Pattern Analysis and Machine Intelligence*, vol. 12, no. 5, pp. 447–458.CrossRefGoogle Scholar - [75]Tan, A-H. (1995), “Adaptive resonance associative map,”
*Neural Networks*, vol. 8, no. 3, pp. 437–446.CrossRefGoogle Scholar - [76]Valiant, L.G. (1984), “A theory of the learnable,”
*Communications of the ACM*, vol. 27, no. 11, pp. 1134–1142.MATHCrossRefGoogle Scholar - [77]Vapnik, V.N. and Chervonenkis, Y.A. (1971), “On the uniform convergence of relative frequencies of events to their probabilities,”
*Theory of Probability and its Applications*, vol. 16, no. 2, pp. 264–280.MathSciNetMATHCrossRefGoogle Scholar - [78]Vidyasagar, M. (1997), A
*Theory of Learning and Generalization*, Springer-Verlag, London, England.Google Scholar - [79]Waugh, S. (1995), “Extending and benchmarking Cascade-Correlation,” Ph.D. Thesis, Dept. Computer Science, University of Tasmania.Google Scholar
- [80]Williamson, J.R. (1996), “Gaussian ARTMAP: A neural network for fast incremental learning of noisy multidimensional maps,”
*Neural Networks*, vol. 9, no. 5, pp. 881–897.CrossRefGoogle Scholar - [81]Xu, H., Hsia, Y-T., and Smets, P. (1996), “Transferable belief model for decision making in the valuation-based systems,”
*IEEE Trans. Systems*,*Man*,*and Cybernetics–A*, vol. 26, no. 6, pp. 698–707.CrossRefGoogle Scholar - [82]Yager, R.R. (1996), “On the aggregation of prioritized belief structured belief structures,”
*IEEE Trans. Systems*,*Man*,*and Cybernetics–A*, vol. 26, no. 6, pp. 708–717.CrossRefGoogle Scholar - [83]Zadeh, L.A. (1965), “Fuzzy sets,”
*Information and Control*, vol. 8, pp. 338–353.MathSciNetMATHCrossRefGoogle Scholar - [84]Zadeh, L.A. (1984), “Review of books: A mathematical theory of evidence,”
*The AI Magazine*, pp. 81–83.Google Scholar - [85]Zadeh, L.A. (1996), “Fuzzy logic = Computing with words,”
*IEEE Trans. Fuzzy Systems*, vol. 4, no. 2, pp. 103–111.MathSciNetCrossRefGoogle Scholar - [86]Zeng, X-J. and Singh, M.G. (1997), “Fuzzy bounded least-squares method for the identification of linear systems,”
*IEEE Trans. Systems*,*Man*,*Cybernetics — A*, vol. 27, no. 5, pp. 624–635.CrossRefGoogle Scholar - [87]Zimmermann, H.J. (1991),
*Fuzzy Set Theory and Its Applications*, Kluwer Academic Publishers, Norwell, MA.MATHCrossRefGoogle Scholar - [88]Zouhal, L.M. and Denoeux, T. (1998), “An evidence-theoretic k-NN Rule with parameter optimization,”
*IEEE Trans. Systems*,*Man*,*and Cybernetics–C*, vol. 28, no. 2, pp. 263–271.CrossRefGoogle Scholar - [89]Zurada, J.M., Marks II, R.J., and Robinson, C.J. (Eds.) (1994),
*Computational Intelligence — Imitating Life*, IEEE Press, New York, NY.Google Scholar