On approximately identifying concept classes in the limit

  • Satoshi Kobayashi
  • Takashi Yokomori
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 997)


In this paper, we introduce various kinds of approximations of a concept and propose a framework of approximate learning in case that a target concept could be outside the hypothesis space. We present some characterization theorems for approximately identifiability. In particular, we show a remarkable result that the upper-best approximate identifiability from complete data is collapsed into the upper-best approximate identifiability from positive data. Further, some other characterizations for approximate identifiability from positive data are presented, where we establish a relationship between approximate identifiability and some important notions in quasi-order theory and topology theory. The results obtained in this paper are essentially related to the closure property of concept classes under infinite intersections (or infinite unions). We also show that there exist some interesting example concept classes with such properties (including specialized EFS's) by which an upper-best approximation of any concept can be identifiable in the limit from positive data.


Open Cover Concept Class Infinite Sequence Inductive Inference Closure Property 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [Ang80]
    D. Angluin. Inductive inference of formal languages from positive data. Information and Control, vol. 45, pp.117–135, 1980CrossRefGoogle Scholar
  2. [ASY92]
    S. Arikawa, T. Shinohara and A. Yamamoto. Learning Elementary Formal Systems. Theoretical Computer Science, 95, pp.97–113, 1992CrossRefGoogle Scholar
  3. [Gol67]
    E. Mark Gold. Language identification in the limit. Information and Control, vol. 10, pp.447–474, 1967CrossRefGoogle Scholar
  4. [Hai69]
    L. H. Haines. On Free Monoids Partially Ordered by Embedding, Journal of Combinatorial Theory, 6, pp.94–98, 1969Google Scholar
  5. [Hau92]
    D. Haussler. Decisin Theoretic Generalizations of the PAC model for Neural Net and Other Learning Applications. Information and Computation, Vol. 100, No. 1, pp.78–150, 1992Google Scholar
  6. [Kru60]
    J. B. Kruskal. Well-Quasi-Ordering, the Tree Theorem and Vazsonyi's Conjecture. Trans. Amer. Math. Soc., 95, pp.210–225, 1960 (May)Google Scholar
  7. [Kru72]
    J. B. Kruskal. The Theory of Well-Quasi-Ordering: A Frequently Discovered Concept. Journal of Combinatorial Theory (A), 13, pp.297–305, 1972Google Scholar
  8. [KSS92]
    M. J. Kearns, R. E. Schapire and L. M. Sellie. Toward Efficient Agnostic Learning, Proc. of 5th Annual Workshop on Computational Learning Theory, pp.341–351, 1992Google Scholar
  9. [KY94a]
    S. Kobayashi. and T. Yokomori. Families of Noncounting Languages and Its Learnability from Positive Data. Technical Report CSIM 94-03, University of Electro-Communications, September, 1994Google Scholar
  10. [KY94b]
    S. Kobayashi and T. Yokomori. An Extended Rough Set Theory Toward Approximate Learning of Formal Languages. Proc. of International Workshop on Rough Sets and Soft Computing, pp.482–489, 1994Google Scholar
  11. [KY94c]
    S. Kobayashi and T. Yokomori. Some Characterizations of Approximately Identifiable Language Classes. Technical Report CSIM 94-06, Department of Computer Science and Information Mathematics, University of Electro-Communications, December, 1994Google Scholar
  12. [MS93]
    T. Moriyama and M. Sato. Properties of Language Classes with Finite Elasticity. Proc. 4th Workshop on Algorithmic Learning Theory, Lecture Notes in Artificial Intelligence 744, pp.187–196, 1993.Google Scholar
  13. [MSW90]
    T. Motoki, T. Shinohara and K. Wright. The correct definition of finite elasticity: corrigendum to identification of unions. Proc. of 4th Workshop on Computational Learning Theory, pp.375–375, 1991Google Scholar
  14. [Muk93a]
    Y. Mukouchi and S. Arikawa. Inductive Inference Machines That Can Refute Hypothesis Spaces. Proc. of 4th Workshop on Algorithmic Learning Theory, Lecture Notes in Artificial Intelligence 744, Springer-Verlag, pp. 123–136, 1993Google Scholar
  15. [Muk94]
    Y. Mukouchi. Inductive Inference of an Approximate Concept from Positive Data, in Proc. of 5th International Workshop on Algorithmic Learning Theory, Lecture Notes in Artificial Intelligence 872, Springer-Verlag, pp.484–499, 1994Google Scholar
  16. [Sak91]
    A. Sakurai. Inductive Inference of Formal Languages from Positive Data Enumerated Primitive-Recursively, Proc. of 2nd Workshop on Algorithmic Learning Theory, pp.73–83, 1991Google Scholar
  17. [SM93]
    M. Sato and T. Moriyama. Inductive Inference of Length Bounded EFS's from Positive Data, in preparation, 1993Google Scholar
  18. [Paw91]
    Z. Pawlak. Rough Sets: Theoretical Aspects of Reasoning about Data. Kulwer Academic Publishers, 1991Google Scholar
  19. [PV88]
    L. Pitt and L. G. Valiant. Computational Limitations on Learning from Examples, Journal of the ACM, Vol. 35, No. 4, pp.965–984, October 1988CrossRefGoogle Scholar
  20. [Shi94]
    T. Shinohara. Rich Classes Inferable from Positive Data: Length Bounded Elementary Formal Systems. Information and Computation, 108, pp.175–186, 1994CrossRefGoogle Scholar
  21. [Smu61]
    R. M. Smullyan. Theory of Formal Systems. Princeton Univ. Press, 1961Google Scholar
  22. [Val84]
    L. G. Valiant. A Theory of the Learnable, Communications of the ACM, Vol. 27, No. 11, pp.1134–1142, November 1984CrossRefGoogle Scholar
  23. [Wri89]
    K. Wright. Identification of unions of languages drawn from an identifiable class. Proc. of 2nd Workshop on Computational Learning Theory, pp.328–333, 1989Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1995

Authors and Affiliations

  • Satoshi Kobayashi
    • 1
  • Takashi Yokomori
    • 1
  1. 1.Department of Computer Science and Information MathematicsThe University of Electro-CommunicationsTokyoJapan

Personalised recommendations