# On approximately identifying concept classes in the limit

## Abstract

In this paper, we introduce various kinds of approximations of a concept and propose a framework of approximate learning in case that a target concept could be outside the hypothesis space. We present some characterization theorems for approximately identifiability. In particular, we show a remarkable result that the upper-best approximate identifiability from *complete* data is collapsed into the upper-best approximate identifiability from *positive* data. Further, some other characterizations for approximate identifiability from positive data are presented, where we establish a relationship between approximate identifiability and some important notions in quasi-order theory and topology theory. The results obtained in this paper are essentially related to the closure property of concept classes under *infinite* intersections (or *infinite* unions). We also show that there exist some interesting example concept classes with such properties (including specialized EFS's) by which an upper-best approximation of *any* concept can be identifiable in the limit from positive data.

## Keywords

Open Cover Concept Class Infinite Sequence Inductive Inference Closure Property## Preview

Unable to display preview. Download preview PDF.

## References

- [Ang80]D. Angluin. Inductive inference of formal languages from positive data.
*Information and Control*, vol. 45, pp.117–135, 1980CrossRefGoogle Scholar - [ASY92]S. Arikawa, T. Shinohara and A. Yamamoto. Learning Elementary Formal Systems.
*Theoretical Computer Science*,**95**, pp.97–113, 1992CrossRefGoogle Scholar - [Gol67]E. Mark Gold. Language identification in the limit.
*Information and Control*, vol. 10, pp.447–474, 1967CrossRefGoogle Scholar - [Hai69]L. H. Haines. On Free Monoids Partially Ordered by Embedding,
*Journal of Combinatorial Theory*,**6**, pp.94–98, 1969Google Scholar - [Hau92]D. Haussler. Decisin Theoretic Generalizations of the PAC model for Neural Net and Other Learning Applications.
*Information and Computation*, Vol. 100, No. 1, pp.78–150, 1992Google Scholar - [Kru60]J. B. Kruskal. Well-Quasi-Ordering, the Tree Theorem and Vazsonyi's Conjecture.
*Trans. Amer. Math. Soc.*,**95**, pp.210–225, 1960 (May)Google Scholar - [Kru72]J. B. Kruskal. The Theory of Well-Quasi-Ordering: A Frequently Discovered Concept.
*Journal of Combinatorial Theory (A)*,**13**, pp.297–305, 1972Google Scholar - [KSS92]M. J. Kearns, R. E. Schapire and L. M. Sellie. Toward Efficient Agnostic Learning,
*Proc. of 5th Annual Workshop on Computational Learning Theory*, pp.341–351, 1992Google Scholar - [KY94a]S. Kobayashi. and T. Yokomori. Families of Noncounting Languages and Its Learnability from Positive Data. Technical Report CSIM 94-03, University of Electro-Communications, September, 1994Google Scholar
- [KY94b]S. Kobayashi and T. Yokomori. An Extended Rough Set Theory Toward Approximate Learning of Formal Languages.
*Proc. of International Workshop on Rough Sets and Soft Computing*, pp.482–489, 1994Google Scholar - [KY94c]S. Kobayashi and T. Yokomori. Some Characterizations of Approximately Identifiable Language Classes. Technical Report CSIM 94-06, Department of Computer Science and Information Mathematics, University of Electro-Communications, December, 1994Google Scholar
- [MS93]T. Moriyama and M. Sato. Properties of Language Classes with Finite Elasticity.
*Proc. 4th Workshop on Algorithmic Learning Theory*, Lecture Notes in Artificial Intelligence**744**, pp.187–196, 1993.Google Scholar - [MSW90]T. Motoki, T. Shinohara and K. Wright. The correct definition of finite elasticity: corrigendum to identification of unions.
*Proc. of 4th Workshop on Computational Learning Theory*, pp.375–375, 1991Google Scholar - [Muk93a]Y. Mukouchi and S. Arikawa. Inductive Inference Machines That Can Refute Hypothesis Spaces.
*Proc. of 4th Workshop on Algorithmic Learning Theory*, Lecture Notes in Artificial Intelligence**744**, Springer-Verlag, pp. 123–136, 1993Google Scholar - [Muk94]Y. Mukouchi. Inductive Inference of an Approximate Concept from Positive Data, in
*Proc. of 5th International Workshop on Algorithmic Learning Theory*, Lecture Notes in Artificial Intelligence**872**, Springer-Verlag, pp.484–499, 1994Google Scholar - [Sak91]A. Sakurai. Inductive Inference of Formal Languages from Positive Data Enumerated Primitive-Recursively,
*Proc. of 2nd Workshop on Algorithmic Learning Theory*, pp.73–83, 1991Google Scholar - [SM93]M. Sato and T. Moriyama. Inductive Inference of Length Bounded EFS's from Positive Data, in preparation, 1993Google Scholar
- [Paw91]Z. Pawlak. Rough Sets: Theoretical Aspects of Reasoning about Data. Kulwer Academic Publishers, 1991Google Scholar
- [PV88]L. Pitt and L. G. Valiant. Computational Limitations on Learning from Examples,
*Journal of the ACM*, Vol. 35, No. 4, pp.965–984, October 1988CrossRefGoogle Scholar - [Shi94]T. Shinohara. Rich Classes Inferable from Positive Data: Length Bounded Elementary Formal Systems.
*Information and Computation*,**108**, pp.175–186, 1994CrossRefGoogle Scholar - [Smu61]R. M. Smullyan. Theory of Formal Systems.
*Princeton Univ. Press*, 1961Google Scholar - [Val84]L. G. Valiant. A Theory of the Learnable,
*Communications of the ACM*, Vol. 27, No. 11, pp.1134–1142, November 1984CrossRefGoogle Scholar - [Wri89]K. Wright. Identification of unions of languages drawn from an identifiable class.
*Proc. of 2nd Workshop on Computational Learning Theory*, pp.328–333, 1989Google Scholar