Abstract
Recent works have shown the advantage of using Active Learning methods, such as the Query by Committee (QBC) algorithm, to various learning problems. This class of Algorithms requires an oracle with the ability to randomly select a consistent hypothesis according to some predefined distribution. When trying to implement such an oracle, for the linear separators family of hypotheses, various problems should be solved. The major problem is time-complexity, where the straight-forward Monte Carlo method takes exponential time.
In this paper we address some of those problems and show how to convert them to the problems of sampling from convex bodies or approximating the volume of such bodies. We show that recent algorithms for approximating the volume of convex bodies and approximately uniformly sampling from convex bodies using random walks, can be used to solve this problem, and yield an eficient implementation for the QBC algorithm. This solution suggests a connection between random walks and certain properties known in machine learning such as ε-net and support vector machines. Working out this connection is left for future work.
Partially supported by project I403-001.06/95 of the German-Israeli Foundation for Scientific Research [GIF].
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
R. Bachrach, S. Fine, and E. Shamir. Query by committee, linear separation and random walks-full version. 1998. Available at http://www.cs.huji.ac.il/labs/learning/Papers/MLT list.html.
C. Burges. A tutorial on support vector machines for pattern recognition. 1998. Available at http://svm.research.bell-labs.com/SVMdoc.html.
D. Cohn, L. Atlas, and R. Ladner. Training connectionist networks with queries and selective sampling. Advanced in Neural Information Processing Systems 2, 1990.
M. Dyer, A. Frieze, and R. Kannan. A random polynomial time algorithm for approximating the volume of convex bodies. Journal of the Association for Com-puting Machinery, 38, Number 1:1–17, 1991.
H.G. Eggleston. Convexity. Cambridge Univ. Press, 1958.
Y. Freund, H. Seung, E. Shamir, and N. Tishby. Selective sampling using the query by committee algorithm. Macine Learning, 28:133–168, 1997.
L. Grotchel, L. Lovasz, and A. Schrijver. Geometric algorithms and Combinatorial Optimization. Springer-Verlag, Berlin, 1988.
D. Haussler, M. Kearns, and R.E. Schapie. Bounds on the sample complexity of bayesian learning using information theory and the vc dimension. Machine Learning, 14:83–113, 1994.
L. Lovasz and M. Simonovits. Random walks in a convex body and an improved volume algorithm. Random Structures and Algorithms, 4, Number 4:359–412, 1993.
D.A. McAllester. Some pac-bayesian theorems. Proc. of the Eleventh Annual Conference on Computational Learning Theory, pages 230–234, 1998.
H.S. Seung, M. Opper, and H. Sompolinsky. Query by committee. Proc. of the Fith Workshop on Computational Learning Theory, pages 287–294, 1992.
J. Shawe-Taylor and R.C. Williamson. A pac analysis of a bayesian estimator. Proc. of the Tenth Annual Conference on Computational Learning Theory., 1997.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1999 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bachrach, R., Fine, S., Shamir, E. (1999). Query by Committee, Linear Separation and Random Walks. In: Fischer, P., Simon, H.U. (eds) Computational Learning Theory. EuroCOLT 1999. Lecture Notes in Computer Science(), vol 1572. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-49097-3_4
Download citation
DOI: https://doi.org/10.1007/3-540-49097-3_4
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-65701-9
Online ISBN: 978-3-540-49097-5
eBook Packages: Springer Book Archive