Bayesian Active Learning Using Arbitrary Binary Valued Queries

  • Liu Yang
  • Steve Hanneke
  • Jaime Carbonell
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6331)


We explore a general Bayesian active learning setting, in which the learner can ask arbitrary yes/no questions. We derive upper and lower bounds on the expected number of queries required to achieve a specified expected risk.


Active Learning Bayesian Learning Sample Complexity Information Theory 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Bshouty, N.H., Li, Y., Long, P.M.: Using the doubling dimension to analyze the generalization of learning algorithms. Journal of Computer and System Sciences 75, 323–335 (2009)zbMATHCrossRefMathSciNetGoogle Scholar
  2. Cover, T.M., Thomas, J.A.: Elements of information theory. John Wiley & Sons, Inc., Chichester (2006)zbMATHGoogle Scholar
  3. Dasgupta, S.: Coarse sample complexity bounds for active learning. In: Advances in Neural Information Processing Systems, vol. 18 (2005)Google Scholar
  4. Freund, Y., Seung, H.S., Shamir, E., Tishby, N.: Selective sampling using the query by committee algorithm. Machine Learning 28, 133–168 (1997)zbMATHCrossRefGoogle Scholar
  5. Gupta, A., Krauthgamer, R., Lee, J.R.: Bounded geometries, fractals, and lowdistortion mbeddings. In: Proceedings of the 44th Annual IEEE Symposium on Foundations of Computer Science (2003)Google Scholar
  6. Hanneke, S.: A bound on the label complexity of agnostic active learning. In: Proceedings of the 24th International Conference on Machine Learning (2007a)Google Scholar
  7. Hanneke, S.: Teaching dimension and the complexity of active learning. In: Proceedings of the 20th Annual Conference on Learning Theory (2007b)Google Scholar
  8. Haussler, D., Kearns, M., Schapire, R.: Bounds on the sample complexity of Bayesian learning using information theory and the VC dimension. Machine Learning 14, 83–113 (1994a)zbMATHGoogle Scholar
  9. Haussler, D., Littlestone, N., Warmuth, M.: Predicting {0, 1}-functions on randomly drawn points. Information and Computation 115, 248–292 (1994b)zbMATHCrossRefMathSciNetGoogle Scholar
  10. Kulkarni, S.R., Mitter, S.K., Tsitsiklis, J.N.: Active learning using arbitrary binary valued queries. Machine Learning 11, 23–35 (1993)zbMATHCrossRefGoogle Scholar
  11. Seung, H.S., Opper, M., Sompolinsky, H.: Query by committee. In: Proceedings of the 5th Workshop on Computational Learning Theory (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Liu Yang
    • 1
  • Steve Hanneke
    • 2
  • Jaime Carbonell
    • 3
  1. 1.Machine Learning DepartmentCarnegie Mellon University 
  2. 2.Department of StatisticsCarnegie Mellon University 
  3. 3.Language Technologies InstituteCarnegie Mellon University 

Personalised recommendations