Machine Learning

, Volume 72, Issue 1–2, pp 113–138 | Cite as

Learning large-alphabet and analog circuits with value injection queries

  • Dana Angluin
  • James Aspnes
  • Jiang Chen
  • Lev Reyzin


We consider the problem of learning an acyclic discrete circuit with n wires, fan-in bounded by k and alphabet size s using value injection queries. For the class of transitively reduced circuits, we develop the Distinguishing Paths Algorithm, that learns such a circuit using (ns)O(k) value injection queries and time polynomial in the number of queries. We describe a generalization of the algorithm to the class of circuits with shortcut width bounded by b that uses (ns)O(k+b) value injection queries. Both algorithms use value injection queries that fix only O(kd) wires, where d is the depth of the target circuit. We give a reduction showing that without such restrictions on the topology of the circuit, the learning problem may be computationally intractable when s=n Θ(1), even for circuits of depth O(log n). We then apply our large-alphabet learning algorithms to the problem of approximate learning of analog circuits whose gate functions satisfy a Lipschitz condition. Finally, we consider models in which behavioral equivalence queries are also available, and extend and improve the learning algorithms of (Angluin in Proceedings of the Thirty-Eighth Annual ACM Symposium on Theory of Computing, pp. 584–593, 2006) to handle general classes of gate functions that are polynomial time learnable from counterexamples.


Value injection queries Learning circuits Query learning 


  1. Aho, A. V., Garey, M. R., & Ullman, J. D. (1972). The transitive reduction of a directed graph. SIAM Journal on Computing, 1, 131–137. MATHCrossRefMathSciNetGoogle Scholar
  2. Akutsu, T., Kuhara, S., Maruyama, O., & Miyano, S. (2003). Identification of genetic networks by strategic gene disruptions and gene overexpressions under a boolean model. Theoretical Computer Science, 298(1), 235–251. MATHCrossRefMathSciNetGoogle Scholar
  3. Alon, N., Yuster, R., & Zwick, U. (1995). Color-coding. Journal of the ACM, 42(4), 844–856. MATHCrossRefMathSciNetGoogle Scholar
  4. Angluin, D., & Kharitonov, M. (1995). When won’t membership queries help? Journal of Computer and System Sciences, 50(2), 336–355. MATHCrossRefMathSciNetGoogle Scholar
  5. Angluin, D., Frazier, M., & Pitt, L. (1992). Learning conjunctions of Horn clauses. Machine Learning, 9, 147–164. Google Scholar
  6. Angluin, D., Hellerstein, L., & Karpinski, M. (1993). Learning read-once formulas with queries. Journal of the ACM, 40, 185–210. MATHCrossRefMathSciNetGoogle Scholar
  7. Angluin, D., Aspnes, J., Chen, J., & Wu, Y. (2006). Learning a circuit by injecting values. In Proceedings of the thirty-eighth annual ACM symposium on theory of computing (pp. 584–593). New York: ACM. CrossRefGoogle Scholar
  8. Angluin, D., Aspnes, J., Chen, J., & Reyzin, L. (2007). Learning large-alphabet and analog circuits with value injection queries. In Twentieth annual conference on learning theory (pp. 51–65), June 2007. Google Scholar
  9. Bshouty, N. H. (1995). Exact learning boolean functions via the monotone theory. Information and Computation, 123(1), 146–153. MATHCrossRefMathSciNetGoogle Scholar
  10. Downey, R. G., & Fellows, M. R. (1999). Parameterized complexity. Berlin: Springer. Google Scholar
  11. Ideker, T., Thorsson, V., & Karp, R. (2000). Discovery of regulatory interactions through perturbation: inference and experimental design. In Pacific symposium on biocomputing (Vol. 5, pp. 302–313). Google Scholar
  12. Jackson, J. C. (1997). An efficient membership-query algorithm for learning DNF with respect to the uniform distribution. Journal of Computer and System Sciences, 55(3), 414–440. MATHCrossRefMathSciNetGoogle Scholar
  13. Jackson, J. C., Klivans, A. R., & Servedio, R. A. (2002). Learnability beyond AC. In STOC ’02: proceedings of the thirty-fourth annual ACM symposium on theory of computing (pp. 776–784). New York: ACM. CrossRefGoogle Scholar
  14. Jeffreys, H., & Jeffreys, B. (1988). Methods of mathematical physics (3rd ed.), Cambridge: Cambridge University Press. Google Scholar
  15. Kearns, M., & Valiant, L. (1994). Cryptographic limitations on learning boolean formulae and finite automata. Journal of the ACM, 41(1), 67–95. MATHCrossRefMathSciNetGoogle Scholar
  16. Kharitonov, M. (1993). Cryptographic hardness of distribution-specific learning. In STOC ’93: proceedings of the twenty-fifth annual ACM symposium on theory of computing (pp. 372–381). New York: ACM. CrossRefGoogle Scholar
  17. Linial, N., Mansour, Y., & Nisan, N. (1993). Constant depth circuits, Fourier transform, and learnability. Journal of the ACM, 40(3), 607–620. MATHCrossRefMathSciNetGoogle Scholar
  18. Littlestone, N. (1988). Learning quickly when irrelevant attributes abound: a new linear-threshold algorithm. Machine Learning, 2(4), 285–318. Google Scholar
  19. Maass, W., & Turan, G. (1989). On the complexity of learning from counterexamples. In The 30th annual symposium on foundations of computer science (pp. 262–267). Google Scholar
  20. Niedermeier, R. (2006). Invitation to fixed-parameter algorithms. London: Oxford University Press. MATHGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2008

Authors and Affiliations

  • Dana Angluin
    • 1
  • James Aspnes
    • 1
  • Jiang Chen
    • 2
  • Lev Reyzin
    • 1
  1. 1.Computer Science DepartmentYale UniversityNew HavenUSA
  2. 2.Center for Computational Learning SystemsColumbia UniversityNew YorkUSA

Personalised recommendations