Abstract
In this paper, a new powerful technique, denoted as pack-based strategy is introduced in the context of statistical learning theory. This strategy allows us to derive bounds on the number of required samples that are manageable for “reasonable” values of probabilistic confidence and accuracy. Using this technique for feasibility and optimization problems involving Boolean expressions consisting of polynomials, we prove that the number of required samples grows with the accuracy parameter ∈ as 1/∈ ln 1/∈. This is a significant improvement when compared to the existing bounds which depend on 1/∈2 ln 1/∈2. We also apply this strategy to convex optimization problems. In this case, we show that the required sample size is inversely proportional to the accuracy for fixed confidence.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alamo, T., Tempo, R., Camacho, E.F.: Revisiting statistical learning theory for uncertain feasibility and optimization problems. In: Proceedings of the 46th IEEE Conference on Decision and Control, New Orleans, USA (December 2007)
Alamo, T., Tempo, R., Camacho, E.F.: Improved sample size bounds for probabilistic robust control design: a pack-based strategy. In: Proceedings of the 46th IEEE Conference on Decision and Control, New Orleans, USA (December 2007)
Alamo, T., Tempo, R., Rodríguez, D., Camacho, E.F.: A sequentially optimal randomized algorithm for robust LMI feasibility problems. In: Proceedings of the European Control Conference, 2007, Kos, Greece (July 2007)
Calafiore, G., Campi, M.C.: Uncertain convex programs: randomized solutions and confidence levels. Math. Program. 102, 25–46 (2005)
Calafiore, G., Campi, M.C.: The scenario approach to robust control design. IEEE Transactions on Automatic Control 51(5), 742–753 (2006)
Calafiore, G., Dabbene, F.: An iterative localization method for probabilistic feasibility of uncertain LMIs. In: Proceedings of the 45th IEEE Conference on Decision and Control, San Diego, USA (December 2006)
Calafiore, G., Polyak, B.T.: Stochastic algorithms for exact and approximate feasibility of robust LMIs. IEEE Transactions on Automatic Control 46(11), 1755–1759 (2001)
Chernoff, H.: A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. Annals of mathematical statistics 23, 493–507 (1952)
Fujisaki, Y., Dabbene, F., Tempo, R.: Probabilistic design of LPV control systems. Automatica 39, 1323–1337 (2003)
Fujisaki, Y., Kozawa, Y.: Probabilistic robust controller design: probable near minimax value and randomized algorithms. In: Calafiore, G., Dabbene, F. (eds.) Probabilistic and Randomized Methods for Design under Uncertainty, Springer, London (2006)
Hoeffding, W.: Probability inequalities for sums of bounded random variables. Journal of American Statistics Association 58, 13–30 (1963)
Kanev, S., De Schutter, B., Verhaegen, M.: An ellipsoid algorithm for probabilistic robust controller design. Systems and Control Letters 49, 365–375 (2003)
Koltchinskii, V., Abdallah, C.T., Ariola, M., Dorato, P., Panchenko, D.: Improved sample complexity estimates for statistical learning control of uncertain systems. IEEE Transactions on Automatic Control 45(12), 2383–2388 (2000)
Polyak, B.T., Tempo, R.: Probabilistic robust design with linear quadratic regulators. Systems & Control Letters 43, 343–353 (2001)
Tempo, R., Bai, E.-W., Dabbene, F.: Probabilistic robustness analysis: explicit bounds for the minimum number of samples. Systems & Control Letters 30, 237–242 (1997)
Tempo, R., Calafiore, G., Dabbene, F.: Randomized Algorithms for Analysis and Control of Uncertain Systems. Communications and Control Engineering Series. Springer, London (2005)
Vapnik, V.N.: Statistical Learning Theory. John Wiley & Sons, New York (1998)
Vapnik, V.N., Chervonenkis, A. Ya.: On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications 16, 264–280 (1971)
Vapnik, V.N., Chervonenkis, A. Ya.: Necessary and sufficient conditions for the uniform convergence of means to their expectations. Theory of Probability and its Applications 26(3), 532–553 (1981)
Vidyasagar, M.: A Theory of Learning and Generalization: with Applications to Neural Networks and Control Systems. Springer, London (1997)
Vidyasagar, M.: Statistical learning theory and randomized algorithms for control. Control Systems Magazine 18(6), 69–85 (1998)
Vidyasagar, M.: Randomized algorithms for robust controller synthesis using statistical learning theory. Automatica 37, 1515–1528 (2001)
Vidyasagar, M., Blondel, V.D.: Probabilistic solutions to some NP-hard matrix problems. Automatica 37(9), 1397–1405 (2001)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Alamo, T., Tempo, R., F., E. (2008). Statistical Learning Theory: A Pack-based Strategy for Uncertain Feasibility and Optimization Problems. In: Blondel, V.D., Boyd, S.P., Kimura, H. (eds) Recent Advances in Learning and Control. Lecture Notes in Control and Information Sciences, vol 371. Springer, London. https://doi.org/10.1007/978-1-84800-155-8_1
Download citation
DOI: https://doi.org/10.1007/978-1-84800-155-8_1
Publisher Name: Springer, London
Print ISBN: 978-1-84800-154-1
Online ISBN: 978-1-84800-155-8
eBook Packages: EngineeringEngineering (R0)