Complexity in Information Theory pp 63-76 | Cite as

# Introduction to Information-Based Complexity

## Abstract

Information-based complexity is based on three assumptions: information is partial, contaminated, and it costs. Its goal is to create a general theory about problems with such information, and to apply the results to solving specific problems in varied disciplines. Some of the questions asked in information- based complexity are listed and some results are indicated. Information-based complexity is contrasted with combinatorial complexity which studies problems such as the traveling salesman problem and linear programming. In combinatorial complexity, information is assumed to be complete, exact, and free. The relation between information theory and information-based complexity is briefly discussed.

## Keywords

Travel Salesman Problem Human Visual System Clock Synchronization Geophysical Exploration Intrinsic Uncertainty## Preview

Unable to display preview. Download preview PDF.

## References

- [1]Boult, T. and Sikorski, K. (1986), Complexity of Computing Topological Degree of Lipschitz Functions in
*n*Dimensions, to appear in*Journal of Complexity*,**2**.Google Scholar - [2]Brillouin, L. (1956),
*Science and Information Theory*, Academic Press, New York.MATHGoogle Scholar - [3]Fisher, R. A. (1950),
*Contributions to Mathematical Statistics*, John Wiley and Sons, New York.MATHGoogle Scholar - [4]Gallager, R. G. (1968),
*Information Theory and Reliable Communication*, John Wiley and Sons, New York.MATHGoogle Scholar - [5]Grimson, W. E. L. (1981),
, MIT Press, Cambridge, Mass.Google Scholar**From Images to Surfaces: A Computational Study of the Human Early Visual System** - [6]Kacewicz, B. Z. (1984), How to Increase the Order to Get Minimal-Error Algorithms for Systems of ODE,
*Numer. Math.*,**45**, 93–104.MathSciNetMATHCrossRefGoogle Scholar - [7]Kadane, J. B., G. W. Wasilkowski, and H. Woźniakowski (1984), Can Adaption Help on the Average for Stochastic Information?,
*Report*, Columbia University.Google Scholar - [8]Kolmogorov, A. N. and V. M. Tihomirov, ɛ-Entropy and ɛ-Capacity of sets in Functional Spaces,
*Usp. Mat. Nauk.***14**, 3–80 (in Russian), (1959). Translation in*Amer. Math. soc. Trans.***17**, 277–364, (1961).MathSciNetMATHGoogle Scholar - [8a]Kolmogorov, A. N. and V. M. Tihomirov, ɛ-Entropy and ɛ-Capacity of sets in Functional Spaces,
*Usp. Mat. Nauk*.**14**, 3–80 (in Russian), (1959). Translation in*Amer. Math. Soc. Transl.*17, 277 – 364, (1961).MathSciNetGoogle Scholar - [9]Kuczyński, J. (1986), On the Optimal Solution of Large Eigenpair Problems, to appear in
*Journal of Complexity*,**>2**.Google Scholar - [10]Kullback, S. (1961),
*Information Theory and Statistics*, John Wiley and Sons, New York.Google Scholar - [11]Lee, D. (1985), Optimal Algorithms for Image Understanding: Current Status and Future Plans,
*Journal of Complexity*,**1**, 138–146.MathSciNetMATHCrossRefGoogle Scholar - [12]Lee, D. and G. W. Wasilkowski (1985), Approximation of Linear Functionals on a Banach Space with a Gaussian Measure,
*Report*, Columbia University. To appear in*Journal of Complexity*,*2*(1986).Google Scholar - [13]Marr, D. (1981), VISION:
*A Computational Investigation in the Human Representation and Processing of Visual Information*, W. H. Freeman, San Francisco.Google Scholar - [14]Marr, D. and T. Poggio (1977), From Understanding Computation to Understanding Neural Circuitry,
*Neuroscience Research Program Bulletin***15**, 470–488.Google Scholar - [15]Marschak, J. and R. Radner (1972),
*Economic Theory of Teams*, Yale University Press, New Haven.MATHGoogle Scholar - [16]Milanese, M., R. Tempo, and A. Vicino (1986), Strongly Optimal Algorithms and Optimal Information in Estimation Problems, to appear in
*Journal of Complexity*,**2**.Google Scholar - [17]Nemirovsky, A. S. and D. B. Yudin (1983),
*Problem Complexity and Method Efficiency in Optimization*, Wiley-Interscience, New York.MATHGoogle Scholar - [18]Pinsker, M. S. (1964),
*Information and Information Stability of Random Variables and Processes*, Holden-Day, San Francisco.MATHGoogle Scholar - [19]Shannon, C. E. (1948), Mathematical Theory of Communication,
*Bell Syst. Tech. J.***27**, 379–423, 623–658.MathSciNetMATHGoogle Scholar - [19a]Shannon, C. E. (1948), Mathematical Theory of Communication,
*Bell Syst. Tech. J.***2**, 379–423, 623–658.MathSciNetMATHGoogle Scholar - [20]Sikorski, K. (1984), Optimal Solution of Nonlinear Equations Satisfying a Lipschitz Condition,
*Numer. Math.***43**, 225–240.MathSciNetMATHCrossRefGoogle Scholar - [21]Sikorski, K. (1985), Optimal Solutions of Nonlinear Equations,
*Journal of Complexity*,**1**.Google Scholar - [22]Theil, H. (1967),
*Economics and Information Theory*, North-Holland, Amsterdam.Google Scholar - [23]Traub, J. F. (1985), Information, Complexity, and the Sciences, University Lecture, Columbia University.Google Scholar
- [24]Traub, J. F., G. W. Wasilkowski, and H. Woźniakowski (1983),
*Information, Uncertainty, Complexity*, Addison-Wesley, Reading, Mass.MATHGoogle Scholar - [25]Traub, J. F., G. W. Wasilkowski, and H. Woźniakowski (1984a), Average Case Optimality for Linear Problems,
*Journal Theoret. Comp. Sci.*,**29**, 1–25.CrossRefGoogle Scholar - [26]Traub, J. F., G. W. Wasilkowski, and H. Woźniakowski (1984b), When is Non-adaptive Information as Powerful as Adaptive Information?,
*Proceedings of the 23rd IEEE Conference on Decision and Control*, 1536–1540.Google Scholar - [27]Traub, J. F. and H. Woźniakowski (1984a),
*A General Theory of Optimal Algorithms*, Academic Press, New York, NY.MATHGoogle Scholar - [28]Traub, J. F. and H. Woźniakowski (1984a), Information and Computation, chapter in
*Advances in Computers***23**, M. C. Yovits, editor, Academic Press, New York 35–92.Google Scholar - [29]Traub, J. F. and H. Woźniakowski (1984b), On the Optimal Solution of Large Linear Systems,
*Journal of the ACM*,**31**, 545–559.MATHCrossRefGoogle Scholar - [30]Traub, J. F. and H. Woźniakowski (1986a), Information-based Complexity, to appear in
*Annual Review of Computer Science***1**, Annual Reviews, Inc., Palo Alto.Google Scholar - [31]Traub, J. F. and H. Woźniakowski (1986b), Measures of Uncertainty and Information, in progress.Google Scholar
- [32]Twomey, S. (1977),
*Introduction to the Mathematics of Inversion in Remote Sensing and Indirect Measurement*, Developments in Geomathematics**3**, Elsevier, Amsterdam.Google Scholar - [33]Wasilkowski, G. W. (1985), Average Case Optimality,
*Journal of Complexity***1**, 107–117.MathSciNetMATHCrossRefGoogle Scholar - [34]Wasilkowski, G. W. (1985), Clock Synchronization Problem with Random Delays,
*Report*, Columbia University.Google Scholar - [35]Wasilkowski, G. W. and H. Woźniakowski (1984), Can Adaption Help on Average?,
*Numer. Math.*,**44**, 169 – 190.MathSciNetMATHCrossRefGoogle Scholar - [36]Werschulz, A. G. (1985a), Complexity of Differential and Integral Equations,
*Journal of Complexity*,**1**.Google Scholar - [37]Werschulz, A. G. (1985b), What is the Complexity of Ill-posed Problems?,
*Report*, Columbia University.Google Scholar - [38]Wiener, N. (1948),
*Cybernetics*, John Wiley and Sons, New York.Google Scholar - [39]Woźniakowski, H. (1985), A Survery of Information-Based Complexity,
*Journal of Complexity***1**, 11 – 44.MathSciNetMATHCrossRefGoogle Scholar