Skip to main content
SpringerLink
Log in

Applications of Kolmogorov Complexity in the Theory of Computation

  • Chapter
Complexity Theory Retrospective

Abstract

This exposition gives a brief introduction to the main ideas of Kolmogorov complexity that have been useful in the area of computational complexity theory. We demonstrate how these ideas can actually be applied and provide a detailed survey of the abundant applications of this elegant notion in computational complexity theory. (Note : Preliminary versions of parts of this paper appeared in: Proc. 3rd IEEE Structure in Complexity Theory Conference, Computer Society Press, Washington D.C., 1988, pp. 80–102; and Uspekhi Mat. Nauk, 43:6 (1988), pp. 129–166 (in Russian).)

The work of the first author was partially performed at Aiken Computation Lab, Harvard University and supported by NSF Grant DCR-8606366, Office of Naval Research Grant N00014-85-k-0445, Army Research Office Grant DAAL03-86-K-0171, and NSERC Operating Grant OGP0036747.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. S.O. Aanderaa. On k-tape versus (k-l)-tape real-time computation. In R.M. Karp, editor, Complexity of Computation, pages 75–96, American Math. Society, Providence, R.I., 1974.

    Google Scholar 

  2. L. Adleman. Time, space, and randomness. Technical Report MIT/LCS/79/TM-131, Massachusetts Institute of Technology, Laboratory for Computer Science, March 1979.

    Google Scholar 

  3. E. Allender. Invertible Functions. PhD thesis, Georgia Institute of Technology, 1985.

    Google Scholar 

  4. E. Allender. Some consequences of the existence of pseudorandom generators. In Proc. 19th ACM Symposium on Theory of Computation, pages 151–159, 1987.

    Google Scholar 

  5. E.A. Allender and R.S. Rubinstein. P-printable sets. SIAM J. on Computing, 17, 1988.

    Google Scholar 

  6. E. Allender and O. Watanabe. Kolmogorov complexity and degrees of tally sets. In Proceedings 3rd Conference on Structure in Complexity Theory, pages 102–111, 1988.

    Chapter  Google Scholar 

  7. Y.M. Barzdin’. Complexity of programs to determine whether natural numbers not greater than n belong to a recursively enumerable set. Soviet Math. Dokl., 9: 1251–1254, 1968.

    Google Scholar 

  8. J. Balcazar and R. Book. On generalized Kolmogorov complexity. In Proc. 1st Structure in Complexity Theory Conference, Lecture Notes in Computer Science, vol 223, pages 334–340, Springer Verlag, Berlin, 1986.

    Google Scholar 

  9. A. Borodin and S. Cook. A time-space tradeoff for sorting on a general sequential model of computation. In 12th ACM Symp. on Theory of Computing, 1980.

    Google Scholar 

  10. C.H. Bennett. Dissipation, information, computational complexity and the definition of organization. In D. Pines, editor, Emerging Syntheses in Science, Addison-Wesley, Reading, MA, 1987. (Proceedings of the Founding Workshops of the Santa Fe Institute, 1985, pages 297–313.).

    Google Scholar 

  11. C.H. Bennett. On the logical “depth” of sequences and their reducibilities to random sequences. In R. Herken, editor, The Universal Turing Machine - A Half-Century Survey, pages 227–258, Oxford University Press, 1988.

    Google Scholar 

  12. A. Borodin, M.J. Fischer, D.G. Kirkpatrick, N.A. Lynch, and M. Tompa. A time-space tradeoff for sorting and related non-oblivious computations. In 20th IEEE Symposium on Foundations of Computer Science, pages 319–328, 1979.

    Google Scholar 

  13. L. Berman and J. Hartmanis. On isomorphisms and density of NP and other complete sets. SIAM J. on Computing, 6: 305–327, 1977.

    Article  MathSciNet  MATH  Google Scholar 

  14. R. Book, P. Orponen, D. Russo, and O. Watanabe. Lowness properties of sets in the exponential-time hierarchy. SIAM J. Computing, 17: 504–516, 1988.

    Article  MathSciNet  MATH  Google Scholar 

  15. J. Cai and L. Hemachandra. Exact counting is as easy as approximate counting. Technical Report 86-761, Computer Science Department, Cornell University, Ithaca, N.Y., 1986.

    Google Scholar 

  16. G.J. Chaitin. On the length of programs for computing finite binary sequences: statistical considerations. J. Assoc. Comp. Mach., 16: 145–159, 1969.

    Article  MathSciNet  MATH  Google Scholar 

  17. G.J. Chaitin. Algorithmic Information Theory. IBM J. Res. Dev., 21: 350–359, 1977.

    Article  MathSciNet  MATH  Google Scholar 

  18. M. Chrobak. Hierarchies of one-way multihead automata languages. Theoretical Computer Science, 48:153–81, 1986. (also in 12th ICALP, Springer LNCS 194: 101–110, 1985 ).

    Article  MathSciNet  Google Scholar 

  19. M. Chrobak and M. Li. k+1 heads are better than k for PDAs. In Proc. 27th IEEE Symposium on Foundations of Computer Science, pages 361–367, 1986.

    Google Scholar 

  20. R.R. Cuykendall. Kolmogorov information and VLSI lower bounds. PhD thesis, University of California, Los Angeles, Dec. 1984.

    Google Scholar 

  21. J. Carter and M. Wegman. Universal classes of hashing functions. J. Comp. Syst. Sciences, 18: 143–154, 1979.

    Article  MathSciNet  MATH  Google Scholar 

  22. R.P. Daley. On the inference of optimal descriptions. Theoretical Computer Science, 4: 301–309, 1977.

    Article  MathSciNet  MATH  Google Scholar 

  23. P. Duris, Z. Galil, W. Paul, and R. Reischuk. Two nonlinear lower bounds for on-line computations. Information and Control, 60: 1–11, 1984.

    Article  MathSciNet  MATH  Google Scholar 

  24. M. Dietzfelbinger. Lower bounds on computation time for various models in computational complexity theory. PhD thesis, University of Illinois at Chicago, 1987.

    Google Scholar 

  25. P. Erdös and J. Spencer. Probabilistic methods in combinatorics. Academic Press, New York, 1974.

    MATH  Google Scholar 

  26. R. Floyd. Review 14. Comput. Rev., 9: 280, 1968.

    MathSciNet  Google Scholar 

  27. H. Gallaire. Recognition time of context-free languages by online Turing machines. Information and Control, 15: 288–295, 1969.

    Article  MathSciNet  MATH  Google Scholar 

  28. Z. Galil, R. Kannan, and E. Szemeredi. On nontrivial separators for k-page graphs and simulations by nondeterministic one-tape Turing machines. In Proc. 18th ACM Symp. on Theory of Computing, pages 39–49, 1986.

    Google Scholar 

  29. M. Gereb and M. Li. Lower bounds in string matching. In preparation.

    Google Scholar 

  30. Z. Galil and J. Seiferas. Time-space optimal matching. In Proc. 13th ACM Symposium on Theory of Computing, 1981.

    Google Scholar 

  31. Y. Goldberg and M. Sipser. Compression and Ranking. In Proc. 17th Assoc. Comp. Mach. Symposium on Theory of Computing, pages 440–448, 1985.

    Google Scholar 

  32. J. Hartmanis. Generalized Kolmogorov complexity and the structure of feasible computations. In Proc. 24th IEEE Symposium on Foundations of Computer Science, pages 439–445, 1983.

    Google Scholar 

  33. L. Hemachandra. Can P and NP manufacture randomness? Technical Report 86–795, Computer Science Department, Cornell University, Ithaca, N.Y., December 1986.

    Google Scholar 

  34. L. Hemachandra. On ranking. In Proc. 2nd Ann. IEEE Conference on Structure in Complexity Theory, pages 103–117, 1987.

    Google Scholar 

  35. J. Hartmanis and L. Hemachandra. On sparse oracles separating feasible complexity classes. In Proc. 3rd Symposium on Theoretical Aspects of Computer Science, Lecture Notes in Computer Science, vol. 210, pages 321–333, Springer Verlag, Berlin, 1986.

    Google Scholar 

  36. M.A. Harrison and O.H. Ibarra. Multi-head and multi-tape pushdown automata. Information and Control, 13: 433–470, 1968.

    Article  MathSciNet  MATH  Google Scholar 

  37. J.E. Hopcroft and J.D. Ullman. Introduction to Automata Theory, Languages, and Computation. Addison-Wesley, 1979.

    MATH  Google Scholar 

  38. D.T. Huynh. Non-uniform complexity and the randomness of certian complete languages. Technical Report 85-34, Iowa State University, December 1985.

    Google Scholar 

  39. D.T. Huynh. Resource-bounded Kolmogorov complexity of hard languages. In Proc. 1st Structure in Complexity Theory Conference, Lecture Notes in Computer Science, vol. 223, pages 184–195, Springer Verlag, Berlin, 1986.

    Google Scholar 

  40. F. Meyer auf der Heide and A. Wigderson. The complexity of parallel sorting. In Proc. 17th ACM Symp. on Theory of computing, pages 532–540, 1985.

    Google Scholar 

  41. O.H. Ibarra and C.E. Kim. On 3-head versus 2 head finite automata. Acta Infomatica, 4: 193–200, 1975.

    Article  MathSciNet  MATH  Google Scholar 

  42. Y. Itkis and L.A. Levin. Power of fast VLSI models is insensitive to wires’ thinnes. In Proc. 30th IEEE Symposium on Foundations of Computing, 1989.

    Google Scholar 

  43. IM] A. Israeli and S. Moran. Private communication.

    Google Scholar 

  44. Ker-I Ko. Resource-bounded program-size complexity and pseudorandom sequences. Technical Report, Department of computer science, University of Houston, 1983.

    Google Scholar 

  45. A.N. Kolmogorov. Three approaches to the quantitative definition of information. Problems in Information Transmission, 1: 1–7, 1965.

    MathSciNet  Google Scholar 

  46. L.A. Levin. Universal search problems. Problems in Information Transmission, 9: 265–266, 1973.

    Google Scholar 

  47. L.A. Levin. Laws of information conservation (non-growth) and aspects of the foundation of probability theory. Problems in Information Transmission, 10: 206–210, 1974.

    Google Scholar 

  48. L.A. Levin. Do chips need wires? Manuscript/NSF proposal MCS-8304498, 1983. Computer Science Department, Boston University.

    Google Scholar 

  49. L.A. Levin. Randomness conservation inequalities; information and independence in mathematical theories. Information and Control, 61: 15–37, 1984.

    Article  MathSciNet  MATH  Google Scholar 

  50. M. Li. Lower Bounds in Computational Complexity, Report TR-85-663. PhD thesis, Computer Science Department, Cornell University, March 1985.

    Google Scholar 

  51. M. Li. Simulating two pushdowns by one tape in 0(n**1.5 (log n)**0.5) time. In Proc. 26th IEEE Symposium on the Foundations of Computer Science, 1985.

    Google Scholar 

  52. M. Li, L. Longpré, and P.M.B. Vitányi. On the power of the queue. In Structure in Complexity Theory, Lecture Notes in Computer Science, volume 223, pages 219–233, Springer Verlag, Berlin, 1986.

    Google Scholar 

  53. L. Longpré. Resource bounded Kolmogorov complexity, a link between computational complexity and information theory, Techn. Rept. TR-86-776. PhD thesis, Computer Science Department, Cornell University, 1986.

    Google Scholar 

  54. M. Loui. Optimal dynamic embedding of trees into arrays. SIAM J. Computing, 12: 463–472, 1983.

    Article  MathSciNet  MATH  Google Scholar 

  55. R. Lipton and R. Sedgewick. Lower bounds for VLSI. In 13th ACM Symp. on Theory of computing, pages 300–307, 1981.

    Google Scholar 

  56. M. Li and P.M.B. Vitányi. An Introduction to Kolmogorov Complexity and Its Applications. (To appear, Addison-Wesley).

    Google Scholar 

  57. M. Li and P.M.B. Vitânyi. Kolmogorov Complexity and its Applications. J. van Leeuwen, editor, Handbook of Theoretical Computer Science, North-Holland. To appear.

    Google Scholar 

  58. M. Li and P.M.B. Vitányi. Kolmogorovskaya slozhnost’ dvadsat’ let spustia. Uspekhi Mat Nauk, 43:6:129–166, 1988. (In Russian; = Russian Mathematical Surveys).

    Google Scholar 

  59. M. Li and P.M.B. Vitányi. Tape versus queue and stacks: the lower bounds. Information and Computation, 78: 56–85, 1988.

    Article  MathSciNet  MATH  Google Scholar 

  60. M. Li and P.M.B. Vitányi. Inductive reasoning and Kolmogorov complexity. In Proc. 4th IEEE Conference on Structure in Complexity Theory, pages 165–185, 1989.

    Google Scholar 

  61. M. Li and P.M.B. Vitányi. A new approach to formal language theory by Kolmogorov complexity. In 16th International Conference on Automata, Languages and Programming, Lecture Notes in Computer Science, vol. 372, pages 488–505, Springer Verlag, Berlin, 1989.

    Chapter  Google Scholar 

  62. M. Li and Y. Yesha. New lower bounds for parallel computation. In Proc. 18th Assoc. Comp. Mach. Symposium on Theory of Computing, pages 177–187, 1986.

    Google Scholar 

  63. M. Li and Y. Yesha. String-matching cannot be done by 2-head 1-way deterministic finite automata. Information Processing Letters, 22: 231–235, 1986.

    Article  MathSciNet  MATH  Google Scholar 

  64. W. Maass. Combinatorial lower bound arguments for deterministic and nondeterministic Turing machines. Trans. Amer. Math. Soc., 292: 675–693, 1985.

    Article  MathSciNet  MATH  Google Scholar 

  65. H.G. Mairson. The program complexity of searching a table. In Proceedings 24th IEEE Symposium on Fundations of Computer Science, pages 40–47, 1983.

    Google Scholar 

  66. K. Mehlhorn. On the program-size of perfect and universal hash functions. In Proc. 23rd Ann. IEEE Symposium on Foundations of Computer Science, pages 170–175, 1982.

    Google Scholar 

  67. S. Miyano. A hierarchy theorem for multihead stack-counter automata. Acta Informatica, 17: 63–67, 1982.

    Article  MathSciNet  MATH  Google Scholar 

  68. S. Miyano. Remarks on multihead pushdown automata and multihead stack automata. Journal of Computer and System Sciences, 27: 116–124, 1983.

    Article  MathSciNet  MATH  Google Scholar 

  69. W. Maass and G. Schnitger. An optimal lower bound for Turing machines with one work tape and a two-way input tape. In Structure in Complexity Theory, Lecture Notes in Computer Science, volume 223, pages 249–264, Springer Verlag, Berlin, 1986.

    Google Scholar 

  70. W. Maass, G. Schnitger, and E. Szemeredi. Two tapes are better than one for off-line Turing machines. In Proc. 19th ACM Symposium on Theory of Computing, pages 94–100, 1987.

    Google Scholar 

  71. B. K. Natarajan. 1988. Personal communication.

    Google Scholar 

  72. C.G. Nelson. One-way automata on bounded languages. Technical Report TR14-76, Harvard University, July 1976.

    Google Scholar 

  73. I. Parberry. A complexity theory of parallel computation. PhD thesis, Warwick University, 1984.

    Google Scholar 

  74. W. Paul. Kolmogorov’s complexity and lower bounds. In Proc. 2nd International Conference on Fundamentals of Computation Theory, September 1979.

    Google Scholar 

  75. W.J. Paul. On-line simulation of k-f 1 tapes by k tapes requires nonlinear time. Information and Control, 1–8, 1982.

    Google Scholar 

  76. W. Paul. On heads versus tapes. Theoretical Computer Science, 28: 1–12, 1984.

    Article  MathSciNet  MATH  Google Scholar 

  77. G. Peterson. Succinct representations, random strings and complexity classes. In Proc. 21st IEEE Symposium on Foundations of Computer Science, pages 86–95, 1980.

    Google Scholar 

  78. R. Paturi and J. Simon. Lower bounds on the time of probabilistic on-line simulations. In Proc. 24th Ann. IEEE Symposium on Foundations of Computer Science, page 343, 1983.

    Google Scholar 

  79. R. Paturi, J. Simon, R.E. Newman-Wolfe, and J. Seiferas. Milking the Aanderaa argument. April 1988. University of Rochester, Comp. Sci. Dept., Rochester, N.Y.

    Google Scholar 

  80. W.J. Paul, J.I. Seiferas, and J. Simon. An information theoretic approach to time bounds for on-line computation. J. Computer and System Sciences, 23: 108–126, 1981.

    Article  MathSciNet  MATH  Google Scholar 

  81. H. Rogers, Jr. Theory of Recursive Functions and effective computability. McGraw-Hill, New York, 1967.

    Google Scholar 

  82. A. Rosenberg. Nonwriting extensions of finite automata. PhD thesis, Harvard University, 1965.

    Google Scholar 

  83. A. Rosenberg. On multihead finite automata. IBM J. Res. Develop., 10: 388–394, 1966.

    Article  MathSciNet  MATH  Google Scholar 

  84. S. Reisch and G. Schnitger. Three applications of Kolmogorov-complexity. In Proc. 23rd Ann. IEEE Symposium on Founda-tions of Computer Science, pages 45–52, 1982.

    Google Scholar 

  85. J. Seiferas. The symmetry of information, and An application of the symmetry of information. Notes, August 1985. Computer Science Dept, University of Rochester.

    Google Scholar 

  86. J. Seiferas. A simplified lower bound for context-free-language recognition. Information and Control, 69: 255–260, 1986.

    Article  MathSciNet  MATH  Google Scholar 

  87. M. Sipser. A complexity theoretic approach to randomness. In Proceedings 15th Assoc. Comp. Mach. Symposium on Theory of Computing, pages 330–335, 1983.

    Google Scholar 

  88. R. J. Solomonoff. A formal theory of inductive inference, Part 1 and Part 2. Information and Control, 7: 1–22, 224–254, 1964.

    Article  MathSciNet  MATH  Google Scholar 

  89. J. Storer. Data Compression: Method and Theory, chapter 6. Computer Science Press, Rockville, MD, 1988.

    Google Scholar 

  90. I.H. Sudborough. Computation by multi-head writing finite automata. PhD thesis, Pennsylvania State University, University Park, 1974.

    Google Scholar 

  91. I.H. Sudborough. One-way multihead writing finite automata. Information and Control, 30:15–20, 1976. (Also FOCS 1971 ).

    Google Scholar 

  92. C.D. Thompson. Area-Time complexity for VLSI. In Proc. 11th ACM Symp. on Theory of Computing, pages 81–88, 1979.

    Google Scholar 

  93. L. Valiant and G. Brebner. Universal Schemes for parallel com-munication. In Proc. 13th ACM Symp. on Theory of Computing, pages 263–277, 1981.

    Google Scholar 

  94. P.M.B. Vitányi. On the simulation of many storage heads by one. Theoretical Computer Science, 34:157–168, 1984. (Also, ICALP ’83.).

    Google Scholar 

  95. P.M.B. Vitányi. On two-tape real-time computation and queues. Journal of Computer and System Sciences, 29: 303–311, 1984.

    Article  MathSciNet  MATH  Google Scholar 

  96. P.M.B. Vitanyi. An N**1.618 lower bound on the time to simulate one queue or two pushdown stores by one tape. Information Processing Letters, 21: 147–152, 1985.

    Article  MathSciNet  MATH  Google Scholar 

  97. P.M.B. Vitanyi. Square time is optimal for the simulation of a pushdown store by an oblivious one-head tape unit. Information Processing Letters, 21: 87–91, 1985.

    Article  MathSciNet  MATH  Google Scholar 

  98. U. Vazirani and V. Vazirani. A natural encoding scheme proved probabilistic polynomial complete. In Proc. 23rd IEEE Symp. on Foundations of Computer Science, pages 40–44, 1982.

    Google Scholar 

  99. O. Watanabe. Comparison of polynomial time completeness notions. Theoretical Computer Science, 53, 1987.

    Google Scholar 

  100. A. Yao. Theory and application of trapdoor functions. In Proceedings 23rd IEEE Symposium on Foundations of Computer Science, pages 80–91, 1982.

    Google Scholar 

  101. Y. Yesha. Time-space tradeoffs for matrix multiplication and discrete Fourier transform on any general random access computer. J. Comput. Syst. Sciences, 29: 183–197, 1984.

    Article  MathSciNet  MATH  Google Scholar 

  102. A.C.-C. Yao and R.L. Rivest. k+1 heads are better than k. J. Assoc. Comput. Mach.25:337–340, 1978. (also see Proc. 17th FOCS, pages 67–70, 1976 ).

    Article  MathSciNet  MATH  Google Scholar 

  103. A.K. Zvonkin and L.A. Levin. The complexity of finite objects and the development of the concepts of information and ran-domness by means of the Theory of Algorithms. Russ. Math. Surv., 25: 83–124, 1970.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Author notes

  1. Ming Li

    Present address: Computer Science Department, University of Waterloo, Waterloo, Ontario, Canada, N2L 3G1

Authors and Affiliations

  1. Department of Computer Science, York University, Ontario, M3J 1P3, Canada

    Ming Li

  2. Centrum voor Wiskunde en Informatica and Faculteit Wiskunde en Informatica, Universiteit van Amsterdam, Kruislaan 413, 1098 SJ, Amsterdam, The Netherlands

    Paul M. B. Vitányi

Authors
  1. Ming Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul M. B. Vitányi
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. College of Computer Science, Northeastern University, Boston, MA, 02115, USA

    Alan L. Selman

Rights and permissions

Reprints and permissions

Copyright information

© 1990 Springer-Verlag New York Inc.

About this chapter

Cite this chapter

Li, M., Vitányi, P.M.B. (1990). Applications of Kolmogorov Complexity in the Theory of Computation. In: Selman, A.L. (eds) Complexity Theory Retrospective. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-4478-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-1-4612-4478-3_8

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4612-8793-3

  • Online ISBN: 978-1-4612-4478-3

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation