Skip to main content
SpringerLink
Log in

Capacity of Higher-Dimensional Constrained Systems

  • Conference paper
Coding Theory and Applications

Part of the book series: CIM Series in Mathematical Sciences ((CIMSMS,volume 3))

Abstract

One-dimensional constrained systems, also known as discrete noiseless channels and sofic shifts, have a well-developed theory and have played an important role in applications such as modulation coding for data recording. Shannon found a closed form expression for the capacity of such systems in his seminal paper, and capacity has served as a benchmark for the efficiency of coding schemes as well as a guide for code construction. Advanced data recording technologies, such as holographic recording, may require higher-dimensional constrained coding. However, in higher dimensions, there is no known general closed form expression for capacity. In fact, the exact capacity is known for only a few higher-dimensional constrained systems. Nevertheless, there have been many good methods for efficiently approximating capacity for some classes of constrained systems. These include transfer matrix and spatial mixing methods. In this article, we will survey progress on these and other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adler, R., Coppersmith, D., Hassner, M.: Algorithms for sliding block codes – an application of symbolic dynamics to information theory. IEEE Trans. Inf. Theory 29, 5–22 (1983)

    Article  MathSciNet  MATH  Google Scholar 

  2. Ashley, J., et al.: Holographic data storage. IBM J. Res. Dev. 44, 341–366 (2000)

    Article  Google Scholar 

  3. Baxter, R.: Hard hexagons: exact solution. Physics A 13, 1023–1030 (1980)

    Article  Google Scholar 

  4. Berger, R.: The undecidability of the domino problem. Mem. Am. Math. Soc. 66, 1–72 (1966)

    Google Scholar 

  5. Blahut, R., Weeks, W.: The capacity and coding gain of certain checkerboard codes. IEEE Trans. Inf. Theory 44, 1193–1203 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  6. Calkin, N., Wilf, H.: The number of independent sets in a grid graph. SIAM J. Discret. Math. 11, 54–60 (1998)

    Article  MathSciNet  MATH  Google Scholar 

  7. Censor, K., Etzion, T.: The positive capacity region of two-dimensional run-length-constrained channels. IEEE Trans. Inf. Theory 52, 5128–5140 (2006)

    Article  MathSciNet  Google Scholar 

  8. Chan, Y., Rechnitzer, A.: Accurate lower bounds on two-dimensional constraint capacities from corner transfer matrices. IEEE Trans. Inf. Theory 60, 3845–3858 (2014)

    Article  MathSciNet  Google Scholar 

  9. Engel, K.: On the Fibonacci number of an m by n lattice. Fibonacci Q. 28, 72–78 (1990)

    MathSciNet  MATH  Google Scholar 

  10. Fagnani, F., Zampieri, S.: Minimal and systematic convolutional codes over finite Abelian groups. Linear Algebra Appl. 378, 31–59 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  11. Forchhammer, S., Justesen, J.: Entropy bounds for constrained two-dimensional random fields. IEEE Trans. Inf. Theory 45, 118–127 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  12. Fornasini, E., Valcher, M.: Algebraic aspects of two-dimensional convolutional codes. IEEE Trans. Inf. Theory 40, 1068–1082 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  13. Fouldadgar, A., Someone, O., Erkip, E.: Constrained codes for joint energy and information transfer with receiver energy utilization requirements. In: Proceedings of IEEE International Symposium on Information Theory, Honolulu, pp. 991–995 (2014)

    Google Scholar 

  14. Friedland, S.: On the entropy of Zd subshifts of finite type. Linear Algebra Appl. 252, 199–220 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  15. Friedland, S., Lundow, P., Markstrom, K.: The 1-vertex transfer matrix and accurate estimation of channel capacity. IEEE Trans. Inf. Theory 56, 3692–3699 (2010)

    Article  MathSciNet  Google Scholar 

  16. Gamarnik, D., Katz, D.: Sequential cavity method for computing free energy and surface pressure. J. Stat. Phys. 137, 205–232 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  17. Golin, M.J., Yong, X., Zhang, Y., Sheng, L.: New upper and lower bounds on the channel capacity of read/write isolated memory. Discret. Appl. Math. 140, 35–48 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  18. Hochman, M., Meyerovitch, T.: A characterization of the entropies of multidimensional shifts of finite type. Ann. Math. 171(3), 2011–2038 (2012)

    Article  MathSciNet  Google Scholar 

  19. Karabed, R., Marcus, B.: Sliding-block coding for input-restricted channels. IEEE Trans. Inf. Theory 34, 2–26 (1988)

    Article  MathSciNet  Google Scholar 

  20. Kastelyn, P.: The statistics of dimers on a lattice. Physica A 27, 1209–1225 (1961)

    Google Scholar 

  21. Kato, A., Zeger, K.: On the capacity of two-dimensional run length constrained channels. IEEE Trans. Inf. Theory 45, 1527–1540 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  22. Kitchens, B.: Multidimensional convolutional codes. SIAM J. Discret. Math. 15, 367–381 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  23. Lieb, E.: Residual entropy of square ice. Phys. Rev. 162, 162–172 (1967)

    Article  Google Scholar 

  24. Lind, D.: The entropies of topological Markov shifts and a related class of algebraic integers. Ergod. Theory Dyn. Syst. 4, 283–300 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  25. Lind, D., Marcus, B.: An Introduction to Symbolic Dynamics and Coding. Cambridge University Press, Cambridge (1995, reprinted 1999)

    Google Scholar 

  26. Lind, D., Schmidt, K., Ward, T.: Mahler measure and entropy for commuting automorphisms of compact groups. Invent. Math. 101, 593–629 (1990)

    Article  MathSciNet  MATH  Google Scholar 

  27. Louidor, E., Marcus, B.: Improved lower bounds on capacities of symmetric 2-dimensional constraints using rayleigh quotients. IEEE Trans. Inf. Theory 56, 1624–1639 (2010)

    Article  MathSciNet  Google Scholar 

  28. Louidor, E., Marcus, B., Pavlov, R.: Independence entropy of Zd shift spaces. Acta Appl. Math. 126, 297–317 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  29. Marcus, B., Pavlov, R.: Computing bounds on entropy of \(\mathbb{Z}^{d}\) stationary Markov random fields. SIAM J. Discret. Math. 27, 1544–1558 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  30. Marcus, B., Pavlov, R.: An integral representation for topological pressure in terms of conditional probabilities (2013, to appear). Isr. J. Math. arXiv:1309.1873v2

  31. Marcus, B., Roth, R., Siegel, P.: Constrained systems and coding for recording channels. In: Pless, V.S., Huffman, W.C. (eds.) Handbook of Coding Theory, vol. II, chapter 20. Elsevier Press, Amsterdam/New York (1998)

    Google Scholar 

  32. Markley, N., Paul, M.: Maximal measures and entropy for \(\mathbb{Z}^{\nu }\) subshifts of finite type. In: Devaney, R., Nitecki, Z. (eds.) Classical Mechanics and Dynamical Systems. Dekker Notes, vol. 70, pp. 135–157. Dekker, New York (1981)

    Google Scholar 

  33. Meyerovitch, T., Pavlov, R.: Entropy and measures of maximal entropy for axial powers of subshifts. Proc. Lond. Math. Soc 109(4), 921–945 (2014)

    Article  MathSciNet  Google Scholar 

  34. Ordentlich, E., Roth, R.: Two-dimensional weight-constrained codes through enumeration bounds. IEEE Trans. Inf. Theory 46, 1292–1301 (2000)

    Article  MathSciNet  MATH  Google Scholar 

  35. Pavlov, R.: Approximating the hard square entropy constant with probabilistic methods. Ann. Probab. 40, 2362–2399 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  36. Poo, T.L., Chaichanavong, P., Marcus, B.: Trade-off functions for constrained systems with unconstrained positions. IEEE Trans. Inf. Theory 52, 1425–1449 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  37. Roth, R., Siegel, P., Wolf, J.: Efficient coding scheme for the hard-square model. IEEE Trans. Inf. Theory 47, 1166–1176 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  38. Schwartz, M., Vardy, A.: New bounds on the capacity of multidimensional run-length constraints. IEEE Trans. Inf. Theory 57, 4373–4382 (2011)

    Article  MathSciNet  Google Scholar 

  39. Shannon, C.: A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423, 623–656 (1948)

    Article  MathSciNet  Google Scholar 

  40. Simon, B.: The Statistical Mechanics of Lattice Gases. Princeton University Press, Princeton (1993)

    Book  MATH  Google Scholar 

  41. Tal, I., Roth, R.: Bounds on the rate of 2-d bit-stuffing encoders. IEEE Trans. Inf. Theory 56, 2561–2567 (2010)

    Article  MathSciNet  Google Scholar 

  42. Wang, Y.: System for encoding and decoding data in machine readable graphic form. US Patent 5,243,655 (1993)

    Google Scholar 

  43. Wang, Y., Yin, Y., Zhong, S.: Approximate capacities of two-dimensional codes by spatial mixing. In: Proceedings of IEEE International Symposium on Information Theory, Honolulu, pp. 1061–1065 (2014)

    Google Scholar 

Download references

Acknowledgements

This article is a summary of the lecture I gave, by the same title, at the 4th International Castle Conference on Coding Theory held in Palmela, Portugal. It is a pleasure to thank the organizers for putting together such a stimulating conference, cutting across many topics of both theoretical and practical importance.

Author information

Authors and Affiliations

  1. Department of Mathematics, University of British Columbia, Vancouver, BC, Canada

    Brian Marcus

Authors
  1. Brian Marcus
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Brian Marcus .

Editor information

Editors and Affiliations

  1. Department of Mathematics, University of Aveiro, Aveiro, Portugal

    Raquel Pinto

  2. Department of Electrical and Computer Engineering, University of Porto, Porto, Portugal

    Paula Rocha Malonek

  3. Department of Mathematics, University of Aveiro, Aveiro, Portugal

    Paolo Vettori

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Marcus, B. (2015). Capacity of Higher-Dimensional Constrained Systems. In: Pinto, R., Rocha Malonek, P., Vettori, P. (eds) Coding Theory and Applications. CIM Series in Mathematical Sciences, vol 3. Springer, Cham. https://doi.org/10.1007/978-3-319-17296-5_1

Download citation

Publish with us

Policies and ethics

Search

Navigation