Skip to main content
SpringerLink
Log in

An improved analysis of least squares superposition codes with bernoulli dictionary

  • Original Paper
  • Information Theory and Statistics
  • Published:
Japanese Journal of Statistics and Data Science Aims and scope Submit manuscript

Abstract

For the additive white Gaussian noise channel with average power constraint, sparse superposition codes (or sparse regression codes), proposed by Barron and Joseph in 2010, achieve the capacity. While the codewords of the original sparse superposition codes are made with a dictionary matrix drawn from a Gaussian distribution, we consider the case that it is drawn from a Bernoulli distribution. We show an improved upper bound on its block error probability with least squares decoding, which is fairly simplified and tighter bound than our previous result in 2014.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

References

  • Arikan, E. (2009). Channel polarization. IEEE Transactions Information Theory, 55(7), 3051–3073.

    Article  MathSciNet  Google Scholar 

  • Barbier, J., & Krzakala, F. (2014). Replica analysis and approximate message passing decoder for superposition codes. In Proc. 2014 IEEE int. symp. inf. theory, Honolulu, HI, USA, June 29–July 4, pp. 1494–1498.

  • Barbier, J., & Krzakala, F. (2017). Approximate message-passing decoder and capacity-achieving sparse superposition codes. IEEE Transactions Information Theory, 63(8), 4894–4927.

    Article  MathSciNet  Google Scholar 

  • Barron, A. R., & Cho, S. (2012). High-rate sparse superposition codes with iteratively optimal estimates. In Proc. 2012 IEEE int. symp. inf. theory, Boston, MA, USA, July 1–6, pp. 120–124.

  • Barron, A. R., & Joseph, A. (2010a). Least squares superposition coding of moderate dictionary size, reliable at rates up to channel capacity. In Proc. 2010 IEEE int. symp. inf. theory, Austin, Texas, USA, June 13–18, pp. 275–279.

  • Barron, A. R., & Joseph, A. (2010b). Towards fast reliable communication at rates near capacity with Gaussian noise. In Proc. 2010 IEEE. Int. symp. inf. theory, Austin, Texas, USA, June 13–18, pp. 315–319.

  • Berrou, C., Glavieux, A., & Thitimajshima, P. (1993). Near Shannon limit error-correcting coding: turbo codes. In Proc. Int. Conf. Commun, Geneva, Switzerland, May, pp. 1064–1070.

  • Bourbaki, N. (1986). Functions of a real variable, (Japanese translation). Tokyo: TokyoTosho.

    Google Scholar 

  • Cho, S., & Barron, A. R. (2013). Approximate iterative Bayes optimal estimates for high-rate sparse superposition codes. In The sixth workshop on information-theoretic methods in science and engineering.

  • Cover, T. M., & Thomas, J. A. (2006). Elements of information theory. New York: Wiley-Interscience.

    MATH  Google Scholar 

  • Joseph, A., & Barron, A. R. (2012). Least squares superposition codes of moderate dictionary size are reliable at rates up to capacity. IEEE Transactions Information Theory, 58(5), 2541–2557.

    Article  MathSciNet  Google Scholar 

  • Joseph, A., & Barron, A. R. (2014). Fast sparse superposition codes have near exponential error probability for \(R < mathcal {C}\). IEEE Transactions Information Theory, 60(2), 919–942.

    Article  MathSciNet  Google Scholar 

  • Kudekar, S., Richardson, T. J., & Urbanke, R. (2011). Threshold saturation via sparitial coupling: why convolutional LDPC ensembles perform so well over the BEC. IEEE Transactions Information Theory, 57(2), 803–834.

    Article  MathSciNet  Google Scholar 

  • Rush, C., Greig, A., & Venkataramanan, R. (2017). Capacity achieving sparse regression codes via approximate message passing decoding. IEEE Transactions Information Theory, 63(3), 1476–1500.

    Article  MathSciNet  Google Scholar 

  • Rush, C., & Venkataramanan, R. (2018). Finite sample analysis of approximate message passing. IEEE Transactions Information Theory, 64(11), 7264–7286.

    Article  MathSciNet  Google Scholar 

  • Reed, I. S., & Solomon, G. (1960). Polynomial codes over certain finite fields. Journal of SIAM, 8, 300–304.

    MathSciNet  MATH  Google Scholar 

  • Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379–423.

    Article  MathSciNet  Google Scholar 

  • Takeishi, Y., Kawakita, M., & Takeuchi, J. (2014). Least squares superposition codes with Bernoulli dictionary are still reliable at rates up to capacity. Bell System Technical Journal, 60(5), 2737–2750.

    MathSciNet  MATH  Google Scholar 

  • Venkataramanan, R., Tatikonda, S., & Barron, A. R. (2019). Sparse regression codes. Foundations and Trends in Communications and Information Theory, 15(1–2), 85–283. https://doi.org/10.1561/0100000092.

    Article  MATH  Google Scholar 

  • Osada, N. (2008). A story: numerical analysis part 3; the Euler-MacLaurin formula. Rikei Heno Suugaku, July 2008. (in Japanese) http://www.lab.twcu.ac.jp/~osada/rikei/rikei2008-7.pdf. Accessed 8 June 2019

Download references

Acknowledgements

The authors thank Professor Andrew R. Barron for his valuable comments. This research was partially supported by JSPS KAKENHI Grant numbers JP16K12496 and JP18H03291.

Author information

Authors and Affiliations

  1. Graduate School of Information Science and Electrical Engineering, Kyushu University, Motooka 744, Nishi-ku, Fukuoka, Fukuoka, 819-0395, Japan

    Yoshinari Takeishi

  2. Mitsubishi Electric Information Network Corporation, 4-6-8 Shibaura Minato-Ku, Tokyo, 108-0023, Japan

    Yoshinari Takeishi

  3. Faculty of Information Science and Electrical Engineering, Kyushu University, Motooka 744, Nishi-ku, Fukuoka, Fukuoka, 819-0395, Japan

    Jun’ichi Takeuchi

Authors
  1. Yoshinari Takeishi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun’ichi Takeuchi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jun’ichi Takeuchi.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This material was presented in part at IEEE International Symposium on Information Theory 2016, in Barcelona, Spain.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Takeishi, Y., Takeuchi, J. An improved analysis of least squares superposition codes with bernoulli dictionary. Jpn J Stat Data Sci 2, 591–613 (2019). https://doi.org/10.1007/s42081-019-00057-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42081-019-00057-9

Keywords

Search

Navigation