Skip to main content
SpringerLink
Log in

Big Data on the Rise?

Testing Monotonicity of Distributions

  • Conference paper
  • First Online:
Automata, Languages, and Programming (ICALP 2015)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 9134))

Included in the following conference series:

Abstract

The field of property testing of probability distributions, or distribution testing, aims to provide fast and (most likely) correct answers to questions pertaining to specific aspects of very large datasets. In this work, we consider a property of particular interest, monotonicity of distributions. We focus on the complexity of monotonicity testing across different models of access to the distributions [5, 7, 8, 20]; and obtain results in these new settings that differ significantly (and somewhat surprisingly) from the known bounds in the standard sampling model [1].

The full version of this work is available as [4].

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Batu, T., Kumar, R., Rubinfeld, R.: Sublinear algorithms for testing monotone and unimodal distributions. In: Proceedings of STOC, pp. 381–390. ACM, New York (2004)

    Google Scholar 

  2. Birgé, L.: On the risk of histograms for estimating decreasing densities. The Annals of Statistics 15(3), 1013–1022 (1987)

    Article  MATH  MathSciNet  Google Scholar 

  3. Canonne, C.L.: A Survey on Distribution Testing: Your Data is Big. But is it Blue? Electronic Colloquium on Computational Complexity (ECCC) 22, 63 (2015)

    Google Scholar 

  4. Canonne, C.L.: Big Data on the Rise: Testing monotonicity of distributions. ArXiV:abs/1501.06783 (2015)

  5. Canonne, C.L., Ron, D., Servedio, R.A.: Testing probability distributions using conditional samples. ArXiV:abs/1211.2664, November 2012

  6. Canonne, C.L., Ron, D., Servedio, R.A.: Testing equivalence between distributions using conditional samples. In: Proceedings of SODA, pp. 1174–1192. SIAM (2014), see also [5] (full version)

    Google Scholar 

  7. Canonne, C.L., Rubinfeld, R.: Testing probability distributions underlying aggregated data. In: Proceedings of ICALP, pp. 283–295 (2014)

    Google Scholar 

  8. Chakraborty, S., Fischer, E., Goldhirsh, Y., Matsliah, A.: On the power of conditional samples in distribution testing. In: Proceedings of ITCS, pp. 561–580. ACM, New York (2013)

    Google Scholar 

  9. Daskalakis, C., Diakonikolas, I., Servedio, R.A.: Learning \(k\)-modal distributions via testing. In: Proceedings of SODA, pp. 1371–1385. SIAM (2012)

    Google Scholar 

  10. Daskalakis, C., Diakonikolas, I., Servedio, R.A., Valiant, G., Valiant, P.: Testing \(k\)-modal distributions: Optimal algorithms via reductions. In: Proceedings of SODA, pp. 1833–1852. SIAM (2013)

    Google Scholar 

  11. Fischer, E.: The art of uninformed decisions: A primer to property testing. BEATCS 75, 97–126 (2001)

    MATH  Google Scholar 

  12. Goldreich, O. (ed.): Property Testing: Current Research and Surveys. LNCS, vol. 6390. Springer, Heidelberg (2010)

    Google Scholar 

  13. Goldreich, O., Ron, D.: On testing expansion in bounded-degree graphs. Technical report, TR00-020, Electronic Colloquium on Computational Complexity (ECCC) (2000)

    Google Scholar 

  14. Guha, S., McGregor, A., Venkatasubramanian, S.: Streaming and sublinear approximation of entropy and information distances. In: Proceedings of SODA, pp. 733–742. SIAM, Philadelphia (2006)

    Google Scholar 

  15. Paninski, L.: A coincidence-based test for uniformity given very sparsely sampled discrete data. IEEE Transactions on Information Theory 54(10), 4750–4755 (2008)

    Article  MathSciNet  Google Scholar 

  16. Pearson, K.: Contributions to the Mathematical Theory of Evolution. Philosophical Transactions of the Royal Society of London. (A.) 185, 71–110 (1894)

    Google Scholar 

  17. Ron, D.: Property Testing: A Learning Theory Perspective. Foundations and Trends in Machine Learning 1(3), 307–402 (2008)

    Article  Google Scholar 

  18. Ron, D.: Algorithmic and analysis techniques in property testing. Foundations and Trends in Theoretical Computer Science 5, 73–205 (2010)

    Article  MathSciNet  Google Scholar 

  19. Rubinfeld, R.: Taming Big Probability Distributions. XRDS 19(1), 24–28 (2012)

    Article  MathSciNet  Google Scholar 

  20. Rubinfeld, R., Servedio, R.A.: Testing monotone high-dimensional distributions. Random Structures and Algorithms 34(1), 24–44 (2009)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Columbia University, New York, USA

    Clément L. Canonne

Authors
  1. Clément L. Canonne
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Clément L. Canonne .

Editor information

Editors and Affiliations

  1. Reykjavik University, Reykjavik, Iceland

    Magnús M. Halldórsson

  2. Kyoto University, Kyoto, Japan

    Kazuo Iwama

  3. The University of Tokyo, Tokyo, Japan

    Naoki Kobayashi

  4. Technische Universiteit Eindhoven, Eindhoven, The Netherlands

    Bettina Speckmann

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Canonne, C.L. (2015). Big Data on the Rise?. In: Halldórsson, M., Iwama, K., Kobayashi, N., Speckmann, B. (eds) Automata, Languages, and Programming. ICALP 2015. Lecture Notes in Computer Science(), vol 9134. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-47672-7_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-47672-7_24

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-47671-0

  • Online ISBN: 978-3-662-47672-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation