Advertisement

Dynamic Table: A Scalable Storage Structure in the Cloud

  • Hanchen Su
  • Hongyan Li
  • Xu Cheng
  • Zhiqiang Liu
Conference paper
  • 1.3k Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7901)

Abstract

Big data bring us not only constantly growing data volume, dynamic and elastic storage demands, diversified data structures, but also different data features. Apart from the traditional dense data, more and more “sparse” data emerged and account for the majority of the massive data. How to adapt to the characteristics of the sparse data without losing sight of the traits of the dense data is a challenge. This paper studies how to integrate row and column data-layouts for both dense and sparse datasets in the cloud. A new NF2 scalable storage structure named “Dynamic Table” based on the key-value storage is proposed. The formal definition of dynamic table and implemention on HDFS is also introduced.

Keywords

Massive Data NF2 Cloud Computing HDFS 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Gantz, J.F.: The Expanding Digital Universe. International Data Corporation (2007)Google Scholar
  2. 2.
    Doan, A., Naughton, J.F., Baid, A., et al.: Information extraction challenges in managing unstructured data. ACM SIGMOD Record Archive 37(4), 14–20 (2008)CrossRefGoogle Scholar
  3. 3.
    Lux, M., Chatzichristofis, S.A.: LIRe: Lucene Image Retrieval – An Extensible Java CBIR Library. In: MM 2008 Proceedings of the 16th ACM international conference on Multimedia, pp. 1085–1088 (2008)Google Scholar
  4. 4.
    Tamura, H., Mori, S., Yamawaki, T.: Textural features corresponding to visual perception. IEEE Transactions on Systems, Man, and Cybernetics 8(6), 460–472 (1978)CrossRefGoogle Scholar
  5. 5.
    Hirata, K., Kato, T.: Query by Visual Example - Content-Based Image Retrieval. In: Pirotte, A., Delobel, C., Gottlob, G. (eds.) EDBT 1992. LNCS, vol. 580, pp. 56–71. Springer, Heidelberg (1992)CrossRefGoogle Scholar
  6. 6.
  7. 7.
    He, Y., Lee, R.B., Huai, Y., et al.: A Fast and Space-efficient Data Placement Structure in MapReduce-based Warehouse Systems. In: Proceedings of the IEEE International Conference on Data Engineering (ICDE), pp. 1199–1208 (2011)Google Scholar
  8. 8.
    Beckmann, J.L., Halverson, A., Krishnamurthy, R., et al.: Extending RDBMSs to support sparse datasets using an interpreted attribute storage format. In: Proceedings of the 22nd International Conference on Data Engineering ICDE, pp. 58–74 (2006)Google Scholar
  9. 9.
    Abadi, D.J.: Column Stores For Wide and Sparse Data. In: Proceedings of CIDR, pp. 292–297 (2007)Google Scholar
  10. 10.
  11. 11.
    Apache Cassandra, http://cassandra.apache.org/
  12. 12.
    Chang, F., Dean, J., Ghemawat, J., et al.: Bigtable: A Distributed Storage System for Structured Data. J. ACM Transactions on Computer Systems 26, 1–26 (2008)zbMATHCrossRefGoogle Scholar
  13. 13.
  14. 14.
  15. 15.

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Hanchen Su
    • 1
  • Hongyan Li
    • 1
  • Xu Cheng
    • 1
  • Zhiqiang Liu
    • 1
  1. 1.Key Laboratory of Machine Perception (Peking University), Ministry of Education School of Electronics Engineering and Computer SciencePeking UniversityBeijingChina

Personalised recommendations