Advertisement

A Bloom Filter-Based Index for Distributed Storage Systems

  • Zhu WangEmail author
  • Chenxi Luo
  • Tiejian Luo
  • Xia Chen
  • Jinzhong Hou
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 373)

Abstract

The indexing technique, which is capable of locating an item, is a key component in distributed storage systems. There have been many solutions for the index in distributed systems. One of the problems is the large number of items and the (relatively) low space available for the index. In this paper we propose a bloom filter based schema for the representation and lookup of items in the distributed systems. In each node, the method selects items and inserts them into a probabilistic data structure. After gathering all the data structures, the index node is in possess of all objects information and is capable of locating items in the system. To reduce the false checking times of the index, we choose items to be recorded in reference with the Internet user behavior pattern. We further use theoretical and experimental analysis to test our proposal. Results show that our method can achieve high performance with limited index space.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Bloom, B.H.: Space/time trade-offs in hash coding with allowable errors. Communications of the ACM 13, 422–426 (1970)CrossRefzbMATHGoogle Scholar
  2. 2.
    Tarkoma, S., Rothenberg, C.E., Lagerspetz, E.: Theory and practice of bloom filters for distributed systems. IEEE Communications Surveys & Tutorials 14(1), 131–155 (2012)CrossRefGoogle Scholar
  3. 3.
    Luo, T., Wang, Z., Wang, X.: A privacy preserving model for ownership indexing in distributed storage systems. In: Proceedings of the Workshops of the EDBT/ICDT 2014 Joint Conference (EDBT/ICDT 2014). CEUR Workshop Proceedings, pp. 416–420 (2014)Google Scholar
  4. 4.
    Luo, C., Wang, Z., Chen, X., Luo, T.: Lba privacy preserving index and its theoretical analysis in cloud storage systems. In: Proceedings of the 2nd International Workshop on Security and Forensics in Communication Systems, SFCS 2014, pp. 37–44. ACM, New York (2014)CrossRefGoogle Scholar
  5. 5.
    Chierichetti, F., Kumar, R., Raghavan, P.: Compressed web indexes. In: Proceedings of the 18th International Conference on World Wide Web, pp. 451–460. ACM (2009)Google Scholar
  6. 6.
    Breslau, L., Cao, P., Fan, L., Phillips, G., Shenker, S.: Web Caching and Zipf-like Distributions: Evidence and Implications. In: IEEE INFOCOM, vol. 1, pp. 126–134 (1999)Google Scholar
  7. 7.
    Mullin, J.K.: A second look at bloom filters. Communications of the ACM 26(8), 570–571 (1983)CrossRefMathSciNetGoogle Scholar
  8. 8.
    Ficara, D., Giordano, S., Procissi, G., Vitucci, F.: Multilayer compressed counting bloom filters. In: INFOCOM 2008. The 27th Conference on Computer Communications, pp. 311–315. IEEE (2008)Google Scholar
  9. 9.
    Song, H., Dharmapurikar, S., Turner, J., Lockwood, J.: Fast hash table lookup using extended bloom filter: an aid to network processing. In: Proceedings of the 2005 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, SIGCOMM 2005, pp. 181–192. ACM, New York (2005)CrossRefGoogle Scholar
  10. 10.
    Bonomi, F., Mitzenmacher, M., Panigrah, R., Singh, S., Varghese, G.: Beyond bloom filters: from approximate membership checks to approximate state machines. In: Proceedings of the 2006 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, SIGCOMM 2006, pp. 315–326. ACM, New York (2006)CrossRefGoogle Scholar
  11. 11.
    Wang, Z., Luo, T.: Intelligent video content routing in a direct access network. In: 2011 3rd Symposium on Web Society (SWS), pp. 147–152. IEEE (2011)Google Scholar
  12. 12.
    Jokela, P., Zahemszky, A., Esteve Rothenberg, C., Arianfar, S., Nikander, P.: Lipsin: line speed publish/subscribe inter-networking. In: Proceedings of the ACM SIGCOMM 2009 Conference on Data Communication, SIGCOMM 2009, pp. 195–206. ACM, New York (2009)CrossRefGoogle Scholar
  13. 13.
    Zhu, Y., Jiang, H., Wang, J., Xian, F.: Hba: Distributed metadata management for large cluster-based storage systems. IEEE Transactions on Parallel and Distributed Systems 19(6), 750–763 (2008)CrossRefGoogle Scholar
  14. 14.
    Kotera, I., Egawa, R., Takizawa, H., Kobayashi, H.: Modeling of cache access behavior based on zipf’s law. In: Proceedings of the 9th Workshop on MEmory Performance: DEaling with Applications, Systems and Architecture, MEDEA 2008, pp. 9–15. ACM, New York (2008)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Zhu Wang
    • 1
    Email author
  • Chenxi Luo
    • 2
  • Tiejian Luo
    • 2
  • Xia Chen
    • 3
  • Jinzhong Hou
    • 2
  1. 1.Xingtang Telecommunications Technology Co., Ltd.BeijingChina
  2. 2.University of Chinese Academy of Sciences (UCAS)BeijingChina
  3. 3.SAPPRFTBeijingChina

Personalised recommendations