Advertisement

A Hybrid Index Model for Efficient Spatio-Temporal Search in HBase

  • Chengyuan Zhang
  • Lei ZhuEmail author
  • Jun Long
  • Shuangqiao Lin
  • Zhan Yang
  • Wenti Huang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11154)

Abstract

With advances in geo-positioning technologies and geo-locat-ion services, there are a rapidly growing massive amount of spatio-tempor-al data collected in many applications such as location-aware devices and wireless communication, in which an object is described by its spatial location and its timestamp. Consequently, the study of spatio-temporal search which explores both geo-location information and temporal information of the data has attracted significant concern from research organizations and commercial communities. This work study the problem of spatio-temporal k-nearest neighbors search (STkNNS), which is fundamental in the spatial temporal queries. Based on HBase, a novel index structure is proposed, called Hybrid Spatio-Temporal HBase Index (HSTI for short), which is carefully designed and takes both spatial and temporal information into consideration to effectively reduce the search space. Based on HSTI, an efficient algorithm is developed to deal with spatio-temporal k-nearest neighbors search. Comprehensive experiments on real and synthetic data clearly show that HSTI is three to five times faster than the state-of-the-art technique.

Notes

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China (61702560, 61379110, 61472450), the Key Research Program of Hunan Province (2016JC2018), Natural Science Foundation of Hunan Province (2018JJ3691), and Science and Technology Plan of Hunan Province (2016JC2011).

References

  1. 1.
    Wang, Y., Lin, X., Wu, L., Zhang, W.: Effective multi-query expansions: collaborative deep networks for robust landmark retrieval. IEEE Trans. Image Process. 26(3), 1393–1404 (2017)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Wu, L., Huang, X., Zhang, C., Shepherd, J., Wang, Y.: An efficient framework of Bregman divergence optimization for co-ranking images and tags in a heterogeneous network. Multimed. Tools Appl. 74(15), 5635–5660 (2015)CrossRefGoogle Scholar
  3. 3.
    Wang, Y., Zhang, W., Wu, L., Lin, X., Fang, M., Pan, S.: Iterative views agreement: an iterative low-rank based structured optimization method to multi-view spectral clustering. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, IJCAI 2016, New York, NY, USA, 9–15 July 2016, pp. 2153–2159 (2016)Google Scholar
  4. 4.
    Wang, Y., Lin, X., Zhang, Q.: Towards metric fusion on multi-view data: a cross-view based graph random walk approach. In: 22nd ACM International Conference on Information and Knowledge Management, CIKM 2013, San Francisco, CA, USA, 27 October–1 November 2013, pp. 805–810 (2013)Google Scholar
  5. 5.
    Zhang, C., Zhang, Y., Zhang, W., Lin, X.: Inverted linear quadtree: efficient top K spatial keyword search. IEEE Trans. Knowl. Data Eng. 28(7), 1706–1721 (2016)CrossRefGoogle Scholar
  6. 6.
    Wu, L., Wang, Y., Li, X., Gao, J.: What-and-where to match: deep spatially multiplicative integration networks for person re-identification. Pattern Recognit. 76, 727–738 (2018)CrossRefGoogle Scholar
  7. 7.
    Wu, L., Wang, Y., Ge, Z., Hu, Q., Li, X.: Structured deep hashing with convolutional neural networks for fast person re-identification. Comput. Vis. Image Underst. 167, 63–73 (2018)CrossRefGoogle Scholar
  8. 8.
    Liu, A., Liu, X., Long, J.: A trust-based adaptive probability marking and storage traceback scheme for wsns. Sensors 16(4), 451 (2016)MathSciNetCrossRefGoogle Scholar
  9. 9.
    Wu, L., Wang, Y., Gao, J., Li, X.: Deep adaptive feature embedding with local sample distributions for person re-identification. Pattern Recognit. 73, 275–288 (2018)CrossRefGoogle Scholar
  10. 10.
    Wang, Y., Zhang, W., Wu, L., Lin, X., Zhao, X.: Unsupervised metric fusion over multiview data by graph random walk-based cross-view diffusion. IEEE Trans. Neural Netw. Learn. Syst. 28(1), 57–70 (2017)CrossRefGoogle Scholar
  11. 11.
    Wang, Y., Lin, X., Wu, L., Zhang, W., Zhang, Q., Huang, X.: Robust subspace clustering for multi-view data by exploiting correlation consensus. IEEE Trans. Image Process. 24(11), 3939–3949 (2015)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Wang, Y., Wu, L., Lin, X., Gao, J.: Multiview spectral clustering via structured low-rank matrix factorization. IEEE Trans. Neural Netw. Learn. Syst. 29(10), 4833–4843 (2018)CrossRefGoogle Scholar
  13. 13.
    Wang, Y., Wu, L.: Beyond low-rank representations: orthogonal clustering basis reconstruction with optimized graph structure for multi-view spectral clustering. Neural Netw. 103, 1–8 (2018)CrossRefGoogle Scholar
  14. 14.
    Wu, L., Wang, Y., Li, X., et al.: Deep attention-based spatially recursive networks for fine-grained visual recognition. IEEE Trans. Cybern. PP(99), 1–12 (2018)Google Scholar
  15. 15.
    Guttman, A.: R-trees: a dynamic index structure for spatial searching. In: SIGMOD 1984, Proceedings of Annual Meeting, Boston, Massachusetts, 18–21 June 1984, pp. 47–57 (1984)Google Scholar
  16. 16.
    Beckmann, N., Kriegel, H., Schneider, R., Seeger, B.: The R*-tree: an efficient and robust access method for points and rectangles. In: Proceedings of the 1990 ACM SIGMOD International Conference on Management of Data, Atlantic City, NJ, 23–25 May 1990, pp. 322–331 (1990)Google Scholar
  17. 17.
    Finkel, R.A., Bentley, J.L.: Quad trees: a data structure for retrieval on composite keys. Acta Inf. 4, 1–9 (1974)CrossRefGoogle Scholar
  18. 18.
    Brown, R.A.: Building a balanced k-d tree in O(kn log n) time. CoRR abs/1410.5420 (2014)Google Scholar
  19. 19.
    Fox, A.D., Eichelberger, C.N., Hughes, J.N., Lyon, S.: Spatio-temporal indexing in non-relational distributed databases. In: Proceedings of the 2013 IEEE International Conference on Big Data, Santa Clara, CA, USA, 6–9 October 2013, pp. 291–299 (2013)Google Scholar
  20. 20.
    Eldawy, A., Mokbel, M.F.: A demonstration of spatialhadoop: an efficient mapreduce framework for spatial data. PVLDB 6(12), 1230–1233 (2013)Google Scholar
  21. 21.
    Wang, Y., Lin, X., Wu, L., Zhang, W., Zhang, Q.: Exploiting correlation consensus: towards subspace clustering for multi-modal data. In: Proceedings of the ACM International Conference on Multimedia, MM 2014, Orlando, FL, USA, 03–07 November 2014, pp. 981–984 (2014)Google Scholar
  22. 22.
    Aji, A., et al.: Hadoop-GIS: a high performance spatial data warehousing system over mapreduce. PVLDB 6(11), 1009–1020 (2013)Google Scholar
  23. 23.
    Wang, Y., Lin, X., Zhang, Q., Wu, L.: Shifting hypergraphs by probabilistic voting. In: Tseng, V.S., Ho, T.B., Zhou, Z.-H., Chen, A.L.P., Kao, H.-Y. (eds.) PAKDD 2014. LNCS (LNAI), vol. 8444, pp. 234–246. Springer, Cham (2014).  https://doi.org/10.1007/978-3-319-06605-9_20CrossRefGoogle Scholar
  24. 24.
    Nishimura, S., Das, S., Agrawal, D., El Abbadi, A.: MD-HBase: a scalable multi-dimensional data infrastructure for location aware services. In: 12th IEEE International Conference on Mobile Data Management, MDM 2011, Luleå, Sweden, 6–9 June 2011, vol. 1, pp. 7–16 (2011)Google Scholar
  25. 25.
    Morton, M.G.: A Computer Oriented Geodetic Data Base and a New Technique in File Sequencing. International Business Machines Company, New York (1966)Google Scholar
  26. 26.
    Hsu, Y., Pan, Y., Wei, L., Peng, W., Lee, W.: Key formulation schemes for spatial index in cloud data managements. In: 13th IEEE International Conference on Mobile Data Management, MDM 2012, Bengaluru, India, 23–26 July 2012, pp. 21–26 (2012)Google Scholar
  27. 27.
    Zhang, N., Zheng, G., Chen, H., Chen, J., Chen, X.: HBaseSpatial: a scalable spatial data storage based on HBase. In: 13th IEEE International Conference on Trust, Security and Privacy in Computing and Communications, TrustCom 2014, Beijing, China, 24–26 September 2014, pp. 644–651 (2014)Google Scholar
  28. 28.
    Chen, X.Y., Zhang, C., Ge, B., Xiao, W.D.: Efficient historical query in HBase for spatio-temporal decision support. Int. J. Comput., Commun. Control. 11(5), 613–630 (2016)CrossRefGoogle Scholar
  29. 29.
    Roussopoulos, N., Kelley, S., Vincent, F.: Nearest neighbor queries. In: Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, San Jose, California, 22–25 May 1995, pp. 71–79 (1995)Google Scholar
  30. 30.
    Zheng, Y., Xie, X., Ma, W.: GeoLife: a collaborative social networking service among user, location and trajectory. IEEE Data Eng. Bull. 33(2), 32–39 (2010)Google Scholar
  31. 31.
    Yuan, J., Zheng, Y., Xie, X., Sun, G.: Driving with knowledge from the physical world. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, USA, 21–24 August 2011, pp. 316–324 (2011)Google Scholar
  32. 32.
    Yuan, J., et al.: T-Drive: driving directions based on taxi trajectories. In: Proceedings of 18th ACM SIGSPATIAL International Symposium on Advances in Geographic Information Systems, ACM-GIS 2010, San Jose, CA, USA, 3–5 November 2010, pp. 99–108 (2010)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Chengyuan Zhang
    • 1
    • 2
  • Lei Zhu
    • 1
    • 2
  • Jun Long
    • 1
    • 2
  • Shuangqiao Lin
    • 1
    • 2
  • Zhan Yang
    • 1
    • 2
  • Wenti Huang
    • 1
    • 2
  1. 1.School of Information ScienceCentral South UniversityChangshaPeople’s Republic of China
  2. 2.Big Data and Knowledge Engineering InstituteCentral South UniversityChangshaPeople’s Republic of China

Personalised recommendations