Abstract
Due to the advance of diverse techniques such as social networks and sensor networks, the volume of data to be processed has rapidly increased. After Google proposed the MapReduce framework which processes big data using large clusters of commodity machines, the MapReduce framework is considered as an effective processing paradigm for a massive data set. However, in the view of the performance, a problem of the MapReduce framework is that an efficient access method (i.e., an index) is not supported. Thus, whole data should be retrieved even though a user wants to access a small portion of data. In this paper, we propose an efficient method constructing quadtrees on the MapReduce framework. Our technique reduces the index construction time utilizing a sampling technique to partition a data set. In addition, using the constructed quadtree as well as the MapReduce framework, a subset of data to be retrieved is easily identified and is processed in parallel. Our experimental result demonstrates the efficiency of our proposed algorithm with diverse environments.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Dean, J., Ghemawat, S.: Mapreduce: Simplified data processing on large clusters. Communication of the ACM 51(1), 107–113 (2008)
Jestes, J., Yi, K., Li, F.: Building wavelet histograms on large data in mapreduce. In: Proceedings of VLDB, pp. 109–120 (2012)
Lu, W., Shen, Y., Chen, S., Ooi, B.C.: Efficient processing of k nearest neighbor joins using mapreduce. In: Proceedings of VLDB, pp. 1016–1027 (2012)
Zhang, X., Chen, L., Wang, M.: Efficient multi-way theta-join processing using mapreduce. In: Proceedings of VLDB, pp. 1184–1195 (2012)
Wang, Y., Wang, S.: Research and implementation on spatial data storage and operation based on hadoop platform. In: Proceedings of International Conference on Geoscience and Remote Sensing, pp. 275–278 (2010)
Finkel, R., Bentley, J.: Quad trees a data structure for retrieval on composite keys. Acta Informatica 4(1), 1–9 (1974)
Comer, D.: The ubiquitous b-tree. ACM Comput. Surv. 11(2), 121–137 (1979)
Beckmann, N., Kriegel, H.P., Schneider, R., Seeger, B.: The r*-tree: An efficient and robust access method for points and rectangles. In: Proceedings of ACM SIGMOD, pp. 322–331 (1990)
Apache: Apache hadoop (2010), http://hadoop.apache.org
Xia, C., Lu, H., Ooi, B.C., Hu, J.: Gorder: An efficient method for knn join processing. In: Proceedings of VLDB, pp. 756–767 (2004)
Zhang, B., Zhou, S., Guan, J.: Adapting skyline computation to the mapreduce framework: algorithms and experiments. In: Xu, J., Yu, G., Zhou, S., Unland, R. (eds.) DASFAA Workshops 2011. LNCS, vol. 6637, pp. 403–414. Springer, Heidelberg (2011)
Vitter, J.S.: Random sampling with a reservoir. ACM Transactions on Mathematical Software 11(1), 37–57 (1985)
Devore, J.L.: Probability and statistics for engineering and the science, 4th edn. Duxbury Press (1995)
Vershynin, R.: How close is the sample covariance matrix to the actual covariance matrix? Journal of Theoretical Probability 25(3), 655–686 (2012)
Zhang, S., Han, J., Liu, Z., Wang, K., Feng, S.: Spatial queries evaluation with mapreduce. In: Proceedings of GCC, pp. 287–292 (2009)
Wu, X., Carceroni, R., Fang, H., Zelinka, S., Kirmse, A.: Automatic alignment of large-scale aerial rasters to road-maps. In: Proceedings of ACM GIS, pp. 17:1–17:8 (2007)
Akdogan, A., Demiryurek, U., Banaei-Kashani, F., Shahabi, C.: Voronoi-based geospatial query processing with mapreduce. In: Proceedings of IEEE CloudCom, pp. 9–6 (2010)
Wang, K., Han, J., Tu, B., Dai, J., Zhou, W., Song, X.: Accelerating spatial data processing with mapreduce. In: Proceedings of IEEE ICPADS, pp. 229–236 (2010)
Cary, A., Sun, Z., Hristidis, V., Rishe, N.: Experiences on processing spatial data with mapreduce. In: Winslett, M. (ed.) SSDBM 2009. LNCS, vol. 5566, pp. 302–319. Springer, Heidelberg (2009)
Schlosser, S.W., Ryan, M.P., Taborda, R., López, J., O’Hallaron, D.R., Bielak, J.: Materialized community ground models for large-scale earthquake simulation. In: Proceedings of ACM/IEEE Conference on Supercomputing (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Noh, H., Min, JK. (2013). An Efficient Data Access Method Exploiting Quadtrees on MapReduce Frameworks. In: Hong, B., Meng, X., Chen, L., Winiwarter, W., Song, W. (eds) Database Systems for Advanced Applications. DASFAA 2013. Lecture Notes in Computer Science, vol 7827. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-40270-8_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-40270-8_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-40269-2
Online ISBN: 978-3-642-40270-8
eBook Packages: Computer ScienceComputer Science (R0)