Advertisement

Accessing Scientific Data: Simpler is Better

  • Mirek Riedewald
  • Divyakant Agrawal
  • Amr El Abbadi
  • Flip Korn
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2750)

Abstract

A variety of index structures has been proposed for supporting fast access and summarization of large multidimensional data sets. Some of these indices are fairly involved, hence few are used in practice. In this paper we examine how to reduce the I/O cost by taking full advantage of recent trends in hard disk development which favor reading large chunks of consecutive disk blocks over seeking and searching. We present the Multiresolution File Scan (MFS) approach which is based on a surprisingly simple and flexible data structure which outperforms sophisticated multidimensional indices, even if they are bulk-loaded and hence optimized for query processing. Our approach also has the advantage that it can incorporate a priori knowledge about the query workload. It readily supports summarization using distributive (e.g., count, sum, max, min) and algebraic (e.g., avg) aggregate operators.

Keywords

Hard Disk Query Processing Index Structure Range Query Query Performance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Beckmann, N., Kriegel, H.-P., Schneider, R., Seeger, B.: The R*-tree: An efficient and robust access method for points and rectangles. In: Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 322–331 (1990)Google Scholar
  2. 2.
    Berchtold, S., Böhm, C., Kriegel, H.-P.: Improving the query performance of high-dimensional index structures by bulk-load operations. In: Schek, H.-J., Saltor, F., Ramos, I., Alonso, G. (eds.) EDBT 1998. LNCS, vol. 1377, pp. 216–230. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  3. 3.
    Berchtold, S., Keim, D.A., Kriegel, H.-P.: The X-tree: An index structure for high-dimensional data. In: Proc. Int. Conf. on Very Large Databases (VLDB), pp. 28–39 (1996)Google Scholar
  4. 4.
    Bernstein, P.A., et al.: The Asilomar report on database research. SIGMOD Record 27(4), 74–80 (1998)CrossRefGoogle Scholar
  5. 5.
    Böhm, C., Kriegel, H.-P.: Dynamically optimizing high-dimensional index structures. In: Zaniolo, C., Grust, T., Scholl, M.H., Lockemann, P.C. (eds.) EDBT 2000. LNCS, vol. 1777, pp. 36–50. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  6. 6.
    Chazelle, B.: A functional approach to data structures and its use in multidimensional searching. SIAM Journal on Computing 17(3), 427–462 (1988)zbMATHCrossRefMathSciNetGoogle Scholar
  7. 7.
    Winter Corporation. Database scalability program (2001), http://www.wintercorp.com
  8. 8.
    Gaede, V., Günther, O.: Multidimensional access methods. ACM Computing Surveys 30(2), 170–231 (1998)CrossRefGoogle Scholar
  9. 9.
    Ganger, G.R., Worthington, B.L., Patt, Y.N.: The DiskSim Simulation Environment Version 2.0 Reference Manual (1999)Google Scholar
  10. 10.
    Gray, J., Chaudhuri, S., Bosworth, A., Layman, A., Reichart, D., Venkatrao, M., Pellow, F., Pirahesh, H.: Data cube: A relational aggregation operator generalizing group-by, cross-tab, and sub-totals. Data Mining and Knowledge Discovery, 29–53 (1997)Google Scholar
  11. 11.
    Guttman, A.: R-trees: A dynamic index structure for spatial searching. In: Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 47–57 (1984)Google Scholar
  12. 12.
    Hahn, C.J., Warren, S.G., London, J.: Edited synoptic cloud reports fromsh ips and land stations over the globe (1982-1991), http://cdiac.esd.ornl.gov/ftp/ndp026b (1996)
  13. 13.
    Jagadish, H.V., Lakshmanan, L.V.S., Srivastava, D.: Snakes and sandwiches: Optimal clustering strategies for a data warehouse. In: Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 37–48 (1999)Google Scholar
  14. 14.
    Kotidis, Y., Roussopoulos, N.: An alternative storage organization for ROLAP aggregate views based on cubetrees. In: Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 249–258 (1998)Google Scholar
  15. 15.
    Lang, C.A., Singh, A.K.: Modeling high-dimensional index structures using sampling. In: Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 389–400 (2001)Google Scholar
  16. 16.
    Lazaridis, I., Mehrotra, S.: Progressive approximate aggregate queries with a multi-resolution tree structure. In: Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 401–412 (2001)Google Scholar
  17. 17.
    Pagel, B.-U., Korn, F., Faloutsos, C.: Deflating the dimensionality curse using multiple fractal dimensions. In: Proc. Int. Conf. on Data Engineering (ICDE), pp. 589–598 (2000)Google Scholar
  18. 18.
    Poosala, V., Ioannidis, Y.E.: Selectivity estimation without the attribute value independence assumption. In: Proc. Int. Conf. on Very Large Databases (VLDB), pp. 486–495 (1997)Google Scholar
  19. 19.
    Proietti, G., Faloutsos, C.: I/O complexity for range queries on region data stored using an R-tree. In: Proc. Int. Conf. on Data Engineering (ICDE), pp. 628–635 (1999)Google Scholar
  20. 20.
    Riedewald, M., Agrawal, D., El Abbadi, A.: pCube: Update-efficient online aggregation with progressive feedback and error bounds. In: Proc. Int. Conf. on Scientific and Statistical Database Management (SSDBM), pp. 95–108 (2000)Google Scholar
  21. 21.
    Riedewald, M., Agrawal, D., El Abbadi, A.: Efficient integration and aggregation of historical information. In: Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 13–24 (2002)Google Scholar
  22. 22.
    Roussopoulos, N., Kotidis, Y., Roussopoulos, M.: Cubetree: Organization of and bulk updates on the data cube. In: Proc. ACM SIGMOD Int. Conf. on Management of Data, pp. 89–99 (1997)Google Scholar
  23. 23.
    Ruemmler, C., Wilkes, J.: An introduction to disk drive modeling. IEEE Computer 27(3), 17–28 (1994)Google Scholar
  24. 24.
    Seeger, B.: An analysis of schedules for performing multi-page requests. Information Systems 21(5), 387–407 (1996)CrossRefMathSciNetGoogle Scholar
  25. 25.
    Seeger, B., Larson, P.-A., McFayden, R.: Reading a set of disk pages. In: Proc. Int. Conf. on Very Large Databases (VLDB), pp. 592–603 (1993)Google Scholar
  26. 26.
    Shukla, A., Deshpande, P., Naughton, J.F., Ramasamy, K.: Storage estimation for multidimensional aggregates in the presence of hierarchies. In: Proc. Int. Conf. on Very Large Databases (VLDB), pp. 522–531 (1996)Google Scholar
  27. 27.
    Tao, Y., Papadias, D.: Adaptive index structures. In: Proc. Int. Conf. on Very Large Databases (VLDB), pp. 418–429 (2002)Google Scholar
  28. 28.
    Tao, Y., Papadias, D., Zhang, J.: Cost models for overlapping and multi-version structures. In: Proc. Int. Conf. on Data Engineering (ICDE), pp. 191–200 (2002)Google Scholar
  29. 29.
    Theodoridis, Y., Sellis, T.K.: A model for the prediction of R-tree performance. In: Proc. Symp. on Principles of Database Systems (PODS), pp. 161–171 (1996)Google Scholar
  30. 30.
    Thompson, D.A., Best, J.S.: The future of magnetic data storage technology. IBM Journal of Research and Development 44(3), 311–322 (2000)CrossRefGoogle Scholar
  31. 31.
    Transaction Processing Performance Council. TPC benchmarks, http://www.tpc.org
  32. 32.
    Weber, R., Schek, H.-J., Blott, S.: A quantitative analysis and performance study for similarity-search methods in high-dimensional spaces. In: Proc. Int. Conf. on Very Large Databases (VLDB), pp. 194–205 (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2003

Authors and Affiliations

  • Mirek Riedewald
    • 1
  • Divyakant Agrawal
    • 2
  • Amr El Abbadi
    • 2
  • Flip Korn
    • 3
  1. 1.Cornell UniversityIthaca
  2. 2.University of CaliforniaSanta Barbara
  3. 3.AT&T Labs-ResearchFlorham Park

Personalised recommendations