Abstract
Data cube has been playing an essential role in fast OLAP(on-line analytical processing) in many multidimensional data warehouse. We often execute range queries on aggregate cube computed by pre-aggregate technique in MOLAP. For the cube with d dimensions, it can generate 2d cuboids. But in a high-dimensional data warehouse (such as the applications of bioinformatics and statistical analysis, etc.), we build all these cuboids and their indices and full materialized the data cube impossibly. In this paper, we propose a multi-dimensional hierarchical fragmentation of the fact table based on dimension hierarchical encoding. This method partition the high dimensional data cube into shell mini-cubes. Using dimension hierarchical encoding and pre-aggregated results, OLAP queries are computed online by dynamically constructing cuboids from the fragment data cubes. Such an approach permits a significant reduction of processing and I/O overhead for many queries by restricting the number of fragments to be processed for both the fact table and bitmap encoding data. This method also supports parallel I/O and parallel processing as well as load balancing for disks and processors. We have compared the methods of our parallel method with the other existed ones such as partial cube by experiment. The analytical and experimental results show that the method of our parallel method proposed in this paper is more efficient than the other existed ones.
The research in the paper is supported by the National Natural Science Foundation of China under Grant No. 60473012; the Natural Science Foundation of Jiangsu Province under Grant No. BK2005047, BK2004052 and BK2005046; the National Tenth-Five High Technology Key Project of China under Grant No. 2003BA614A; the Tenth-Five High Technology Key Project of JiangSu Province of China under Grant No. BG2004034.
Chapter PDF
References
Chaudhuri, U., Dayal, U.: Data Warehousing and OLAP for Decision Support. ACM SIGMOD Record 26, 507–508 (1997)
Gray, J., Chaudhuri, S., Bosworth, A., Layman, A., Reichart, D., Venkatrao, M., Pellow, F., Pirahesh, H.: Data Cube: A Relational Aggregation Operator Generalizing Group-by, Cross-tab and Subtotals. Data Mining and Knowledge Discovery 1, 29–54 (1997)
Agarwal, S., Agrawal, R., Deshpande, P.M., Gupta, A., Naughton, J.F., Ramakrishnan, R., Sarawagi, S.: On the Computation of Multidimensional Aggregates. In: VLDB, pp. 506–521 (1996)
Beyer, K., Ramakrishnan, R.: Bottom-up Computation of Sparse and Iceberg Cubes. ACM SIDMOD, 359–370 (1999)
Han, J., Pei, J., Dong, G., Wang, K.: Efficient Computation of Iceberg Cubes with Complex Measures. ACM SIGMOD, 1–12 (2001)
Xin, D., Han, J., Li, X., Wah, B.W.: Star-cubing:Computing Iceberg Cubes by Top-down and Bottom-up Integration. In: VLDB, pp. 476–487 (2003)
Harinarayan, V., Rajaraman, A., Ullman, J.D.: Implementing Data Cubes Efficiently. ACM SIGMOD, 205–216 (1996)
Wang, W., Lu, H., Feng, J., Yu, J.X.: Condensed Cube: An Effective Approach to Reducing Data Cube Size. In: ICDE, pp. 155–165 (2002)
Sismanis, Y., Deligiannakis, A., Kotidis, Y., Roussopoulos, N.: Hierarchical Dwarfs for the Rollup Cube. In: VLDB, pp. 540–551 (2004)
Lakshmanan, L.V.S., Pei, J., Zhao, Y.: Q-trees: An Efficient Summary Structure for Semantic OLAP. ACM SIGMOD, 64–75 (2003)
Shanmugasundaram, J., Fayyad, U.M., Bradley, P.S.: Compressed Data Cubes for OLAP Aggregate Query Approximation on Continuous Dimensions. ACM SIGKDD, 223–232 (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 IFIP International Federation for Information Processing
About this paper
Cite this paper
Hu, K., Chen, L., Gu, Q., Li, B., Dong, Y. (2005). A Parallel and Distributed Method for Computing High Dimensional MOLAP. In: Jin, H., Reed, D., Jiang, W. (eds) Network and Parallel Computing. NPC 2005. Lecture Notes in Computer Science, vol 3779. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11577188_30
Download citation
DOI: https://doi.org/10.1007/11577188_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-29810-6
Online ISBN: 978-3-540-32246-7
eBook Packages: Computer ScienceComputer Science (R0)