Abstract
With the increasing popularity of operational data warehousing, the ability to load data quickly and continuously into an RDBMS is becoming more and more important. However, in the presence of materialized join views, loading data concurrently into multiple base relations of the same materialized join view can cause a severe deadlock problem. To solve this problem, we propose reordering the data to be loaded so that at any time, for any materialized join view, data is only loaded into one of its base relations. Also, for load transactions on the relations that contain “aggregate” attributes, we propose using pre-aggregation to reduce the number of SQL statements in the load transactions. The advantages of our methods are demonstrated through experiments with a commercial parallel RDBMS.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Babcock, B., Babu, S., Datar, M., et al.: Models and Issues in Data Stream Systems. In: PODS 2002, pp. 1–16 (2002)
Bernstein, P.A., Hadzilacos, V., Goodman, N.: Concurrency Control and Recovery in Database Systems. Addison-Wesley, Reading (1987)
Bernstein, P.A., Hsu, M., Mann, B.: Implementing Recoverable Requests Using Queues. In: SIGMOD Conf. 1990, pp. 112–122 (1990)
Brobst, S., Rarey, J.: The Five Stages of an Active Data Warehouse Evolution (2001), http://www.ncr.com/online_periodicals/brobst.pdf
Brobst, S.: Personal communication (2003)
Dver, A.: Real-time Enterprise. Business week December 2, 2002 issue (2002)
Gawlick, D.: Processing “Hot Spots” in High Performance Systems. In: Proc. IEEE Compcon Spring 1985 (1985)
J. Gehrke, F. Korn, and D. Srivastava. On Computing Correlated Aggregates over Continual Data Streams. In: SIGMOD Conf. 2001, pp. 13–24 (2001)
Gray, J., Lorie, R.A., Putzolu, G.R., et al.: Granularity of Locks and Degrees of Consistency in a Shared Data Base. In: IFIP Working Conference on Modeling in Data Base Management Systems 1976, pp. 365–394 ( 1976)
Gray, J., Reuter, A.: Transaction Processing: Concepts and Techniques. Morgan Kaufmann Publishers, San Francisco (1993)
Graefe, G., Zwilling, M.J.: Transaction Support for Indexed Views. In: SIGMOD Conf. 2004 (2004)
Luo, G., Naughton, J.F., Ellmann, C.J., et al.: Locking Protocols for Materialized Aggregate Join Views. In: VLDB 2003, pp. 596–607 (2003)
Luo, G., Naughton, J.F., Ellmann, C.J., et al.: A Comparison of Three Methods for Join View Maintenance in Parallel RDBMS. In: ICDE 2003, pp. 177–188 (2003)
Luo, G., Naughton, J.F., Ellmann, C.J., et al.: Transaction Reordering and Grouping for Continuous Data Loading. Full version (2006), available at http://www.cs.wisc.edu/~gangluo/tpump_full.pdf
O’Gorman, K., Abbadi, A.E., Agrawal, D.: Multiple Query Optimization by Cache-Aware Middleware using Query Teamwork. In: ICDE 2002, p. 274 (2002)
Oracle Streams (2002), http://otn.oracle.com/products/dataint/htdocs/streams_fo.html
Oracle9i Database Daily Feature - MERGE Statement (2002), http://technet.oracle.com/products/oracle9i/daily/Aug24.html
Poess, M., Floyd, C.: New TPC Benchmarks for Decision Support and Web Commerce. SIGMOD Record 29(4), 64–71 (2000)
Pooloth, K.: High Performance Inserts on DB2 UDB EEE using Java (2002). http://www7b.boulder.ibm.com/dmdd/library/techarticle/0204pooloth/0204pooloth.html#overview
Teradata Parallel Data Pump Reference (2002), http://www.info.ncr.com/eDownload.cfm?itemid=023390001
Zhuge, Y., Garcia-Molina, H., Wiener, J.L.: The Strobe Algorithms for Multi-Source Warehouse Consistency. In: PDIS 1996, pp. 146–157 (1996)
A. Zimmerman, E. Nelson. In Hour of Peril, Americans Moved to Stock up on Guns and TV Sets. Wall Street Journal Newsletter (September 18, 2001), http://www.swcollege.com/econ/street/html/sept01/sept18_2001.html
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Luo, G., Naughton, J.F., Ellmann, C.J., Watzke, M.W. (2007). Transaction Reordering and Grouping for Continuous Data Loading. In: Bussler, C., Castellanos, M., Dayal, U., Navathe, S. (eds) Business Intelligence for the Real-Time Enterprises. BIRTE 2006. Lecture Notes in Computer Science, vol 4365. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73950-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-540-73950-0_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73949-4
Online ISBN: 978-3-540-73950-0
eBook Packages: Computer ScienceComputer Science (R0)