Abstract
In the context of near-real-time data warehousing the user’s updates generated at data source level need to be stored into warehouse as soon as they occur. Before loading these updates into the warehouse they need to be transformed, often using a join operator between the stream of updates and disk-based master data. In this context a stream-based algorithm called X-HYBRIDJOIN (Extended Hybrid Join) has been proposed earlier, with a favourable asymptotic runtime behavior. However, the absolute performance was not as good as hoped for. In this paper we present results showing that through properly tuning the algorithm, the resulting “Tuned X-HYBRIDJOIN” performs significantly better than that of the previous X-HYBRIDJOIN, and better as other applicable join operators found in literature. We present the tuning approach, based on measurement techniques and a revised cost model. To evaluate the algorithm’s performance we conduct an experimental study that shows that the Tuned X-HYBRIDJOIN exhibits the desired performance characteristics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Naeem, M.A., Dobbie, G., Weber, G.: X-HYBRIDJOIN for Near-real-time Data Warehousing. In: Fernandes, A.A.A., Gray, A.J.G., Belhajjame, K. (eds.) BNCOD 2011. LNCS, vol. 7051, pp. 33–47. Springer, Heidelberg (2011)
Anderson, C.: The Long Tail: Why the Future of Business is Selling Less of More. Hyperion (2006)
Naeem, M.A., Dobbie, G., Weber, G.: HYBRIDJOIN for Near-real-time Data Warehousing. IJDWM: International Journal of Data Warehousing and Mining 7(4), 21–42 (2011)
Golab, L., Tamer Özsu, M.: Processing Sliding Window Multi-Joins in Continuous Queries over Data Streams. In: VLDB 2003, Berlin, Germany, pp. 500–511 (2003)
Babu, S., Widom, J.: Continuous queries over data streams. SIGMOD Rec. 30(3), 109–120 (2001)
Polyzotis, N., Skiadopoulos, S., Vassiliadis, P., Simitsis, A., Frantzell, N.E.: Supporting Streaming Updates in an Active Data Warehouse. In: ICDE 2007: IEEE 23rd International Conference on Data Engineering, Los Alamitos, CA, USA, pp. 476–485 (2007)
Polyzotis, N., Skiadopoulos, S., Vassiliadis, P., Simitsis, A., Frantzell, N.: Meshing Streaming Updates with Persistent Data in an Active Data Warehouse. IEEE Trans. on Knowl. and Data Eng. 20(7), 976–991 (2008)
Naeem, M.A., Dobbie, G., Weber, G.: R-MESHJOIN for Near-real-time Data Warehousing. In: DOLAP 2010: Proceedings of the ACM 13th International Workshop on Data Warehousing and OLAP. ACM, Toronto (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Naeem, M.A., Dobbie, G., Weber, G., Bajwa, I.S. (2012). Efficient Usage of Memory Resources in Near-Real-Time Data Warehousing. In: Chowdhry, B.S., Shaikh, F.K., Hussain, D.M.A., Uqaili, M.A. (eds) Emerging Trends and Applications in Information Communication Technologies. IMTIC 2012. Communications in Computer and Information Science, vol 281. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28962-0_32
Download citation
DOI: https://doi.org/10.1007/978-3-642-28962-0_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28961-3
Online ISBN: 978-3-642-28962-0
eBook Packages: Computer ScienceComputer Science (R0)