Abstract
With the broad range of data available on the World Wide Web and the increasing use of social media such as Facebook, Twitter, YouTube, etc. a “Big Data” notion has emerged. This latter has become an important aspect in nowadays business since it is full of important knowledge that is crucial for effective decision making. However, this kind of data brings with it new problems and challenges for the Decision Support System (DSS) that must be addressed. In this paper, we propose a new approach called BigDimETL (Big Dimensional ETL) that deals with ETL (Extract-Transform-Load) development process. Our approach focuses on integrating Big Data taking into account the MultiDimensional Structure (MDS) through a MapReduce paradigm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Arres, B., Kabachi, N., Boussaid, O.: Building OLAP cubes on a cloud computing environment with MapReduce. In: ACS International Conference on Computer Systems and Applications, AICCSA, pp. 1–5 (2013)
Bala, M., Boussaïd, O., Alimazighi, Z.: P-ETL: parallel-ETL based on the MapReduce paradigm. In: 11th IEEE/ACS International Conference on Computer Systems and Applications, AICCSA, pp. 42–49 (2014)
Bellatreche, L., Schneider, M., Mohania, M., Bhargava, B.: PartJoin: an efficient storage and query execution for data warehouses. In: Kambayashi, Y., Winiwarter, W., Arikawa, M. (eds.) DaWaK 2002. LNCS, vol. 2454, pp. 296–306. Springer, Heidelberg (2002). doi:10.1007/3-540-46145-0_29
Berro, A., Megdiche, I., Teste, O.: Graph-based ETL processes for warehousing statistical open data. In: Proceedings of the 17th International Conference on Enterprise Information Systems, pp. 271–278 (2015)
Chung, W.C., Lin, H.P., Chen, S.-H., et al.: JackHare: a framework for SQL to NoSQL translation using MapReduce. Autom. Softw. Eng. 21(4), 489–508 (2014)
Dean, J., Ghemawat, S.: MapReduce: simplified data processing on large clusters. Commun. ACM 51(1), 107–113 (2008)
Deb Nath, R.P., Hose, K., et al.: Towards a programmable semantic extract-transform-load framework for semantic data warehouses. In: Proceedings of the ACM Eighteenth International Workshop on Data Warehousing and OLAP, pp. 15–24 (2015)
Akkaoui, Z., Mazón, J.-N., Vaisman, A., Zimányi, E.: BPMN-based conceptual modeling of ETL processes. In: Cuzzocrea, A., Dayal, U. (eds.) DaWaK 2012. LNCS, vol. 7448, pp. 1–14. Springer, Heidelberg (2012). doi:10.1007/978-3-642-32584-7_1
El-Sappagh, S.H.A., Hendawi, A.M.A., El Bastawissy, A.H.: Original article: a proposed model for data warehouse ETL processes. J. King Saud Univ. Comput. Inf. Sci. 23(2), 91–104 (2011)
Jaspreet Kaur, K.K.: A new improved vertical partitioning scheme for non relational databases using greedy method. Int. J. Adv. Res. Comput. Commun. Eng. 2 (2013)
Kimball, R., Ross, M.: The Data Warehouse Toolkit: The Complete Guide to Dimensional Modeling, 2nd edn. Wiley, Hoboken (2002)
Kraiem, M.B., Feki, J., Khrouf, K., et al.: Modeling and OLAPing social media: the case of Twitter. Soc. Netw. Anal. Min. 5(1), 47:1–47:15 (2015)
Liu, X., Thomsen, C., Pedersen, T.B.: ETLMR: a highly scalable dimensional ETL framework based on MapReduce. Trans. Large-Scale Data Knowl. Cent. Syst. 8, 1–31 (2013)
Liu, X., Thomsen, C., Pedersen, T.B.: CloudETL: scalable dimensional ETL for hive. In: 18th International Database Engineering & Applications Symposium, IDEAS, pp. 195–206 (2014)
Oliveira, B., Belo, O.: Using REO on ETL conceptual modelling: a first approach. In: Proceedings of the Sixteenth International Workshop on Data Warehousing and OLAP, DOLAP 2013, pp. 55–60 (2013)
Orlando, S., Orsini, R., Raffaetà, A., Roncato, A., Silvestri, C.: Trajectory data warehouses: design and implementation issues. JCSE 1(2), 211–232 (2007)
Silva, D., Fernandes, J.M., Belo, O.: Assisting data warehousing populating processes design through modelling using coloured petri nets. In: 2013 - Proceedings of the 3rd International Conference on Simulation and Modeling Methodologies, pp. 35–42 (2013)
Thusoo, A., Sarma, J.S., Jain, N., Shao, Z., Chakka, P., Anthony, S., Liu, H., Wyckoff, P., Murthy, R.: Hive: a warehousing solution over a map-reduce framework. Proc. VLDB Endow. 2(2), 1626–1629 (2009)
Trujillo, J., Luján-Mora, S.: A UML based approach for modeling ETL processes in data warehouses. In: Song, I.-Y., Liddle, S.W., Ling, T.-W., Scheuermann, P. (eds.) ER 2003. LNCS, vol. 2813, pp. 307–320. Springer, Heidelberg (2003). doi:10.1007/978-3-540-39648-2_25
Vassiliadis, P., Simitsis, A., Skiadopoulos, S.: Conceptual modeling for ETL processes. In: Proceedings of the 5th ACM International Workshop on Data Warehousing and OLAP, DOLAP 2002, pp. 14–21. ACM, New York (2002)
Vassiliadis, P., Vagena, Z., et al.: ARKTOS: towards the modeling, design, control and execution of ETL processes. Inf. Syst. 26(8), 537–561 (2001)
White, T.: Hadoop: The Definitive Guide. O’Reilly Media, Inc., Sebastopol (2012)
Acknowledgments
This work is dedicated to the soul of my supervisor, Dr. Lotfi Bouzguenda, who left us in juin 2016. We are very grateful for his help, his advice and his prestigious remarks. May his soul rest in peace.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Mallek, H., Ghozzi, F., Teste, O., Gargouri, F. (2017). BigDimETL: ETL for Multidimensional Big Data. In: Madureira, A., Abraham, A., Gamboa, D., Novais, P. (eds) Intelligent Systems Design and Applications. ISDA 2016. Advances in Intelligent Systems and Computing, vol 557. Springer, Cham. https://doi.org/10.1007/978-3-319-53480-0_92
Download citation
DOI: https://doi.org/10.1007/978-3-319-53480-0_92
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-53479-4
Online ISBN: 978-3-319-53480-0
eBook Packages: EngineeringEngineering (R0)