Life and Death of Data in Data Lakes: Preserving Data Usability and Responsible Governance
Data crossing seeks the extraction of novel knowledge through correlations and dependencies among heterogeneous data, and is considered a key process in sustainable science to push back the current frontiers of knowledge, especially to address challenges such as the socio-economic impacts of climate change. To tackle such complex challenges, interdisciplinary approaches and data sharing methodologies are ubiquitous, with a strong focus on data openness and ensuring that the fair principles hold. Data lakes are data repositories, recently developed to store such big heterogeneous data that are then available for crossing and be exploited without a priori objectives regarding their usage (unlike data warehouses). Such data lakes can then be used to populate Open and Linked Open Data in a central location regardless of its source or format. In this context of no prior knowledge regarding its usage, it may be tempting to store and share all the available data. However, this comes with two main disadvantages: (1) overwhelming amount of data that could prevent end users from exploiting the data, (2) and environmental reasons (energy consumption of data storage). Moreover, data of poor quality may deserve the lake usability and be deleted. We thus claim in this position paper that a data life cycle must be designed so as to integrate data death for some of the data. The choice of the data to be stored regarding the ones to forget is then of crucial importance in data lakes. We propose here some first positions for this aspect of data governance.
KeywordsData lakes Web of data Data life cycle and data governance Sustainability
Supported by PHC CEDRE 42415YJ, French Ministry of European and Foreign Affairs (MEAE), French Ministry of Higher Education, Research and Innovation (MESRI) and Lebanese Ministry of Education and Higher Education (MEHE).
- 1.Boly, A., Hébrail, G.: Forgetting data intelligently in data warehouses. In: 2007 IEEE International Conference on Research, Innovation and Vision for the Future in Computing and Communication Technologies, RIVF 2007, Hanoi, Vietnam, 5–9 March 2007, pp. 220–227. IEEE (2007). https://doi.org/10.1109/RIVF.2007.369160
- 2.Fang, H.: Managing data lakes in big data era: what’s a data lake and why has it became popular in data management ecosystem. In: International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER), pp. 820–824. IEEE (2015)Google Scholar
- 3.Gartner: Gartner Says Beware of the Data Lake Fallacy (2014). http://www.gartner.com/newsroom/id/2809117
- 4.Gaucherel, C., Gouyon, P., Dessalles, J.L.: Information. The Hidden Side of Life. ISTE (2019)Google Scholar
- 5.IBM: Governing and Managing Big Data for Analytics and Decision Makers (2014). http://www.redbooks.ibm.com/abstracts/redp5120.html?Open
- 9.Madera, C., Laurent, A.: The next information architecture evolution: the data lake wave. In: Chbeir, R., Agrawal, R., Biskri, I. (eds.) Proceedings of the 8th International Conference on Management of Digital EcoSystems, MEDES 2016, Biarritz, France, 1–4 November 2016. pp. 174–180. ACM (2016). https://doi.org/10.1145/3012071
- 11.Mentzer, J., Witt, W.D., et al.: Defining supply chain (SC) management. J. Bus. Logist. 22(2) (2001). https://doi.org/10.1002/j.2158-1592.2001.tb00001.xCrossRefGoogle Scholar