Skip to main content
SpringerLink
Log in

Dealing with Dark Data – Shining a Light

  • Conference paper
  • First Online:
Knowledge Management in Organisations (KMO 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1825))

Included in the following conference series:

  • 607 Accesses

Abstract

Organizations today are gathering and storing more and more data in the belief that it is necessary for compliance, legal reasons or that it may be necessary in the future. Most of this data is considered dark as it is unstructured, uncatalogued, unmanaged, and unanalyzed. Big data consists of structured data (business critical and redundant obsolete and trivial (ROT) data) and unstructured data being dark data. This dark data can be in data silos isolated to specific departments or sectors in an organization unable to be accessed and analyzed by other departments in the organization. Organizations waste time and operating budgets searching for this data and storing the data. Data management practices, policies and procedures need to be reviewed by organizations. The creation of a position solely to be responsible for the storage, curation, and general good health of data should be considered. Dark data can have inherent security risks for organizations that can damage reputations, harm revenue, and leave the organization vulnerable to cybersecurity threats and risks such as personal data breaches or stolen data. Data governance principles need to be established and implemented in all organizations. The three main components of data governance are people (roles, responsibilities, working groups and committees), processes, and tools and technology. This paper presents a brief review of the various aspects of dark data and their implications for organisations.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Ackoff, R.L.: From data to wisdom. J. Appl. Syst. Anal. 16, 3–9 (1989)

    Google Scholar 

  2. Allen, G.D.: Hierarchy of knowledge – from data to wisdom. Int. J. Curr. Res. Multidisc. (IJCRM) 2(1), 15–23 (2017)

    Google Scholar 

  3. Turban, E., Rainer, R.K., Potter, R.E.: Introduction to Information Technology, 3rd edn. Wiley, New York (2005)

    Google Scholar 

  4. Snyder, H.: Literature review as a research methodology - An overview and guidelines (2019)

    Google Scholar 

  5. Braun, V., Clarke, V.: Using thematic analysis in psychology (2006)

    Google Scholar 

  6. Ashbel, A.: Dark data: a challenge enterprise data management can’t ignore. https://bluexp.netapp.com/blog/cds-blg-dark-data-a-challenge-enterprise-data-management-cant-ignore. Accessed 11 Nov 2022

  7. Splunk: The state of dark data (2019). Accessed 11 Nov 2022

    Google Scholar 

  8. Goetz, T.: Freeing the dark data of failed scientific experiment. Wired Mag. 15(10), 7–12 (2007). http://www.wired.com/science/discoveries/magazine/15-10/st_essay. Accessed 11 Nov 2022

  9. Grimm, D.J.: The dark data quandary. Am. Univ. Law Rev. 68(3), 768 (2019)

    Google Scholar 

  10. Martin, E.J.: Dark Data: Analyzing Unused and Ignored Information (2016)

    Google Scholar 

  11. Heidorn, P.B.: Shedding light on the dark data in the long tail of science. Libr. Trends 57(2), 280–299 (2008)

    Article  Google Scholar 

  12. Heidorn, P.B., Stahlman, G.R., Steffen, J.: The astrolabe project: identifying and curating astronomical ‘dark data’ through development of cyberinfrastructure resources. Astrophys. J. Suppl. Ser. 236(1), 3 (2018). https://doi.org/10.1051/epjconf/201818603003,lastaccessed2022/11/11

    Article  Google Scholar 

  13. Schembera, B., Durán, J.M.: Dark data as the new challenge for big data science and the introduction of the scientific data officer (2019)

    Google Scholar 

  14. Ajis, A.F.M., Zakaria, S., Ahmad, A.R.: Demystifying dark data characteristics in small and medium enterprises: a Malaysian experience (2022)

    Google Scholar 

  15. Gartner Inc.: Innovation Insight: File Analysis Innovation Delivers an Understanding of Unstructured Dark Data, Alan Dayley, March (2013)

    Google Scholar 

  16. Cadariu, S.: Dark Data at the Enterprise Level: What is it and What Risks Does it Pose? https://www.aitimejournal.com/dark-data-at-the-enterprise-level-what-is-is-and-what-risks-does-it-pose. Accessed 11 Nov 2022

  17. Cadariu, S.: Data Fabric and Cloud Computing as Enterprise Technologies, https://www.aitimejournal.com/data-fabric-and-cloud-computing-as-enterprise-technologies, last accessed 2022/11/11

  18. Veritas: The databerg report - see what others don’t (2015)

    Google Scholar 

  19. Dimitrov, W., Siarova, S., Petkova, L.: Types of dark data and hidden cybersecurity risks (2018)

    Google Scholar 

  20. Jackson, T.W., Hodgkinson, I.R.: Keeping a lower profile: how firms can reduce their digital carbon footprints (2022)

    Google Scholar 

  21. Intel: A vision for big data

    Google Scholar 

  22. Zikopoulos, P., Eaton, C., Dirk, D., Deutsch, T., Lapis, G.: Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data, 1st edn. Mcgraw-Hill, New York (2012)

    Google Scholar 

  23. McKinsey: Big Data: The Next Frontier For Innovation, Competition and Creativity (2011)

    Google Scholar 

  24. Schniederjans, M.J., Schniederjans, D.G., Starkey, C.M.: Business Analytics Principles, Concepts and Applications, 1st edn. Gill Editorial Services, Pearson Education, Inc., Upper Saddle River (2014)

    Google Scholar 

  25. Imdad, M., et al.: Dark Data: Opportunities and Challenges (2020)

    Google Scholar 

  26. Gimpel, G.: Dark data: the invisible resource that can drive performance now (2021)

    Google Scholar 

  27. CommVault: 5 ways to illuminate your dark data (2014)

    Google Scholar 

  28. Ryan, S.: Illuminating Dark Data (2014)

    Google Scholar 

  29. Hand, D.J.: Dark data: why what you don’t know matters (2020)

    Google Scholar 

  30. Ajis, A.F.M., Ishak, I., Harun, Q.N.: Modelling dark data management framework - a grounded theory (2022)

    Google Scholar 

  31. Bertino, E.: Data protection from insider threats. Synthesis Lect. Data Manage. 4(4), 1–91 (2012). https://doi.org/10.2200/S00431ED1V01Y201207DTM028

    Article  Google Scholar 

  32. CommVault.: Turning dark data into smart data (2014)

    Google Scholar 

  33. Kevin, N.M., Wanyaga, F.M., Kibaara, D., Dinda, W.A., Ngatia, J.K.: Dark data: business analytical tools and facilities for illuminating dark data (2016)

    Google Scholar 

  34. The Data Governance Institute: Definitions of Data Governance (2015). https://datagovernance.com/defining-data-governance/. Accessed 08 Dec 2022

  35. Newman, D., Logan, D.: Governance is an essential building block for enterprise information system. Gartner Research (2006). https://www.gartner.com/en/documents/492444. Accessed 08 Dec 2022

  36. Almeida, B.: Data governance challenges just got easier to solve (2021). https://bluexp.netapp.com/blog/clc-blg-data-governance-just-got-easier-to-solve. Accessed 11 Nov 2022

  37. Henderson, D.: DAMA-DMBOK-Data-Management-Body-of-Knowledge, 2nd edn. (2017)

    Google Scholar 

  38. Wilkinson, M.D., et al.: The FAIR guiding principles for scientific data management and stewardship. Sci. Data 3 (2016). https://www.nature.com/articles/sdata201618. Accessed 11 Nov 2022

  39. Panian, Z.: Some practical experiences in data governance (2010)

    Google Scholar 

  40. Koltay, T.: Data governance, data literacy and the management of data quality. IFLA J. 42(4), 303–312 (2016)

    Article  Google Scholar 

  41. Tallon, P.P., Ramirez, R.V., Short, J.E.: The information artifact in it governance: toward a theory of information governance. J. Manage. Inf. Syst. 30(3), 141–177 (2014)

    Article  Google Scholar 

  42. Donnelley Financial Solutions: Understanding Risk: The Dark Side of Data (2022). https://www.dfinsolutions.com/sites/default/files/documents/2022-10/DealMaker_Meter_Security_Report. Accessed 08 Dec 2022

  43. Al-Ruithe, M., Benkhelifa, E., Hameed, K.: Systematic literature review of data governance & cloud data governance (2019)

    Google Scholar 

  44. Mahanti, R.: Data governance components and framework. In: Mahanti, R. (ed.) Data Governance Success, pp. 127–166. Springer, Singapore (2021). https://doi.org/10.1007/978-981-16-5086-4_5, https://doi.org/10.1007/978-981-16-3583-0_4. Accessed 08 Dec 2022

Download references

Author information

Authors and Affiliations

  1. Stewart Barr & Associates, Adelaide, Australia

    Graham Gordon Chant

Authors
  1. Graham Gordon Chant
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Graham Gordon Chant .

Editor information

Editors and Affiliations

  1. Staffordshire University, Stoke-on-Trent, UK

    Lorna Uden

  2. National University of Kaohsiung, Kaohsiung, Taiwan

    I-Hsien Ting

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chant, G.G. (2023). Dealing with Dark Data – Shining a Light. In: Uden, L., Ting, IH. (eds) Knowledge Management in Organisations. KMO 2023. Communications in Computer and Information Science, vol 1825. Springer, Cham. https://doi.org/10.1007/978-3-031-34045-1_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-34045-1_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-34044-4

  • Online ISBN: 978-3-031-34045-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation