Towards Big Data Quality Framework for Malaysia’s Public Sector Open Data Initiative

  • Mohamad Taha IjabEmail author
  • Azlina Ahmad
  • Rabiah Abdul Kadir
  • Suraya Hamid
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10645)


This paper is about the conceptual development of the Big Data Quality Framework for Malaysia’s Public Sector Open Data Initiative (My-PSODI). At the moment, there is a lack of Big Data Quality Framework in existence particularly that is focusing on the specific context and needs of Malaysia’s Public Sector Open Data initiative. Most of existing data quality frameworks are catering the needs of traditional data types (i.e., structured data) and are very generic in nature. Due to the explosion of big data which consists mostly of unstructured data and structured data, and Malaysia’s vision of leveraging data in modernizing its service delivery, a new framework addressing the needs of Big Data for Malaysia is needed. Based on an extensive literature review, we develop a conceptual framework and systematic methodologies of how to construct the said framework to its fruition.


Big data Open data Data quality framework 


  1. 1.
    Open Data Barometer (2017).
  2. 2.
    MAMPU Analitis Data Raya Sektor Awam (DRSA): Strategi, Cabaran dan Halatuju (2013).
  3. 3.
    MAMPU. Garis Panduan Analitis Data Raya Sektor Awam – Program Kesedaran Dasar dan Garis Panduan ICT Sektor Awam (2016).
  4. 4.
    Laranjeiro, N., Soydemir, S.N., Bernardino, J.: A survey on data quality: classifying poor data. In: IEEE 21st Pacific Rim International Symposium on Dependable Computing, 18–20 November, Zhangjiajie, China (2015)Google Scholar
  5. 5.
    Khoury, M.J., Ioannidis, J.P.A.: Big data meets public health. Science 346(6213), 1054–1055 (2014). doi: 10.1126/science.aaa2709 CrossRefGoogle Scholar
  6. 6.
    Gartner, Big Data Definition (2012).
  7. 7.
    Abdel Hafez, H.A.: Mining big data in telecommunications industry: challenges, techniques, and revenue opportunity. Int. J. Comput. Electr. Autom. Control Inf. Eng. 10(1), 183–190 (2016)Google Scholar
  8. 8.
    Cai, L., Zhu, Y.: The challenges of data quality and data quality assessment in the big data era. Data Sci. J. 14(2), 1–10 (2015)Google Scholar
  9. 9.
    Saha, B., Srivastava, D.: Data quality: the other face of big data. In: IEEE 30th International Conference on Data Engineering (ICDE), 31 March–4 April, Chicago, IL (2014)Google Scholar
  10. 10.
    Chen, M., Song, M., Han, J., Haihong, E.: Survey on data quality. In: 2012 World Congress on Information and Communication Technologies (WICT), 30 October–2 November, Trivandrum, India (2012)Google Scholar
  11. 11.
    NIST: NIST Big Data Interoperability Framework, vol. 1, Definitions (2015).
  12. 12.
    Lucas, A.: Corporate data quality management: from theory to practice. In: 5th Iberian Conference on Information Systems and Technologies (CISTI), 16–19 June, Santiago, Spain (2010)Google Scholar
  13. 13.
    Abdullah, N., Ismail, S.A., Sophiayati, S., Mohd Sam, S.: Data quality in big data: a review. Int. J. Adv. Soft Comput. Appl. 7(3), 16–27 (2015)Google Scholar
  14. 14.
    Fan, W., Geerts, F.: Foundations of data management. Synth. Lect. Data Manag. 4(5), 1–217 (2012). Morgan & ClaypoolGoogle Scholar
  15. 15.
    General Administration of Quality Supervision, Inspection and Quarantine of the People’s Republic of China. Quality management systems-fundamentals and vocabulary (GB/T19000—2008/ISO9000:2005), Beijing (2008)Google Scholar
  16. 16.
    Wang, R.Y., Strong, D.M.: Beyond accuracy: what data quality means to data consumers. J. Manag. Inf. Syst. 12(4), 5–33 (1996)CrossRefGoogle Scholar
  17. 17.
    Crosby, P.B.: Quality is Free: The Art of Making Quality Certain. McGraw-Hill, New York (1988)Google Scholar
  18. 18.
    Juran, J.M.: Juran on Leadership for Quality: An Executive Handbook. The Free Press, New York (1989)Google Scholar
  19. 19.
    Alexander, J.E., Tate, M.A.: Web Wisdom: How to Evaluate and Create Information on the Web. Erlbaum, Mahwah (1999)Google Scholar
  20. 20.
    Shanks, G., Corbitt, B.: Understanding data quality: social and cultural aspects. In: Proceedings of the 10th Australasian Conference on Information Systems, pp. 785–797. MCB University Press Ltd., Wellington (1999)Google Scholar
  21. 21.
    Zhu, X., Gauch, S.: Incorporating quality metrics in centralised/distributed information retrieval on the world wide web. In: SIGIR 2000 Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 24–28 July, Athens, Greece (2000)Google Scholar
  22. 22.
    Batini, C., Scannapeico, M.: Data Quality: Concepts, Methodologies and Techniques. Springer, Berlin (2006)zbMATHGoogle Scholar
  23. 23.
    Krogstie, J.: Capturing enterprise data integration challenges using a semiotic data quality framework. Bus. Inf. Syst. Eng. 57(1), 27–36 (2015)CrossRefGoogle Scholar
  24. 24.
    Taleb, I., Dssouli, R., Serhani, M.A.: Big data pre-processing: a quality framework. In: 4th IEEE International Congress on Big Data, Santa Clara, CA 29 October–1 November (2015)Google Scholar
  25. 25.
    Juddoo, S.: Overview of data quality challenges in the context of big data. In: International Conference on Computing, Communication and Security (ICCCS), Mauritius, 4–5 December (2015)Google Scholar
  26. 26.
    Batini, C., Rula, A., Scannapieco, M., Viscusi, G.: From Data Quality to Big Data Quality, Big Data Concepts, Methodologies, Tools, and Applications, pp. 1934–1956. IGI Global, Hershey (2016)CrossRefGoogle Scholar
  27. 27.
    Ijab, M.T., Ahmad, A., Abdul Kadir, R.: Challenge of data quality: towards a big data quality framework. In: IMPACT: Technologies for Society’s Well-Being, Universiti Kebangsaan Malaysia (UKM), p. 44 (2016)Google Scholar
  28. 28.
    Economic Planning Unit - EPU Malaysia, 11th Malaysia Plan (2017).
  29. 29.
    Gurin, J.: Big Data and Open Data: What’s What and Why Does It Matter? The Guardian (2014).

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Mohamad Taha Ijab
    • 1
    Email author
  • Azlina Ahmad
    • 1
  • Rabiah Abdul Kadir
    • 1
  • Suraya Hamid
    • 2
  1. 1.Institute of Visual InformaticsUniversiti Kebangsaan Malaysia (UKM)BangiMalaysia
  2. 2.Faculty of Computer Science and Information TechnologyUniversiti MalayaKuala LumpurMalaysia

Personalised recommendations