A method to evaluate quality of modelling languages based on the Zachman reference taxonomy

  • Fáber D. GiraldoEmail author
  • Sergio España
  • William J. Giraldo
  • Óscar Pastor
  • John Krogstie


The model-driven engineering (MDE) paradigm promotes the use of conceptual models in information systems (IS) engineering and research. As engineering products, conceptual models must be of high quality, which applies to both conceptual models and the modelling language used to build them. Quality is a growing concern in the MDE field; however, studies such as Giraldo, F.D. et al. Software Quality Journal, pp. 1–66 (2016b) and Goulão, M. et al. Software Quality Journal, pp. 1–33 (2016) demonstrate the divergence in several approaches that are proposed for addressing this topic. Due to the many challenges, divergences, and trends for quality assessment and assurance in the MDE context, one way to perform a quality evaluation process is to use an approach where the applicability and goals of modelling languages (and artifacts) can be compared with respect to the essential principles of the development of IS. We propose using principles from an IS architecture reference (i.e., the Zachman framework) as a taxonomy that is applied on the modelling languages used in information system development in order to perform analytic procedures. We also demonstrate that this taxonomy can be considered as a formal context for the application of the formal concept analysis (FCA) method. This paper derives formal, methodological, and technological requirements for a modelling language quality evaluation method (MMQEF) with the potential to tackle some of the open MDE quality challenges. In addition, a tool that operationalizes the taxonomic evaluation procedure and the FCA analytic method is also presented. In this work, we discuss how this taxonomy supports analytics that are in modelling languages for quality purposes through its management of the semantics.


Quality Model-driven engineering Information systems Modelling language evaluation Reference taxonomy The MMQEF method 



F.G. would like to thank COLCIENCIAS (Colombia) for funding this work through the COLCIENCIAS Grant call 512-2010. This work has been supported by the Generalitat Valenciana Project IDEO (PROMETEOII/2014/039), the European Commission FP7 Project CaaS (611351), and ERDF structural funds. F.G. would to thank César A. Cataño and Juan D. Fernández for their support in the implementation of EMAT tool.


  1. Aagesen, G., & Krogstie, J. (2015). BPMN 2.0 for modeling business processes, (pp. 219–250). Heidelberg: Berlin.Google Scholar
  2. da Silva, A.R. (2015). Model-driven engineering: a survey supported by the unified conceptual model. Computer Languages, Systems & Structures, 43, 139–155.MathSciNetCrossRefGoogle Scholar
  3. de Graaf, K., Liang, P., Tang, A., van Hage, W., van Vliet, H. (2014). An exploratory study on ontology engineering for software architecture documentation. Computers in Industry, 65(7), 1053–1064.CrossRefGoogle Scholar
  4. de la Vara, J.L., Díaz, J.S., Pastor, O. (2007). Integración de un entorno de producción automática de software en un marco de alineamiento estratégico (in Spanish). In Anais do WER07 - workshop em engenharia de requisitos, Toronto, Canada, May 17-18, 2007 (pp. 68–79).Google Scholar
  5. Espańa, S., González, A., Pastor, Ó. (2009). Communication analysis: a requirements engineering method for information systems, (pp. 530–545). Berlin: Springer.Google Scholar
  6. España, S., Ruiz, M., González, A. (1). Systematic derivation of conceptual models from requirements models: a controlled experiment. In 2012 6th international conference on research challenges in information science (RCIS).Google Scholar
  7. España Cubillo, S. (2012). Methodological integration of communication analysis into a model-driven software development framework. PhD thesis.Google Scholar
  8. Frankel, D.S., Harmon, P., Mukerji, J., Odell, J., Owen, M., Rivitt, P., Rosen, M., Soley, R.M. (2003). The Zachman Framework and the OMG’s model driven architecture.Google Scholar
  9. Ganter, B., & Wille, R. (1999). Concept lattices of contexts, (pp. 17–61). Berlin: Springer.Google Scholar
  10. Garner, B., & Raban, R. (1999). Context management in modeling information systems (is). Information and Software Technology, 41(14), 957–961.CrossRefGoogle Scholar
  11. Giraldo, F.D., España, S., Pastor, O. (2016a). Evidences of the mismatch between industry and academy on modelling language quality evaluation. CoRR, arXiv:1606.02025.
  12. Giraldo, F.D., España, S., Pastor, Ó. , Giraldo, W.J. (2016b). Considerations about quality in model-driven engineering. Software Quality Journal, pp. 1–66.Google Scholar
  13. González, A., España, S., Ruiz, M., Pastor, Ó. (2011). Systematic Derivation of Class Diagrams from Communication-Oriented Business Process Models, pages 246–260 Springer Berlin Heidelberg. Heidelberg: Berlin.Google Scholar
  14. Goulão, M., Amaral, V., Mernik, M. (2016). Quality in model-driven engineering: a tertiary study. Software Quality Journal, pp. 1–33.Google Scholar
  15. Guarino, N., & Welty, C.A. (2000). A formal ontology of properties. In Proceedings of the 12th European workshop on knowledge acquisition, modeling and management, EKAW ’00 (pp. 97–112). London: Springer.Google Scholar
  16. Henderson-Sellers, B., & Gonzalez-Perez, C. (2010). Granularity in conceptual modelling: application to metamodels. In Parsons, J., Saeki, M., Shoval, P., Woo, C., Wand, Y. (Eds.) Conceptual modeling - ER 2010, volume 6412 of lecture notes in computer science (pp. 219–232). Berlin: Springer.Google Scholar
  17. ISO/IEC/IEEE. (2011). Iso/iec/ieee systems and software engineering – architecture description. ISO/IEC/IEEE 42010:2011(E) (Revision of ISO/IEC 42010:2007 and IEEE Std 1471-2000), pp. 1–46.Google Scholar
  18. Kingston, J. (2008). Multi-perspective ontologies: resolving common ontology development problems. Expert Systems with Applications, 34(1), 541–550.CrossRefGoogle Scholar
  19. Kingston, J., & Macintosh, A. (2000). Knowledge management through multi-perspective modelling: representing and distributing organizational memory. Knowledge-Based Systems, 13(2-3), 121–131.CrossRefGoogle Scholar
  20. Krogstie, J. (2012). Quality of models, (pp. 205–247). London: Springer.Google Scholar
  21. Kruchten, P. (2000). The rational unified process: an introduction, 2nd. Boston: Addison-Wesley Longman Publishing Co., Inc.Google Scholar
  22. Laware, G., & Kowalkowski, F. (2005). The business value of taxonomies and ontologies for web & knowledge management practices. pp. 41–47. cited By 0.Google Scholar
  23. Martin, R., & Robertson, E.L. (1999). Formalization of multi-level zachman frameworks (technical report no. 522). Technical report, Computer Science Department, Indiana University.Google Scholar
  24. Mohagheghi, P., Dehlen, V., Neple, T. (2009). Definitions and approaches to model quality in model-based software development - a review of literature. Information and Software Technology, 51(12), 1646–1669. Quality of {UML} Models.CrossRefGoogle Scholar
  25. Molina, A.I., Giraldo, W.J., Ortega, M., Redondo, M.A., Collazos, C.A. (2014). Model-driven development of interactive groupware systems: integration into the software development process. Science of Computer Programming, 89(Part C(0)), 320–349.CrossRefGoogle Scholar
  26. Moody, D. (2009). The physics of notations: toward a scientific basis for constructing visual notations in software engineering. IEEE Transactions on Software Engineering, 35(6), 756–779.CrossRefGoogle Scholar
  27. Muntermann, J., Nickerson, R., Varshney, U. (2015). Towards the development of a taxonomic theory. In Proceedings of the Twenty-first Americas Conference on Information Systems (AMCIS), Puerto Rico, 2015, AMCIS’15. AIS Electtronic Library (AISeL).Google Scholar
  28. Noran, O. (2003). An analysis of the zachman framework for enterprise architecture from the {GERAM} perspective. Annual Reviews in Control, 27(2), 163–183.CrossRefGoogle Scholar
  29. Object and Reference Model Subcommittee of the Architecture Board. (2005). A Proposal for an MDA Foundation Model ormsc/05-04-01. Technical report, Object Management Group.Google Scholar
  30. Olivé, A. (2001). Taxonomies and derivation rules in conceptual modeling. In Proceedings of the 13th International Conference on Advanced Information Systems Engineering, CAiSE ’01 (pp. 417–432). London: Springer.Google Scholar
  31. OMG. (2003). Mda guide version 1.0.1.Google Scholar
  32. OMG. (2014). MDA Guide revision 2.0.Google Scholar
  33. Pastor, Ó., & España, S. (2012). Full model-driven practice: from requirements to code generation, (pp. 701–702). Berlin: Springer.Google Scholar
  34. Pastor, O., Insfrán, E., Pelechano, V., Romero, J., Merseguer, J. (2013). OO-METHOD: an OO software production environment combining conventional and formal methods, (pp. 139–152). Berlin: Springer.Google Scholar
  35. Pastor, O., & Molina, J.C. (2007). Model-driven architecture in practice: a software production environment based on conceptual modeling. New York: Springer.Google Scholar
  36. Priss, U. (2006). Formal concept analysis in information science. Annual Rev. Info. Sci & Technol., 40(1), 521–543.CrossRefGoogle Scholar
  37. Romero, J., Jaen, J., Vallecillo, A. (2009). Realizing correspondences in multi-viewpoint specifications. In Enterprise distributed object computing conference, 2009. EDOC ’09. IEEE International (pp. 163–172).Google Scholar
  38. Rueda, U., España, S., Ruiz, M. (2015). GREAT Process modeller user manual. CoRR, arXiv:1502.07693.
  39. Shuman, E.A. (2010). Understanding executable architectures through an examination of language model elements. In Proceedings of the 2010 summer computer simulation conference, SCSC ’10 (pp. 483–497). San Diego: Society for Computer Simulation International.Google Scholar
  40. Siau, K., & Rossi, M. (1998). Evaluation of information modeling methods - a review. In HICSS (5) (pp. 314–322).Google Scholar
  41. Smith, R. (2013). On the value of a taxonomy in modeling. In Tolk, A. (Ed.) Ontology, epistemology, and teleology for modeling and simulation, volume 44 of intelligent systems reference library (pp. 241–254). Berlin: Springer.Google Scholar
  42. Sowa, J.F., & Zachman, J.A. (1992). Extending and formalizing the framework for information systems architecture. IBM Systems Journal, 31(3), 590–616.CrossRefGoogle Scholar
  43. Tang, A., Han, J., Chen, P. (2004). A comparative analysis of architecture frameworks. In 2004 11th Asia-pacific software engineering conference (pp. 640–647).Google Scholar
  44. Wegmann, A., Kotsalainen, A., Matthey, L., Regev, G., Giannattasio, A. (2008). Augmenting the zachman enterprise architecture framework with a systemic conceptualization. In Proceedings of the 2008 12th international IEEE enterprise distributed object computing conference, EDOC ’08 (pp. 3–13). Washington: IEEE Computer Society.Google Scholar
  45. Welty, C., & Guarino, N. (2001). Supporting ontological analysis of taxonomic relationships. Data Knowledge Engineering, 39(1), 51–74.CrossRefGoogle Scholar
  46. Wolff, K.E. (1993). A first course in formal concept analysis. StatSoft, 93, 429–438.Google Scholar
  47. Zachman, J.A. (1987). A framework for information systems architecture. IBM Systems Journal, 26(3), 276–292.CrossRefGoogle Scholar
  48. Zhao, L., Letsholo, K., Chioasca, E.-V., Sampaio, S., Sampaio, P. (2012). Can business process modeling bridge the gap between business and information systems?. In Proceedings of the 27th Annual ACM Symposium on Applied Computing, SAC ’12 (pp. 1723–1724). New York: ACM.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.SINFOCI Research GroupUniversity of QuindíoArmeniaColombia
  2. 2.Department of Information and Computing SciencesUtrecht UniversityUtrechtNetherlands
  3. 3.PROS Research CentreUniversitat Politècnica de ValènciaValenciaSpain
  4. 4.Department of Computer and Information ScienceNorwegian University of Science and TechnologyTrondheimNorway

Personalised recommendations