Software Quality Models: A Comprehensive Review and Analysis

Document Type: Research Paper


1 Department of Mathematics and Computer Science, Allameh Tabataba'i University, Tehran, Iran

2 Faculty of Computer and Information Technology Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran



One of the major challenges in software engineering is how to respond to the desolate state of high-quality software development in a timely and cost-effective manner. Software quality refers to the abilities related to the properties of software products in meeting users' requirements. Many studies have been conducted in an attempt to formalize the quality of software. However, according to the recent researches, the lack of comprehensive quality model is rooted in neglecting all quality aspects. In this study, we review nineteen quality models and classify them from three different perspectives, including structural, behavioral, and basic and derived aspects. The main aim of this paper is to specify and extract the more comprehensive set of quality factors to evaluate software quality. Moreover, this paper compares the different quality models and analyzes the factors to draw the necessary aspects in comprehensive quality models. Since the software quality involves several engineering tasks and several players who deal with quality concepts during software life cycle according to their various roles, in various phases and different artifacts, comprehensive quality models must consider many factors. These factors are in different aspects such as the measurement time in different development phases, product as well as process-related quality factors, a set of quality metrics measureable on the different type of artifacts such as document, model and source code, and finally a specific mechanism to apply dynamic weights to quality factors to determine their impacts on final quality of a product based on its application domain.

Graphical Abstract

Software Quality Models: A Comprehensive Review and Analysis


[1]     N. Gorla and S. C. Lin, “Determinants of software quality: A survey of information systems project managers,” Information and Software Technology, vol. 52, no. 6, pp. 602-610, 2010.

[2]     R. Nienaber and E. Cloete, “A software agent framework for the support of software project management,” in Proc. Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists on Enablement Through Technology, pp. 16-23, 2003.

[3]     R. W. Hoyer and B. Y. Hoyer, “What is quality?,Quality Progress, vol. 34, no. 7, p. 52, 2001.

[4]     F. Deissenboeck and E. Juergens, K. Lochmann, and S. Wagner., “Software quality models: Purposes, usage scenarios and requirements,” in Proc. International Conference on Software Engineering Software Quality, pp. 9-14, 2009.

[5]     R. G. Dromey, “A model for software product quality,” IEEE Transactions on Software Engineering, vol. 21, no. 2, pp. 146-162, 1995.

[6]     J. P. Cavano and J. A. McCall, “A framework for the measurement of software quality,” in Proc. Software Quality Assurance Workshop on Functional and Performance Issues, pp. 133-139, 1978.

[7]     B. Behkamal, M. Kahani, and M. K. Akbari, “Customizing ISO9126 quality model for evaluation of B2B applications,” Information and Software Technology, vol. 51, no. 3, pp. 599-609, 2009.

[8]     M. F. Berto and A. Vallecillo, “Quality attributes for COTS components,” I+D Computacion, vol. 1, no. 2, pp. 128-144, 2002.

[9]     A. Alvaro, E. S. Almeida, and S. R. L. Meira, “Towards a software component quality model,” presented at the 5th International Conference on Quality Software (QSIC), Bangalore, India,  2005.

[10]  Y. Fernández, C. Cruz, and J. L. Verdegay, “A new model based on soft computing for evaluation and selection of software products,” IEEE Latin America Transactions, vol. 16, no. 4, pp. 1186-1192, 2018. 

[11]  S. Wagner, “Cost-Optimization of analytical software quality assurance: models, data, case studies,” VDM Verlag, 2008.

[12]  D. Coleman, B. Lowther, and P. Oman, “The application of software maintainability models in industrial software systems,” Journal of Systems and Software, vol. 29, no. 1, pp. 3-16, 1995.

[13]  M. A. Akbar, J. Sang, A. A. Khan, F. E. Amin, M. Shafig, S. Hussain, H. Hu, M. Elahi, H. Xiang, “Improving the quality of software development process by introducing a new methodology—AZ-model,” IEEE Access, vol. 6, pp. 4811-4823, 2017.

[14]  T. A. Alrawashdeh, M. Muhairat, and A. Althunibat, “Evaluating the quality of software in ERP systems using the ISO9126 model,” International Journal of Ambient Systems and Applications (IJASA), vol. 1, no. 1, pp. 1-9, 2013.

[15]  T. Bakota, P. Hegedűs, P. Körtvélyesi, R. Ferenc, and T. Gyimóthy, “A probabilistic software quality model,” presented at the 27th IEEE International Conference on Software Maintenance (ICSM), pp. 243-252, 2011.

[16]  J. Bansiya and C. G. Davis, “A hierarchical model for object-oriented design quality assessment,” IEEE Transactions on Software Engineering, vol. 28, no. 1, pp. 4-17, 2002.

[17]  L. Bettini, D. D. Ruscio, L. Iovino, and A. Pierantonio, “Quality-Driven detection and resolution of metamodel smells,” IEEE Access, vol. 7, pp. 16364-16376, 2019.

[18]  B. W. Boehm, Characteristics of Software Quality, vol. 1, North-Holland, Amsterdam, 1978, p. 169.

[19]  B. W. Boehm, J. R. Brown, and M. Lipow, “Quantitative evaluation of software quality,” in Proc. The 2nd International Conference on Software Engineering, pp. 592-605, 1976.

[20]  M. Broy, F. Deissenboeck, and M. Pizka, “Demystifying maintainability,” in Proc. The 4th Workshop on Software Quality, pp. 21-26. ACM Press, 2006.

[21]  J. A. Dallal and A. Abdin, “Empirical evaluation of the impact of object oriented code refactoring on quality attributes: A systematic literature review,” IEEE Transactions on Software Engineering, vol. 44, no. 1, pp. 44-69, 2018.

[22]  F. Deissenboeck, S. Wagner, M. Pizka, S. Teuchert, and J. F. Girard, “An activity-based quality model for maintainability,” in Proc. IEEE International Conference on Software Maintenance (ICSM), pp. 184-193, 2007.

[23]  N. Fenton, “Software measurement: A necessary scientific basis,” IEEE Transactions on Software Engineering, vol. 20, no. 3, pp. 199-206, 1994.

[24]  N. E. Fenton and M. Neil, “A critique of software defect prediction models,” IEEE Transactions on Software Engineering, vol. 25, no. 5, pp. 675-689, 1999.

[25]  X. Franch and J. P. Carvallo, “Using quality models in software package selection,” IEEE software, vol. 20, no. 1, pp. 34-41, 2003.

[26]  C. Frye, “CMM founder: Focus on the product to improve quality,” 2008.

[27]  E. Georgiadou, “GEQUAMO - A generic, multilayered, customizable, software quality model,” Software Quality Journal, vol. 11, no. 4, pp. 313-323, 2003.

[28]  R. Grady and D. Caswell, “Software metrics: establishing a company-wide program”. Prentice Hall, 1987

[29]  ISO/IEC., Systems and Software Engineering - Systems and Software Quality Requirements and Evaluation (Square) - System and Software Quality Models, 2011.

[30]  ISO/IEC 14598: Information Technology – Software Product Evaluation, 1999.

[31]  ISO/IEC 9126-1., Software Engineering - Product Quality -Part 1: Quality Model, International Organization for Standardization, Geneva, Switzerland, 2001.

[32]  ISO/IEC 25000., Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE). International Organization for Standardization, Geneva, Switzerland, 2014.

[33]  ISO/IEC 25022., Systems and software engineering — Systems and software quality requirements and evaluation (SQuaRE) - Measurement of quality in use, International Organization for Standardization, Geneva, Switzerland, 2016.

[34]  B. Kitchenham, “Towards a constructive quality model - Part 1: Software quality modelling, measurement and prediction,” Software Engineering Journal, vol. 2, no. 4, pp. 105-126, 1987.

[35]  B. Kitchenham and S. L. Pfleeger, “Software quality: The elusive target,” IEEE software, vol. 13, no. 1, pp. 12-21, 1996.

[36]  B. Kitchenham, S. Linkman, A. Pasquini, and V. Nanni, “The SQUID approach to defining a quality model,” Software Quality Journal, vol. 6, no. 3, pp. 211-233, 1997.

[37]  M. R. Lyu, Handbook of Software Reliability Engineering, vol. 222, CA: IEEE computer society press, 1996.

[38]  T. J. McCabe, “A complexity measure,” IEEE Transactions on Software Engineering, vol. SE-2, no. 4, pp. 308-320, 1976.

[39]  J. P. Miguel, D. Mauricio, G. Rodríguez “A review of software quality models for the evaluation of software products,” International Journal of Software Engineering and Applications (IJSEA), vol. 5, no. 6, pp. 31-54, 2014.

[40]  J. D. Musa, Software Reliability Engineering: More Reliable Software, Faster and Cheaper, Tata McGraw-Hill Education, 2004, p. 632.

[41]  J. D. Musa, A. Iannino, and K. Okumoto, Software Reliability: Measurement, Prediction, Application. McGraw-Hill, Inc., 1987.

[42]  S. Neuhaus, T. Zimmermann, C. Holler, and A. Zeller, “Predicting vulnerable software components,” in Proc. The 14th ACM Conference on Computer and Communications Security, pp. 529-540, 2007.

[43]  L. Olsina, G. Lafuente, and G. Rossi, “Specifying quality characteristics and attributes for websites,” Web Engineering, pp. 266-278, 2001.

[44]  R. Ploesch, H. Gruber, G. Pomberger, M. Saft, and S. Schiffer, “Tool support for expert-centred code assessments,” in Proc. The 1st International Conference on Software Testing, Verification and Validation (ICST), pp. 258-267, 2008.

 [45]  R. Plösch, H. Gruber, A. Hentschel, C. Körner et al., “The EMISQ method and its tool support-expert-based evaluation of internal software quality,” Innovations in Systems and Software Engineering, vol. 4, no. 1, pp. 3-15, 2008.

[46]  F. Radulovic and R. García-Castro, “Extending software quality models - A sample in the semantic technologies domain,” in Proc. The 23rd International Conference on Software Engineering and Knowledge Engineering (SEKE), pp. 25-30, 2011.

[47]  L. Samoladas, G. Gousios, D. Spinellis, and I. Stamelos, “The SQO-OSS quality model: Measurement based open source software evaluation,” in Proc. IFIP International Federation for Information Processing, vol. 275, pp. 237-248, 2008.

[48]  J. Tian, “Quality-evaluation models and measurements,” IEEE software, vol. 21, no. 3, pp. 84-91, 2004.

[49]  P. Vizarreta, K. Trivedi, B. Helvik, P. Heegaard, A. Blenk, W. Kellerer, and C. M. Machuca, “Assessing the maturity of sdn controllers with software reliability growth models,” IEEE Transactions on Network and Service Management, vol. 15, no. 3, pp. 1090-1104, 2018.

[50] S. Wagner, Software Product Quality Control, Berlin: Springer, 2013, pp. 1-12.