Document Type: Original Research Paper


1 Faculty of Computer and Information Technology Engineering, Qazvin Branch, Islamic Azad University, Qazvin, Iran

2 Department of Mathematics and Computer Science, Allameh Tabataba'i University, Tehran, Iran


Background and Objectives: One of the major challenges in software engineering is how to respond to the desolate state of high-quality software development in a timely and cost-effective manner. Many studies have been conducted in an attempt to formalize the quality of software. However, according to the recent researches, the lack of comprehensive quality model is rooted in neglecting all quality aspects.
Methods: In this study, we review nineteen quality models and classify them from three different perspectives, including structural, behavioral, and basic and derived aspects. The main aim is to specify and extract the more comprehensive set of quality factors to evaluate software quality.
Results: This paper compares the different quality models and analyzes the factors to draw the necessary aspects in comprehensive quality models. Since the software quality involves several engineering tasks and several players who deal with quality concepts during software life cycle according to their various roles, in various phases and different artifacts, comprehensive quality models must consider many factors.
Conclusion: These factors are in different aspects such as the measurement time in different development phases, product as well as process-related quality factors, a set of quality metrics measureable on the different type of artifacts such as document, model and source code, and finally a specific mechanism to apply dynamic weights to quality factors to determine their impacts on final quality of a product based on its application domain. 


Main Subjects

[1] N. Gorla, S. C. Lin, “Determinants of software quality: A survey of information systems project managers,” Information and Software Technology, 52(6): 602-610, 2010.

[2] R. Nienaber, E. Cloete, “A software agent framework for the support of software project management,” in Proc. Annual Research Conference of the South African Institute of Computer Scientists and Information Technologists on Enablement Through Technology: 16-23, 2003.

[3] R. W. Hoyer, B. Y. Hoyer, “What is quality?” Quality Progress, 34(7): 52-62, 2001.

[4] F. Deissenboeck, E. Juergens, K. Lochmann, and S. Wagner., “Software quality models: Purposes, usage scenarios and requirements,” in Proc. International Conference on Software Engineering Software Quality: 9-14, 2009.

[5] R. G. Dromey, “A model for software product quality,” IEEE Transactions on Software Engineering, 21(2): 146-162, 1995.

[6] J. P. Cavano, J. A. McCall, “A framework for the measurement of software quality,” in Proc. Software Quality Assurance Workshop on Functional and Performance Issues: 133-139, 1978.

[7] B. Behkamal, M. Kahani, M. K. Akbari, “Customizing ISO9126 quality model for evaluation of B2B applications,” Information and Software Technology, 51(3): 599-609, 2009.

[8] M. F. Berto, A. Vallecillo, “Quality attributes for COTS components,” I+D Computacion, 1(2): 128-144, 2002.

[9] Y. Fernández, C. Cruz, J. L. Verdegay, “A new model based on soft computing for evaluation and selection of software products,” IEEE Latin America Transactions, 16(4): 1186-1192, 2018. 

[10] D. Coleman, B. Lowther, P. Oman, “The application of software maintainability models in industrial software systems,” Journal of Systems and Software, 29(1): 3-16, 1995.

[11] M. A. Akbar, J. Sang, A. A. Khan, F. E. Amin, M. Shafig, S. Hussain, H. Hu, M. Elahi, H. Xiang, “Improving the quality of software development process by introducing a new methodology—AZ-model,” IEEE Access, 6: 4811-4823, 2017.

[12] T. A. Alrawashdeh, M. Muhairat, A. Althunibat, “Evaluating the quality of software in ERP systems using the ISO9126 model,” International Journal of Ambient Systems and Applications (IJASA), 1(1): 1-9, 2013.

[13] J. Bansiya, C. G. Davis, “A hierarchical model for object-oriented design quality assessment,” IEEE Transactions on Software Engineering, 28(1): 4-17, 2002.

[14] L. Bettini, D. D. Ruscio, L. Iovino, , A. Pierantonio, “Quality-Driven detection and resolution of metamodel smells,” IEEE Access, 7: 16364-16376, 2019.

[15] J. A. Dallal, A. Abdin, “Empirical evaluation of the impact of object oriented code refactoring on quality attributes: A systematic literature review,” IEEE Transactions on Software Engineering, 44(1): 44-69, 2018.

[16] F. Deissenboeck, S. Wagner, M. Pizka, S. Teuchert, J. F. Girard, “An activity-based quality model for maintainability,” in Proc. IEEE International Conference on Software Maintenance (ICSM)): 184-193, 2007.

[17]  N. Fenton, “Software measurement: A necessary scientific basis,” IEEE Transactions on Software Engineering, 20(3): 199-206, 1994.

[18] N. E. Fenton, M. Neil, “A critique of software defect prediction models,” IEEE Transactions on Software Engineering, 25(5): 675-689, 1999.

[19]  X. Franch, J. P. Carvallo, “Using quality models in software package selection,” IEEE software, 20(1): 34-41, 2003.

[20] E. Georgiadou, “GEQUAMO - A generic, multilayered, customizable, software quality model,” Software Quality Journal, 11(4): 313-323, 2003.

[21] B. Kitchenham, “Towards a constructive quality model - Part 1: Software quality modelling, measurement and prediction,” Software Engineering Journal, 2(4): 105-126, 1987.

[22] B. Kitchenham, S. L. Pfleeger, “Software quality: The elusive target,” IEEE software, 13(1): 12-21, 1996.

[23] B. Kitchenham, S. Linkman, A. Pasquini, , V. Nanni, “The SQUID approach to defining a quality model,” Software Quality Journal, 6(3): 211-233, 1997.

[24] T. J. McCabe, “A complexity measure,” IEEE Transactions on Software Engineering, SE-2(4): 308-320, 1976.

[25] J. P. Miguel, D. Mauricio, G. Rodríguez “A review of software quality models for the evaluation of software products,” International Journal of Software Engineering and Applications (IJSEA), 5(6): 31-54, 2014.

[26] L. Olsina, G. Lafuente, G. Rossi, “Specifying quality characteristics and attributes for websites,” Web Engineering): 266-278, 2001.

[27] R. Ploesch, H. Gruber, G. Pomberger, M. Saft, S. Schiffer, “Tool support for expert-centred code assessments,” in Proc. The 1st International Conference on Software Testing, Verification and Validation (ICST): 258-267, 2008.

[28] R. Plösch, H. Gruber, A. Hentschel, C. Körner et al., “The EMISQ method and its tool support-expert-based evaluation of internal software quality,” Innovations in Systems and Software Engineering, 4(1): 3-15, 2008.

[29] J. Tian, “Quality-evaluation models and measurements,” IEEE software, 21(3): 84-91, 2004.

[30] P. Vizarreta, K. Trivedi, B. Helvik, P. Heegaard, A. Blenk, W. Kellerer, C. M. Machuca, “Assessing the maturity of sdn controllers with software reliability growth models,” IEEE Transactions on Network and Service Management, 15(3): 1090-1104, 2018.

[31] A. Alvaro, E.S. de Almeida, S.R. de Lemos Meira 2010. A Software Component Quality Framework. ACM SIGSOFT SEN 35, 1(Nov. 2010), 1-18, 2010.

[32] S. Wagner, Cost-Optimization of analytical software quality assurance: models, data, case studies, VDM Verlag, 2008.

[33] B. W. Boehm, Characteristics of Software Quality, 1, North-Holland, Amsterdam,:169, 1978.

[34] R. Grady, D. Caswell, Software metrics: establishing a company-wide program, Prentice Hall, 1987.

[35] M. R. Lyu, Handbook of Software Reliability Engineering, 222, CA: IEEE computer society press, 1996.

[36] J. D. Musa, Software Reliability Engineering: More Reliable Software, Faster and Cheaper, Tata McGraw-Hill Education: 632, 2004.

[37] J. D. Musa, A. Iannino, K. Okumoto, Software Reliability: Measurement, Prediction, Application. McGraw-Hill, Inc., 1987.

[38] S. Wagner, Software Product Quality Control, Berlin: Springer: 1, 2013.

[39] C. Frye, “CMM founder: Focus on the product to improve quality,” 2008.

[40] A. Alvaro, E. S. Almeida, S. R. L. Meira, “Towards a software component quality model,” presented at the 5th International Conference on Quality Software (QSIC), Bangalore, India, 2005.

[41] T. Bakota, P. Hegedűs, P. Körtvélyesi, R. Ferenc, T. Gyimóthy, “A probabilistic software quality model,” presented at the 27th IEEE International Conference on Software Maintenance (ICSM): 243-252, 2011.

[42] B. W. Boehm, J. R. Brown, M. Lipow, “Quantitative evaluation of software quality,” in Proc. The 2nd International Conference on Software Engineering: 592-605, 1976.

[43]  M. Broy, F. Deissenboeck, M. Pizka, “Demystifying maintainability,” in Proc. The 4th Workshop on Software Quality: 21-26. ACM Press, 2006.

[44] S. Neuhaus, T. Zimmermann, C. Holler, A. Zeller, “Predicting vulnerable software components,” in Proc. The 14th ACM Conference on Computer and Communications Security: 529-540, 2007.

[45] F. Radulovic, R. García-Castro, “Extending software quality models - A sample in the semantic technologies domain,” in Proc. The 23rd International Conference on Software Engineering and Knowledge Engineering (SEKE): 25-30, 2011.

[46] L. Samoladas, G. Gousios, D. Spinellis, I. Stamelos, “The SQO-OSS quality model: Measurement based open source software evaluation,” in Proc. IFIP International Federation for Information Processing, 275: 237-248, 2008.

[47] ISO/IEC., Systems and Software Engineering - Systems and Software Quality Requirements and Evaluation (Square) - System and Software Quality Models, 2011.

[48] ISO/IEC 14598: Information Technology – Software Product Evaluation, 1999.

[49] ISO/IEC 9126-1., Software Engineering - Product Quality -Part 1: Quality Model, International Organization for Standardization, Geneva, Switzerland, 2001.

[50] ISO/IEC 25000, Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE). International Organization for Standardization, Geneva, Switzerland, 2014.

[51] ISO/IEC 25022, Systems and software engineering — Systems and software quality requirements and evaluation (SQuaRE) - Measurement of quality in use, International Organization for Standardization, Geneva, Switzerland, 2016.