Introduction

IASC

Intelligent Automation & Soft Computing

2326-005X 1079-8587

Tech Science Press

USA

14639

10.32604/iasc.2021.014639

Article

CMMI Compliant Workflow Models to Establish Configuration Management Integrity in Software SMEs

Ali

Islam

1 Khan

Musawwer

1 Mehmood

Waqar

1 Nisar

Wasif

1 Aslam

Waqar

2 Saleem

Muhammad Qaiser

3 Omer

Majzoob K.

3 Shafiq

Muhammad

4 shafiq@ynu.ac.kr 1Department of Computer Science, COMSATS University Islamabad, Wah Campus, Wah Cantt, Pakistan 2Department of Computer Science & IT, The Islamia University of Bahawalpur, Bahawalpur, Pakistan 3College of Computer Science and Information Technology, Al Baha University, Al Bahah, Saudi Arabia 4Department of Information and Communication Engineering, Yeungnam University, Gyeongsan, 38541, Korea

*Corresponding author: Muhammad Shafiq. Email: shafiq@ynu.ac.kr

01 01 2021

27 3 605 623 05 10 2020 30 11 2020

2021

Ali et al.

This work is licensed under a Creative Commons Attribution 4.0 International License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Capability Maturity Model Integration (CMMI) is a world-renowned framework for software process improvement, which specifies “What-To-Do” in terms of requirements. However, it leaves the “How-To-Do” part regarding implementation to implementers. The software industry especially software SMEs (SSMEs) faces difficulties in implementing the Specific Practices (SPs) of Various Process Areas (PAs). Configuration Management Process Area (CM-PA) is usually ignored despite its acknowledged importance in the software development process. Establishing integrity is one of the three Specific Goals (SGs) that CMMI ver. 1.3 requires for successful implementation of CM-PA. This goal is achieved through the implementation of two SPs (i.e., 3.1 and 3.2). In order to enable aforesaid SSMEs, pertinent research work regarding the implementation of PAs at CMMI Level-II was studied and Workflow Models (WFMs) were devised after sifting through all the relevant material. The models were assessed through Expert Panel Review (EPR) and further confirmed by conducting case studies. This work also contributes to the implementation of CM-PA. The results from EPR and case studies are promising since they not only testifies the clarity, learnability, usability, usefulness of the models but also proves its applicability to SSMEs. The proposed WFMs have a strong theoretical basis and practically proven. More industrial case studies are suggested to evaluate models for the upcoming versions of CMMI frameworks.

Software configuration management capability maturity model integration software process improvement software SMEs

1 Introduction

The success or failure of an organization largely hinges on quality of products or services it provides. Everyone desires to have software product(s) that operate reliably without errors or being crashed. One way to enhance software quality is to improve software development processes. That’s why many SSMEs take interest in SPI. Improving the software process continually and appraising it regularly for effectiveness helps in meeting the customer’s expectations and is bound to pave a way towards a high-quality software.

There is no doubt that CMMI enables software development industry to take quality of software process to a next higher level. However, no significant number of SSMEs are opting for adoption. Like many other researcher, Gang Xu et al. [1] pointed out that CMMI offers software companies only guidelines, not the clear workflow models resulting in increased budget.

CMMI Level-II [2] consists of seven PAs including CM-PA. As elaborated in the next section, variety of research work has been carried out for implementation of PAs at CMMI Level-II, However, presently, no workflow model was found for SPs wise implementation of CM-PA particularly to help SSMEs as shown in Tab. 1. Therefore there is an intense need to devise the tailorable workflow models for SPs of CM-PA.

Table 1 Summary of workflow models devised earlier for various SPs of PAs at CMMI Level-II

No	PAs at CMMI Level-II	Work	Reference
1	Configuration Management (CM)	X	X
2	Measurement & Analysis (M&A-PA)	X	X
3	Project Monitoring & Control (PM&C)	X	X
4	Requirements Management (REQM)	RCM, WFM for SP 1.3 & 1.4, REQM	[3–6]
5	Project Planning (PP)	WFM for SP 1.3	[7]
6	Process and Product Quality Assurance (PPQA)	WFM for All SPs	[8]
7	Supplier Agreement Management (SAM)	WFM	[9]

As per CMMI for Dev Ver. 1.3, CM-PA has three SGs and are achieved through implementation of seven SPs collectively. The focus of this study is to achieve the third goal (SG-3) of CM-PA by devising WFMs for implementation of two associated SPs (i.e., 3.1 and 3.2). As a vehicle to achieve the research objective, research questions are formulated as given in Tab. 2.

Table 2 Research questions

ID	Research Question	Motivation
RQ-A	How to implement the associated SPs to achieve “Establish Integrity” goal of CM-PA at CMMI Level-II in SSMEs?	To devise WFMs for implementation of SPs contributing to SG-3 of CM-PA and
RQ-B	What is the expert’s perception about “Practice Coverage” of the proposed WFMs specifically w.r.t SSMEs?	assess the coverage of sub-practices,
RQ-C	What is the expert’s perception about “Usefulness” of the proposed WFMs taking SSMEs into account?	evaluate its utility
RQ-D	What is the expert’s perception about “Ease of Learning & Usage” of the proposed WFMs in the context of SSMEs?	appraise its ease of learning, usability &
RQ-E	What is the expert’s perception about “Applicability” of the proposed WFMs to SSMEs?	Judge its implement-ability in SSMEs.

Organization of the paper is as follows. Sec 2 traverses through the earlier work and its limitations. Sec 3 throw light on methodology adopted and criteria for validation of the workflow model. Proposed models are elaborated in sec 4. Validation of models, threats to validity and their mitigation is given in sec 5, Sec 6 concludes the study and finally sec 7 highlights potential future work.

2 Related Work

In order to help software development firms in implementing the best practices of REQM-PA, Niazi et al. [3] devised the CMMI compliant Requirements Change Management (RCM) Model. The model has five stages “Request”, “Validate”, “Implement”, “Verify” & “Update” and was evaluated through EPR process. Keshta [4] devised WFMs for SPs 1.3 & 1.4 of the REQM-PA having six stages “Initiate”, “Validate”, “Implement”, “Verify”, “Update” and “Release”. EPR was used to validate the models against the specified criteria. Applicability of models to small & medium sized software development organizations (SMSDOs) was evaluated in Saudi Arabian software industry. Tariq [5] has suggested to include an additional SP in REQM-PA for Software as a Services (SAAS) and carried out validation through a case study “Allwebid”. Batti [6] proposed a six-phased methodology to deal with changing requirements i.e., “Initiate”, ”Receipt”, ”Approve/Disapprove”, ”Evaluate”, ”Implement” and ”Configure” with CCB to act as central player and as a process owner. Keshta [7] also devised a WFM for implementation of SP 1.3 of PP-PA and defined phases for a project life cycle keeping in view the SMSDOs. The model comprises of four stages “Plan”, “Design”, “Review” and ”Update/Rework”. Keshta [8] further developed a WFMs for all SPs of PPQA-PA in perspective of SMSDOs. Both SP-1.1 & SP-1.2 of PPQA comprise of four stages i.e., “Plan”, “Prepare”, “Audit” & “Report”. The models were validated making use of EPR. Vivatanavorasin et al. [9] presented a three layered WFM for SAM-PA having “Contextual layer”, “Elaboration layer”, and “Definition layer”. As a proof of concept prototype, Supplier Agreement Management Tool was developed. In order to adopt CM process in DevOps environment, Erik Hochbergs and Laroy Nilsson Sjödahl [10] prepared guidelines after exploring literature and interviewing key professionals of software companies. Syahrul Fahmy et al. [11] highlighted the evolution of SCM since its beginning and appreciated that SCM techniques are also being applied to other areas.

Resources are meagre in small software companies (SSCs) as compared to medium and large companies. Tuape and Ayalew [12] underscored that SPI frameworks are usually framed keeping big companies in view and thus software quality is usually compromised in SSCs. The authors identified three factors that tend to affect development process in SSC’s generally and African SSCs particularly. Victor José et al. [13] worked on how to implement the measurement process in line with the CMMI in companies whose primary business is maintenance instead of development. Tadele [14] devised a simple and easy to use framework amalgamating the CMMI ver 2.0 and DevOps to assist small companies. This framework is claimed to be comparatively cheaper & easily implementable and was validated through case study in few companies where substantial improvement was seen after implementation. Definition of SMEs varies w.r.t countries and time span. Few, collected from various studies, are given in Tab. 3.

Table 3 Categorization of SMEs

Country	Small Company	Medium Company	Reference
Country	Number of employee	Number of employee	Reference
Pakistan	10~35	36~99	Dasanayaka [15]
Korea	11~49	50~199	Sanath Divakara [16]
Turkey	03~49	50~250	Hande Karadag [17]
Saudi Arabia	06~49	50~249	Abhishek Tripathii [18]

3 Methodology Adopted and Criteria for Validation of Workflow Model

Research methodology need to be devised very carefully as it has profound impact on the validity and reliability of study results. The research methodology used for this research has six major stages and is illustrated in Fig. 1.

Figure 1 Methodology adopted

Success of a study largely depends on formulating of a sound evaluation criteria. Criteria for validation of the models in this study, because of the similar nature, has been derived from work of Niaz [3], Keshta [4,7,8] and Vivatanavorasin [9] respectively and is elaborated in Tab. 4.

Table 4 Validation criteria

Criteria	Elaboration
Satisfaction of SPs	The proposed WFMs should address the practices where necessary to ensure achieving the goals set by CMMI v1.3 specifications.
Satisfaction of Users	Models’ should satisfy users and help them to achieve their needs and objectives.
Ease of Learning & Use	Models shall be simple, easy to understand and comfortable to follow.
Applicability of the models in SSMEs.	The WFMs shall be implementable in SSMEs i.e., it shall enable them to achieve the integrity goal of CM-PA.

4 Proposed Workflow Models for Establishing Integrity

As per CMMI Framework 1.3, the SG-1 “Establish Baselines” of CM-PA serve to establish baselines, the SG-2 “Track and Control Changes” assist in maintaining the baselines whereas the SG-3 “Establish Integrity” basically establish records and appraise the integrity of the baselines. The later goal is achieved through implementation of two SPs. The proposed WFMs for the aforesaid SPs, in this work, are composed of core stages. In fact, the activities involved in a particular SP have been logically grouped with logical sequence into stages. The proposed WFMs are constructed using well known Entry-Task-Verification-eXit (ETVX) model. Each activity is accompanied with the actor having generic title who has to perform it and the potential artifacts to be created. These actors are taken from a sample SSMEs. The implementers may tailor it as per their working environment. Further, inputs and outputs of the workflow along with the associated processes/stages are also indicated.

4.1 WFM for SP 3.1 – “Establish Configuration Management Records”

First SP of the said goal is to “Establish and maintain records describing the CIs”. In order to keep brevity, only three among many of the findings from literature are given in Tab. 5 supporting each stage i.e., “Planning”, “Recording”, “Revision”, and “Sharing Reports” of the proposed WFM for SP 3.1 of CM-PA and is depicted in Fig. 2 followed by the associated process guide in Tab. 6. Rationale for provision of process guide in tabular format is to achieve brevity and to provide structured information to implementers.

Table 5 Evidences from literature supporting each stage of WFM for SP 3.1

No.	Evidence from Literature	Author’s Point of View	Author & reference
A - Planning Stage
1	CMMI for Development	It’s an organizational prerogative, due to supportive nature of the PA, to select CIs and its control level. In fact, understanding CIs status is quite time-taking without adequate description of CIs.	Chrissis et al. [19]
2	Introduction to Software Process Improvement	G. Oregan emphasized to include the activity of establishing records in the process, make part of checklists as well as template.	G. O’Regan [20]
3	Guide to Software Engineering Body of Knowledge V 3.0	SCM Planning shall be consistent with the organizational context and project plan. It terms SCM Plan as living document serving as a reference for the SCM process.	SWEBOK V 3.0 [21]
B – Recording Stage
1	Introduction to Software Quality	G. O’Regan suggested a role of librarian to establish a library (filing structure) for to record CM activities. Configuration manager may act as librarian.	G. O’Regan [22]
2	CMMI for Development	Chrisis emphasized that ample information be recorded to be able to maintain the CIs differentiation between baselines easily.	Chrissis et al. [19]
3	WFM for SP 2.2 of PPQA.	The author included the “Record” stage in WFM for SP “Establish Records” of PPQA-PA with evidences from literature.	Keshta [8]
C - Revision Stage
1	CMMI for Development	The author considers the version control as critical and suggested “Sequential” as standard way for identification of versions.	Chrissis et al. [19]
2	Introduction to Software Quality	G. O’Regan stressed that on each change in document, next version shall be assigned and history shall be updated.	G. O’Regan [22]
3	WFM for SP 2.2 of PPQA	The author included the “Revise” stage in WFMs for SP “Establish Records” of PPQA-PA with evidences from literature.	Keshta [8]
D - Sharing Reports Stage
1	Workflow Model for PPQA-PA	The author included the “Report” stage in both WFMs for SP 1.1 and SP 1.2 of PPQA-PA with evidences from literature.	Keshta [8]
2	Introduction to Software Quality	The Status Accounting Reports shall include Baseline Status, Baseline Differences, Problems reports, Change Request etc.	O’Regan [22]
3	Workflow Model for PPQA-PA	The author included the “Share” stage in workflow models for SP “Establish Records” of PPQA-PA with evidences from literature.	Keshta [8]

Figure 2 Workflow model for SP-3.1

Table 6 Process guide for SP 3.1

Purpose	In order to maintain control over the configuration of project CIs and provide management status of the project, CM records of the CIs needs to be recorded throughout the SDLC.
Scope	This process applies to all activities that are performed throughout the project life cycle.
Abbreviations	SAR BDR		Status Accounting ReportsBaselines Difference Report
Entry criteria	A Baseline has been released to the client. A request for the status has been received from PM through email
Inputs to the workflow and associated SPs	Input Work-Products			Associated PA/SP
	SCM Plan Organizational/Project Measures/Metrics Configuration Management System (CMS)			PP-PA Status Accounting Information Requirements Establish CMS
Stage	Process Activities				Activity Roles	Potential Records
A. Planning	A.1	Status accounting information are identified/selected.			PM	Status Accounting Information
	A.2	The version of CIs that constitute the baselines			SCM TM	–
	A.3	The latest version of the baselines shall be specified.			SCM TM	–
B. Recording	B.1	The Status accounting information shall be incorporated into SCM repository.			SCM TL, SCM TM	Rev. Repository Structure
	B.2	Authorized users shall have access to the repository.			SCM TM	Roles/privileges
	B.3	The SCM repository shall be configured to auto-inform authorized users about change of any CI.			SCM TM	Roles/privileges
	B.4	In order to know the content and status of CIs and render the older versions recoverable, all SCM activities shall be recorded with sufficient details.			SCM TM	Logs
C. Revision	C.1	The changes shall be identified based on updates.			SCM TM	–
	C.2	Following shall be confirmed prior to revision of status and preparing history of CIs. The old version be recoverable. Status accounting info are incorporated into SCM Repository and access provided to users The repository be configured to send the alerts automatically on change of CIs. No errors are there in any CIs/records.			SCM TMs	Corrective Actions
	C.3	Status of CIs shall be revised and revision history shall be prepared accordingly.			SCM TMs	Revision History of CIs
E. Sharing Reports	D.1	SCM Reports shall be generated periodically or on demand including structure reports, add-hoc reports and difference b/w successive baselines reports.			SCM TMs, SCM TL	SCM Report
	D.2	SCA reports shall be distributed to stakeholders.			SCM TL	Dissemination log
Interfaces	Artifacts generated from PP, PPQA, PMC, REQM, MA, and SCM Process Areas are referred to SCM process area to be stored and maintained in the CMS. QC department performs the testing according the defined process and notifies SCM about the resolution of all bugs or the completion of functionality. The schedule of configuration audit is received from the Project Planning Process Area
Outputs of WFM and associated PAs/SPs	Output Work-Products				Associated PA/SP
	SCM Reports Corrective Actions Internal Configuration Checklist, NCs, Audit Reports				Perform Configuration AuditsPPQA-PAEstablish CM Records, PPQA
Exit criteria	SA Reports and Baselines Difference Reports are generated/viewed/evaluated. Product has been released to the client and acceptance from client has been received.
Measures	Number of releases.
Verification points	PM in coordination with Dev TL and SCM TL reviews the Configuration Status Accounting process and work products at points identified by the Project Plan and Project Schedule. QA evaluates the Configuration Status Accounting process and designated work products. Top Management periodically reviews the Status Accounting activities. Refer PMC Process.
Training	Training on Configuration Status Accounting process/Templates
Tools	MS Word, MS Excel, VSS/TFS/Any CM Tool
Assumptions	PM may request the project status/baseline difference report any time throughout the SDLC. Frequency of reports generation can be defined as per the project’s needs. SCM TL verifies the updates in the status report on weekly basis.
Exemptions	Tailoring Guidelines
Applicable standards & documents	Documentation Standards Manual, PMC Process Guide/MA Process Guide SCM Report template

4.2 WFM for SP-3.2 of CM-PA “Perform Configuration Audits”

The purpose of SP-3.2 is to appraise the integrity of the baselines. As per CMMI, CM Audit is defined as “Audit is to verify that a CIs or a collection of CIs that make up a baseline conforms to a specified standard or requirement”. Only three among the findings from literature are given in Tab. 7 supporting each stage of the aforementioned WFM i.e., “Planning”, “Preparation”, “Conduction” and “Closure” and is depicted in Fig. 3 followed by the Process Guide in Tab. 8.

Table 7 Evidences from literature supporting each stage of WFM for SP 3.2

No.	Evidence from Literature	Author’s Point of View	Author & reference
A - Planning Stage
1	International Journal of Government Auditing, International Standard of Supreme Audit Institutions (ISSAIs).	Performance audit guides published by various member countries e.g., Bangladesh, Kosovo, India, Pakistan etc. all follow the same standard that has four main stages namely “Planning”, “Execution”, “Reporting”, and “Follow-up”.	INTOSAI [23,24]
2	Workflow model for Audit	Planning is essential for any activity and audit is not an exception. Planning is included as a stage in this workflow.	SASQAG [25]
3	WFMs for SP-1.1 and SP-1.2 of PPQA-PA	The “Plan” stage was included in WFMs for auditing the said SPs with evidences from literature.	Keshta [8]
B – Preparation Stage
1	Workflow model for Audit	An audit is as much successful as how much “Preparation” has been carried out prior to the conduction of audit and hence necessary to include this step in the said workflow.	SASQAG [25]
2	Introduction to Software Process Improvement.	Prepared an audit 15 points checklist to help CM Auditor.	O’Regan [20]
3	WFMs for SP-1.1 and SP-1.2 of PPQA-PA	The “Prepare” stage in WFMs for auditing both the SPs was included with adequate evidences from literature.	Keshta [8]
C – Conduction Stage
1	Audit Guide	The “Execution” in ISSAI is similar to “Conduction” stage.	Audit Guide [24]
2	Workflow model for Audit	The SASQAG Audit Workflow includes the “Conduct Audit” as a major step.	SASQAG [25]
3	WFMs for SP-1.2 of PPQA-PA	“Audit” stage is included in WFMs for both with adequate evidences from literature.	Keshta [8]
D – Closure Stage
1	Workflow model for Audit	The SASQAG Audit Workflow includes the “Close Out Audit” as a major step and is similar to closure stage.	SASQAG [25]
2	ISSAIs	The activities carried out in this stage of proposed WFM are addressed in “Reporting” & “Follow-up” stages of ISSAI.	Audit-Guide [24]
3	WFMs for SP-1.2 of PPQA-PA	“Report” stage is included with evidences from literature. Basically reporting is covered in this stage in our WFM.	Keshta [8]

Figure 3 Workflow model for SP-3.2

Table 8 Process guide for SP 3.2

Purpose	Configuration audits confirm that the resulting baselines/documentation conform to a specified standard and to ensure accuracy, consistency, and completeness of CIs.
Scope	Scope of this process is to audit the CM department where integrity of all the project artifacts is evaluated. Audit scope is specified in the audit plan by the management.
Abbreviations	NCs SCML CMA		Non-conformancesSoftware Configuration Management LibraryConfiguration Management Audit
Entry criteria	Once notification for conducting audit is received as per the Configuration Management Plan
Inputs to the workflow and associated PAs/SPs	Input Work-Products			Associated PA/SP
	Project Plan/SCM Plan CIs and Baselines Change Requests Configuration Management System (CMS)			PP-PACreate BaselinesTrack Change RequestsEstablish CMS
Stage	Process Activities				Activity Roles	Potential Records
A. Planning	A.1	Organizational level quality standards, processes, plans are established.			PM, SCM TL, Dev TL	Processes, Standards, Plans
	A.2	An independent auditor is identified and audit criteria for SCM Audit is specified.			PM, SCM TL, SQTL	Audit Criteria
	A.3	Employees shall be encouraged to participation in identifying & reporting CM Issues.			PM, SCM TL	SCM Issues
	A.4	The SCM Audit Plan shall be finalized.			PM, SCM TL, SQA TL	Correspondence with Auditor
B. Preparation	B.1	Checklist is prepared/reviewed for SCM Audit.			SQA/Auditor	SCM Audit Checklist
	B.2	Internal Audit shall be carried out prior to as a preparation to external audit.			SQA TL/PPQA	Internal Audit Report
	B.3	SCM shall be facilitated in obtaining requisite info/work products.			PM	–
	B.4	Schedule is confirmed and communicated to stakeholders.			PM, SQA TL, SCM TL	Corresp. with stakeholders
C. Conduction	C.1	SQA TL shall act a facilitator to the Auditor.			SQA TL	–
	C.2	The integrity of baselines are assessed as per defined audit criteria			SCM Auditor	Audit Notes
	C.3	CMS records and CIs shall be tallied for connect identification.			SCM Auditor	Audit Notes
	C.4	Structure and integrity of CIS in CMS shall be reviewed			SCM Auditor	Audit Notes
	C.5	Disposition of change requests shall be checked.			SCM Auditor	Audit Notes
	C.6	The completeness, correctness and consistency of CIs shall be confirmed.			SCM Auditor	Audit Notes
	C.7	Compliance of CIs is evaluated against standards.			SCMAuditor	Audit Notes
	C.8	Non-compliance, observation and improvement opportunities are collected and noted down.			SCM Auditor	Audit Notes
	C.9	The agreed upon audit findings shall be documented and shared.			Auditor, PM, SCM TL	Audit Notes
D. Closure	D.1	Audit findings/NCs shall be analysed for root causes and potential impact.			SCM TL, SQA TL	Root Causes/Impacts
	D.2	Audit findings are compared across the projects and/or organization to figure out trends, best practices and lessons learned.			SCM TL, SQA TL	Trends, best practices, lessons learned.
	D.3	Audit findings (NCs, observation, issues etc.) shall be discussed with relevant stakeholders.			SCM TL, SQA TL	Suggested CAs
	D.4	The issues shall be escalated to higher management for which the CAs are not feasible and approval is taken for organization-wide implementation of improvement opportunities.			SCM TL, PM	Management Directives/Decisions/Approvals
	D.5	Corrective actions shall be taken to close findings/NCs and improvements shall be incorporated for all instances of NCs.			SCM TL, SQA TL	CA reports
	D.6	Corrective actions shall be tracked to closure for all instances of NCs.			SCM TL	Follow-up Status
	D.7	Audit Status Report shall be prepared for publishing/sharing with stakeholders.			PM, SQA TL, SCM	Corrective Actions Reports
Interfaces	Artifacts generated from all PAs are referred to SCM-PA to be maintained in the CMS. QC department performs testing and notifies SCM about the resolution of all bugs. The schedule of configuration audit is received from the PP-PA.
Outputs of the workflow and associated PAs/SPs	Output Work-Products				Associated PA/SP
	Updated project plans Corrective Actions Internal Configuration Checklist, NCs, Audit Reports				PP-PA Track Change Requests Establish CM Records
Exit criteria	The CM Audits have been conducted and corrective actions have been tracked to closure. Audit report published.
Measures	Ratio of (No of NCs closed/No of NCs identified)
Verification points	PM shall verify the conduction of CM Audits as per PM&C process guide. QC shall perform testing activity to ensure fulfillment of specified requirements. Management shall periodically review the activities, status and configuration audit findings.
Training	Training on Perform Configuration Audit process and CM standards shall be conducted. Templates/checklist usage training.
Tools	MS Word, MS Excel
Assumptions	Trained Human Resource, Hardware, Software, tools and Facilities are available.
Exemptions	Tailoring Guidelines
Applicable standards & related documents	Availability of Documentation Standards Manual, CM Standards etc. Tailoring Guidelines PMC Process Guide PPQA Process Guide

5 Validation of Proposed Models, Threats to Validity and Their Mitigation 5.1 Validation Through EPR

In order to perform initial evaluation of the proposed models, an expert panel review was carried out where opinions on the models, based on the specified criteria, were taken from 10 experts. The experts having expertise in various domains (SPI, Project Management, Configuration Management and Software Development) were selected required for the study from Pakistani software industry and are enlisted along with the experience in Tab. 9.

Table 9 Profiles of the panel members

Domain	No of Experts	Overall Knowledge/Experience	CMMI/SPI Knowledge & Experience
SPI Experts/CMMI Auditors	2	20, 17	20, 16
QA Managers	2	19, 17	19, 15
Project Managers	2	20, 15	15, 15
Configuration Managers/CM Auditors	2	20, 14	18, 13
Senior Software Engineers	2	16, 13	15, 12

As in [26–30] the researchers are free to frame their own criteria. In this study, the experts were classified into 3 groups based on their experience and knowledge. Experts having experience less than 15 years were classified as Junior, the experts having experience greater than 17 years as Senior and the remaining were classified as In-between. As per this criteria, the panel comprised of 10 experts with 4 seniors, 2 juniors and 4 In-between.

The questionnaire was formulated specifically to obtain the panel opinion on the proposed WFMs after studying the questionnaire designed for a similar work by Niazi [3] and Keshta [4,8]. The questionnaire comprised of three parts including a cover letter elucidating the purpose; demographics; and expert’s opinion. The later part addresses the validation criteria specified in Tab. 4. The questionnaire was reviewed by an academician to make it more legible and refined. The questionnaire comprised seven questions, as shown in Tabs. 11 and 12. Question 8 not shown in questionnaire was used to collect feedback to improve the models. Five point Likert measure with relative weight is given in Tab. 10.

Table 10 Relative weight of five-point liker measure

-	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
Weight	1	2	3	4	5
Mean	1.00–1.80	1.81–2.60	2.61–3.40	3.41–4.20	4.21–5.00

Summary of responses to questions about SP-3.1 and SP-3.2 of CM-PA are shown in Tabs. 11 and 12.

Table 11 Summary of the responses to the proposed model for Sp-3.1

		Question	Measure	D/“2”	N/“3”	A/“4”	SA/“5”	Mean	Rlt
Practice Satisfaction(Answer to RQ-A,B)	Q-1	As per CMMI Ver. 1.3, the proposed model would help to satisfy the practice and contribute towards the achievement of relevant goal.(Strongly Agree – Strongly Disagree)	Freq	3	0	4	3	3.7	Agree
	Q-1		%age	30	0	40	30	3.7	Agree
	Q-2	How much our proposed WFM covers the SPs and Sub-SPs of the CM-PA?(Fully Covered 5 – 1 Not Yet)	Freq	0	2	2	6	4.4	Fully Covered
	Q-2		%age	0	20	20	60	4.4	Fully Covered
User Satisfaction(Answer to RQ-C)	Q-3	The proposed WFMs would prove useful for SSMEs.(Very Useful 5 – 1 Not at all)	Freq	2	1	3	4	3.9	Useful
	Q-3		%age	20	10	30	40	3.9	Useful
	Q-4	The use of the proposed WFM shall prove instrumental to improve software development process and contribute to quality of the products produced through it in SSMEs. (Strongly Agree – Strongly Disagree)	Freq	4	0	3	3	3.5	Agree
	Q-4		%age	40	0	30	30	3.5	Agree
Ease of Learning & Use (Answer to RQ-D)	Q-5	How clearly the said WFMs represents the relevant SP.(Very Clear 5 – 1 Not at all)	Freq	2	0	5	3	3.9	Clear
	Q-5		%age	20	0	50	30	3.9	Clear
	Q-6	In order to use the WFMs, how much CMMI knowledge would be required?(Not at all 5 – 1 Too Much)	Freq	3	0	3	4	3.8	Little
	Q-6		%age	30	0	30	40	3.8	Little
Implement-ability(Answer to RQ-E)	Q-7	Our proposed WFMs can be implemented in SSMEs with little tailoring/tweaking.(Strongly Agree – Strongly Disagree)	Freq	4	0	3	3	3.5	Agree
Implement-ability(Answer to RQ-E)	Q-7		%age	40	0	30	30	3.5	Agree

Table 12 Summary of the responses to the proposed model for SP- 3.2

		Question	Measure	D/“2”	N/“3”	A/“4”	SA/“5”	Mean	Rlt
Practice Satisfaction(Answer to RQ-A,B)	Q-1	As per CMMI Ver. 1.3, the proposed model would help to satisfy the practice and contribute towards the achievement of relevant goal. (Strongly Agree – Strongly Disagree)	Freq	3	0	3	4	3.8	Agree
	Q-1		%age	30	0	30	40	3.8	Agree
	Q-2	How much our proposed WFM covers the SPs and Sub-SPs of the CM-PA?(Fully Covered 5 – 1 Not Yet)	Freq	0	2	3	5	4.3	Fully Covered
	Q-2		%age	0	20	30	50	4.3	Fully Covered
User Satisfaction(Answer to RQ-C)	Q-3	The proposed WFMs would prove useful for SSMEs.(Very Useful 5 – 1 Not at all)	Freq	3	0	4	3	3.7	Useful
	Q-3		%age	30	0	40	30	3.7	Useful
	Q-4	The use of the proposed WFM shall prove instrumental to improve software development process and contribute to quality of the products produced through it in SSMEs. (Strongly Agree – Strongly Disagree)	Freq	3	0	5	2	3.6	Agree
	Q-4		%age	30	0	50	20	3.6	Agree
Ease of Learning & Use (Answer to RQ-D)	Q-5	How clearly the said WFMs represents the relevant SP.(Very Clear 5 – 1 Not at all)	Freq	2	0	4	4	4.0	Clear
	Q-5		%age	20	0	40	40	4.0	Clear
	Q-6	In order to use the WFMs, how much CMMI knowledge would be required?(Not at all 5 – 1 Too Much)	Freq	3	0	3	4	3.8	Little
	Q-6		%age	30	0	30	40	3.8	Little
Implement-ability(Answer to RQ-E)	Q-7	Our proposed WFMs can be implemented in SSMEs with little tailoring/tweaking.(Strongly Agree – Strongly Disagree)	Freq	2	0	5	3	3.9	Agree
Implement-ability(Answer to RQ-E)	Q-7		%age	40	0	30	30	3.9	Agree

As obvious from the results of EPR, the expert are strongly agree or agree that the models are instrumental in facilitating the implementation of the SPs, supportive in achieving SG-3, provide coverage to the sub-practices, are easy to learn, believed to be very useful to the software industry, perceived to improve process and contribute to the quality of the software produced. Further, it shall particularly support SSMEs in implementing the said SPs. However, it transpired from the EPR that a little knowledge about CM-PA of CMMI and CM domain is required. Results also suggest that there is a room for improvement in the WFMs.

5.2 Validation Through Case Studies

As a confidence measure, two Pakistani software SMEs were selected for carrying out case studies. The SSMEs were willing to publish the outcome of the study, however asked for non-disclosure of the SMEs names and project’s info. As a confidentiality measure, the companies in this study are referred as Small Software Enterprise (SSE) and Medium Software Enterprise (MSE). Brief introduction of the companies are tabulated as under in Tab. 13.

Table 13 Software setups participated in study

Type of Setup	Emp(s) Strength	Core Business Activities
SSE	32	Software development and provision of support to Pakistani Sugar Mills.
MSE	83	Development of ERP for SMEs and provision of maintenance support.

After necessary coordination at management level, an opening sessions were arranged in both the SSMEs for participating employees including project manager, quality manager, configuration manager and relevant desirous system analysts, developers and testing professionals. The two companies’ collaborated and about 30 professionals participated. A brief presentation was given over the objectives of the study in these sessions. Soft copies of the models, templates, forms, guides were provided as well as an envelope full of hard copies was handed over for implementation in their environment within one month duration. After implementation in both the SSMEs, SCAMPI Type-“C” & Type-“B” appraisals were conducted against the said SPs to evaluate its effectiveness by the lead auditor with appraisal team members (ATM). Finally, a closing session was conducted to get feedback from participating professionals. The appraisal results were encouraging and appreciated by the lead auditor. Confidence of the lead auditor reflected from his statement that both the SSMEs fulfill the requirements of the said SPs and will certainly result in “Fully-Implemented” if SCAMPI type “A” is conducted. In closing session, feedback was collected through the questionnaire that was originally designed for EPR.

5.3 The Proposed WFMs in Comparison with Earlier Models Developed for Various PAs of CMMI

A detailed comparison of the proposed models with the existing models are inscribed in Tab. 14. The comparison criteria was taken from the work of Niazi [3], Keshta [4,7,8] and further refined.

Table 14 The proposed model in comparison with the earlier models found in literature

Comparison Criteria	The Proposed Models	References to Earlier Models
Comparison Criteria	The Proposed Models	[3]Niazi	[4]Keshta	[5]Anum	[6]Bhatti	[7]Keshta	[8]Keshta	[9]Viva
Is WFMs compliant to CMMI representation-(staged/phased)?	☼	☼	☼		☼	☼	☼	☼
Do WFMs achieve objectives of SG-2 of CM-PA?	☼	☼	☼	☼		☼	☼
Are the WFMs devised SP-wise?	☼	☼	☼	☼		☼	☼
Do it satisfy the relevant SPs (SP-2.1 & SP-2.2)?	☼	☼	☼			☼	☼	☼
Do the WFM cover the Sub-SPs?	☼
Do it satisfy user?	☼	☼	☼			☼	☼
Are the WFMs easy to learn/easy to use?	☼	☼	☼		☼	☼	☼
Do the WFMs applicable to Software SMEs?	☼		☼			☼	☼
Do the WFMs follow the ETVX Model?	☼
Have the associated templates, forms, checklists developed?	☼		☼			☼	☼	☼
Is the Process Guide prepared?	☼
Does the model address the overall CM-PA?	☼

5.4 Limitation of the Study/Threats to Validity and Their Mitigation Strategies

The limitations of the study, threats to its validity are explained in Tab. 15 along with mitigation strategies.

Table 15 Limitations of the study/threats to validity and mitigation strategies.

Limitation/Threats to Validity	Mitigation Strategy
Presence of the closed-ended questions in the questionnaire may not have captured the true respondent’s feelings.	The impact was reduced by adding the open-ended questions as well. This added to the veracity of the response.
The panel members may have varying interpretation of the questions/WFMs and responded accordingly.	The questionnaire, due to close relevancy, was taken from Keshta’s work. This was more refined by adding coverage of the framework at sub-practices level and reviewed by another academician.
The responses may have been limited to the knowledge and experiences of the respondents.	As a confidence building measure, experts with rich industry experience were selected. Presence of the world-renowned experts in the panel added to the effectiveness of the review process.
There might be a difference between responses received from Junior, In-between and Senior experts.	Fortunately, the number of senior and In-between experts exceeded the Junior experts. Further, the p > 0.05 of Chi-Square (X²) test when ∞ is 0.05 & degree of freedom is 2 against the responses which is indicative of insignificant variation among the responses provided by Senior, In-between and Junior experts.
The possibility that ordinary literature review process may have failed to see the relevant research work.	As per Hossain et al. [31], this cannot be taken as systematic omission.
Results and conclusions may not be valid for diversified or a typical environments.	In addition to the EPR, case studies were conducted in Pakistani Software Industry. Though results may be generalized for Pakistani SSMEs, however more case studies be carried out for other countries.

6 Conclusion

Designing a workflow model to achieve SG-3 “Establish Integrity” of CM-PA at CMMI maturity level-II and its validation was the main objective of this study. Five research questions (RQ-A~RQ-E) were formulated for the purpose. Further WFMs were devised for two SPs contributing to the aforesaid goal. It is clearly indicated in the Tabs. 11 and 12 that which question of questionnaire addresses which research question making use of which validation criteria. Responses from the experts satisfied the said criteria. The results were further affirmed through conducting case studies. It is worth-mentioning that case studies demonstrated the ability of Pakistani SSMEs to adopt the proposed models with little tailoring to adjust their contexts. Satisfactory comments from participating organizations and experts speaks well of the WFMs and add to the confidence in the evaluation results. In face-to-face discussion with the participating professionals, it transpired that they had no problem in understanding/usage of the models with associated templates, forms, checklists and process guides as helping tools. The WFMs were refined after several rounds of improvements by incorporating suggestions from academicians, professionals and finally feedback from case studies. This work shall be continued to develop WFMs for other SPs of this PA, other PAs of Level-II as well as higher levels for which the workflow models are not developed yet. The models also need to be revised/validated for future versions of the CMMI.

Funding Statement: The author(s) received no specific funding for this study.

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the present study.

References [1]

Xu,

Hu,

Yu,

Lv,

Qu et al.

, “ Supporting flexibility of the CMMI process framework with a multi-layered process model,” in Web Information System and Application Conference (WISA 2013), Yangzhou, China, pp. 409– 414, 2013.

[2]

CMMI Product Team , “ CMMI for Development, Version 1.3,” 2010.

[3]

Niazi,

Hickman,

Ahmad and

Ali Babar , “A model for requirements change management: Implementation of CMMI level 2 specific practice,” in 9th International Conference on Product-Focused Software Process Improvement (PROFES 2008), Springer Berlin Heidelberg, Monte Porzio Catone, Italy, vol. 5089, pp. 143– 157, 2008.

[4]

Keshta,

Niazi and

Alshayeb , “ Towards implementation of requirements management specific practices (SP1.3 and SP1.4) for Saudi Arabian small and medium sized software development organizations,” IEEE Access, vol. 5, pp. 24162– 24183, 2017.

[5]

Tariq,

S. A.

Khan and

Iftikhar , “ Remapping of CMMI level-2 KPA’s for development process improvement of software-as-a-service (SaaS) cloud environment,” in Int. Conf. on Open Source Systems and Technologies (ICOSST), Lahore, Pakistan, pp. 43– 51, 2014.

[6]

M. W.

Bhatti,

Hayat,

Ehsan,

Ishaque,

Ahmed et al.

, “ A methodology to manage the changing requirements of a software project,” in Int. Conf. on Computer Information Systems and Industrial Management Applications (CISIM), Krackow, Poland, pp. 319– 322, 2010.

[7]

Keshta

, “ A model for defining project lifecycle phases: Implementation of CMMI level 2 specific practice,” Journal of King Saud University - Computer and Information Sciences, pp. 1–10, 2019.

[8]

Keshta,

Niazi and

Alshayeb , “ Towards implementation of process and product quality assurance process area for Saudi Arabian small and medium sized software development organizations,” IEEE Access, vol. 6, pp. 41643– 41675, 2018.

[9]

Vivatanavorasin,

Prompoon and

Surarerks , “ A process model design and tool development for supplier agreement management of CMMI: Capability level 2,” in XIII ASIA PACIFIC Software Engineering Conf. (APSEC’06), Kanpur, India, pp. 385– 392, 2006.

[10]

Hochbergs

, MS Thesis by Erik Hochbergs and Laroy Nilsson Sjödahl from Department of Computer Science LTH, Lund University, Lund Sweden, 2020.

[11]

Fahmy,

Deraman,

Yahaya,

Nasir and

Shamsudin , “ The evolution of software configuration management,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 9, no. 1.3, pp. 50– 63, 2020.

[12]

Tuape

, “ Factors affecting development process in small software companies,” in IEEE/ACM Symposium on Software Engineering in Africa (SEiA), Montréal, Québec, Canada, pp. 16– 23, 2019.

[13]

José,

Camila,

Balancieri and

A. C.

Rouiller , “ The measurement process in micro and small software maintenance companies: Empirical study,” Independent Journal of Management & Production (IJM&P), vol. 11, pp. 519– 538, 2020.

[14]

T. M.

Eshete

, Thesis of MS in Software Engineering by Tadele Mengiste Eshete A, School of Research and Postgraduate Studies Faculty F Computing, Bahir Dar University Bahir Dar Institute of Technology, DSpace Institution, DSpace Repository, 2020. http://dspace.org.

[15]

Dasanayaka

, “ SMEs in globalized world: A brief note on basic profiles of Pakistan’s small and medium scale enterprises and possible research directions,” Research Journal of the Institute of Business Administration Karachi – Pakistan, vol. 3, no. 1, pp. 69– 90, 2008.

[16]

Divakara and

H. N. S. K.

Surangi , “ A literature review on small & medium size enterprises,” Journal of Business and Technology, vol. 2, no. 3, pp. 23, 2018.

[17]

Karadag

, “ The role and challenges of small and medium-sized enterprises (SMEs) in emerging economies: An analysis from Turkey,” Business and Management Studies, vol. 1, no. 2, pp. 179– 188, 2015.

[18]

Tripathi

, “ SMEs in Saudi Arabia - an innovative tool for country’s economic growth,” Science International (Lahore), vol. 31, no. 2, pp. 261– 267, 2019.

[19]

Mary Beth,

Konrad and

Shrum , CMMI for Development. Third Edition, Addison Wesley, Pearson Education, Boston, USA, 2017.

[20]

O’Regan

, Introduction to Software Process Improvement. London Dordrecht Heidelberg New York: Springer, 2011.

[21]

Abran,

J. W.

Moore,

Dupuis,

Dupuis and

L. L.

Tripp , “ Guide to the Software Engineering Body of Knowledge (SWEBOK), A Project of the IEEE Computer Society, Version 3.0, 2014.

[22]

O’Regan

, Introduction To Software Quality Assurance. Ireland, Cham Heidelberg New York Dordrecht London: Springer, 2014.

[23]

J. C.

Blockwood, “ Editor, Professional skepticism: A model for public sector auditing,” International Journal of Governemnt Auditing, vol. 47, no. 3, pp. 45, 2020, [Online]. Available: www.intosaijournal.org.

[24]

Guidelines

, “ ISSAI Guidelines on Performance Audit,” 2010.

[25]

Gilchrist

, “ Software Process Reviews/Audits Process Overview - SASQAG,” 2010.

[26]

S. U.

Khan,

Niazi and

Ahmad , “ Empirical investigation of success factors for offshore software development outsourcing vendors,” Institute of Engineering and Technology Software, vol. 6, no. 1, pp. 1– 15, 2012.

[27]

M. I. U.

Lali,

R. U.

Mustafa,

Saleem,

M. S.

Nawaz,

Zia et al.

, “ Finding healthcare issues with search engine queries and social network data,” International Journal on Semantic Web and Information Systems, vol. 13, no. 1, pp. 48– 62, 2017.

[28]

Shahzad,

Al-Ohali and

Abdullah , “ Trivial model for mitigation of risks in software development life cycle,” International Journal of the Physical Sciences, vol. 6, no. 8, pp. 2072– 2082, 2011.

[29]

Shahzad and

Afzal Safvi , “ Effective risk mitigation: A user prospective,” International Journal of Mathematics and Computers in Simulation, vol. 2, no. 1, pp. 70– 80, 2008.

[30]

Shafiq,

Ahmad and

J. G.

Choi , “ Public system usability analysis for the valuation of cognitive burden and interface standardization: A case study of cross-ATM design,” Journal of Organizational Computing and Electronic Commerce, vol. 27, no. 2, pp. 162– 196, 2017.

[31]

Hossain,

Ali Babar and

H. Y.

Paik , “ Using scrum in global software development: A systematic literature review,” in Fourth IEEE Int. Conf. on Global Software Engineering (ICGSE 2009), Limerick, Ireland, pp. 175– 184, 2009.