<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.1 20151215//EN" "http://jats.nlm.nih.gov/publishing/1.1/JATS-journalpublishing1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" article-type="research-article" dtd-version="1.1">
<front>
<journal-meta>
<journal-id journal-id-type="pmc">IASC</journal-id>
<journal-id journal-id-type="nlm-ta">IASC</journal-id>
<journal-id journal-id-type="publisher-id">IASC</journal-id>
<journal-title-group>
<journal-title>Intelligent Automation &#x0026; Soft Computing</journal-title>
</journal-title-group>
<issn pub-type="epub">2326-005X</issn>
<issn pub-type="ppub">1079-8587</issn>
<publisher>
<publisher-name>Tech Science Press</publisher-name>
<publisher-loc>USA</publisher-loc>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="publisher-id">21769</article-id>
<article-id pub-id-type="doi">10.32604/iasc.2022.021769</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Article</subject>
</subj-group>
</article-categories>
<title-group>
<article-title>Strengthening Software Make <italic>vs.</italic> Buy Decision: A Mixed-Method Approach</article-title><alt-title alt-title-type="left-running-head">Strengthening Software Make <italic>vs.</italic> Buy Decision: a Mixed-Method Approach</alt-title><alt-title alt-title-type="right-running-head">Strengthening Software Make <italic>vs.</italic> Buy Decision: a Mixed-Method Approach</alt-title>
</title-group>
<contrib-group content-type="authors">
<contrib id="author-1" contrib-type="author">
<name name-style="western"><surname>Javeed</surname><given-names>Farooq</given-names></name>
<xref ref-type="aff" rid="aff-1">1</xref>
</contrib>
<contrib id="author-2" contrib-type="author">
<name name-style="western"><surname>Shahzad</surname><given-names>Basit</given-names></name>
<xref ref-type="aff" rid="aff-2">2</xref>
</contrib>
<contrib id="author-3" contrib-type="author">
<name name-style="western"><surname>Shaikh</surname><given-names>Asadullah</given-names></name>
<xref ref-type="aff" rid="aff-3">3</xref>
</contrib>
<contrib id="author-4" contrib-type="author" corresp="yes">
<name name-style="western"><surname>Al Reshan</surname><given-names>Mana Saleh</given-names></name>
<xref ref-type="aff" rid="aff-3">3</xref><email>manaalreshan@gmail.com</email>
</contrib>
<contrib id="author-5" contrib-type="author">
<name name-style="western"><surname>Ahmad</surname><given-names>Saba</given-names></name>
<xref ref-type="aff" rid="aff-1">1</xref>
</contrib>
<contrib id="author-6" contrib-type="author">
<name name-style="western"><surname>Alshahrani</surname><given-names>Hani</given-names></name>
<xref ref-type="aff" rid="aff-3">3</xref>
</contrib>
<contrib id="author-7" contrib-type="author">
<name name-style="western"><surname>Rajab</surname><given-names>Khairan</given-names></name>
<xref ref-type="aff" rid="aff-3">3</xref>
</contrib>
<aff id="aff-1"><label>1</label><institution>Department of Computer Science and Information Technology, University of Sargodha</institution>, <addr-line>Sargodha, 40100</addr-line>, <country>Pakistan</country></aff>
<aff id="aff-2"><label>2</label><institution>Department of Engineering, National University of Modern Languages</institution>, <addr-line>Islamabad, 44000</addr-line>, <country>Pakistan</country></aff>
<aff id="aff-3"><label>3</label><institution>College of Computer Science and Information System, Najran University</institution>, <addr-line>Najran, 61441</addr-line>, <country>Saudi Arabia</country></aff>
</contrib-group><author-notes><corresp id="cor1">&#x002A;Corresponding Author: Mana Saleh Al Reshan. Email: <email>manaalreshan@gmail.com</email></corresp></author-notes>
<pub-date pub-type="epub" date-type="pub" iso-8601-date="2021-10-4"><day>04</day>
<month>10</month>
<year>2021</year></pub-date>
<volume>32</volume>
<issue>1</issue>
<fpage>193</fpage>
<lpage>205</lpage>
<history>
<date date-type="received"><day>14</day><month>7</month><year>2021</year></date>
<date date-type="accepted"><day>18</day><month>8</month><year>2021</year></date>
</history>
<permissions>
<copyright-statement>&#x00A9; 2022 Javeed et al.</copyright-statement>
<copyright-year>2022</copyright-year>
<copyright-holder>Javeed et al.</copyright-holder>
<license xlink:href="https://creativecommons.org/licenses/by/4.0/">
<license-p>This work is licensed under a <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</ext-link>, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
</license>
</permissions>
<self-uri content-type="pdf" xlink:href="TSP_IASC_21769.pdf"></self-uri>
<abstract>
<p>Over the past few decades, multiple software development process models, tools, and techniques have been used by practitioners. Despite using these techniques, most software development organizations still fail to meet customer&#x2019;s needs within time and budget. Time overrun is one of the major reasons for project failure. There is a need to come up with a comprehensive solution that would increase the chances of project success. However, the &#x201C;make <italic>vs.</italic> buy&#x201D; decision can be helpful for &#x201C;in time&#x201D; software development. Social media have become a popular platform for discussion of all sorts of topics, so software development is no exception. Software developers discuss all the pros and cons of making <italic>vs.</italic> buy decisions on Twitter and other social media platforms. Twitter trending is a typical feature that evaluates the level of popularity of a specific event on online networking. A mixed-method approach comprising of interviews of software industry experts and Twitter data extraction is applied to scrutinize the effective decision of software build <italic>vs.</italic> buy decision. The findings of the analysis show that software makes <italic>vs.</italic> buy decisions depend on several factors including cost, development technology, software development team skills, and time. Based on the finding of the study a framework is proposed for the decision to build versus buy in Small and medium-sized enterprises (SMEs). Furthermore, the framework has been designed to statistically indicate make versus buy decisions of the organization and to suggest appropriate choices based on different parameters.</p>
</abstract>
<kwd-group kwd-group-type="author">
<kwd>Software make</kwd>
<kwd>software buy decisions</kwd>
<kwd>twitter classification</kwd>
<kwd>decision framework</kwd>
<kwd>SMEs</kwd>
</kwd-group>
</article-meta>
</front>
<body>
<sec id="s1">
<label>1</label>
<title>Introduction</title>
<p>Software development is one of the leading industries among other commercial industries. Although the software has been developed for the past many decades, however, the development process still needs improvements to predict its faultless development [<xref ref-type="bibr" rid="ref-1">1</xref>]. A big consideration in software development is the decision to &#x201C;build versus buy&#x201D;. Particularly, commercial off the shelf is available for organizations of all types however all those organizations are facing difficulty in taking the right decision. Previously, researchers have made arguments that support one approach over the other, while some are acknowledged as realities. Because of advancements in the software industry, few of the arguments have become unsubstantial [<xref ref-type="bibr" rid="ref-2">2</xref>]. Software leaders have questioned whether it is better to make software in-house or buy from the market. SMEs (Small and Medium Enterprises) play a very significant role in the development of any country [<xref ref-type="bibr" rid="ref-3">3</xref>] because of having a global impact on the GDP. Smaller-scale enterprises focus on maintaining viability, rise in profits, and increases in sales. A big consideration in development for the organization is the decision to build <italic>vs.</italic> buy. Build implies that custom subsystems will be built from its core essential parts, then integrates into a final product. Buy implies buying the subsystems and maintaining distance from customizing tasks [<xref ref-type="bibr" rid="ref-2">2</xref>]. Many organizations take a strategic approach to make <italic>vs.</italic> buy decisions [<xref ref-type="bibr" rid="ref-4">4</xref>]. Organizations that do not consider the build/buy option as an open door are more likely to employ sources of reference where social values based on expert peer acceptance are typically dominated [<xref ref-type="bibr" rid="ref-5">5</xref>]. When a business requires software, it has the option of building it or acquiring it. The decision to acquire more can be made in two ways: whether to buy the entire solution or just a few components. Modifiable off-the-shelf (MoTS) and Commercial off-the-shelf (COTS) solutions are available (CoTS). If the decision is made to construct software, the appropriate environment for software development must be provided. However, deciding whether to build or buy is not a simple decision that takes the careful evaluation of facts, understanding of the organization&#x2019;s strengths and shortcomings, economic situation, and other parameters [<xref ref-type="bibr" rid="ref-2">2</xref>].</p>
<p>In the development of software in SMEs, the build versus buy decision might take any form, as shown in <xref ref-type="fig" rid="fig-1">Fig. 1</xref>. The pathways of software build <italic>vs.</italic> buy decision are depicted in <xref ref-type="fig" rid="fig-1">Fig. 1</xref>. Build <italic>vs.</italic> buy is a difficult decision to make because it involves many factors such as capacity, organizational resources, sufficient time to be available, and the presence of required expertise in the organization; if all of these factors are present, the decision will be in favor of build; otherwise, an alternative approach will be considered. We gathered the elements from the literature and then updated them with expert opinions gleaned from social media.</p>
<fig id="fig-1">
<label>Figure 1</label>
<caption>
<title>Software build versus buy decision</title></caption>
<graphic mimetype="image" mime-subtype="png" xlink:href="IASC_21769-fig-1.png"/>
</fig>
<p>Social Media are a popular platform to discuss and experience things in a different way. Not only it brings change into someone&#x2019;s opinion but also becomes the cause of social change [<xref ref-type="bibr" rid="ref-6">6</xref>]. Twitter has risen as a broadly used microblogging service. It has a huge and rapidly rising user. They post status (messages) called tweets. It is a short message with a length of 140 characters [<xref ref-type="bibr" rid="ref-7">7</xref>] initially which has upgraded the character length now. Hence, Twitter data is also used as the base to classify Build <italic>vs.</italic> Buy. Classification is the categorization of data into one or more groups. Data is classified into two categories as labeled and unlabelled data. Unlabelled data is labeled into some classes by using machine learning algorithms. Commonly there are two types: classic binary and modern multiclass [<xref ref-type="bibr" rid="ref-8">8</xref>]. Multiple tools are used for the classification of data such as WEKA. WEKA stands for &#x201C;Waikato Environment for Knowledge Analysis&#x201D; [<xref ref-type="bibr" rid="ref-9">9</xref>]. It has many built-in features like feature generation, feature selection, classification, and evaluation of the selected model. Moreover, the result of the classifier depends on the nature of the data. Therefore, the objective of this research is to identify the factors that help organizations to take an appropriate decision either software should be in-house or buy from vendors.</p>
<p>The organization of the study is as follows. Section 2 elaborates the related work from literature, section 3 discusses the methodology adopted. Section 4 elaborates on the results of the study whereas, the conclusion and future work is presented in section 5.</p>
</sec>
<sec id="s2">
<label>2</label>
<title>Related Work</title>
<p>Previously, software tasks are considered as sequences of codes. Single person interpreted and solved all problems of obligatory software where no team management is needed [<xref ref-type="bibr" rid="ref-10">10</xref>]. IT can be utilized to recognize and find people with specific expertise nevertheless of a person&#x2019;s physical location [<xref ref-type="bibr" rid="ref-11">11</xref>]. To write the code people started hiring others as &#x201C;developers&#x201D; particularly in modern programming languages. Developers levels are different and hence the code may contain different types of vulnerabilities. The level of developer from given source code is detected by [<xref ref-type="bibr" rid="ref-12">12</xref>]. Moreover, new concepts of some different roles in software development are introduced by the industry i.e., users and developers. Many organizations take a strategic approach to make <italic>vs.</italic> buy decisions. Organizations that did not see the build/buy choice as an open door tend to use points of reference [<xref ref-type="bibr" rid="ref-13">13</xref>]. Commercial-off-the-shelf (COTS) may not fully fit all requirements [<xref ref-type="bibr" rid="ref-14">14</xref>]. According to Xu [<xref ref-type="bibr" rid="ref-15">15</xref>], outsourcing is entirely over an in-house generation if the yield of seller&#x2019;s production is adequately low or its economies of the extension are remarkably appealing. Cortellessa [<xref ref-type="bibr" rid="ref-16">16</xref>] divided the requirements into two parts; functional and non-functional for components. Some companies take the decision of make, buy and make or buy separately on four different theories that are used to elaborate the decision of make or buy for companies (transaction cost economics composite, neoclassical economics/rational choice decision, the resource-based view of strategy and institutional theory explanations) [<xref ref-type="bibr" rid="ref-17">17</xref>]. According to Daneshgar et al. [<xref ref-type="bibr" rid="ref-18">18</xref>], discoveries make the required decision more time-consuming. Simply they acknowledged that if dealer support is insufficient, the decision to buy is mostly among SMEs. Cost is always a big consideration; its risk remains the same in both cases [<xref ref-type="bibr" rid="ref-19">19</xref>,<xref ref-type="bibr" rid="ref-20">20</xref>]. Availability of multiple solutions to requirement fit leads to buying in many cases. Instead of the typical software development approach build <italic>vs.</italic> buy decision needs the contribution of people (networks, data centers, etc.) that may gather data from different techniques by Torrecilla-Salinas [<xref ref-type="bibr" rid="ref-20">20</xref>] to explore that the agile techniques enrich the build <italic>vs.</italic> buy decision. The organization mostly goes for a third option except for build <italic>vs.</italic> buy but to cope with the risk factor [<xref ref-type="bibr" rid="ref-21">21</xref>]. In the process of literature review, multiple factors have been identified that affect the organization&#x2019;s development method. Factors that affect the build <italic>vs.</italic> buy decision of SEMs are summarized in <xref ref-type="table" rid="table-1">Tab. 1</xref>. Previous research [<xref ref-type="bibr" rid="ref-2">2</xref>,<xref ref-type="bibr" rid="ref-18">18</xref>] has backed up these factors, which have been used to create a generic set of criteria.</p>
<table-wrap id="table-1"><label>Table 1</label>
<caption>
<title>Factors that affect the build <italic>vs.</italic> buy decision of SEMs</title></caption>
<table><colgroup><col align="left"/><col align="left"/><col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">F&#x0023;</th>
<th align="left">Factor</th>
<th align="left">Reference</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">1</td>
<td align="left">The factor of strategy and competitive advantage</td>
<td align="left">[<xref ref-type="bibr" rid="ref-18">18</xref>]</td>
</tr>
<tr>
<td align="left">2</td>
<td align="left">Scale and complexity factor</td>
<td align="left">[<xref ref-type="bibr" rid="ref-18">18</xref>], [<xref ref-type="bibr" rid="ref-25">25</xref>],[<xref ref-type="bibr" rid="ref-26">26</xref>&#x2013;<xref ref-type="bibr" rid="ref-29">29</xref>]</td>
</tr>
<tr>
<td align="left">3</td>
<td align="left">Cost factor</td>
<td align="left">[<xref ref-type="bibr" rid="ref-18">18</xref>], [<xref ref-type="bibr" rid="ref-21">21</xref>],[<xref ref-type="bibr" rid="ref-29">29</xref>],[<xref ref-type="bibr" rid="ref-30">30</xref>]</td>
</tr>
<tr>
<td align="left">4</td>
<td align="left">The factor of requirement fit</td>
<td align="left">[<xref ref-type="bibr" rid="ref-18">18</xref>],[<xref ref-type="bibr" rid="ref-30">30</xref>]</td>
</tr>
<tr>
<td align="left">5</td>
<td align="left">Time factor</td>
<td align="left">[<xref ref-type="bibr" rid="ref-18">18</xref>], [<xref ref-type="bibr" rid="ref-26">26</xref>],[<xref ref-type="bibr" rid="ref-27">27</xref>], [<xref ref-type="bibr" rid="ref-28">28</xref>],[<xref ref-type="bibr" rid="ref-29">29</xref>]</td>
</tr>
<tr>
<td align="left">6</td>
<td align="left">In-house expertise factor</td>
<td align="left">[<xref ref-type="bibr" rid="ref-17">17</xref>],[<xref ref-type="bibr" rid="ref-18">18</xref>]</td>
</tr>
<tr>
<td align="left">7</td>
<td align="left">The factor of risk involved</td>
<td align="left">[<xref ref-type="bibr" rid="ref-17">17</xref>],[<xref ref-type="bibr" rid="ref-18">18</xref>], [<xref ref-type="bibr" rid="ref-25">25</xref>], [<xref ref-type="bibr" rid="ref-30">30</xref>], [<xref ref-type="bibr" rid="ref-31">31</xref>]</td>
</tr>
<tr>
<td align="left">8</td>
<td align="left">The factor of the Support structure</td>
<td align="left">[<xref ref-type="bibr" rid="ref-17">17</xref>],[<xref ref-type="bibr" rid="ref-18">18</xref>],</td>
</tr>
<tr>
<td align="left">9</td>
<td align="left">Operational factor</td>
<td align="left">[<xref ref-type="bibr" rid="ref-18">18</xref>], [<xref ref-type="bibr" rid="ref-30">30</xref>]</td>
</tr>
<tr>
<td align="left">10</td>
<td align="left">The factor of commoditization, flexibility, and change.</td>
<td align="left">[<xref ref-type="bibr" rid="ref-18">18</xref>], [<xref ref-type="bibr" rid="ref-21">21</xref>], [<xref ref-type="bibr" rid="ref-29">29</xref>], [<xref ref-type="bibr" rid="ref-31">31</xref>]</td>
</tr>
<tr>
<td align="left">11</td>
<td align="left">Economic factor</td>
<td align="left">[<xref ref-type="bibr" rid="ref-21">21</xref>], [<xref ref-type="bibr" rid="ref-25">25</xref>], [<xref ref-type="bibr" rid="ref-28">28</xref>], [<xref ref-type="bibr" rid="ref-29">29</xref>]</td>
</tr>
<tr>
<td align="left">12</td>
<td align="left">The factor of Standards involved</td>
<td align="left">[<xref ref-type="bibr" rid="ref-26">26</xref>], [<xref ref-type="bibr" rid="ref-27">27</xref>], [<xref ref-type="bibr" rid="ref-29">29</xref>],[<xref ref-type="bibr" rid="ref-32">32</xref>]</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Machine learning (ML) is a branch of artificial intelligence. It works on the two approaches supervised and unsupervised learning. In supervised learning provided output is used for the training of the input data. For the algorithm, a training labeled data set has been used under supervised learning. Different supervised techniques are support vector machine (SVM), na&#x00EF;ve Bayes (NB), Logistic regression, Decision tree, and Maximum Entropy. Clustering is an absolute unsupervised learning methodology. Some unsupervised algorithms are HMM, K-Mean, and Neural Network. Classification of data is necessary to acquire desirable results. Commonly there are two types, classic binary, and modern multi-class. Social media, nowadays, has numerous impacts on society. Many social sites are famous including Facebook, Instagram, Viber, WhatsApp, Twitter, Linked In, and many more. Twitter as a social networking site, started on 21 March 2006. Through tweets, people can express their views and emotions precisely due to limited character space, which is 140 in numbers now its 280-character space [<xref ref-type="bibr" rid="ref-22">22</xref>]. With some limit&#x2019;s tweets can be searched by API, a search facility provided by Twitter [<xref ref-type="bibr" rid="ref-23">23</xref>]. Per day production of tweets is half of a billion by its users [<xref ref-type="bibr" rid="ref-24">24</xref>]. It is a faster way to gain knowledge. Tweet data is collected and data is divided into two classes; yes, and no.</p>
<p>The literature gives rise to different questions like how to build or buy decisions should be taken for different organizations? What are the factors that affect the organization&#x2019;s development method? and what are the reasons behind the build <italic>vs.</italic> buy decision? Therefore, the objective of this research is to identify the factors that help organizations to take an appropriate decision either software should be in-house built or buy from vendors. Another objective is to suggest an appropriate decision to build <italic>vs.</italic> buy in SMEs with the help of a framework. Machine learning classification is used in this study to classify data into two categories i.e., build &#x0026; buy.</p>
<p>In the literature study, the most significant factors studied by many researchers are &#x201C;Time&#x201D; and &#x201C;Risk,&#x201D; followed by economics, cost, complexity, and changeability. The variables &#x201C;strategy and competitive advantage,&#x201D; &#x201C;support structure,&#x201D; and &#x201C;operational factor&#x201D; are the ones that have received the least attention.</p>
</sec>
<sec id="s3">
<label>3</label>
<title>Methodology</title>
<p>To make the study more complete, a mixed-method approach based on qualitative and quantitative methodologies is used. This research tries to determine the best decision for businesses to build <italic>vs.</italic> buy based on the characteristics listed in <xref ref-type="table" rid="table-1">Tab. 1</xref> and gleaned through a literature evaluation.</p>
<p>The methodology adopted in this study is based on four steps.<list list-type="roman-lower"><list-item><label>i.</label>
<p>First, factors influencing the decision to make <italic>vs.</italic> buy are found in the literature, with a total of twelve factors indicated in <xref ref-type="table" rid="table-1">Tab. 1</xref>.</p></list-item><list-item><label>ii.</label>
<p>Second, a survey questionnaire is constructed to collect data from IT specialists, owners, developers, and senior management from various small to medium enterprises based on these factors. All of the information acquired in the orderly survey is done so by asking the appropriate questions and following a well-defined process [<xref ref-type="bibr" rid="ref-32">32</xref>]. To organize the information, different prioritization strategies are used. Pilot research is conducted by sending questioners to ten industry experts, after which the questionnaire is changed and given to other organizations based on the input received from the experts. The relevant information is extracted from the dataset using the SPSS program [<xref ref-type="bibr" rid="ref-33">33</xref>].</p></list-item><list-item><label>iii.</label>
<p>Then, to cross-validate, the survey findings, interviews with IT specialists from five SMEs are undertaken.</p></list-item><list-item><label>iv.</label>
<p>Finally, using Twitter, compare the results of the survey and interview with overall software-industry practices in different countries. Machine learning techniques are used to gather and classify Twitter Tweets.</p></list-item></list></p>
<p>Data collection is the key part of these types of studies. Numerous ways are used for data pool such as interviews, surveys, meetings, and an online survey. The collection of data is a crucial requirement to establish the cogency of this approach. For the survey, data is collected by digital means. It is conceivable to compute the reliability and the capability of summing up results to a bigger population. Furthermore, the profundity of a quantitative report is restricted [<xref ref-type="bibr" rid="ref-28">28</xref>]. The online questionnaire is sent to 65 organizations in Pakistan among which 58 had responded. 10 responses are not filled while 2 are not filled by relevant persons. Thus, valid data of 46 organizations are utilized in this study. The survey consists of a five-point Likert scale i.e., from 0-strongly disagree, to 4-strongly agree is applied to get responses from experts.</p>
<p>For the experts&#x2019; interview phase, face to face interview session of 30 min for each interviewee is conducted. Interviewees are asked five open-ended questions regarding build <italic>vs.</italic> buy decisions in their organizations. The findings of the interviews are summarized and analyzed for common themes.</p>
<p>For this study, 569 Tweeter tweets are collated are classified. All tweets of IT experts (known) from 2009 to the present are collected during the two months. Tweets collection is an essential and basic phase of this approach therefore by using API search, tweets are collected from Twitter. The pre-processing technique for withdrawal of inappropriate characters and elimination of other frequent words is applied. All data is pre-processed before giving it to classifiers. Furthermore, Tweets Data, Outlier Removal, Keyword Stemming, Spelling Correction, and Category Ranking steps have been performed. After the execution of a classifier, the next stage is the validation of the model by a cross-validation method. There are multiple techniques used for this purpose. Precision, accuracy level, recall, and F-measure are frequently utilized [<xref ref-type="bibr" rid="ref-22">22</xref>]. Recall and precision are inversely proportional to each other; when recall increases precision decreases and when recall decreases precision increases [<xref ref-type="bibr" rid="ref-23">23</xref>].</p>
</sec>
<sec id="s4">
<label>4</label>
<title>Data Analysis and Results</title>
<p>The outcomes of the survey&#x2019;s data analytics with SPSS, interview sessions, and Tweeter data analysis using machine learning algorithms are discussed in this section.</p>
<sec id="s4_1">
<label>4.1</label>
<title>Survey Results</title>
<p>The data used in this study is largely dependent on the response from the software houses. The data analysis is performed on survey data with the help of the SPSS tool. The demographic analysis is shown in <xref ref-type="fig" rid="fig-2">Fig. 2a</xref> indicates that10.9&#x0025; of the organization&#x2019;s owners gave responses, while another 10.9&#x0025; of responses are given by senior management and 15.2&#x0025; of responses were given by middle-level management. 45.7&#x0025; of responses came from IT experts and 8.7&#x0025; of responses were given by entry-level employees or others. Similarly, <xref ref-type="fig" rid="fig-2">Fig. 2b</xref> indicates that 69&#x0025; of the interview respondents are IT experts, while 18&#x0025; are executives and 13&#x0025; of the respondents belong to senior management.</p>
<fig id="fig-2">
<label>Figure 2</label>
<caption>
<title>Demographic information of respondents (a) Responses of different organizational representatives (b) Expert types for the interview</title></caption>
<graphic mimetype="image" mime-subtype="png" xlink:href="IASC_21769-fig-2.png"/>
</fig>
<p>According to the survey data correlation analysis in SPSS, there is an inverse relationship between organization expertise and the choice to buy software; this indicates that the greater the expertise, the lower the possibilities of buying, and the lower the expertise, the higher the possibilities of buying, as shown in <xref ref-type="table" rid="table-2">Tab. 2</xref>. Pearson correlation tells about the direction and strength of the relationship between organization expertise and the decision of software buying.</p>
<table-wrap id="table-2"><label>Table 2</label>
<caption>
<title>Correlations of organization&#x2019;s expertise</title></caption>
<table><colgroup><col align="left"/><col align="left"/><col align="left"/><col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">Correlation Factors</th>
<th align="left"/>
<th align="left">Organization Expertise</th>
<th align="left">What does your organization do when the project is out of organizational expertise</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">Organization Expertise</td>
<td align="left">Pearson Correlation<break/>Sig. (2-tailed) N</td>
<td align="left">1<break/>46</td>
<td align="left">-.682&#x002A;&#x002A;<break/>000<break/>46</td>
</tr>
<tr>
<td align="left">What does your organization do when the project is out of organizational expertise</td>
<td align="left">Pearson Correlation<break/>Sig. (2-tailed) N</td>
<td align="left">-.682&#x002A;&#x002A;<break/>000<break/>46</td>
<td align="left">1<break/>46</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>The findings confirmed that there is a high association between organization expertise and the decision of software buying. The values of Pearson -.682&#x002A;&#x002A; confirmed the inverse proportion between the variables. The value -.682 is near 1 which means that there is a high association between the variables. Conclusively, one variable is negatively associated with the other variable. The findings of the study are consistent with [<xref ref-type="bibr" rid="ref-15">15</xref>,<xref ref-type="bibr" rid="ref-16">16</xref>,<xref ref-type="bibr" rid="ref-34">34</xref>&#x2013;<xref ref-type="bibr" rid="ref-36">36</xref>]. Also, there is a direct proportion between organization capacity and the decision of software build means that higher will be the capacity of the organization, higher will be chances of build and lower will be the capacity of organization lower will be the chances of the build as shown in <xref ref-type="table" rid="table-3">Tab. 3</xref>.</p>
<table-wrap id="table-3"><label>Table 3</label>
<caption>
<title>Correlations of organization&#x2019;s capacity</title></caption>
<table><colgroup><col align="left"/><col align="left"/><col align="left"/><col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">Correlation Factors</th>
<th align="left"/>
<th align="left">Organization Capacity</th>
<th align="left">What does your organization do when the project is out of organizational capacity</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">Organization Capacity</td>
<td align="left">Pearson Correlation<break/>Sig. (2-tailed) N</td>
<td align="left">1<break/>46</td>
<td align="left">.716&#x002A;&#x002A;<break/>000<break/>46</td>
</tr>
<tr>
<td align="left">What does your organization do when the project is out of organizational capacity</td>
<td align="left">Pearson Correlation<break/>Sig. (2-tailed) N</td>
<td align="left">.716&#x002A;&#x002A;<break/>000<break/>46</td>
<td align="left">1<break/>46</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Correlation is significant at the 0.01 level. The findings confirmed that there is a high association between organization capacity and the decision of software building. The values of Pearson .716&#x002A;&#x002A; confirmed the direct proportion between the variables. The value .716 is near 1 which means that there is a high association between the variables. It also means that one variable is positively associated with the other variable. So our hypothesis &#x201C;higher will be the capacity higher will be the chances of build in house&#x201D; is proved. The finding of this hypothesis is in line with [<xref ref-type="bibr" rid="ref-15">15</xref>,<xref ref-type="bibr" rid="ref-16">16</xref>,<xref ref-type="bibr" rid="ref-37">37</xref>,<xref ref-type="bibr" rid="ref-38">38</xref>]. Similarly, there is a direct proportion between Situational factors (Time, Cost, and Budget) and software building decisions, shown in <xref ref-type="table" rid="table-4">Tab. 4</xref>.</p>
<table-wrap id="table-4"><label>Table 4</label>
<caption>
<title>Correlations of organization&#x2019;s situational factor</title></caption>
<table><colgroup><col align="left"/><col align="left"/><col align="left"/><col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">Correlation Factors</th>
<th align="left"/>
<th align="left">Situational Factors (Time, Cost, and Budget)</th>
<th align="left">Considering the risk factors, you will go for in house build</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">Situational Factors (Time, Cost, and Budget)</td>
<td align="left">Pearson Correlation<break/>Sig. (2-tailed) N</td>
<td align="left">1<break/>46</td>
<td align="left">638&#x002A;&#x002A;<break/>000<break/>46</td>
</tr>
<tr>
<td align="left">Considering the risk factors, you will go for in house build</td>
<td align="left">Pearson Correlation<break/>Sig. (2-tailed) N</td>
<td align="left">.638&#x002A;&#x002A;<break/>000<break/>46</td>
<td align="left">1<break/>46</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>As stated, correlation is significant at the 0.01 level. The findings confirmed that there is highly association between situational factors (time, cost, and budget) and the decision of software building. The values of Pearson .638&#x002A;&#x002A; confirmed the direct proportion between the variables. The value .638 is near 1 which means that there is a high association between the variables. It also means that one variable is positively associated with the other variable. Hence, our hypothesis &#x201C;There is a direct proportion between Situational factors (time, cost and budget) and software building decision&#x201D; is correct. The results of the hypotheses are consistent with previous studies [<xref ref-type="bibr" rid="ref-17">17</xref>], [<xref ref-type="bibr" rid="ref-18">18</xref>,<xref ref-type="bibr" rid="ref-39">39</xref>,<xref ref-type="bibr" rid="ref-40">40</xref>].</p>
<p>In <xref ref-type="fig" rid="fig-3">Fig. 3</xref>, the learning curve shows that when a new project comes in, the organization checks its expertise. If the required expertise is higher in the organization, then more chances of build and if the expertise is low fewer chances of build and go for the buy. If the situational factors (time, cost, etc.) are higher than the decision made by the organization is build and vice versa. Also, the budget and capacity of the organization are checked, and if it is out of budget go for buy else build.</p>
<fig id="fig-3">
<label>Figure 3</label>
<caption>
<title>The learning curve for the build <italic>vs.</italic> buy decision</title></caption>
<graphic mimetype="image" mime-subtype="png" xlink:href="IASC_21769-fig-3.png"/>
</fig>
</sec>
<sec id="s4_2">
<label>4.2</label>
<title>Interview Findings</title>
<p>According to the first interviewee, organizational scale, capacity, and resources affect the build <italic>vs.</italic> buy decision. And it&#x2019;s better to build in-house if expertise is present. If the technology changes help can be taken from the vendors, then make sure to build in-house. Similarly, the second respondent agreed that the build versus buy decision depends on the time and cost. And it&#x2019;s better to build in-house if there is no issue of budget to complete the project on time. New technology can be adapted to compete in the market and get help from vendors. In contrast to previous comments, the third responded view is that the organizational requirements, risk factors, capacity, and resources affect the build <italic>vs.</italic> buy decision. If solutions are available, then go for them. Build something that the market wants. The point of view of the 4th respondent is organizational requirements, risk factors, resources, and capacity affect the build <italic>vs.</italic> buy decision. If solutions are available to go for it. If time is limited, will buy, and on the other hand if have a better solution outsource take it. Finally, the last respondent argues that build <italic>vs.</italic> buy depends on multiple factors and the size of the project. Try to build a system on your own if the technology change hires a new skilled person and builds in-house. However, if there is no possible solution then go for the buy option.</p>
</sec>
<sec id="s4_3">
<label>4.3</label>
<title>Result of Machine Learning Classifiers</title>
<p>Machine learning classifiers are used for the training, testing, and classification of data. Tweets of software industry experts are collected and are used for further verification of evaluated results of previous sections. Results are in line with the previous study [<xref ref-type="bibr" rid="ref-39">39</xref>]. <xref ref-type="table" rid="table-5">Tab. 5</xref> presents the results of all classifiers including logistic regression, na&#x00EF;ve Bayes, simple logistic, Bayes net, voted perceptron, and zeroR on the scale of precision, recall, f-measure, and accuracy.</p>
<table-wrap id="table-5"><label>Table 5</label><caption><title>Results of all classifiers</title></caption>
<table>
<colgroup>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th align="left">Classifier</th>
<th align="left">Precision</th>
<th align="left">Recall</th>
<th align="left">F-measure</th>
<th align="left">Accuracy (&#x0025;)</th>
</tr>
</thead>
<tbody>
<tr>
<td align="left">Logistic regression</td>
<td align="left">0.885</td>
<td align="left">0.885</td>
<td align="left">0.892</td>
<td align="left">91.03</td>
</tr>
<tr>
<td align="left">Naive bayes</td>
<td align="left">0.877</td>
<td align="left">0.877</td>
<td align="left">0.859</td>
<td align="left">84.3</td>
</tr>
<tr>
<td align="left">Simple logistic</td>
<td align="left">0.851</td>
<td align="left">0.851</td>
<td align="left">0.874</td>
<td align="left">90.6</td>
</tr>
<tr>
<td align="left">Bayes net</td>
<td align="left">0.838</td>
<td align="left">0.838</td>
<td align="left">0.875</td>
<td align="left">91.5</td>
</tr>
<tr>
<td align="left">Voted preception</td>
<td align="left">0.875</td>
<td align="left">0.875</td>
<td align="left">0.881</td>
<td align="left">91.3</td>
</tr>
<tr>
<td align="left">ZeroR</td>
<td align="left">0.838</td>
<td align="left">0.838</td>
<td align="left">0.875</td>
<td align="left">91.5</td>
</tr>
</tbody>
</table>
</table-wrap><p>Using the Logistic Regression classification technique, we detected 518 correctly identified tweets and 51 incorrectly categorized tweets, resulting in a 91 percent accuracy rate. The weighted average for Precision is 0.885, Recall 0.910, and F-Measure is 0.892 as shown in <xref ref-type="fig" rid="fig-4">Fig. 4</xref>.</p>
<fig id="fig-4">
<label>Figure 4</label>
<caption>
<title>Logistic regression results on build <italic>vs.</italic> buy</title></caption>
<graphic mimetype="image" mime-subtype="png" xlink:href="IASC_21769-fig-4.png"/>
</fig>
<p>By applying the Na&#x00EF;ve Bayes classification algorithm, 480 correctly classified tweets are found, incorrectly classified tweets are 89, and the accuracy rate is 84 percent. The weighted average for Precision is 0.877, Recall 0.844, and F-Measure is 0.859. Similarly, by applying the Simple Logistic classification algorithm, we have founded 516 correctly classified, incorrectly classified tweets are 53, and the accuracy rate is 90 percent. The weighted average for Precision is 0.851, Recall 0.907, and F-Measure is 0.874. By applying Bayes Net classification algorithm, we have founded 521 correctly classified tweets, incorrectly classified tweets are 48, and the accuracy rate is 91 percent.</p>
<p>The weighted average for Precision is 0.838, Recall 0.916, and F-Measure is 0.875. It is also analyzed that by applying the Voted Perceptron classification algorithm, we have founded 520 correctly classified tweets, incorrectly classified tweets are 49, and the accuracy rate is 91 percent. The weighted average for Precision is 0.875, Recall 0.914, and F-Measure is 0.881. Furthermore, by applying the ZeroR classification algorithm, we have founded 521 correctly classified tweets, incorrectly classified tweets are 48, and the accuracy rate is 91 percent. The weighted average for Precision is 0.838, Recall 0.916, and F-Measure is 0.875. The classification characteristics values of various methods are represented as a chart in <xref ref-type="fig" rid="fig-5">Fig. 5</xref>.</p>
<fig id="fig-5">
<label>Figure 5</label>
<caption>
<title>Results of Classifiers</title></caption>
<graphic mimetype="image" mime-subtype="png" xlink:href="IASC_21769-fig-5.png"/>
</fig>
</sec>
<sec id="s4_4">
<label>4.4</label>
<title>Proposed Framework</title>
<p>Based on the findings of the survey analysis and machine learning classification a framework for the build <italic>vs.</italic> buys decision is prosed as shown in <xref ref-type="fig" rid="fig-6">Fig. 6</xref> Expertise is drawn from a variety of fields in the proposed framework, including programming, database management, networking, and Android. A specific amount of time is allotted to accomplish a project or meet deadlines; finance includes resources (skilled labor costs, equipment costs, etc.); and capacity includes infrastructure, personnel, and machinery.</p>
<fig id="fig-6">
<label>Figure 6</label>
<caption>
<title>A proposed framework for the build <italic>vs.</italic> buy decision</title></caption>
<graphic mimetype="image" mime-subtype="png" xlink:href="IASC_21769-fig-6.png"/>
</fig>
<p>The greater the expertise within an organization, the more the chances of developing, and the lower the knowledge within an organization, the higher the possibilities of buying. More situational elements (time, cost budget) increase the likelihood of building, while fewer elements increase the likelihood of buying. If an organization&#x2019;s resources and capacity are limited, the chances of a build are low; conversely, if the organization&#x2019;s capacity is high, the probabilities of a build are higher. When a new project comes in, the framework shows that the organization checks to see if it has enough competence. The project is denied if the requisite expertise is not present inside the organization. Also, the organization&#x2019;s budget and capacity are examined, and if all of the categories are greater, the organization should build rather than buy.</p>
</sec>
</sec>
<sec id="s5">
<label>5</label>
<title>Conclusions and Future Work</title>
<p>For a long time, commercial of the shelf is available for organizations of all types, however, the problem is the right choice of &#x2018;build versus buy&#x2019;. All those organizations are facing difficulty in taking the right decision due to which a deep understanding of the system is required to find out core factors on which the decision to build or buy should be taken. In this research, we tried to help organizations in taking accurate decisions for software development. We used both qualitative and quantitative approaches to get the most appropriate results. Firstly, core factors are identified on which the decision depends. It shows that the capacity and situational factors (time, cost, and budget) are directly proportional, higher these factors best to build in-house and vice versa. Furthermore, organizational expertise is indirectly proportional to buy decisions, higher than the expertise fewer chances of buy and lower the expertise more chances of buy. Secondly, the context-based analysis of Twitter data is performed. Different machine learning algorithms have been used to get the finest results from Twitter data. Data are gathered about the Make <italic>vs.</italic> Buy decision. Dataset is passed through two stages (training and testing) to attain the best result from tweet data. A close examination of different machine learning classifiers has been done. Bayes Net and ZeroR have shown the best results inaccuracy.</p>
<p>This study is conducted on a limited number of SME organizations and the proposed framework is based on identified factors. The researchers can apply the identified elements in their research. In the future, the approach could aid major organizations in deciding whether to build or buy. The biasness can be handled by using data collected from various social media platforms. Furthermore, the tweets covering images can also be taken under observation.</p>
</sec>
</body>
<back><fn-group>
<fn fn-type="other">
<p><bold>Funding Statement:</bold> The authors would like to acknowledge the support of the Deputy for Research and Innovation, Ministry of Education, Kingdom of Saudi Arabia for this research through a grant (NU/IFC/INT/01/008) under the institutional Funding Committee at Najran University, Kingdom of Saudi Arabia.</p>
</fn>
<fn fn-type="conflict">
<p><bold>Conflicts of Interest:</bold> The authors declare that they have no conflicts of interest to report regarding the present study.</p>
</fn>
</fn-group>
<ref-list content-type="authoryear">
<title>References</title>
<ref id="ref-1"><label>[1]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>J. C.</given-names> <surname>Lee</surname></string-name>, <string-name><given-names>Y. C.</given-names> <surname>Shiue</surname></string-name> and <string-name><given-names>C. Y.</given-names> <surname>Chen</surname></string-name></person-group>, &#x201C;<article-title>Examining the impacts of organizational culture and top management support of knowledge sharing on the success of software process improvement</article-title>,&#x201D; <source>Computers in Human Behavior</source>, vol. <volume>54</volume>, pp. <fpage>462</fpage>&#x2013;<lpage>474</lpage>, <year>2016</year>.</mixed-citation></ref>
<ref id="ref-2"><label>[2]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>B.</given-names> <surname>Shahzad</surname></string-name>, <string-name><given-names>A. M.</given-names> <surname>Abdullatif</surname></string-name>, <string-name><given-names>N.</given-names> <surname>Ikram</surname></string-name> and <string-name><given-names>A.</given-names> <surname>Mashkoor</surname></string-name></person-group>, &#x201C;<article-title>Build software or buy a study on developing large scale software</article-title>,&#x201D; <source>IEEE Access</source>, vol. <volume>5</volume>, pp. <fpage>24262</fpage>&#x2013;<lpage>24274</lpage>, <year>2017</year>.</mixed-citation></ref>
<ref id="ref-3"><label>[3]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>I.</given-names> <surname>Ahmed</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Shahzad</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Umar</surname></string-name> and <string-name><given-names>B. A.</given-names> <surname>Khilji</surname></string-name></person-group>, &#x201C;<article-title>Information technology and SMEs in Pakistan</article-title>,&#x201D; <source>International Business Research</source>, vol. <volume>3</volume>, no. <issue>4</issue>, pp. <fpage>237</fpage>&#x2013;<lpage>240</lpage>, <year>2010</year>.</mixed-citation></ref>
<ref id="ref-4"><label>[4]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>I.</given-names> <surname>Shin</surname></string-name></person-group>, &#x201C;<article-title>Adoption of enterprise application software and firm performance</article-title>,&#x201D; <source>Small Business Economics</source>, vol. <volume>26</volume>, no. <issue>3</issue>, pp. <fpage>241</fpage>&#x2013;<lpage>256</lpage>, <year>2006</year>.</mixed-citation></ref>
<ref id="ref-5"><label>[5]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>B. S.</given-names> <surname>Buchowicz</surname></string-name></person-group>, &#x201C;<article-title>A process model of make-<italic>vs</italic>.-buy decision-making</article-title>,&#x201D; <source>The Case of Manufacturing Software IEEE Transactions on Engineering Management</source>, vol. <volume>38</volume>, no.<issue>1</issue>, pp. <fpage>24</fpage>&#x2013;<lpage>32</lpage>, <year>1991</year>.</mixed-citation></ref>
<ref id="ref-6"><label>[6]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>T.</given-names> <surname>Dong</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Liang</surname></string-name> and <string-name><given-names>X.</given-names> <surname>He</surname></string-name></person-group>, &#x201C;<article-title>Social media and internet public events</article-title>,&#x201D; <source>Telematics and Informatics</source>, vol. <volume>34</volume>, no. <issue>3</issue>, pp. <fpage>726</fpage>&#x2013;<lpage>739</lpage>, <year>2016</year>. [Online] Available: <uri xlink:href="http://dx.doi.org/10.1016/j.tele.2016.05.024">http://dx.doi.org/10.1016/j.tele.2016.05.024</uri>.</mixed-citation></ref>
<ref id="ref-7"><label>[7]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>S. E.</given-names> <surname>Shukri</surname></string-name>, <string-name><given-names>R. I.</given-names> <surname>Yaghi</surname></string-name>, <string-name><given-names>I.</given-names> <surname>Aljarah</surname></string-name> and <string-name><given-names>H.</given-names> <surname>Alsawalqah</surname></string-name></person-group>, &#x201C;<article-title>Twitter sentiment analysis: a case study in the automotive industry</article-title>,&#x201D; in <conf-name>Applied Electrical Engineering and Computing Technologies (AEECT), IEEE Jordan Conference</conf-name>, <conf-loc>Amman, Jordon</conf-loc>, pp. <fpage>1</fpage>&#x2013;<lpage>5</lpage>, <year>2015</year>.</mixed-citation></ref>
<ref id="ref-8"><label>[8]</label><mixed-citation publication-type="book"><person-group person-group-type="author">S. H. Peled, D. Roth and D. Zimak</person-group>, &#x201C;<article-title>Constraint classification for multiclass classification and ranking</article-title>,&#x201D; in <conf-name>NIPS&#x2019;02: Proceedings of the 15th International Conference on Neural Information Processing Systems</conf-name>, <conf-loc>MIT Press, 55 Hayward St., Cambridge, MA, United States</conf-loc>, pp. <fpage>809</fpage>&#x2013;<lpage>816</lpage>, <year>2002</year>.</mixed-citation></ref>
<ref id="ref-9"><label>[9]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Hall</surname></string-name>, <string-name><given-names>E.</given-names> <surname>Frank</surname></string-name>, <string-name><given-names>G.</given-names> <surname>Holmes</surname></string-name>, <string-name><given-names>B.</given-names> <surname>Pfahringer</surname></string-name>, <string-name><given-names>P.</given-names> <surname>Reutemann</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<article-title>The WEKA data mining software: An update</article-title>,&#x201D; <source>ACM SIGKDD Explorations Newsletter</source>, vol. <volume>11</volume>, no. <issue>1</issue>, pp. <fpage>10</fpage>&#x2013;<lpage>18</lpage>, <year>2009</year>.</mixed-citation></ref>
<ref id="ref-10"><label>[10]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>E. W.</given-names> <surname>Dijkstra</surname></string-name></person-group>, &#x201C;<article-title>Complexity controlled by hierarchical ordering of function and variability</article-title>,&#x201D; in <conf-name>Software Engineering: Report on a Conference Sponsored by the NATO Science Committee</conf-name>, <conf-loc>Garimisch, Germany</conf-loc>, pp. <fpage>181</fpage>&#x2013;<lpage>185</lpage>, <year>1968</year>.</mixed-citation></ref>
<ref id="ref-11"><label>[11]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>N.</given-names> <surname>Mehta</surname></string-name>, <string-name><given-names>D.</given-names> <surname>Hall</surname></string-name> and <string-name><given-names>T.</given-names> <surname>Byrd</surname></string-name></person-group>, &#x201C;<article-title>Information technology and knowledge in software development teams: The role of project uncertainty</article-title>,&#x201D; <source>Information and Management</source>, vol. <volume>51</volume>, no. <issue>4</issue>, pp. <fpage>417</fpage>&#x2013;<lpage>429</lpage>, <year>2014</year>.</mixed-citation></ref>
<ref id="ref-12"><label>[12]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>F.</given-names> <surname>Javeed</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Siddique</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Munir</surname></string-name>, <string-name><given-names>B.</given-names> <surname>Shehzad</surname></string-name> and <string-name><given-names>M. I.</given-names> <surname>Lali</surname></string-name></person-group>, &#x201C;<article-title>Discovering software developer&#x2019;s coding expertise through deep learning</article-title>,&#x201D; <source>IET Software</source>, vol. <volume>14</volume>, no. <issue>2</issue>, pp. <fpage>213</fpage>&#x2013;<lpage>220</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-13"><label>[13]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>H.</given-names> <surname>Dar</surname></string-name>, <string-name><given-names>M. I.</given-names> <surname>Lali</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Ashraf</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Ramzan</surname></string-name>, <string-name><given-names>T.</given-names> <surname>Amjad</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<article-title>A systematic study on software requirements elicitation techniques and its challenges in mobile application development</article-title>,&#x201D; <source>IEEE Access</source>, vol. <volume>6</volume>, pp. <fpage>63859</fpage>&#x2013;<lpage>63867</lpage>, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-14"><label>[14]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>P. C.</given-names> <surname>Jha</surname></string-name>, <string-name><given-names>V.</given-names> <surname>Bali</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Narula</surname></string-name> and <string-name><given-names>M.</given-names> <surname>Kalra</surname></string-name></person-group>, &#x201C;<article-title>Optimal component selection based on cohesion &#x0026; coupling for component based software system under build-or-buy scheme</article-title>,&#x201D; <source>Journal of Computational Science</source>, vol. <volume>5</volume>, no. <issue>2</issue>, pp. <fpage>233</fpage>&#x2013;<lpage>242</lpage>, <year>2014</year>.</mixed-citation></ref>
<ref id="ref-15"><label>[15]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>W.</given-names> <surname>Xiuting</surname></string-name> and <string-name><given-names>D.</given-names> <surname>Haibo</surname></string-name></person-group>, &#x201C;<article-title>Analysis of buying decision making process on Chinese clothing for Ethiopian young consumers</article-title>,&#x201D; in <conf-name>2011 Int. Conf. on Product Innovation Management (ICPIM 2011)</conf-name>, <conf-loc>Hubei, China</conf-loc>, pp. <fpage>754</fpage>&#x2013;<lpage>757</lpage>,. <publisher-name>IEEE</publisher-name>, <year>2011</year>.</mixed-citation></ref>
<ref id="ref-16"><label>[16]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>V.</given-names> <surname>Cortellessa</surname></string-name>, <string-name><given-names>F.</given-names> <surname>Marinelli</surname></string-name> and <string-name><given-names>P.</given-names> <surname>Potena</surname></string-name></person-group>, &#x201C;<article-title>An optimization framework for build-or-buy decisions in software architecture</article-title>,&#x201D; <source>Computers &#x0026; Operations Research</source>, vol. <volume>35</volume>, no. <issue>10</issue>, pp. <fpage>3090</fpage>&#x2013;<lpage>3106</lpage>, <year>2008</year>.</mixed-citation></ref>
<ref id="ref-17"><label>[17]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>H. J.</given-names> <surname>Rosenberg</surname></string-name>, <string-name><given-names>N. P.</given-names> <surname>Mols</surname></string-name> and <string-name><given-names>A. R.</given-names> <surname>Villadsen</surname></string-name></person-group>, &#x201C;<article-title>Make and buy&#x2014;an alternative to make or buy? An investigation of four theoretical explanations in danish municipalities</article-title>,&#x201D; <source>International Journal of Public Administration</source>, vol. <volume>34</volume>, no. <issue>8</issue>, pp. <fpage>539</fpage>&#x2013;<lpage>552</lpage>, <year>2011</year>.</mixed-citation></ref>
<ref id="ref-18"><label>[18]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>F.</given-names> <surname>Daneshgar</surname></string-name>, <string-name><given-names>G. C.</given-names> <surname>Low</surname></string-name> and <string-name><given-names>L.</given-names> <surname>Worasinchai</surname></string-name></person-group>, &#x201C;<article-title>An investigation of &#x2018;build <italic>vs.</italic> buy&#x2019;decision for software acquisition by small to medium enterprises</article-title>,&#x201D; <source>Information and Software Technology</source>, vol. <volume>55</volume>, no. <issue>10</issue>, pp. <fpage>1741</fpage>&#x2013;<lpage>1750</lpage>, <year>2013</year>.</mixed-citation></ref>
<ref id="ref-19"><label>[19]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S. K.</given-names> <surname>Bhatti</surname></string-name>, <string-name><given-names>M. I.</given-names> <surname>Lali</surname></string-name>, <string-name><given-names>B.</given-names> <surname>Shahzad</surname></string-name>, <string-name><given-names>F.</given-names> <surname>Javid</surname></string-name>, <string-name><given-names>F. U.</given-names> <surname>Mangla</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<article-title>Everaging the big data produced by the network to take intelligent decisions on flow management</article-title>,&#x201D; <source>IEEE Access</source>, vol. <volume>6</volume>, pp. <fpage>12197</fpage>&#x2013;<lpage>12205</lpage>, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-20"><label>[20]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>C. J.</given-names> <surname>Torrecilla-Salinas</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Sede&#x00F1;o</surname></string-name>, <string-name><given-names>M. J.</given-names> <surname>Escalona</surname></string-name> and <string-name><given-names>M.</given-names> <surname>Mej&#x00ED;as</surname></string-name></person-group>, &#x201C;<article-title>Estimating, planning and managing agile Web development projects under a value-based perspective</article-title>,&#x201D; <source>Information and Software Technology</source>, vol. <volume>61</volume>, pp. <fpage>124</fpage>&#x2013;<lpage>144</lpage>, <year>2015</year>.</mixed-citation></ref>
<ref id="ref-21"><label>[21]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>B.</given-names> <surname>Shahzad</surname></string-name>, <string-name><given-names>K. M.</given-names> <surname>Awan</surname></string-name>, <string-name><given-names>M. I.</given-names> <surname>Lali</surname></string-name> and <string-name><given-names>W. A.</given-names> <surname>Aslam</surname></string-name></person-group>, &#x201C;<article-title>Identification of patterns in failure of software projects</article-title>,&#x201D; <source>Joutnal of Information Science Engineering</source>, vol. <volume>33</volume>, no. <issue>6</issue>, pp. <fpage>1465</fpage>&#x2013;<lpage>1479</lpage>, <year>2017</year>.</mixed-citation></ref>
<ref id="ref-22"><label>[22]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>M. S.</given-names> <surname>Tiwana</surname></string-name>, <string-name><given-names>F.</given-names> <surname>Javeed</surname></string-name>, <string-name><given-names>M. I.</given-names> <surname>Lali</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Dar</surname></string-name></person-group> <conf-loc>and</conf-loc> <person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Bilal</surname></string-name></person-group><conf-loc>, &#x201C;</conf-loc><article-title>Comparative analysis of context based classification of twitter</article-title><conf-loc>,&#x201D;</conf-loc> in <conf-name>2018 Fourth Int. Conf. on Advances in Computing, Communication &#x0026; Automation (ICACCA)</conf-name><conf-loc>, Subang Jaya, Malaysia, pp.</conf-loc> <fpage>1</fpage>&#x2013;<lpage>5</lpage>, <publisher-name>IEEE</publisher-name>, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-23"><label>[23]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>A. M.</given-names> <surname>Abdullatif</surname></string-name>, <string-name><given-names>B.</given-names> <surname>Shahzad</surname></string-name> and <string-name><given-names>A.</given-names> <surname>Hussain</surname></string-name></person-group>, &#x201C;<article-title>Evolution of social media in scientific research: A case of technology and healthcare professionals in Saudi universities</article-title>,&#x201D; <source>Journal of Medical Imaging and Health Informatics</source>, vol. <volume>7</volume>, no. <issue>6</issue>, pp. <fpage>1461</fpage>&#x2013;<lpage>1468</lpage>, <year>2017</year>.</mixed-citation></ref>
<ref id="ref-24"><label>[24]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>B.</given-names> <surname>Shahzad</surname></string-name>, <string-name><given-names>K. M.</given-names> <surname>Awan</surname></string-name>, <string-name><given-names>A. M.</given-names> <surname>Abdullatif</surname></string-name>, <string-name><given-names>M. I.</given-names> <surname>Lali</surname></string-name>, <string-name><given-names>M. S.</given-names> <surname>Nawaz</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<article-title>Quantification of productivity of the brands on social media with respect to their responsiveness</article-title>,&#x201D; <source>IEEE Access</source>, vol. <volume>7</volume>, pp. <fpage>9531</fpage>&#x2013;<lpage>9539</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-25"><label>[25]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S. T.</given-names> <surname>Bakhsh</surname></string-name>, <string-name><given-names>B.</given-names> <surname>Shahzad</surname></string-name> and <string-name><given-names>S.</given-names> <surname>Tahir</surname></string-name></person-group>, &#x201C;<article-title>Risk management approaches for large scale software development</article-title>,&#x201D; <source>Journal of Information Science and Engineering</source>, vol. <volume>33</volume>, no. <issue>6</issue>, pp. <fpage>1547</fpage>&#x2013;<lpage>1560</lpage>, <year>2017</year>.</mixed-citation></ref>
<ref id="ref-26"><label>[26]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>K.</given-names> <surname>Saleem</surname></string-name>, <string-name><given-names>B.</given-names> <surname>Shahzad</surname></string-name>, <string-name><given-names>M. A.</given-names> <surname>Orgun</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Al-Muhtadi</surname></string-name>, <string-name><given-names>J. J.</given-names> <surname>Rodrigues</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<article-title>Design and deployment challenges in immersive and wearable technologies</article-title>,&#x201D; <source>Behaviour and Information Technology</source>, vol. <volume>36</volume>, no. <issue>7</issue>, pp. <fpage>687</fpage>&#x2013;<lpage>698</lpage>, <year>2017</year>.</mixed-citation></ref>
<ref id="ref-27"><label>[27]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>V.</given-names> <surname>Cortellessa</surname></string-name>, <string-name><given-names>F.</given-names> <surname>Marinelli</surname></string-name> and <string-name><given-names>P.</given-names> <surname>Potena</surname></string-name></person-group>, &#x201C;<article-title>Automated selection of software components based on cost/reliability tradeoff,&#x201D; in <italic>European Workshop on Software Architecture, Nantes, France</italic></article-title>&#x201D;, pp. <fpage>66</fpage>&#x2013;<lpage>81</lpage>, <year>2006</year>.</mixed-citation></ref>
<ref id="ref-28"><label>[28]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Choshin</surname></string-name> and <string-name><given-names>A.</given-names> <surname>Ghaffari</surname></string-name></person-group>, &#x201C;<article-title>An investigation of the impact of effective factors on the success of e-commerce in small-and medium-sized companies</article-title>,&#x201D; <source>Computers in Human Behavior</source>, vol. <volume>66</volume>, pp. <fpage>67</fpage>&#x2013;<lpage>74</lpage>, <year>2017</year>.</mixed-citation></ref>
<ref id="ref-29"><label>[29]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>J. M.</given-names> <surname>M&#x00FC;ller</surname></string-name></person-group>, &#x201C;<article-title>Business model innovation in small-and medium-sized enterprises: Strategies for industry 4.0 providers and users</article-title>,&#x201D; <source>Journal of Manufacturing Technology Management</source>, vol. <volume>30</volume>, no. <issue>8</issue>, pp. <fpage>1127</fpage>&#x2013;<lpage>1142</lpage>, <year>2019</year>. Available: 10.1108/jmtm-01-2018-0008.</mixed-citation></ref>
<ref id="ref-30"><label>[30]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>D. A.</given-names> <surname>Pacheco</surname></string-name>, <string-name><given-names>C. S.</given-names> <surname>Caten</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Jung</surname></string-name>, <string-name><given-names>C. F.</given-names> <surname>Sassanelli</surname></string-name> and <string-name><given-names>S.</given-names> <surname>Terzi</surname></string-name></person-group>, &#x201C;<article-title>Overcoming barriers towards sustainable product-service systems in small and medium-sized enterprises: State of the art and a novel decision matrix</article-title>,&#x201D; <source>Journal of Cleaner Production</source>, vol. <volume>222</volume>, pp. <fpage>903</fpage>&#x2013;<lpage>921</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-31"><label>[31]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>K. R.</given-names> <surname>Fowler</surname></string-name> and <string-name><given-names>S. A.</given-names> <surname>Dyer</surname></string-name></person-group>, &#x201C;<article-title>A model for the build-versus-buy decision in developing embedded systems</article-title>,&#x201D; <source>IEEE Systems Journal</source>, vol. <volume>14</volume>, no. <issue>1</issue>, pp. <fpage>1317</fpage>&#x2013;<lpage>1324</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-32"><label>[32]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>O.</given-names> <surname>Gomez</surname></string-name>, <string-name><given-names>P.</given-names> <surname>Wriedt</surname></string-name> and <string-name><given-names>F.</given-names> <surname>Zhao</surname></string-name></person-group>, &#x201C;<article-title>Build or Buy: A case study for ERP system selection in SMEs</article-title>,&#x201D; in <conf-name>Int. Conf. on Human-Computer Interaction</conf-name>, <conf-loc>Toronto, Canada</conf-loc>, pp. <fpage>23</fpage>&#x2013;<lpage>33</lpage>, <year>2016</year>.</mixed-citation></ref>
<ref id="ref-33"><label>[33]</label><mixed-citation publication-type="journal">SPSS Software, &#x201C;International Business Machines Corporation,&#x201D; (Accessed 6 May 2021), 2021. Available: <uri xlink:href="https://www.ibm.com/analytics/spss-statistics-software">https://www.ibm.com/analytics/spss-statistics-software</uri>.</mixed-citation></ref>
<ref id="ref-34"><label>[34]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>P.</given-names> <surname>Verschuren</surname></string-name></person-group>, &#x201C;<article-title>Case study as a research strategy: Some ambiguities and opportunities</article-title>,&#x201D; <source>International Journal of Social Research Methodology</source>, vol. <volume>6</volume>, no. <issue>2</issue>, pp. <fpage>121</fpage>&#x2013;<lpage>139</lpage>, <year>2003</year>.</mixed-citation></ref>
<ref id="ref-35"><label>[35]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>E.</given-names> <surname>Lloret</surname></string-name> and <string-name><given-names>M.</given-names> <surname>Palomar</surname></string-name></person-group>, &#x201C;<article-title>Text summarisation in progress: A literature review</article-title>,&#x201D; <source>Artificial Intelligence Review</source>, vol. <volume>37</volume>, no. <issue>1</issue>, pp. <fpage>1</fpage>&#x2013;<lpage>41</lpage>, <year>2012</year>.</mixed-citation></ref>
<ref id="ref-36"><label>[36]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>Q. N.</given-names> <surname>Naveed</surname></string-name>, <string-name><given-names>M. R. N. M.</given-names> <surname>Qureshi</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Shaikh</surname></string-name>, <string-name><given-names>A. O.</given-names> <surname>Alsayed</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Sanober</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<article-title>Evaluating and ranking cloud-based e-learning critical success factors (CSFs) using combinatorial approach</article-title>,&#x201D; <source>IEEE Access</source>, vol. <volume>7</volume>, pp. <fpage>157145</fpage>&#x2013;<lpage>157157</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-37"><label>[37]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M. A.</given-names> <surname>Memon</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Shaikh</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Sulaiman</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Alghamdi</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Alrizq</surname></string-name> <etal>et al.</etal></person-group>, &#x201C;<article-title>Time and quantity based hybrid consolidation algorithms for reduced cost products delivery</article-title>,&#x201D; <source><italic>Computers, Materials</italic> and Continua</source>, vol. 69, no. 1, pp. <fpage>409</fpage>&#x2013;<lpage>432</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-38"><label>[38]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>N. A.</given-names> <surname>Mahoto</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Iftikhar</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Shaikh</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Asiri</surname></string-name> and <string-name><given-names>A.</given-names> <surname>Alghamdi</surname></string-name></person-group>, &#x201C;<article-title>An intelligent business model for product price prediction using machine learning approach</article-title>,&#x201D; <source>Intelligent Automation &#x0026; Soft Computing</source>, vol. <volume>30</volume>, no.<issue>1</issue>, pp. <fpage>147</fpage>&#x2013;<lpage>159</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-39"><label>[39]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>A.</given-names> <surname>Sahaym</surname></string-name>, <string-name><given-names>A. A.</given-names> <surname>Datta</surname></string-name> and <string-name><given-names>S.</given-names> <surname>Brooks</surname></string-name></person-group>, &#x201C;<article-title>Crowdfunding success through social media: Going beyond entrepreneurial orientation in the context of small and medium-sized enterprises</article-title>,&#x201D; <source>Journal of Business Research</source>, vol. <volume>125</volume>, pp. <fpage>483</fpage>&#x2013;<lpage>494</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-40"><label>[40]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>P.</given-names> <surname>Maroufkhani</surname></string-name>, <string-name><given-names>M. L.</given-names> <surname>Tseng</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Iranmanesh</surname></string-name>, <string-name><given-names>W. K.</given-names> <surname>Ismail</surname></string-name> and <string-name><given-names>H.</given-names> <surname>Khalid</surname></string-name></person-group>, &#x201C;<article-title>Big data analytics adoption: Determinants and performances among small to medium-sized enterprises</article-title>,&#x201D; <source>International Journal of Information Management</source>, vol. <volume>54</volume>, no. <issue>102190</issue>, pp. <fpage>1</fpage>&#x2013;<lpage>15</lpage>, <year>2020</year>.</mixed-citation></ref>
</ref-list>
</back>
</article>