As our goal is to better cluster tasks and services to divide-and-conquer massive manufacturing resources, in order to simplify the supply-demand adaptation problem in SCM. Therefore, distinguish from traditional methods, we consider the potential relationships between tasks or services to improve the performance of clustering. Inspired by reference [18], a collaborative network model is established to provide a knowledge representation basis for the supply-demand adaptation of service resources, as well as to better divide-and-conquer the supply-demand adaptation problem.

(1) Network Model for Manufacturing Tasks

The tasks in SCM can be divided into indissoluble meta-tasks and composite tasks composed of meta-tasks, and the combination of meta-tasks into a composite task follows the rules of a working procedure and time sequence. Therefore, it is necessary to construct a task network (TN) model based on a weighted directed graph, in which positive and negative weights represent the directions of the edges:

(1)TN=<T,ET>

(2)T={Tpri1,Tpri2,…Tpril}

(3)ET={eij},ET={eij};i,j=1,2,…,l

(4)Tpri=<InputT,OutputT,StT,EtT,Rc,…>where T represents the set of task nodes; ET is the set of edges, which represent relationships between tasks; Tpri is a meta-task, and its features are extracted from a specific description of the physical manufacturing resources [21]; InputT, OutputT, StT and EtT are the input, output, starting time and ending time of the task, respectively; Rc={rc1,rc2,…} is a collection of the task’s consumption of various customized QoS indicator attributes; eij represents the type of edge between the i-th and j-th meta-tasks; and wij is the weight of an edge. There are three kinds of relationships that need to be defined:

Definition 1. Similar Tasks: When eij=−1, OutputTi=OutputTj or InputTi=InputTj, Tprii and Tprij are similar tasks, and the edge between them is undirected;

Definition 2. Complementary Tasks: When eij=1 and OutputTi=InputTj,EtTi≰EtTj, or eij=1,OutputTj=InputTi, and EtTj≰EtTi, Tprii and Tprij are complimentary tasks, and the edge between them is undirected;

Definition 3. Sequential Tasks: When eij=1,wij=1,OutputTi=InputTj,and EtTi≤EtTj, or eij=1,wij=−1,OutputTj=InputTi,and EtTj≤EtTi, Tprii and Tprij are sequential tasks, and the edge between them is directed from the preorder to the postorder.

(2) Network Model for Manufacturing Services

Service providers generally do not provide sequential services in the process of the supply-demand adaptation of service resources driven by task requirements [22], which makes the use of sequential relationships inappropriate for describing the services. Therefore, the service network (SN) model that is to be built is based on an undirected graph:

(5)SN=<S,ES>

(6)S=(Spri1,Spri2,…,Sprim)

(7)ES={eij};i,j=1,2,…,m

(8)Spri=<Inputs,Outputs,Sts,Ets,Rg>where S represents the set of service nodes; ES is the set of edges, which represent relationships between services; Spri is a meta-service; InputS, OutputS, StS and EtS are the input, output, starting time and ending time of the service, respectively; R={rg1,rg2,…} is a collection of the service’s gross of various customized QoS indicator attributes; and eij represents the type of edge between the i-th and j-th meta-services, which can indicate two kinds of relationships:

Definition 4. Similar Services: When OutputSi=OutputSj or InputSi=InputSj, Sprii and Sprij are similar services, eij=−1;

Definition 5. Complementary Services: When OutputSi=InputSj or OutputSj=InputSi, Sprii and Sprij are complementary services, and eij=1.

This paper considers the supply-demand adaptation problem of SCM in the adaptation domain:

(9)M=<TN,SN>

Assume that Tprik∈TN and Spril∈SN represent the k-th meta-task in TN and the l-th meta-service in SN, respectively. When task Tprik is assigned to Spril, some of the QoS indicator resources of Spril are occupied. In most cases, multiple tasks will match multiple services. To determine the best combination, a generalized QoS-driven supply-demand adaptation model is introduced in this paper:

(10)maxQoS=Q(ibr,obr)where Q is a function of various evaluation indexes. Different definitions of Q represent different preferences of decision-makers; therefore, the idea of QoS here is generalized.

ibr is the local equilibrium degree of service, and it measures the difference level of QoS indexes within the service:

(11)ibr=∑i=1m(roi−RO)2/pwhere roi represents the occupancy rate of the i-th indicator, RO represents the average occupancy rate of various indicators, and p is the total number of indicators.

obr is the global equilibrium degree of services; it measures the difference level of the occupancy rate of normalized QoS indexes between services and is expressed in the form of variance:

(12)obr=∑i=1n(ROi−RO¯)2/mwhere ROi represents the mean occupancy rate of various indicators in the i-th service, RO¯ is the mean value of RO, and m is the number of services in the adaptation domain.

On the basis of the intelligent adaptation model of cloud manufacturing established in this paper, multiple constraints of service resource adaptation on both sides of supply and demand of intelligent cloud manufacturing should also be considered. The adaptation problem in this paper follows the principles of function matching and time domain (TD) matching. That is, the functions provided by the service must meet the requirements of the task, and the time of the task must be within the cycle of the service. Therefore, the matching of Tprik and Spril should satisfy the constraints below:

Constraint 1. Functionality Constraint: InputTk⊆InputSl and OutputTk⊆OutputSl.

Constraint 2. TD Constraint: StTk≥Stsl and EtTk≤Etsl.

For the indexes in Rc and Rg, the upper limit of the index, described below, should also be satisfied.

Constraint 3. Index Upper Limit Constraint:

(13)∀i∈[1,m],∀j∈[1,l],∑k=1prcijk≤rgijwhere m is the number of indicators, n is the number of services in M, p is the number of tasks assigned to the corresponding service, rcijk is the occupation of the i-th index of the k-th task assigned to the j-th service, and rgij represents the upper limit of the i-th index of the j-th service.

Considering the high spatiotemporal search complexity of the collaborative network model established in this paper, which is large scale and highly hybrid in the adaptation domain M=<TN,SN>, the computational speed is extremely slow. To overcome the disadvantages mentioned above, a novel method is proposed to cluster the nodes in TN and SN and then sample multiple service and task subdomains from the clusters. Finally, the global network model is divided into various adaptation subdomains that have similar distributions to the global model to enable divide-and-conquer.

(1) Optimization of Adaptation Domains Based on DGC

Optimization of adaptation domains of massive supply and demand data includes two steps: Clustering adaptation domains to form supply and demand service resource sets of different functional groups, and sampling from the above different resource sets to form multiple sub-adaptation domains.

The collaborative supply-demand network model is a graph network that considers the properties of the supply-demand and their relationships between the supply and demand. Inspired by [24], this paper proposes a novel graph clustering method based on graph neural networks and deep learning.

This method uses a graph attention network (GAT) to generate a unique vector representation of the nodes in the network model; next, it divides the nodes into several categories according to these vector representations with clustering algorithms. Then, it assigns the same pseudo-label to nodes in the same category; finally, it uses the pseudo-labels to train the GAT to achieve a satisfactory clustering result. The clustering process is shown in Fig. 3.

The GATs can handle both directed and undirected graph, due to its vertex-by-vertex operation mode, that is, each operation needs to loop through all the vertices on the graph. It means that the problem of directed graph can be solved by getting rid of the constraint of Laplacian matrix. So, the proposed DGC, which adopted GAT, can operate both the directed TN and the undirected SN.

The sampling algorithm guarantees the distribution consistency between the generated adaptation subdomain and the entire adaptation domain by ensuring that the subdomain covers the samples in each cluster and that the proportion of samples from each cluster in the subdomain is the same as that in the entire domain. Before sampling, to eliminate the interference of singular clusters, which have very small sample sizes, each singular cluster should be merged into the corresponding cluster with the smallest distance from its cluster center. After that, the number of samples in the smallest cluster should be ensured to be much larger than the number of subdomains n. On this premise, random sample replication is used for each cluster to supplement its number of samples, an integer multiple of n. Finally, the entire domain is divided into several adaptation subdomains Msubs after AS:

(14)Msubs={msub1,msub2,…,msubk,…,msubn}

(15)msubk=<tsubi,ssubj>

(16)ssubj∈Ssubs={ssub1,ssub2,…,ssubj,…,ssubn}

(17)tsubi∈Tsubs={tsub1,tsub2,…,tsubi,…,tsubn}where msubk=<tsubi,ssubj> represents one adaptation subdomain; Ssubs and Tsubs are sets of service and task subdomains, respectively; and ssubj and tsubi are a service and a task subdomains in Ssubs and Tsubs, respectively.

(2) Recursive Optimization Mechanism of Adaptation Domains

In the case of huge supply and demand adaptation data, the complexity of sub-adaptation domain is still too high after optimization of the adaptation domain. The adaptive domain optimization method proposed in this paper can be applied to sub-adaptive domains to further reduce the complexity of adaptation. That is, the optimization method can be used recursively until the complexity of the sub-adaptation domain is reduced to a reasonable value.

(3) Divide-and-conquer Algorithm Based on DGC and AS

To simplify the network model, first is to classify supplies and demands through DGC algorithm. Second, adaptation subdomains with similar distributions and less complexity are formed by applying AS to the clustered supply and demand instances. The divide-and-conquer algorithm for adaptation based on DGC and AS is shown in Algorithm 1:

The supplies and demands may appear or disappear at any time in SCM. The traditional optimization algorithm must address the problem that a tiny fluctuation will lead to a large change in the solution space. Therefore, this paper adopts the DRL algorithm, which has the abilities of migration and dynamic learning, to independently find the best adaptation scheme under a wide range of complicated decision-making [25]. At the same time, the parameters generated by DRL can be transferred among subdomains to improve the efficiency and results of adaptation tasks in other adaptation subdomains.

(1) Transformation of an Adaptation Task into an RL Problem

The adaptation environment in this paper is denoted as SDEnv, and the adaptation process is described as follows: In the adaptation domain msubi, each task finds the most suitable service according to the current state of services. Accordingly, each such process is regarded as a STEP in the training of the RL, and an EPISODE is ended after all tasks have been allocated or when there are abnormalities. We set the REWARD as a polynomial in the QoS and denote an OBSERVATION of RL as ⟨tcur,Ssub′⟩, where tcur is the feature set of the current task to be assigned and Ssub′ is the feature set of all services as well as the assigned tasks in the adaptation domain. An ACTION is the sequence number of the service that matches the task. In addition, if the matching result given by RL does not meet the constraint of supply-demand adaptation, the REWARD is directly set to a negative value as punishment [26]. The details of SDEnv are shown in Tab. 1.

(2) Collaborative Mechanism of S-D Adaptation Based on TL

The distributions among adaptation subdomains are similar, so the parameters of DRL can be transferred among subdomains at a low cost. Coupled with the randomness of DRL, it is easy to find the optimal strategy using parallel training. Therefore, we combine the parameter transfer mechanism with RL; that is, each subdomain performs an independent RL process and carries out parameter transfer among processes simultaneously. Transfer is achieved through an external shared parameter pool, where each process uploads its parameters to the pool and searches for better parameters from the pool to apply to itself according to the rules.

(3) Supply-demand Adaptation Algorithm Based on DRL Powered by a Transfer Mechanism

It is necessary to establish a proper external shared parameter pool to share the experiences and conduct transfer learning in order to perform supply-demand adaptation and result optimization. This paper takes the double DQN (DDQN) algorithm [27,28] as the baseline, as shown in Algorithm 2:

To achieve the intelligent dynamic adaptation of massive service resources, we propose a divide-and-conquer method based on DGC simplify the complex adaptation problem. In this experiment, both the Cora and the supply-demand service datasets are used to verify the validity of the method.

(1) Results on the Cora Dataset

In accordance with reference [30], the first 140 nodes in the dataset are selected for training, and the last 1000 nodes are selected for testing, while another 500 nodes are selected in the middle for cross-validation. The number of clusters is set to 7 according to the number of categories in the Cora dataset to obtain the best clustering results. Based on the parameter setting in reference [30] and the pretraining carried out before the experiment, the specific parameters are shown in Tab. 2:

Where Epochs is the number of GAT’s training cycles; Episodes is the number of cycles, which includes the complete procedure of GAT forward propagation, clustering, pseudo-label generation and GAT training, processed by DGC; and Patience is the threshold at which the training of the GAT stops prematurely.

The adjusted Rand index (ARI), adjusted mutual information (AMI)-based scores, the mean of homogeneity and completeness (V-measure), Fowlkes-Mallows scores and the silhouette coefficient (SC) are used to evaluate the clustering algorithms. Only the SC is used for situations where the ground truth is unknown; therefore, it is also used to evaluate the results on the supply-demand service dataset. We also use the results of K-means (KM), the Gaussian mixture model (GMM), the spectral clustering algorithm (SCA) and k-means++ (KMPP), which are implemented in scikit-learn, as comparisons. Since none of the above algorithms can properly process graph data, only the features of the nodes in the graph are considered, when clustering with these algorithms.

As shown in Fig. 4, the performance of DGC on various evaluation indicators increases with the number of iterations and tends to level off and eventually converge after 5-6 iterations. This is because the training of the DGC makes its neural network increasingly close to reality, leading to the correction and redistribution of pseudo-labels. The results of the clustering algorithms other than DGC fluctuate randomly, mainly because they have no “learning” process. Additionally, the result of the first episode of DGC is worse than that of its counterparts because the weights of the GAT are initialized randomly. It is also clear that the similarity between the results of DGC and the ground truth shows an obvious advantage over other traditional algorithms. That is, the results of DGC better represent the real categories of the original data.

Fig. 5a shows the variation in the number of samples in each cluster of DGC. In the first few episodes, it varies greatly because the DGC has not been fully trained. As training goes on, fewer samples are redistributed, and a balanced state is reached within approximately 5–6 episodes. This result is consistent with Fig. 4; that is, when the clustering results tend to be stable, the evaluation indicators also tend to be at their highest value. Fig. 5b shows the number of iterations required for the DGC and KMPP algorithms to converge. The KMPP algorithm fluctuates irregularly, while the DGC algorithm, as the neural network is trained, gradually reduces the number of iterations. Similarly, it reaches a balanced state at 5–6 episodes, which is consistent with the previous conclusion.

(2) Results on the Supply-demand Service Dataset

The clustering result of the DGC algorithm on the task network and service network is also verified in this paper. We set different numbers of clustering categories and then calculate the mean value in each case. The results are shown in Fig. 6.

Fig. 6 shows the results on the supply-demand service dataset and are consistent with the results on the Cora dataset. The clustering results of DGC improve as training deepens, and finally, it is far superior to the traditional algorithm. In summary, these experimental results show the effectiveness of the simplification method for the proposed collaborative network model. The divide-and-conquer algorithm based on DGC and AS can perform better classification and improve the distribution consistency between the adaptation subdomain and the entire domain. It also reduces the complexity of the intelligent adaptation algorithm and reduces the cost of the transfer of experience. Finally, it significantly reduces the difficulty of dynamic adaptation of large amounts of resources.

In Section 3, the problem of supply-demand adaptation was transformed into a problem of RL, and the DDQN algorithm, which has stable performance, was selected as the baseline for this problem. After some pretraining sets, the parameters of DDQN are shown in Tab. 3.

In this table, Memory Size represents the maximum number of trajectories stored in the playback library; ε-greedy and Increment are the maximum probability of nonrandom exploration and the increment rate of it in each iteration, respectively; and TargetNetReplacementIteration represents the update interval of the target net. Due to the randomness of RL, the indicators will fluctuate greatly. Therefore, in a total of 60,000 episodes, we calculated the mean value every 500 episodes, and then formed 120 data points.

(1) Results of Intelligent Adaption

In this paper, the results of the intelligent supply-demand adaptation method for the model based on the generalized QoS are shown in Fig. 7.

Fig. 7 shows the changes in the normalized QoS value and the two evaluation indicators. As shown in the figure, the algorithm begins to learn in the 24th evaluation period after storing 200,000 experiences. Since the QoS function we set prefers the obr value, it decreases after learning, while the other indicator increases accordingly due to the strict relationship between them. Once learning starts, the indicators gradually become stable after 20 more evaluation periods, and the normalized QoS increases in fluctuation and finally tends toward stability, indicating that the method in this paper can ultimately achieve the goal of supply-demand adaptation.

(2) Effectiveness in Solving the Dynamic Supply-demand Adaptation Problem

We studied the probability of completing the adaptation task and the change in the cumulative reward obtained after completion.

Fig. 8a shows the variation in the average failure rate of the adaptation tasks. For the subdomain, the failure rate reaches as high as 85% before learning begins, due to the random exploration of the DDQN. However, once learning starts, it rises briefly and then rapidly falls. After reaching a certain value, the failure rate declines slowly, with fluctuations, and eventually falls below 30%. This indicates that the RL algorithm has learned the law of supply-demand adaptation and is able to process the situation and then allocate the tasks effectively. In contrast, the adaptation task in the entire adaptation domain, which doesn’t adopt the divide-and-conquer method, cannot make any progress under the corresponding computational cost due to the large search space caused by the dimensional explosion. This indirectly demonstrates the practicality and effectiveness of the proposed divide-and-conquer method. Fig. 8b shows the change in the cumulative reward, which has the similar change law shown in Fig. 8a, and it also shows the effectiveness of the DRL algorithm in solving the problem of supply-demand adaptation.

In summary, RL has inherent advantages in solving dynamic supply-demand adaptation problems. When a task or service is introduced or removed, this algorithm can adapt according to previous experience without re-optimization, which greatly improves the efficiency of adaptation. Therefore, the method proposed in this paper can learn the laws of adaptation and can solve the dynamic adaptation problem.

(3) Advantages of TL in Solving Massive Supply-demand Adaptation Problems

In this paper, TL is used to share experiences in the adaptation subdomains to an external shared parameter pool as candidate experiences for other subdomains, and the performance improvements are investigated.

Fig. 9a shows the variation in the failure rate of the TL and non-TL methods. After the use of TL, the failure rate immediately decreases by approximately 30% compared with that of the non-TL method, indicating that the experience in the subdomain has some applicability. With the deepening of training, the final failure rate of TL is 5%–10% lower than that without TL, indicating that TL can improve the performance of adaptation. Fig. 9b shows how the rewards change, and the same conclusion can be drawn from this figure.

We also recorded the change in the adaptation results and adaptation efficiency in each adaptation subdomain by turning the TL mechanism on and off.

As shown in Fig. 10a, the initial adaptation result of TL was greatly improved, with an average of 37.3% improvement in any subdomain. For the final adaptation result, there was an improvement in most cases, with the average reaching 4.7%. Fig. 10b shows a comparison of the number of evaluation periods needed with and without TL when achieving 80% of the optimal adaptation result. This indicator shows the advantage of TL in improving training efficiency. In most cases, TL has an advantage of more than 20 evaluation periods, and the mean number is 24.647.

In summary, using TL to transfer parameters between adaptation subdomains can greatly improve not only the results but also the efficiency of adaptation. The excellent results of TL also indirectly show that there is a very similar distribution rule in the adaptation subdomains and the entire adaptation domain, which proves the great benefits of applying the divide-and-conquer method. Therefore, as a part of the framework proposed in this paper, TL greatly improves the ability to perform massive supply-demand adaptation.