Here we utilize gradient descent to learn node convergence weight from real bandwidth data. In brief, we iteratively calculate the cost between the estimated bandwidth (which is derived from node convergence weight) and the actual bandwidth, and update convergence weight with gradient descent [40], until the cost becomes minimal.

First, we initialize node convergence weights as small positive numbers that are randomly generated within (0, 1), and substitute these weights and the known bandwidth requirements of lower nodes into Eq. (1) to derive the estimated bandwidth of the upper node y¯ .

Then we calculate the cost for gradient descent as follows:(3) cost=(y−y¯)2 where y is the actual bandwidth of the converged node.

The idea of gradient descent is to minimize cost by gradually adjusting node convergence weights. Specifically, the gradient of the cost function can be computed as:(4) ∂cost∂wi=2×(y−y¯)×Bi where w_i and B_i are the convergence weight and the bandwidth of the node i, respectively.

We gradually adjust the convergence weight of each node as follows:(5) wi←wi−α1×∂cost∂wi where α₁ is the learning rate of gradient descent. Its value will be determined by experiments later.

Eqs. (4) and (5) will be executed iteratively until the cost function converges to its minimum. The obtained convergence weight w_i essentially reflects the radio of node i’s bandwidth that converges to its upper node.

We use correlation coefficient calculation to decide which node characteristics (e.g., number or type of application) have more significant impacts on convergence weight. Only these key node characteristics will be concerned for bandwidth estimation. This is to simplify the acquirement process of node characteristics as well as the subsequent non-linear multivariate learning.

Specifically, for all N nodes, we calculate correlation coefficient between each candidate characteristic and the set of convergence weights as follows:(6) corr(w,Xj)=cov(w,Xj)var(w)⋅var(Xj) where w is the set of the convergence weights of all nodes (obtained in Section 5.1), X_j is the set of the values of a candidate characteristic for all nodes, cov(w, X_j) is the covariance of w and X_j, and var(w) and var(X_j) are the variance of w and X_j, respectively.

After calculating the correlation coeffecients for all candidate characteristics, we choose the key characteristics with higher correlation coeffecients for the non-linear multivariate learning in the next subsection. The number of key characteristics should be carefuly decided to balance between implemation complexity and bandwidth estimation accuracy. This will be further disccussed in Section 6.

At last, we exploit nonlinear multivariate fitting to reveal how a node’s convergence weight is affected by its characteristics. Here, “nonlinear” is to reflect the complexity of such relationship, and “multivariate” is to reflect the combined impacts of multiple node characteristics. Multivariate nonlinear fitting means using mathematical model to express the nonlinear relationship between different characteristics and the convergence weight.

Since we have derived the convergence weight in Section 5.1 and the key characteristics in Section 5.2, we only need to determine the rest unknown parameters in Eq. (2), namely, the multinomial functions f( ⋅ ) and g( ⋅ ), and the coefficients β_j and β_k+1. This is accomplished through fitting Eq. (2) to the convergence weight obtained in Eq. (5).

First, suppose that we have decided there are k key characteristics and the power of f( ⋅ ) is n (we will discuss how to derive them later). The nonlinear influence of each characteristic X_j on the convergence weight can be expressed by:(7) f(Xi)=a1Xi+a2Xi2+……+anXjn

and the nonlinearly correlated influence of the k key characteristics on the convergence weight can be expressed by:

(8) g(X1,X2…,Xk)=b(1,2)X1X2+b(1,3)X1X3…+b(1,2,…,k)X1X2…Xk where the coefficients a₁, …, a_n, b_(1,2), …, b_(1,2,…,k) will be learned later. It is noteworthy that the complexity of g( ⋅ ) is o(2^k), which is why we should keep the number of node characteristics k as small as possible.

Second, we substitute Eqs. (7) and (8) into Eq. (2) to express the fitted convergence weight w¯i for node i as follows:(9) w¯i=∑j=1kβj⋅f(Xj)+βk+1⋅g(X1,X2…Xk)

Third, we compare the fitted convergence weight w¯i in Eq. (9) to the convergence weight w_i derived by Eq. (5), i.e.,(10) loss=∑i=1n(wi−w¯i)2

Now we can learn the undetermined parameters via multivariate gradient descent [10]:(11) θ=θ←α2⋅∂loss∂θ where α₂ is the learning rate, and its value will be determined by experiments later. θ represents multiple variables a₁, …, a_n, b_(1,2), …, b_(1,2,…,k), β₁, …, β_k+1.

The above Eqs. (10) and (11) will be executed iteratively until the loss function Eq. (10) converges to its minimum. By repeating this process on all node convergence weights, the values of a₁, …, a_n, b_(1,2), …, b_(1,2,…,k), β₁, …, β_k+1 are learned.

Finally, we determine the hyperparameters that should be set before learning, namely, the learning rates α₁ in Eq. (5) and α₂ in Eq. (11), the number of the key node characteristics n, and the power of nonlinear fitting k in Eq. (7).

We begin with the learning rates α₁ and α₂. We initially set them to very small values, e.g., 10⁻⁸, and observe that whether the cost in Eq. (3) or the lost in Eq. (10) steadily decreases as we perform linear fitting in Eq. (4) or nonlinear fitting in Eq. (11), respectively. If the decrement is too slow, we gradually increase α₁ or α₂ to learn the unknown parameters more drastically. On the other hand, if the decrement is unstable, we gradually decrease α₁ or α₂ to learn more cautiously. We keep adjusting α₁ and α₂ until the decrement is stable and notable. The resulting α₁ and α₂ will be used for learning later.

Afterwards, we determine the number of the key node characteristics n, and the power of nonlinear fitting k. For practical considerations, we should set them as small as possible (otherwise, there are too many variables to learn in Eqs. (7) and (8)). Therefore, we gradually increase them from n = 1 and k = 1. For each pair of (n, k), we use Eq. (6) to find the k key node characteristics, use Eqs. (9)–(11) to derive the other unknown parameters in Eqs. (7) and (8), and use Eqs. (1) and (2) to obtain bandwidth estimations for all upper nodes. This increment process stops as the bandwidth estimation accuracy has become reasonably high and the accuracy increment has become marginal. The pair of (n, k) that has the highest bandwidth estimation accuracy will be used for learning later.

After determining the hyperparameters α₁, α₂, n, and k, we can learn all parameters’ values in Eqs. (1) and (2) from a present network with Eqs. (3)–(11). Then we can use Eqs. (1) and (2) to estimate the bandwidth requirements of new networks.

We learn the estimation parameters from the network in Fig. 4. This network is based on the regional power grid of a moderate-sized city in China, which has 2 regional dispatching centers, 9 220 kV substations, and 1 110 kV substation. It represents a “present network” where the nodes’ bandwidth consumptions have been known. During learning, Z2 node is used for validating, and the rest nodes are used for fitting.

We first determine the linear learning rate α₁ and the nonlinear learning rate α₂. We increase α₁ and α₂ from 10⁻⁸ to 10⁻⁴, respectively, and find that the fitting costs of Eqs. (4) and (11) stably decrease only when α_1 ≤10⁻⁶ and α_2 ≤10⁻⁶. Since larger learning rates lead to quicker learning, we choose α₁=10⁻⁶ and α₂=10⁻⁶.

Next, we investigate 8 node characteristics that maybe related to convergence weight, as listed in Tab. 2.

We calculate correlation coefficient as Eq. (6) to find that there are k = 3 node characteristics closely related to convergence weight, which are: node’s total bandwidth > number of real-time applications > node strength.

We show 4 node characteristics with the highest correlation coefficient values in Tab. 3, which are node’s total bandwidth, number of real-time applications, node strength, and number of normal applications. It can be seen that while the first 3 characteristics have correlation coefficients larger than 0.6, the fourth characteristic (i.e., number of normal applications) drastically drops to 0.36. Such result indicates that number of normal applications (and the characteristics after it) has statistically ignorable impacts on convergence weight.

The above result is reasonable. Firstly, a node’s total bandwidth basically reflects how important it is for smart grid (as complex applications tend to be deployed in critical grid sites), which means that a node with higher bandwidth requirement often has proportionally more data to upload. Secondly, most real-time applications need to communicate with dispatching center, so a node with more real-time applications implies that it has more data to upload. Thirdly, a node with higher node strength means that it is connected by many other nodes, so it tends to have more data to upload. Later we will show that our estimation based on these three characteristics indeed achieves rather high accuracy.

Afterwards, we determine the power of Eq. (7), n. According to Eq. (9), the value of n directly affects the fitting performance from node characteristics to convergence weight. To demonstrate this, we draw the fitting curves relating the 3 node characteristics (for conciseness, we only show each node’s total bandwidth on the x-axis) to the 11 nodes’ convergence weights in Fig. 5. Observe that the fitting curve can barely match itself to all the points when n ≤ 2, which means that the relationship between the node characteristics and the convergence weight is too complicated for these values of n to capture. When n ≥ 3, the fitting curve is able to reach most of the points, and thereby the convergence weight is well related to the node characteristics.

Nevertheless, according to machine learning theory, n being too large will lead to overfitting, that is, our method can achieve high accuracy during learning, but its accuracy will drop if it is applied to nodes excluded by learning process (i.e., Z2 in this case). This statement is verified by Tab. 4, where we compute the average bandwidth estimation accuracy across the network for different values of n. Observe that although the fitting accuracy keeps increasing as n grows, the validating accuracy for Z2 reaches the maximum at n = 3, and decreases as n becomes larger. This clearly indicates that overfitting occurs for n > 3. Combing this result with Fig. 5, we let n = 3.

Finally, with k = 3 and n = 3, we can further derive the values of a₁, …, a_n, b_(1,2), …, b_(1,2,…,k), β₁, …, β_k+1. The deriving process and the final results are omitted here.

Now we use the results in Section 6.1 to estimate the network bandwidth in Fig. 6. This network is based on a small city’s power grid, which has 1 regional dispatching centers, 1 220 kV substations, 5 110 kV substation, and 1 35 kV substation. This represents a “new network” where only applications’ bandwidth requirements are known. Note that the network is highly heterogeneous with 4 different types of nodes, which is difficult for bandwidth estimation.

We are mostly interested in the estimation accuracy of the dispatching center, because it is the most important node in smart grid’s control system, and its estimation accuracy basically depends on the estimation accuracies of all the other nodes. Tab. 5 compares our method with 3 recent works in [2,4,6], where t, a, and b mean that network topology, applications, and node’s total bandwidth are considered in estimation, respectively.

Observe that our method (t + a + b) is just slightly higher than the actual bandwidth requirement of 759.56 Mbps, which outperforms the alternative methods (PCCM [2], CA [4], and CBT [6]) by 12.6% to 56.5% higher accuracy. The major reason is that our method accomplishes finer-grained estimation via considering network topology (node strength), applications (number of real-time applications), and node bandwidth as key node characteristics (see Section 6.1). In contrast, the alternative methods only consider one or two of these characteristics, thus they can hardly differentiate the convergence impacts of different lower-voltage nodes on the dispatching center.

In fact, Tab. 5 also shows that our method achieves higher accuracy as it takes more characteristics into consideration, i.e., t < (t + a) < (t + a + b). Such result further proves that finer-grained estimation leads to higher accuracy.

We further investigate how the selection of node characteristics affects convergence weight learning, and eventually affects bandwidth estimation. This can be clearly illustrated by the ranking of learned convergence weights of different methods in Tab. 6.

By considering node strength (t), number of real-time applications (a), and node bandwidth (b), our method derives the same ranking as the actual network does. As the matter of fact, for the dispatching center (S7), the ranking of the convergence weights of all nodes should be: itself (S7), the nodes linked by lower-voltage nodes (S2, S6, and S4), the nodes without lower-voltage nodes (S0, S1, S3), and the lowest-voltage nodes (S5, S8). Our method’s fine-grained recognition of different nodes is the key to accurate bandwidth estimation.

On the other hand, because PCCM ignores network topology and node bandwidth, it cannot fully recognize the differences among various nodes to learn convergence weights in a heterogeneous network like Fig. 6. This explains why PCCM has rather low estimation accuracy in Tab. 5.

Both CA and CBT have considered topology for estimation, so they can roughly infer how the nodes converges the dispatching center, and hence make partially correct rankings in Tab. 6. However, note that both CA and CBT incorrectly rank S3 after S5 due to their neglection of node bandwidth. This explains their relatively low estimation accuracy in Tab. 5.