The input data X∈Rw×m is a multivariate time series consisting of m variables, with the length of each variable being w. The standard Transformer network maps all the variables in X at the same time to a d-dimensional vector through an embedding layer and then performs feature extraction on it. The d-dimensional vectors at different time points form a matrix Ut=[u1,u2,⋯,ui,⋯,uw], where Ut∈Rw×d, ui represents the d-dimensional vectors of all variables at the time ti after embedding, as is shown in Eq. (1): (1)ui=Wpx(ti)+bp,where x(ti)=[x1(ti),x2(ti),⋯,xi(ti),⋯,xm(ti)], x(ti)∈Rm, and xi(ti) represent the sample values of the i-th variable at the time i; Wp∈Rd×m is the weight coefficient matrix; and bp is the bias.

The output feature matrix Ut partitions the multi-dimensional time series of the input into blocks along the time dimension, forming a feature representation centered on time, thus enabling the model to better capture the dynamic evolution patterns of the multi-variable time series. However, this approach causes the independent features of different variables to be intertwined into multi-dimensional features, making it difficult for the model to distinguish and capture the dynamic correlations between different variables. Therefore, the paper constructs a parallel embedding layer structure to solve this problem, as shown in Fig. 2.

Fig. 2a shows the temporal embedding layer, which retains the feature embedding approach used in the standard Transformer model, and this operation ensures that the model can fully capture the evolution patterns of the sequence along the time axis. Fig. 2b shows the variable embedding layer, which embeds all the sample values of a single variable at all time points into a d-dimensional vector, and after embedding, the m variables form the matrix Vα=[v1,v2,⋯,vj,⋯,vm], which Vα∈Rm×d, vj represents the d-dimensional vector obtained by embedding the sample values of the j-th variable at all time points, as shown in Eq. (2): (2)vj=Wqx(αj)+bq,where x(αj)=[x1(αj),x2(αj),⋯,xj(αj),⋯,xw(αj)], x(αj)∈Rw, xj(αj) is the sample value of the j-th variable at time j; Wq∈Rd×w is the weight coefficient matrix; and bq is the bias.

The output matrix Vα partitions the input time series in the variable dimension, forming a feature representation centered on the variable, allowing the model to capture the dynamic correlation between different variables. Therefore, by designing the parallel embedding layer, the model can simultaneously pay attention to the temporal dependency and dynamic correlation between variables in the time-series data.

To effectively integrate the outputs of the dual-tower encoder, this paper designs an attention-gate control mechanism. The attention gate control mechanism is composed of an attention module and a GLU module, which realizes the nonlinear integration of the output matrices Xoutput1∈Rw×d and Xoutput2∈Rm×d of the dual-tower encoder. This operation can adaptively adjust the attention distribution of the output feature information, and focus on the key feature information of the dual-tower encoder outputs, thereby achieving precise gating and flow control. Its operation flow is as follows: (1)

Feature Linear Transformation: Linear transformation is applied to the feature matrices Xoutput1 and Xoutput2, mapping them to the query, key, and value matrices Q, K, and V, the mapping formula is as shown in Eq. (3): (3){Q=(Xoutput1;Xoutput2)WQK=(Xoutput1;Xoutput2)WKV=(Xoutput1;Xoutput2)WV,where WQ, WK, WV∈Rd×d represent the weight matrices for the query, key, and value, respectively.

This paper employs busbar voltage magnitude data of individual nodes in the power system as the original dataset. The voltage curves under different system states are shown in Fig. 3a,b. As evident from these figures, when the system is subjected to a certain disturbance, the busbar voltage in a stable state can recover to its original stable state or attain a new stable state over time. However, in an unstable state, the busbar voltage fluctuates continuously with varying vibration trends. Combining the changes in rotor angle curves can demonstrate the differences between the two instability modes, as shown in Fig. 3c.

Considering the potential data loss during PMU data collection, this paper constructs a 30-dimensional trajectory cluster dataset based on the original dataset to prevent the data-driven model from becoming inoperable due to the loss of busbar data [24]. The operating state of the power system is divided into three cases: the stable state is labeled 0, the aperiodic unstable state is labeled 1, and the oscillatory unstable state is labeled 2. The stability of the power system is judged by the transient stability index (TSI) of the generator rotor angle after the fault, and the aperiodic instability and oscillatory instability are further distinguished based on the divergence of the power angle curve. TSI calculation formula is as shown in Eq. (6): (6)TSI=360∘−|Δδmax|360∘+|Δδmax|where Δδmax represents the maximum relative power angle difference between any two generators. When TSI>0, the sample is classified as a stable state. When TSI<0, the sample is classified as an unstable state.

This paper designs a TSA model composed of the dual-tower Transformer encoder, fully connected layer, and Softmax classifier, as shown in Fig. 4. The model assessment process is as follows:

(1)

A 30-dimensional trajectory cluster dataset is obtained from the power system, as shown in the data processing section of Fig. 4, where Xi(w,m) represents the m-th feature of the i-th sample at the w-th time point. Then, each X is normalized to obtain the normalized data X^.

In order to effectively reduce the probability of misclassification of unstable samples, this paper has developed a multi-stage TSA method, which includes pre-training, re-training, and online assessment stages.

Self-supervised learning [26], as an important branch of unsupervised learning [27], possesses the ability to autonomously mine and extract the intrinsic representation characteristics of unlabeled data. In this paper, we introduce the masked prediction task into self-supervised learning as an auxiliary strategy, utilizing unlabeled samples to conduct pseudo-supervised training of the model. The specific operation of self-supervised learning based on masked prediction is as follows: (1)

Data masking: For each unlabeled input sample X of the model, create an independent binary noise mask Noise∈Rw×m. The elements of Noise are 0 and 1, with 0 segments being the masked segments, which account for a proportion of r, and the distribution follows a geometric distribution with a mathematical expectation of E. Multiply the noise mask with the input data element-wise, and use the masked segments to obscure the input data, generating the masked data X~=[x~1,x~2,⋯,x~i,⋯,x~w], where x~i is the i-th vectors.

The initial update in Stage 1 alone cannot enhance the model’s assessment performance to meet the predetermined standard. However, after the initial update in Stage 1, the model has learned rich data representations of the new system from unlabeled samples. So, when optimizing the pre-trained model in Stage 1, no more labeled samples are needed, greatly reducing the time required for sample generation. The update steps are as follows: (1)

Key Sample Selection Mechanism II: Using the pre-trained model from the first stage, perform masked predictions on unlabeled samples, and define the top Ω samples with the highest predicted loss value as key unlabeled samples.

In this simulation, the software uses PSD-BPA software, the assembly language is Python, and the deep learning framework is PyTorch. The case study uses the IEEE New England 10-machine 39-bus system and IEEE 47-machine 140-bus system for simulation experiments. The database setting is shown in Table 1, where S1 and S2 are the source domain databases, and A-F are the target domain databases. The case study considers 9 load levels at 80%, 85%, 90%, … , 120%, and the fault location is selected at 0%, 10%, 20%, … , 90% of the line, with the fault type set as a three-phase short-circuit fault. The fault duration is randomly set within 0.1 to 0.2 s, and the simulation time is 20 s. The S1 database has 12,240 samples, including 7986 stable samples, 3042 aperiodic unstable samples, and 1212 oscillatory unstable samples. The S2 database has 22,140 samples, comprising 15,201 stable samples, 3853 aperiodic unstable samples, and 3086 oscillatory unstable samples. The datasets are divided into training, validation, and test sets in an 8:1:1 ratio.

To evaluate the model performance comprehensively, we introduced a confusion matrix, as shown in Table 2, and three classification assessment indicators, as shown in Eqs. (8)–(10): (8)PACC=T00+T11+T22T0+T1+T2×100%, (9)F1i=2×TiiT0i+T1i+T2i×TiiTi0+Ti1+Ti2TiiT0i+T1i+T2i+TiiTi0+Ti1+Ti2, (10)Gmean=T00T0×T11T1×T22T23,where PACC represents the overall assessment accuracy rate, F1i is the harmonic mean of the precision and recall for samples labeled as i, and Gmean represents the comprehensive performance index, which is an important indicator reflecting the model’s discriminative performance on minority sample.

To assess the dual-tower Transformer’s performance, we compared it with six other models: dual-tower FNN-Transformer, standard Transformer, CNN+LSTM, LSTM, CNN, and SVM. The dual-tower FNN-Transformer integrates an FNN module for nonlinear transformation, while the CNN+LSTM model separately extracts and analyzes spatial and temporal features. The model parameters are as follows: the dual-tower Transformer and standard Transformer models have 8-head attention mechanisms and 4 sub-modules. The LSTM algorithm uses 4 hidden layers with a hidden layer dimension of 150. The CNN model comprises 4 convolutional layers with a kernel size of 3 × 3, and each layer is followed by a 2 × 2 max pooling layer. The SVM kernel function uses the radial basis function, with hyperparameters C = 10 and kernel function parameters. All models are set with an Epoch size of 200, a Batch size of 128, an initial learning rate of Lr = 0.001, and an L2 regularization term of 0.1. The average of 20 repeated experiments is taken as the evaluation result, as shown in Table 3. Simultaneously, to test the working efficiency of the dual-tower Transformer model, the training durations of both the dual-tower Transformer model and the dual-tower FNN-Transformer model were recorded, with the results shown in Table 4.

Table 3 demonstrates that the Transformer model outperforms LSTM, CNN, and SVM in assessment tasks, due to its multi-head attention mechanism that targets key features. The dual-tower Transformer, which extracts data from both time and variable dimensions, exhibits superior performance. Although the CNN+LSTM model, which captures features from both dimensions, performs better than each model, it still lags behind the dual-tower Transformer. Retraining the dual-tower Transformer using a mixed sampling set significantly enhanced indicators Gmean, validating the strategy’s effectiveness. According to Table 4, during the training process of the two systems, the dual-tower Transformer model significantly outperformed the dual-tower FNN-Transformer model in terms of training speed. This validates that the Transformer model, which utilizes GLU instead of FNN for nonlinear transformation, possesses higher training efficiency while maintaining superior evaluation performance.

To verify the effectiveness and explainability of the designed dual-tower Transformer in allocating attention to feature information, this experiment randomly selected 50 samples for testing. Heatmaps in Figs. 6 and 7 visualize the evolution process of attention weight allocation in Encoder I and Encoder II, respectively, with color depth indicating weight (darker means higher).

Figs. 6 and 7 illustrate the changes in attention weights of the tower encoder in time and variable dimensions over training epochs. In Fig. 6, as the number of training rounds increases, the attention mechanism of Encoder I increasingly focuses on features from later periods, and this conforms to the system’s state gradually becoming clearer as time passes after the fault occurs. In Fig. 7, the attention weights of Encoder II are evenly distributed among features initially but develop a clear bias by the 100th round, showing the model learns to emphasize key features. The figures demonstrate that the feature extraction network based on the model can dynamically adjust the feature weights and verify its interpretability.

The source domain model is tested on different target domain datasets, respectively, and the average of 20 repeated experiments is taken as the evaluation result. The results are shown in Table 5.

Table 5 shows that the evaluation performance of the source domain model in each target domain has dropped sharply, with accuracy rates below 80%, which fails to meet the accuracy requirements for transient stability assessment. Therefore, the source domain model needs to be updated to adapt to the new system structure. Since the I_MMD values between the six target domains and the source domain dataset are all greater than 0.5, this paper utilizes the two-stage updating scheme to update the source domain model.

In the first stage update, the mask ratio is set to 0.15, with a geometric distribution whose mathematical expectation is 3. In the second stage, 1500 critical samples are used, the initial learning rate is set to 0.001, and 100 epochs are run, with other settings remaining the same as in Section 5.2. The assessment result is the average of 20 repeated experiments, as shown in Table 6.

Table 6 shows that the model’s assessment performance significantly improves after two-stage updating, achieving over 97% accuracy even in cases of two bus outages, verifying the scheme’s effectiveness.

To verify the performance of the key sample selection mechanism based on the first-stage transfer model, 1500 key samples selected by the selection mechanism were compared with 1500 random samples through t-random nearest neighbor embedding dimensionality reduction visualization, the distribution of samples as shown in Fig. 8.

From Fig. 8, it can be seen that the overlap among the key samples is more significant than that among the random samples, and the proportion of minority class samples increases. This is because the samples in the overlapping area and the minority class samples are more difficult to distinguish. Therefore, these key samples have higher value information, verifying the effectiveness of the key sample selection mechanism.

To further verify the effectiveness of the two-stage model updating schemes, this paper designs four additional updating schemes for comparison testing:

Option 1: Retrain the model using all labeled samples.

Option 2: Based on the source domain model, randomly select 1500 samples for labeling, and fine-tune the output layer parameters of the model.

Option 3: Use self-supervised learning to initially improve the model’s generalization performance. Then, randomly select 1500 unlabeled samples and label them using long-term simulation techniques. Finally, fine-tune the model using the labeled samples.

Option 4: Utilize active learning to screen high-value samples, and subsequently subject the screened 1500 unlabeled samples to long-term simulation to generate labels. These labeled samples are then used to further optimize the model.

Option 5: The two-stage update scheme proposed in this paper.

To test the update effectiveness of each scheme, experiments were conducted on different target domains for the five update schemes, with two performance indicators, PACC and Gmean, selected for comparative analysis, as shown in Fig. 9. To further test the update efficiency of the two-stage update scheme, experiments were carried out on target domain F of the IEEE 47-machine 140-bus system using Option 1, Option 4, and Option 5. The comparison benchmark was set at an accuracy rate of 95%, and the update time of the models in the three schemes was compared, as shown in Table 7.

From Fig. 9, although Option 1 can improve the evaluation performance of the model through a large number of training samples and demonstrates similar performance to the proposed update scheme in this paper on the target domain F, according to Table 7, the retraining time required for Option 1 is 5347.38 s, far exceeding that of Option 5, and cannot meet the timely demand for model updates. The performance of Option 2 is inferior to that of Option 5, because when I_MMD > 0.5, the distribution difference between the target domain data and the source domain data is significant, and merely fine-tuning the output layer cannot fully capture the sample characteristics under the new system. Although Option 3 utilizes self-supervised learning in the first stage to enhance the model’s generalization performance, the randomly selected labeled samples are not representative, leading to lower update performance across various target domains compared to Option 5. Option 4 employs active learning to update the model, but the performance after the update is significantly lower than that achieved using Option 5.

To verify the impact of sample size on the transfer results, we selected 500 to 3100 key samples from the target domain, with 200 samples as the interval, according to the five updating schemes, and tested them. The results are shown in Fig. 10.

From Fig. 10, with just 900 labeled samples, the updated model’s accuracy in option 5 is notably higher, outperforming other schemes. This is due to the proposed scheme enhancing model generalization through self-supervised learning in stage 1, and a sample selection mechanism identifying valuable samples. Thus, the model significantly improves with limited labeled samples, validating the two-stage updating scheme’s effectiveness.