Since the identification of DNNs’ vulnerability to adversarial attacks, several effective attack algorithms have been proposed [13,39–41]. These methods can be categorized as white-box attacks and black-box attacks based on the adversary’s knowledge. White-box attacks such as the fast gradient sign method (FGSM) [39], projection gradient descent method (PGD) [13], and Carlini Wagner Attack (CW) [40], have full access to all the parameter information of the attacked model. To comprehensively evaluate the effectiveness of the proposed defense method, we employ PGD [13], FGSM [39], CW [40], and AutoAttack (AA) [41].

To mitigate the threat of adversarial attacks, various defense methods have been proposed [5,10,14]. AT [10–13], which adds adversarial perturbations to the inputs during training, has proven to be one of the most effective approaches for enhancing the DNNs’ adversarial robustness. Madry et al. [13] formulated standard AT (SAT) as a minimax optimization problem, where the inner maximization represents the attack strategy guiding AE generation. Solving the inner maximization problem in SAT is achieved using the PGD attack.

Several studies have proposed methods to improve the performance of SAT. Zhang et al. [10] introduced TRadeoff-inspired Adversarial DEfense via Surrogate-loss minimization (TRADES) to balance adversarial robustness and clean accuracy. Wang et al. [42] further improved performance by Misclassification-Aware Adversarial Training (MART). While these methods employed fixed attack strategies, other studies [12,36–38] demonstrated that employing different attack strategies at different training phases can further improve AT. Cai et al. [36] introduced curriculum adversarial training (CAT), which employs AEs generated by attacks with varying strengths in a curriculum manner. Zhang et al. [37] proposed friendly adversarial training (FAT) that trains a DNN using both wrongly-predicted and correctly-predicted AEs. Wang et al. [38] introduced First-Order Stationary Condition for constrained optimization (FOSC) as a quantitative indicator for assessing AE convergence quality. However, these methods rely on manually designed metrics to evaluate the AE difficulty and still use a single strategy at each stage, thus limiting the robustness improvement and requiring domain expertise. Jia et al. [12] proposed a learnable attack strategy that allows the strategy model to automatically produce sample-dependent attack strategies using a gaming mechanism. However, when directly applied to AD, this method generated the AEs that are independent of the teacher model and not applicable for distillation, thus limiting the closer match between teacher and student models.

To address this limitation and generate sample-dependent attack strategies advantageous to distillation, reference [12] is improved and introduced into the AD framework. This improvement considers the differences in output between the target and teacher models, resulting in a closer match between them.

Recently, there has been a growing body of research highlighting the potential for improving AT through the integration of KD. KD offers data-driven soft labels to smooth the hard labels. In Adversarially Robust Distillation (ARD) [19], the target model was encouraged to mimic the softmax output of an adversarially pre-trained teacher model on clean input when facing an adversary. In Robust Soft Label Adversarial Distillation (RSLAD) [21], the generation of AEs and the training of target models were guided by the Robust Soft Labels (RSLs) derived from adversarially pre-trained teachers. Adversarial Knowledge Distillation (AKD) [20] leveraged a linear combination of the AEs’ predictions from the teacher model and the original labels, effectively guiding the student model’s predictions on AE.

However, these methods only utilize the knowledge of adversarially pre-trained teachers to enhance the adversarial robustness of the target model, overlooking considerations related to clean accuracy. Chen et al. [23] imposed the adversarial predictions of the target model to mimic those of standard teachers and robust teachers, hereinafter referred to as self-teacher training (STS). This method notably improves accuracy on both clean samples and AEs, yet it heavily relies on trust in teacher models. IAD [25] highlighted the diminishing reliability of teacher guidance, advocating for a gradual development of confidence in the student model’s adversarial robustness while partially trusting the teacher model. The methods mentioned earlier predominantly focus on distilling logit knowledge from the teacher model. Vanilla Feature Distillation Adversarial Training (VFD-Adv) [22] distilled feature knowledge from the teacher’s intermediate layer, aligning features of clean examples from the teacher model with those from the student model in the feature space. We utilize logit distillation since it requires less computational and storage costs and logits are at a higher semantic level than deep features.

The baseline in our paper is STS presented in [23], and we use the same AD framework. The adversarial robustness and clean accuracy of the target model are simultaneously improved by leveraging the standard and robust teachers to provide clean and robust knowledge, respectively. Recognizing the decreasing reliability of teacher knowledge during training [24], we update the parameters of teacher models by incorporating supervision from the training set and AEs, as well as feedback from the target model’s performances on the validation set and AEs.

Current AD techniques employ the conventional two-stage offline KD technique, where the teacher model’s parameters remain unchanged during the distillation process. However, this technique cannot guarantee a match between the teacher and student models, especially when there is a significant difference in predictive performance between them. Additionally, two-stage offline KD cannot adjust the knowledge transfer process in real time based on the learning status of the student model. To address these issues, L2T distillation has been proposed [26], which involves training the student model and fine-tuning the teacher model simultaneously, allowing the teacher model to adjust its behavior based on the feedback from the student model.

Online distillation [27–30] is a commonly used L2T algorithm, which involves simultaneously training the student and teacher models and ensuring similarity between their outputs on the training set by minimizing the Kullback-Leibler (KL) divergence between them. However, this only considers the knowledge transfer on the training set without considering the validation performance of the student model. Meta distillation [31–34] addresses this issue by fine-tuning the teacher model to minimize the loss of the updated student on the validation set. However, the teacher model only receives supervision from the student model, which can result in performance degradation.

Recently, Ren et al. [35] proposed a novel L2T distillation framework called Learning Good Teacher Matters (LGTM), which introduced the distillation influence to assign a loss weight to each training sample based on the student model’s performance on the validation set. However, this method does not consider the accuracy of the target model on AEs as feedback to fine-tune the robust teacher.

To improve the reliability and effectiveness of the standard and robust teachers’ knowledge in the generalization ability of the target model on both clean samples and AEs, LGTM [35] is extended and incorporated into the AD framework. We use feedback from the target model on the validation data and AEs to update both standard and robust teachers. Fine-tuning the teachers narrows the capacity gap between the teacher and target models and makes teacher models more adaptable to the stronger AEs, increasing their reliability. Additionally, due to the involvement of teacher knowledge in the AE generation in our method, more reliable teachers can also improve the quality of AEs.

Existing AD techniques employ fixed and sample-agnostic attack strategies that are centered around the target model, which leads to AEs being irrelevant to the teacher models and unsuitable for AD. Besides, static teachers face challenges in accurately predicting stronger AEs generated by the increased robustness of the target model. Distilling unreliable knowledge can hurt the performance of the target model. To enhance the suitability of AEs for distillation and improve the reliability and effectiveness of teachers’ knowledge in promoting the generalization performance of the target model, LDAS&ET-AD is proposed to generate AEs by leveraging a learnable distillation attack strategies generating mechanism that considers prediction differences between the teacher and target models, as well as update teachers by using a teacher evolution strategy that takes into account the performance of the target model on validation set and AEs. The proposed AD framework, depicted in Fig. 1, comprises a target model, a strategy model, and standard and adversarially pre-trained teacher models.

The training process consists of two stages: Generating AEs and fine-tuning teachers.

In the stage of generating AEs, the target model and the strategy model compete with each other in minimizing or maximizing the same objective function. The strategy model is trained to automatically generate attack strategies that produce AEs capable of misleading the target model and inducing maximum divergence between the target and teacher models. The target model is trained to defend against AEs generated by the attack strategies while receiving guidance from both standard and adversarially pre-trained teachers to minimize the prediction distance with them.

In the subsequent section, we provide a detailed description of the learnable distillation attack strategies generating mechanism that considers prediction differences, as well as the teacher evolution strategy that takes into account the validation performance of the target model. The equation symbols and abbreviations used throughout this paper are summarized in Tables 1 and 2, respectively.

An attack strategy is determined by the values chosen for attack parameters, such as the maximal perturbation strength ε, attack step size α, and attack iteration I. These parameters play a crucial role in the inner optimization problem of AT, significantly impacting performance. Given a clean image xcleani and its ground truth (GT) label ycleani, the generation of AE xadvi using an attack strategy a at the m^th training step can be defined as follows: (1)xadvi=xcleani+δ=g(xcleani,a,θtarm)

where δ represents the adversarial perturbation, θtarm is the parameters of the target model at the m^th training step, g(⋅) denotes the PGD attack employed in our method following [13]. Concretely, PGD recursively searches: (2)xadvi(n+1)=∏ℬε[xadvi(0)]xadvi(n+1)+α⋅sign(∇xadvi(n)L(fθtarm(xadvi(n)),ycleani))

until a stopping criterion is met. xadvi(n) are the AEs at step n, and ∏ℬε[xadvi(0)](⋅) is the projection function that projects the AEs back into the ε-ball centered at xadvi(0), L(fθtarm(xadvi(n)),ycleani) is the cross-entropy (CE) loss between fθtarm(xadvi(n)) and ycleani. For simplicity, we annotate ∂f/∂v as ∇v.

Current AD techniques still rely on fixed sample-agnostic and student-egocentric attack strategies, where the attack parameters are artificially set and remain unchanged during training. The loss function for current AD at the m^th training step can be expressed as: (3)minθtarm⁡E(xcleanr,ycleanr)∼Dtrain[maxδ=(xadvr−xcleanr)∈Ω𝒫⁡(1−λstd−λadv)⋅L(fθtarm(xadvr),ycleanr)+λstd⋅KL(fθtarm(xadvr),fθstd−T(xadvr))+λadv⋅KL(fθtarm(xadvr),fθadv−T(xadvr))]

where fθtarm(⋅) denotes the target model at the m^th training step, fθstd−T(⋅) and fθadv−T(⋅) represent the static standard and adversarially pre-trained teachers, respectively. Dtrain denotes the distribution of the clean training examples xcleanr and their GT labels ycleanr. Ω𝒫 represents a bound defined as Ω𝒫={δ:||δ||𝒫≤ε}. xadvr are the AEs of xcleanr. L(fθtarm(xadvr),ycleanr) represents the CE loss of the target model between fθtarm(xadvr) and ycleanr. KL(fθtarm(xadvr),fθstd−T(xadvr)) and KL(fθtarm(xadvr),fθadv−T(xadvr)) are the KL divergence between fθtarm(xadvr) and fθstd−T(xadvr) and between fθtarm(xadvr) and fθadv−T(xadvr). λstd and λadv are hyperparameters that control the guidance ratio of the standard and robust teachers, respectively. The target model is trained to minimize both the AT loss and the prediction distance with standard and adversarial pre-trained teachers on AEs. The process of AE generation in existing AD methods is illustrated in Fig. 2a, which results in AEs unsuitable for distillation, limiting the closeness between the teacher and target models.

As the robustness of the target model increases and AEs become stronger, the reliability of static teachers’ knowledge diminishes. This unreliable guidance not only negatively impacts the performance of the target model, but also affects the quality of AEs that rely on the knowledge of teacher models. To enhance the reliability and effectiveness of teachers’ knowledge in promoting the generalization performance of the target model, a teacher evolution strategy is introduced in our AD framework. This strategy takes into consideration the validation performance of the target model. The feedback for fine-tuning teachers is determined by the adversarial distillation influence, which extends the distillation influence proposed in [35].

In accordance with previous studies [24], we reported the test accuracy at both the best checkpoint and the last checkpoint. The best checkpoint of ST is chosen based on its performance on clean test examples, while the best checkpoints of SAT [13], TRADES [10], LAS-AT [12], STS [23], IAD [25], and our LDAS&ET-AD are selected based on their robustness against the PGD attack.

We conducted a set of ablation studies to better grasp the impact of each component in our LDAS&ET-AD.

Firstly, the learnable distillation attack strategies generating mechanism in our LDAS&ET-AD was replaced with the fixed distillation attack strategy in STS [23] considering prediction differences between student and teacher, denoted as Fixed Distillation Attack Strategy and Evolvable Teachers Adversarial Distillation (FDAS&ET-AD), to verify the effectiveness of the introduction of learnable attack strategies. Besides, this mechanism was replaced with the learnable attack strategies in LAS-AT [12], denoted as Learnable Attack Strategy and Evolvable Teachers Adversarial Distillation (LAS&ET-AD), to demonstrate the importance of the consideration of the prediction differences.

Secondly, we fine-tuned the model parameters of one, denoted as Learnable Distillation Attack Strategies and Evolvable Robust Teachers Adversarial Distillation (LDAS&ERoT-AD) and Learnable Distillation Attack Strategies and Evolvable Standard Teachers Adversarial Distillation (LDAS&EStT-AD), or none, denoted as Learnable Distillation Attack Strategies Adversarial Distillation (LDAS-AD), of the two pre-trained teachers in our LDAS&ET-AD. The purpose was to illustrate the different effects of each teacher’s update on performance improvement. Subsequently, the test clean and adversarial accuracy of the trained target models were evaluated. The results of the ablation studies are presented in Table 8.

As shown in Table 8, our LDAS&ET-AD outperforms all five variants against all four attacks. Firstly, our LDAS&ET-AD automatically generates sample-dependent and increasingly stronger attack strategies, enabling the creation of AEs that can adapt to more robust target models. Consequently, our LDAS&ET-AD outperforms FDAS&ET-AD, resulting in improvements of 1.81%, 0.67%, 2.27%, 0.96%, and 2.18% in clean, FGSM, PGD-20, CW_∞, and AA accuracy, respectively. Furthermore, by incorporating prediction differences into the learnable attack strategies, AEs are not only able to mislead the target model but also maximize the prediction discrepancy between the target and teacher models, achieving a closer match between them. Therefore, our LDAS&ET-AD outperforms LAS&ET-AD in terms of clean, FGSM, PGD-20, CW_∞, and AA accuracy by 1.01%, 0.29%, 1.80%, 0.06%, and 1.11%, respectively. These findings highlight the superiority of introducing the learnable attack strategies and prediction differences into the AD framework due to the generation of AEs that are more suitable for AD and a closer match between the teacher and target models.

Secondly, fine-tuning only the adversarially pre-trained teacher in LDAS&ERoT-AD ensures the reliability and effectiveness of adversarial knowledge which aims to guide the target model in accurately classifying AEs. Therefore, LDAS&ERoT-AD outperforms LDAS-AD solely in terms of adversarial robustness. LDAS&EStT-AD, on the other hand, only updates the standard pre-trained teacher to enhance the quality of clean knowledge, which is designed to specifically enhance the clean accuracy of the target model. LDAS&EStT-AD achieves higher accuracy on clean samples compared to LDAS-AD. Our LDAS&ET-AD, which fine-tunes both teacher models, shows improved clean, FGSM, PGD-20, CW_∞, and AA accuracy by 0.71%, 0.21%, 0.39%, 0.31%, and 0.86%, respectively, compared to the methods that either do not update or only update one teacher. The experimental results indicate that fine-tuning both robust and standard teachers has positive effects on improving both clean accuracy and adversarial robustness of the target model, highlighting the potential of evolvable standard and robust teachers.

To verify the superiority of the learnable distillation attack strategies generating mechanism in our LDAS&ET-AD over other dynamic hand-crafted attack strategies generating methods, we replaced it with CAT [36], FOCS [38], and FAT [37] and considered prediction differences, denoted as Curriculum Distillation Attack Strategy and Evolvable Teachers Adversarial Distillation (CDAS&ET-AD), First-Order Stationary Condition Distillation Attack Strategy and Evolvable Teachers Adversarial Distillation (FOCSDAS&ET-AD), and Friendly Distillation Attack Strategy and Evolvable Teachers Adversarial Distillation (FriDAS&ET-AD), respectively. The results are shown in Table 9.

The obtained results demonstrate that the learnable distillation attack strategies generating mechanism in our LDAS&ET-AD outperforms all three variants. This improvement can be attributed to the AEs being more suitable for AD of the increasingly robust target model. Specifically, compared to the best variant, our LDAS&ET-AD achieves higher accuracy in clean, FGSM, PGD-20, CW_∞, and AA attacks by 0.96%, 0.32%, 1.04%, 0.23%, and 1.14%, respectively. These findings emphasize the advantages of introducing learnable attack strategies in the proposed LDAS&ET-AD method for generating AEs suitable for AD when compared to other dynamic hand-crafted attack strategy methods.

To assess the superiority of the teacher fine-tuning strategy in our LDAS&ET-AD over other teacher fine-tuning methods based on L2T distillation, we replace it with (1) meta distillation [31], which considers feedback from the target model on the validation set while all training samples equally and solely receiving supervision from the target model, referred to as Learnable Distillation Attack Strategy and Mate Adversarial Distillation (LDAS&meta-AD) and (2) online distillation [27], which enforces similarity between the outputs of the target and teacher models on the training set without considering the target model’s performance on the validation set, denoted as Learnable Distillation Attack Strategy and Online Adversarial Distillation (LDAS&OL-AD). The results are presented in Table 10.

Table 10 demonstrates that our LDAS&ET-AD outperforms all two variants, achieving the highest test accuracy on both clean samples and AEs. Our LDAS&ET-AD uses the target model’s performance on the verification set as feedback to assign the loss weight of each training sample for fine-tuning of teacher models, enhancing the effectiveness of the teachers’ knowledge in the generalization ability of the target model on both clean samples and AEs. This improvement is achieved by introducing adversarial distillation influence. Additionally, the training of teacher models is also supervised by the AEs of the training set, improving the reliability of their knowledge. Our LDAS&ET-AD demonstrates significant improvements compared to the best variant, achieving enhancements of 0.96%, 0.32%, 1.04%, 0.23%, and 1.14% on clean, FGSM, PGD-20, CW_∞, and AA accuracy, respectively. These results validate the effectiveness of the teacher fine-tuning teacher strategy in our proposed LDAS&ET-AD, surpassing other teacher fine-tuning methods.

The hyperparameter k controls the alternating update of θtar and θstra. Every k times θtar are updated, θstra are updated once. It affects not only performance but also training efficiency. Firstly, the efficiency of the proposed method decreases with the increase of k. Smaller k results in more frequent updates of θstra, thus requiring more training time. Secondly, selecting an appropriate k is crucial for the adversarial robustness of the target model. If k is too small, the target model’s discrimination ability towards attack strategies generated by the strategy model may be impaired. This, in turn, affects the diversity of attack strategies and the update stability of the teacher models. On the other hand, if k is excessively large, the generation ability of the strategy model may be compromised, resulting in insufficiently effective AEs for updating the teacher and target models. To determine the optimal k, we conducted experiments on hyperparameter selection. The performance results are depicted in Fig. 4, and the efficiency results are listed in Table 11.

Fig. 4 shows how the selection of k impacts the clean and PGD-20 accuracy of the target model and there is a trade-off between these two metrics. Specifically, when k is too small, the target model exhibits poor discrimination ability against attack strategies generated by the frequently updated strategy model. Consequently, the diversity of attack strategies diminishes, leading to low-quality AEs, lower adversarial robustness, and higher clean accuracy of the target model. Conversely, when k is too large, the attack strategies generated by the strategy model with low generation ability become approximately fixed, preventing the target model from achieving optimal adversarial robustness. The results in Fig. 4 indicate that the proposed LDAS&ET-AD achieves the best adversarial robustness when k is set to 30.

Table 11 demonstrates that the training time of the proposed LDAS&ET-AD decreases with the increase of k. As k increases, the update frequency of the strategy model decreases, resulting in a decrease in the overall training time.

Considering both efficiency and adversarial robustness, we set k to 30.

The hyperparameters αstd and αadv play vital roles in controlling the self-evolution of standard and robust teachers and impact the guidance quality for classifying clean samples and AEs. Specifically, an excessive focus on self-evolution may lead to neglecting the feedback provided by the target model, resulting in guidance not meeting the target model’s needs. Besides, a lack of focus on self-evolution may hinder teachers from enhancing their abilities, reducing the reliability of teacher knowledge. The controlled experiments were conducted to analyze the impact of self-evolution on the target model’s performance.

We fix αstd at 0.6 and vary αadv from {1.0, 0.9, 0.8, 0.7, 0.6, 0.5} to evaluate the adversarial robustness of ResNet-18 on CIFAR-10 against PGD-20 attack since αadv controls fine-tuning of the robust teacher and mainly affects the robustness of the target model. Fig. 5a demonstrates that when αadv is too large, the feedback from the target model has little influence on the robust teacher’s update. Consequently, the fine-tuned robust teacher does not significantly improve the generalization performance of the target model on AEs. Conversely, when αadv is too small, neglect of self-evolution causes performance degradation. The guidance from low-performance teachers impairs the robustness of the target model. Both situations result in suboptimal adversarial robustness. The results indicate that LDAS&ET-AD achieves the best robustness when αadv is set to 0.7.

Next, we fix αadv at 0.7 and vary αstd from {1.0, 0.9, 0.8, 0.7, 0.6, 0.5} to evaluate the clean accuracy of ResNet-18 on CIFAR-10 since αstd controls fine-tuning of the standard teacher and mainly affects the clean accuracy of the target model. Consistent with the analysis of αadv, both excessively large and small αadv hamper the fine-tuning of the standard teacher, preventing the target model from achieving optimal generalization on clean samples. Fig. 5b illustrates that LDAS&ET-AD performs well when αstd is set to 0.8.

In conclusion, we set αstd to 0.8 and αadv to 0.7 to strike a balance between self-evolution and target model feedback, ensuring the best performance of LDAS&ET-AD.

The hyperparameters αL2 and αL3 balance the trade-off between evaluating robustness loss term and predicting clean samples loss term in attack strategies generating mechanism. When αL2 is relatively large compared to αL3, it may result in lower clean accuracy. Conversely, an excessively large αL3 can lead to insufficient attention to robustness, resulting in low adversarial robustness. We present the performance of our proposed LDAS&ET-AD with various αL2 and αL3 pairs on CIFAR-10 using the ResNet-18 target model in Table 12.

Firstly, we fix αL2 at 2 and vary αL3 from {2, 4, 6} to evaluate performance. As αL3 increases, the total loss function places more emphasis on the robustness evaluation loss term. The results in Table 12 demonstrate that the clean accuracy shows a downward trend. Although the best robustness is achieved when αL3 is 4, there is only a small improvement compared to when αL3 is 2. The robustness continues to increase, and the optimal value is still achieved when αL3 is 4, but the difference is minimal compared to when αL3 is 2.

Secondly, we fix αL3 at 4 and vary αL2 from {2, 4, 6}. As αL2 increases, the total loss function focuses more on the clean accuracy loss term. It can be observed from the results in Table 12 that increasing αL2 leads to robustness decreasing and clean accuracy with little change. This indicates that when αL3 is fixed at 4, the clean accuracy is not sensitive to the change of αL2.

Although the performance of the target model is affected by αL2 and αL3, the changes do not occur within a large range. Therefore, the proposed method is not highly sensitive to these two hyperparameters, which aligns with the observation in [12]. Overall, we set αL2 to 2 and αL3 to 4.

The proposed method entails a higher training complexity than the baseline, primarily due to the training of the strategy model parameters and the fine-tuning of the teacher models. However, our LDAS&ET-AD offers pronounced improvements over state-of-the-art adversarial defense methods. Specifically, the sample-dependent attack strategies generated by the strategy model in the game with the target model are highly effective, as are the more reliable teacher models fine-tuned according to the validation performance of the target model. In contrast, fixed hand-crafted attack strategies and static teacher models are far less effective.

Besides, to ensure a suitable trade-off between efficiency and robustness during the training of the strategy model, we have considered various factors, including the frequency of updating parameters. We have also introduced a finite difference approximation [35] to address the slowness of computing per-sample gradients and improve computational efficiency. Importantly, no additional complexity is introduced in the inference stage. However, we acknowledge that further work is necessary to reduce the training complexity of our approach.