This section covers studies that have utilized CNNs to classify retinal images with different objectives. Numerous studies in existing literature have concentrated on enhancing robustness. For example, the work done in [15], the authors built a robust binary classification model to detect diabetic retinopathy (DR). The Indian IDRiD and OCT datasets were used. The study applied hold-out validation (15,400 training, 3720 validations, and 400 testing sets). The ResNet-50 model with pre-trained weights (ImageNet) was fine-tuned to the proposed datasets. Adversarial training and distillation were performed to address the FGSM and L-BFGS attacks. The distillation technique outperformed adversarial training, reducing the success rates of both attacks (FGSM and L-BFGS-B by 60% and 82%, respectively).

In contrast, the authors of [16] proposed a light, robust CNN model to address the sustainability and security of retinal OCT classification. The study used Kermany OCT, COVID, and Kidney Stone CT datasets. An under-sampling technique was used to balance the data. Hold-out validation was applied (80% training and 20% for testing). Cross-validation with 10 folds was applied to the training set. The study designed a custom lightweight CNN model. No defense mechanisms were applied. The model was evaluated using four adversarial FGSM examples. Achieving 94% clean accuracy on the Kermany dataset.

Furthermore, some retinal OCT studies have utilized CNN with a focus on accuracy enhancements. For example, in [17], the study aimed to develop a system for detecting retinal diseases using OCT images. This study proposes two models for feature extraction (ResNet-50 and AlexNet) and uses the Kermany dataset. The proposed CNN network is used in parallel to extract the features. Two feature selection techniques were applied in parallel (PCA and Entropy-based Ant Colony System). The selected features were combined into an optimized vector and then fed into Machine Learning models, such as K-Nearest Neighbors(kNN), Linear Support Vector Machine (LSVM), Linear Discriminant (LD), and Decision trees. The LD classifier achieved high accuracy (99.9%).

Similarly, the authors in [18] addressed the challenge of multi-class accuracy (retinal disease classification) under a limited and imbalanced dataset, proposing a modified VGG16-based CNN with data augmentation, class weighting, and explainability techniques (Grad-CAM) using a small Mendeley fundus dataset of 302 images. The model achieved 93.42% training accuracy and 77.5% validation accuracy, with improved interpretability and class balance.

All these studies were limited to CNNs and did not address the improvement in the robustness and efficiency of ViT models.

This section covers studies in the existing literature that utilize Vision Transformers as the main model. Most studies that employed Transformers in Retinal OCT classification focused on efficiency. For example, the authors in [10] designed a light approach for Diabetic Retinopathy Classification. This study proposed resizing the images to 896 × 896 pixels and splitting them into 16 patches (224 × 224 pixels). This study proposes a ViT-small model as a feature extractor. The Classification tokens (CLS) were combined into a matrix (16 × 384) and fed into the Global Instance Computing Block (GICB) layer to compute attention scores, then fed to a Multi-Layer Perceptron (MLP) layer with a residual. The proposed approach was applied to APTOS 2019 and Messidor-1 datasets. The inference time is reduced by 62%, and the accuracy is improved by 2.1% for the APTOS and 12.1% for the Messidor dataset.

Extending the focus to efficiency, the authors in [19] evaluated a list of ViT models (ViT, T2T ViT, and Mobile-ViT) for retinal OCT classification. This study proposed mobile-ViT as the main approach. The public OCT and Mendeley datasets were used. Hold-out validation was applied to propose 10% as the validation dataset. Mobile-ViT outperformed all the models by achieving 99.1% accuracy with better efficiency.

Some studies have employed ViTs without considering sustainability, such as the authors in [9], who compared two Vision Transformer variants (ViT-16 and ViT-32) in the binary classification of retinal OCT images. The ViT models were trained to detect Diabetic Macular Edema (DME), using three optimizers: Adam, Stochastic Gradient Descent (SGD), and Root Mean Square Propagation (RMSProp). Two balanced datasets were proposed: KMC and Mendeley Datasets. A custom classification head was added. The GRAD-CAM was used for visualization. ViT-16 outperformed all the baselines across both datasets. The Adam optimizer achieved the best results, with 100% recall and stable training.

However, a limited number of OCT studies focused on ViT robustness against adversarial attacks, such as in [8]. This study proposed adversarial training as a defense mechanism. Exploring the performance using a binary labeled list of medical datasets: Fundoscopy, Chest X-ray, ISIC2019, Breast Histology. Images were normalized using ImageNet normalization (during the preprocessing phase). The proposed model is a pretrained ViT–Tiny model. The study employed hold-out validation (85% of the training set and 15% of the testing set). All models were evaluated against multiple attacks (FGSM, BIM, PGD, and CW) using these hyperparameters (ε varied between 0.25/255 and 5/255 for FGSM, PGD-20 steps, and BIM-40 steps). The ViT model outperformed all CNNs on balanced datasets in terms of both clean accuracy and adversarial robustness.

The authors of [10,19] proposed lightweight models without evaluating their robustness to attacks. In [9], ViT variants were compared based on performance, without considering robustness against attacks. The study in [8] focused on robustness enhancement but utilized an imbalanced dataset and applied adversarial training without considering proper ImageNet normalization.

Many studies have employed a hybrid framework combining a CNN with a Transformer to classify retinal OCT images in different research directions. For example, the authors of [20] aimed to enhance efficiency by proposing a hybrid CNN-Vision Transformer architecture on OCT2017, OCT-C8, and an external validation dataset. The model architecture comprises a list of blocks. The Depthwise Convolution DW block uses depth convolution for local lesion scanning. The MobileViT block (MV) is based on Mobile-ViT for learning global patterns. The Convolutional Block Attention Module (CBAM) block focuses on the relevant regions using average pooling, with a final layer for classification. The proposed approach achieved a high performance, but without enhancing the resistance of the model to attacks.

Similarly, the authors of [21] proposed an MGR-GAN to classify retinal OCT images. This hybrid model architecture combines a Transformer with a CNN. The UCSD OCT dataset was used in this study. The study designed a generator (Transformer-based) to generate images, and a discriminator (ResNet) to distinguish between generated and original images. The discriminator is also used for feature extraction and classification with three convolutional layers (F1, F2, and F3) and a softmax layer. The model achieved a 99% accuracy rate. This study did not focus on the security and sustainability of the model.

Similarly, the authors [22] worked to address many problems in the related work, including the need for large, balanced, diverse datasets, and the reduction of health risks in Fluorescein Angiography (FA) imaging that requires dye injection. The study used a large ultra-widefield retinal imaging dataset with 1198 patients, along with Messidor-2 for external evaluation, and proposed a hybrid approach combining a pix2pixHD-based GAN for multi-phase FA image synthesis and a Swin Transformer for diabetic retinopathy classification. The results demonstrated that the generator achieved high realism (SSIM = 0.82–0.85) with an AUC of 0.910, and an accuracy of 82.9%.

Similarly, the authors of [23] proposed an RAD-IoMT defense mechanism. It is a transformer-based detector that detects adversarial attacks and then passes only clean attacks to the CNN classifier. This study used three datasets (retinal OCT, skin cancer, and chest X-ray). The VGG-16 (standard 138 million parameters) model was trained as a classifier. The transformer was trained using binary classification to detect attacks. The study performed four attacks (white box: PGD and FGSM, and black box: AGN and AUN). The detector achieved an accuracy between 92% and 95%, whereas the classifier achieved an accuracy between 86% and 97%. However, deploying two complex models (detector and classifier) incurs an extra cost (heavy deployment). Additionally, if the attacker fools the filter (detector), the classifier crashes (a Single Point of Failure). Table 1 summarizes the results of the previous studies.

This study adopts the Retinal OCT classification C-8 dataset [24] because this dataset was designed for multi-classification projects, including eight classes. This dataset is publicly available on Kaggle, labeled, and categorized with a balanced class distribution, including 24,000 images. This dataset contains seven retinal OCT diseases: Age-related Macular Degeneration (AMD), Choroidal Neovascularization (CNV), Central Serous Retinopathy (CSR), Diabetic Macular Edema (DME), Diabetic Retinopathy (DR), Drusen, Macular Hole, and Normal Retina. This makes the dataset ideal for building predictive models. Fig. 2 shows an example of each class. In addition to the C-8 dataset, this study incorporates the OCTID dataset, which is available on Kaggle and consists of five classes: Age-related Macular Degeneration (AMD), Central Serous Retinopathy (CSR), Diabetic Retinopathy (DR), Macular Hole (MH), and Normal retina. The dataset contains around 588 OCT images. OCTID is widely used to evaluate model generalization under limited data scenarios. In this study, it is used to address domain shift through a small fine-tuning subphase [25].

Furthermore, the UCSD-3 dataset (derived from the UCSD OCT dataset) is included for additional evaluation. This dataset is also available on Kaggle and consists of three classes: Choroidal Neovascularization (CNV), Diabetic Macular Edema (DME), and Normal retina. In this study, it is used as an external evaluation dataset. The test set includes 750 images, which are used for performance assessment [26].

This study proposes training a Vision Transformer model to classify retinal OCT diseases. The selected model backbone was ViT/B-16. This model serves as the standard and original baseline for Vision Transformer models. This architecture has demonstrated a robust performance in retinal OCT studies. However, further improvements in efficiency and robustness are required. This architecture is based on the transformer model [2]. It splits images into patches, each of which is treated as a token. It was pre-trained on ImageNet1k. Patches are processed through a linear layer to form a vector, which is referred to as patch embedding. Positional embeddings were added to preserve the spatial ordering. In addition, a special classification token (CLS) is appended to summarize all patches and capture all the relationships and information needed to help in the prediction (holding the final decision). The vectors were then processed using blocks. Each block contained a multihead self-attention (MHSA) and an MLP layer. Through the self-attention mechanism, the image patches attend to each other to model long-range dependencies. MHSA computes the attention scores using a dot product. ViT splits embeddings into heads. For each head, the Core Attention Formula (1) was applied [2]. (1)Attention(Q,K,V)=softmax(QKT/sqrt(dk))

The Softmax is applied to assign attention weights to each patch, and the CLS token is then processed through a linear layer and softmax to generate probabilities for each class. After MHSA, the embeddings were processed through an MLP layer (with an activation function) to capture the structures and features inside each image. The backbone architecture consisted of 12 blocks, 12 attention heads, 64 head dimensions, a 4.0 MLP ratio with GELU activation, 196 patches including the CLS patch, 86 million parameters, and a classification head. It is the base ViT model version, with a 16 × 16 patch size, 224 × 224 image size, and 768 embedding lengths. The attention mechanism enables the ViT model to capture the relationships between patches and comprehend the differences between various regions within the images. Fig. 3 shows the architecture of the vision transformer [2].

In this phase, the selected ViT model is trained. The proposed training pipeline starts with hold-out validation, 70% training, 15% validation, and 15% testing. This study proposed the application of clean training in two rounds. First, the model is trained to build a reliable classifier before pruning. Second, we fine-tune the pruned model to help it adapt to the new architecture. Two rounds were conducted using the same setup. Only a different number of epochs was used (first round 10 epochs, second round 2 epochs). An NVIDIA Tesla GPU P100 was utilized to train and evaluate the model (provided by Kaggle), 32 batch size and four CPU workers to prepare batches during training.

The training set was utilized for building the model, and the validation set for model selection, hyperparameter tuning, and pruning. The test set was used in the evaluation phase. The pipeline started resizing and then cropping to 224 × 224. Augmentation (random zoom, rotation (+−20), brightness (0.3), contrast adjustment (0.3), flipping (p = 0.5), and shift (+−10)). Subsequently, ImageNet normalization was applied. In clean training, ImageNet normalization can be applied directly during the preprocessing phase without any issues. However, in the adversarial training phase, this study proposed a custom pipeline. The training setup includes cross-entropy as a loss function (difference between actual and predicted values) and an AdamW optimizer to change the model parameters according to gradients to reduce the loss. Weight decay (L2-regularization to prevent parameters from having large values) with the Warmup scheduler (to control the LR values during training), Automatic Mixed Precision (AMP) (to reduce training cost), and LR cosine. A list of anti-softmax saturation techniques was proposed: Temperature scaling 1.5, Post-Attention Layer Norm after attention residual, Label Smoothing 0.1, and Gradient Clipping 1.0.

In each epoch, the model switched from the training model to the validation model. This assists in monitoring the performance of the model during the training. This study proposes a light-validation setup that includes six batches (192 images). Early stopping is triggered to stop the training after three rounds without any enhancements. In the validation and evaluation, the image was resized and cropped to 224 × 244 pixels. However, without augmentation. In addition, light hyperparameter tuning was proposed to select the optimal values for two parameters: the Learning Rate and weight decay. The hyperparameter tuning mechanism was based on five training epochs (8% sample from the training set), with a small evaluation (while training) by a light validation of three batches. This selects the optimal values through the proposed search space for each parameter, as follows:

Learning rate search space: {5 × 10⁵, 1 × 10⁴, 2 × 10⁴, 3 × 10⁴}

The values selected based on hyperparameter tuning are (LR = 5 × 10⁵ and Weight decay = 0.01). Fig. 4 summarizes the proposed clean training pipeline.

This phase prunes the proposed trained model after clean training. This study proposes a pruning mechanism based on a validation set. An importance score was assigned to each block of the ViT model.

In the ViT structure, a single gradient is specified for each parameter within a single neuron. The gradient during training represents two values: magnitude and direction. The magnitude represents the impact of this parameter change on the loss. The direction illustrates how the parameter should be adjusted to minimize loss. The direction was ignored by considering only the magnitude. This determines the importance of the parameter. A neuron with high-magnitude parameters (weights) is an especially important neuron, while a block that holds a particularly important list of neurons is also particularly important. The absolute gradient value represents the magnitude without direction (i.e., it does not include positive or negative signs in the absolute values). The average absolute value for all gradients belonging to a single block represents the importance of the block (how strongly this block impacts).

The proposed mechanism computes the average values of the absolute gradients within each block. Starting with 30 batches of images from the validation set. With forward and backward passes (backpropagation), gradients are calculated for each block. The forward pass is used to compute the loss, and the backward pass computes the gradients. The average of the absolute gradients can be computed as (sum of all gradients divided by the number of gradients). Then, a ranking phase was conducted to identify high- or low-impact blocks (ranked by importance). A low average has a minimal impact. The pruned architecture was fine-tuned using clean training within 2 epochs. This allows the model to adapt to the new architecture and achieve high performance. The proposed ViT backbone included 12 blocks. This mechanism proposes determining the search space of architectures to determine the optimal one. The proposed search space consisted of 12 blocks, resulting in 12/2 = 6. Then, 6 + 2 and 6 − 2. The proposed search space consists of (4, 6, 8) blocks. In the experimental phase, we conducted a series of evaluations to assess each option within the search space. Starting from removing eight blocks and keeping four, pruning six, or pruning four. The study selects the most efficient architecture that only drops the accuracy by less than 2% to find a sustainable model without degrading the performance. The search space represents approximately 30%, 50%, and 70% reductions in model depth. This selection provides well-separated and meaningful compression regions. Where 8 blocks approximate the upper range (layers 7–11), 6 blocks capture the central 50% compression point, and 4 blocks represent the lower range (layers 1–5). This search space provides a representative summary of performance-efficiency trade-off across the full depth (1–12 blocks), enabling effective analysis without exhaustively evaluating all configurations. The next phase involves the proposed adversarial training pipeline. Fig. 5 shows the pruning approach.

The adversarial training phase is an extended training phase after cleaning to make the model robust against adversarial attacks. Adversarial training shows the model slightly modified images during training, which forces the model to learn a feature while staying stable even when the input changes. The model learned to focus on global patterns instead of intricate details, making its layers and tokens more robust. This phase applies adversarial training to the pruned model by applying the same anti-saturation techniques applied to the clean training pipeline (temperature scaling 1.5, post-attention layer norm, label smoothing 0.1, and Gradient Clipping 1.0). The hold-out validation includes a 70% training set, 15% testing set, 15% validation set. The validation set consisted of a stratified sample of 125 images belonging to each class. Validation with only the forward pass (without gradients). The model switches from training mode to validation mode after the end of each training epoch. Validation of 2800 images to maintain clean generalization. Validating clean performance is important to ensure that adversarial training does not harm the performance against clean data. Adversarial training is more complex than clean training and requires more resources. This study proposes the application of AMP precision, while hyperparameter tuning and augmentation were ignored in this phase to avoid additional costs. The same GPU engine (p100) was used in the training and evaluation phases.

The images were resized to 224 × 224 pixels. ImageNet normalization is then applied. ImageNet normalization is important for pretrained models (such as ViTs) to ensure consistency in the input distribution. Adversarial attacks apply perturbations to pixel space. The difference between spaces can change the determined epsilon parameter, which harms adversarial training. ImageNet normalization is represented by formula (2) below [27]:(2)xc′=(xc−μc)/σchere, c ∈ {R, G, B} denotes to the RGB channel, x_c ∈ [0, 1] is the pixel space, and μc, σc denote the channel-wise mean and standard deviation computed from the ImageNet dataset. Adversarial perturbations are defined in the pixel space as x~c=xc+ε, where ε is the perturbation magnitude. The transformation to ImageNet space is computed as follows.

Thus, the effective perturbation in pixel space becomes Δ·xc = ε·σc. The effective perturbation is approximately 0.069, which is smaller than ε=0.3. This demonstrates that applying perturbations in normalized space leads to weaker adversarial perturbations during training and evaluation. This study proposes wrapping ImageNet normalization into the model. This adds the attack’s perturbations on pixels directly (within the pixel space 0, 1) and then performs normalization.

The value 0.9 corresponds to the correctly perturbed pixel value in the original space. Wrapping normalization in a model is the correct normalization mechanism for adversarial training. Adversarial training generates attack samples by identifying spots to fool the model based on the gradient direction, thereby increasing loss. The model was then trained on the generated adversarial samples to become robust against these spots, thereby reducing the loss. The gradient direction guides adversarial sample generation and guides model training on these samples. Eq. (3) represents the adversarial training equation [28]. (3)min_E(x,y)[max_L(fθ(x+δ),y)]s.t.||δ||∞≤ε

The proposed model was trained using a combination of clean and adversarial examples. Clean samples were injected with PGD and BIM samples, 32 batch size with (four CPU workers) for all the samples (1:1:1 Clean: PGD: BIM). The batch size consisted of 96 images. The gradients of the loss were computed by automatic differentiation in Pytorch and used to construct the PGD and BIM perturbations under the L-infinity constraint. These perturbations were dynamically updated at each iteration using the current model parameters. The training hyperparameters were assigned based on the default setup and the most common practice (ε = 8/255, α = 2/255, PGD-7 steps, BIM-10), including six training epochs and early stopping triggered by the validation loss (two epochs without improvements, the model will stop training), and a cross-entropy loss function (with label smoothing 0.1). AdamW optimizer (Learning Rate: 1e−4 and weight decay: 0.01 assigned manually without tuning). In this phase, the PGD and BIM generated adversarial samples retained the original ground-truth labels of their corresponding clean images, and the model was trained jointly on clean and perturbed samples using these labels. Formula (4) represents the PGD attack, and Formula (5) represents the BIM attack. Fig. 6 shows the adversarial training pipeline [29,30]. After the adversarial training, the model was fine-stuned for two epochs using clean training to adapt to domain shift on a mixed dataset composed of OCT-C8 and OCTID samples. During this stage, most model parameters were frozen, and updates were applied only to the classification head, the last transformer block, and the final normalization layer. The fine-tuning was performed on 5000 images and included only the shared classes between the two datasets.(4)x_adv = Proj_ε(x+α⋅ sign(∇_xJ(θ,x,y))) (5)x_t∧(+1)= Clip_(x,ε∧∞)(x_t+α⋅ sign(∇_xJ(θ,x_t,y)))

The evaluation plan included several experiments to evaluate the proposed model (GreenShield-ViT). The study plan explored the pruning search space to select the optimal model. This comparison was used to investigate sustainability enhancements. t-Distributed Stochastic Neighbor Embedding (t-SNE) plots and attention rollout-visualization are investigated to ensure that clusters remain clearly separated and the attention is in the same relevant regions. Following adversarial training, the GreenShield-ViT model’s performance and calibration were assessed against adversarial samples. Five repeated stratified hold-out experiments were conducted to ensure statistical robustness. The experiments were performed using different random seeds (42, 123, 2024, 3407, and 9999). In each trial, the dataset was randomly split into 70% training, 15% validation, and 15% testing while preserving class distribution. The GreenShield-ViT is benchmarked against Mobile-ViT Extra Extra Small (XXS), ViT-Small (ViT-Tiny/16), and ViT-Tiny (ViT-Tiny/16). These baselines were selected from related work sections [8,10,19]. We explored their robustness against attacks after applying the proposed clean and adversarial training pipelines to all models. Investigating their robustness against attacks. The robustness evaluation is based on a small set of white-box attacks, including (FGSM, PGD, BIM, and the proposed hybrid model) [6]. The proposed hybrid attack is a combination of FGSM and PGD attacks. The hybrid attack begins with an FGSM step (based on the epsilon range and gradient direction) to make a significant jump in reaching the bound (the allowed value based on the epsilon range). Subsequently, if the gradient direction changes, it includes the PGD steps. If the direction remains the same, the steps are projected to prevent pixels from moving farther than ±ε. This attack is designed to initiate from the boundary rather than starting from the pixel value. This attack is introduced as an additional evaluation scenario to assess the robustness of the proposed model under mixed iterative attack strategies. This setting reflects a potential adversarial scenario where multiple attack strategies are combined to fool the model. This evaluation also assesses the robustness against unexpected attack strategies that were not explicitly used during training. The same evaluation hyperparameters were applied to all attacks. Default hyperparameters were selected (following widespread practice). The hyperparameters of the proposed evaluation attack are listed in Table 2. Additionally Fig. 7 shows how the adopted parameters can generate a visually imperceptible perturbation. The per-pixel difference was computed, and matplotlib was utilized to highlight the changed pixels (the highest perturbation magnitude).

After demonstrating that the proposed model is more robust than the baselines, the model was fine-tuned using a combined dataset from OCT-C8 and OCTID to mitigate domain shift and improve generalization. It was subsequently evaluated on three datasets (OCT-C8, UCSD-3, and OCTID), where UCSD-3 represents an unseen dataset for assessing generalization, and OCTID provides an additional evaluation under a different domain shift [25,26]. The UCSD-3 dataset was proposed solely for evaluation to ensure that the model can generalize well against external samples beyond the main dataset. This dataset is available on Kaggle with three labels (CNV, DRUSEEN, Normal). The test set contained 750 images. All the experiments were conducted using PyTorch and Torchvision. Finally, the model is evaluated under different epsilon levels (2/255, 4/255, 8/255, 16/255) to ensure that the model is robust beyond the common evaluation practice. Fig. 8 shows the attacks on the different epsilon levels. In addition, the model is evaluated against PGD-7 with three random restarts and Transfer-PGD attack, which was generated on a surrogate original ViT model before pruning (12-blocks).

A set of evaluation metrics was adapted to interpret the findings and results. This study evaluates the model performance based on the Accuracy, Precision, Recall, F1-score, and average of max softmax probabilities to check the softmax saturation. Prior studies have demonstrated that the average max softmax probability (Avg MPS) is a valid metric to check softmax saturation [12,13]. When the maximum softmax probability ranges between (0.97–1.00), then the softmax is saturated. In addition, the Expected Calibration Error (ECE) measures the gap between the confidence and accuracy of the model (higher ECE indicates a higher gap) [31]. The efficiency of the model is evaluated using Floating-point operations per second (FLOPs), which is computed using fvcore and ptflops (in Python), Number of Model parameters, Model Size (in MB), Inference Time [32], Inference Memory usage [33], and Inference Carbon Footprint (CO₂). The carbon footprint refers to the greenhouse gas emissions generated by inference measured in CO₂-equivalent units. Carbon emissions were computed using the Green Algorithm estimation tool [34]. Pytorch Compute Unified Device Architecture (CUDA) was used for timing and memory measurements. The robustness of the Model was evaluated using the model’s accuracy against each attack [35], Attack Success Rate, the robustness gap between clean and adversarial accuracies, and the improvement rate before and after applying adversarial training [36]. The formula below represents the carbon footprint equation [34]. (6)Carbon Footprint=Energy Needed*Carbon Intensity 

In this experiment, the lightest architecture was selected without degrading accuracy by more than 2%. The results demonstrated that the 6-block model (pruning 6) was the optimal model, balancing performance and efficiency. The study selected the 6-block as the GreenShield-ViT model architecture. The 4-block model is the lightest, but the performance is reduced from 96.2% to 89.9% (6.3% reduction). While the 6-block model reduced from 96.2% to 95.6% (0.6% reduction), which is less than 2%. The 8-block model achieved an impressive performance, but the 6-block model achieved a competitive performance with significant enhancement of efficiency. The stability of block importance reveals that Transformer layers have unequal contributions, where lower-ranked blocks provide limited information and can be removed. Pruning up to 6 blocks maintains performance, whereas further pruning eliminates critical feature representation and results in performance degradation. All models were evaluated under identical experimental conditions. All experiments were conducted in a Kaggle notebook using an NVIDIA Tesla P100 GPU, with the same software framework (PyTorch and TIMM), identical input resolution (224 × 224), and the same test dataset. Under these standardized settings, all measurements were made to provide unbiased comparisons across models. The pruning was conducted based on 30 batches from the validation set. The results demonstrate that the top-rank blocks remain consistent across different batch sizes (30 and 40 batches), indicating that importance estimation is stable, and based on this observation, 30 batches were selected. Table 3 demonstrated stability in ranking. Table 4 presents the results of the pruning search space, and Fig. 9 shows the efficiency enhancement. Fig. 10 shows the ROC curve. The t-SNE plot (Fig. 11) shows that class separability is preserved after pruning, as the clusters remain clearly separated. In addition, the attention rollout visualization (Fig. 12) confirms that the model focuses on similar clinically relevant regions before and after pruning. Tables 5 and 6 demonstrate consistent performance across five trials.

The results indicate that reducing the number of transformer blocks does not significantly degrade performance beyond a certain depth. This is because earlier transformer layers capture the most critical structural and textural features of retinal OCT images, while deeper layers provide only marginal refinements. Given the structured nature of OCT data, redundant deeper representations can be removed without sacrificing discriminative capability. This explains why the 6-block configuration achieves a near-optimal balance between efficiency and accuracy, while the 8-block configuration offers minimal additional performance gains at a higher computational cost.

This experiment investigated the robustness of the selected pruned model (6-Block) against adversarial attacks. Adversarial training was applied to the pruned model (GreenShield-ViT). This model represents the proposed approach, including all phases in this experiment. This experiment included the results before and after the proposed adversarial training to demonstrate the enhancement. The results indicated that the proposed adversarial training enhanced the robustness and calibration of the model. Before applying adversarial training, the model was unstable, achieving low accuracy with a high confidence rate. Table 7 shows the enhancement in the robustness. The calibration enhancements are demonstrated in Table 8 using the average of the maximum softmax probability (Avg. MSP) and expected calibration error (ECE). Fig. 13 illustrates the alignment between the accuracy and average MSP before and after adversarial training. As shown in Tables 9 and 10, the model demonstrates consistent performance across all trials, with minimal variation in both clean and adversarial accuracy, confirming its stability.

To address potential concerns regarding overfitting and result saturation, we conducted five independent experimental trials using different random seeds. The results, summarized in Tables 7 and 8, report mean, standard deviation, and 95% confidence intervals for all evaluation metrics. The observed low variance across trials confirms the stability and generalization capability of the proposed GreenShield-ViT model, even under adversarial conditions.

This experiment aims to evaluate the robustness of the baselines (Mobile-ViT, ViT-Tiny, and ViT-Small) against adversarial attacks (FGSM, PGD, BIM, and the hybrid attack). The robustness performance is evaluated to compare baselines with the proposed model and identify the most robust, lightweight model. The same proposed pipeline was applied to all models. Exploring the robustness of these lightweight models addresses a research gap in prior OCT work. To ensure a fair comparison, all models were trained using identical data splits, preprocessing steps, hyperparameter settings, and adversarial training configuration, and evaluated under the same attack conditions. The results demonstrated that the models achieved impressive performance against clean samples, with a significant robustness gap. Furthermore, larger models are not necessarily more robust, as ViT-Small achieved an impressive performance against PGD and BIM samples, which were included in training, but performed poorly against FGSM and Hybrid attack, which indicates attack-specific overfitting and limited generalization. Table 11 presents the results of the baseline models, and Fig. 14 illustrates how the proposed model is significantly more robust than the baseline models, based on the success rate of each attack.

This experiment aims to fine-tune the model on a mix of OCTID and OCT-C8 datasets. This fine-tuning helps the model to adapt to domain shift. After fine-tuning, the model was evaluated against three datasets: OCT-C8 (15% test set images), OCTID (115 images), and UCSD-3 (735 images). The results indicate that the proposed model achieves a high clean accuracy. The results against the OCTID dataset indicate that the model adapted to domain shift, while the results against UCSD-3 indicate that the model can generalize against unseen data. The significant improvement in the robustness after adversarial training demonstrates that the proposed pipeline builds a robust model. In addition, the findings illustrate a strong alignment between the accuracy and confidence levels after adversarial training. Table 12 lists the experimental results, while Tables 13–15 represent the confusion matrix across different datasets.

This Experiment aims to evaluate the proposed model against varying epsilon levels to ensure that the model can generalize beyond the specific default parameters proposed for evaluation (default in ImageNet 8/255). The results demonstrate that the model can achieve competitive results under standard adversarial benchmark settings (2/255, 4/255, and 8/255), while maintaining acceptable robustness at 16/255, which exceeds commonly adopted research benchmarks. The model was evaluated against PGD-R3(3 random restarts) and Transfer-PGD scenarios. The experimental results are listed in Table 16. Fig. 15 represents the loss level against different epsilon values. The results indicate that the model can achieve a satisfactory performance against different scenarios, while the model does not exhibit gradient masking, as increasing the perturbation magnitude consistently leads to a higher loss and lower accuracy.