The natural immune system exhibits remarkable properties such as adaptability, diversity, distributiveness, and robustness, enabling it to effectively recognize and eliminate harmful pathogens. Artificial Immune Systems (AIS) are a family of bio-inspired computational algorithms developed by modeling these biological immune mechanisms and applying them to a wide range of computational problems. From a computational perspective, AIS approaches offer several advantages that make them attractive for solving complex problems in computer science. By incorporating principles inspired by the natural immune system, AIS models often provide unique capabilities such as adaptability, fault tolerance, and robustness [1]. Many AIS algorithms are considered computationally lightweight in classification and anomaly detection tasks when compared with conventional machine learning methods. Furthermore, previous studies have reported that AIS-based approaches can achieve competitive or superior detection performance in several application domains while maintaining good generalization ability [2].

Among the various AIS algorithms, the Dendritic Cell Algorithm (DCA) is one of the most widely studied models. DCA emulates the behavior of biological dendritic cells (DCs), which play a key role in the innate immune system by detecting potentially harmful signals and coordinating immune responses. Inspired by this biological mechanism, the DCA has been successfully applied to anomaly detection tasks in several domains, including intrusion detection, fault diagnosis, and spam filtering [3]. The algorithm is particularly suitable for binary classification problems, where each input instance is transformed into three biologically inspired signal types: Pathogen-Associated Molecular Patterns (PAMP), Danger Signals (DS), and Safe Signals (SS). These signals are processed together with data identifiers (antigens) through a context assessment mechanism that determines whether a given instance represents normal or anomalous behavior [4].

In general, the DCA consists of four main processing phases: (i) preprocessing and initialization, (ii) detection, (iii) context assessment, and (iv) classification [3,4]. Among these stages, the preprocessing phase plays a critical role because it transforms raw input features into the three signal categories (PAMP, DS, and SS) that drive the entire detection process. Consequently, the effectiveness of the DCA largely depends on the quality of feature selection and the correctness of feature-to-signal mapping. Accurate identification of informative features and appropriate assignment of those features to the corresponding signal categories are essential to ensure that the generated signals are meaningful and discriminative for anomaly detection.

Traditionally, the preprocessing stage of the DCA relies heavily on manual configuration or static feature transformation techniques. Methods such as Principal Component Analysis (PCA) [3], Correlation Coefficient (CC) [4], Information Gain (IG), Rough Set Theory (RST) [5], and Fuzzy Rough Set Theory (FRST) [6] have been widely used to construct reduced feature spaces with lower dimensionality. These techniques typically remove weak or irrelevant attributes based on statistical relevance measures. However, relevance according to these criteria does not necessarily guarantee that a feature belongs to the optimal DCA feature subset, nor does irrelevance imply that a feature is completely unsuitable. As a result, filter-based reduction methods may produce suboptimal signal mappings that negatively affect anomaly classification performance [7].

To further improve preprocessing, several studies have incorporated machine learning and optimization techniques into the DCA framework. For example, Support Vector Machines (SVM) [8] and K-Nearest Neighbors (KNN) [9] have been used to assist feature selection or signal generation. In addition, optimization-based approaches such as Grouping Particle Swarm Optimization (GPSO) [10] and Grouping Genetic Algorithms (GGA) [11] formulate signal categorization as a combinatorial grouping problem. In these approaches, input features are divided into mutually disjoint subsets representing PAMP, DS, SS, and unused features. These techniques partially automate the signal categorization process and reduce manual intervention. However, the resulting mappings remain static, meaning that once the optimization process finishes, the feature-to-signal assignments remain fixed.

A major limitation of evolutionary optimization approaches such as PSO-DCA and GA-DCA is their limited generalization capability. Because these algorithms optimize feature groupings based on a specific dataset, the resulting mappings often reflect dataset-specific statistical patterns rather than generalizable relationships. Consequently, the learned mappings may perform poorly when applied to new datasets or different domains [12]. In addition, PSO is highly sensitive to parameter configuration and does not incorporate the semantic meaning of the DCA signal categories. Similarly, GA-based methods may suffer from instability due to stochastic operations such as mutation and crossover. As a result, these optimization methods typically require re-optimization for each new dataset, which limits their applicability in multi-domain anomaly detection tasks where adaptability and robustness are required.

Another limitation of the traditional DCA framework is the reliance on static and manually defined signal mappings. In many implementations, users must assign features to PAMP, DS, and SS based on intuition or domain knowledge [13]. Once defined, these mappings remain fixed and cannot adapt to evolving data distributions or changing anomaly patterns. This rigid structure limits the ability of the DCA to respond to new or previously unseen data characteristics, thereby reducing its effectiveness in dynamic environments.

To address these challenges, this study proposes a novel adaptive preprocessing framework called Reinforcement Learning-based DCA (RL-DCA). The proposed approach reformulates the feature-to-signal mapping process as a reinforcement learning (RL) task [14,15]. Specifically, the RL-DCA model employs Q-learning [16] to iteratively optimize the assignment of features to the three signal categories (PAMP, DS, and SS) based on reward feedback derived from classification performance. Unlike static signal mappings, the RL agent continuously updates its signal categorization policy through interaction with the environment. This adaptive mechanism enables RL-DCA to automatically adjust signal mappings as data characteristics change, thereby reducing reliance on manual configuration.

Such adaptability is particularly important in domains such as cybersecurity, financial fraud detection, and behavioral analytics, where anomaly patterns evolve over time and static preprocessing methods may quickly become ineffective. By incorporating a self-learning mechanism into the preprocessing phase, RL-DCA can dynamically adapt to changing data distributions and maintain stable anomaly detection performance across heterogeneous datasets.

Motivated by these challenges, this study addresses the following research question:

How can the preprocessing phase of the DCA be made adaptive, learning-driven, and generalizable without relying on manual configuration or static optimization strategies?

To answer this question, the main contributions of this study are summarized as follows:

Adaptive Signal Categorization: An RL-based signal categorization mechanism (RL-DCA) is introduced to automatically learn feature-to-signal mappings in the preprocessing stage of the DCA.

The remainder of this paper is organized as follows. Section 2 presents an overview of the Dendritic Cell Algorithm and its computational principles. Section 3 introduces the reinforcement learning framework based on Q-learning. Section 4 reviews related work on DCA preprocessing and signal categorization. Section 5 describes the proposed RL-DCA model. Section 6 presents the experimental setup and datasets used in the evaluation. Section 7 reports and analyzes the experimental results. Section 8 investigates the sensitivity of the MCAV threshold. Section 9 discusses the implications and limitations of the proposed approach. Finally, Section 10 concludes the paper and outlines directions for future research.

The DCA is an AIS technique inspired by the behaviour of biological DCs in the innate immune system. In immunology, DCs act as professional antigen-presenting cells with the ability to ingest, process, and present antigens to T-cells. Their behaviour is regulated by environmental signals: in the presence of pathogenic or danger cues, DCs transition to a mature state, triggering an immune response; in contrast, exposure primarily to safe signals leads to a semi-mature state that promotes immune tolerance. This dual-state mechanism is the biological foundation for distinguishing between normal and anomalous patterns [3].

For clarity, the key variables and notations used throughout this study are summarized here. The DCA operates using three immunological signal categories: Pathogen-Associated Molecular Patterns (PAMP), Danger Signals (DS), and Safe Signals (SS). During the detection phase, each dendritic cell accumulates three output signals: the costimulatory signal (Csm), the mature output signal (MAT), and the semi-mature output signal (SEMI). The migration threshold (MT) determines when a dendritic cell stops sampling antigens and performs context assessment. The final classification decision is derived using the Mature Context Antigen Value (MCAV), which measures the proportion of mature contexts associated with each antigen. In the reinforcement learning component, α denotes the learning rate, γ represents the discount factor, and ε controls the exploration rate in the ε-greedy policy. These parameters regulate how the RL agent updates its signal assignment strategy during training.

Reinforcement Learning (RL) [6] is a sequential decision-making paradigm in which an agent learns an optimal policy through direct interaction with an environment by maximizing cumulative reward. In the proposed RL-DCA framework, RL is not employed as a conventional classifier. Instead, it serves as a search and optimization mechanism to dynamically learn feature to signal mappings for the DCA. This design directly addresses a key limitation of classical DCA, namely its reliance on manually defined and static signal categorization rules that are often domain dependent and difficult to generalize.

Among various RL approaches, Q-learning is adopted due to its simplicity, model-free nature, and proven convergence properties. Q-learning learns an action–value function Q(s,a), which estimates the expected long-term reward of action a in state s and following the optimal policy thereafter. In the context of RL-DCA, the environment state s is derived from a compact representation of the input features, while the action space consists of three biologically inspired signal categories: PAMP, DS, and SS. Each action corresponds to assigning a particular signal type to the current data instance.

At each interaction step, the RL agent selects an action using an ε-greedy strategy, which balances exploration of alternative signal assignments and exploitation of learned knowledge. The reward function is label guided and designed to reinforce biologically meaningful behavior: assigning PAMP to abnormal samples or SS to normal samples yields a positive reward, while incorrect assignments are penalized. This reward structure allows the agent to gradually learn which feature patterns correspond to each signal category without relying on expert-defined thresholds or fixed heuristics. The Q-values are updated iteratively using the Bellman optimality equation, shown as Eq. (3):(3)Q(s,a)←Q(s,a)+α[r+γmaxQ(s′,a′)−Q(s,a)]where α is the learning rate, γ is the discount factor, r denotes the immediate reward, and s′ represents the next observed state. Through repeated updates, the Q-table gradually converges toward an optimal signal mapping policy that reflects the relationship between feature patterns and immune signal categories. The learned signal vectors are subsequently passed to the DCA, which performs the final immune-inspired aggregation and decision-making process. In this way, Q-learning enhances the adaptability of DCA while preserving its biological interpretation and anomaly detection mechanism.

In the proposed RL-DCA framework, reinforcement learning is used as a label-guided optimization mechanism for signal categorization. Because the reward function uses ground-truth class labels during training, this stage introduces a supervised learning component. However, RL is applied only to optimize signal generation before the standard DCA processing stage, while the final anomaly classification is still performed by the original DCA context assessment and MCAV mechanism.

The preprocessing phase of DCA, particularly feature selection or feature reduction, and signal categorization, plays a crucial role in its anomaly detection performance. This phase determines how raw input features are transformed into the three core signal types: PAMP, DS, and SS [3].

In classical DCA implementations, preprocessing is typically performed manually using rule-based or statistically driven mappings informed by domain expertise. Practitioners select relevant features and assign them to signal categories based on intuition, prior knowledge, or predefined thresholds [3]. Although this approach is simple and interpretable, it introduces subjectivity and rigidity, which limit scalability and adaptability, especially in high-dimensional, noisy, or dynamic environments.

To mitigate the limitations of manual preprocessing, several feature reduction techniques have been incorporated into the DCA framework. Principal Component Analysis (PCA) [8] is among the most widely used methods, where original features are transformed into a smaller set of uncorrelated components before being mapped to DCA signals. PCA improves computational efficiency and reduces redundancy; however, it often sacrifices semantic interpretability, making it difficult to trace how individual features influence signal behavior [9].

Other statistical feature selection methods, such as Information Gain, correlation coefficients, and mutual information, have also been employed to rank and select features prior to signal mapping [10]. While effective in certain scenarios, these approaches generally assume linear relationships and often fail to capture complex feature interactions, reducing their robustness in noisy or non-linear datasets [11].

More recent studies have explored automated feature reduction techniques to further enhance the DCA preprocessing phase. Kernel Principal Component Analysis (KPCA) has been proposed as a nonlinear extension of PCA, enabling projection into kernel-induced feature spaces that better preserve complex relationships and filter noise prior to DCA execution. Autoencoders (AE) have also been adopted to learn compact latent representations through unsupervised neural reconstruction, allowing nonlinear feature compression before mapping reduced features into DCA signals. Hybrid approaches, such as AEkPCA, combine AE-based representation learning with KPCA refinement to further decorrelate and enhance latent features. These methods have demonstrated improved anomaly detection performance across multiple domains. Despite their effectiveness, KPCA-, AE-, and AEkPCA-based pipelines introduce additional computational overhead, hyperparameter sensitivity, and static transformation stages that operate independently of the DCA decision process. Consequently, while these approaches reduce manual intervention, they remain external preprocessing modules rather than adaptive mechanisms integrated within the DCA framework itself [12].

Several studies have focused specifically on improving signal categorization in DCA. Mean Squared Error (MSE) and signal sensitivity measures have been used to evaluate the quality of feature-to-signal assignments. Rough Set Theory (RST) has also been applied to identify minimal feature subsets (reducts) that preserve classification performance [13]. The QuickReduct algorithm, for example, derives efficient feature sets while maintaining DCA accuracy. To handle uncertainty in feature importance, Fuzzy Rough Set Theory (FRST) has been introduced, enabling soft boundaries during categorization [13]. Although these techniques improve robustness, their configurations remain static and highly dependent on dataset characteristics.

To further enhance input signal generation, machine learning classifiers have been integrated with the traditional DCA. In SVM-DCA, Support Vector Machines are used to compute sparse weight matrices, where attributes with larger weights are retained and mapped to DCA signals [14]. Similarly, K-Nearest Neighbors (KNN) has been employed to filter attributes based on pairwise instance differences, forming hybrid KNN-DCA models.

Metaheuristic optimization techniques have also been explored to automate signal mapping. Genetic Algorithms (GA) evolve feature-to-signal assignments using mutation and crossover operations, optimizing classification accuracy [15,16]. Particle Swarm Optimization (PSO) and Genetic Grouping Algorithms (GGA) further frame signal categorization as a clustering task, grouping features into PAMP, DS, SS, and unassigned categories [14,17]. These methods reduce manual configuration but typically converge to fixed mappings. Additional studies have incorporated Bayesian Optimization with resource-efficient strategies such as Hyperband to fine-tune signal fusion parameters [18]. Although these methods enhance discriminative power, they still produce static mappings that do not adapt after training.

A critical limitation across existing DCA-based methods is the lack of adaptability in signal categorization. Most approaches generate fixed feature-to-signal mappings that cannot respond to evolving data distributions, class imbalance, or noise. For example, PCA may perform well on one dataset but poorly on another, while GA- and PSO-based methods are often sensitive to parameter tuning and dataset complexity [19].

Although the DCA is bio-inspired and widely used for anomaly detection [20], it also suffers from similar weaknesses [21]. In its standard form, the DCA relies on experience-based parameter settings and lacks a training phase, resulting in reduced categorization accuracy when handling large-scale or high-dimensional datasets. This rigid design limits robustness under real-world conditions.

Recent advances in artificial intelligence have significantly influenced the development of zero-day and anomaly detection systems. A comprehensive review by Yee et al. [22] highlights that AI-based detection methods can be broadly categorized into machine learning based, deep learning based, hybrid, and anomaly based approaches. While many supervised and deep learning models achieve high detection accuracy, the review emphasizes recurring challenges such as limited adaptability, dependence on labeled data, sensitivity to data distribution shifts, and difficulty handling unseen attack patterns. These limitations indicate that static learning mechanisms may struggle to maintain performance in evolving threat environments.

Similarly, Dai et al. [23] propose a hybrid intrusion detection framework that integrates an autoencoder with Random Forest and XGBoost classifiers to improve detection of zero-day attacks in unseen data. Their results demonstrate that incorporating anomaly detection improves generalization within the same dataset distribution. However, the model structure remains largely dependent on supervised classifiers after anomaly filtering, and adaptation does not occur within the internal feature transformation process itself. These findings collectively suggest that improving generalization requires not only anomaly aware modeling but also mechanisms that allow detection systems to adapt dynamically during operation.

Inspired by advances in adaptive optimization, recent studies have investigated Reinforcement Learning (RL) as a mechanism for enabling self-learning and adaptability in computational models. Liu et al. [24] applied deep RL for feature selection in high-dimensional domains, outperforming static filter and wrapper methods. Other works have used RL to optimize decision thresholds in network anomaly detection [25], adapt detection parameters in artificial immune systems [26], and control parameter updates in swarm intelligence algorithms [27]. These studies consistently demonstrate that RL enables models to adjust their behavior based on environmental feedback rather than relying on fixed configurations.

Motivated by both the limitations identified in recent AI-based zero-day detection frameworks and the adaptive capability demonstrated by RL-driven models, this study introduces reinforcement learning into the Dendritic Cell Algorithm (DCA) to enhance its signal categorization mechanism. Unlike static or handcrafted mappings, RL frames signal categorization as a sequential decision-making process guided by feedback from classification performance.

Specifically, this work replaces the static signal categorization mechanism with a dynamic RL-based strategy using Q-learning [28]. Through continuous interaction with the environment, the RL agent evaluates outcomes and iteratively updates its signal assignment policy. This design enables the DCA to self-adjust its internal signal transformation process in real time, thereby improving adaptability, generalization capability, and robustness across diverse benchmark datasets. A comparative overview of existing DCA preprocessing methods, including their advantages, limitations, and adaptability, is presented in Table 2.

A Data Sets

To evaluate the performance and generalizability of the proposed RL-DCA model, nine publicly available datasets were selected from reputable sources, including the UCI Machine Learning Repository [34], KEEL [35] and established cybersecurity research platforms [36,37]. These datasets represent a diverse set of binary classification tasks, varying significantly in feature (ranging from 3 to 85 features), sample sizes, and class imbalance levels. The datasets are collected from a variety of domains, including healthcare, finance, social behaviour, spam detection, and network intrusion detection. which allows a comprehensive evaluation of RL-DCA across multiple real-world contexts. Table 3 provides the details of each dataset, including the number of features, the total number of samples, and the ratio of class imbalance. To ensure compatibility with the RL-based signal transformation process, all categorical attributes were numerically encoded using label encoding. Numerical features were standardized via Z-score normalization to align data scales and enhance the convergence behavior of the learning algorithm. Data cleaning procedures included the removal of features with over 40% missing values and attributes with zero variance (i.e., those containing a single unique value). Outliers with a Z-score greater than 2.5 were either filtered or imputed using the feature mean, based on the Eq. (13):(13)zi,j=(xi,j−μi)/σjwhere xi,j denotes the value of the ith instance for the jth feature, μi is the mean, and σj is the standard deviation of that feature. This preprocessing ensures that the selected datasets are standardized, clean, and suitable for evaluating the adaptability and robustness of the RL-DCA algorithm across both general classification tasks and real-world anomaly detection scenarios.

B Experiment Setup

In this study, a set of experiments was conducted to evaluate the effectiveness and robustness of the proposed RL-DCA, which replaces the traditional signal transformation process in the DCA. The goal is to assess the ability of RL-DCA to dynamically learn and assign signals (PAMP, DS, SS) to features, thereby improving anomaly detection performance across various types of datasets. The experiments were carried out on nine publicly available datasets, which vary in feature dimensionality, data size, and domain characteristics. Each dataset was pre-processed by removing missing values, encoding categorical features using Label Encoder, and scaling numerical features using

Standard Scaler, following commonly adopted preprocessing practices in anomaly detection studies [38]. To ensure fair comparison and consistency across datasets, the same preprocessing pipeline was applied to all datasets. This approach is widely used in comparative benchmarking studies to avoid dataset dependent bias and to ensure that performance differences are attributed to the learning model rather than preprocessing variations [39].

The experimental protocol involves two main evaluations:

C Reinforcement Learning–Based Signal Transformation

The proposed RL-DCA framework employs a Q-learning agent to dynamically map input features to the DCA signal categories, namely PAMP, DS, and SS. Through interaction with the environment, the agent learns a signal transformation policy that maximizes the expected reward associated with correct anomaly classification. The behaviour of the RL agent is controlled by three main hyperparameters: the learning rate (α), the discount factor (γ), and the exploration rate (ε). These parameters regulate the learning dynamics of the agent, including how new information updates the Q-values, how future rewards are considered, and how the balance between exploration and exploitation is maintained during training.

The learning rate α determines how strongly newly observed rewards influence the update of the Q-values. In this study, α = 0.3 was selected to allow gradual updates to the Q-table while avoiding unstable fluctuations during learning. A moderate learning rate helps maintain stable convergence while still allowing the agent to adapt its policy. The discount factor γ controls the contribution of future rewards in the Q-value update. A value of γ = 0.9 enables the agent to consider long-term rewards while still emphasizing immediate feedback from the environment. This encourages the agent to learn signal categorization strategies that improve overall detection performance over time.

The exploration rate ε defines the probability of selecting a random action instead of the currently optimal action. In this work, ε = 0.1 was used to ensure sufficient exploration of the action space while allowing the agent to increasingly exploit the best-performing signal mappings identified during training.

The selected parameter configuration (α = 0.3, γ = 0.9, ε = 0.1) follows commonly adopted settings in Q-learning-based optimisation methods and was further validated through empirical experimentation. The convergence behaviour of the RL agent is illustrated in Fig. 6, which shows the evolution of the average reward during training on the SP dataset. As training progresses, the learning curve gradually stabilizes, indicating that the RL agent converges toward a consistent signal transformation policy. This behaviour demonstrates that the selected hyperparameter configuration supports stable policy learning within the RL-DCA framework.

D Enhanced DCA Classification

The transformed signals are processed using the standard DCA mechanism, where classification decisions are made based on the Mature Context Antigen Value (MCAV). A decision threshold of 0.85 is applied to distinguish between normal and anomalous antigens. This threshold value has been widely adopted in DCA related studies [40] and was further vali dated experimentally in this work to achieve a suitable balance between detection accuracy and false alarm rate. To ensure statistical reliability, each experiment was repeated for 10 independent runs. Datasets were randomly split into 80% training and 20% testing sets, and 10-fold cross-validation was employed. Model performance was evaluated using multiple classification metrics [41], including accuracy Eq. (14), precision Eq. (15), sensitivity (recall) Eq. (16), specificity Eq. (17), and F1-score Eq. (18), which are standard evaluation measures in anomaly detection and classification research. These metrics are defined as follows:(14)Accuracy=TP+TNTP+TN+FP+FN (15)Precision=TPTP+FP (16)Sensitivity (Recall)=TPTP+FN (17)Specificity=TNTN+FP (18)F1−score=2×Precision×RecallPrecision+Recall

To reduce the effect of randomness due to different data splits and initialization conditions, the reported results represent the mean and standard deviation across the repeated runs. The relatively small standard deviation values observed in the experiments indicate that the proposed RL-DCA model produces stable and consistent performance. Where TP, TN, FP, and FN denote true positives, true negatives, false positives, and false negatives, respectively. These metrics collectively provide a comprehensive assessment of classification performance by capturing overall correctness, anomaly detection reliability, detection completeness, normal-class recognition capability, and the balance between precision and recall. All simulations were executed using Python 3.10 in PyCharm on a Windows 10 machine with an Intel Core i7-10510U CPU (1.8 GHz) and 16 GB RAM. Visualization of results was performed using the matplotlib library.

The RL-DCA introduces an RL mechanism for signal categorization prior to executing the standard DCA pipeline. The runtime of RL-DCA therefore depends on the number of iterations over the dataset, the Q-table update operations, and the signal construction steps defined in Algorithm 1. According to Gu et al. [42], the runtime complexity of the standard DCA is bounded by O(n²), where n is the data size. Thus, this study focuses on the runtime of the complete RL-DCA. Table 4 shows the details of the primitive operations Algorithm 1, together with the number of times each operation is executed. For each input sample, the algorithm performs state construction, Q-table initialization (if required), ε-greedy action selection, reward calculation, Q-value update, and signal construction. All operations are bound by constant time except the Q-update, which requires computing a maximum over the action space. Since the action space is fixed to three categories PAMP, DS, SS), this operation also executes in constant time. Therefore, the runtime of the Q-learning stage scales linearly with the dataset size n. The runtime of the Q-learning stage can be expressed as Eq. (19):(19)T(n)=c0+n×(c1+c2+c3+c4+c5)where c0 is the initialization cost, and c1,…,c5 denote the constant costs of state processing, action selection, reward computation, Q-table update, and signal construction. Simplifying, we obtain Eq. (20):(20)T(n)=o(n)

Since the Q-learning stage is followed by the standard DCA classification phase with complexity o(n2), the overall runtime of RL-DCA using is:(21)TRL−DCA(n)=o(n)+o(n2)=o(n2)

As shown in Eq. (21), the worst-case runtime complexity of RL-DCA is O(n2), dominated by the standard DCA phase. The Q-learning signal mapping contributes only to the linear overhead. To further verify this analysis, the empirical runtime was calculated across multiple benchmark datasets. Fig. 7 illustrates the average execution time of RL-DCA in seconds. The results show that RL-DCA maintains competitive runtime performance, scaling efficiently even on large datasets such as NSL-KDD and UNSW15. Considering the considerable accuracy improvements achieved, the additional computational overhead introduced by the RL mechanism is considered acceptable.

The proposed RL-DCA model was evaluated against six baseline methods, namely GGA-DCA [14], GPSO-DCA [17], PCA-DCA, SVM-DCA, KNN, DT [14] and SVM [43] using nine benchmark datasets spanning biomedical, financial, behavioral, and cybersecurity domains. The evaluation employed multiple performance metrics, including accuracy (Table 5), precision (Table 6), specificity (Table 7), F1-score, and area under the ROC curve (AUC). For all metrics, the reported results represent mean performance values with corresponding standard deviations computed over multiple experimental runs. Tables 5–7 present detailed numerical comparisons of accuracy, precision, and specificity across all datasets. To improve readability and facilitate interpretation of the remaining performance metrics, the F1-score and AUC results are summarized using graphical representations rather than extensive numerical tables.

As shown in Tables 5–7, RL-DCA demonstrates consistently strong performance across mostbenchmark datasets. In terms of classification accuracy, RL-DCA achieves high detection rates on complex datasets such as NSL-KDD, DP, SP, and SPCHEES, outperforming or matching classical DCA variants and conventional machine learning classifiers. Similar trends are observed for precision and specificity, indicating that RL-DCA effectively balances correct positive detection while minimizing false alarms across diverse data characteristics.

Since the reward function in RL-DCA is label-guided during the learning phase, the reinforcement learning component introduces supervised information into the signal categorization process. To provide a fair and comprehensive evaluation, the experimental comparison includes not only traditional DCA variants but also conventional supervised machine learning classifiers such as SVM. This comparison allows the proposed method to be evaluated against both immune-inspired approaches and standard supervised learning baselines.

The comparative F1-score performance of all algorithms across the benchmark datasets is illustrated in Fig. 8. This figure highlights the overall balance between precision and recall achieved by each method. RL-DCA attains superior or competitive F1-scores on most datasets, particularly in cybersecurity and structured domains such as NSL-KDD, SP, DP, and UNSW15. These results demonstrate that the adaptive signal categorization enabled by reinforcement learning enhances the robustness of the DCA framework, especially in complex or high-dimensional environments where static signal mappings and traditional classifiers exhibit reduced effectiveness. RL-DCA demonstrates consistently competitive or superior performance across most datasets, highlighting its ability to balance precision and recall across diverse domains.

The discriminative capability of the evaluated algorithms is further examined using the AUC metric, as shown in Fig. 9. The graphical comparison reveals that RL-DCA maintains strong discrimination performance across most datasets, achieving high AUC values on NSL-KDD, DP, SP, and SPCHEES. Although RL-DCA exhibits relatively lower AUC performance on certain imbalanced financial datasets such as GCD and COIL2000, it remains competitive with existing DCA variants and conventional classifiers, confirming its ability to generalize across heterogeneous data distributions.

To further analyze the experimental results, a paired t-test was conducted to determine whether statistically significant differences exist between the proposed RL-DCA model and other signal categorization algorithms of the DCA framework (referred to as “Comparisons”). The statistical test was performed under a significance level of α = 0.05.

The following hypotheses were considered:H0:μRL−DCA=μComparisons H1:μRL−DCA≠μComparisonswhere μRL−DCA represents the mean classification accuracy obtained by the proposed RL-DCA model, and μComparisons represents the mean accuracy obtained by the baseline algorithms. Table 8 presents the t-test results based on classification accuracy between RL-DCA and other DCA-based signal categorization algorithms. With ten experimental runs, the degree of freedom is df = 9. At a significance level of α = 0.05, the corresponding critical t-value from the student’s t-distribution table is 2.262. If the calculated t-value is less than 2.262, the null hypothesis H0 is accepted, indicating that no statistically significant difference exists between the compared methods. Conversely, if the calculated t-value exceeds 2.262, the null hypothesis is rejected, indicating a statistically significant performance difference. As shown in Table 8, all calculated t-values are greater than 2.262, demonstrating that the proposed RL-DCA model achieves statistically significant improvements compared with the other signal categorization algorithms of the DCA framework across the evaluated benchmark datasets. These results confirm that the reinforcement learning based signal categorization mechanism effectively enhances the classification capability of the traditional DCA model.

In the traditional DCA, classification is performed using the MCAV, where samples are labeled as anomalous when their MCAV exceeds a predefined threshold θ. Previous studies commonly adopted a threshold value of θ=0.85, however, the influence of this parameter on detection performance requires further investigation. Therefore, a threshold sensitivity analysis was conducted by varying the MCAV threshold within the range:θ∈{0.50,0.65,0.75,0.85,0.95}

The analysis was performed on two representative datasets with different class distributions. The NSL-KDD dataset, which contains relatively balanced attack and normal samples, was selected to represent balanced scenarios. In contrast, the COIL2000 dataset, which is highly imbalanced, was used to evaluate the behavior of the algorithm under skewed class distributions.

Fig. 10 illustrates the F1-score obtained under different threshold values for both datasets. The results indicate that the proposed RL-DCA model maintains relatively stable performance across a wide range of threshold settings. For the NSL-KDD dataset, the performance remains consistently high across all evaluated thresholds, indicating strong separability between normal and attack traffic. For the COIL2000 dataset, the F1-score improves as the threshold increases from 0.50 to approximately 0.65 and remains relatively stable between 0.65 and 0.95.

Overall, the results demonstrate that RL-DCA is not overly sensitive to moderate variations in the MCAV threshold. Among the tested values, the threshold 0.85 achieves the highest or near-highest F1-score across the evaluated datasets, providing a balanced trade-off between anomaly detection capability and false positive control. Therefore, this threshold was selected for the subsequent experiments.

Furthermore, the threshold sensitivity analysis highlights a limitation of the traditional DCA classification mechanism: the final decision depends on a manually selected threshold parameter, which may not adapt optimally to different data distributions.

This section discusses the experimental results of the proposed RL-DCA model and explains their implications with respect to the research question. Instead of repeating the numerical values reported in Tables 5–7, the discussion focuses on the main performance patterns, the observed trade-offs, and the limitations identified across the evaluated datasets.

Overall, the results show that RL-DCA achieves the highest or near-highest performance on most datasets and evaluation metrics. The improvements are particularly clear in datasets with high dimensionality or complex feature relationships, such as NSL-KDD and UNSW15. In these datasets, traditional DCA variants that rely on static signal mappings or heuristic optimization methods tend to perform less effectively. The results suggest that learning the signal categorization directly from classification feedback helps the model generate more informative signals. As a result, RL-DCA improves the ability of the DCA framework to distinguish between normal and anomalous patterns across different application domains.

On structured datasets such as DP, RL-DCA achieves performance comparable to the best classical machine learning classifiers while showing lower variation across repeated runs. This indicates that the reinforcement learning component converges to a stable signal mapping strategy and reduces sensitivity to initialization. In contrast, conventional classifiers tend to show larger performance degradation on cybersecurity datasets. This behaviour is consistent with known challenges in these datasets, such as class imbalance, sparse features, and complex attack patterns.

Although RL-DCA improves performance in most cases, the results also reveal several dataset-dependent limitations. For example, the GCD and COIL2000 datasets show relatively high accuracy but lower F1-score and precision. This behaviour is mainly caused by class imbalance, where the large number of majority-class instances makes it more difficult to correctly identify rare events. A similar trade-off appears in the UNSW15 dataset. In this case, RL-DCA achieves strong accuracy, but slightly lower specificity and AUC compared with GGA-DCA and GPSO-DCA. This suggests that the model tends to prioritize detecting anomalous instances, which increases sensitivity but may also produce more false positives. Such behaviour can be useful in security applications where missing an attack is more critical than raising additional alarms. However, it also highlights the need for better balance between sensitivity and specificity when dealing with highly imbalanced datasets.

The results provide a clear answer to the research question of this study. The findings show that replacing fixed signal mappings with a learning-based signal categorization mechanism allows the DCA preprocessing stage to become more adaptive and less dependent on manual configuration. By learning signal assignments directly from classification outcomes, RL-DCA can adjust to different dataset characteristics and maintain competitive performance across multiple domains.

It should be noted that the reinforcement learning component uses class labels within the reward function to guide the learning of signal assignments. Therefore, the signal categorization stage becomes label guided during training. However, reinforcement learning is used only to optimize the preprocessing stage of the DCA, while the final anomaly decision is still performed by the original DCA context assessment and MCAV classification mechanism.

The findings also support the main contributions of this work. First, the proposed adaptive signal categorization mechanism improves anomaly detection performance in most evaluated datasets. Second, the experiments conducted on nine benchmark datasets demonstrate that the approach can generalize across different application areas. Third, the runtime analysis shows that the additional computational cost introduced by the learning mechanism remains moderate and acceptable compared with the benefits of removing manual signal design.

Despite these advantages, several limitations remain. The results show that the model is still affected by class imbalance in some datasets, particularly GCD, COIL2000, and UNSW15, where minority-class detection performance is reflected by lower F1-score, specificity, or AUC values. In addition, the current classification stage still relies on a fixed MCAV threshold. A single threshold value may not always be optimal for datasets with different class distributions. Future work will therefore focus on improving the classification stage of the DCA and exploring adaptive threshold strategies. A data-driven threshold adjustment mechanism could further improve the balance between detection sensitivity and false-positive rates.

This study proposes a new adaptive framework, called RL-DCA, which integrates reinforcement learning with the DCA to achieve dynamic signal categorization. Unlike traditional DCA approaches that rely on fixed rules, static feature transformations, or manually designed preprocessing, the proposed method learns the mapping between features and immunological signals directly from data. By using Q-learning, the RL agent iteratively updates signal assignments based on classification feedback, allowing the model to automatically discover more informative signal representations for anomaly detection.

The proposed framework preserves the immune-inspired structure of the original DCA while improving its preprocessing stage through an adaptive learning mechanism. In RL-DCA, the agent explores different signal assignments for PAMP, DS, and SS using a Q-table and an ε-greedy learning strategy. This dynamic process allows the model to adjust signal categorization according to the characteristics of the dataset, reducing dependence on manual configuration and improving generalization.

Experimental results on nine benchmark datasets from healthcare, finance, social behavior, spam detection, and cybersecurity domains demonstrate the effectiveness of the proposed approach. RL-DCA consistently achieves higher or competitive performance compared with existing DCA variants, including PCA-DCA, GGA-DCA, and GPSO-DCA, as well as classical machine learning classifiers such as SVM, KNN, and Decision Tree. These results show that adaptive signal categorization improves anomaly detection performance and enhances the robustness of the DCA framework across different application domains.

Future work will focus on extending the RL-DCA framework to support multi-class classification and online anomaly detection in streaming environments. Such extensions will enable the model to operate in real-time scenarios and further improve its applicability to practical problems such as network intrusion detection, fraud detection, and adaptive cybersecurity monitoring.

Overall, the results demonstrate that incorporating reinforcement learning into the DCA provides an effective way to transform static signal categorization into a data-driven and adaptive process, improving the flexibility and generalization capability of immune-inspired anomaly detection systems.