There are generally two approaches for addressing multi-label classification issues: problem transformation and algorithmic adaptation. Problem transformation involves converting the multi-label classification task into multiple single-label classification tasks, allowing the use of established, robust single-label classification algorithms. For instance, the Binary Relevance (BR) method transforms the multi-label problem into a series of binary classification tasks [23]. The BR algorithm uses the entire dataset and trains a binary classifier for each label. When training a classifier for a specific label, only the information pertinent to that label is considered, and other labels are ignored. This approach, however, has a notable disadvantage: it overlooks label correlation. Consequently, label combinations that are dependent on one another may not be predicted in certain datasets. In contrast, the Classifier Chains (CC) algorithm mitigates these issues, delivering improved classification performance with the added benefit of a low time complexity of the BR algorithm [24]. The CC algorithm enhances the BR algorithm by transitioning from individually training binary classifiers to a sequential training process. However, a notable limitation of this method is the difficulty and complexity involved in detecting such correlations, rendering them non-trivial to ascertain. The algorithm adaptation method diverges from the problem transformation approach in its fundamental premise: rather than altering the multi-label classification problems or data, it enhances existing single-label classification algorithms to accommodate and resolve multi-label classification issues directly. The advantage of this strategy is that it circumvents the potential loss of information that can occur during the transformation process. Multi-Label Learning-K-Nearest Neighbor (ML-KNN), which is a multi-label iteration of the K-Nearest Neighbor (KNN) [25], serves as an example of this approach.

Feature selection [3] is one of the most important techniques used in machine learning to preprocess data, aimed at curating a subset of the most pertinent features from a vast pool to facilitate accurate classification. Essentially, feature selection can be regarded as a combinatorial optimization problem. It has been applied in different fields, including traveling thief problems [26], text feature selection [27], cancer classification [28], and network traffic identification [29]. Meanwhile, feature selection can be combined with different intelligent optimization algorithms to solve practical problems, such as the Greylag Goose Optimization Algorithm [30], the Hybrid Sine-Cosine Chimp Optimization Algorithm [31] and the WOA [32]. For multi-label data, the filtered subset of features enables the classifier to assign multiple labels to individual instances effectively. In feature selection, it is crucial to identify the optimal subset of features by finding those that are both highly correlated with labels and non-redundant. For example, Asilian Bidgoli et al. [33] proposed an enhanced iteration of the super-multi-objective binary Non-Dominated Sorting Genetic Algorithm-Ⅲ (NSGA-III) algorithm, which optimized the Hamming loss, feature subset size, feature label relevance, and computational complexity of features concurrently. Hashemi et al. [34] introduced the Multi-label Graph-based Feature Selection (MGFS). They first calculated the correlation distance between features and each class label, creating the Correlation Distance Matrix (CDM). They constructed a complete weighted feature map using the Euclidean distance applied to the CDM. Finally, the importance of the graph nodes was determined using the weighted PageRank algorithm of mean. Gonzalez-Lopez et al. [35] utilized two techniques, Euclidean Paradigm Maximization and Geometric Mean Maximization, to select features. The experiments affirmed that the proposed distributed approach effectively selected features with maximum relevance and minimum redundancy, while fully respecting predefined time constraints.

The ACO algorithm is a widely used optimization technique that imitates the foraging pattern of real-life ants. These ants maintain specific search patterns through a special chemical substance called pheromones. In recent years, this algorithm has seen an increased application in the realm of multi-label feature selection. Ants are a common sight in nature and follow specific patterns when searching for food. They maintain these patterns through the use of a special chemical called pheromones. In nature, ants aim to take the shortest path between their nest and their food source. As they travel, they deposit pheromones on the path, which gradually evaporate, leading to a decrease in their concentration along all pathways. Volatilization is a crucial process that aids in overcoming the challenge of being trapped in local minima and enhances exploration of the search space. Therefore, the concentration of pheromones can significantly influence the path that ants follow, making them more likely to choose paths with higher pheromone concentrations. As a result, ants gradually tend to traverse the shortest path. ACO algorithms frequently convert feature selection processes into the process of an ant colony wandering through a graph. The graph is generally composed of the correlation between features and labels, as well as the redundancy between features. The wandering process of the ant colony is primarily impacted by the pheromone update rule and the state transfer rule.

In the feature selection process of ACO, the choice of the subsequent feature usually hinges on the redundancy between it and the previous one. However, this static redundancy strategy will result in poor prediction accuracy for certain datasets.

To evaluate the redundancy of previously chosen features and potential ones, a dynamic computational strategy is proposed as in Eq. (1). Suppose that k−th an ant currently selects a feature set with feature number: (1)avgsim(Fj)=m−1∑Fz∈Fm−1ksim(Fj,Fz),Fj∈X−Fm−1kwhere Fj consists of unreachable features to be selected, X is the original set of features, (m−1) denotes the number of elements in the set Fm−1k, and sim(Fj,Fz) represents the similarity between two elements, Fj and Fz. Eq. (1) illustrates the average redundancy between the candidate features and the selected features, which will vary throughout the wandering process.

Label relevance is paramount in multi-label learning. To improve the performance of multi-label learning through feature selection, the impact of label correlation needs to be considered. For filtered methods, information theory techniques are often utilized to measure label correlation. The pairwise correlations between labels are computed using symmetric uncertainty in information theory. The formula for symmetric uncertainty is given as Eq. (2): (2)SU=MI(X;Y)H(X)+H(Y) (3)H(X)=−∑i=1np(xi)lg⁡p(xi) (4)H(X|Y)=−∑i=1n∑j=1mp(xi,yj)lg⁡(xi|yj) (5)MI(X;Y)=H(X)−H(X|Y)

Eq. (2) calculates the pairwise correlation between labels, serving as a heuristic factor for ACO. Here, H(x) defines the entropy definition of the random variable X. H(X|Y) denotes the conditional entropy, which measures the residual uncertainty of random variable X given Y and MI(X;Y), quantifies the information that one variable receives from the other and indicates the amount of information shared by the variables X and Y. The term p(xi) represents the probability of xi. p(xi,yj) is the joint probability of xi and yj.

The state transition rule is employed during runtime to construct ant paths: (6)Pik(t)={[τi(t)][η(Li,Lj)]β∑u∈Nik[τi(t)][η(Li,Lu)]β,∀i∈Nik,ifg>g00,otherwise (7)j=argmaxu∈N{[τi(t)][η1(Li,Lu)]β},if g<g0where τi(t) is the pheromone at time t, η(Li,Lj) is a heuristic factor which generally consists of the correlation of features, η(Li,Lj)=SU(Li,Lj), Nik is the set of unvisited labels of ant k, g0 is a constant used to balance exploration and exploitation, g is a random value. The parameter β distributed in [0,1] controls the trade-off between the pheromone value and the heuristic information. If β=0, the influence of the heuristic information is ignored. And the ant's path selection process is only related to the pheromone, making decisions based only on the previous action. And the pheromone is updated according to Eq. (8): (8)τi(t+1)=(1−ρ)τit+LC(i)∑j=1qLC(j)where q is the number of raw tag, τit and τi(t+1) are the pheromone values of tag i at t and (t+1) moments. Respectively, ρ is the pheromone evaporation parameter, and LC(i) is the counter corresponding to the tag i. The weight vector for label lb is attained at the conclusion of ant colony iteration, as demonstrated in Eq. (9): (9)lb=τ

In this subsection, feature relevancy and feature label relevancy are computed separately, and then the label relevancy obtained from the ACO algorithm is integrated into the feature label relevancy. Cosine similarity and Pearson correlation coefficient are used in calculating the correlations. Cosine similarity is a mathematical measure, as shown in Eq. (10): (10)cos⁡ine(A,B)=cos⁡(θ)=A⋅B||A||∗||B||where θ is the angle between the nonzero vectors A and B, ∥A∥ and ∥B∥ are their respective magnitude.

The Pearson correlation coefficient is one of the most popular similarity measures, calculated as shown in Eq. (11): (11)correlation(A,B)=|∑i=1n(Ai−A¯)(Bi−B¯)(∑i=1n(Ai−A¯)2)(∑i=1n(Bi−B¯)2)|where A and B are two n-dimensional (number of instances) features vectors, A¯ and B¯ are the average of the feature in the entire dataset. If they are linearly correlated, the value of the correlation coefficient will be closer to one, otherwise if they are independent then the value is zero.

To separately evaluate the redundancy between features and the correlation between features and class labels, the redundancy matrix fCorr and the incidence matrix fCorr for m×m and m×q dimensions are computed. Additionally, the label correlations obtained from ACO in the previous section are integrated into the correlation matrix. The integration process illustrated in Eq. (12): (12)flCorr=flCorr∗lb

The proposed approach weights the relevance between each feature and all class labels through labels, preserving and incorporating all feature-label relevance.

The construction of the subset is also the process of information accumulation, feedback, and iteration. The coding method used pheromones as feature weights, that is, it obtains feature ordering by sorting the pheromone values, thus obtaining the feature subset. During the process of wandering of ant colony, the characteristics of the next visit are determined by state transition rules, which encompass two strategies, a probabilistic transfer strategy, and a greedy transfer strategy: (13)Pik(t)={[τi(t)]α[η1(Fi,Fj)η2(Fj)]β∑u∈Nik[τi(t)]α[η1(Fi,Fu)η2(Fu)]β,∀i∈Nik,ifg>g00,otherwise (14)j=argmaxu∈N{[τi(t)]α[η1(Fi,Fu)η2(Fi,Fu)]β},ifg<g0 (15)η1(Fi,Fj)=1(flCorr(Fi,Fj))∗(1+avgsim(Fj)) (16)η2(Fj)=flCorr(Fj)

(17)g0=(gs−ge)×(1−kNF)r+gewhere Nik is the set of features that have not been visited by the ant k; τi(t) is the value of the pheromone of the feature i related to the pheromone in the t−th generation; the parameter g is a random variable uniformly distributed in [0,1], and g0∈[0,1] is an adaptive value with iteration; NF is the number of features that need to be selected by the ant in each iteration; r is the parameter controlling the decay rate. Through the probabilistic transfer rule defined in Eq. (13), the ant has the probability to select any feature that has not yet been visited; through the greedy rule in Eq. (14), the ant can select the feature with the lowest static redundancy and dynamic redundancy among unvisited features, as well as the highest feature-label relevance. The frequency with which an ant visits a feature will be stored in a vector called a feature counter (FC). Each time the ant traverses a feature, it adds one to the FC value corresponding to that feature. At the end of each iteration, Eq. (18) updates the amount of pheromone for each feature.

(18)τi(t+1)=(1−ρ)τit+FC(i)∑j=1mFC(j)where m is the number of features, τit and τi(t+1) are the pheromone values of tag i at t and (t+1) moments. Respectively, ρ is the pheromone evaporation parameter, and FC(i) is the counter corresponding to the tag i.

The ant colony selects features and accumulates pheromone values using the aforementioned strategy. The values in the pheromone correspond to the weights of the features. Algorithm 2 provides a detailed description of the proposed method. The specific flow chart is shown in Fig. 1. The computational complexity of the proposed method is analyzed. Firstly, the label weight lb is obtained by Algorithm 1. The computational complexity of uncertainty matrix SU is O(q2), and the main cycle is O(nCycle∗nAnt∗NF). Then, the feature label correlation matrix flCorr is obtained with computational complexity O(m2+mq). Therefore, the total computational complexity of algorithm 2 is O(q2+m2+mq+nCycle∗nAnt∗NF).

In this work, comprehensive experiments are carried out on authentic public datasets from various domains, including images, text, and biology, as shown in Table 1. It illustrates the number of instances, the number of features, the number of labels, cardinality (average number of labels for per instance), density, and domains for each data set.

The study presents an analysis and discussion of the seven publicly accessible multi-label datasets that were utilized, as well as the five filter methods that were compared, while the performance of all methods is evaluated using four multi-label categorical evaluation metrics. The paper compares the proposed method with MLACO [18], MCLS [36], ELA-Chi [37], PPT-Chi [37], and PPT-MI [37]. MLACO is based on the graph frame and adopts ACO. MCLS uses popular learning transformation and constraints on label space. PPT-CHI, PPT-MI, and ELA-Chi are classic problem-based transformation methods based on mutual information.

The parameters for the comparison methods have been specified. In the MLACO [18] algorithm, the maximum number of iterations is set to 40 (nCycle=40), with a pheromone decay rate of 0.1 (ρ=0.1) and the number of ants fixed at 25 (nAnt=25). The quantity of features selected by each ant in every iteration is defined as half the original number of features NF=m2. g0 the power parameters for the start value, termination value, and polynomial decay rate are configured as 1, 0, and 0.7, and the parameter β is established at 1. The settings of the proposed method are consistent with MLACO. The number of unique parameter neighbors in the Manifold-based Constraint Laplacian Score (MCLS) [36] is set to 5. The pruning threshold for Pruned Problem Transformation-Chi (PPT-Chi) [37] and Pruned Problem Transformation for Multi-Label (PPT-MI) [37] is set to 6. Entropy-based Label Assignment-Chi (ELA-Chi) [37] does not require to have a parameter set. The classifier of the experiment utilizes lazy learning-based ML-KNN [25] which is often used to access the performance of multi-label feature selection methods, with the number of neighbors set to 10. Four multi-label evaluation metrics are used: Hamming loss, One-error, Average-precision, and Micro-F1. Smaller values for Hamming loss and One-error are better, and larger values for Average-precision and Micro-F1 are better. For each dataset and algorithm, the average of 20 independent runs of these evaluation metrics is computed for all graphs and tables. Additionally, the dataset is partitioned using 5-fold cross-validation. The feature subset is obtained by sorting the features obtained by all methods. In order to evaluate the efficiency of feature selection methods, we first select the most representative d features through each method as the final feature subset, where d∈{10,20,30,40,50,60,70,80,90,100}. And then we classify these selected feature subsets. Since the feature dimension of the datasets Flags and Emotions are less than 100, we set them as d∈{2,4,6,8,10,12,16,18} and d∈{10,20,30,40,50,60,70} respectively on these two datasets. In order to further analyze the performance of each method, the first 30 features of each method are selected as the feature subset of multi-label learning.

Figs. 2–5 illustrate the performance variance of all methods on seven datasets with different sizes of feature subsets. Based on the results, the following observation can be deduced: From the results shown in Fig. 2a, it can be seen that the proposed method outperforms all other methods, achieving the best results. Furthermore, with an increasing number of features, the proposed method exhibits remarkable stability and consistently surpass the compared algorithms. It is observed from Fig. 2b highlights that the performance metrics, particularly one-error and average-precision, exhibit stability with the incremental increase in features. Despite the observed fluctuations and a decline in performance concerning hamming loss and micro-F1 measures, the proposed method consistently maintains a considerable superiority to comparative approaches. The observations from Fig. 3a reveal that all methods exhibit their highest variability on the Flags dataset, which can be attributed to its inherently low feature dimensionality. Additionally, it is discernible that an increase in the number of features does not consistently enhance the classification performance of algorithms. Analysis across the four evaluative metrics indicates that the performance of the proposed method falls short of the benchmark set by comparative methodologies exclusively when dealing with subsets of features with the limited size. Fig. 3b presents the results on the Emotions dataset. The overall trend is the same as the performance on the Scene dataset, with the difference being that the gap between the other methods and the proposed method is more pronounced, especially in hamming loss measures.

As depicted in Fig. 4a, the performance of the proposed method is observed to be inferior to that of PPT-MI for feature counts less than 60. Beyond the threshold of 60 features, the proposed method asserts its unequivocal superiority across various evaluation metrics. This observation underscores the nuanced efficacy of the proposed method in relation to feature quantity and its comparative advantage over a spectrum of evaluative criteria. According to the results in Fig. 4b, it displays that the proposed method does not match the MCLS in terms of one-error rate and average-precision. Nevertheless, it remains competitive relative to other evaluated methods, such as MLACO. Moreover, for most other evaluation metrics, the proposed method outperforms all compared counterparts. The noticeable decrease in the effectiveness of the proposed method, especially in the Yeast dataset, can likely be attributed to the minimal correlation between labels within the dataset. From the Fig. 5, it can be seen that the performance of the proposed method is monotonically improved with a relatively small feature subset size. Although it is not as stable as the other methods as the feature subset size increases, but remains highly competitive in terms of classification error rate and accuracy. From the overall trend, it can be seen that with the increasing size of feature subset, the classification error of the proposed method consistently decreases, the classification accuracy continues to rise. This improvement sets the proposed method apart from the compared methods, demonstrating its superior performance.

To further demonstrate the effectiveness of the proposed method, quantitative analysis is employed to demonstrate its performance. This involves the use of a feature subset composed of the top 30 features derived from the feature ranking outcomes. The rankings of each method on various datasets are presented in the Tables 2–5 with AvgRank signifying the mean ranking of the performance of each method on all datasets. The boldface in the table indicates the best performance, and the numbers in parentheses indicate the relative ranking of the six algorithms on each evaluation metric for each dataset. In the analysis presented across four tables, the proposed method consistently ranks the top position, with the only exception being its position on the Chess dataset. However, on other data sets, the proposed method significantly performs better than other methods. Additionally, the proposed method attains the highest AvgRank, which is an aggregate performance measure across all metrics. On the four evaluation metrics across the seven datasets, the method achieves average improvements of 6.81%, 18.01%, 6.33%, and 25.14%, respectively, in comparison to MLACO. These improvements indicates that dynamic redundancy and label correlation significantly enhances the search capabilities of the ACO algorithm. Therefore, the proposed method achieves highly competitive performance compared with all the comparing methods.

Based on the previous experimental results, the aim is to compare and analyze the performance of the implemented multi-label feature selection methods. The superiority of the proposed methods is further evaluated. In this subsection, two statistical tests commonly used for multi-label feature selection, namely Friedman test and Posterior Nemenyi test, are used to verify the significance of the variances among the methods.

Initially, the proposed method is tested to obtain multi-label feature selection. If the calculated value Friedman statistic FF is greater than the critical value, the assumption that all methods perform equally is rejected at the significance level a. During this testing phase, the critical value of 2.534 is set and found. Subsequently, the Friedman statistic and critical value for the four evaluation indexes are obtained, as presented in Table 6.

The results from Table 6 indicate that the Friedman statistics for all four evaluation indicators are significantly higher than the critical value, thus rejecting the null hypothesis made. The statistical differences between the methods are also compared visually using the post-hoc test Nemenyi test. The smaller the mean ranking is the better the characterization performance. The mean ranking difference between two methods is compared with the critical distance, if the mean ranking difference is greater than the critical distance then there is a significant difference between the two. The critical distance is indicated by a solid red line. CD (Critical Difference) is calculated as shown in Eq. (19):

(19)CD=qak(k+1)6Nwhere qa=3.031 is the parameter at significance level a of 0.05, then the critical distance CD=2.850 is calculated.

From Fig. 6, it can be seen that the performance difference between the implemented method and MCLS, ELA-CHI, and PPT-Chi is the most significant. Although the difference between PPT-MI and MLACO is not significant, within a critical distance, the proposed method evaluates first in the ranking, while MLACO is ranked back on average. Taken together, the proposed method has strong competitiveness and gains some performance improvement compared with MLACO. This verifies the effectiveness of incorporating dynamic redundancy and label correlation as correlation information.