Most papers in the literature favor the use of statistical techniques (such as chi-square test, Fisher’s score, information gain, etc.) to select or reduce features due to their speed and low computational cost. For example, Cai et al. [10] proposed JOWMDroid, a feature-based approach to detect Android malware combining weight mapping optimization with classifier parameter optimization. Their idea is to utilize information gain to select the most relevant features from eight categories of features extracted from APK files. Next, three machine learning algorithms were used to calculate the initial weight for each feature and then map them to the final weights. The final step is to utilize the differential evolution algorithm to jointly optimize the parameters of the weighting function and the classifier.

Another work dealing with feature reduction for Android malware detection was proposed by Xie et al. [11]. In their approach, they employed a two-step feature selection method consisting of using InfoGain for an initial selection and applying the chi-square test for further reduction to remove redundant and irrelevant features. For the classification of Android malware, they used a stacking method of 5 base classifiers with the use of the Genetic Algorithm to optimize the hyperparameters of the model.

In [12], the authors introduced a malware detection framework called FAMD (Fast Android Malware Detector). It utilizes the FCBF (Fast Correlation-Based Filter) algorithm and the N-Gram technique to process the features and reduce their dimensionality. Then, the CatBoost classifier is utilized for malware classification. Experimental tests demonstrated a malware detection accuracy of 97.40% and a malware family classification accuracy of 97.38%.

In [7], the authors investigated the utility of the feature subset selection methods for Android malware detection by comparing and contrasting these methods along several factors. They utilized various learning algorithms to empirically evaluate the predictive accuracy of the feature subset selection methods and compare their predictive accuracy and execution times. The experiment findings demonstrated that feature selection is essential for increasing learning model accuracy and reducing runtime. The outcomes also illustrated that different learning algorithms perform differently when it comes to feature selection techniques, and no particular feature selection approach consistently outperforms the others.

Onwuzurike et al. [9] proposed a system that aims to detect Android malware from a behavioral perspective. It is based on abstracting the API calls executed by the application and building behavioral models through the use of Markov chains. The authors employed principal component analysis (PCA) to lower the dimensionality of the feature space and hence the computational and memory complexity of their system.

In [17], a novel method for selecting features called the selection of relevant attributes was developed by the authors in order to improve locally extracted features through the use of classical feature selectors (SAILS). This mechanism was constructed on top of conventional feature selection methods, including “mutual information”, “distinguishing feature selector”, and “Galavotti-Sebastiani-Simi”. It aimed to discover prominent system calls from Android applications.

Thiyagarajan et al. suggested in [8] a malware detection method based on the permissions requested by the application. Their idea was to minimize the data size by decreasing the number of permissions through a set of data reduction techniques (chi-square, permission ranking with a negative rate, support-based pruning, association-based pruning, and PCA). The reduced permissions were utilized for classifying the samples as malware or benign with a decision tree algorithm and categorizing the malware samples through the use of the K-means clustering algorithm.

In other cases, authors have preferred to benefit from the strength of neural networks for feature selection. For instance, Liu et al. [13] proposed an unsupervised feature learning method named “Subspace Based Restricted Boltzmann Machines” (SRBM) to reduce the data dimensionality in mobile malware detection. Their method includes searching for suitable subspaces over the entire feature set using a clustering method and learning the features in each feature subspace using Restricted Boltzmann Machines. Then, all learned features are concatenated to represent the original features in a lower dimension. The authors illustrated that their method outperforms other feature reduction methods including “RBM”, “Stacked Auto Encoder”, “Principal Components Analysis”, and “Agglomeration algorithms” in terms of clustering evaluation metrics.

In [14], Wu et al. proposed a feature reduction framework called DroidRL for Android malware detection. They used the Double Deep Q Network (DDQN) and the recurrent neural network (RNN) algorithms to select a valid subset of features over a larger range. They also attempted to determine the semantic relevance of features by using word embedding for the input features. The experiments conducted by the authors showed that their approach reduced the number of features from 1083 to 24 while maintaining high accuracy.

In [15], the authors proposed a method named DroidDomTree that searches the dominance tree of API calls included in the Android APK in order to find malicious modules. To efficiently select features, they developed a weighting scheme for assigning weights to each node in the dominance tree. This scheme aims to find the key modules that help in detecting malicious elements. In the experimental tests, the method presented detection rates between 98.1% and 99.3% when applied with eight machine learning classifiers. In another paper, Sharma et al. [16] have chosen a simple technique to reduce features based on calculating the importance of each feature using the feature_importances property of the Random Forest classifier, and then evaluating it using various machine learning algorithms.

Table 1 presents a comparative summary of the different works discussed in this section. These works denoted the importance of feature reduction methods in improving the performance of An- droid malware detection and classification systems. In the current paper, we tried to exploit a more innovative technique other than those described in the literature, that is, the Attention mechanism. Despite its importance, this mechanism is not yet well exploited in the field of Android malware detection. Among the few works done in this context, we can cite the paper by Wu et al. [18], in which the authors proposed a neural network approach to classify Android malware based on two layers: attention layer and multilayer perceptron (MLP). The attention layer is intended to learn feature weights, which can be thought of as scores of relevance between the features and classification outcomes. Then, the MLP maps the weighted features to classify the samples as benign or malicious. In this approach, the Attention-based feature importance is associated with an MLP classifier to classify Android samples. However, in our opinion, we believe that the classification process requires a more robust algorithm than a simple MLP. For this reason, we propose in this paper a new classification approach for android applications based on the Attention mechanism and the LightGBM algorithm. The Attention technique is utilized for determining the importance of features and reducing their dimensionality, while the LightGBM algorithm (with Bayesian optimization of hyperparameters) is utilized for performing an efficient classification of Android samples based on the set of selected features. To the best of our knowledge, the system that we propose in the current article is the first solution combining an Attention mechanism and a distributed gradient boosting framework like LightGBM to classify Android malware.

The feature extraction process is based on the technique utilized in the “CCCS-CIC-AndMal-2020” dataset [21] that we tested during the experimentation of the approach. This process consists in statically analyzing the Android application by reverse engineering its APK file. The extracted features contain a large set of information, including:

Activities: the user interfaces of the Android app.

A feature vector is then created from the numerical values of the collected features. The dimension of the vector equals 9504.

Each instance of the dataset includes 9504 features. This number of features is very large and can impact the performance of the classifier in terms of speed and accuracy. In our approach, we decided to utilize a selection method to reduce the number of features and select only those that help in enhancing the performance of the classifier. We started by eliminating features with null or empty values, which reduced the number of features to only 5911. Then, we applied a feature selection algorithm based on the Attention mechanism. This idea aimed to calculate the weight of each feature through the use of a neural network that contains an Attention layer that allows paying attention to the features that are more crucial than the others in the classification process. The features with the highest scores are selected to take part in the classification task. The following paragraph describes in detail how this step works.

The Attention mechanism is a neural network concept that has gained much popularity in recent years and allows paying more attention to certain parts when processing data. It utilizes a weighted vector to help the neural architecture understand the relationship between the input elements and the target and estimate which part of the data is more important than others for the task at hand. In our approach, we used the Attention mechanism to select the most important features that help enhance the performance of the classifier.

The general architecture of the Attention-based feature importance is presented in Fig. 2. It is a neural network architecture. The input to this network is a feature vector containing the initial features of the model. This vector is connected to an attention layer to compute the weighted values of each feature based on its importance to the classification process. In this way, a weighted feature vector is created, which is then connected to a Dense layer. The purpose of the Dense layer is to compute a “y” score in order to predict whether the instance is classified as malware or normal. After the training process of the neural attention network is complete, the weighted feature vector is utilized for determining which features have higher weighted values than the others. The features with higher weighted values mean that they are more crucial for the classification task.

In order to explain the feature extraction process, consider X as the set of feature vectors extracted from the Android samples and xi as the i-th sample of the set X denoted as (xi(1),xi(2),…,xi(j),…,xi(N)), where xi(j)(1≤j≤N) refers to the j-th feature of the i-th sample. Each feature vector xi is assigned a label value yi∈{0,1}, where 0 means that the sample is classified as normal and 1 means that it is classified as malware.

The implementation of the Attention layer is inspired by the work of Wu et al. [18]. It consists of using an adapted fully connected network and a SoftMax function to calculate the weight of each feature. We started by using the following equation to evaluate how closely the output feature matches the input feature:

(1)ei(j)=∑k=1Nxi(k)wkjwhere wkj represents a weighting parameter learned during the training of the fully connected network in the attention layer and ei(j) denotes the output of the fully connected network at the j-th position, which can be viewed as an association of a set of features with varying relevance to the input feature at position j. Thus, training the neural model allows the parameter wkj to be assigned a relevant value to indicate the relationship between the j-th input feature and other input features.

Next, in order to determine the weights of the input features at various places, we apply a SoftMax function to the output of the fully connected network. The output of the attention layer is a vector denoted by αi=(αi(1),αi(2),αi(3),…,αi(N)) calculated as follows:

(2)αi(j)=exp⁡(ei(j))∑k=1Nexp⁡(ei(k))where αi(j) denotes the weight of the j-th feature in the i-th sample and reflects its importance based on the classification results.

Then, in order to generate the weighted feature vector, we weight the input feature vector by the Attention vector αi as follows:

(3)ci=αixiTwhere ci denotes the weighted feature vector of the i-th sample.

Finally, we calculate the classification result yi by mapping the input vector ci into a binary prediction value.

Once the training process is complete, the Attention-based model assigns different weights to all features based on their contribution to the classification results. The features with higher weights indicate that they are more important for the relevance of the classification results, while the features with lower weights are less important. The weighted feature vector assists in selecting the top n features. These features are considered more crucial than the others, and our main classifier (LightGBM) is applied to them. The choice of the parameter n affects the classification results, so it is essential to select a suitable value. In our case, we tried to choose the best value for n based on the results of the experiments performed.

LightGBM [26] is a distributed gradient boosting algorithm released by Microsoft in 2017 for machine learning tasks. LightGBM is based on decision trees and can be used for several machine learning tasks such as classification, ranking and regression. It uses leaf-wise tree growth instead of the level-wise-tree growth which is widely used in several tree-based learning algorithms. LightGBM has outperformed several machine learning algorithms in multiple applications thanks to its characteristics, including efficiency, speed, and low memory consumption. In our model, LightGBM plays a role in classifying the Android samples as normal or malware after selecting the most important features from the previous step.

The implementation of LightGBM requires the use of a set of parameters called hyper-parameters, such as the number of leaves per tree, the maximum tree depth, and the learning rate. The mentioned parameters significantly affect how well the LightGBM algorithm performs and produces results. The choice of these hyper-parameters is crucial to achieving good results [27]. In our approach, a Bayesian optimization technique was utilized for identifying the best parameters for this model.

Bayesian optimization is a useful technique to optimize black-box functions that are expensive to evaluate [28,29]. The optimization problem can be formulated as follows:

(4)x∗=argmaxx∈χf(x)where x∗ represents the LightGBM model’s hyper-parameters that need to be optimized. The symbol χ denotes the search space for the hyper-parameters. The objective function is denoted by f(x) and indicates how well the LightGBM model performs given the selected hyper-parameters. Accuracy is the measurement criterion we chose to evaluate how well the objective function performed. Therefore, the goal of the optimization is to determine the collection of hyper-parameters x∗ with which the function f(x) performs best. The optimization procedure usually involves numerous iterations. The objective function yields an observed result yi=f(xi) that will be added to the historical set D=(x1,y1),…,(xi,yi) and utilized for updating the surrogate probability model in order to generate the next proposal. The optimization procedure for the LightGBM model is presented in Algorithm 1.

After completing the optimization process and selecting the best hyperparameters, the classification process of the LightGBM model starts to identify whether the Android sample should be classified as normal or malware.

In order to test and assess our approach, we utilized the CCCS-CIC-AndMal-2020 dataset [21]. This is a recent Android malware dataset created by the Canadian Institute for Cybersecurity (CIC). The dataset comprises 400 K Android applications (200 K are benign and 200 K are malware). The Android malware data is divided into the following 14 malware categories: Adware, Backdoor, FileInfector, No_Category, Potentially Unwanted Apps (PUA), Ransomware, Riskware, Scareware, Trojan, Banker Trojan, Dropper Trojan, SMS Trojan, Spy Trojan, and Zero-Day. Table 2 presents the number of families and samples for each of the 14 malware categories. In the Adware category, for example, there are 47,210 samples in the dataset that belong to this type of malware. Also in this category, we found 48 malware families, including Dowgin, Adflex, Airpush, Baiduprotect, etc. In the experimental tests, 12 malware categories were utilized, with the exception of No_Category and Zero Day, because the data in these categories was incomplete. The features of the dataset contain a lot of information, including:

Activities: the user interfaces of the Android app.

The hyperparameters of the LightGBM classifier greatly affect the quality of the classification results. The integration of Bayesian optimization into this approach has remarkably helped us in carefully selecting the right parameters. To accomplish this task, we have selected a set of values for each parameter of the LightGBM classifier (Learning rates, Number of iterations, Number of leaves, Bagging fraction, Feature fraction, Min data in leaf and Max depth). We then enter these values as input to Algorithm 1 described above. The role of this optimization algorithm is to try different configurations of these parameters in several iterations with the aim of obtaining an optimal configuration for the best classification accuracy. Table 3 describes the selected hyperparameters for this classification model.

The classification model used in this approach plays the role of a binary classifier whose goal is to classify Android samples into two classes: normal and malware. For this purpose, we grouped all the samples in the dataset belonging to the different malware families into a single class labeled “Malware”, and the other class labeled “Normal” is intended for benign samples. In order to train and assess the model, we divided the dataset as follows: 80% for training and 20% for testing. The experiments conducted in this work include 7 classification models (LightGBM, Random Forest, AdaBoost, Naive Bayes, Decision Tree, XGBoost and K-Nearest-Neighbor). For each model we performed 5 tests with the features of the dataset: applying the model (i) to the totality of the features without reduction (5911 features), (ii) to the 200 best features of the dataset selected thanks to our Attention-based feature importance method, (iii) to the 300 best features, (iv) to the 500 best features, and finally (v) to the 1000 best features. The aim is to evaluate the effectiveness of the Attention-based feature importance method in association with the LightGBM model (as well as the other machine learning algorithms) and to identify whether it can contribute to the improvement of the classification model performance. In order to evaluate the performed experimentations, we calculated 5 measures: accuracy, precision, recall, F1-score and false alarm rate (FAR) as well as the speed of training.

Table 4 illustrates the results of the experiments described above. Concerning LightGBM, the model applied to the top 300 features (in terms of importance) obtained the best results in the 5 evaluation measures with an accuracy of 0.987110, a precision of 0.988937, a recall of 0.982342, an F1-score of 0.985628, and a FAR of 0.008990. These scores slightly exceeded the results obtained from the LightGBM model applied to the totality of features without reduction (5911 features). The latter exhibited an accuracy of 0.986462, a precision of 0.988772, a recall of 0.981053, an F1-score of 0.984897, and a FAR of 0.009114. Furthermore, the LightGBM model with 300 features took less time to complete the training with 28.099 s in comparison with 80.043 s for the LightGBM model without the feature reduction. Fig. 3 presents the ROC curve and the confusion matrix of the LightGBM model with the top 300 features.

Concerning the other classification models, the application of our feature reduction technique based on the Attention mechanism showed better classification performance with all algorithms except Naive Bayes. For example, for Random Forest, the use of the 200 best features showed an accuracy equal to 0.986086, exceeding that with all the features of the dataset. For AdaBoost, similarly, the use of the top 200 and 300 features showed better performance in terms of accuracy, recall, F1-score and training time. Regarding the Decision Tree algorithm, we obtained the best results with the top 500 features. The selection of the first 300 features with the XGBoost model showed the best results in terms of accuracy, precision, recall, F1-score, FAR and training time. For the K-Nearest-Neighbor model, accuracy reached its maximum with the top 1000 features. The only case where the feature reduction technique failed to improve the classification performance was with the Naive Bayes algorithm.

Comparing all the results obtained from the different experiments, we can conclude that the LightGBM model with the top 300 features is the best performing model for the Android malware dataset. These comparative results confirm that the feature importance technique based on the Attention mechanism proposed in this paper has shown its ability to improve the classification results of the LightGBM model. The strength of this technique is that it can pay attention to features that play an important role in the classification results and ignore those that are less important. This technique has both reduced the training time and improved the classification results.

In this part, we are looking to evaluate our approach in the context of a multiclass classification. Our target is to classify the malware samples of the dataset into 12 malware categories as previously mentioned in the description of the dataset. This experiment consists in conducting a comparative evaluation among (i) the LightGBM model without feature reduction, (ii) the LightGBM model with selection of the best features as we did in the previous experiment, and (iii) the model described by the authors of the dataset paper (deep learning model). The benchmarking comprises 4 measures: precision, recall, and F1-score of each malware category as well as the overall model accuracy.

Table 5 shows the classification results of the different models tested in this comparison. This time the LightGBM model applied to the 500 best features provided the best results with an accuracy of 0.947711 in comparison with 0.946933 for LightGBM without feature reduction, 0.946349 for LightGBM with top 200 features, 0.946960 for LightGBM with top 300 features, 0.947461 for LightGBM with top 1000 features, and 0.93 for the deep learning model of the authors of the dataset.

These results demonstrated once again the effectiveness of the feature importance technique based on the Attention mechanism. This effectiveness is also associated with the choice of the value n that determines the number of important features to be selected. Choosing this number carefully is very essential to avoid ignoring some important features or including some features that have a negative effect on the classification results.

In recent years, a number of studies have been conducted with the aim of proposing relevant solutions for the identification and classification of malware in Android mobile environments. Table 6 compares some of these studies, including the one we propose in this paper. To achieve a fair and equitable comparison, we only selected the works that were evaluated using the CCCS-CIC-AndMal-2020 dataset. Different techniques were used in these works, including classical machine learning algorithms such as Random Forest and SVM as well as deep learning algorithms such as CNN and LSTM. The comparison showed that our approach outperformed the other works in terms of binary and multi-class classification accuracy.

The experimental results shown that our proposed approach is very effective. This approach made it possible to minimize the execution time of the model by effectively reducing the dimensionality of the data while improving the accuracy of the classification. The experimental findings have also shown that the LightGBM model performs better than other machine learning algorithms on this type of dataset.

Although the proposed approach has been shown to be effective with the CCCS-CIC-AndMal-2020 dataset and although we believe it can be properly adapted to other types of datasets, it would be more appropriate to test this approach with other Android malware samples. This will help to thoroughly investigate the performance of the Attention mechanism and verify its ability to efficiently analyze other types of features. Another limitation of this approach must be considered in our future work. The data collected in the CCCS-CIC-AndMal-2020 dataset contains static analysis of Android applications. This static analysis is very useful in identifying Android malware due to the large number of features we can extract from the application APK file. However, it is not able to detect complex Android malwares as their malicious actions can only be observed during execution. Therefore, it is important to extend the current approach to support hybrid application analysis that includes static data from the application and dynamic monitoring of its behavior at runtime.