Step I Data Acquisition: An extensive collection of five types of malicious samples including shell attacks, accessing sensitive directories, stealing communication protocol traffic, filling tunnel reserved words, and common network attacks, and using 0–5 tags to indicate classification methods, such as obtaining them from websites such as GitHub to build concealment tunnel sandbox, etc.

Step II Feature Database Construction: Collect feature words from the perspectives of shell commands, sensitive directories, communication protocol keywords, tunnel reserved words, and common network attack keywords, and build a characteristic database.

Step III Data Preprocessing: After three steps of hexadecimal decoding, common encryption method decoding, and text feature representation, the original ICMP hexadecimal is converted into a tensor that the model can learn.

Step IV Model Construction: Construction of ICMPTend covert tunnel detection model base on SVM classifier.

Step V Convert the data part of the ICMP traffic to be detected into a tensor that can represent features as model inputs. After the Step VI detection phase, the output of the model is a label, which can indicate the specific attack intention of the hidden tunnel or confirm that there is no covert tunnel.

Extracting features from the perspective of specific attack intentions of ICMP covert tunnels to identify hidden tunnels is essentially a multi-classification task of ICMP hidden tunnels based on attack intentions. In this paper, we mainly consider a large number of attack intents in covert tunnels, such as shell attacks, access to sensitive directories, stealing communication protocol traffic, filling tunnel reserved words, and common network attacks. There are five types of specific attack intentions. The benign samples come from normal ICMP traffic in the campus network of Beijing University of Posts and Telecommunications (BUPT), with a total of 1,000; the malicious samples come from the following sources: (1)

Sample ICMP tunnel traffic collected from sites such as GitHub, counting 442 entries.

A total of 4,462 malicious samples with malicious attack intent were obtained, and the number of samples with specific attack intent of 5 types is shown in Tab. 2. The samples are divided into the training set and testing set in the ratio of 7:3 for experiment.

Feature words are mainly composed of letters, numbers and special symbols, and different feature databases have different construction methods. Some commercial software constructs feature databases by directly querying the feature signature of malware [6]. In [18], the feature databases were constructed by directly cutting words. In our paper, we found that ICMP covert tunnel traffic has obvious and specific attack intentions in the data part, such as SHELL_ATTACKS, ACCESS_SENSITIVE_DIRS, etc. In order to extract keywords that can represent these attack intents, corresponding to the above attack intents, feature keywords from the perspective of shell commands, sensitive directories and their operations, communication protocols and related information, tunnel reserved words and common network attacks. The database is constructed as follows: (1)

SHELL_ATTACKS keywords: Shell attacks are essentially composed of various shell commands, and shell commands are divided into built-in commands and external commands. Therefore, this paper combines the malicious samples in the training set to collects 78 keywords of built-in shell commands and 33 common keywords. There are 111 external commands, such as the built-in command keyword “kill” for forcibly terminating the startup process and the external command keyword “sh” for starting a shell script, which constitute the keyword set for shell attacks.

The final set of these five types of keywords are combined into a feature database (FD) containing 637 unique feature words. The composition and description of the feature database are shown in Tab. 3.

Data preprocessing is the process of converting the hexadecimal data of ICMP data part into tensors that can be recognized by the machine learning model after hexadecimal decoding, string decoding, and text feature representation. The specific process is as follows:

Step 1: Hexadecimal Decode

The data field of the original ICMP traffic stores data in the form of a hexadecimal stream. In order to extract the text features of the transmitted content, the hexadecimal data needs to be decoded. The decoding function is shown in Eq. (1). (1)decode(HEX)=encodedString

As shown in Fig. 2, the ICMP covert tunnel is to transmit a shell attack statement-“docker pull nginx; L2Jpbi9zaCBzaGVsbC5zaA==”, but the actual after hexadecimal encoding, what is passed is “646f636b6572 ……”, where “docker” is encoded as “646f636b6572”, “pull” is encoded as “707566c6c”, and “nginx” is encoded as “6e67696e78”. Both the training model and the prediction stage need to encode the ICMP data.

Step 2: String Decode

With the continuous development of various encryption technologies, attackers use Uniform Resource Locator (URL) encoding [21], BASE64 encoding [22] and other encoding methods [23] to encode attack traffic to evade the detection of security detection system and thus hide their information. Attack intent, normal URL decoding and BASE64 decoding of text can effectively restore the original traffic and improve the detection efficiency. The decoding function is shown in Eq. (2). (2)decodebase64,URL(encodedString)=String

As shown in Fig. 3, the hexadecimal decoded text “docker pull nginx; L2Jpbi9zaCBzaGVsbC5 zaA ==” is decoded by BASE64 and becomes “docker pull nginx; /bin/sh shell.sh”, the key part of the shell attack “/bin/sh shell.sh” is restored.

In this paper, we use word frequency-inverse document frequency (TF-IDF) [24] for text feature representation, through which the text content can be converted into a feature-representing tensor, which can be input into the model for learning. TF-IDF is a statistical method used to evaluate the importance of a word to a sample in the training set. The core idea is that the importance of a word increases in proportion to the number of times it appears in the sample, but it is not in the sample. The number of occurrences is inversely proportional to the frequency in the training set. The algorithm flow of text feature representation is as follows. The text feature representation of the example ICMP covert tunnel traffic is shown in Fig. 4. 1)

Each word in the FD is numbered, which corresponds to the index in the feature vector with a latitude size of, m=|FD|v→=(v0→,v1→,⋯,vm−1→)vi→=0 for i=0,1,…,m (637 in this paper).

vi in Eq. (3) represents the mapping of the corresponding numbered words in the feature database. The corresponding TF-IDF value is then calculated for this flow. v→=(v0→,v1→,⋯,fj,i,⋯,vm−1→)

In order to clarify the specific attack intentions of ICMP covert tunnel, ICMP traffic flow needs to be classified into multiple categories. There are six categories of multi-classification, namely, normal traffic, shell attack, sensitive directory access, communication protocol traffic stealing, filling tunnel reserved words, and common network attacks as shown in the aforementioned Tab. 1. ICMPTend model construction process is shown in Fig. 5, which is divided into two phases: training and testing. In the training phase, the input is pre-labeled benign and malicious sample data in the training set, and the output is a multi-class model with attack intention prediction capabilities. In the prediction phase, the input is the actual data part of the unlabeled ICMP traffic, and the output is the possible attack intent of the traffic. The purpose is to obtain a multi-classification model ICMPTend, which can predict the attack intention of ICMP traffic in real situations.

ICMPTend uses SVM as the classification algorithm. SVM has maintained its unique advantage in solving classification problems for small and medium samples, high-dimensional, and linearly indistinguishable datasets. The ICMP covert tunnel dataset constructed in this paper happens to be small-sample, high-dimensional, and linearly indistinguishable, so theoretically SVM is suitable for the situation in this paper.

The ICMPTend receives the data part of ICMP traffic as input, and outputs the label of the category to which the traffic belongs. The label corresponds to the specific attack intent. Suppose the training set contains the data part of ICMP traffic. {(d1,y1),(d2,y2),…,(dT,yT)},dt(t=1,2,…,T) denotes the data portion of a flow. yt∈{0,1,…,5} indicates the label of ICMP traffic, 0 to 5 indicate normal traffic class, shell attack class, sensitive directory access class, communication protocol traffic stealing class, filling tunnel reserved word class, and common network attack class, respectively. The model first needs to obtain the feature representation of the data part, i.e., dt→vt,vt∈Rn. Next, a classification function needs to be fitted, assuming that the predicted label is y^t. Then y^t=f(vt) gives the result that the data portion of an ICMP traffic is predicted to be a certain class. P(y^t=f(vt)),t=0,1,…,5 indicates the probability that the data part is predicted to be a certain category, Finally, the prediction label corresponding to max(P(y^t=f(vt))),t=0,1,…,5 is selected as the final prediction output, which corresponds to the specific covert tunnel attack intent. During training, the model needs to minimize the loss function ∑1T⁡Ly^t≠yt over the entire training set, and L denotes the function that calculates the loss in case of classification errors.

In order to answer Q1 and Q2, we build a feature dictionary based on the attack intent of shell attacks, access to sensitive directories, stealing communication protocol traffic, filling tunnel reserved words, and common network attacks, and construct feature vectors based on the feature dictionary as input to the model. In order to verify the effectiveness of the feature construction method in this paper, the feature vector is input into a separate model for training and prediction, and the effectiveness of the detection is evaluated.

The results of the comparison experiments using SVM, logistic regression (LR), and Naive Bayesian (NB) models are shown in Tab. 5. It can be found that inputting the feature vectors constructed by the feature construction method in this paper into multiple models for testing has achieved good results in terms of accuracy, recall, F1 score, and accuracy. Even in the NB model, which has the worst combined effect, the precision, recall, F1 score and accuracy reach at least 0.86, 0.80, 0.83, and 0.89, respectively. This indicates that the method of constructing features in this paper is effective.

Comparing the detection effect of the SVM model with that of the LR and NB models, the SVM model is also superior to other models in all aspects. This show that the SVM model is more appropriate in discerning the specific attack intent of the covert tunnel. The feature construction method in this paper is effective and SVM can be used as a classifier for covert tunnel specific attack intent detection [26,27].

Observation 1: (1) The keywords of shell attacks, access to sensitive directories, stealing communication protocol traffic, filling tunnel reserved words, and common network attacks are often found in the data portion of malicious traffic in ICMP covert tunnels, so extracting these types of keywords to construct features would be effective; (2) The SVM is suitable for high-dimensional, linearly indistinguishable data, and ICMP traffic happens to be high-dimensional and linearly indistinguishable, so SVM is more appropriate than other machine learning models.

The general method of using keywords for classification in machine learning is to use the collection of all words in the training set after sample word separation to form a vocabulary to form a large-dimensional vocabulary, which often requires further dimensionality reduction. In order to verify the effectiveness of this dimensionality reduction method, a comparative experiment before and after dimensionality reduction was constructed. The pre-dimensionalization experiment is to split the data part of each traffic in the training set with spaces and special symbols, and use the set of all words obtained after splitting as a feature dictionary, which contains about 30,000 words. The dimensionality reduction experiment adopts the feature dictionary construction method of this article. The other experimental steps are the same.

As shown in Tab. 6, after dimensionality reduction, the precision, recall, and F1 scores of each category are higher than those before dimensionality reduction by at least 0.02, 0.01, and 0.02, the accuracy rate is improved by 0.05, and the training time after dimensionality reduction is also reduced to about 1/8 of that before dimensionality reduction. This show that the dimensionality reduction method used in this paper is effective, not only improves the training speed, but also improves the evaluation indicators of each category. Before and after dimensionality reduction, the accuracy rate can be improved, and the time efficiency can be significantly improved.

In terms of CPU resource consumption, as shown in Fig. 6, after dimensionality reduction, the CPU usage during training is significantly lower than before. Only a single core is required to meet the training requirements, while before dimensionality reduction, an eight-core CPU is required to meet the training requirements. This shows that the dimensionality reduction method used in this paper can effectively reduce the consumption of CPU resources.

In terms of memory resource consumption, as shown in Fig. 7, the memory utilization during training after dimensionality reduction is 45.4% lower than that before dimensionality reduction. This shows that the dimensionality reduction method used in this paper can make more effective use of memory resources.

Observation 2: Using the feature lexicon constructed in this paper, the dimension of the final feature vector is reduced from more than 30,000 to 637 compared to the dictionary constructed after direct word segmentation. Therefore, dimensionality reduction can undoubtedly improve the efficiency of the model. At the same time, choosing an appropriate dimensionality reduction method can filter out noise or irrelevant information, thereby helping the model to better learn the main features, and the model can obtain better detection results.

In order to clarify the gap of detection capability between attack intention detection based on multi classification and anomaly detection based on binary classification, we aggregate all types of malicious samples into malicious samples, and conducts anomaly detection experiments based on two classifications. And compare the results of the anomaly detection experiment with the results of the attack intention detection experiment.

The experimental results are shown in Tab. 7. The macro-average accuracy, macro-average recall, and accuracy of attack intent detection based on multi-classification are 0.05, 0.06, and 0.05 lower than that of anomaly detection based on two-classification, and the gap is controlled within 0.1. At the same time, the lowest accuracy rate and the lowest recall rate in the multi-classification also reached 0.89 and 0.86 respectively, achieving a better multi-classification effect.

Observation 3: Compared with anomaly detection based on binary classification, attack intention detection based on multi-classification has better detection capabilities.