When inputting high-dimensional features into machine learning models, there will be some features in the model that are not related to the training target or the features are redundant [17], and these redundant features not only make the prediction results of the algorithm inaccurate, but also consume computing time and computer memory. There are many excellent algorithms for the selection of data features, such as: Lasso [18], Ridge, Principal Component Analysis [19], etc. This paper adopts Ridge algorithm which is faster in calculation and better in effect.

Enumerate the expression form of the ridge regression algorithm, in Eq. (1), μ is called zero parameter.

(1) β^Ridge=(X′X+μI)−1X′y

Take the value of the minimum penalty likelihood function as the estimated value of the regression coefficient, in Eq. (2), The penalty is Pλ(|B|)=λ∑j=1P|βj|m , m>0 , λ is the adjustment parameter.

(2) β∧=arg⁡minβ⁡(‖Y−βX‖2+Pλ|β|)

When m is 2, that is the Ridge penalty item. The expression form of Ridge regression can be obtained in Eq. (3).

(3) β^Ridge=arg⁡minβ⁡{∑i=1N(yi−β0−∑i=1Pxijβj)2+λ∑i=1Pβj2}

In Ridge regression, the high-dimensional data X has been centered and standardized, so that the size of the standardized ridge regression coefficients can be directly compared to judge the importance of high-dimensional features. The size of the regression coefficient reflects the importance of high-dimensional features to Y(y1,y2,...,yN) in the model. In this paper, the low-importance features are eliminated, and the high-importance input model is used for training.

Sparrow Search Algorithm (SSA) [20] is a group behavior algorithm inspired by the foraging behavior and anti-predation behavior of sparrows. Individuals in the population are divided into discoverers, followers and alerters according to the division of labor. The discoverers mainly provide foraging directions and areas for the entire population. The followers follow the discoverers to forage. The alerters are responsible for monitoring the foraging area. The optimization of the model parameters is achieved through the process of updating the position of the three.

Suppose the total number of sparrow individuals is n . The dimension of the variable to be optimized is d . Then the position of the population can be expressed as:

(4) X=[x1,1x1,2⋯x1,dx2,1x2,2⋯x2,d⋮⋮⋱⋮xn,1xn,2⋯xn,d]

The discoverers are responsible for guiding the population to find food and guiding the population to a safe location. The location update formula is as follows:

(5) Dxi,jt+1={xi,jt⋅exp⁡(−iα×T),R2>STxi,jt+Q⋅L,R2≥ST

In Eq. (5), t represents the current iteration number, xi,jt represents the position of the i sparrow in the j dimension at t iteration, i=1,2,⋯,n , j=1,2,⋯,d . α is a random number between 0 and 1. T is the maximum number of iterations. R2 is warning value between 0 and 1. ST is the preset safety threshold between 0.5 and 1. Q is a Gaussian distribution random number. L a is a matrix whose shape is (1∗d) and elements are all 1.

The followers will always follow the discoverers and compete for food resources in order to obtain more food resources. When the fitness of the discoverers is low, the followers will move to other positions, The follower’s position update formula is as follows:

(6) Fxi,jt+1={Q⋅exp⁡(xworstt−xi,jti2),i>n2xPt+1+|xi,jt−xPt+1|⋅A+⋅L,i≤n2

In Eq. (6), xworstt represents the position of the individual with the lowest fitness in t iteration. xPt+1 represents the position of the individual with the highest fitness in t iterations. A+=AT(AAT)−1 , A is shape of (1∗d) and each element of a is randomly preset to −1 or 1.

When the population realizes the danger, alerters will quickly make an anti-predation response. The update formula of the position of the alerters is as follows:

(7) xi,jt+1={xbestt+β⋅|xi,jt−xbestt|,fi≠fgxbestt+k⋅(xi,jt−xbestt|fi−fw|+ε),fi=fg

In Eq. (7), xbestt represents the global optimal position in t iterations. β is the step size control parameter, which is a Gaussian distribution random number with mean 0 and variance 1. k is a random number between −1 and 1. fi represents the fitness of the current individual. fg and fw represents the fitness of the current global best and worst individuals. ε is the smallest constant used to avoid the situation where the denominator is 0. It can be seen from this formula, fi≠fg means that the individual is at the periphery of the population and needs to constantly change positions to obtain higher fitness. fi=fg means that the individual at the center of the population is aware of the danger, and will continue to approach other nearby sparrows to stay away from the danger area.

Random Forest (RF) [21] is a flexible and easy-to-use machine learning algorithm. It uses multiple regression trees as a basis for training and incorporates the idea of bagging. In the tree training process, random feature selection is used to reduce the correlation between sample features, thereby solving the problem of overfitting of a single decision tree model, so that the model has a better prediction effect. The basic process sees Fig. 2 below.

From the flowchart, we can see the Random Forest generation process, and the details are presented below:

Random replacement sampling in training samples from the original training set, repeating S times;

In addition, the Random Forest incorporates the bootstrap idea when selecting samples, that is, sampling with replacement. The out-of-bag data generated by the bootstrap algorithm can be used to test the generalization ability of the model.

For the establishment of this model, this paper adopts predictive Random Forest, selects the optimal feature j and segmentation position s .

(8) Rm(j,s)=minj,s[minc1∑(yi−c1)2+minc2∑(yi−c2)2]

(9) R1(j,s)={x|x(j)≤s},R2(j,s)={x|x(j)>s}

In Eq. (8), ci is to divide the set sample, m=1,2 . Then this paper uses the selected (j,s) to divide the area to find the corresponding output value in Eq. (10).

(10) cm=1Nm∑xi∈Rm(j,s)yi

The smaller the cm is, the better the split performance of the selected feature and the segmentation point is. Continue to perform the above steps on the sub-regions to generate the optimal RF model.

Ridgecv model is trained using 5-fold cross-validation and training regression coefficients are sorted by the size. The larger the regression coefficient, the higher the influence of the molecular descriptor on the change of biological activity. The top 20 molecular descriptors are shown in the Fig. 3. In this paper, the top 20 to 100 drug compound variables with a high degree of influence are selected through the sorted characteristic regression coefficients and divide into features_num at intervals of 10 features. features_num = [20,30,40,50,60,70,80,90,100]. Divide the dataset by features_num and put the divided dataset into the model for training.

Like other intelligence optimization algorithms, SSA has the problem of easily falling into local optimum. In the later stage of the traditional SSA algorithm iteration, the position between the three sparrows will be updated in a small range near the optimal point, which is prone to the situation that the position update in a small range is stagnant. To solve this problem, this paper proposes an Improved Sparrow Search Algorithm(ISSA). We add dynamic adaptive weights to the sparrow finder position update formula to optimize the local exploration problem of the model. The formula is as follows:

(11) xi,jt+1={w⋅(xi,jt⋅exp⁡(−iα⋅T)),R2<STxi,jt+Q⋅L,R2≥ST

(12) w=sin⁡(π⋅t2⋅T+π)+b

In Eq. (12), T represents the maximum number of iterations. t represents the current iteration number. b represents the bias term. The Eq. (12) curve figure is as follows.

It can be seen from Fig. 4 that w is constantly changing as the number of iterations is updated. The sine function controls the range of w within the range of [−1,0], and adjusts the range of w by modifying the bias term b . This paper sets b to 1. Giving the discoverer a larger weight in the early stage of the algorithm iteration is conducive to the global search. In the later stage of the algorithm search, w decreases slowly, and there is sufficient time for local exploration. And because w has a small decrease, it can also make a relatively large weight in the later stage of the iteration, thereby speeding up the speed of local exploration. The involvement of this weight also speeds up the overall convergence speed of the algorithm to a certain extent.

In order to illustrate the convergence effect of the ISSA algorithm proposed in this paper, this paper uses the Rosenbrock function to conduct simulation experiments. When it is a binary function, as shown in Fig. 5. The Rastrigin formula is as follows:

(13) Rosenbrock=∑i=1N−1100(xi−1−xi2)2+(1−xi)2

In this paper, the independent fitness convergence training of the Rosenbrock function is performed, and the results are shown in Fig. 6. Within 500 iterations, SSA reached the convergence state at 27 iterations, and the convergence fitness value precision reached 10e−7, while ISSA reached the convergence state at 41 iterations, and the convergence fitness value precision reached 10e−23. The fitness convergence accuracy is much higher than that of SSA, which shows that the improved ISSA algorithm has much higher convergence fitness accuracy than the ordinary SSA algorithm.

Considering the time benefit and model accuracy of ISSA algorithm parameter optimization, too large an optimization range will lead to long model training time, and too small an optimization range will lose model accuracy, so it is necessary to limit the optimization range of parameters. Among the parameters required by the RandomForestRegressor function, n_estimators, max_depth, and max_features have a greater impact on the accuracy of the RF model. The parameters of max_depth and n_estimators are limited to the optimization range by using RF model training with MSE and R2 as the judgment criteria. The smaller the MSE, the higher the accuracy of the prediction of biological activity. The higher the R2 , the better the model, and the stronger the interpretation of the biological activity by the molecular descriptor features of the selected compound. As shown in Fig. 7. As the number of iterations increases, MSE keeps decreasing. After 175 iterations, MSE and R2 are almost stable with small fluctuations. This shows that when n_estimators = 175 or so, the model is stable and the accuracy reaches the highest level, so the range of n_estimators is set to [160,180]. Similarly, according to the curves shown in Figs. 7c and 7d, when max_depth = 16, the degree of fluctuation is small, and the range of max_depth is set to [10,30]. In order to satisfy the characteristics of RF model selection of features, this paper selects max_features, but considering that the number of molecular descriptors input in each training is different, the optimization range is not limited, and the ISSA algorithm directly performs the optimization operation. The parameter optimization range can be seen in Table 1.

This paper combines the ISSA with the RF algorithm to optimize the parameters of the RF model. First, the sparrow population is initialized. The the RF model is trained to calculate its fitness value. This paper sets the fitness function as the RMSE of RF model training and each iteration searches towards a position with a lower RMSE. Update the positions of discoverers, follower and alerters through the change of fitness value until the training termination condition is met. Finally, the ERα gene activity was predicted by the RF model with optimal parameters, and the model was tested by the validation set. The predicted model structures of the drug compound molecules that antagonize the activity of the ERα gene by the ISSA-RF model are shown in Fig. 8.