Weather prediction is an important research direction [12–15]. The main tasks include rainfall prediction, temperature prediction, air pollution prediction, etc. Recent research on PM2.5 concentration prediction is generally based on deep learning methods, which convert the PM2.5 concentration prediction problem into a data mining problem. Therefore, it is necessary to introduce the PM2.5 prediction models using deep learning methods. With the rapid growth of air pollution data, deep learning methods have been further applied in PM2.5 prediction and proven effective prediction performance. In order to capture the complex temporal characteristics contained in the air pollutant data series, recurrent neural networks such as Long Short-Term Memory (LSTM) [4] have been widely used in PM2.5 prediction and achieved good performance [5,6]. These studies show that LSTM can achieve better results than traditional machine learning methods when modeling long-term sequence prediction problems. However, these methods only consider the temporal characteristics of PM2.5 concentration series, and lack the utilization of spatial characteristics that reflect the correlation between different monitoring stations.

However, the aforementioned deep learning methods usually only consider temporal characteristics of PM2.5 concentration data. In the real world, the PM2.5 concentration data of different regions are spatially interrelated, and PM2.5 pollutants between areas could be transmitted and diffused to each other. In order to learn the spatial correlation, convolutional neural networks are introduced to extract the spatial characteristics of the time series of PM2.5 pollutants [16,17]. Many research works combine convolutional neural networks (CNN) and LSTM to learn the temporal and spatial dependence of urban PM2.5 concentration [7–9]. Attention ConvLSTM Encoder-Forecaster (AttEF) [18] integrates the attention mechanism into ConvLSTM encoder-forecaster to solve the loss of important spatiotemporal information, which has achieved good performance in precipitation nowcasting. These methods combine CNN and LSTM to form a spatiotemporal prediction model for PM2.5 concentration. But convolutional neural networks can only be used to process data in Euclidean space, and there are still shortcomings in capturing spatial features. In general, deep learning methods have achieved good results in PM2.5 concentration prediction. How to learn the spatial dependence of PM2.5 concentration data between different monitored cities needs further research. In addition, the influence of meteorological factors needs to be considered in the forecasting process.

Recently, graph neural networks have received increasing attention from researchers due to their ability to learn graph structure information, representing complex non-Euclidean spatial information [19]. Considering the non-Euclidean distribution among air monitoring stations in different cities, only using convolutional neural networks is not enough to capture complex spatial information. Therefore, the graph neural network (GNN) model based on the graph structure can better learn the spatial correlation between PM2.5 monitoring concentration data in different cities. PM2.5-GNN integrates domain knowledge into graph-structured data to explicitly model the long-term spatiotemporal dependence in the PM2.5 forecasting process [20]. In addition to GNN, graph convolutional neural networks also play an essential role in air pollutant prediction. In the GLSTM model [21], the graph convolutional network is combined with LSTM to introduce spatiotemporal information into PM2.5 concentration prediction. Hierarchical graph convolutional networks are adopted to model air pollutants’ diffusion process more effectively in air quality prediction [22]. GCLSTM proposes a hybrid model combining graph convolutional network and LSTM to model and predict the continuous changes of PM2.5 concentration [23]. The above methods use graph neural networks to learn node features on a fixed graph, and cannot dynamically learn the weights of edges representing correlations between nodes. Graph convolutional networks or graph neural networks obtain node representations by aggregating the proximity information of target nodes. However, the relationship between different PM2.5 monitoring sites is not just a connection of 0 or 1. The spread of air pollution between cities is also closely related to meteorological conditions. The connection relationship between nodes needs to be more optimized to a more accurate value to obtain a richer expression. Therefore, we use graph attention network to learn the spatial relationship of PM2.5 concentration data in this paper. Compared with other types of graph networks, graph attention networks use the attention mechanism to learn the relative importance of different neighbor nodes. This method can effectively improve the expressive ability of the graph network.

For the air pollutant prediction problem, it is vital to learn the spatial characteristics and dependencies contained in the original data. From a spatial perspective, neighboring cities generally have similar air pollution conditions, and air pollutants could spread and affect each other between neighboring cities. The current research work either ignores the mutual influence between different city nodes or introduces prior knowledge to establish node correlations. In GAT-GRU prediction model, the graph attention network is used to capture the spatial dependence of PM2.5 concentration monitoring data. Compared with graph convolutional network, GAT can assign different weights to the neighbor nodes of the target node according to their importance.

Unlike the general GAT-based forecasting model, the input h of the GAT layer in the GAT-GRU model is obtained by combining two parts: PM2.5 concentration data x for a period of time, and the domain knowledge s reflecting the city’s meteorological conditions during this period. Meteorological conditions are closely related to the generation and spread of air pollutions. Therefore, these factors need to be fully considered in the PM2.5 concentration prediction process.

The GAT layer in the prediction model is mainly composed of two parts: (1) calculate the attention coefficient. (2) aggregate features of neighbor nodes to get node representation. Fig. 3 shows the calculation process of the graph attention mechanism. First of all, the attention coefficient represents the importance of neighboring nodes to the target node. The following formula can calculate the attention coefficient: Figure 2 (2)eij=attention(Whi,Whj)=attention(W[xi,si],W[xj,sj])

eij represents the attention coefficient between neighboring node i and target node j. The value of the attention coefficient reflects the strength of the relationship between the two nodes. To make the attention coefficient comparable between all nodes, the softmax function is used to normalize the attention coefficient, and the formula is as follows: (3)αij=softmax(eij)=exp⁡(eij)∑k∈Ni⁡exp⁡(eik)

The calculation formula of the final result of the attention coefficient is shown in formula (4). The specific attention operation in GAT is to splicing the feature vectors of two nodes together, and then doing an inner product with the weight vector a→T. (4)αij=exp(LeakyReLU(a→T[W[x→i,s→i]||W[x→j,s→j]]))∑k∈Ni⁡exp(LeakyReLU(a→T[W[x→i,s→i]||W[x→k,s→k]])),where Ni represents all neighboring nodes of node i, || represents the concatenation operation, LeakyReLU denotes the nonlinear activation function. After calculating the weight of each city node’s neighboring city nodes, the output of the GAT layer can be obtained by aggregating the information of the neighboring nodes. (5)hi′=σ(∑j∈Ni⁡αijWhj),where σ represents the activation function, hi′ is the node feature vector obtained calculated by the attention mechanism. In addition to a separate attention mechanism, multi-head attention can ensure the stability of the attention mechanism. Multi-head attention allows the model to have the ability to learn relevant information from different subspaces. K represents the number of attention layers in the multi-head attention mechanism. (6)hi′=σ(1K∑k=1K∑j∈NiαijkWkhj)

In order to enhance the stability of the results, the multi-head attention mechanism is used in the GAT layer. As shown in formula (6), the results of K independent attention operations are aggregated to obtain the final feature representation.

The PM2.5 concentration data recorded by the air pollution monitoring stations is stored in the form of time series. The time series of PM2.5 concentrations have remarkable features such as periodicity, proximity and trend. Periodicity means that the PM2.5 concentration fluctuates cyclically over a longer period of time. Proximity means that the PM2.5 concentration values are closer when the time period is similar. Trend means that the change of PM2.5 concentration has a trend of increase or decrease in a period of time. Therefore, it is very important to model the temporal dependence of PM2.5 concentration data. With the development of deep learning, the recurrent neural network has become an effective method in time series modeling. Many PM2.5 prediction methods use LSTM as the basic model for learning temporal dependencies [7–9]. This paper uses a variant of the recurrent neural network called gated recurrent unit (GRU) to process air pollution data. Fig. 3 shows the overall structure of the gated recurrent unit.

Gated recurrent unit contains two gates: reset gate rt and update gate zt. The reset gate rt determines the combination of the new input information xt and the previous memory state ht−1. The update gate zt determines the amount of past state information ht−1 that continues to be saved in the current state  ht. Fig. 3 shows the internal structure of the GRU and the connection between the update gate and the reset gate. The following is the calculation formula of GRU: (7)zt=σ(Wz⋅[ht−1,xt]) (8)rt=σ(Wr⋅[ht−1,xt])

zt and rt represent the output of reset gate and update gate, Wz and Wr represent learnable parameters. xt represents the input data at the current time t. In the GAT-GRU prediction model, xt includes PM2.5 concentration data and meteorological characteristic data. (9)h~t=tanh⁡(W⋅[rt∗ht−1,xt]) (10)ht=(1−zt)∗ht−1+zt∗h~t

ht represents the output at time t, ht−1 and xt represent the output of the previous time t−1 and the input of this time t.

In order to model the spatiotemporal dependence of PM2.5 concentration sequence, this paper proposes the GAT-GRU model composed of graph attention mechanism and gated recurrent unit. The input of the GAT-GRU model includes the node features matrix S∈RN×s, the PM2.5 concentration data X∈RN×1 and the adjacency matrix A∈RN×N. N represents the number of cities with PM2.5 concentration monitoring data. The node features matrix St∈RN×s represents the meteorological knowledge. Fig. 4 shows the overall architecture of the GAT-GRU model.

In this paper, the PM2.5 concentration data Xt and additional factors St at time t are used as the input of the model, and the output predicted value Xt+1′ of the GAT-GRU model and the additional factors St+1 at the next moment are used as the input for the next step to continue the prediction. The basic unit of the GAT-GRU prediction model is the GAT-GRU cell. Each GAT-GRU cell mainly consists of three parts: the GAT layer, the GRU layer and fully connected layer. The proposed model predicts the future PM2.5 concentration value in a rolling manner. Formula (11) to formula (17) represent the calculation process of the GAT-GRU model. (11)Xt′=[fc(Xt),St],where fc() represents the fully connected layer, which changes the dimension of input or output. Xt′ denotes the concatenation of the input concentration data Xt and additional factors St. In this part, the PM2.5 concentration data and meteorological feature data at the current time t are processed as the input of the graph attention network. Meteorological feature data, as important factors related to PM2.5 concentration, provide important information for prediction models. (12)g(Xt′)=[f(A,Xt′),Xt′],

f(A,Xt′) represents the graph attention network layer. A is an adjacency matrix representing the spatial connection between different cities. Then the output of GAT layer is used as the input of GRU to obtain temporal dependence. (13)zt=σ(Wz⋅[g(Xt′),ht−1]) (14)rt=σ(Wr⋅[g(Xt′),ht−1]) (15)h~t=tanh⁡(W⋅[rt∗ht−1,g(Xt′)]) (16)ht=(1−zt)∗ht−1+zt∗h~t,the above four formulas are the calculation formulas of GRU. ht−1 is the output of the GAT-GRU cell at the last time step t−1, which is used as the previous state’s input at time t. g(Xt′) is the input of GRU model. (17)Xt+1′=fc(ht),where Xt+1′ denotes the predicted results of PM2.5 concentration in the next time t+1. In the GAT-GRU model, a rolling prediction model is used to predict the PM2.5 concentration after T time steps. Xt+1′ and St+1 are used together as the input of prediction cell for the next time step.

We summarize the learning process of the GAT-GRU prediction model in Algorithm 1 below.

We conduct experiments on a GPU server with a single 2080ti which has 11G video memory. We use PyTorch as the deep learning running framework of the server. The initial hidden state h0 of GRU is initialized with a zero tensor. The input data of all experiments is 1 step (3 h), and the output prediction result is 24 steps (72 h). Therefore, it means that 3 h of historical data is used in the experiment to implement the prediction of the PM2.5 concentration value after 72 h in the future. All models are trained for 100 epochs. The learning rate is 0.0005, and the batchsize is set to 64.

A total of 5 evaluation indicators are used in the experiment to evaluate the predictive performance of the GAT-GRU model. These indicators can be divided into two categories. The commonly used indicators to measure prediction accuracy in prediction models are mean absolute error (MAE) and root mean square error (RMSE). The other is the commonly used accuracy evaluation indicators in meteorology: critical success index (CSI), false alarm rate (FAR) and probability of detection (POD).

Specifically, the calculation methods of MAE and RMSE are as follows: (18)MAE(x,x^)=1|Ω|∑iεΩ⁡|xi−x^i| (19)RMSE(x,x^)=1|Ω|∑iεΩ⁡(xi−x^i)2,where x^i and xi respectively represent the predicted value and the ground truth, Ω denotes the total number of data samples.

Following PM2.5-GNN, the calculation methods of three meteorological evaluation indicators are as follows: (20)CSI=hitshits+misses+falsealarms (21)POD=hitshits+misses (22)FAR=falsealarmshits+falsealarms,where hits means the predicted value and the true value are both 1, and misses indicates that the predicted value is 0 while the true value is 1, and falsealarms means that the predicted value is 1 and the true value is 0.

As shown in Section 3, the dataset we used in the experiment is KnowAir, which contains the PM2.5 concentration data and meteorological attribute data of 184 cities in China collected from the real world from January 1, 2015 to December 31, 2018. From this dataset, we obtain two datasets (Dataset 1 and Dataset 2) for experiments. Dataset 1 represents the air pollution prediction under normal circumstances, and Dataset 2 selects the PM2.5 data in winter with severe air pollution for prediction. Tab. 2 shows the segmentation method of the two datasets in the experiment.

In the PM2.5 concentration prediction experiment, the following models are used as baselines compared with the proposed GAT-GRU model. For the fairness of the comparison of experimental results, we add meteorological knowledge as part of the model input when conducting experiments on all baselines. (1)

MLP [26]: MLP is a classic multi-layer neural network model, which generally consists of an input layer, a hidden layer and an output layer. The representation of the node is used as the input of the multi-layer perceptron to obtain the prediction result finally.

Experiment 1: comparison with baselines. Tabs. 3 and 4 show the PM2.5 concentration prediction performance of our proposed method and other methods used as baselines. As mentioned above, we conduct experiments on two real-world datasets. Experimental results include MAE, RMSE, CSI, POD and FAR. All the best experimental results are highlighted in bold. From the results in the table, we can see that the prediction performance of the methods that use the recurrent neural network to model the time characteristics is better than the MLP model. Furthermore, the predictive models that use graph structure to model spatial dependence have better performance, such as GC-LSTM, PM2.5-GNN and the proposed GAT-GRU model. These results mean that it is vital to model spatiotemporal dependence for PM2.5 prediction problem. Both GAT-GRU and PM2.5-GNN introduce new information and knowledge, such as meteorological attributes of cities and edge attributes obtained from wind speed and direction between city nodes. The experimental results prove that the introduction of meteorological knowledge can effectively improve the accuracy of prediction. In the experimental results of Dataset 1, the MAE and RMSE of the GAT-GRU model are 34.56 and 42.79, which are better than the results of other models. In the experimental results of Dataset 2, the results of GAT-GRU are also basically stronger than other models. Compared with the graph neural network in GC-LSTM and PM2.5-GNN, the graph attention network can effectively model the dynamic connection between monitoring nodes, especially under the condition of the integration of meteorological knowledge. In the experiments of the two datasets, the POD (Probability of Detection) indicator of PM2.5-GNN is better than the GAT-GRU model. Since PM2.5-GNN utilizes edge attributes composed of wind speed and wind direction between city nodes, more accurate PM2.5 propagation information can effectively enhance the probability of detection. In general, compared with other methods, the proposed GAT-GRU model achieves better prediction performance.

Experiment 2: The influence of meteorological knowledge. Tab. 5 shows the results of ablation experiments on whether meteorological knowledge is incorporated in the GAT-GRU model. Taking the experimental results on Dataset 1 as an example, the predicted MAE and RMSE of the GAT-GRU model (without meteorological knowledge) are 42.21 and 50.61. With the use of meteorological knowledge, MAE and RMSE are reduced by 7.65 and 7.82 respectively. The experimental results show that the use of meteorological knowledge effectively improves the results of PM2.5 concentration prediction. In addition, the MAE and RMSE of the ablation experiment on Dataset 2 have a more significant decrease, which proves that the result of the use of meteorological knowledge on Dataset 2 is better than that of Dataset 1.

Experiment 3: multi-head attention mechanism. Tab. 6 shows the experimental results of ablation for the multi-head attention mechanism. In the spatiotemporal prediction problem, we mainly consider two indicators, MAE and RMSE. As can be seen from the table, the best prediction results on Dataset 1 and Dataset 2 can be obtained when the numbers of multi-head attention mechanisms are 2 and 6. The results of Experiment 3 demonstrate the effectiveness of the multi-head attention mechanism.