Current time series forecasting methods can be divided into traditional statistics-based methods and deep learning-based methods.

Traditional statistics-based methods, such as the autoregressive integrated moving average (ARIMA) [12] and Kalman filter models [13], have had extensive theoretical research in the past. However, these traditional statistical-based methods perform poorly on complex time series.

Deep learning can automatically learn and model the hidden features of complex series based on raw data and can achieve better forecasting accuracy on complex time series datasets. Therefore, more research is now based on deep learning methods.

Recurrent neural networks (RNNs) [14] play an important role in time series forecasting [15–17]. However, when the given time series is long, much of the information contained earlier in the time series will be lost [18].

In recent years, transformer-based models [19] have been widely used for time series forecasting tasks [20]. Informer [21] reduces the computational complexity of the self-attention mechanism and performs well in long-term time series forecasting. Autoformer [22] uses autocorrelation attention and seasonal decomposition methods for model construction, reducing the required computational workload and improving forecasting accuracy. FEDformer [23] uses the seasonal trend decomposition method and frequency-enhanced attention. PatchTST [24] achieves SOTA accuracy on several long-term time series forecasting datasets based on channel independence and subseries-level patch input mechanism. However, when these transformer-based models deal with long sequence data, they always require considerable computational resources and memory. In addition, using self-attention mechanisms for time series modeling may lead to the loss of temporal information [10].

TCNs are also popular for time series forecasting tasks [25–28]. MISO-TCN [29] is a lightweight and novel weather forecasting model based on TCN. TCN-CBAM [30] uses the convolutional block attention module for the prediction of chaotic time series. VMD-GRU-TCN [31] integrates variational modal decomposition, gated recurrent unit and TCN into a hybrid network for high and low frequency load forecasting. Veg-W2TCN [32] combines multi-resolution analysis wavelet transform and TCN for vegetation change forecasting. TCNs use dilated causal convolution in the temporal dimension to learn temporal dependencies [33,34]. For vanilla convolution operations, the size of the receptive field is linearly related to the number of convolution layers, which leads to an inability to handle long sequence inputs. Dilated causal convolution is a very efficient structure that helps a model achieve exponential receptive field sizes. For an input sequence X={x0,x1,…,xt}, the dilated causal convolution operation for the element located at s can be formalized as: (1)F(s)=∑i=0k−1f(i)⋅xs−diwhere fn:{0,…,k−1}→R is a convolutional filter, d is the dilation factor, and k is the convolutional filter size. The use of a larger d allows the top output to represent a larger range of inputs, which effectively expands the receptive field of the convolutional network, as shown in Fig. 1.

Dilated causal convolution can capture the long-term dependencies of the time series, but the causal convolution structure limits the receptive field and results in severe information loss during dilated convolution. Therefore, we propose a new dilated convolution method to replace dilated causal convolution in TCNs.

Forecasting models often suffer badly from distribution shift in time series. Domain adaptation [35] and domain generalization [36] are common methods to alleviate the distribution shift [37,38]. However, these methods are often complex, and it is not easy to define the domain in non-stationary time series. Recently, some simple and effective methods have been proposed to address the distribution shift problem. The Revin method [39] normalizes each input sequence and then non-normalizes the model output sequence. However, this method prevents the model from obtaining features such as the magnitude of data changes. The NLinear model [10] uses a method of subtracting the end value of the input sequence from each value of the input sequence and restoring the end value to the output sequence. However, there is a problem of significant differences within the input sequence, which undermines the effectiveness of model training. Our proposed method minimizes the difference between and within input sequences without losing information in the original data.

The overall architecture of our proposed model is shown in Fig. 3. First, the DCM carries out a difference operation on the input time series to obtain difference sequences with smaller discrepancies in values, and then the Padding operation is used to extend the sequence length to meet the requirements of dilated convolution. Next, the Embedding operation maps the padded sequences to a high-dimensional space, followed by multiple layers of SDC to extract temporal dependencies of the input. Finally, the Decoding operation is used and the DCM restores the original input values to the Decoding output to obtain the prediction result.

DCM is used to address the problem of distribution shift. It contains a difference stage and a compensation stage.

In the difference stage, a difference sequence is obtained through differencing adjacent elements in an input sequence. For an input sequence X={x0,x1,…,xt} with length t, the difference stage can be formalized as:

(2)Xd={x1−x0,x2−x1,…,xt−xt−1}

In the compensation stage, to compensate for the information loss caused by difference operation, the last element value of the original sequence is added back to the output. For the output sequence Y={y0,y1,…,yq} of length q, the compensation stage can be formalized as:

(3)Y={y0+xt,y1+xt,…,yq−1+xt,yq+xt}

To ensure that the sequence length can meet the requirements of the SDC, we pad the difference sequence Xd with zeros, and the length L of the padded sequence is:

(4)L=kcwhere k is the convolutional filter size, and c is the number of layers of the Muti-layer SDC.

The process of padding and embedding the sequence Xd can be formalized as:

(5)Xp=Padding(Xd),

(6)H0=Embedding(Xp)where Embedding(.) is a linear layer that maps the padded sequence Xp∈Rvar×L to H0∈Rdim×L, var denotes the number of variates in time series, dim is a hyperparameter that represents the hidden variable dimension of the model, H0 is the input of Muti-layer SDC.

Due to the limitation of dilated causal convolution on the receptive field, we propose the SDC to be used instead of dilated causal convolution. Our SDC has the following two similarities with dilated causal convolution: (1) The size of the receptive field of the output sequence elements is exponentially related to the number of network layers, so very large receptive fields can be achieved by stacking a few convolutional layers. (2) The input and output sequences are of equal length, which facilitates the stacking of layers to obtain a larger receptive field. The uniqueness of SDC is that it uses multiple convolutional filters to convolve the elements of adjacent subsequences. The elements in a subsequence share a receptive field. Fig. 4 shows a 3-layer SDC structure.

At the SDC layer, an input or output sequence is divided into several subsequences. For multi-layer SDC, the initial subsequence length is 1, and the length will increase with the number of SDC layers by SDC operation. Each SDC layer uses k convolutional filters of size k, which means that the size and number of convolutional filters used in each layer are equal. This setting ensures that the input and output sequences possess equal lengths.

Specifically, at SDC Layer i, k input subsequences are convolved to generate an output subsequence whose length is the sum of the lengths of these input subsequences. Therefore, we can calculate the length of subsequences for each layer. The output sequence Hi of SDC Layer i can be expressed based on subsequence as:

(7)Hi={hi0,0,…,hia,b,…,hiri−1,pi−1}where hia,b denotes the a-th element of the b-th subsequence, the subsequence length ri=ki, ki is the i-th power of k, pi is the number of the subsequences, and pi=L/ri.

The detailed process of the SDC operation for convolving subsequences is as follows: firstly, the elements in the neighboring subsequences are convolved using several different filters, and elements in the same subsequence are not convolved with each other. Then the output elements generated by convolving the same elements but with different convolution filters are adjacent to each other, and the elements generated by the same subsequences are merged into a new subsequence. E.g., in Fig. 4, convolution between h10,0 and h10,1, not between h10,0 and h11,0, the output elements h20,0 and h21,0 generated by the same input elements are adjacent to each other.

We first formalize SDC operations from the perspective of a single output element. The SDC operation on the a-th element h^ia,b of the b-th subsequence can be formalized as:

(8)h^ia,b=fdc(Hi−1,a,b)=∑n=0k−1fm(n)⋅hi−1r.,p.+nwhere Hi−1 is the input sequence of Layer i, k is the convolutional filter size, fm:{0,…,k−1}→R is the m-th convolution filter, m=amodk, r.=⌊ak⌋, and p.=b⋅k.

Next, based on the SDC operation for a single element, we extend to the SDC operation for the sequence. The SDC operation for the input sequence Hi−1 can be formalized as:

(9)H^i=FDC(Hi−1) ={h^i0,0,…,h^iri−1,pi−1} ={fdc(Hi−1,0,0),…,fdc(Hi−1,ri−1,pi−1)}

For a SDC Layer i, the output Hi is obtained after SDC operation and residual connection, the process can be formalized as:

(10)Hi=ReLU(H^i+Hi−1)where ReLU(⋅) is the rectified linear unit (ReLU) activation function [40].

To enhance the comprehensibility of the SDC layer, we utilize pseudo-code to illustrate the process in Algorithm 1.

The Decoding module includes 2 fully-connected layers. The one maps Hc∈Rdim×L to Hd∈Rtvar×L, tvar denotes the number of target variates to be predicted. And then the other one maps Hd∈Rtvar×L to Y ∈Rtvar×pL, pL denotes the time step of the target variates.

ETT (Electricity Transformer Temperature): This dataset consists of the electricity transformer temperature derived from two different countries in China for two years. ETTh1 and ETTh2 correspond to data collected at one-hour intervals from these two counties. The target “oil temperature” was predicted based on past data. The train/val/test is 12/4/4 months.

ECL (Electricity Consumption Load): This dataset collects the electricity consumption loads (kWh) of 321 customers. We set “MT_320” as the target value. The training, validation, and test sets were scaled to 0.6/0.2/0.2.

WTH (Weather): This dataset collects four years of climate data from 2010 to 2013; it covers 1600 locations in the United States, with data points collected every hour. Each data point consists of the target value “wet bulb”. The training, validation, and test sets were scaled to 0.6/0.2/0.2.

To verify the superiority of our proposed method, we compared it with four SOTA models: PatchTST, FEDformer, Autoformer, and Informer. In addition, our proposed method was also compared with the classic models TCN and LSTM.

PatchTST (2023) [24]: This is a transformer-based model. it achieves SOTA accuracy on ETTh1 and ETTh2 datasets based on channel independence and subseries-level patch input mechanism.

FEDformer (2022) [23]: This is a transformer-based model. It uses the seasonal trend decomposition method and a frequency-enhanced attention module. Since the frequency-enhanced attention module has linear complexity, FEDformer is more efficient than the standard transformer. It achieves the best forecasting performance on the benchmark dataset in comparison with previous state-of-the-art algorithms.

Autoformer (2021) [22]: This is a transformer-based model. It uses auto-correlation attention and seasonal decomposition for model construction, reducing the required computational effort and achieving improved forecasting accuracy.

Informer (2021) [21]: This is an efficient transformer-based model that uses the ProbSparse self-attention mechanism to reduce its computational complexity. The model performs well in long-term time series prediction tasks.

TCN (2018) [9]: This is a convolutional network for sequence modeling that uses dilated causal convolution. It outperforms recurrent neural networks in many sequence modeling tasks.

LSTM (1997) [41]: This is a recurrent neural network whose gating mechanism alleviates the problem of gradient disappearance and allows the model to capture long-term dependencies.

We used the following evaluation metrics:

(1) Mean squared error (MSE):

(11)MSE=1n∑(y−y∗)2

(2) Mean absolute error (MAE):

(12)MAE=1n∑|y−y∗|where n is the number of time steps to be predicted, y is the predicted value, and y∗ is the ground truth.

For all the compared models, the same parameter settings were used for the training process, with predicted sequence lengths of 24, 48, 168, 336, and 720. The models were optimized using the adaptive moment estimation (Adam) optimizer with learning rates starting at 1e-3. The total number of epochs is 8 with proper early stopping. We used Mean Squared Error (MSE) as our loss function. The inputs of each dataset were zero-mean normalized. Following the previous related work [42], we used a 10-residual-block stack in the TCN. Referring to Informer [21] and PatchTST [24], the input length was selected from {168, 336, 512, 720}. The other settings used their default values.

For better performance of TSCND, we set the convolutional filter size k to 2 and the hidden layer dimension dim to 64 based on the validation dataset. To allow the output elements to have a look back at the entire input sequence while the model has as little computational effort as possible, the number of model layers is computed from the input length t, and the number of model layers c=⌊logk⁡t⌋+1.

Tables 1 and 2 list the forecasting results of the comparison models and our model. Our model achieved the best results in 32/40 cases and the second-best results in 6/8 cases where it did not achieve the best results. Compared with the SOTA model PatchTST, our model yields an overall 7.3% relative MSE reduction, and when the prediction length is 720, the improvement can be more than 15%. TSCND performs slightly worse than PatchTST on the MSE metrics in the ETTh1 dataset. This is due to the presence of many outliers in the ETTh1 dataset, and the self-attention mechanism employed by PatchTST is more advantageous than convolutional networks in dealing with outliers. Meanwhile, TSCND outperforms PatchTST on the MAE metric in the ETTh1 dataset. This metric is insensitive to outliers, which supports the above view and suggests that TSCND is better at predicting the normal value of ETTh1.

Compared to TCN, our method brings improvement in all cases on ETTh1, ETTh2, and ECL datasets, and brings improvement in 7/10 cases on WHT datasets. It achieved better results than TCN, with an average MSE reduction of 19.4%. In the short-term forecasting of WTH, the results of TSCND and TCN are similar for the following reasons. The time series of the WTH dataset is relatively stationary, thus the DCM method is of little help to the model. And when the prediction length is short, this advantage of SDC using subsequences instead of elements to extract features is not obvious, so TSCND and TCN perform similarly. However, when predicting long-term time series, TSCND is superior to TCN.

To evaluate the efficiency of our proposed model in handling inputs of different lengths, we chose PatchTST and TCN for comparison. We compared the time required for training each model for 1 epoch on the ETTh2 dataset under different input sequence lengths. The results are shown in Table 3. Compared with PatchTST and TCN, it reduces training time by over 40% on average. The reason for the faster training speed of TSCND compared to TCN is that (1) TSCND uses only one convolutional layer per layer, while TCN uses two convolutional layers per layer. (2) TSCND is designed with the number of layers adjusted according to the length of the inputs, while TCN has a fixed number of layers. TSCND uses fewer layers when processing short inputs. (3) Since the stride length of the convolutional layers in TCN is shorter than in TSCND, the elements of each layer in TCN participate in the convolution more times.

To evaluate the proposed method for addressing the distribution shift problem, we conducted experiments on the ETTh1 dataset using our proposed method, the Revin method (Revin) [39], and the NLinear method (SubLast) [10]. The experimental results are shown in Table 4, and it can be seen that our method achieves the best performance in almost all cases. Compared with the original input, our method yields an overall 52.6% relative MSE reduction, indicating the effectiveness of the proposed method. We conducted experiments using CPU AMD Ryzen 7 5800 to compare the time taken by our method with the other methods. The results of CPU times are shown in Table 5. Our method takes more time than the SubLast method but much less time than the Revin method. This time is almost negligible compared to the overall training time of the model. Therefore, we believe that our method remains an alternative to SubLast method where efficiency is pursued.

To test whether a subsequence can capture more temporal dependencies than a single element, we tested three different structures of the Decoder Layer for making predictions using SDC output.

Structure A: This is also the structure as that in TSCND. The structure uses the whole output sequence to predict each future time point, which is shown as Structure A in Fig. 5.

Structure B: In this structure, each single output element of the Multi-layer SDC is used to predict a time point independently, which is shown as Structure B in Fig. 5.

Structure C: This is similar to the dilated causal convolution, where only one output element can obtain information about the whole input sequence. The structure predicts each time point using a single element of the Multi-layer SDC outputs, which is shown as Structure C in Fig. 5.

We conducted experiments using above structures on the ECL dataset, and the experimental results are shown in Fig. 6. The prediction error of Structure A is much lower than the other two structures under all lengths, especially for long lengths. Using output sequences rather than single elements for prediction improves prediction accuracy, indicating that SDC can enhance feature extraction by using subsequences to extract features from receptive fields.