We propose new hybrid Lagrange neural networks called LaNets to predict the numerical solutions of partial differential equations. That is, we embed Lagrange interpolation and small sample learning into deep neural network frameworks. Concretely, we first perform Lagrange interpolation in front of the deep feedforward neural network. The Lagrange basis function has a neat structure and a strong expression ability, which is suitable to be a preprocessing tool for prefitting and feature extraction. Second, we introduce small sample learning into training, which is beneficial to guide the model to be corrected quickly. Taking advantages of the theoretical support of traditional numerical method and the efficient allocation of modern machine learning, LaNets achieve higher predictive accuracy compared to the stateoftheart work. The stability and accuracy of the proposed algorithm are demonstrated through a series of classical numerical examples, including onedimensional Burgers equation, onedimensional carburizing diffusion equations, twodimensional Helmholtz equation and twodimensional Burgers equation. Experimental results validate the robustness, effectiveness and flexibility of the proposed algorithm.
In this paper, we consider partial differential equations (PDEs) of the general form in
where
At first, traditional numerical methods, including finite element method [
Neural network is a black box model, its approximation ability depends partly on the depth and width of the network, and thus too many parameters will cause a decrease in computational efficiency. One may use Functional Link Artificial Neural Network (FLANN) [
On the other hand, deep learning is a type of learning that requires a lot of data. The performance of deep learning depends on largescale and highquality sample sets but the cost of data acquisition is prohibitive. Moreover, sample labeling also needs to consume a lot of human and material resources. Therefore, a popular learning paradigm named Small Sample Learning (SSL) [
In this paper, we integrate Lagrange interpolation and small sample learning with deep neural networks frameworks to deal with the problems in existing models. Specifically, we replace the first hidden layer of the deep neural network with a Lagrange block. Here, Lagrange block is a preprocessing tool for preliminary fitting and feature extraction of input data. The Lagrange basis function has a neat structure and strong expressive ability, so it is fully capable of better extracting detailed features of input data for feature enhancement. The main thought of Lagrange interpolation is to interpolate the function values of other positions between nodes through the given nodes, so as to make a prefitting behaviour without adding any extra parameters. Then, the enhanced vector is input to the subsequent hidden layer for the training of the network model. Furthermore, we add the residual of a handful of observations into cost function to rectify the model and improve the predictive accuracy with less label data. This composite neural network structure is quite flexible, mainly in that the structure is easy to modify. That is, the number of polynomials and hidden layers can be adjusted according to the complexity of different problems.
The structure of this paper is as follows. In
In this section, we start with illustrations on Lagrange interplotation polymonials. After that, we discuss the framework of LaNets. And finally, we clarify the detatils of the proposed algorithm.
Lagrange interpolation is a kind of polynomial interpolation methods proposed by JosephLouis Lagrange, a French mathematician in the 18th century, for numerical analysis [
Assuming
Here, the polynomial
As shown in
The problem we aim to solve is described as
We continue to approximate
Here,
An entire overview of this work is shown in Algorithm 1. In the algorithm description, we consider the spatiotemporal variables
In this section, we verify the performance and accuracy of LaNets numerically through experiments with benchmark equations. In
In this subsection, we demonstrate the predictive accuracy of our method on three onedimensional timedependent PDEs including Burgers equation, carburizing constant diffusion coefficient equation and carburizing variable diffusion coefficient equation.
We start with the following onedimensional timedependent Burgers equation in
Here, the LaNets model consists of one Lagrange block and 7 hidden layers with 20 neurons in each layer. Lagrange block contains three Lagrange basis functions. By default, the Lagrange block is composed of three Lagrange basis functions unless otherwise specified.
To further verify the effectiveness of the proposed algorithm, we compare the predicted solution with the analytical solution provided in the literature [
A more detailed numerical result is summarized in
0  50  100  200  300  

PINNs  1.6 
       
Benchmark    1.1 
7.9 
5.5 
2.4 
LaNets    4.77 
3.84 
1.11 
5.13 
We consider the onedimensional carburizing diffusion model [
Constant diffusion coefficient
We start with the constant diffusion coefficient
In this numerical experiment, we take
Regarding the training set, we take
In order to evaluate the performance of our algorithm in multiple ways, we compare the simulation results with the simulation results obtained by the PINNs model and our earlier model. The results for the three models are shown in
0
50
100
200
300
PINNs
6.72




Benchmark

2.38
1.48
1.19
1.22
LaNets

2.13
1.43
1.31
1.35
Variable diffusion coefficient
In this experiment, the carburizing diffusion coefficient
The analytical solution corresponding to this setting is
In this example, we have
Further, we make a contrast between the simulation results obtained by the proposed model and the benchmark model. The detailed results for them are displayed in
0  50  100  200  300  

PINNs  2.25 
       
Benchmark    5.93 
5.95 
6.29 
1.20 
LaNets    1.38 
1.67 
3.22 
2.07 
In this section, we consider twodimensional problems including the timeindependent Helmholtz equation and timedependent Burgers equation to verify the effectiveness of the LaNets model. These two types of twodimensional problems aim to demonstrate the generalization ability of our methods.
In this example, we consider a timeindependent twodimensional Helmholtz equation as
Here, we take
The training set of this example is generated according to the exact solution in the above equation. The problem is solved using the 4layer LaNets model on the domain
The visual comparison among LaNets, benchmark and PINNs results is displayed in
0  50  100  200  300  

PINNs  4.95 
       
Benchmark    1.10 
6.03 
2.43 
1.26 
LaNets    7.31 
6.32 
5.28 
1.23 
In the last experiment, we consider a twodimensional timedependent Burgers equation as
In this example, we take
The decline curve of the loss function is shown in
0  50  100  200  300  

PINNs  1.64 
       
Benchmark    5.13 
3.81 
2.7 
4.0 
c    2.19 
3.69 
2.39 
2.06 
In this paper, we propose hybrid Lagrange neural networks called LaNets to solve partial differential equations. We first perform Lagrange interpolation through Lagrange block in front of deep feedforward neural network architecture to make prefitting and feature extraction. Then we add the residuals of small sample data points in the domain into the cost function to rectify the model. Compared with the singlelayer polynomial network, LaNets greatly increase the reliability and stability. And compared with general deep feedforward neural network, the proposed model improves the predictive accuracy without adding any extra parameters. Moreover, the proposed model can obtain more accurate prediction with less label data, which makes it possible to save a lot of manpower and material resources and improve computational efficiency. A series of experiments demonstrate the effectiveness and robustness of the proposed method. In all cases, our model shows smaller predictive errors. The numerical results verify that the proposed method improves the predictive accuracy, robustness and generalization ability.
This research was supported by NSFC (No. 11971296), and National Key Research and Development Program of China (No. 2021YFA1003004).