Several studies have combined computer science and materials science, with a particular emphasis on utilizing image processing methods for analyzing the microstructure images of materials. Varde [10] proposed a computational estimation method called AutoDomainMine, based on graph data mining. By integrating clustering and classification techniques, this method discovered knowledge from existing experimental data and utilized it for estimation. The main objective of this framework was to estimate the graphical results of experiments based on input conditions. Similar graph data mining methods can be employed for image deblurring tasks to analyze and extract patterns and features from image data to achieve image deblurring goals. Pan et al. [11] reviewed the evolution and impact of material microstructures during cutting processes, presenting a thermal-force-microstructure coupled modeling framework. They analyzed microstructural changes such as white layer formation, phase transformation, and dynamic recrystallization under different materials and cutting conditions, as well as the effects of these changes on cutting forces and surface integrity. Vibration of cutting tools or materials can cause motion in image acquisition devices (such as cameras) during the capturing process, resulting in image blurring. Therefore, studying the deblurring of material microstructures holds significant importance.

Many traditional image enhancement methods employ regularization and manually crafted prior images for blur kernel estimation [12]. Subsequent iterative optimization is used to gradually recover a clear image. However, this conventional approach involves intricate blur kernel estimation, leading to laborious sharpening, subpar real-time performance, and algorithmic limitations. To enhance the quality of image deblurring, many methods based on convolutional neural networks (CNN) have been proposed [13–15]. Chakrabarti [13] designed a neural network to generate a global blur kernel for non-blind deconvolution. Song et al. [14] proposed a method using a neural network for reliable detection of motion blur kernels to detect image forgeries. Wang et al. [15] proposed a network-based framework that learned to remove raindrops by learning motion blur kernels. Sun et al. [16] predicted the probability distribution of non-uniform motion blur using CNNs. However, most neural-network-based methods still rely on blur models to solve the blur kernel, limiting their performance.

In recent years, with the development of deep learning, a series of methods based on deep learning have been used for image deblurring [17,18]. Zhang et al. [19] proposed DMPHN, which is the first multi-scale network based on the multi-patch method for single-image deblurring. Chen et al. [20] proposed HINet, a deep image restoration network based on the HIN block. Fanous et al. [5] presented GANscan, a method for restoring sharp images from motion-blurred videos. The method was applied to reconstruct tissue sections under the microscope. Liang et al. [21] directly deblurred raw images using deep learning-based busy image-to-image blind-deblurring. DID-ANet [4] was designed specifically for single-image blur removal caused by camera misfocus. MedDeblur [18] was developed to remove blur in medical images due to patient movement or breathing. Xu et al. [22] proposed a deep-learning-based knowledge-enhanced image deblurring method for quality inspection in yarn production. Restormer [23] is an efficient transformer model that can be utilized for image restoration tasks at high resolutions. This model is effective for restoring high-resolution images. Chen et al. [7] found that nonlinear activation functions are not necessary and can be replaced or omitted, and developed NAFNet for both image denoising and deblurring. Due to the impressive performance of NAFNet in deblurring tasks, we are currently implementing modifications to its architecture.

In recent years, attention mechanisms have proven to be highly effective in various computer vision tasks [24,25]. As a result, attention-based methods have gradually been adopted for the task of image deblurring [26,27]. MSAN [28] is a convolutional neural network architecture based on attention that efficiently and effectively generalizes motion deblurring. D3-Net [26] can be used for deblurring, dehazing, and object detection, with the addition of a classification attention feature loss to improve deblurring and dehazing performance. Cui et al. [27] proposed a dual-domain attention mechanism that enhances feature expression in both spatial and frequency domains. Ma et al. [29] proposed an attention-based dehazing algorithm for deblurring to improve defect detection in inspection image pipelines. Shen et al. [30] introduced a supervised human-perception attention mechanism model, which performs exceptionally well in motion deblurring in an end-to-end manner. MALNET [31] is a lightweight network based on attention mechanisms, which also performs well in image deblurring. Zhang et al. [32] proposed an attention-based inter-frame compensation scheme for video deblurring. In this work, we also incorporate attention mechanisms into our image deblurring network to improve its deblurring capability.

The network structure of this paper is shown in Fig. 1. It follows a classical U-shape structure, which is an improvement from NAFNet [7]. The structure comprises an encoder and a decoder, both belonging to the MID-Block. An attention mechanism is introduced between the blocks to improve the image restoration quality of the network.

MID-Block is the basic building block of MIDNet. To avoid high complexity between blocks, MID-Block does not use any nonlinear activation functions such as ReLU, GELU, and Softmax. We construct a MID-Block using analogies with NAFNet blocks, as illustrated in Fig. 2.

To stabilize the training process, the input is first passed through Layer Normalization. Next, the input undergoes convolution operations and is then processed by SimpleGate (SG) [7], which is a variant of Gated Linear Units (GLU) [33]. The GLU formula is as follows: (1)Gate(X,f,g,σ)=f(X)⊙σ(g(X))

In Eq. (1), X represents the feature map, f and g function as linear transformers, σ represents a nonlinear activation function, such as Sigmoid, and ⊙ represents element-wise multiplication.

The GLU increases the intra-block complexity, which is not desirable. To remedy this issue, we reconsider the activation function in the block, specifically GELU [34], which is expressed as: (2)GELU(x)=xϕ(x)where ϕ represents the cumulative distribution function of the standard normal distribution. According to reference [34], it is suggested that the Gaussian Error Linear Unit (GELU) activation function can be effectively approximated and implemented by employing the following methodology: (3)0.5x(1+tanh[2/π(x+0.044715x3)])

In Eqs. (1) and (2), GELU is a specific case of GLU, where the activation functions f and g are identity functions and the parameter σ is substituted with ϕ. The GLU incorporates nonlinearity and is not reliant on the parameter σ. Even in the absence of the parameter σ, the expression Gate(X)=f(X)⊙g(X) retains its nonlinearity. According to reference [7], we suggest a simple adjustment to GLU: Split the feature map into two parts along the channel dimension and multiply them. This could be done using a basic element-wise multiplication, which is represented by Eq. (4). (4)SimpleGate(X,Y)=X⊙Y

In Eq. (4), X and Y represent feature maps of equal proportions.

The gating unit SG is a neural network component illustrated in Fig. 3, which is used in the processing of feature maps. It operates by splitting the feature map into two parts along the channel dimension, which is then multiplied to generate the final output. By splitting the feature map in this manner, SG can selectively emphasize or de-emphasize specific channels in the feature map, which can be useful for enhancing certain features or suppressing noise in the signal. This process is often referred to as channel-wise gating.

Our novel approach introduces Simplified Channel Attention (SCA) [7], a new component that utilizes channel-wise attention to enhance relevant features in data. Compared to other approaches, SCA has a simpler structure which offers ease of implementation. Additionally, it adds minimal computational overhead to models, hence enhancing the efficiency of our approach. Please refer to Fig. 4 for an illustration of SCA.

SCA determines channel attention by computing the average of the feature map along the spatial dimensions and applying a fully connected layer to generate a channel-wise attention vector. This attention vector is then multiplied with the original feature map to selectively amplify important channels in the data while suppressing irrelevant or noisy channels.

Our experiments demonstrate that incorporating SCA into a standard convolutional neural network yields improved performance, highlighting the efficacy of enhancing feature representation using channel attention. SCA can be easily integrated into existing neural network architectures and represents a useful tool for improving the performance of deep learning models in a variety of applications.

SCA is derived from Channel Attention (CA) [35], which can be expressed by the Eq. (5). (5)CA(X)=X⊗σ(W2max(0,W1pool(X)))

In Eq. (5), X denotes the feature map, pool denotes the global average pooling operation, σ denotes an activation function such as Sigmoid, W_1, and W₂ denote fully connected layers, and cross multiplication is the channel multiplication operation. By simplifying the Eq. (5), we can finally obtain SCA, as shown in Eq. (6). (6)SCA(X)=X⊗Wpool(X)

With the advancement of deep learning techniques, significant progress has been made in image restoration. The NAFNet model, in particular, has shown significant performance in various applications. However, a limitation of NAFNet is that the skip connections used for feature aggregation between the encoder and decoder have the potential to disrupt the feature distribution, resulting in inconsistencies between these components. Another shortcoming of NAFNet is that it only employs an intra-block attention mechanism and ignores attention-based skip connections.

To address these challenges, we introduce a soft attention mechanism to capture the latent relationship between the encoder and decoder more adaptively. We refer to the proposed soft attention mechanism as ATT. The architecture of the attention gate ATT is shown in Fig. 5. Specifically, the proposed attention gate ATT aggregates features from different blocks using a weighting scheme based on their relevance to the current image restoration task, instead of simple element-wise addition used in conventional skip connections. This allows the model to selectively focus on the most informative features while suppressing the irrelevant ones.

Moreover, our attention mechanism enables us to incorporate attention-based skip connections, which further enhance the feature aggregation process. By attending to the most informative features in the encoder and decoder, the model can effectively alleviate issues related to feature distribution inconsistency and strengthen the correlation between these components. The formula of the soft attention mechanism can be expressed as follows: (7)Watt=ψT(σ1(WaTxa+WbTxb+bf))+bψ (8)x^b=σ2(Watt)∗xb

In Eqs. (7) and (8), σ1 and σ2 denote activation functions. The attention gate is represented by a set of parameters through mathematical formulas, including linear transformations Wa, Wb, ψ, and biases bf, bψ. The linear transformations are obtained by performing convolution operations on the input tensors. The output of the attention gate is the product of the input feature map and the attention coefficient.

The paper utilizes multi-loss functions, as shown in Eq. (9), which comprise the deblurring loss, edge loss, and FFT loss. The hyperparameters λ1 and λ2 are assigned the values of 0.05 and 0.01, respectively. (9)L=Ld+λ1Le+λ2Lf

We utilize a dataset containing 120 paired images with both low and high quality to investigate material microstructure fuzziness. Specifically, low-quality images in this dataset are directly obtained from observations captured through the SEM rather than artificially blurred using blur kernels or algorithms. This approach replicates real-world scenarios more accurately while simultaneously presenting greater challenges for the process of deblurring. When low-quality images are captured in practice, operators take repeated images until high-quality ones are achieved. Consequently, we meticulously selected 120 matching low and high-quality images that met stringent criteria. All images are subsequently adjusted to 256 * 256 pixels. Several cropped images are displayed in Fig. 6. The dataset is randomly divided into a training set comprising 108 image pairs and a test set containing 12 image pairs.

We optimize the model using Adam (β1 = 0.9, β2 = 0.999) for 200 K iterations with a cosine annealing schedule that decreases the learning rate from 10–3 to 10–7. We crop the images to a size of 256 * 256 pixels and apply rotation and flipping as data augmentation techniques. We employed the skip-init method to ensure stable training and implemented our code in the PyTorch framework. We evaluate our model using peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) metrics. All experiments are conducted on an NVIDIA Tesla V100 GPU.

PSNR and SSIM are employed as quantitative evaluation metrics, with larger values indicating superior image quality. They are calculated according to Eqs. (13) and (14). (13)PSNR=10⋅log10⁡(MAX2MSE) (14)SSIM(x,y)=(2μxμy+C1)(2σxy+C2)(μx2+μy2+C1)(σx2+σy2+C2)

In Eq. (13), MAX represents the maximum pixel value of the image, typically 255 when each pixel is represented by an 8-bit binary. MSE (Mean Squared Error) is the mean squared error value between the blurred image and the clear image. In Eq. (14), x and y denote the original image and the deblurred image, respectively. μx and μy represent the mean pixel values of images x and y, σx and σy represent the standard deviations of pixel values in images x and y, and σxy is the covariance between the pixel values of the two images. C1 and C2 are constants introduced to prevent division by zero in the denominator.

To assess the generalizability of models trained on natural images to material microstructure fuzziness data, we conduct a series of relevant studies. Specifically, we employ pre-trained weights from the original papers of DMPHN, HINet, Restormer, and NAFNet methods to conduct inference on material blurry images. The deblurred images are displayed in Fig. 7, while the corresponding PSNR and SSIM values are summarized in Table 1.

As observed in Fig. 7, these methods exhibit certain levels of processing applied to the blurry images. However, their ability to achieve satisfactory deblurring outcomes remains limited, with minimal improvement over the initial blurry images. By referring to Table 1, the PSNR and SSIM values of both the original blurry and clear images are provided in the input row. Notably, these methods yield relatively low PSNR and SSIM scores, with instances where deblurred images demonstrate worse performance compared to their initial states.

Interestingly, these methods have demonstrated proficiency on the GoPro dataset and have exhibited effective deblurring outcomes on real-world blurry images. Consequently, we postulate that their subpar performance on material images may be attributed to external factors rather than the inherent limitations of the methods themselves.

Upon meticulous scrutiny of the GoPro dataset, a notable distinction emerges in the PSNR values of its blurry images, which average approximately 23. In contrast, the blurry images originating from our material microstructure exhibit a lower PSNR value of approximately 21. Building upon these observations, a hypothesis arises: The relatively lower quality of material images, resulting in reduced information content, poses a heightened challenge for the deblurring process. Consequently, this challenge could potentially contribute to network degradation and the suboptimal performance observed.

Furthermore, an additional factor potentially influencing the subpar deblurring results is the unique visual characteristics inherent to material microstructures, setting them apart from real-world blurry images. This disparity in appearance might contribute to reduced reliability in the neural network’s performance when confronted with material microstructure fuzziness data. To address this challenge, we advocate for a proactive solution: Retraining and fine-tuning these methods using material blurry images. Our approach involves freezing the majority of the model layers and selectively unfreezing a small subset for training purposes. We apply data augmentation techniques, such as flip and rotate, to the dataset during the training process. Hyperparameters, including learning rate, batch size, and number of iterations, are adjusted based on the specific model to achieve optimal performance. Additionally, appropriate regularization strategies are employed to mitigate overfitting problems. Such an approach holds the promise of enhancing the network’s capability to effectively restore blurry images of materials. In line with this recommendation, we embarked on the process of retraining and fine-tuning these methods. To gauge the efficacy of this intervention, we present the deblurring outcomes in Fig. 8.

This study utilizes a dataset of material blurry images to conduct a detailed analysis of the deblurring capability of the original method compared to the retraining and fine-tuning methods. The outcomes of this comparison reveal a significant enhancement in deblurring quality for material images through retraining and fine-tuning, surpassing the performance of the no-training scenario and yielding satisfactory results. Notably, the process of retraining and fine-tuning contributes to the restoration of intricate features within material images, underscoring the pivotal role of material-specific data in optimizing deblurring effectiveness. These findings offer fresh insights into the efficacy of retraining and fine-tuning strategies in effectively addressing the intricate deblurring challenges posed by material images. Furthermore, they provide valuable guidance for the future development of more potent deblurring methodologies within the domain of material science and engineering. Importantly, this study also serves as a demonstration of the potential of deep learning techniques in enhancing the quality of visual data across a wide spectrum of scientific and industrial applications.