Since the threshold value of the pavement gray map is related to the average value of pixel brightness, Cheng et al. [18] proposed a real-time image threshold algorithm that reduces sample space and differences to determine appropriate thresholds while reducing sample space. Uneven shadows and lighting are common in photos taken in real scenes, which can seriously affect crack detection by threshold segmentation. Then, in order to reduce the influence of noise in the image background, the researchers [4,5] used an edge detection algorithm to detect cracks. With the rapid development of deep learning, unprecedented breakthroughs have been made in the field of computer vision. Many deep learning methods have also been applied to crack detection tasks. Dung et al. [19] proposed a coding-decoding full convolutional network for crack detection, which has shown improved accuracy in predicting crack path and density. Liu et al. [20] proposed a network architecture composed of a full convolutional network and a deep supervision network. Zou et al. [21] proposed DeepCrack, an end-to-end trainable deep convolutional neural network for automatic crack detection by learning advanced features of crack representation. Due to the limitations of the receptive field of convolution, some researchers have attempted to use atrous convolution [22]. Hybrid atrous convolutional network (HACNet) had been proposed by Chen et al. [16], which used an atrous convolutional network with an appropriate expansion rate to expand the receiving field while maintaining the same spatial resolution. Zhou et al. [23] proposed an attention mechanism and hybrid pool module to capture both long-range and short-range dependence in crack detection. Qu et al. [24] proposed a concrete pavement crack detection algorithm based on attention mechanism and multi-feature fusion. Yang et al. [25] designed an end-to-end deep crack segmentation network is proposed, which combined progressive and hierarchical context fusion.

The methods previously described predominantly utilize a singular encoder-decoder structure, which fails to capture certain edge detail information and global information effectively. Therefore, we propose a network structure based on dual-encoder feature extraction for crack characterization, designed for dual-level extraction of feature information, ensuring the generation of more precise and accurate prediction maps.

Due to limited computing resources in actual application scenarios, almost no deep learning network models have been used for crack detection tasks. So a lightweight network for crack detection is urgently needed. In computer vision tasks, knowledge distillation [26] and network pruning are commonly used to build lightweight models; however, these methods are only effective for complex models with high efficiency. Therefore, the most feasible way to achieve a lightweight crack detection network is by building an efficient network structure. For example, MobileNets use intermediate expansion layers that employ lightweight depthwise convolutions to filter nonlinear feature sources [27]. Recently, many lightweight networks based on deep learning have emerged. Liao et al. [28] used the modified residual network to build a lightweight network architecture with an encoder-decoder structure. Zhang et al. [29] proposed a lightweight U-Net model based on attention fusion. Deng et al. [30] proposed asymmetric architectures by gradually fusing information from astrous convolutional layers to reduce network parameters and improve computational and detection performance. These lightweight networks confirm their utility by reducing the size and running time of the model while maintaining similar performance parameters.

Although the aforementioned lightweight networks have achieved certain effects, there still exists an issue of low accuracy, which creates a gap between their performance and the practical application requirements in real-world scenarios. Therefore, dual-encoder network with low parameter count is proposed to address the issues of large parameter size and low accuracy.

There is some unimportant information in the feature extracted by the encoder, which makes it difficult to filter out irrelevant information about cracks. A “Squeeze and Excitation” (SE) block was proposed by Hu et al. [31] to model the interdependencies among channels by adaptively recalibrating the channel feature response mode and enhancing the representation capability of CNN by improving the spatial coding quality of the entire feature hierarchy. Chen et al. [32] proposed feature maps extracted from the convolutional neural network would be used in Transformer as input sequence to extract global context information, and at last, encoded features would be fused with CNN feature graphs to generate feature maps. In order to capture large receptive field contextual information, Liu et al. [33] designed a self-attention module with 1 × 1 convolution kernel was proposed to extract context information efficiently across feature channels. For inadequate local feature processing and information loss caused by pooling operations, Yang et al. [34] proposed a multi-scale triple attention network for end-to-end pixel-level crack detection. Zhao et al. [35] made the model adaptively combine local crack features with their global dependencies by establishing a connection between feature interdependencies in channel and spatial dimensions.

In the methods described above, the employed attention mechanisms, while effective in filtering out non-crack information, concurrently increase the complexity of the network model due to their substantial parameter size. Consequently, the implementation of attention mechanisms with smaller parameter sizes, capable of efficiently filtering irrelevant information within the network, can enhance performance without increasing the complexity of the model.

In our paper, crack detection is treated as a pixel-level segmentation task, where “0” represents “non-crack pixels” and “1” represents “crack pixels”. This is shown in Table 2, Our network model achieves the highest F1 value of 0.874 and MIoU of 0.883 on the DeepCrack dataset. Compared with all the contrastive methods listed in Table 2, the image segmentation evaluation metrics employed in this study have shown optimal outcomes in every aspect. Benefiting from atrous convolution and lightweight attention [17], as demonstrated in Fig. 1, We have developed a lightweight network featuring dual-encoder structure, consisting of shallow feature extraction module and large kernel atrous convolution module integrated with lightweight attention mechanism. On this basis, features extracted from the two encoders are cross-fused in the manner depicted in Fig. 1, effectively generating the final predictive image. The shallow feature extraction module (SFEM) is divided into three stages and comprises seven convolution blocks. As shown in Fig. 3, the detailed information of our model and the specific configurations of each module are elaborately depicted. Each block consists of one convolution layer with 3 × 3 kernel, followed by one batch normalization (BN) layer and one rectified linear unit (ReLU) layer. Specifically, the first stage consists of two convolution blocks with 64 channels each, the second stage is composed of two convolution blocks with 128 channels each, and the third stage includes three convolution blocks with 256 channels each. These three stages together form the SFEM, serving as one of the encoders. The large kernel atrous convolution (LKAC) module is divided into upper, middle, and lower parts. The upper part consists of one convolution layer with 3 × 3 kernel and 48 channels, followed by one batch normalization (BN) layer and one ReLU layer. The middle part is primarily composed of the coordination attention atrous convolution (CAAC) module. The lower part includes two convolution layers, each with 1 × 1 kernels and one channel, corresponding to the outputs of the two CAAC modules. Finally, the results from these two layers are concatenated. In the network’s decoder, the feature cross-fusion stage is employed. Specifically, the output feature map from the first stage of the SFEM is added to the result of the first CAAC module to generate the first side output. The output feature map from the second stage of SFEM is directly used as the second side output. The output from the third stage of SFEM is added to the output of the LKAC encoder to form the third side output. Finally, the feature map obtained by adding these three side outputs is concatenated with the output of the LKAC encoder to produce the final prediction map.

Ordinary convolution has some fatal defects, such as frequent upsampling and pooling operations, which will lead to the loss of internal data structure and loss of spatial hierarchical information. We will avoid these operations and use atrous convolutions for feature extraction. As shown in Fig. 1, the CAAC module comprises four groups of atrous convolution blocks, each characterized by varying dilation rates. Each group predominantly consists of one 3 × 3 atrous convolution layer, followed by one BN layer and one ReLU layer. After cascading these four groups of atrous convolution blocks, the output is processed through two 1-D convolution blocks. The first 1-D convolution block includes one 1 × 3 convolution layer, one BN layer, and one ReLU layer, while the second 1-D convolution block is composed of one 3 × 1 convolution layer, one BN layer, and one ReLU layer.

Inspired by HACNet [16], the dilation rates for these four groups of atrous convolution blocks are sequentially set as {2,4,8,18}. For the atrous convolution block with dilation rate of 2, coordination attention (CA) mechanism is adopted. This is due to the more complex nature of the information features extracted in the atrous convolution layers with smaller dilation rates. Since 1-D convolution has the advantages of fewer parameters and less computing resources, we cascade and fuse the outputs of all atrous convolution blocks and pass a pair of 1-D convolution before the final output to reduce the number of channels of the feature map and achieve the effect of refining the features. In this module, the receptive field (RF) size in the j th convolutional layer is calculated as follows: (1)RFj=(RFj−1−1)×sj+1+(kj−1)×rj,

where RFj is the size of the RF in the j th layer and the same RFj−1 is the RF size at layer ( j − 1). In the j th layer, sj, kj, rj represent the dilation rate, kernel size, and stride, respectively.

Recently, the attention mechanism has been widely used in crack detection methods based on deep learning [23,24,29]. The features extracted by the crack detection network model contain a lot of detailed information, but also contain a lot of noise, which leads to the final result is not clear. Currently, prevailing crack detection methodologies employ the attention mechanism. This approach enhances performance while simultaneously increasing the parameter count and computational resource requirements. To solve this problem, coordinated attention with a small number of parameters and occupying small computational resources is used in our proposed network. The diagram of coordinated attention intends to be designed as shown in Fig. 4. In order for the attention block to capture long-distance interactions with precise location information, we decompose the global pooling into a pair of 1-D average pooling to aggregate features along the horizontal and vertical directions, respectively, to generate bidirectional spatial feature perception. For example, if we set the pooling kernels to be (H,1), (1,W), and input T, the output of the c-th channel at height h can be written as,

(2)mch(h)=1W∑0≤i<Wtc(h,i),

Similarly, the output of the c-th channel at width w can be formulated as, (3)mcw(w)=1H∑0≤i<Htc(j,w).

The feature aggregation of formulas (2) and (3) in the generation process can be expressed as, (4)n=ζ(C(⟨mh,mw⟩)),

where ⟨⋅,⋅⟩ represents the concatenation them along the horizontal and vertical, ζ is a non-linear activation function, C is the convolution operation. Then, we divide n into horizontal nw and vertical nh. What’s more, Ch and Cw are 1 × 1 convolutions that transform nh and nw to have the same number of channels as the input T. The outputs kh and kw are then expanded and used as attention weights, respectively. Finally, the output O via attention can be formulated as, (5)kh=γ(Ch(nh)), (6)kw=γ(Cw(nw)), (7)Oc(i,j)=kch(i)×kcw(j)×tc(i,j),

where γ denotes the sigmoid function.

In our proposed model, the loss function is an integral part, supervising our side outputs as well as the final output, so that the prediction results are closer to the label map quickly. We denote the training set by S=(Rn,Kn),n=1,…,N, where Rn stands for the original image and Kn for the labeled image. W denotes all parameters of the whole network, and all the side output layers can be viewed as classifiers with parameter w(m). F¯ denotes the result of the model prediction. Then the side loss function Lside can be formulated as, (8)Lside(R,K,W,w)=∑m=13αmlsidem(R,K,W,w(m))=∑m=13αmθ(F¯,K,W,w(m)),

where θ denotes the modified cross-entropy function, αm is a hyper-parameters representing the loss weight for each side output layer, which can be rewritten as, (9)θ=−u0∑i∈G+log⁡Pr(ki=1|R;W,w(m))−u1∑i∈G−log⁡Pr(ki=0|R;W,w(m)),where u0,u1 denote the class loss weights for cracks and non-cracks, respectively, G+ and G− represent the total number of cracked and non-cracked pixels. Pr(⋅) means the probability of positive or negative for a pixel in the predicted map. ki∈{1,0} refers to the predicted results of the m-th side output layer. The final fusion loss obtained by concatenating all the side output layers can be expressed as, (10)Lfuse(R,K,W)=−u0∑i∈G+log⁡Pr(ki=1|R;W)−u1∑i∈G−log⁡Pr(ki=0|R;W),

Therefore, the total loss function of the model can be simplified as, (11)L=Lside(R,K,W,w)+Lfuse(R,K,W).

Both our proposed model and compared models are implemented on PyTorch, a public deep learning framework. In the proposed network, batch normalization and ReLU are used after each convolutional layer in order to make the model converge faster during training. In the model, the initial learning rate is set to 1e-4 and is reduced to 10 times every 50 epochs, the training epoch is set to 500. We adopt stochastic gradient descent (SGD) as the optimizer with weight decay (2e-4) and momentum (0.9). Experiments are implemented with the 4-core Inter (R) Xeno (R) Sliver CPU and the Tesla A100 40 GB GPU on the Ubuntu 16.04 system.

We train and test on the DeepCrack dataset, and verify the effectiveness of the model on CFD and Crack500 datasets. A concise introduction of each dataset is given below.

(1) DeepCrack [20]: This dataset contains 537 crack images with a resolution of 544 × 384. It is divided into two groups, the training set contains 300 images and the test set contains 234 images.

(2) CFD [39]: This dataset contains 118 crack images with a resolution of 480 × 320. Each image has its corresponding pixel-wise label image. Our model is tested on the CFD dataset.

(3) Crack500 [40]: This dataset contains 500 images with a resolution of 2000 × 1500. These images were divided into 3 sets, 250 training sets, 50 validation sets, and the remaining 200 as the test set.

(1) U-Net [11]: The model is composed of U-shaped encoder-decoder structure and skip connection layer.

(2) UHDN [37]: The prediction of the image is realized by encoder-decoder architecture with hierarchical feature learning and dilated convolution.

(3) DeepLabV3+ [15]: It is a combination of the advantages of the spatial pyramid pooling module and the encoder-decoder structure.

(4) DeepCrack [20]: This is a fully convolutional network and refines the result with guided filtering and conditional random fields.

(5) HACNet [16]: It uses a hybrid approach to concatenate atrous convolutions with different dilation rates to aggregate features.

(6) FPHBN [39]: The network aggregates contextual information into low-level features through a feature pyramid.

(7) CrackSegNet [38]: This network consists of backbone network, dilated convolution, spatial pyramid pooling and skip connection modules.

(8) CrackW-Net [36]: Based on U-Net, a skip-level round-trip sampling block is proposed.

Parameter count (Params) is an important evaluation criterion for a lightweight model. We measure the running speed of these models in frames per second (FPS). In addition, the Params and FLOPs of each convolution layer can be expressed as follows: (12)Params=(CinT2+1)Cout, (13)FLOPs=2HW(CinT2+1)Cout,where T denotes the size of convolution kernel, Cin and Cout indicate the nuber of channels for input and output feature maps, respectively. H and W respent the height and width of the feature map, respectively. These methods’ performance is evaluated using precision (P), recall (R), and F-score (F1), the pixel accuracy (PA), mean pixel accuracy (MA), mean intersection over union (MIoU). The equations are calculated as, (14)P=TPTP+FP, (15)R=TPTP+FN, (16)F1=2PRP+R,where TP denotes true positives, FP means false positives and FN refers false negatives. (17)PA=∑i=0kpii∑i=0k∑j=0kpij, (18)MA=1k+1∑i=0kpii∑j=0kpij,

(19)MIoU=1k+1∑i=0kpii∑j=0kpij+∑j=0kpji−pii,where k means the number of classes, pii denotes the number of pixels with classification i that are predicted to be i, while pij indicates the number of pixels with classification i but predicted to be j.

To evaluate the validity of our experiment, we introduce eight metrics used in crack detection: Precision, Recall, F-score, MIoU, Params, Training time, FPS, FLOPs. Where, Params, FPS and FLOPs evaluate the complexity of the model, and Training time denotes the time for the network to run one epoch.

(1) Results on DeepCrack

Fig. 5a displays the Precision-Recall curves on the DeepCrack dataset, with our model reaching the upper right corner. It exceeds current crack detection methods by achieving the highest F1 value of 0.874. As illustrated in Table 2, our method compared to FPHBN and CrackW-Net, shows an improvement of 5.3% and 6.6% on the F1 value, and an enhancement of 4.3% and 5.3% on the MIoU, respectively. The two methods mentioned above can be attributed to the utilization of distinct fusion techniques combined with encoder-decoder network structures. What’s more, our method achieves the highest MIoU value of 0.883. Fig. 6 presents the feature maps of test results on three datasets: DeepCrack, CFD, and Crack500, comparing our method with others. The first two rows are derived from the DeepCrack dataset, the middle two rows from the CFD dataset, and the final two rows from the Crack500 dataset. Notably, Fig. 6c illustrates the results from our model, where it can be observed that the edge details of the cracks are clearer and more abundant, showing the highest conformity with the label images. In our model, there are three sideoutput feature maps. Table 3 presents the evaluation metrics on the DeepCrack dataset, showing that all metrics gradually increase, with the results reaching their optimum after fusion.

(2) Results on CFD and Crack500

We use these two datasets to verify the generalization of our approach. The trained models on DeepCrack dataset are used to predict the maps on CFD and Crack500 datasets. As demonstrated in Table 4 our approach achieves the highest F1 value 0.640 and MIoU value 0.736 on CFD dataset, respectively. As shown in Table 5, our method reaches the highest F1 value 0.630 and MIoU value 0.725 on Crack500 dataset, separately. Compared to HACNet [16], our method also employs a similar atrous convolution extraction module. However, in contrast to the single encoder feature extraction used by HACNet, our dual encoder approach significantly enhances performance. In terms of the F1 and MIoU metrics, our method shows an improvement of 3.2% and 2.2%, respectively, over HACNet on the CFD dataset. On the Crack500 dataset, the F1 and MIoU improved by 5% and 3.2%, respectively. Figs. 5b and 5c show that our method has a better Precision-Recall curve than other methods on both the CFD and Crack500 datasets. Table 6 displays the sideoutput and post-fusion results for the CFD and Crack500 datasets. The fusion of the three sideoutputs achieved the highest F1 and MIoU.

(3) Model Complexity

In Table 7, Params and training time are shown. In terms of the Params metric, HACNet has the lowest at 0.21 M, showing a significant difference compared to other methods. However, the parameter count of our method is also modest at 1.95 M, closely aligning with that of HACNet. Notably, our training time is just 20 s, which represents a significant improvement in comparison. As shown in Table 8, the complexity of the model is evaluated using FPS and FLOPs. The above evaluation of the complexity of our model is based on the DeepCrack dataset. Benefiting from the lightweight design of the network architecture, our proposed method achieves the greatest training time value of 20 s and params only 1.95 M. We use lightweight modules CA, SFEM and LKAC to make the model much less complex than the comparison methods.

This section primarily serves to validate the effectiveness of each component of our model, which includes three principal modules. As shown in Table 9, crack detection can be effectively enhanced by proposed network architectures and modules. We used a shallow feature extraction module (SFEM) as a feature extraction module and take large kernel atrous convolution (LKAC) as another feature extraction module, combined them and added coordination attention (CA) successively. As can be seen from the results in Table 9, the F1 values of the two feature extraction modules are very similar on the DeepCrack dataset, and the F1 value increased by about 1% when they are combined. But the F1 value decreased by 2% and 5% because of adding CA to the LKAC network in DeepCrack and CFD, respectively. In order to reduce the non-crack information in the results after the combination of SFEM and LKAC, we try to add CA module to filter the characteristic information. In Table 9, the LKAC module exhibits a slightly higher precision on the CFD dataset compared to our model. This is attributed to the LKAC module primarily utilizing atrous convolution for crack feature extraction, without employing upsampling and pooling layers, thus leading to less information loss and a slight improvement in accuracy.