The following is the setting of FPN used to construct the feature pyramid. The original feature representations are denoted as P={P3,P4,P5}, which correspond to the multilevel feature maps 𝒞={𝒞3,𝒞4,𝒞5} with predetermined strides {8, 16, 32} in feature hierarchy of the input image. To address the challenges that lead to defect missed detection due to insulators with small defect objects, low proportion of pixels occupied, and serious loss of detailed features during convolution down samplings, we introduce the high-resolution feature map 𝒞2 after two down samplings in the backbone network and fuse it with P2 of FPN. This way can enhance the detailed features of small objects in the neck network and alleviate the situation of serious loss of the detailed features in the deeper layers. The specific steps are: (1) P3 is upsampled and then feature fused with 𝒞2 to obtain P2 which increases the detail features in the neck network that are favorable for small object detection; (2) the feature map P2 is input into the YOLOv5 head after convolution, batch normalization, and Leaky ReLU activation, which ultimately improves the detection of tiny objects.

To tackle the issue of insufficient feature fusion across different scales, which hampers the network of ability to adapt to significant variations in object scales, S-ASFF is added to the neck network, inspired by the Adaptive Spatial Feature Fusion (ASFF) [37] module. In addition, to reduce the number of parameters, we remove the feature fusion layers of ASFF-1, ASFF-2, and ASFF-3. By utilizing the feature fusion mechanism of S-ASFF, the model can adaptively learn the weights of the different scale feature layers of the FPN at each same position, so that the most important feature layers dominate the fusion. The comparison between the ASFF module and the S-ASFF module is shown in Fig. 2. The original ASFF contains three fusion layers while S-ASFF has only one fusion layer which integrates the feature layer P2 of FPN with more detailed features. The S-ASFF can fuse the bottom feature maps with higher resolution in the FPN so that it can improve the detection ability of smaller objects.

As shown in Fig. 3, S-ASFF is specifically divided into three steps: (1) up-sampling P={P3,P4,P5} to expand the scale to ensure that the scales of each feature layer are consistent in the process of fusion; (2) spatial filtering for each position after up-sampling to learn the relative importance of different scale layers and to enhance the perceptibility of the scales; and (3) spatial fusion across scales to enhance the adaptability to changes in scales. As shown in Fig. 3, where P5 is the highest layer and P2 is the lowest layer. After up-sampling, the feature maps of each layer are denoted as xk,k∈[2,5].

Both scale dimension and spatial dimension are taken into account in S-ASFF which processes spatial weights to the scaled feature maps at each level. A softmax activation function with a control factor ρ is used to calculate the spatial mask indicating the relative importance of the corresponding positions for the layers across the scales. Taking layer of x5 as an example, the formula for calculating a mask value at pixel (i, j) of the 5-th layer is as follows:

(1)αij5=eρij5eρij2+eρij3+eρij4+eρij5,where each pixel corresponds to a control factor ρij to generate αij. The αij for levels referred to as [αij2,αij3,αij4,αij5].

For aggregation at scales and filtering of conflicts in space, the formula is as follows:

(2)yij′=αij2⋅xij2+αij3⋅xij3+αij4⋅xij4+αij5⋅xij5,where xijk, k ∈ [2,5] denotes the feature tensor at pixel (i, j) and αij2+αij3+αij4+αij5=1. yij′ denotes a feature tensor as output after being processed by S-ASFF.

To enhance the transform modeling ability of the convolution neural network, the model can adaptively adjust the shape of the convolution kernel to adapt to object features with different morphologies, to enhance the localization ability of insulator defects with fuzzy edge morphology, and then the missed detection of insulator defects is reduced. EDAM is introduced to improve the perception of object morphology. As shown in Fig. 4, the given feature tensor F∈R𝒞×H×W is the feature space based on S-ASFF, and after a general convolution, the feature map y is obtained. ⊙ indicates that K sampling points of a pixel are matched with the attention mask value, respectively, and ∑k=1K⊙ indicates that K sampling points are paid attention to. The adaptive sampling process is a self-learning process to obtain the offset when the network is trained. EDAM enhances the attention ability of the DAM so that the model can focus on more meaningful locations. Specifically, it zeroes in on the attention values of unimportant sample points by fusing the gating activation function, prompting the model to transition from learning a limited set of sample points near a reference point to focusing on a few crucial sample points within that set.

The mask is obtained based on the gating activation function ∅. This activation function zeroes out the negative weights so that the network can learn how to enhance expansion ways for the perceptual domain of attention, extracting effective discriminant features. The formula is represented as follows:

(3)∅(x)=max(0,tanh(x−φ)+tanh(φ)1+tanh(φ))∈[0,1],where x is the feature map after convolution, φ is a predefined hyperparameter and tanh is the hyperbolic tangent function.

Based on the features in the across-scale fusion space, the output after attention for K sampling points on a pixel p0 is represented as follows:

(4)F˙(p0)=∑k=1K⁡Ak⋅wk⋅x(p0+pk+Δpk),where F˙ is the feature space for EDAM. For a pixel p0 on the feature map y after general convolution, K is the total number of sampling points on a pixel p0. We set K=|ℛ|. The range of the convolution kernel is defined by ℛ. In this paper, ℛ={((−1,−1).(−1,0),…,(0,1),(1,1))}, which is represented that 3×3 convolution kernel with dilation 1. Ak is a mask value for the k-th sampling point by gating activation function ∅. pk+Δpk is the offset position of the k-th sampling point. It is worth noting that when the offset coordinates are floating point numbers, we use the nearest neighbor interpolation method to round the offset coordinates to obtain the revised coordinates and determine the offset position pk+Δpk.

This paper utilizes a dataset of 1600 raw images. We first collated insulator images containing flashover and breakage from the Electric Power Research Institute (EPRI) and public datasets UPID [38,39]. And then we performed a pre-processing operation in images: Cleaning up the damaged images and resizing them. In addition, image flipping, saturation adjustment, contrast adjustment, and noise addition were used to expand the data. Second, a dataset in YOLO format was produced for training and evaluation. Specifically, the images were labeled using the Labelimg tool to obtain the object categories and coordinate information of ground truth. There are three object classes: Pollution_flashover, broken, and insulator. According to statistics, the sample size of these three classes is 1994, 861, and 1466 in order. The dataset was partitioned into a training set and a test set, with an 8:2 ratio.

The label correlogram for objects of each size in the dataset is shown in Fig. 5. It can help identify patterns or correlations in the distribution of object annotations across different classes and scales. It reveals if certain classes are more likely to appear at specific scales [40]. x and y are the coordinates of the bounding box of the object, and width and height are the size of the bounding box of the object, respectively. From the three coordinate graphs with red labeled boxes, it can be shown that most of the bounding box widths and heights are less than 1/4 or 1/8 of the original size of the entire image. However, because the size of the feeding neural network is fixed at 640 × 640, most of the object pixels in the actual trained images take up a smaller proportion. Overall, the object bounding boxes are distributed in all parts of the x-axis, indicating that the object scale varies greatly. In addition, this dataset covers most of the defect scenes in practical applications which makes the trained model generalizable. To further reduce the network overfitting problem, the model was enhanced with a mosaic data enhancement method for the samples in the dataset before training. Furthermore, data enhancement strategies such as random scaling and random cropping were used to improve the model classification performance.

For a more comprehensive evaluation of the model, the paper employed three metrics: Precision (P), Recall (R), and F1-Score, to account for the comprehensive prediction of breakage and flashover. Additionally, mAP was utilized as an overarching performance metric to characterize and assess the model’s quality. All the experiments in this paper were conducted in the hardware environment shown in Table 1.

The network was fed with images of size 640 × 640, and several key parameters were configured. The batch size was set to 8, momentum to 0.937, initial learning rate to 0.01, and weight decay to 0.0005. The network was trained from scratch for 300 epochs. For the constructed dataset, the number of object classes is inevitably unbalanced. To alleviate the impact of this problem, we adopted YOLOv5 with a class imbalance strategy as the baseline. This strategy is proposed by YOLOv5. The setting of class weights and image weights was introduced. The class weights require calculating the number of labels for each class in the dataset and then taking the reciprocal of the number of class labels. In other words, the greater the number of labels of a certain class, the smaller the weight in the image containing that class. Calculating the sum of all the class weights contained in an image is the image weight. The greater the image weight, the greater the probability of the image being sampled. In particular, the image is selected according to the image weight by random selection method and the number of images is the same as that of the training set. This means that the larger image weight will be more likely to be selected for training. This method can increase the training proportion of the small number of object classes during training so that the model can learn the features of each object class more balanced and prevent over-fitting. The training results are depicted in Fig. 6. As the number of training epochs gradually increased to 300, the loss curves stabilized, indicating effective training.

In this section, we assessed the detection capabilities of our method. For a fair comparison, we re-implemented the relevant models and employed the same evaluation dataset to calculate performance metrics, including precision, recall, and F1-Score, for the different models. All results are presented in Table 2. The detection model based on Faster RCNN and SSD has low detection precision and recall for flashover and breakage as small objects due to the limitations of the basic network framework, so they are difficult to apply in practice. Compared with YOLOv5s, both YOLOv7 and YOLOv8s have better performance. However, when applied to YOLOv5s, YOLOv7, and YOLOv8s, respectively, our proposed method can bring them greater precision gains and recall rate gains. In particular, when the YOLOv5s baseline was combined with our proposed method, the precision on flashover and breakage defects was improved by 5.7% and 4.7%, respectively. This exceeds the YOLOv8s baseline, even the model YOLOv8s + Ours, and significantly narrows the performance gap with YOLOv7. As for the higher gains of our method on YOLOv5, the possible reason is that the YOLOv5 model is not robust enough. There are more robust network structures and performance due to the optimization and improvement of YOLOv7 and YOLOv8 in the aspects of model feature extraction and strategy for matching positive and negative samples.

We also provide the deployment performance of each model in practical applications, as shown in Table 3. In the task of power line inspection, the deployment of deep learning models is a comprehensive process involving several key factors. How efficiently the model is deployed on specific hardware for optimal performance is critical. YOLOv7 and YOLOv8s are higher than YOLOv5s in FLOPs, Parameters, and Model Size. For deploying models on hardware devices that do not support GPU, YOLOv5s is more suitable. However, the YOLOv5s is significantly inferior to the YOLOv5s + Ours in terms of precision. In actual model deployment, compared with YOLOv7 + Ours and YOLOv8s + Ours, YOLOv5s + Ours has the advantages of small model size, high precision, low computational efficiency and flexible deployment. If optimal precision needs to be considered, YOLOv7 as the baseline can be chosen to detect flashover and breakage defects of power line insulators.

The results show that our proposed method is effective and lightweight in solving small objects, scale variations, and weak edge morphological features. Firstly, for strategy adjustment of the YOLOv5 network structure, although this strategy only uses simply high-resolution feature map, it brings two major improvement advantages to the model: (1) reduces the loss of details of small objects; (2) brings smaller object-scale information to the second component (S-ASFF). This balances the preference for learning large and small objects, allowing the model to learn tiny object features that are more difficult to learn. Secondly, for S-ASFF, the improved performance gains benefit from the fact that the most important scale layer dominates each position in the feature pyramid with minimal computational cost. For better fusion, we weigh the scale to which the feature pyramid should be scaled with the least computational cost while preserving as much of the small object information as possible from the first proposed component (strategy). Other fusion ways all cause secondary loss of small object information or increase unnecessary computing costs. Finally, due to EDAM introducing the gating activation function ∅ with tanh instead of the sigmoid in DAM, EDAM can suppress all attention values with negative input to 0, which is similar to the negative half region of the ReLU. This will increase the sparsity of attention. Compared to DAM which indiscriminately learns from a small group of sampling points near a certain pixel point, EDAM can focus on more meaningful location learning discriminant features, resulting in improved performance. In addition, EDAM alleviates gradient dispersion because the sensitive range of gradient change is expanded from [0,0.25] to [0,1], and the gradient is more stable during training. Furthermore, our method is a more generic variant method compared to the existing solutions on the YOLO series, which can improve the performance of models in the YOLO family.

All in all, in this paper, a general enhancement method is designed, which reasonably uses high-resolution feature maps and can simply and effectively improve the adaptability of variable multi-scale features and the perception of weak edge features with less increase of parameters and computation. It significantly enhances the detection performance of the YOLO series on difficult small objects. The missing detection of defects in power lines is reduced.

To verify the detection performance of our model on large-scale objects and make the results more convincing, we trained and tested various mainstream algorithms on the public dataset UPID and compared them with the IFD dataset constructed in this paper. As shown in Table 4. For large-scale objects, the AP of our model on both the IFD dataset and the UPID public dataset is best.

Object detection was performed on the test set and the results were visualized. Fig. 7 shows some of the results. The experiment proves that the detection model has strong generalization capability and wide application potential.

In this section, we conducted essential ablation experiments to quantitatively analyze the effectiveness of the various equipment proposed. We gradually introduced the relevant modules into the network during training. YOLOv5s served as the baseline method for all ablation studies. For convenience, the strategy adjustment of YOLOv5s network structure is denoted as SAStrgy. The results are shown in Table 5.

We note that directly adding an SAStrgy makes slight degradation for the performance of breakage but improves the precision of flashover. The result suggests an unstable effect. We consider possible reasons for this: The plain YOLOv5s has no advantage for the detection of small objects due to variable scale and weak edge features besides complex backgrounds. The method has difficulty in precise judgment for the defect location and damage degree. For specific defect detection tasks, further optimized strategies are necessary to improve detection performance.

The results show that S-ASFF and EDAM, based on the addition of a SAStrgy, boost the performance of the baseline: The detection precision of flashover is improved by 3.01% and that of breakage by 4.58%. However, EDAM improves the model performance most significantly. The detection precision of flashover is boosted by 4.71% and recall by 2.96% over YOLOv5s. The detection precision of breakage is increased by 5.74% and recall by 1.05%. This is because S-ASFF provides key details to EDAM and reduces interference from complex backgrounds. It enables EDAM to more accurately expand the perceptual domain of attention and extract discriminative features.

To illustrate the benefits of our model more visually. This is shown in Fig. 8. The first row shows an example of breakage and the second row shows an example of flashover. The layer23 will be further used to predict small objects. It is used together with layer 26 and layer 29 as input to yolo heads for classification and regression tasks. Before S-ASFF (as shown in layer 21), features are retained in the object regions because of the stronger semantic information. On the contrary, after the object scale perception of S-ASFF (as shown in layer 22), features are extracted around each object. Compared with layer 21, large-scale insulators and small-scale defect (flashovers and breakages) objects significantly improve the feature distinction in scale and are more sensitive to location. After EDAM (as shown in layer 23), distinguishing features are highlighted from key information retained by the previous module, and object sizes and edge morphological features are precisely learned. At the same time, one may notice that the defect objects can be learned in both layer 26 and layer 29 because YOLOv5 allows ground truth boxes to perform anchor matching in all prediction layers simultaneously to increase the number of positive samples. However, layer 23 contains more accurate small object size and richer edge morphology. Therefore, for SAStrgy, adding a tiny object prediction layer to directly predict the feature map of layer23 is conducive to improving the precision of smaller objects, especially for detecting difficult small objects.

We also recorded the testing time of our method based on YOLOv5. Specifically, with a batch size of 16, our method achieved an inference time of 6.1 ms and a non-maximum suppression (NMS) time of 0.9 ms. Consequently, the total processing time for an image with a size of 640 pixels was 7.1 ms.