<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.1 20151215//EN" "http://jats.nlm.nih.gov/publishing/1.1/JATS-journalpublishing1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:mml="http://www.w3.org/1998/Math/MathML" xml:lang="en" article-type="research-article" dtd-version="1.1">
<front>
<journal-meta>
<journal-id journal-id-type="pmc">CMC</journal-id>
<journal-id journal-id-type="nlm-ta">CMC</journal-id>
<journal-id journal-id-type="publisher-id">CMC</journal-id>
<journal-title-group>
<journal-title>Computers, Materials &#x0026; Continua</journal-title>
</journal-title-group>
<issn pub-type="epub">1546-2226</issn>
<issn pub-type="ppub">1546-2218</issn>
<publisher>
<publisher-name>Tech Science Press</publisher-name>
<publisher-loc>USA</publisher-loc>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="publisher-id">42313</article-id>
<article-id pub-id-type="doi">10.32604/cmc.2023.042313</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Article</subject>
</subj-group>
</article-categories>
<title-group>
<article-title>Visualization for Explanation of Deep Learning-Based Fault Diagnosis Model Using Class Activation Map</article-title>
<alt-title alt-title-type="left-running-head">Visualization for Explanation of Deep Learning-Based Fault Diagnosis Model Using Class Activation Map</alt-title>
<alt-title alt-title-type="right-running-head">Visualization for Explanation of Deep Learning-Based Fault Diagnosis Model Using Class Activation Map</alt-title>
</title-group>
<contrib-group>
<contrib id="author-2" contrib-type="author">
<name name-style="western"><surname>Guo</surname><given-names>Youming</given-names></name></contrib>
<contrib id="author-1" contrib-type="author" corresp="yes">
<name name-style="western"><surname>Wu</surname><given-names>Qinmu</given-names></name><email>qmwu@gzu.edu.cn</email></contrib>
<aff><institution>School of Electrical Engineering, Guizhou University</institution>, <addr-line>Guiyang, 550025</addr-line>, <country>China</country></aff>
</contrib-group>
<author-notes>
<corresp id="cor1"><label>&#x002A;</label>Corresponding Author: Qinmu Wu. Email: <email>qmwu@gzu.edu.cn</email></corresp>
</author-notes>
<pub-date date-type="collection" publication-format="electronic"><year>2023</year></pub-date>
<pub-date date-type="pub" publication-format="electronic"><day>29</day><month>11</month><year>2023</year></pub-date>
<volume>77</volume>
<issue>2</issue>
<fpage>1489</fpage>
<lpage>1514</lpage>
<history>
<date date-type="received">
<day>25</day><month>5</month><year>2023</year>
</date>
<date date-type="accepted">
<day>11</day><month>8</month><year>2023</year>
</date>
</history>
<permissions>
<copyright-statement>&#x00A9; 2023 Guo and Wu</copyright-statement>
<copyright-year>2023</copyright-year>
<copyright-holder>Guo and Wu</copyright-holder>
<license xlink:href="https://creativecommons.org/licenses/by/4.0/">
<license-p>This work is licensed under a <ext-link ext-link-type="uri" xlink:type="simple" xlink:href="https://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</ext-link>, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
</license>
</permissions>
<self-uri content-type="pdf" xlink:href="TSP_CMC_42313.pdf"></self-uri>
<abstract>
<p>Permanent magnet synchronous motor (PMSM) is widely used in various production processes because of its high efficiency, fast reaction time, and high power density. With the continuous promotion of new energy vehicles, timely detection of PMSM faults can significantly reduce the accident rate of new energy vehicles, further enhance consumers&#x2019; trust in their safety, and thus promote their popularity. Existing fault diagnosis methods based on deep learning can only distinguish different PMSM faults and cannot interpret and analyze them. Convolutional neural networks (CNN) show remarkable accuracy in image data analysis. However, due to the &#x201C;black box&#x201D; problem in deep learning models, the diagnostic results regarding providing accurate information to the user are uncertain. This paper proposes a motor fault diagnosis method based on improved deep residual network (ResNet) and gradient-weighted class activation mapping (Grad-CAM) to analyze demagnetization and eccentricity faults of permanent magnet synchronous motors, and the uncertainty limitation of fault diagnosis based on the convolutional neural network is overcome by the visual interpretation method. The improved ResNet is formed by using ResNet9 as the backbone network, replacing the last convolution layer with a atrous spatial pyramid pooling (ASPP), and adding a multi-scale feature fusion module and attention channel mechanism (CAM). The proposed model not only retains the effective extraction of image features by ResNet9 but also enhances the sensitivity field of the network through the hollow convolution pyramid and realizes the feature fusion of the web on different scales through the multi-scale feature fusion module (MSFFM), further improving the diagnostic accuracy of the network on different types of fault features. The diagnostic effect of the network is verified on the self-made data set, which mainly includes five states: normal (He), 25% demagnetization (De25), 50% demagnetization (De50), 10% static eccentricity (Se10), and 20% static eccentricity (Se20). The number of pictures in the training set is 6000, and the number in the test set is 1500. The average diagnostic accuracy of the improved ResNet on this dataset is 99.00%, which is 1.04%, 8.89%, 4.58%, and 7.22% higher than that of the multi-column convolutional neural network (MCNN), Bi-directional long short-term memory (Bi-LSTM), deep belief network (DBN), and recurrent neural network (RNN) models, respectively. Finally, gradient activation heat maps were used to globally average pool the final output feature map of the network to obtain feature weights. They were superimposed with the original image to get gradient activation heat maps of different grayscale images. The warmer the tone of the heat map, the greater the impact on the network diagnosis results, and then the demagnetization and eccentricity fault characteristics of the permanent magnet synchronous motor were determined-visual characterization of quantitative analysis.</p>
</abstract>
<kwd-group kwd-group-type="author">
<kwd>Permanent magnet synchronous motor</kwd>
<kwd>convolutional neural network</kwd>
<kwd>fast fourier transform</kwd>
<kwd>ASPP</kwd>
<kwd>Grad-CAM</kwd>
<kwd>fault diagnosis</kwd>
</kwd-group>
<funding-group>
<award-group id="awg1">
<funding-source>National Natural Science Foundation of China</funding-source>
<award-id>51867006</award-id>
<award-id>51867007</award-id>
</award-group>
<award-group id="awg2">
<funding-source>Natural Science and Technology Foundation of the Guizhou Province</funding-source>
<award-id>[2018]5781</award-id>
<award-id>[2018]1029</award-id>
</award-group>
</funding-group>
</article-meta>
</front>
<body>
<sec id="s1">
<label>1</label>
<title>Introduction</title>
<p>Due to its wide range of constant power speed, better dynamic performance, easy maintenance, and other advantages [<xref ref-type="bibr" rid="ref-1">1</xref>], permanent magnet synchronous motor has been widely used in social production because of its extensive speed range, excellent dynamic performance, and relatively simple structure [<xref ref-type="bibr" rid="ref-2">2</xref>]. Therefore, it is of great social and economic significance to analyze and diagnose the faults of PMSM.</p>
<p>To date, the diagnosis methods of motor faults are mainly based on three aspects: model, signal, and data-driven method [<xref ref-type="bibr" rid="ref-3">3</xref>]. The model-based method usually requires establishing a mathematical model and using equations to describe the troubled motor&#x2019;s rigid structure accurately. Modeling methods include methods based on classical state estimation or process parameter estimation [<xref ref-type="bibr" rid="ref-4">4</xref>], finite element method, etc. [<xref ref-type="bibr" rid="ref-5">5</xref>,<xref ref-type="bibr" rid="ref-6">6</xref>]. This method conducts fault analysis based on the model, which has the advantage of going deep into the essence of motor operation but has the disadvantage of relying on an accurate mathematical model of the motor. The second approach is signal-based and does not rely on precise mathematical models. At present, there are several methods to diagnose different types of motor faults by signal: (1) Spectrum analysis; (2) Vector decomposition method [<xref ref-type="bibr" rid="ref-7">7</xref>]; (3) Time-frequency analysis method [<xref ref-type="bibr" rid="ref-8">8</xref>], etc. But these methods require corresponding expertise. The method of motor fault diagnosis based on data drive originates from the extensive application and progress of massive AI technologies in different fields [<xref ref-type="bibr" rid="ref-9">9</xref>], such as aerospace [<xref ref-type="bibr" rid="ref-10">10</xref>], industrial production [<xref ref-type="bibr" rid="ref-11">11</xref>] and construction engineering [<xref ref-type="bibr" rid="ref-12">12</xref>], and so on. Unlike the signal-based fault diagnosis method, a neural network does not need the corresponding expert knowledge. It only needs to import a large amount of data to train the neural network. After its related parameters are fixed, the fault diagnosis of different types of motor faults can be carried out.</p>
<p>Various deep and machine-learning algorithms have been widely used in mechanical fault diagnoses [<xref ref-type="bibr" rid="ref-13">13</xref>]. Literature [<xref ref-type="bibr" rid="ref-14">14</xref>] and literature [<xref ref-type="bibr" rid="ref-15">15</xref>] proposed dislocated time series convolutional neural network (DTS-CNN) and hierarchical convolutional neural networks (TICNN) to detect motor bearing faults by processing motor vibration signals. In literature [<xref ref-type="bibr" rid="ref-16">16</xref>], the stator current of PMSM is transformed by fast fourier transform (FFT), and its amplitude is imported into one-dimensional CNN for training to diagnose the demagnetization fault and bearing fault of PMSM. Tamilselvan and Wang proposed a fault diagnosis method of DNN based on a constrained Boltzmann machine is offered. Still, this method requires many sensors to collect related signals. Each layer is treated as a deep network structure in this method and trained. Although this method has an excellent diagnostic effect on machine faults, it has a lot of background noise which is difficult to eliminate.</p>
<p>Literature [<xref ref-type="bibr" rid="ref-17">17</xref>] proposed an uncrewed aerial vehicle (UAV) image target detection based on the improved you only look once (YOLO) algorithm. From the perspective of model efficiency, it effectively solved the real-time target detection problem of multi-scale targets by using target box dimension clustering, pre-training network classification, multi-scale detection training, and changing the screening rules of candidate boxes. Literature [<xref ref-type="bibr" rid="ref-18">18</xref>] proposed an automatic weed detection system based on UAV images based on CNN. The overall accuracy of the convolutional neural network-learning vector quantization (CNN-LVQ) model developed for weed detection was significantly improved through strict hyperparameter adjustment. Literature [<xref ref-type="bibr" rid="ref-19">19</xref>] optimized the climate depth long short-term memory (CDLSTM) model for predicting temperature and rainfall in Himalayan states. It uses Facebook&#x2019;s FB-Prophet model to predict and evaluate the performance of the developed CDLSTM model. It has higher accuracy and a lower error rate than the previous algorithm. Literature [<xref ref-type="bibr" rid="ref-20">20</xref>] proposed a brain tumor classification model (DCNNBT) based on a new deep convolutional neural network. By scaling the image resolution, depth of layers, and width of channels and strictly optimizing the hyperparameters, the model achieved a classification accuracy of up to 99.18%, which was significantly higher than other studies based on the same database. Literature [<xref ref-type="bibr" rid="ref-21">21</xref>] proposed the application of a five-layer CNN to large-scale plant classification in the natural environment. Plant identification based on experimental data has high accuracy. The model achieved the highest recognition rate of 96% on the NU108 dataset, and the accuracy of NU101 drone images reached 97.8%. However, most fault diagnosis methods based on deep learning are black boxes. It is impossible to know which image features have been classified based on the neural network, and the mapping established by the neural network needs descriptive analysis. Therefore, a complete CNN and Grad-CAM method for motor fault diagnosis is proposed in this paper, and demagnetizing faults and eccentricity faults of PMSMs are diagnosed.</p>
<p>The major innovations of the article are as follows:</p>
<p>1. Multi-scale feature fusion module, attention mechanism, and atrous convolution pyramid are added to the traditional ResNet network, which enhances the feature extraction capability of the model and improves the fault diagnosis accuracy of the motor. In this paper, a multi-scale feature fusion module, attention mechanism, and atrous convolution pyramid are added to the traditional ResNet network, which enhances the feature extraction capability of the model and improves the fault diagnosis accuracy of the motor.</p>
<p>2. Grad-CAM is introduced to solve the &#x201C;black box&#x201D; problem in the fault diagnosis process of the CNN, and the diagnosis based on the features of the image is studied, and the original frequency signal corresponding to the features on the image is determined.</p>
<p>The rest of this paper is arranged as follows: The second section introduces the establishment of symmetrical and asymmetric PMSM models on the finite element software AltairFlux, and co-simulation with MATLAB to obtain the current, torque and speed data of different faulty motors during operation. The third section converts the time-domain signals of stator currents of other defective motors into frequency-domain signals and converts the frequency-domain signals into grayscale images to make data sets and convolutional neural network architecture based on attention channel, multi-scale feature fusion module, and ASPP was constructed, and training parameters were configured. In the fourth section, the produced data set is imported into the built CNN for training and compared with other algorithms. Then, Grad-cam was used to analyze the gray image of the demagnetization fault and eccentricity fault and find out the fault characteristics of the demagnetization fault and eccentricity fault.</p>
</sec>
<sec id="s2">
<label>2</label>
<title>Build Different Types of Fault Models of PMSM</title>
<p>AFinite element modeling software-Altair Flux is often used for modeling and simulation in the electromagnetic field [<xref ref-type="bibr" rid="ref-22">22</xref>]. Taking it to create a model can be divided into four steps: establishing the motor geometric model, setting the physical properties, setting the model-solving parameters and solving them, and processing the results. At every step of the process, Flux will save relevant files. As long as the &#x201C;post-processing&#x201D; is performed to visualize or save the results, the solution results of each parameter can be obtained.</p>
<p>Motor faults can be divided into electrical defects, mechanical faults, magnetic faults, and so on. In this article, the static eccentric responsibility (Se10, Se20) and the demagnetization fault (De25, De50) are taken as examples, and the CNN is built, and the feature visualization technology is used to diagnose these two kinds of faults.</p>
<p>Based on the geometric parameters of PMSM in the Flux official tutorial, this paper redesigned the electrical parts of the motor and the structure of the permanent magnet and established an 8-pole, 48-slot PMSM model, whose structural parameters are shown in <xref ref-type="table" rid="table-1">Table 1</xref>.</p>
<table-wrap id="table-1">
<label>Table 1</label>
<caption>
<title>Geometric parameters of PMSM model</title>
</caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th colspan="2" align="center">Motor parameter</th>
</tr>
</thead>
<tbody>
<tr>
<td>Number of stator slots</td>
<td>48</td>
</tr>
<tr>
<td>Air gap length (m)</td>
<td>0.0008</td>
</tr>
<tr>
<td>Stator outer diameter (m)</td>
<td>0.141</td>
</tr>
<tr>
<td>The outer diameter of the rotor (m)</td>
<td>0.092</td>
</tr>
<tr>
<td>Rotor shaft radius (m)</td>
<td>0.056</td>
</tr>
<tr>
<td>Depth of groove (m)</td>
<td>0.003</td>
</tr>
<tr>
<td>Winding phase number</td>
<td>3</td>
</tr>
<tr>
<td>Winding shape</td>
<td>Single-pole double-layer winding</td>
</tr>
<tr>
<td>Pole number</td>
<td>8</td>
</tr>
<tr>
<td>Type of embedded magnet</td>
<td>V</td>
</tr>
<tr>
<td>Magnet material</td>
<td>NdFeB</td>
</tr>
<tr>
<td>Permanent magnet width (m)</td>
<td>0.0054</td>
</tr>
<tr>
<td>The thickness of the permanent magnet (m)</td>
<td>0.0005</td>
</tr>
<tr>
<td>Maximum motor power (kW)</td>
<td>60</td>
</tr>
<tr>
<td>Top motor speed (rpm)</td>
<td>4000</td>
</tr>
</tbody>
</table>
</table-wrap>
<sec id="s2_1">
<label>2.1</label>
<title>Establishment of PMSM Model for Demagnetization Fault and Eccentricity Fault</title>
<p>When the motor appears to have an eccentric weakness, its topology will become asymmetric. When the strange fault model of the engine is built in Flux, the intact geometric construction of the PMSM needs to be contrived. In this paper, 10% and 20% static eccentricity fault models of permanent magnet synchronous motors are built in Flux. As shown in <xref ref-type="fig" rid="fig-2">Fig. 2a</xref>, the coordinates Or and Os of the stator center of PMSM and the rotor axis are set as (0.06,0) and (0,0), respectively, so that the distance between them is 0.06 mm (uniform air gap length is 0.6 mm). In this way, the motor model with a 10% static eccentricity fault is obtained <xref ref-type="fig" rid="fig-1">Fig. 1b</xref> shows the setting method of the PMSM model for 20% eccentric fault. That is, the fixed and rotor axes are set as (0.12,0) and (0.0), respectively, so that the distance between the fixed and rotor axes is 0.12 mm. Then in mechanical properties, set the center of rotation of the rotor to (0.0) so that the center of rotation of the rotor coincides with the axis of the rotor. Thus, the PMSM model with 10% and 20% static eccentricity faults is obtained.</p>
<fig id="fig-1">
<label>Figure 1</label>
<caption>
<title>Static eccentric fault motor model: (a) 10% static eccentric; (b) 20% static eccentricity</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-1.tif"/>
</fig>
<fig id="fig-2">
<label>Figure 2</label>
<caption>
<title>Magnetic curves of NdFeB and its demagnetized materials (operating temperature: 20&#x00B0;C)</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-2.tif"/>
</fig>
<p>Due to its high energy density and low price, neodymium boron (NdFeB) is commonly used to make permanent magnets for permanent magnet synchronous motors. However, it also has the disadvantages that the magnetization intensity is greatly affected by working conditions, and the demagnetization is irreversible. The demagnetization failure of PMSM is usually uniform demagnetization of permanent magnets.</p>
<p>In Flux, it can be described by &#x201C;Remanent flux density&#x201D; and &#x201C;Relative permeability&#x201D; to the type of permanent magnet and the residual magnetic of permanent magnet materials Br, the slope of the magnetic curve is defined. In this paper, three magnetic curves are defined respectively, which are the magnetic curves of ordinary (He), 25% (De25), and 50% (De50) demagnetized materials. An PMSM model with demagnetization faults of different degrees in permanent magnets on opposite pairs is established.</p>
<p>As shown in <xref ref-type="fig" rid="fig-2">Fig. 2</xref>, yellow, blue, and red lines in three different colors represent normal, 25% demagnetization, and 50% demagnetization magnetic profile curves of permanent magnets. Their remanence Br can be set to 1.2, 0.9, and 0.6 T, respectively, and their permeability can be set to 1.05. <xref ref-type="fig" rid="fig-3">Figs. 3a</xref> and <xref ref-type="fig" rid="fig-3">3b</xref> respectively represent the sub-densities of permanent magnets with 25% and 50% demagnetization. It can be seen from the figure that the more profound the demagnetization degree of permanent magnets, the more sparse the magnetic densities.</p>
<fig id="fig-3">
<label>Figure 3</label>
<caption>
<title>Magnetic density distribution diagram of the permanent magnet of the motor with demagnetizing failure: (a) Demagnetizing 25%; (b) Demagnetizing 50%</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-3.tif"/>
</fig>
</sec>
<sec id="s2_2">
<label>2.2</label>
<title>Co-Simulation and Result Analysis</title>
<p>In this paper, finite element models of different types of PMSM established in Altair Flux were imported into a vector control system in MATLAB for co-simulation. Current strategies used in the vector control system <inline-formula id="ieqn-1"><mml:math id="mml-ieqn-1"><mml:msub><mml:mi>i</mml:mi><mml:mrow><mml:mi>d</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:math></inline-formula> and using the space vector pulse width modulation (SVPWM) motor power supply voltage vector modulation algorithm. <xref ref-type="fig" rid="fig-4">Fig. 4</xref> shows the PMSM finite element model in a loop simulation system co-simulated by AltairFlux and MATLAB-Simulink [<xref ref-type="bibr" rid="ref-23">23</xref>].</p>
<fig id="fig-4">
<label>Figure 4</label>
<caption>
<title>Magnetic curves of NdFeB and its demagnetized materials (operating temperature: 20&#x00B0;C)</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-4.tif"/>
</fig>
<p><xref ref-type="fig" rid="fig-5">Fig. 5</xref> respectively shows the stator three-phase current simulation results of 25% demagnetization, 50% demagnetization, and normal PMSM at a rated speed of 4000 rpm and load 0 N. <xref ref-type="fig" rid="fig-5">Figs. 5a</xref>&#x2013;<xref ref-type="fig" rid="fig-5">5c</xref> are the simulation results of stator currents of three types of motors, respectively. It can be seen from the figure that demagnetization of the engine will increase the time for the motor to enter the steady state. <xref ref-type="fig" rid="fig-5">Fig. 5d</xref> shows the speed variation of the three motors. It can be seen that demagnetizing failure will increase the time required for the motor speed to reach a steady state. <xref ref-type="fig" rid="fig-5">Fig. 5e</xref> shows the electromagnetic torque comparison. In the same way, the greater the demagnetizing intensity, the longer the adjustment time required for the motor to reach a steady state and the smaller the electromagnetic torque. In summary, demagnetizing failure is equivalent to increasing the engine load, thus increasing the adjustment time for the motor to reach a steady state.</p>
<fig id="fig-5">
<label>Figure 5</label>
<caption>
<title>PMSM Simulation results: (a) De25 PMSM stator current; (b) De50 PMSM stator current; (c) Normal PMSM stator current; (d) Speed of comparison; (e) Comparison of electromagnetic torque</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-5.tif"/>
</fig>
<p><xref ref-type="fig" rid="fig-6">Fig. 6</xref> shows the simulation results of typical, 10% eccentric, and 20% strange permanent magnet synchronous motor when the given speed is 4000 rpm, and the load is 0 N.m. <xref ref-type="fig" rid="fig-6">Figs. 6a</xref>&#x2013;<xref ref-type="fig" rid="fig-6">6c</xref> are	 the simulation results of stator currents of three types of motors, respectively, and (d) and (e) are the comparison of three motor speeds and electromagnetic torques. It can be seen that pure static eccentricity fault has no apparent influence on the stator current, rotational speed, and electromagnetic torque of PMSM [<xref ref-type="bibr" rid="ref-24">24</xref>].</p>
<fig id="fig-6">
<label>Figure 6</label>
<caption>
<title>PMSM simulation results: (a) Normal PMSM stator current; (b) Se10 PMSM stator current; (c) Se20 PMSM stator current; (d) Speed of comparison; (e) Comparison of electromagnetic torque</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-6.tif"/>
</fig>
</sec>
</sec>
<sec id="s3">
<label>3</label>
<title>Data Set Making and CNN Model Building</title>
<sec id="s3_1">
<label>3.1</label>
<title>Stator Current FFT Transformation</title>
<p>FFT is a standard signal processing method that can quickly realize signal transformation from a time domain to a frequency domain. Discrete fourier transform (DFT) is the most basic method, digital signal processing of the input signal by frequency <inline-formula id="ieqn-2"><mml:math id="mml-ieqn-2"><mml:msub><mml:mi>f</mml:mi><mml:mrow><mml:mi>s</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> interval sampling after n times, such as DFT processing of n discrete points get n the plural. Each complex number represents the amplitude and phase information of the frequency component with <inline-formula id="ieqn-3"><mml:math id="mml-ieqn-3"><mml:msub><mml:mi>f</mml:mi><mml:mrow><mml:mi>s</mml:mi></mml:mrow></mml:msub><mml:mrow><mml:mo>/</mml:mo></mml:mrow><mml:mi>n</mml:mi></mml:math></inline-formula> as the interval, to obtain the spectrum information of the input signal [<xref ref-type="bibr" rid="ref-25">25</xref>].</p>
<p>For a time-domain discrete signal X(n) with length n, the DFT transformation result is X(k), then:</p>
<p><disp-formula id="eqn-1"><label>(1)</label><mml:math id="mml-eqn-1" display="block"><mml:mrow><mml:mtext>X</mml:mtext></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mi>k</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mi>D</mml:mi><mml:mi>F</mml:mi><mml:mi>T</mml:mi><mml:mrow><mml:mo>{</mml:mo><mml:mi>x</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>n</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>}</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:munderover><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:munderover><mml:mi>x</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>n</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>&#x2217;</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>&#x2212;</mml:mo><mml:mi>j</mml:mi><mml:mfrac><mml:mrow><mml:mn>2</mml:mn><mml:mi>&#x03C0;</mml:mi></mml:mrow><mml:mi>N</mml:mi></mml:mfrac><mml:mi>n</mml:mi><mml:mi>k</mml:mi></mml:mrow></mml:msup><mml:mo>,</mml:mo><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mn>1</mml:mn><mml:mo>,</mml:mo><mml:mo>&#x2026;</mml:mo><mml:mo>,</mml:mo><mml:mi>N</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:math></disp-formula></p>
<p>If the following definition is made:</p>
<p><disp-formula id="eqn-2"><label>(2)</label><mml:math id="mml-eqn-2" display="block"><mml:msub><mml:mi>W</mml:mi><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>&#x2212;</mml:mo><mml:mi>j</mml:mi><mml:mfrac><mml:mrow><mml:mn>2</mml:mn><mml:mi>&#x03C0;</mml:mi></mml:mrow><mml:mi>N</mml:mi></mml:mfrac></mml:mrow></mml:msup></mml:math></disp-formula></p>
<p><disp-formula id="eqn-3"><label>(3)</label><mml:math id="mml-eqn-3" display="block"><mml:msubsup><mml:mi>W</mml:mi><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mi>n</mml:mi><mml:mi>k</mml:mi></mml:mrow></mml:msubsup><mml:mo>=</mml:mo><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mo>&#x2212;</mml:mo><mml:mi>j</mml:mi><mml:mfrac><mml:mrow><mml:mn>2</mml:mn><mml:mi>&#x03C0;</mml:mi></mml:mrow><mml:mi>N</mml:mi></mml:mfrac><mml:mi>n</mml:mi><mml:mi>k</mml:mi></mml:mrow></mml:msup></mml:math></disp-formula></p>
<p>Then X(k) can be expressed as:</p>
<p><disp-formula id="eqn-4"><label>(4)</label><mml:math id="mml-eqn-4" display="block"><mml:mrow><mml:mtext>X</mml:mtext></mml:mrow><mml:mrow><mml:mo>(</mml:mo><mml:mi>k</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mi>D</mml:mi><mml:mi>F</mml:mi><mml:mi>T</mml:mi><mml:mrow><mml:mo>{</mml:mo><mml:mi>x</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>n</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>}</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:munderover><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>n</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:mrow></mml:munderover><mml:mi>x</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>n</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>&#x2217;</mml:mo><mml:msubsup><mml:mi>W</mml:mi><mml:mrow><mml:mi>N</mml:mi></mml:mrow><mml:mrow><mml:mi>n</mml:mi><mml:mi>k</mml:mi></mml:mrow></mml:msubsup><mml:mo>,</mml:mo><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:mn>0</mml:mn><mml:mo>,</mml:mo><mml:mn>1</mml:mn><mml:mo>,</mml:mo><mml:mo>&#x2026;</mml:mo><mml:mo>,</mml:mo><mml:mi>N</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn></mml:math></disp-formula></p>
<p>FFT mainly uses periodicity, Symmetry, and scaling of rotation factor to carry out butterfly iterative calculation of DFT to reduce the computation amount of DFT, but its basic properties are still similar to DFT.</p>
<p>In the PMSM fault model built by AltairFlux and MATLAB, the simulation model collected 8000 data points in the time domain signal of stator current at 0.25 s, and the sampling frequency was 32 kHz. The spectrum diagram of the stator three-phase currents of the standard motor, the 25% and 50% demagnetization fault motor, and the 10% and 20% static eccentricity fault motor obtained through the co-simulation is shown in <xref ref-type="fig" rid="fig-7">Fig. 7</xref>. Since bilateral spectra are symmetrical, only unilateral spectra can be analyzed.</p>
<fig id="fig-7">
<label>Figure 7</label>
<caption>
<title>Spectrum diagram of stator current of PMSM with different faults: (a) Normal PMSM; (b) De25 PMSM; (c) De50 PMSM; (d) Se10 PMSM; (e) Se20 PMSM</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-7.tif"/>
</fig>
</sec>
<sec id="s3_2">
<label>3.2</label>
<title>The Frequency Domain Signal is Converted into a Gray Image</title>
<p>Because high-dimensional features often contain more data features, this paper first upgraded the frequency domain signal of motor stator current into the two-dimensional gray image, then produced a data set and imported it into CNN [<xref ref-type="bibr" rid="ref-26">26</xref>]. The scattering of data sample values within the range of gray values can be deemed as a probability problem. Because of the significant number theorem, on the condition the amount of data is large enough, the data will obey the Gaussian distribution. Therefore, the mapping function of data scattering into the gray range is set as a one-dimensional Gaussian distribution function in this paper, as shown in the <xref ref-type="disp-formula" rid="eqn-5">formula (5)</xref>. <xref ref-type="fig" rid="fig-8">Fig. 8</xref> shows the process of image conversion based on the autocorrelation matrix [<xref ref-type="bibr" rid="ref-27">27</xref>].</p>
<fig id="fig-8">
<label>Figure 8</label>
<caption>
<title>Image conversion flow chart based on autocorrelation matrix</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-8.tif"/>
</fig>
<p><disp-formula id="eqn-5"><label>(5)</label><mml:math id="mml-eqn-5" display="block"><mml:mi>f</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>x</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:mn>255</mml:mn><mml:mo>&#x00D7;</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:msqrt><mml:mn>2</mml:mn><mml:mi>&#x03C0;</mml:mi><mml:mi>&#x03C3;</mml:mi></mml:msqrt></mml:mfrac><mml:msup><mml:mi>e</mml:mi><mml:mrow><mml:mfrac><mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>x</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>&#x2212;</mml:mo><mml:mi>u</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msup><mml:msup><mml:mi>&#x03C3;</mml:mi><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msup></mml:mfrac></mml:mrow></mml:msup></mml:math></disp-formula></p>
<p>where <inline-formula id="ieqn-4"><mml:math id="mml-ieqn-4"><mml:msub><mml:mi>x</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> are elements in the ith row and the Jth column of the autocorrelation matrix; &#x03BC; is the mean value of the original data; &#x03C3;<sup>2</sup> is the variance of the original data.</p>
<p>If <inline-formula id="ieqn-5"><mml:math id="mml-ieqn-5"><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:mi>j</mml:mi><mml:mo>)</mml:mo></mml:mrow></mml:math></inline-formula> it represents the intensity of pixels in the generated row I and column j and <inline-formula id="ieqn-6"><mml:math id="mml-ieqn-6"><mml:mi>S</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:mn>1</mml:mn><mml:mo>)</mml:mo></mml:mrow></mml:math></inline-formula> represents the I-th data point in the original frequency domain signal, then:</p>
<p><disp-formula id="eqn-6"><label>(6)</label><mml:math id="mml-eqn-6" display="block"><mml:mi>P</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:mi>j</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>=</mml:mo><mml:msup><mml:mi>S</mml:mi><mml:mrow><mml:mi>T</mml:mi></mml:mrow></mml:msup><mml:mrow><mml:mo>(</mml:mo><mml:mn>1</mml:mn><mml:mo>,</mml:mo><mml:mi>j</mml:mi><mml:mo>)</mml:mo></mml:mrow><mml:mo>&#x2217;</mml:mo><mml:mi>S</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:mi>i</mml:mi><mml:mo>,</mml:mo><mml:mn>1</mml:mn><mml:mo>)</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:mi>f</mml:mi><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>x</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow></mml:math></disp-formula></p>
<p>Therefore, the pixels in the I-th row and J-th column of the grayscale image generated by the image conversion method based on the autocorrelation matrix reflect the intensity of the I-th data point in the original data, and the generated grayscale map is symmetrical about its diagonal.</p>
</sec>
<sec id="s3_3">
<label>3.3</label>
<title>Make a Data Set</title>
<p>In this paper, an image conversion method based on an autocorrelation matrix is used to convert the frequency domain signal of motor stator current into a gray-level image. In the spectrum diagram, 500 data points are sampled every 30 data points and dimensionally upgraded. The way of data interception is shown in <xref ref-type="fig" rid="fig-9">Fig. 9</xref>:</p>
<fig id="fig-9">
<label>Figure 9</label>
<caption>
<title>Data interception mode</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-9.tif"/>
</fig>
<p>There are 8000 data points in the time domain signal of single-phase current of different fault motors obtained by co-simulation, and 8000 data points in the bilateral spectrum obtained by FFT. By converting the two-sided range of the three-phase current into the gray image, the original gray image of each type of fault is 793. After data enhancement, 20% of the grayscale images are randomly selected as the test set, and the data set information is shown in <xref ref-type="table" rid="table-2">Table 2</xref> and the grayscale images converted from frequency domain signals of PMSM with different fault types are shown in <xref ref-type="fig" rid="fig-10">Fig. 10</xref>.</p>
<table-wrap id="table-2">
<label>Table 2</label>
<caption>
<title>Dataset content</title>
</caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th>Motor type</th>
<th>Label</th>
<th>Number of raw data</th>
<th>Number of expanded data</th>
<th>Number of training sets</th>
<th>Number of test sets</th>
</tr>
</thead>
<tbody>
<tr>
<td>Normal</td>
<td>He</td>
<td>793</td>
<td>2000</td>
<td>1600</td>
<td>400</td>
</tr>
<tr>
<td>25% demagnetizing</td>
<td>De25</td>
<td>793</td>
<td>1800</td>
<td>1440</td>
<td>360</td>
</tr>
<tr>
<td>50% demagnetizing</td>
<td>De50</td>
<td>793</td>
<td>1500</td>
<td>1200</td>
<td>300</td>
</tr>
<tr>
<td>10% static eccentric</td>
<td>Se10</td>
<td>793</td>
<td>1200</td>
<td>960</td>
<td>240</td>
</tr>
<tr>
<td>20% static eccentric</td>
<td>Se20</td>
<td>793</td>
<td>1000</td>
<td>800</td>
<td>200</td>
</tr>
</tbody>
</table>
</table-wrap><fig id="fig-10">
<label>Figure 10</label>
<caption>
<title>Grayscale image converted from PMSM frequency domain signals of different faults (load is 0 N.m): (a) De25; (b) De50; (c) Normal; (d) Se10; (e) Se20</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-10.tif"/>
</fig>
<p>The training of CNN generally requires large data sets, and reading grayscale images in data sets one by one will occupy a lot of computing resources. In the TensorFlow framework, sample images can be made into TFRecord files to compress the data set into binary coding. During training, it can be decoded and then imported into CNN for training. This method will significantly save computing resources.</p>
</sec>
<sec id="s3_4">
<label>3.4</label>
<title>The Construction of the CNN Model</title>
<sec id="s3_4_1">
<label>3.4.1</label>
<title>Multiscale Feature Fusion</title>
<p>CNN was first proposed by French American computer programmer Yang Likun in 1989, inspired by primate visual system neurons. It is a deep feedforward neural network [<xref ref-type="bibr" rid="ref-28">28</xref>], which has a solid ability to extract input data features and is used explicitly for processing input data with network-like structures. However, with the increasing number of CNN layers, the image information contained in FeatureMap gradually decreases in the forward propagation process. As a result, the neural network gradient disappears. ResNet adds a skip join before each convolutional layer, which not only solves the problem of vanishing gradients. It also speeds up the training process of the network and is beneficial to the backpropagation of the slope.</p>
<p>Literature [<xref ref-type="bibr" rid="ref-29">29</xref>] improved the Selective Convolutional Kernel Network (SKNet) through skip connections, which learned the weight of shallow image information through skip links and fused it with the extracted global features to obtain more image elements [<xref ref-type="bibr" rid="ref-30">30</xref>]. Although external feature maps contain a lot of semantic information, their resolution could be higher. The semantic information level of deep feature maps is lower but has a higher resolution [<xref ref-type="bibr" rid="ref-31">31</xref>]. This problem can be solved by collecting image features of different levels simultaneously. In addition, the attention module is widely used in various CNN structures. This paper also introduces an MSFFM (Multi-scale feature fusion module) based on a channel attention module to fuse image features of different levels [<xref ref-type="bibr" rid="ref-32">32</xref>]. To solve the semantic gap problem in the process of feature fusion, MSFFM calculates the relevance between pixels of varying feature maps through the number and multiplication operation between matrix elements and uses this relevance as the weight vector of deep feature maps:</p>
<p><disp-formula id="eqn-7"><label>(7)</label><mml:math id="mml-eqn-7" display="block"><mml:msub><mml:mi>s</mml:mi><mml:mrow><mml:mi>j</mml:mi><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mrow><mml:mi>exp</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>P</mml:mi><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>&#x2217;</mml:mo><mml:msub><mml:mi>Q</mml:mi><mml:mrow><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow></mml:mrow><mml:mrow><mml:munderover><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover><mml:mi>exp</mml:mi><mml:mo>&#x2061;</mml:mo><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>P</mml:mi><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>&#x2217;</mml:mo><mml:msub><mml:mi>Q</mml:mi><mml:mrow><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow></mml:mrow></mml:mfrac></mml:math></disp-formula></p>
<p><inline-formula id="ieqn-7"><mml:math id="mml-ieqn-7"><mml:msub><mml:mi>S</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> is used to measure the correlation between the <inline-formula id="ieqn-8"><mml:math id="mml-ieqn-8"><mml:mi>i</mml:mi><mml:mi>t</mml:mi><mml:mi>h</mml:mi></mml:math></inline-formula> channel in shallow feature maps and the <inline-formula id="ieqn-9"><mml:math id="mml-ieqn-9"><mml:mi>j</mml:mi><mml:mi>t</mml:mi><mml:mi>h</mml:mi></mml:math></inline-formula> medium in deep feature maps. <inline-formula id="ieqn-10"><mml:math id="mml-ieqn-10"><mml:mi>N</mml:mi></mml:math></inline-formula> is the number of pixels on the feature map of different <inline-formula id="ieqn-11"><mml:math id="mml-ieqn-11"><mml:mi>P</mml:mi></mml:math></inline-formula> media, and <inline-formula id="ieqn-12"><mml:math id="mml-ieqn-12"><mml:mi>Q</mml:mi></mml:math></inline-formula> is the shallow and deep features maps created by different convolution layers in CNN, respectively, where <inline-formula id="ieqn-13"><mml:math id="mml-ieqn-13"><mml:mrow><mml:mo>{</mml:mo><mml:mi>P</mml:mi><mml:mo>,</mml:mo><mml:mi>Q</mml:mi><mml:mo>}</mml:mo></mml:mrow><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>R</mml:mi><mml:mrow><mml:mi>C</mml:mi><mml:mo>&#x2217;</mml:mo><mml:mi>N</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula>. As shown in <xref ref-type="fig" rid="fig-11">Fig. 11</xref>, firstly, the generated feature map is imported into the corresponding convolution layer, the number of channels and the size of the feature map are compressed, and the feature map is generated feature maps <inline-formula id="ieqn-14"><mml:math id="mml-ieqn-14"><mml:mi>A</mml:mi></mml:math></inline-formula> and <inline-formula id="ieqn-15"><mml:math id="mml-ieqn-15"><mml:mi>B</mml:mi></mml:math></inline-formula>, <inline-formula id="ieqn-16"><mml:math id="mml-ieqn-16"><mml:mrow><mml:mo>{</mml:mo><mml:mi>A</mml:mi><mml:mo>,</mml:mo><mml:mi>B</mml:mi><mml:mo>}</mml:mo></mml:mrow><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>R</mml:mi><mml:mrow><mml:mi>C</mml:mi><mml:mo>&#x2217;</mml:mo><mml:mi>H</mml:mi><mml:mo>&#x2217;</mml:mo><mml:mi>W</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula>. <inline-formula id="ieqn-17"><mml:math id="mml-ieqn-17"><mml:mi>H</mml:mi></mml:math></inline-formula> and <inline-formula id="ieqn-18"><mml:math id="mml-ieqn-18"><mml:mi>W</mml:mi></mml:math></inline-formula> is the size of the feature map [<xref ref-type="bibr" rid="ref-33">33</xref>]. Then, shallow feature <inline-formula id="ieqn-19"><mml:math id="mml-ieqn-19"><mml:mi>A</mml:mi></mml:math></inline-formula>s and in-depth feature <inline-formula id="ieqn-20"><mml:math id="mml-ieqn-20"><mml:mi>B</mml:mi></mml:math></inline-formula>s were fused and reshaped into <inline-formula id="ieqn-21"><mml:math id="mml-ieqn-21"><mml:mi>P</mml:mi></mml:math></inline-formula> and <inline-formula id="ieqn-22"><mml:math id="mml-ieqn-22"><mml:mi>Q</mml:mi></mml:math></inline-formula>. Respectively, where <inline-formula id="ieqn-23"><mml:math id="mml-ieqn-23"><mml:mi>N</mml:mi><mml:mo>=</mml:mo><mml:mi>H</mml:mi><mml:mo>&#x2217;</mml:mo><mml:mi>W</mml:mi></mml:math></inline-formula> is the number of elements on different feature plots? Then, multiply the two matrices, transpose, and compute the spatial attention graph with the softmax function <inline-formula id="ieqn-24"><mml:math id="mml-ieqn-24"><mml:mi>S</mml:mi><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>R</mml:mi><mml:mrow><mml:mi>N</mml:mi><mml:mo>&#x2217;</mml:mo><mml:mi>N</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula>. Then multiply the matrix between <inline-formula id="ieqn-25"><mml:math id="mml-ieqn-25"><mml:mi>Q</mml:mi></mml:math></inline-formula> and the spatial attention graph <inline-formula id="ieqn-26"><mml:math id="mml-ieqn-26"><mml:mi>S</mml:mi></mml:math></inline-formula> to generate the feature graph <inline-formula id="ieqn-27"><mml:math id="mml-ieqn-27"><mml:mi>L</mml:mi><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>R</mml:mi><mml:mrow><mml:mi>C</mml:mi><mml:mo>&#x2217;</mml:mo><mml:mi>H</mml:mi><mml:mo>&#x2217;</mml:mo><mml:mi>W</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula>. Finally, bear the elements corresponding to <inline-formula id="ieqn-28"><mml:math id="mml-ieqn-28"><mml:mi>B</mml:mi></mml:math></inline-formula><inline-formula id="ieqn-29"><mml:math id="mml-ieqn-29"><mml:mi>L</mml:mi></mml:math></inline-formula> and to obtain the final <inline-formula id="ieqn-30"><mml:math id="mml-ieqn-30"><mml:mi>O</mml:mi><mml:mo>&#x2208;</mml:mo><mml:msup><mml:mi>R</mml:mi><mml:mrow><mml:mi>C</mml:mi><mml:mo>&#x2217;</mml:mo><mml:mi>H</mml:mi><mml:mo>&#x2217;</mml:mo><mml:mi>W</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula>. The output is as follows:
<disp-formula id="eqn-8"><label>(8)</label><mml:math id="mml-eqn-8" display="block"><mml:msub><mml:mi>O</mml:mi><mml:mrow><mml:mi>j</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mi>&#x03B1;</mml:mi><mml:mo>&#x2217;</mml:mo><mml:munderover><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>=</mml:mo><mml:mn>1</mml:mn></mml:mrow><mml:mrow><mml:mi>N</mml:mi></mml:mrow></mml:munderover><mml:mrow><mml:mo>(</mml:mo><mml:msub><mml:mi>s</mml:mi><mml:mrow><mml:mi>j</mml:mi><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>&#x2217;</mml:mo><mml:msub><mml:mi>q</mml:mi><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub><mml:mo>)</mml:mo></mml:mrow><mml:mo>+</mml:mo><mml:msub><mml:mi>B</mml:mi><mml:mrow><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:math></disp-formula></p>
<fig id="fig-11">
<label>Figure 11</label>
<caption>
<title>Multi-scale feature fusion module</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-11.tif"/>
</fig>
<p>Among <inline-formula id="ieqn-31"><mml:math id="mml-ieqn-31"><mml:mi>&#x03B1;</mml:mi></mml:math></inline-formula> initialize to 0 and gradually learn to allocate more weights <inline-formula id="ieqn-32"><mml:math id="mml-ieqn-32"><mml:msub><mml:mi>q</mml:mi><mml:mrow><mml:mi>i</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula>, represents the <inline-formula id="ieqn-33"><mml:math id="mml-ieqn-33"><mml:mi>i</mml:mi><mml:mi>t</mml:mi><mml:mi>h</mml:mi></mml:math></inline-formula> pixel in the low-dimensional feature maps, and <inline-formula id="ieqn-34"><mml:math id="mml-ieqn-34"><mml:msub><mml:mi>B</mml:mi><mml:mrow><mml:mi>j</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> is the <inline-formula id="ieqn-35"><mml:math id="mml-ieqn-35"><mml:mi>j</mml:mi><mml:mi>t</mml:mi><mml:mi>h</mml:mi></mml:math></inline-formula> channel in the high-dimensional feature map. From <xref ref-type="disp-formula" rid="eqn-8">(8)</xref>, it can be inferred that each position of the final feature <inline-formula id="ieqn-36"><mml:math id="mml-ieqn-36"><mml:mi>O</mml:mi></mml:math></inline-formula> is the weighted sum of all top-quality functions. Because shallow parts generate the last semantic features, they will contain rich semantic information [<xref ref-type="bibr" rid="ref-34">34</xref>].</p>
</sec>
<sec id="s3_4_2">
<label>3.4.2</label>
<title>Channel Attention Module</title>
<p>Shallow features contain a lot of semantic information. The feature graph of every channel will generate a specific feature map, and every feature map will significantly impact the final output. Through the channel attention module shown in <xref ref-type="fig" rid="fig-12">Fig. 12</xref>, the consistency of feature mapping at each layer is enhanced by changing the feature weights of each channel [<xref ref-type="bibr" rid="ref-35">35</xref>]. CAM (Channel Attention Module) is to reweight each channel based on all elements of each feature map. First, the global average pooling layer is used to compress the feature map size and the number of feature channels [<xref ref-type="bibr" rid="ref-36">36</xref>]. Then, the corresponding weight vector is generated by the modified activation function (ReLU) and sigmoid function. Finally, the output feature map is generated by combining the matrix multiplication operation with the input feature map. Integrate the general information into the weight vector to ensure the reliability of feature mapping in turn [<xref ref-type="bibr" rid="ref-37">37</xref>].</p>
<fig id="fig-12">
<label>Figure 12</label>
<caption>
<title>Channel attention module</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-12.tif"/>
</fig>
</sec>
<sec id="s3_4_3">
<label>3.4.3</label>
<title>Atrous Convolutional Spatial Pyramid Pooling</title>
<p>The Atrous Convolutional Spatial pyramid pooling (ASPP) incorporates Aperture Convolutions into SPP. Atrous convolutional can systematically aggregate image feature information at different scales and levels without losing resolution [<xref ref-type="bibr" rid="ref-38">38</xref>].</p>
<p>The basic principle of atrous convolution is shown in <xref ref-type="disp-formula" rid="eqn-9">formula (9)</xref>:</p>
<p><disp-formula id="eqn-9"><label>(9)</label><mml:math id="mml-eqn-9" display="block"><mml:mi>y</mml:mi><mml:mo stretchy="false">[</mml:mo><mml:mi>i</mml:mi><mml:mo stretchy="false">]</mml:mo><mml:mo>=</mml:mo><mml:msub><mml:mo movablelimits="false">&#x2211;</mml:mo><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msub><mml:mi>x</mml:mi><mml:mo stretchy="false">[</mml:mo><mml:mi>i</mml:mi><mml:mo>+</mml:mo><mml:mi>r</mml:mi><mml:mo>&#x2217;</mml:mo><mml:mi>k</mml:mi><mml:mo stretchy="false">]</mml:mo><mml:mi>w</mml:mi><mml:mo stretchy="false">[</mml:mo><mml:mi>k</mml:mi><mml:mo stretchy="false">]</mml:mo></mml:math></disp-formula></p>
<p>In <xref ref-type="disp-formula" rid="eqn-9">Eq. (9)</xref>, i is the coordinates of each element in the two-dimensional matrix, r is the expansion rate of the atrous convolution, and k is the dimension of the convolution kernel [<xref ref-type="bibr" rid="ref-39">39</xref>]. The ordinary convolution can be viewed as an atrous convolution with a rate of expansion of 1.</p>
<p>For standard convolution operations, the sliding step size of the convolution kernel is 1, which can be divided into three situations [<xref ref-type="bibr" rid="ref-40">40</xref>]:</p>
<p>That is, when downsampling is performed while convolution is being performed, the dimension of the convolutional feature map will decline;</p>
<p>1. Indicating a convolution with an average step size of 1;</p>
<p>2. As the feature map is pooled, the dimension of the feature map will increase.</p>
<p>The atrous convolution fills in between the blank elements. It ignores the part or leaves the input unchanged, adding the weight of the convolution kernel parameter to 0, thereby expanding the acceptance domain, convolving with a value more excellent than one can achieve the same effect. However, downsampling will be performed simultaneously with the convolution, which will reduce the feature map&#x2019;s size and is unsuitable for use [<xref ref-type="bibr" rid="ref-41">41</xref>].</p>
<p>Assuming that the atrous fraction of the atrous convolution is r and the dimension of the convolution kernel is k, the extent F of the receptive field obtained is:</p>
<p><disp-formula id="eqn-10"><label>(10)</label><mml:math id="mml-eqn-10" display="block"><mml:mi>F</mml:mi><mml:mo>=</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>r</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2217;</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mi>k</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mn>1</mml:mn><mml:mo stretchy="false">)</mml:mo><mml:mo>+</mml:mo><mml:mi>k</mml:mi></mml:math></disp-formula></p>
<p>The structure of the atrous convolutional spatial pyramid pooling is shown in <xref ref-type="fig" rid="fig-13">Fig. 13</xref>. Compared with SPP, an empty convolution is inserted in front of each parallel pooling window to increase the extraction range of image features. The similar atrous convolution layers have four different expansion rates to extract feature information at different levels and scales [<xref ref-type="bibr" rid="ref-42">42</xref>].</p>
<fig id="fig-13">
<label>Figure 13</label>
<caption>
<title>Structural diagram of ASPP</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-13.tif"/>
</fig>
</sec>
<sec id="s3_4_4">
<label>3.4.4</label>
<title>Construction of a GCNN Model for Atrous Convolutional Pyramid</title>
<p>Based on the traditional CNN model, multi-scale feature modules and attention channels are added to form an 8-layer convolutional neural network, including six convolutional layers and two fully connected layers. Maxpool is used for all pooling layers. In this paper, the improved ResNet network based on MSFFM-CAM-ASPP is named GCNN. An atrous convolution pyramid is introduced behind the sixth pooling layer to enlarge the field of view and enhance the feature extraction capability of the network. After pooling, the first five pooling layers are connected to the fusion layer by multi-scale feature modules, and the data features of different levels are extracted. Finally, the data is fused with the components extracted by ASPP. The network structure is shown in <xref ref-type="fig" rid="fig-14">Fig. 14</xref>.</p>
<fig id="fig-14">
<label>Figure 14</label>
<caption>
<title>GCNN neural network structure</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-14.tif"/>
</fig>
<p>The four-layer ASPP adopts an average pooling layer. Although the highest pooling layer is standard, it can only output a maximum value and is unsuitable for GCNN networks with added atrous convolutional kernels. At each level of ASPP, feature maps with dimensions of (w, h) (w for width, h for height) are equally divided. For example, if it is divided into four modules, the dimension of every module will be (w/2, h/2); If it were divided into nine modules, the Measurement of every module would be (w/3, h/3); If it were divided into 36 modules, the dimension of every module would be (w/6, h/6). This article divides the four-layer feature maps into 1, 4, 9, and 36 modules, respectively, so that 50 sub-regions can be obtained and 50 different sets of image features can be extracted. These features are fused with the feature maps obtained from the first six convolutional layers through jump connections.</p>
<p>Every layer of ASPP in <xref ref-type="fig" rid="fig-14">Fig. 14</xref> has an atrous convolutional kernel, and the dimensions of the convolutional kernel are set to 3 &#x00D7; 3. <xref ref-type="table" rid="table-3">Table 3</xref> shows the parameters of the four-layer ASPP. Atrous convolutional kernels with high expansion rates will lose feature information, affect network accuracy, and affect the accuracy of the network. Therefore, using atrous convolutional kernels with low expansion rates to segment feature maps is common.</p>
<table-wrap id="table-3">
<label>Table 3</label>
<caption>
<title>GCNN network model parameters</title>
</caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th>Network parameters</th>
<th>Parameter selection</th>
</tr>
</thead>
<tbody>
<tr>
<td>Number of layers</td>
<td>8</td>
</tr>
<tr>
<td>Epoch</td>
<td>30</td>
</tr>
<tr>
<td>Loss function</td>
<td>Categorical_crossentropy</td>
</tr>
<tr>
<td>Optimizer</td>
<td>Adagrad</td>
</tr>
<tr>
<td>Activation function</td>
<td>Relu</td>
</tr>
<tr>
<td>Original learning rate</td>
<td>0.01</td>
</tr>
<tr>
<td>Learning rate decay rate</td>
<td>0.85</td>
</tr>
<tr>
<td>Channels</td>
<td>64</td>
</tr>
<tr>
<td>Batch size</td>
<td>16</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
</sec>
<sec id="s3_5">
<label>3.5</label>
<title>Parameter Configuration of the GCNN Model</title>
<p>GCNN network parameters are the optimal hyperparameters determined by the grid search method. The parameters are set reasonably, and the diagnostic accuracy of different faults is close to the maximum accuracy allowed by the theory.</p>
<p>This article adopts a step decay learning rate, and the selected decay learning rate function is:</p>
<p><disp-formula id="eqn-11"><label>(11)</label><mml:math id="mml-eqn-11" display="block"><mml:mi>k</mml:mi><mml:mo>=</mml:mo><mml:msub><mml:mi>k</mml:mi><mml:mrow><mml:mn>0</mml:mn></mml:mrow></mml:msub><mml:mo>&#x2217;</mml:mo><mml:msup><mml:mi>a</mml:mi><mml:mrow><mml:mrow><mml:mo>[</mml:mo><mml:mfrac><mml:mi>n</mml:mi><mml:mi>m</mml:mi></mml:mfrac><mml:mo>]</mml:mo></mml:mrow></mml:mrow></mml:msup></mml:math></disp-formula></p>
<p>Among them, <inline-formula id="ieqn-37"><mml:math id="mml-ieqn-37"><mml:msub><mml:mrow><mml:mtext>k</mml:mtext></mml:mrow><mml:mrow><mml:mn>0</mml:mn></mml:mrow></mml:msub></mml:math></inline-formula> is the learning rate at the beginning of GCNN training; <inline-formula id="ieqn-38"><mml:math id="mml-ieqn-38"><mml:mrow><mml:mtext>a</mml:mtext></mml:mrow></mml:math></inline-formula> is the attenuation coefficient of GCNN; n indicates how many rounds GCNN has trained so far; <inline-formula id="ieqn-39"><mml:math id="mml-ieqn-39"><mml:mrow><mml:mtext>m</mml:mtext></mml:mrow></mml:math></inline-formula> is a constant applied to adjust the variation of GCNN learning rate; <inline-formula id="ieqn-40"><mml:math id="mml-ieqn-40"><mml:mrow><mml:mo>[</mml:mo><mml:mstyle displaystyle="true" scriptlevel="0"><mml:mfrac><mml:mrow><mml:mtext>n</mml:mtext></mml:mrow><mml:mrow><mml:mtext>m</mml:mtext></mml:mrow></mml:mfrac></mml:mstyle><mml:mo>]</mml:mo></mml:mrow></mml:math></inline-formula> stands for rounding; <inline-formula id="ieqn-41"><mml:math id="mml-ieqn-41"><mml:mrow><mml:mtext>k</mml:mtext></mml:mrow></mml:math></inline-formula> stands for the current learning rate of GCNN.</p>
<p>The initial learning rate is 0.01, the attenuation coefficient <inline-formula id="ieqn-42"><mml:math id="mml-ieqn-42"><mml:mrow><mml:mtext>a</mml:mtext></mml:mrow></mml:math></inline-formula> is 0.85, and 30 rounds of training with a <inline-formula id="ieqn-43"><mml:math id="mml-ieqn-43"><mml:mrow><mml:mtext>m</mml:mtext></mml:mrow></mml:math></inline-formula> of 5 can result in faster and better training results. The learning rate of GCNN during the exercise is shown in <xref ref-type="fig" rid="fig-15">Fig. 15</xref> [<xref ref-type="bibr" rid="ref-43">43</xref>]:</p>
<fig id="fig-15">
<label>Figure 15</label>
<caption>
<title>Step attendee&#x2019;s learning rate</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-15.tif"/>
</fig>
<p>GCNN adopts the cross entropy that can be adopted to estimate the contrast between the theoretical value and the actual value to evaluate its loss function. The smaller the cross entropy value is, the closer the two probability distributions are, and it is described as <xref ref-type="disp-formula" rid="eqn-12">formula (12)</xref> [<xref ref-type="bibr" rid="ref-44">44</xref>]:</p>
<p><disp-formula id="eqn-12"><label>(12)</label><mml:math id="mml-eqn-12" display="block"><mml:mi>C</mml:mi><mml:mo>=</mml:mo><mml:mo>&#x2212;</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>n</mml:mi></mml:mfrac><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>x</mml:mi></mml:mrow></mml:munder><mml:mo stretchy="false">[</mml:mo><mml:mrow><mml:mi>y</mml:mi><mml:mo>&#x2217;</mml:mo><mml:mi>l</mml:mi><mml:mi>n</mml:mi><mml:mi>a</mml:mi></mml:mrow><mml:mo>+</mml:mo><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>y</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo>&#x2217;</mml:mo><mml:mi>l</mml:mi><mml:mi>n</mml:mi><mml:mo stretchy="false">(</mml:mo><mml:mn>1</mml:mn><mml:mo>&#x2212;</mml:mo><mml:mi>a</mml:mi><mml:mo stretchy="false">)</mml:mo><mml:mo stretchy="false">]</mml:mo></mml:math></disp-formula></p>
<p>In <xref ref-type="disp-formula" rid="eqn-12">formula (12)</xref>, <inline-formula id="ieqn-44"><mml:math id="mml-ieqn-44"><mml:mrow><mml:mtext>x</mml:mtext></mml:mrow></mml:math></inline-formula> stands for the sample, <inline-formula id="ieqn-45"><mml:math id="mml-ieqn-45"><mml:mrow><mml:mtext>n</mml:mtext></mml:mrow></mml:math></inline-formula> stands for the total amount of models, <inline-formula id="ieqn-46"><mml:math id="mml-ieqn-46"><mml:mrow><mml:mtext>y</mml:mtext></mml:mrow></mml:math></inline-formula> stands for the theoretical value, and it stands for the actual value.</p>
<p>The other model parameters in GCNN, except for ASPP, are shown in <xref ref-type="table" rid="table-3">Table 3</xref>, and the model parameters of ASPP are shown in <xref ref-type="table" rid="table-4">Table 4</xref>, where k is the size of the ordinary convolution kernel, c is the number of characteristic output channels, d is the expansion rate of cavity convolution kernel, p is the filling mode, r is the type of activation function, and b is the offset.</p>
<table-wrap id="table-4">
<label>Table 4</label>
<caption>
<title>The ASPP model parameters</title>
</caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th rowspan="2">Number of layers</th>
<th rowspan="2">Padding</th>
<th colspan="6" align="center">Atrous conv</th>
<th colspan="5" align="center">Conv</th>
<th>BN</th>
<th align="center" colspan="2">Average pooling</th>
</tr>
<tr>
<th>k</th>
<th>c</th>
<th>d</th>
<th>p</th>
<th>r</th>
<th>s</th>
<th>k</th>
<th>c</th>
<th>s</th>
<th>b</th>
<th>r</th>
<th>r</th>
<th>k</th>
<th>s</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>4</td>
<td>3</td>
<td>1024</td>
<td>2</td>
<td>Same</td>
<td>Relu</td>
<td>1</td>
<td>1</td>
<td>1024</td>
<td>1</td>
<td>/</td>
<td>/</td>
<td>Relu</td>
<td>w, h</td>
<td>w, h</td>
</tr>
<tr>
<td>2</td>
<td>4</td>
<td>3</td>
<td>1024</td>
<td>4</td>
<td>Same</td>
<td>Relu</td>
<td>1</td>
<td>1</td>
<td>1024</td>
<td>1</td>
<td>/</td>
<td>/</td>
<td>Relu</td>
<td>w/2, h/2</td>
<td>w/2, h/2</td>
</tr>
<tr>
<td>3</td>
<td>4</td>
<td>3</td>
<td>1024</td>
<td>8</td>
<td>Same</td>
<td>Relu</td>
<td>1</td>
<td>1</td>
<td>1024</td>
<td>1</td>
<td>/</td>
<td>/</td>
<td>Relu</td>
<td>w/3, h/3</td>
<td>w/3, h/3</td>
</tr>
<tr>
<td>4</td>
<td>4</td>
<td>3</td>
<td>1024</td>
<td>12</td>
<td>Same</td>
<td>Relu</td>
<td>1</td>
<td>1</td>
<td>1024</td>
<td>1</td>
<td>/</td>
<td>/</td>
<td>Relu</td>
<td>W/6, h/6</td>
<td>W/6, h/6</td>
</tr>
</tbody>
</table>
</table-wrap>
</sec>
</sec>
<sec id="s4">
<label>4</label>
<title>Results</title>
<sec id="s4_1">
<label>4.1</label>
<title>GCNN Training Results and Comparison Experiment</title>
<p>The produced data set was imported into the built GCNN model for training. The training results are shown in <xref ref-type="fig" rid="fig-16">Fig. 16</xref>. The accuracy of the test set is 99.08%, and the loss function is 0.0035. The diagnostic accuracy of De25, De50, He, Se10, and Se20 motors is 99.80%, 99.60%, 99.70%, 98.10%, and 98.20%, respectively, which has high diagnostic accuracy. In addition, cross-validation was used to repeat the experiment many times to enhance the reliability of training results and avoid the interference of random factors on experimental results. After many experiments, it is found that the average accuracy of the five-fold cross-validation is the highest. The verification results of GCNN are shown in <xref ref-type="table" rid="table-5">Table 5</xref>. The difference between the average and experimental results obtained by random segmentation data is only 0.08%, which can fully prove the reliability of the experimental results.</p>
<fig id="fig-16">
<label>Figure 16</label>
<caption>
<title>GCNN model test results: (a) Accuracy curve; (b) Loss graph; (c) Test result confusion matrix</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-16.tif"/>
</fig><table-wrap id="table-5">
<label>Table 5</label>
<caption>
<title>GCNN 5-fold cross-validation results</title>
</caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th>5-fold cross-validation</th>
<th>Accuracy</th>
</tr>
</thead>
<tbody>
<tr>
<td>1</td>
<td>98.95%</td>
</tr>
<tr>
<td>2</td>
<td>98.77%</td>
</tr>
<tr>
<td>3</td>
<td>99.18%</td>
</tr>
<tr>
<td>4</td>
<td>99.07%</td>
</tr>
<tr>
<td>5</td>
<td>99.03%</td>
</tr>
<tr>
<td>Mean</td>
<td>99.00%</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>To demonstrate the advantage of the GCNN, this paper conducts comparison experiments with MCNN [<xref ref-type="bibr" rid="ref-27">27</xref>], Bi-LSTM [<xref ref-type="bibr" rid="ref-45">45</xref>], DBN [<xref ref-type="bibr" rid="ref-46">46</xref>], and RNN [<xref ref-type="bibr" rid="ref-47">47</xref>], respectively, and the experimental results are shown in <xref ref-type="fig" rid="fig-17">Fig. 17</xref>. The optimal hyperparameters of other comparison algorithms are determined by the grid search method, the parameters are set reasonably, and the diagnostic accuracy of different faults is close to the maximum accuracy allowed by the theory. The data set used is the same data set as GCNN, and both are self-made data sets. It can be seen that the accuracy rates of MCNN, Bi-LSTM, DBN, and RNN on the test set are 97.96%, 90.11%, 94.42%, and 91.78%, respectively, which are much lower than 99.00% of GCNN; the loss functions of the four comparison algorithms are 0.099, 0.242, 0.168, and 0.231, respectively, which are higher than that of GCNN. The comparison experiments thoroughly verify the superiority of GCNN.</p>
<fig id="fig-17">
<label>Figure 17</label>
<caption>
<title>The result of comparison experiments: (a) Accuracy curve; (b) Loss graph</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-17.tif"/>
</fig>
</sec>
<sec id="s4_2">
<label>4.2</label>
<title>Noise Interference Experiment</title>
<p>In order to verify the anti-interference ability and robustness of the model proposed in this paper against noise, Gaussian noise, black pepper and salt noise, white pepper and salt noise, and black pepper and salt mixed noise with white pepper and salt were added to the self-made data set, respectively. The gray images after adding noise are shown in <xref ref-type="fig" rid="fig-18">Figs. 18</xref>&#x2013;<xref ref-type="fig" rid="fig-21">21</xref>. After adding noise, different network fault diagnosis accuracy rates are shown in <xref ref-type="table" rid="table-6">Table 6</xref>, and the parameters of each network for comparison experiment are determined as the optimal parameters by the grid search method.</p>
<fig id="fig-18">
<label>Figure 18</label>
<caption>
<title>Grayscale images of different faults after adding Gaussian noise (load is 0 N.m): (a) De25; (b) De50; (c) Normal; (d) Se10; (e) Se20</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-18.tif"/>
</fig><fig id="fig-19">
<label>Figure 19</label>
<caption>
<title>Grayscale images of different faults after adding Black salt-and-pepper noise (load is 0 N.m): (a) De25; (b) De50; (c) Normal; (d) Se10; (e) Se20</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-19.tif"/>
</fig><fig id="fig-20">
<label>Figure 20</label>
<caption>
<title>Grayscale images of different faults after adding White salt-and-pepper noise (load is 0 N.m): (a) De25; (b) De50; (c) Normal; (d) Se10; (e) Se20</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-20.tif"/>
</fig><fig id="fig-21">
<label>Figure 21</label>
<caption>
<title>Grayscale images of different faults after adding Black and white salt-and-pepper noise (load is 0 N.m): (a) De25; (b) De50; (c) Normal; (d) Se10; (e) Se20</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-21.tif"/>
</fig><table-wrap id="table-6">
<label>Table 6</label>
<caption>
<title>Noise interferes with experimental results</title>
</caption>
<table frame="hsides">
<colgroup>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
<col align="left"/>
</colgroup>
<thead>
<tr>
<th>Network</th>
<th>Gaussian noise</th>
<th>Black salt-and-pepper noise</th>
<th>White salt-and-pepper noise</th>
<th>Black and white salt-and-pepper noise</th>
</tr>
</thead>
<tbody>
<tr>
<td>GCNN</td>
<td>0.9880</td>
<td>0.9887</td>
<td>0.9870</td>
<td>0.9855</td>
</tr>
<tr>
<td>MCNN</td>
<td>0.9435</td>
<td>0.9414</td>
<td>0.9230</td>
<td>0.9032</td>
</tr>
<tr>
<td>Bi-LSTM</td>
<td>0.8416</td>
<td>0.8331</td>
<td>0.8125</td>
<td>0.8087</td>
</tr>
<tr>
<td>DBN</td>
<td>0.8633</td>
<td>0.8515</td>
<td>0.8076</td>
<td>0.8036</td>
</tr>
<tr>
<td>RNN</td>
<td>0.8035</td>
<td>0.7930</td>
<td>0.7986</td>
<td>0.7916</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>It can be seen from the results of noise interference experiments that the fault diagnosis accuracy of the proposed GCNN network for different types of permanent magnet synchronous motors has not changed much in the face of noise interference such as Gaussian noise, black pepper and salt noise, white pepper and salt noise, black and white pepper, and salt noise, and is always maintained within 0.5%. The ability of anti-noise interference and robustness are obviously better than the other four networks.</p>
</sec>
<sec id="s4_3">
<label>4.3</label>
<title>Interpretability Analysis of Fault Characteristics</title>
<sec id="s4_3_1">
<label>4.3.1</label>
<title>The Visualization Technique of CNN&#x2014;Grad-CAM</title>
<p>Different channels of the convolutional neural network convolutional layer can extract other fault features from images, and these fault features are related to PMSM fault types to some extent [<xref ref-type="bibr" rid="ref-48">48</xref>]. The correlation between fault characteristics and fault types of different channels is also distinct, so assigning different weights to these fault characteristics is necessary. However, obtaining the weight parameters of output features of various media from the convolutional neural network is problematic, resulting in the need for more interpretability of the mapping relationship established by deep learning [<xref ref-type="bibr" rid="ref-49">49</xref>].</p>
<p>To solve this problem, the Grad-CAM visualization analysis method is adopted in this paper to calculate the weight k of each channel, and the feature map of each channel is superimposed with the weight k to generate the thermal activation map [<xref ref-type="bibr" rid="ref-50">50</xref>]. The intensity of pixel color in the thermal activation map corresponds to the correlation degree of features, and the closer the pixel color is to the warm color, The more significant the activation intensity of features, the higher the degree of correlation, which can directly reflect the basis for CNN to judge fault characteristics [<xref ref-type="bibr" rid="ref-51">51</xref>].</p>
<p>The principle formula of Grad-CAM is shown in <xref ref-type="disp-formula" rid="eqn-13">formulas (13)</xref> and <xref ref-type="disp-formula" rid="eqn-14">(14)</xref>:</p>
<p><disp-formula id="eqn-13"><label>(13)</label><mml:math id="mml-eqn-13" display="block"><mml:msub><mml:mi>&#x03B1;</mml:mi><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:mfrac><mml:mn>1</mml:mn><mml:mi>Z</mml:mi></mml:mfrac><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>i</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mi>p</mml:mi></mml:mrow></mml:munder><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>j</mml:mi><mml:mo>&#x2208;</mml:mo><mml:mi>q</mml:mi></mml:mrow></mml:munder><mml:mfrac><mml:mrow><mml:mi>&#x03B4;</mml:mi><mml:mi>l</mml:mi></mml:mrow><mml:mrow><mml:mi>&#x03B4;</mml:mi><mml:msubsup><mml:mi>A</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msubsup></mml:mrow></mml:mfrac></mml:math></disp-formula></p>
<p><disp-formula id="eqn-14"><label>(14)</label><mml:math id="mml-eqn-14" display="block"><mml:msub><mml:mi>L</mml:mi><mml:mrow><mml:mi>G</mml:mi><mml:mi>r</mml:mi><mml:mi>a</mml:mi><mml:mi>d</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mi>c</mml:mi><mml:mi>a</mml:mi><mml:mi>m</mml:mi></mml:mrow></mml:msub><mml:mo>=</mml:mo><mml:munder><mml:mo>&#x2211;</mml:mo><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:munder><mml:mrow><mml:mo>|</mml:mo><mml:msub><mml:mi>&#x03B1;</mml:mi><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msub><mml:mo>|</mml:mo></mml:mrow><mml:msup><mml:mi>A</mml:mi><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msup></mml:math></disp-formula></p>
<p>Type of &#x03B1;<sub><italic>k</italic></sub> said the weight of the K-th channel, <inline-formula id="ieqn-47"><mml:math id="mml-ieqn-47"><mml:msubsup><mml:mi>A</mml:mi><mml:mrow><mml:mi>i</mml:mi><mml:mi>j</mml:mi></mml:mrow><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msubsup></mml:math></inline-formula> as the K-th channel output characteristic figure, <inline-formula id="ieqn-48"><mml:math id="mml-ieqn-48"><mml:msup><mml:mi>A</mml:mi><mml:mrow><mml:mi>k</mml:mi></mml:mrow></mml:msup></mml:math></inline-formula> in the element value of coordinates (i, j), p and q respectively extraordinary clinic figure the length and width, <inline-formula id="ieqn-49"><mml:math id="mml-ieqn-49"><mml:mi>Z</mml:mi><mml:mo>=</mml:mo><mml:mi>p</mml:mi><mml:mo>&#x2217;</mml:mo><mml:mi>q</mml:mi></mml:math></inline-formula> and <inline-formula id="ieqn-50"><mml:math id="mml-ieqn-50"><mml:msub><mml:mi>L</mml:mi><mml:mrow><mml:mi>G</mml:mi><mml:mi>r</mml:mi><mml:mi>a</mml:mi><mml:mi>d</mml:mi><mml:mo>&#x2212;</mml:mo><mml:mi>c</mml:mi><mml:mi>a</mml:mi><mml:mi>m</mml:mi></mml:mrow></mml:msub></mml:math></inline-formula> is the Grad-CAM [<xref ref-type="bibr" rid="ref-52">52</xref>].</p>
</sec>
<sec id="s4_3_2">
<label>4.3.2</label>
<title>Method of Determining Fault Characteristics</title>
<p>The PMSM fault diagnosis method based on Grad-CAM is divided into two stages: the selection of the thermal map and the determination of the original data corresponding to the thermal map. The specific implementation steps are described as follows.</p>
<p>Selection of heat map: The time domain signals of PMSM of different fault types obtained by co-simulation have 8000 data points, and the bilateral spectrum diagram generated after FFT transformation also has 8000 data points, which are evenly distributed on the horizontal axis of 32 kHz. Since the bilateral spectrum is axisymmetric, only the half spectrum composed of the first 4000 data points can be analyzed. The gray image based on the autocorrelation matrix takes 500 data points every 30 points to upgrade the dimension, so studying every heat map is unnecessary. Theoretically, all the data points on the single side spectral diagram can be covered by analyzing the Grad-CAM of Nos. 1, 17, 33, 49, 65, 81, 97, 113, and 116 gray images of demagnetization fault and eccentric fault.</p>
<p>Determine the raw data corresponding to the heat map: Since each gray image is dimensionally upgraded from 500 data points, the gray image is strictly diagonal symmetric, and the pixel points in each row and column reflect the intensity of the corresponding data points in the original data. Therefore, the corresponding pixel points in the unilateral spectrum can be found by analyzing the pixels in the warm color region in the thermal map. Thus, the fault characteristics of demagnetization and eccentricity of PMSM are determined.</p>
</sec>
</sec>
<sec id="s4_4">
<label>4.4</label>
<title>Fault Characteristics of Demagnetization and Static Eccentricity of PMSM</title>
<p>Grad-CAM of Nos. 1, 17, 33, 49, 65, 81, 97, 113, 116 gray images of 25% and 50% demagnetization faults were selected for analysis. The Grad-CAM is shown in <xref ref-type="fig" rid="fig-22">Figs. 22</xref> and <xref ref-type="fig" rid="fig-23">23</xref>.</p>
<fig id="fig-22">
<label>Figure 22</label>
<caption>
<title>Grad-CAM corresponding to different gray level images of 25% demagnetization fault: (a) No. 1; (b) No. 17; (c) No. 33; (d) No. 49; (e) No. 65; (f) No. 81; (g) No. 97; (h) No. 113; (i) No. 116</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-22.tif"/>
</fig><fig id="fig-23">
<label>Figure 23</label>
<caption>
<title>Grad-CAM corresponding to different gray level images of 50% demagnetization fault: (a) No. 1; (b) No. 17; (c) No. 33; (d) No. 49; (e) No. 65; (f) No. 81; (g) No. 97; (h) No. 113; (i) No. 116</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-23.tif"/>
</fig>
<p>The analysis of the Grad-CAM of 25% demagnetization and 50%PMSM gray image shows that the fault characteristics of demagnetization faults of different degrees are the same. There are 16 fault characteristics of demagnetization faults of PMSM. They are 140, 280, 440 Hz, 2.88, 3.44, 5.72, 6.68, 7.12, 7.48, 8, 9.68, 10, 12, 12.32, 12.64, 14.96 kHz; It is marked on the spectrum diagram of De25 and De50 fault PMSM, as shown in <xref ref-type="fig" rid="fig-24">Fig. 24</xref>.</p>
<fig id="fig-24">
<label>Figure 24</label>
<caption>
<title> Fault characteristics corresponding to different demagnetizing faults: (a) De25; (b) De50</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-24.tif"/>
</fig>
<p>Similarly, Grad-CAM of Nos. 1, 17, 33, 49, 65, 81, 97, 113, and 116 gray images with 10% and 20% static eccentricity faults are analyzed. The gradient-type activation thermal maps (Grad-CAM) are shown in <xref ref-type="fig" rid="fig-25">Figs. 25</xref> and <xref ref-type="fig" rid="fig-26">26</xref>.</p>
<fig id="fig-25">
<label>Figure 25</label>
<caption>
<title>Grad-CAM corresponding to different gray level images of 10% static eccentricity fault: (a) No. 1; (b) No. 17; (c) No. 33; (d) No. 49; (e) No. 65; (f) No. 81; (g) No. 97; (h) No. 113; (i) No. 116</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-25.tif"/>
</fig><fig id="fig-26">
<label>Figure 26</label>
<caption>
<title>Grad-CAM corresponding to different gray level images of 20% static eccentricity fault: (a) No. 1; (b) No. 17; (c) No. 33; (d) No. 49; (e) No. 65; (f) No. 81; (g) No. 97; (h) No. 113; (i) No. 116</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-26.tif"/>
</fig>
<p>Through the analysis of the Grad-CAM thermal map, it can be found that there are 21 fault features in Se10, were 180 Hz, 2, 3.04, 3.84, 4.16, 4.64, 4.96, 5.44, 5.76, 6.76, 8, 9.8, 10.88, 11.72, 12.24, 13.32, 14.16, 14.64, 14.88, 15.24, 15.72 kHz, it is marked on the PMSM spectrum diagram of Se10 and Se20 faults, as shown in <xref ref-type="fig" rid="fig-27">Fig. 27</xref>.</p>
<fig id="fig-27">
<label>Figure 27</label>
<caption>
<title>Fault characteristics corresponding to different demagnetizing faults: (a) Se10; (b) Se20</title>
</caption>
<graphic mimetype="image" mime-subtype="tif" xlink:href="CMC_42313-fig-27.tif"/>
</fig>
</sec>
</sec>
<sec id="s5">
<label>5</label>
<title>Discussion</title>
<p>In this paper, through the co-simulation of AltairFlux and MATLAB, the time domain signals of stator current in PMSM under different degrees of demagnetization and eccentricity fault are obtained. Then the frequency domain signals of the stator current are received by FFT. Then, it is transformed into a gray image by an image conversion method based on an autocorrelation matrix, and the data set is made. In Resnet-8, a multi-scale feature fusion module is constructed to extract image features of different levels, and the attention mechanism module is introduced to improve the feature extraction ability of CNN further. An ASPP is added in series at the end of CNN, and the sensitivity field of CNN is increased by hollow convolution. Its accuracy on the test set is 99.08%. In addition, given the defect that the traditional data-driven fault diagnosis method is unable to make interpretative analysis, this paper introduces Grad-CAM for interpretative analysis of different degrees of eccentricity and demagnetization faults, finds out their fault characteristics, and marks them on the spectrum diagram. However, the interpretative analysis method based on Grad-CAM proposed in this paper can only determine the general range of the symptoms and cannot accurately locate the fault characteristics. A plan can be found to locate different fault characteristics in the following research accurately.</p>
<p>The experimental data in this paper are all co-simulated by Flux and MATLAB and do not involve complex scenes with multiple heterogeneous targets. However, our research team is purchasing the faulty motor and the corresponding drive platform and plans to simulate better the operation state of permanent magnet synchronous motor on new energy vehicles in future research.</p>
</sec>
<sec id="s6">
<label>6</label>
<title>Conclusions</title>
<p>This paper presents a fault diagnosis method based on improved ResNet9 and Grad-CAM. It is used to diagnose demagnetization and eccentricity faults of permanent magnet synchronous motors and determine their fault frequency. The data set used in this paper was obtained by joint simulation of Flux and MATLAB. The data set contained 7,500 fault images of different types of permanent magnet synchronous motors, the specific parameters shown in <xref ref-type="table" rid="table-2">Table 2</xref>. The overall accuracy of the improved ResNet9 network fault diagnosis results proposed in this paper is 99.00%, which has higher diagnostic accuracy than other traditional deep learning algorithms and has good robustness in the face of different noise interference. In addition, the &#x201C;black box&#x201D; problem in neural network fault diagnosis is solved using Grad-CAM. The demagnetization and eccentricity faults of permanent magnet synchronous motors are analyzed, and their fault characteristics are determined.</p>
</sec>
</body>
<back>
<ack>
<p>The authors would like to thank the Scientific Committee of the National Nature Foundation of China and the Scientific Committee of the Guizhou Provincial Nature Foundation for their funding of this research project.</p>
</ack>
<sec><title>Funding Statement</title>
<p>This study was funded by National Natural Science Foundation of China (Grant Numbers 51867006, 51867007) and the Natural Science and Technology Foundation of the Guizhou Province, China (Grant Numbers [2018]5781, [2018]1029).</p>
</sec>
<sec><title>Author Contributions</title>
<p>Study conception and design: Youming Guo, Qinmu Wu; data collection: Youming Guo; analysis and interpretation of results: Youming Guo; draft manuscript preparation: Youming Guo, Qinmu Wu. All authors reviewed the results and approved the final version of the manuscript.</p>
</sec>
<sec sec-type="data-availability"><title>Availability of Data and Materials</title>
<p>The data using in this paper are collected simulation experiment, and the modeling method is showed clearly in paper.</p>
</sec>
<sec sec-type="COI-statement"><title>Conflicts of Interest</title>
<p>The authors declare that they have no conflicts of interest to report regarding the present study.</p>
</sec>
<ref-list content-type="authoryear">
<title>References</title>
<ref id="ref-1"><label>[1]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>A. G.</given-names> <surname>Espinosa</surname></string-name>, <string-name><given-names>J. A.</given-names> <surname>Rosero</surname></string-name> and <string-name><given-names>J.</given-names> <surname>Cusido</surname></string-name></person-group>, &#x201C;<article-title>Fault detection by means of Hilbert&#x2013;Huang transform of the stator current in a PMSM with demagnetization</article-title>,&#x201D; <source>IEEE Transactions on Energy Conversion</source>, vol. <volume>25</volume>, no. <issue>2</issue>, pp. <fpage>312</fpage>&#x2013;<lpage>318</lpage>, <year>2010</year>.</mixed-citation></ref>
<ref id="ref-2"><label>[2]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>J. D.</given-names> <surname>McFarland</surname></string-name> and <string-name><given-names>T. M.</given-names> <surname>Jahns</surname></string-name></person-group>, &#x201C;<article-title>Investigation of the rotor demagnetization characteristics of interior PM synchronous machines during fault conditions</article-title>,&#x201D; <source>IEEE Transactions on Industry Applications</source>, vol. <volume>50</volume>, no. <issue>4</issue>, pp. <fpage>2768</fpage>&#x2013;<lpage>2775</lpage>, <year>2014</year>.</mixed-citation></ref>
<ref id="ref-3"><label>[3]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S. K.</given-names> <surname>Kommuri</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Defoort</surname></string-name> and <string-name><given-names>H. R.</given-names> <surname>Karimi</surname></string-name></person-group>, &#x201C;<article-title>A robust observer-based sensor fault-tolerant control for PMSM in electric vehicles</article-title>,&#x201D; <source>IEEE Transactions on Industrial Electronics</source>, vol. <volume>63</volume>, no. <issue>12</issue>, pp. <fpage>7671</fpage>&#x2013;<lpage>7681</lpage>, <year>2016</year>.</mixed-citation></ref>
<ref id="ref-4"><label>[4]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>Y.</given-names> <surname>Gao</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Qu</surname></string-name> and <string-name><given-names>D.</given-names> <surname>Li</surname></string-name></person-group>, &#x201C;<article-title>Improved hybrid method to calculate inductances of permanent magnet synchronous machines with skewed stators based on winding function theory</article-title>,&#x201D; <source>Chinese Journal of Electrical Engineering</source>, vol. <volume>2</volume>, no. <issue>1</issue>, pp. <fpage>52</fpage>&#x2013;<lpage>61</lpage>, <year>2016</year>.</mixed-citation></ref>
<ref id="ref-5"><label>[5]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>W.</given-names> <surname>Sun</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Hang</surname></string-name> and <string-name><given-names>S.</given-names> <surname>Ding</surname></string-name></person-group>, &#x201C;<article-title>Electromagnetic parameters analysis of inter-turn short circuit fault in DTP-PMSM based on finite element method</article-title>,&#x201D; in <conf-name>Int. Conf. on Power Electronics Systems and Applications (PESA)</conf-name>, <publisher-loc>Hong Kong, China</publisher-loc>, pp. <fpage>1</fpage>&#x2013;<lpage>4</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-6"><label>[6]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S.</given-names> <surname>Fu</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Qiu</surname></string-name> and <string-name><given-names>L.</given-names> <surname>Chen</surname></string-name></person-group>, &#x201C;<article-title>Adaptive fuzzy observer-based fault estimation for a class of nonlinear stochastic hybrid systems</article-title>,&#x201D; <source>IEEE Transactions on Fuzzy Systems</source>, vol. <volume>30</volume>, no. <issue>1</issue>, pp. <fpage>39</fpage>&#x2013;<lpage>51</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-7"><label>[7]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Eftekhari</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Moallem</surname></string-name> and <string-name><given-names>S.</given-names> <surname>Sadri</surname></string-name></person-group>, &#x201C;<article-title>Online detection of Induction motor&#x2019;s stator winding short-circuit faults</article-title>,&#x201D; <source>IEEE Systems Journal</source>, vol. <volume>8</volume>, no. <issue>4</issue>, pp. <fpage>1272</fpage>&#x2013;<lpage>1282</lpage>, <year>2014</year>.</mixed-citation></ref>
<ref id="ref-8"><label>[8]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>H.</given-names> <surname>Keskes</surname></string-name> and <string-name><given-names>A.</given-names> <surname>Braham</surname></string-name></person-group>, &#x201C;<article-title>Recursive undecimated wavelet packet transform and DAG SVM for Induction motor diagnosis</article-title>,&#x201D; <source>IEEE Transactions on Industrial Informatics</source>, vol. <volume>11</volume>, no. <issue>5</issue>, pp. <fpage>1059</fpage>&#x2013;<lpage>1066</lpage>, <year>2015</year>.</mixed-citation></ref>
<ref id="ref-9"><label>[9]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>N. P.</given-names> <surname>Nguyen</surname></string-name>, <string-name><given-names>N. X.</given-names> <surname>Mung</surname></string-name> and <string-name><given-names>L. N. N.</given-names> <surname>Thanh Ha</surname></string-name></person-group>, &#x201C;<article-title>Finite-time attitude fault tolerant control of quadcopter system via neural networks</article-title>,&#x201D; <source>Mathematics</source>, vol. <volume>8</volume>, no. <issue>9</issue>, pp. <fpage>1541</fpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-10"><label>[10]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>N. P.</given-names> <surname>Nguyen</surname></string-name>, <string-name><given-names>N. X.</given-names> <surname>Mung</surname></string-name> and <string-name><given-names>H. L. N. N.</given-names> <surname>Thanh</surname></string-name></person-group>, &#x201C;<article-title>Adaptive sliding mode control for attitude and altitude system of a quadcopter UAV via the neural network</article-title>,&#x201D; <source>IEEE Access</source>, vol. <volume>9</volume>, pp. <fpage>40076</fpage>&#x2013;<lpage>40085</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-11"><label>[11]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>J.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Liu</surname></string-name> and <string-name><given-names>Z.</given-names> <surname>Wang</surname></string-name></person-group>, &#x201C;<article-title>Convolutional neural network for crowd counting on metro platforms</article-title>,&#x201D; <source>Symmetry</source>, vol. <volume>13</volume>, no. <issue>4</issue>, pp. <fpage>703</fpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-12"><label>[12]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S. M. M.</given-names> <surname>Hossain</surname></string-name>, <string-name><given-names>K.</given-names> <surname>Deb</surname></string-name> and <string-name><given-names>P. K.</given-names> <surname>Dhar</surname></string-name></person-group>, &#x201C;<article-title>Plant leaf disease recognition using depth-wise separable convolution-based models</article-title>,&#x201D; <source>Symmetry</source>, vol. <volume>13</volume>, no. <issue>3</issue>, pp. <fpage>511</fpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-13"><label>[13]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>P. K.</given-names> <surname>Kankar</surname></string-name>, <string-name><given-names>S. C.</given-names> <surname>Sharma</surname></string-name> and <string-name><given-names>S. P.</given-names> <surname>Harsha</surname></string-name></person-group>, &#x201C;<article-title>Fault diagnosis of ball bearings using machine learning methods</article-title>,&#x201D; <source>Expert Systems with Applications</source>, vol. <volume>38</volume>, no. <issue>3</issue>, pp. <fpage>1876</fpage>&#x2013;<lpage>1886</lpage>, <year>2011</year>.</mixed-citation></ref>
<ref id="ref-14"><label>[14]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>R.</given-names> <surname>Liu</surname></string-name>, <string-name><given-names>G.</given-names> <surname>Meng</surname></string-name> and <string-name><given-names>B.</given-names> <surname>Yang</surname></string-name></person-group>, &#x201C;<article-title>Dislocated time series convolutional neural architecture: An intelligent fault diagnosis approach for an electric machine</article-title>,&#x201D; <source>IEEE Transactions on Industrial Informatics</source>, vol. <volume>13</volume>, no. <issue>3</issue>, pp. <fpage>1310</fpage>&#x2013;<lpage>1320</lpage>, <year>2016</year>.</mixed-citation></ref>
<ref id="ref-15"><label>[15]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>W.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Li</surname></string-name> and <string-name><given-names>G.</given-names> <surname>Peng</surname></string-name></person-group>, &#x201C;<article-title>A deep convolutional neural network with new training methods for bearing fault diagnosis under noisy environment and different working load</article-title>,&#x201D; <source>Mechanical Systems and Signal Processing</source>, vol. <volume>100</volume>, no. <issue>6</issue>, pp. <fpage>439</fpage>&#x2013;<lpage>453</lpage>, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-16"><label>[16]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>I. H.</given-names> <surname>Kao</surname></string-name>, <string-name><given-names>W. J.</given-names> <surname>Wang</surname></string-name> and <string-name><given-names>Y. H.</given-names> <surname>Lai</surname></string-name></person-group>, &#x201C;<article-title>Analysis of permanent magnet synchronous motor fault diagnosis based on learning</article-title>,&#x201D; <source>IEEE Transactions on Instrumentation and Measurement</source>, vol. <volume>68</volume>, no. <issue>2</issue>, pp. <fpage>310</fpage>&#x2013;<lpage>324</lpage>, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-17"><label>[17]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>A.</given-names> <surname>Jawaharlalnehru</surname></string-name>, <string-name><given-names>T.</given-names> <surname>Sambandham</surname></string-name>, <string-name><given-names>V.</given-names> <surname>Sekar</surname></string-name>, <string-name><given-names>D.</given-names> <surname>Ravikumar</surname></string-name>, <string-name><given-names>V.</given-names> <surname>Loganathan</surname></string-name> <etal>et al.</etal></person-group><italic>,</italic> &#x201C;<article-title>Target object detection from unmanned aerial vehicle (UAV) images based on improved YOLO algorithm</article-title>,&#x201D; <source>Electronics</source>, vol. <volume>11</volume>, pp. <fpage>2343</fpage>, <year>2023</year>.</mixed-citation></ref>
<ref id="ref-18"><label>[18]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Anul Haq</surname></string-name></person-group>, &#x201C;<article-title>CNN based automated weed detection system using UAV imagery</article-title>,&#x201D; <source>Computer Systems Science and Engineering</source>, vol. <volume>42</volume>, no. <issue>2</issue>, pp. <fpage>837</fpage>&#x2013;<lpage>849</lpage>, <year>2022</year>.</mixed-citation></ref>
<ref id="ref-19"><label>[19]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Anul Haq</surname></string-name></person-group>, &#x201C;<article-title>CDLSTM: A novel model for climate change forecasting</article-title>,&#x201D; <source>Computers, Materials &#x0026; Continua</source>, vol. <volume>71</volume>, no. <issue>2</issue>, pp. <fpage>2363</fpage>&#x2013;<lpage>2381</lpage>, <year>2022</year>.</mixed-citation></ref>
<ref id="ref-20"><label>[20]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Anul Haq</surname></string-name></person-group>, &#x201C;<article-title>DCNNBT: A novel deep convolutional neural network-based brain tumor classification model</article-title>,&#x201D; <source>Fractals</source>, vol. <volume>31</volume>, no. <issue>6</issue>, pp. <fpage>2340102</fpage>&#x2013;<lpage>2340128</lpage>, <year>2023</year>.</mixed-citation></ref>
<ref id="ref-21"><label>[21]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Anul Haq</surname></string-name></person-group>, &#x201C;<article-title>Implementation of CNN for plant identification using UAV imagery</article-title>,&#x201D; <source>International Journal of Advanced Computer Science and Applications (IJACSA)</source>, vol. <volume>14</volume>, no. <issue>4</issue>, pp. <fpage>830</fpage>&#x2013;<lpage>847</lpage>, <year>2023</year>.</mixed-citation></ref>
<ref id="ref-22"><label>[22]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>I. H.</given-names> <surname>Kao</surname></string-name>, <string-name><given-names>W. J.</given-names> <surname>Wang</surname></string-name> and <string-name><given-names>Y. H.</given-names> <surname>Lai</surname></string-name></person-group>, &#x201C;<article-title>Analysis of permanent magnet synchronous motor fault diagnosis based on learning</article-title>,&#x201D; <source>IEEE Transactions on Instrumentation and Measurement</source>, vol. <volume>68</volume>, no. <issue>2</issue>, pp. <fpage>310</fpage>&#x2013;<lpage>324</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-23"><label>[23]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>A. L.</given-names> <surname>Rodriguez</surname></string-name>, <string-name><given-names>L.</given-names> <surname>Huang</surname></string-name> and <string-name><given-names>P.</given-names> <surname>Lombard</surname></string-name></person-group>, &#x201C;<article-title>Vibration analysis of a PMSM through FEM multiphysics simulation with experimental validation</article-title>,&#x201D; in <conf-name>Int. Conf. on Electrical Machines (ICEM)</conf-name>, <publisher-loc>Hong Kong, China</publisher-loc>, pp. <fpage>2560</fpage>&#x2013;<lpage>2566</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-24"><label>[24]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>Y.</given-names> <surname>Qin</surname></string-name> and <string-name><given-names>X.</given-names> <surname>Shi</surname></string-name></person-group>, &#x201C;<article-title>Fault diagnosis method for rolling bearings based on two-channel CNN under unbalanced datasets</article-title>,&#x201D; <source>Applied Sciences</source>, vol. <volume>12</volume>, pp. <fpage>8474</fpage>, <year>2022</year>.</mixed-citation></ref>
<ref id="ref-25"><label>[25]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>C.</given-names> <surname>Fu</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Gui</surname></string-name> and <string-name><given-names>F.</given-names> <surname>Akter</surname></string-name></person-group>, &#x201C;<article-title>Discrete fourier transform (DFT)-based computational intelligence model for urban carbon emission and economic growth</article-title>,&#x201D; <source>Mathematical Problems in Engineering</source>, vol. <volume>2022</volume>, pp. <fpage>1</fpage>&#x2013;<lpage>10</lpage>, <year>2022</year>.</mixed-citation></ref>
<ref id="ref-26"><label>[26]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>A.</given-names> <surname>Abbassi</surname></string-name>, <string-name><given-names>R. B.</given-names> <surname>Mehrez</surname></string-name> and <string-name><given-names>R.</given-names> <surname>Abbassi</surname></string-name></person-group>, &#x201C;<article-title>Improved off-grid wind/photovoltaic/hybrid energy storage system based on a new framework of Moth-Flame optimization algorithm</article-title>,&#x201D; <source>International Journal of Energy Research</source>, vol. <volume>46</volume>, no. <issue>5</issue>, pp. <fpage>6711</fpage>&#x2013;<lpage>6729</lpage>, <year>2022</year>.</mixed-citation></ref>
<ref id="ref-27"><label>[27]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>Z.</given-names> <surname>Li</surname></string-name>, <string-name><given-names>Q.</given-names> <surname>Wu</surname></string-name> and <string-name><given-names>S.</given-names> <surname>Yang</surname></string-name></person-group>, &#x201C;<article-title>Diagnosis of rotor demagnetization and eccentricity faults for PMSM based on deep CNN and image recognition</article-title>,&#x201D; <source>Complex &#x0026; Intelligent Systems</source>, vol. <volume>8</volume>, pp. <fpage>5469</fpage>&#x2013;<lpage>5488</lpage>, <year>2022</year>.</mixed-citation></ref>
<ref id="ref-28"><label>[28]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>O.</given-names> <surname>Moutik</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Sekkat</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Tigani</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Chehri</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Saadane</surname></string-name> <etal>et al.</etal></person-group><italic>,</italic> &#x201C;<article-title>Convolutional neural networks or vision transformers: Who will win the race for action recognitions in visual data</article-title>,&#x201D; <source>Sensors</source>, vol. <volume>23</volume>, no. <issue>2</issue>, pp. <fpage>734</fpage>, <year>2023</year>; <pub-id pub-id-type="pmid">36679530</pub-id></mixed-citation></ref>
<ref id="ref-29"><label>[29]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>A.</given-names> <surname>Barua</surname></string-name>, <string-name><given-names>M. U.</given-names> <surname>Ahmed</surname></string-name> and <string-name><given-names>S.</given-names> <surname>Begum</surname></string-name></person-group>, &#x201C;<article-title>A systematic literature review on multimodal machine learning: Applications, challenges, gaps, and future directions</article-title>,&#x201D; <source>IEEE Access</source>, vol. <volume>11</volume>, pp. <fpage>14804</fpage>&#x2013;<lpage>14831</lpage>, <year>2023</year>.</mixed-citation></ref>
<ref id="ref-30"><label>[30]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>Y.</given-names> <surname>Yu</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>Q.</given-names> <surname>Fu</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Kou</surname></string-name>, <string-name><given-names>F.</given-names> <surname>Huang</surname></string-name> <etal>et al.</etal></person-group><italic>,</italic> &#x201C;<article-title>Techniques and challenges of image segmentation: A review</article-title>,&#x201D; <source>Electronics</source>, vol. <volume>12</volume>, pp. <fpage>1199</fpage>, <year>2023</year>.</mixed-citation></ref>
<ref id="ref-31"><label>[31]</label><mixed-citation publication-type="book"><person-group person-group-type="author"><string-name><given-names>S.</given-names> <surname>Deb</surname></string-name> and <string-name><given-names>S.</given-names> <surname>Sinha</surname></string-name></person-group>, &#x201C;<chapter-title>Comparative improvement of image segmentation performance with graph-based method over watershed transform image segmentation</chapter-title>,&#x201D; In: <person-group person-group-type="editor"><string-name><given-names>R.</given-names> <surname>Natarajan</surname></string-name></person-group> (<role>Ed.</role>), <source>Distributed Computing and Internet Technology: ICDCIT</source>, pp. <fpage>322</fpage>&#x2013;<lpage>332</lpage>, <publisher-loc>Cham, New York, NY, USA</publisher-loc>: <publisher-name>Springer</publisher-name>, <year>2014</year>.</mixed-citation></ref>
<ref id="ref-32"><label>[32]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>H.</given-names> <surname>Wang</surname></string-name> and <string-name><given-names>P. A.</given-names> <surname>Yushkevich</surname></string-name></person-group>, &#x201C;<article-title>Guiding automatic segmentation with multiple manual segmentations</article-title>,&#x201D; in <conf-name>Int. Conf. on Medical Image Computing and Computer-Assisted Intervention</conf-name>, <publisher-loc>Berlin, Heidelberg, Germany</publisher-loc>, <publisher-name>Springer</publisher-name>, pp. <fpage>429</fpage>&#x2013;<lpage>436</lpage>, <year>2012</year>.</mixed-citation></ref>
<ref id="ref-33"><label>[33]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>N.</given-names> <surname>Nikbakhsh</surname></string-name> and <string-name><given-names>Y.</given-names> <surname>Baleghi</surname></string-name></person-group>, &#x201C;<article-title>A new fast method of image segmentation fusion using maximum mutual information</article-title>,&#x201D; in <conf-name>Iranian Conf. on Electrical Engineering (ICEE)</conf-name>, <publisher-loc>Yazd, Iran</publisher-loc>, pp. <fpage>1584</fpage>&#x2013;<lpage>1588</lpage>, <year>2019</year>.</mixed-citation></ref>
<ref id="ref-34"><label>[34]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>Z.</given-names> <surname>Yan</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>Q.</given-names> <surname>Ning</surname></string-name> and <string-name><given-names>Y.</given-names> <surname>Lu</surname></string-name></person-group>, &#x201C;<article-title>Robust image matching based on image feature and depth information fusion</article-title>,&#x201D; <source>Machines</source>, vol. <volume>10</volume>, no. <issue>6</issue>, pp. <fpage>456</fpage>, <year>2022</year>.</mixed-citation></ref>
<ref id="ref-35"><label>[35]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>H.</given-names> <surname>Lee</surname></string-name> and <string-name><given-names>S.</given-names> <surname>Cho</surname></string-name></person-group>, &#x201C;<article-title>Locally adaptive channel attention-based network for denoising images</article-title>,&#x201D; <source>IEEE Access</source>, vol. <volume>8</volume>, pp. <fpage>34686</fpage>&#x2013;<lpage>34695</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-36"><label>[36]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>Y.</given-names> <surname>Yang</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>B.</given-names> <surname>Sun</surname></string-name> and <string-name><given-names>Q.</given-names> <surname>Zhao</surname></string-name></person-group>, &#x201C;<article-title>Channel expansion convolutional network for image classification</article-title>,&#x201D; <source>IEEE Access</source>, vol. <volume>8</volume>, pp. <fpage>178414</fpage>&#x2013;<lpage>178424</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-37"><label>[37]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S.</given-names> <surname>Seong</surname></string-name> and <string-name><given-names>J.</given-names> <surname>Choi</surname></string-name></person-group>, &#x201C;<article-title>Semantic segmentation of urban buildings using a high-resolution network (HRNet) with channel and spatial attention gates</article-title>,&#x201D; <source>Remote Sensing</source>, vol. <volume>13</volume>, pp. <fpage>3087</fpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-38"><label>[38]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M. H.</given-names> <surname>Park</surname></string-name>, <string-name><given-names>J. H.</given-names> <surname>Cho</surname></string-name> and <string-name><given-names>Y. T.</given-names> <surname>Kim</surname></string-name></person-group>, &#x201C;<article-title>CNN model with multilayer ASPP and two-step cross-stage segmentation</article-title>,&#x201D; <source>Machines</source>, vol. <volume>11</volume>, no. <issue>2</issue>, pp. <fpage>126</fpage>, <year>2023</year>.</mixed-citation></ref>
<ref id="ref-39"><label>[39]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S. Q.</given-names> <surname>Du</surname></string-name>, <string-name><given-names>S. J.</given-names> <surname>Tang</surname></string-name>, <string-name><given-names>W. X.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>X. M.</given-names> <surname>Li</surname></string-name>, <string-name><given-names>Y. H.</given-names> <surname>Lu</surname></string-name> <etal>et al.</etal></person-group><italic>,</italic> &#x201C;<article-title>PSCNET: Efficient RGB-D semantic segmentation parallel network based on spatial and channel attention</article-title>,&#x201D; <source>Remote Sensing</source>, vol. <volume>1</volume>, pp. <fpage>129</fpage>&#x2013;<lpage>136</lpage>, <year>2022</year>.</mixed-citation></ref>
<ref id="ref-40"><label>[40]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Hu</surname></string-name>, <string-name><given-names>K.</given-names> <surname>Cui</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>H.</given-names> <surname>Mao</surname></string-name> <etal>et al.</etal></person-group><italic>,</italic> &#x201C;<article-title>Animal migration patterns extraction based on atrous-gated CNN deep learning model</article-title>,&#x201D; <source>Remote Sensing</source>, vol. <volume>13</volume>, no. <issue>24</issue>, pp. <fpage>4998</fpage>, <year>2022</year>.</mixed-citation></ref>
<ref id="ref-41"><label>[41]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Abdullah Alkhonaini</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Ben Haj Hassine</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Obayya</surname></string-name>, <string-name><given-names>F. N.</given-names> <surname>Al-Wesabi</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Mustafa Hilal</surname></string-name> <etal>et al.</etal></person-group><italic>,</italic> &#x201C;<article-title>Detection of lung tumor using ASPP-Unet with a whale optimization algorithm</article-title>,&#x201D; <source>Computers, Materials &#x0026; Continua</source>, vol. <volume>72</volume>, no. <issue>2</issue>, pp. <fpage>3511</fpage>&#x2013;<lpage>3527</lpage>, <year>2022</year>.</mixed-citation></ref>
<ref id="ref-42"><label>[42]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>Y.</given-names> <surname>Li</surname></string-name>, <string-name><given-names>Z.</given-names> <surname>Cheng</surname></string-name>, <string-name><given-names>C.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Zhao</surname></string-name> and <string-name><given-names>L.</given-names> <surname>Huang</surname></string-name></person-group>, &#x201C;<article-title>RCCT-ASPPNet: Dual-encoder remote image segmentation based on transformer and ASPP</article-title>,&#x201D; <source>Remote Sensing</source>, vol. <volume>15</volume>, no. <issue>2</issue>, pp. <fpage>379</fpage>, <year>2023</year>.</mixed-citation></ref>
<ref id="ref-43"><label>[43]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>B.</given-names> <surname>Liu</surname></string-name>, <string-name><given-names>Q.</given-names> <surname>Wu</surname></string-name>, <string-name><given-names>Z.</given-names> <surname>Li</surname></string-name> and <string-name><given-names>X.</given-names> <surname>Chen</surname></string-name></person-group>, &#x201C;<article-title>Research on fault diagnosis of PMSM for electric vehicles based on multi-level feature fusion SPP network</article-title>,&#x201D; <source>Symmetry</source>, vol. <volume>13</volume>, pp. <fpage>1844</fpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-44"><label>[44]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>M.</given-names> <surname>Javed Awan</surname></string-name>, <string-name><given-names>M. S.</given-names> <surname>Mohd Rahim</surname></string-name>, <string-name><given-names>N.</given-names> <surname>Salim</surname></string-name>, <string-name><given-names>M. A.</given-names> <surname>Mohammed</surname></string-name>, <string-name><given-names>B.</given-names> <surname>Garcia-Zapirain</surname></string-name> <etal>et al.</etal></person-group><italic>,</italic> &#x201C;<article-title>Efficient detection of knee anterior cruciate ligament from magnetic resonance imaging using deep learning approach</article-title>,&#x201D; <source>Diagnostics</source>, vol. <volume>11</volume>, no. <issue>1</issue>, pp. <fpage>105</fpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-45"><label>[45]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>L.</given-names> <surname>Chen</surname></string-name>, <string-name><given-names>Z.</given-names> <surname>Zhang</surname></string-name> and <string-name><given-names>J.</given-names> <surname>Cao</surname></string-name></person-group>, &#x201C;<article-title>A novel method of combining nonlinear frequency spectrum and deep learning for complex system fault diagnosis&#x2014;ScienceDirect</article-title>,&#x201D; <source>Measurement</source>, vol. <volume>151</volume>, pp. <fpage>107190</fpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-46"><label>[46]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>T.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>Z.</given-names> <surname>Li</surname></string-name> and <string-name><given-names>Z.</given-names> <surname>Deng</surname></string-name></person-group>, &#x201C;<article-title>Hybrid data fusion DBN for intelligent fault diagnosis of vehicle reducers</article-title>,&#x201D; <source>Sensors</source>, vol. <volume>19</volume>, no. <issue>11</issue>, pp. <fpage>2504</fpage>, <year>2019</year>; <pub-id pub-id-type="pmid">31159290</pub-id></mixed-citation></ref>
<ref id="ref-47"><label>[47]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>J.</given-names> <surname>He</surname></string-name>, <string-name><given-names>J.</given-names> <surname>Lee</surname></string-name> and <string-name><given-names>T.</given-names> <surname>Song</surname></string-name></person-group>, &#x201C;<article-title>Recurrent neural network (RNN) for delay-tolerant repetition-coded (RC) indoor optical wireless communication systems</article-title>,&#x201D; <source>Optics Letters</source>, vol. <volume>44</volume>, no. <issue>15</issue>, pp. <fpage>3745</fpage>&#x2013;<lpage>3748</lpage>, <year>2019</year>; <pub-id pub-id-type="pmid">31368958</pub-id></mixed-citation></ref>
<ref id="ref-48"><label>[48]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>R. R.</given-names> <surname>Selvaraju</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Cogswell</surname></string-name> and <string-name><given-names>A.</given-names> <surname>Das</surname></string-name></person-group>, &#x201C;<article-title>Grad-CAM: Visual explanations from deep networks via gradient-based localization</article-title>,&#x201D; <source>IEEE International Conference on Computer Vision</source>, vol. <volume>128</volume>, no. <issue>2</issue>, pp. <fpage>336</fpage>&#x2013;<lpage>359</lpage>, <year>2020</year>.</mixed-citation></ref>
<ref id="ref-49"><label>[49]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S. H.</given-names> <surname>Wang</surname></string-name>, <string-name><given-names>Y.</given-names> <surname>Zhang</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Cheng</surname></string-name>, <string-name><given-names>X.</given-names> <surname>Zhang</surname></string-name> and <string-name><given-names>Y.</given-names> <surname>Zhang</surname></string-name></person-group>, &#x201C;<article-title>PSSPNN: PatchShuffle stochastic pooling neural network for an explainable diagnosis of COVID-19 with multiple-way data augmentation</article-title>,&#x201D; <source>Computational and Mathematical Methods in Medicine</source>, vol. <volume>2021</volume>, pp. <fpage>1</fpage>&#x2013;<lpage>18</lpage>, <year>2021</year>.</mixed-citation></ref>
<ref id="ref-50"><label>[50]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>A.</given-names> <surname>Chattopadhay</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Sarkar</surname></string-name>, <string-name><given-names>P.</given-names> <surname>Howlader</surname></string-name> and <string-name><given-names>V. N.</given-names> <surname>Balasubramanian</surname></string-name></person-group>, &#x201C;<article-title>Grad-CAM&#x002B;&#x002B;: Generalized gradient-based visual explanations for deep convolutional networks</article-title>,&#x201D; in <conf-name>IEEE Winter Conf. on Applications of Computer Vision (WACV)</conf-name>, <publisher-loc>Lake Tahoe, NV, USA</publisher-loc>, pp. <fpage>839</fpage>&#x2013;<lpage>847</lpage>, <year>2018</year>.</mixed-citation></ref>
<ref id="ref-51"><label>[51]</label><mixed-citation publication-type="conf-proc"><person-group person-group-type="author"><string-name><given-names>R. R.</given-names> <surname>Selvaraju</surname></string-name>, <string-name><given-names>M.</given-names> <surname>Cogswell</surname></string-name>, <string-name><given-names>A.</given-names> <surname>Das</surname></string-name>, <string-name><given-names>R.</given-names> <surname>Vedantam</surname></string-name>, <string-name><given-names>D.</given-names> <surname>Parikh</surname></string-name> <etal>et al.</etal></person-group><italic>,</italic> &#x201C;<article-title>Grad-CAM: Visual explanations from deep networks via gradient-based localization</article-title>,&#x201D; in <conf-name>IEEE Int. Conf. on Computer Vision (ICCV), Venice</conf-name>, <publisher-loc>Venice, Italy</publisher-loc>, pp. <fpage>618</fpage>&#x2013;<lpage>626</lpage>, <year>2017</year>.</mixed-citation></ref>
<ref id="ref-52"><label>[52]</label><mixed-citation publication-type="journal"><person-group person-group-type="author"><string-name><given-names>S.</given-names> <surname>Kolekar</surname></string-name>, <string-name><given-names>S.</given-names> <surname>Gite</surname></string-name>, <string-name><given-names>B.</given-names> <surname>Pradhan</surname></string-name> and <string-name><given-names>A.</given-names> <surname>Alamri</surname></string-name></person-group>, &#x201C;<article-title>Explainable AI in scene understanding for autonomous vehicles in unstructured traffic environments on Indian roads using the inception U-Net model with Grad-CAM visualization</article-title>,&#x201D; <source>Sensors</source>, vol. <volume>22</volume>, no. <issue>24</issue>, pp. <fpage>9677</fpage>, <year>2022</year>; <pub-id pub-id-type="pmid">36560047</pub-id></mixed-citation></ref>
</ref-list>
</back></article>