The MobileNet model focuses on depthwise separable convolutions, a factorized convolution that splits a standard convolution into a depth-wise and a pointwise convolution. The depth-wise convolution used by Mobile Nets utilizes a single filter for each input channel. The outputs of depth-wise convolution are then combined using an 1 × 1 convolution by pointwise convolution. This is divided into two layers by the depthwise separable convolutions; one for filtering and the other for connecting. This factorization has the benefit of significantly lowering the computing time. An improved module with an inverted residual structure has been added to MobileNetV2 [23]. The first layer is 1 × 1 convolution with Rectified Linear Unit (ReLU). The depthwise convolution is the second layer. Another 1 × 1 convolution is used in the third layer with no non-linearity. Deep networks will only have the power of a linear classifier on the non-zero volume output domain if ReLU is applied again.

The projection layer, a pointwise convolutional layer in V2, converts the data with many channels into a tensor with much fewer channels. Before going into the depth-wise convolutional layer, a 1 × 1 expansion of the convolutional layer extends the number of channels based on the expansion factor in the results. Each output of the block is a holdup in the bottleneck residual block. The residual connection exists to aid gradient flow through the network. Batch normalization and the ReLU6 activation features are included in each layer of MobileNetV2.

MobileNet V2 architecture includes 17 bottleneck residual blocks in a row, a standard 1 × 1 convolution, and a classification layer. The pre-trained MobileNetV2 model was adapted for the cephalopod classification task in the proposed work, and its architecture is shown in Fig. 2. In the final classification section, fully connected dense layers with dropouts are added accordingly to achieve higher accuracy of classification. A softmax layer, an output classification layer, replaces the final classification layers of the model. The proposed RAdamGC optimizer is used in the fine-tuned architecture, and it is explained in Section 3.4 to increase the training speed of the network. The learning rate factor is adjusted in fully connected layers to expedite learning in the new ending layers. Training parameters such as learning rate, maximum epochs, mini-batch size, and validation data are being set to train the network using the cephalopod data.

The brain team of google has created the Neural Architecture Search Network (NASNet), which has two main cell functions; (1) Normal cell and (2) Reduction Cell. NASNet [24] has performed its operations on a small dataset before transferring its block to a larger dataset to achieve a higher mean Average Precision (mAP). The output of NASNet is enhanced by a tweaked drop path called Scheduled drop path for successful regularization. In the first NASNet, the number of cells in the architecture is not pre-determined. Normal and reduction cells are used in particular. The size of the feature map is described by normal cells, and the reduction cell returns the reduced feature map in terms of height and width by a factor of two.

The controlled architecture of NASNet, which is built on a Recurrent Neural Network (RNN), predicts the entire structure of the network based on the two initial hidden states. The Controller Architecture employs an RNN-based LSTM model, with softmax prediction for convolutional cells prediction and recursive construction of network motifs. In NASNet Mobile, the input image size used is 224 × 224. For the transfer learning process, NASNetMobile employs ImageNet pre-trained network weights to classify the cephalopods. The classification layers of the pre-trained NASNet Mobile model are fine-tuned as shown in Fig. 3 for the cephalopod classification task. The model is fine-tuned with three fully connected layers with dropouts, a softmax layer, and an output classification layer to achieve a high categorization rate. The learning rate factor with fully connected layers are increased to accelerate the learning in the new final layers with the proposed optimizer. The performance of the model is evaluated by tweaking higher parameters.

The advent of Inception models inhibits the stacking layer mode, allowing us to expand the depth and width in a new way. Because the variety of modules in an inception model is more complex to design than a Visual Geometry Group (VGG). On the other hand, inception models can be built more extensively and broadly under the same computational budget, resulting in greater accuracy than a VGG. The Inception paradigm has gone through four generations; Inception-V1, Inception-V2, Inception-V3, and Inception-V4. Inception-V2 replaces the 5 × 5 convolutional layers of Inception v1with two 3 × 3 convolutional layers in a row. Asymmetric convolutions are used in Inception-V3 and Inception-V4 to expand the depth, although the former has fewer model parameters. The classification layers of the pre-trained InceptionV3 model have been fine-tuned for our cephalopod classification task. The fully connected layers with dropouts, a softmax layer, and an output classification layer replace the final layers of the model. The learning rate factor with fully connected layers is varied to accelerate the learning in the new final layers. The performance is observed by tweaking the other hyperparameters.

A feed-forward network, the Dense Convolutional Network (DenseNet), in which each layer is linked to every other layer. Direct connections L(L + 1)/2 are used in this network. All the feature maps of previous levels are used as inputs into each layer, and the feature maps of the layer are being used as inputs in all the following layers. DenseNets have several compelling advantages, including eliminating the vanishing-gradient problem, improved feature propagation, reuse of features, and a considerable reduction in the number of parameters. For Cephalopod classification, DenseNet201 is used. The DenseNet201 is a 201-layer deep CNN. It is a member of the DenseNet family of image classification models.

Xception is a more advanced variant of the Inception model that independently performs channel-wise and cross-channel convolutions. The primary operator in Xception is a separable convolution, which significantly reduces the parameters of regular convolutions. Xception uses a smaller window (3 × 3) for channel-wise convolution, which is similar to VGG networks. 1 × 1 standard convolutions are performed for cross-channel convolution. Inception models outperform the other types of networks in the classification of accuracy because of the multi-scale processing modules within them.

The ResNet-50 model and aggregation of ResNet models of various depths gained recognition for image classification. ResNet-50 is a residual network of 50 layers that aims to resolve the issue of vanishing gradients in CNN during back-propagation. Increasing the depth of the network can enhance its accuracy if over-fitting is taken into consideration. As the depth of the network rises, the signal needed to modify the weights, which originates from comparing the ground truth and prediction (seen vs. forecasted) at the end of the network, becomes extremely tiny at the early layers. Essentially, it implies that the layers before it is nearly unlearned. The ResNet-50 model relies heavily on the convolution blocks. Multiple filters are applied on these networks to distinguish the images, and the filters are passed over the original image of the strides. The values from the learned filters are multiplied by the values from the images. The outputs of the filters are pooled, thereby down sampling while preserving the essential characteristics.

Rectified Adam, often known as Adam, is a stochastic optimizer variation that adds a term to correct the variance of adaptive learning rate. In order to improve the accuracy in cephalopod classification, the optimizer Rectified Adam proposed in [25] is adopted and fine-tuned.

The values of ϕ(.) and ψ(.) , may be changed to provide different optimization techniques, where ϕ(.) indicates how the momentum is determined at time step t and ψ(.) specifies how the adaptive rate of learning is calculated. For example, in the Adam algorithm,

(1) ϕ(g1,⋯,gt)=(1−β1)∑i=1tβ1t−igi1−β1tandψ(g1,⋯,gt)=1−β2t(1−β2)∑i=1tβ2t−igi2

The function ψ(.) in Eq. (1) is commonly computed as in Eq. (2).

(2) ψ^(g1,⋯,gt)=1−β2ϵ+(1−β2)∑i=1tβ2t−tg12

In generic adaptive optimisation method, calculation of gradients with respect to stochastic objective at time step t, momentum, adaptive learning rate, updating parameters are given as follows (Eqs. (3)–(6)).

(3) gt←∇θft(θt−1)

(4) mt←ϕt(g1,⋯,gt)

(5) lt←ψt(g1,⋯,gt)

(6) θt←θt−1−αtmtlt

In real-world applications, the Exponential Moving Average (EMA) can be interpreted as a close approximation to the Simple Moving Average (SMA), which is given in Eq. (7).

(7) p((1−β2)∑i=1tβ2t−igi21−β2t)≈p(∑i=1f(t,β2)gt+1−i2f(t,β2))

where f(t,β2) is the length of SMA, allowing the SMA and the EMA to share the same “center of mass.” In other words, f(t,β2) satisfies the following condition which is given in Eq. (8).

(8) (1−β2)∑i=1tβ2t−i⋅i1−β2t=∑i=1f(t,β2)(t+1−i)f(t,β2)

By solving Eq. (8), we get the Eq. (9) as follows:

(9) f(t,β2)=21−β2−1−2tβ2t1−β2t

For ease of notation, f(t,β2) is marked as ρt . Also, we refer 21−β2−1asρ∞ (maximum length of the approximated SMA), due to the inequality f(t,β2)≤limt→∞⁡f(t,β2)21−β2−1

The rectification term can be calculated as in Eq. (10):

(10) rt=(ρt−4)(ρt−2)ρ∞(ρ∞−4)(ρ∞−2)ρt

Gradient Centralization (GC) is a technique [26] for bringing gradient vectors to a zero mean. GC improves the gradient of the loss function, enhancing the efficiency and consistency of the training process.

It is assumed that the gradient of an FC or convolution layer via propagating backwards is obtained, then, for a weight vector wi whose gradient is ∇wiL(i=1,2,…,N) , the GC operator, denoted by ΦGC , is given in Eq. (11).

(11) ΦGC(∇wiL)=∇wiL−μ∇wiL

where represent the gradient of L w.r.t. the weight matrix W and weight vector wi, respectively.

The GC formula is pretty straight forward. The mean of the weight matrix's column vectors must be computed, and then the mean must be removed from each column vector.

In optimising the cephalopod classification, GC is embedded with RAdam to boost the performance of the network model and to reduce the training time extensively. It is proposed to have an algorithm RAdamGC to raise the performance of the model and save time for training. To embed GC with RAdam, the parameter updates are carried out as shown in Eqs. (12)–(16) and embedded with RAdam algorithm.

(12) gt←∇θft(θt−1)

(13) gt^←∅GC(gt)

(14) mt←β1mt−1+(1−β1)gt^

(15) vt←β2vt−1+(1−β2)gt^⊙gt^

(16) mt^←mt(1−β1t)

With the equation, gt^←∅GC(gt) gradient centralisation is applied for calculating the gradients at timestamp t with respect to stochastic objective. Then exponential moving 1st moment mt←β1mt−1+(1−β1)gt^ is calculated with GC. Bias corrected moving average is calculated with mt^←mt/(1−β1t) . In the classification task, the proposed algorithm is utilized using initial values of ρt→7 for MobileNetV2 and ρt→6 for InceptionV3 and NASNet Mobile. For MobileNetV2, InceptionV3, and NASNet Mobile, the initial learning rate is 1e−8, 1e−6, and le−5, respectively.

An improved classification model with dense layers is used to improve the accuracy of classification. Initially, the optimizer Adam is used with the default learning rate. According to the results of the fine-tuned models, there is a significant improvement in the inaccuracy. MobileNetV2 has achieved an accuracy of 84.62%, while the other lightweight model NASNet Mobile has achieved only 84.08%. The classification accuracy of InceptionV3 is 84.61%. Fig. 5 compares the models that have and do not have fine-tuning. Fine-tuning has improved the performance of all models.

The performance of MobileNetV2 has significantly improved since the models are fine-tuned. The accuracy of the models NASNET Mobile and InceptionV3 fine-tuned with dense layers have increased by 2%. The accuracy of MobileNetV2 has increased by 4%.

In an optimization algorithm, the Lr is a tweaking parameter that controls the step size at each iteration while progressing towards a minimum of loss function. A higher learning rate causes the loss function to rise, whereas a slower learning rate causes the loss function to decline progressively.

The best learning rate must be determined to minimize the loss function in the cephalopod classification task. To assess the performance of the model, the models are now trained using learning rates of 1e−1, 1e−2, 1e−3, 1e−4, and 1e−5. Tab. 1 gives the accuracy values when the model is tested with varied learning rates. The MobileNetV2 model performs the best for the Lr of 1e−3. It gives an accuracy of 84.71% with Lr of 1e−3. NASNet Mobile gives the accuracy of 84.62% with the Lr of 1e−5. With the Lr, 1e−4, InceptionV3 gives the accuracy of 84.63%.