Fruit classification is found to be one of the rising fields in computer and machine vision. Many deep learning-based procedures worked out so far to classify images may have some ill-posed issues. The performance of the classification scheme depends on the range of captured images, the volume of features, types of characters, choice of features from extracted features, and type of classifiers used. This paper aims to propose a novel deep learning approach consisting of Convolution Neural Network (CNN), Recurrent Neural Network (RNN), and Long Short-Term Memory (LSTM) application to classify the fruit images. Classification accuracy depends on the extracted and selected optimal features. Deep learning applications CNN, RNN, and LSTM were collectively involved to classify the fruits. CNN is used to extract the image features. RNN is used to select the extracted optimal features and LSTM is used to classify the fruits based on extracted and selected images features by CNN and RNN. Empirical study shows the supremacy of proposed over existing Support Vector Machine (SVM), Feed-forward Neural Network (FFNN), and Adaptive Neuro-Fuzzy Inference System (ANFIS) competitive techniques for fruit images classification. The accuracy rate of the proposed approach is quite better than the SVM, FFNN, and ANFIS schemes. It has been concluded that the proposed technique outperforms existing schemes.

In the area of computer science, fruit classification is getting increasingly popular. The Indian economy has been largely dependent on agriculture up until now. The fruit classification system's accuracy is determined by the quality of the collected fruit images, the number of extracted features, the kinds of features, and the selection of optimum classification features from the retrieved features, as well as the type of classifier employed. Images in poor weather reduce the visibility of the obtained fruit images and conceal key features. Image classification is useful for categorizing fruit images and determining what kind of fruit is included in the image. Color image categorization methods, on the other hand, have poor visibility problems [

With the development of computer and machine vision, concerns are growing about the image-based recognition and classification technologies of fruits and vegetables. The deep learning approaches were applied to automatically diagnose the in-field wheat diseases [

To classify the fruit images after enhancement, CNN deep learning method is involved to extract optimal features. RNN is involved to label the optimal features and finally, LSTM is utilized to classify the fruits. From empirical results, it has been proved that deep learning applications classify fruits efficiently [

The suggested study's main contribution is the use of deep learning algorithms to categorize fruit images. Support Vector Machine (SVM), Feed-forward Neural Network (FFNN), CNN, RNN, and Adaptive Neuro-Fuzzy Inference System (ANFIS)classification methods are compared to the suggested methodology. In terms of accuracy, root means square, and coefficient of correlation analysis, the suggested method surpasses current alternatives. The obtained fruit images are enhanced using Type-II fuzzy. The best features are extracted using CNN, and the features are labeled using RNN. The fruit images are classified using LSTM based on optimum and labeled features by CNN and RNN, respectively.

The paper is organized as follows: Related work is discussed in Section 2. Methodology related to the proposed work is discussed in Section 3. Results and conclusion for the fruit images classification approach are discussed in Sections 4 and 5 respectively.

In this section, a comprehensive review of image classifications approaches is discussed. From an extensive literature survey, it has been observed that fruit recognition and classification is still a challenging job. Zaw Min et al. [

Nasiri et al. [

In [

In [

Prof. Lotfi A. Zadeh [

An improved image allocation function g is defined as:

Finally, an improved fruit image is obtained with the support of look-up table-oriented histogram equalization.

Refined gray level g(m) is calculated by νmin and νmax as:

For global and local image contrast improvement, in the proposed study the cosine values of global regions in fruit images are modified.

Fuzzy Entropy, c-partition, fuzzy threshold, and Fuzzy MF's are employed by many investigators [

Were 0 <= k; a <= A1, 0 = l and b <= B−1.

Ja, jb is calculated by:

Membership function (MI(g)) is involved to compute the presence of grey level and described as:

Probability of grey level g defined for normalization of histogram h˜I(g). h˜I(g) is defined in the proposed study as:

The fuzzy function Probability is described as:

It is defined as T = t1, t2, … tn, Q∊n, real e× n matrices set and e is an integer 2 ≤ e≤ n Fuzzy c-partition space for P is fixed to [

Shannon and fuzzy entropy are utilized to identify the c-partition fuzziness. maximize the fuzzy entropy and fuzzy c-partition [

We have explored the fuzzy sure entropy principle for fruit image enhancement according to Fuzzy c-partition. Fuzzy sure entropy is modified as [

During the fruit image enhancement procedure, is it mandatory to subscriptropriate grey level values.

We define the involutive fuzzy complements as follows:

Definition: Let

In this paper, for image I, the membership function μI(ij) is initialized as [

The most often used method for dealing with uncertainty in threshold values of collected images is fuzzy type-II. Gray-level values, which range from 0 to 255, are used here [

In G, d and b fuzzy membership functions are defined as:

Fuzzy sure entropy parameter

And,

Corresponding to the grey level i of the fruit image (I) as [

M (LC)max defines the maximum value of the partition Mα (LC) concerning the range (0, 1) [

M(dark)max and M (bright)max are defined as maximum and minimum values in the proposed work as:

Then, we can assign 𝜖 value as shown below:

The exhausting search procedure is employed in this research work to obtain the optimal values of Type-II fuzzy logic.

Steps used during implementations are as follows:

Step 1: input the image, set L = 256, normalize gray level and initialize. Hmax αopt,

Step 2: Compute Histogram and obtain m according to

Step 3: Compute the membership function μIij according to

Step 4: Compute the probability of the occurrence of the gray level and normalize the histogram according to

Step 5: Initialize

Step 6: Exhausted search approach is used to obtain the pair of 2.3368 and

For given α, according to the

Compute the probabilities of the fuzzy events of d and b by as:

Compute the sure entropy of this partition according as:

If current computed M(d) is greater than M(d), replace M(d) with current computed. In the same way, when the current computed is greater than M(b)max, replace M(b) with the current computed. Similarly, if current Computed H. At the same time, replace α(opt) and

Step 7: Modify

Step 8: Repeat steps 6 and to obtain the ultimate αopt and

Step 9: Obtain the involution memberships (^μ(ij)) according to

Step 10: Evaluate enhanced fruit image as per the equation

During the image enhancement process, an exhaustive research strategy is involved to obtain the effective type-II fuzzy values by setting L from 0 to 255 normal gray level values. Then compute the histogram values to set membership levels for separating the image regions into bright and dark levels. Compute the dark and bright values using fuzzy events. For this involution, a membership strategy is utilized. If satisfactory results are obtained then the visual analysis is done on the images as per the following figure.

In

In the proposed approach, the major objective is to extract the optimal features, label the optimal features and finally classify the fruits based on optimal features by deep learning applications. In this research work, CNN, RNN, and LSTM deep learning models are combined to classify the fruits. During feature extraction using CNN, segment acquired features into different strategies, we divide the features into coarse and fine categories. CNN generator is involved to extract fine and coarse label categories. The two layers can be structured either in a sequential or parallel way. The general architect of CNN is as follows:

IMAGE: Input Image

CONV: Convolutional Layer

RELU: Rectified Linear Unit

POOL: Pooling Layer

NCR: up to 5

NCRP is Large

FC: Contains neurons that connect to the entire input volume, as in ordinary Neural Networks.

Let I will be input image, then convolution operation is defined as:

RELU is a linear activation function that is similar to a filter that allows positive inputs to pass through unchanged while clamping everything else to zero. It is defined as

Let Sp (x, y) be pth sampling image after applying pooling operation on activation map Cp. When filter size m×n = 2 × 2, no padding (P = 0) and stride (s = 2), then AVG pooling is defined as follows

This work is done by the approach used by Guo et al. to label the image features, we employed the same concept to label the fruit image features as shown in

During the categorization policy, firstly we separate training and testing data into 75% and 25%, Then 25% into training, and 75% into testing. Features are extracted but not categorized into different labels efficiently. Therefore, we employed RNN to label the features. The Convolution Neural Network-based developer is unable to exploit the interrelationship between two supervisory signals individually. In case, if the hierarchy has adaptive size, a CNN-based developer is not able to categorize the hierarchical labels. To overcome the limitations of CNN, the RNN-LSTM integrator calculates hierarchical classification by altering the last layer of a CNN with RNN.

RNN is an artificial neural network in which edges between units form a directed cycle (see

During the Proposed classification scheme, the major objective of the study is to classify the fruit images quite impressively. We integrate the RNN and CNN to overcome the limitations of CNN to label and classify the fruits. The flow is depicted in

As shown in

During the inference phase, the CNN-RNN integrator predicts the probability labeling for the current timestamp when the ground truth coarse-fine strategy is not available. To overcome the CNN-RNN limitations, LSTM is used to label the optimal and effective features during image classification.

In the proposed deep learning model CNN, RNN and LSTM were used to classify the fruits. In the proposed model, extracted features using the convolution layer of CNN are labeled are coarse and fine with the help of RNN. Linear, convolution and fully connected layers of CNN extract the best-matched feature based on intensity, texture, and shape. RNN is involved to label these features distinctly. Finally soft-max-layer are used to classify the fruits by the filtering process used by LSTM. Optimal features are utilized in the proposed approach to classifying the fruits.

As shown in

To evaluate the classification performance of the proposed scheme, MATLAB software is used for simulation work on the Intel Core i5 processor with 8−GB RAM. Deep learning applications CNN, RNN, and LSTM are integrated to develop a novel fruit image classification model. SVM, FFNN, and ANFIS classification results are compared with the proposed scheme.

In the proposed study accuracy, root means square, and coefficient of correlation analysis measures are employed to evaluate the performance of the novel fruit image classification approach.

Few 1's determine the quality of fruit images during classification. In the same way, 0's described the failure rate. Classification accuracy is defined as:

Here Tp+Tn+Fp+Fn represent true positive, true negative, false positive, and false negative, correspondingly.

F-measure is used to evaluate the accuracy of the proposed classification scheme as shown in

Here 1 value for F measure means best classifier result and 0 means worst classification results. Precision is the fraction of relevant values among the obtained values and recall is the fraction of relevant values that have been obtained from the total obtained classification values. Precise is positive value and recall is called sensitivity during the classification process.

F-measure is defined as:

Precision and recall are defined as:

Deep Learning models are proposed for fruit image classification. Each image of size 512 × 512 is utilized with the help of MATLAB software for classification purposes. The preliminary step of the proposed study is to enhance the fruit images. Type-II fuzzy is implemented in the proposed scheme. The results of the proposed method are compared with existing competitive methods. Edges are preserved, low pixel saturation during fruit image enhancement by Type-II Fuzzy. The proposed approach successfully solves the problem of classification issues, that occur during the state of art approaches.

Three objectives were achieved, CNN is used to meet the first objective of feature extraction. RNN is involved to meet the labeling of optimal features objective. LSTM is finally used to meet the image classification objectives. The results of this study are compared with the existing classification approaches. The results of SVM are sufficient to meet the standards but fail to provide accurate outcomes during texture features classifications, which are satisfactory by the proposed approach. Results of FFNN are also quite promising but fail in comparison to true positive and true negative analysis, which are better during the proposed classifier. ANFIS has good accuracy rates but fails to match the results of the proposed scheme during f measure quantitative analysis. combination of RNN-CNN has a better classification rate but is inadequate during classification after feature extraction and labeling procedure.

Due to their similar form but the varied texture and intensity characteristics, fruit categorization remains a difficult job. We developed a deep learning method in this paper to classify the fruit images. To create discriminative features, CNN was employed, while RNN was used to generate sequential labels. Based on optimum and labeled features by CNN and RNN, LSTM was used to categorize the fruit images extremely well. On fruit images, extensive experiments were conducted utilizing the suggested method as well as current classification algorithms such as SVM, FFNN, and ANFIS. In terms of accuracy, root means square, and coefficient of correlation analysis, this classification method surpasses current image classification algorithms. This work has been done on the acquired fruit images, which may have limitations in the form of contrast improvement and edge detection. In the future, work be elaborated to overcome these issues.

We deeply acknowledge Taif University for supporting this study through Taif University Researchers Supporting Project Number (TURSP-2020/150), Taif University, Taif, Saudi Arabia.