The published literature on plant species recognition from leaf photos included several works utilizing deep learning techniques. Nguyen et al. [29] investigated GoogLeNet [30], CaffeNet [31], and AlexNet [32] deep learning models to detect different flower species. The researchers used the PlantCLEF2015 dataset of 967 flower species. The proposed method obtained 66.6% accuracy on GoogLeNet, 54.84% on CaffeNet, and 50.60% on AlexNet. Gogul et al. [33] also investigated using InceptionV3, OverFeat, and Xception deep learning methods to distinguish the multiple flower species. The researchers used the Flowers28 dataset of 28 different species of flowers. The OverFeat model achieved 85.71% accuracy, InceptionV3 got 92.41%, and Xception obtained 90.18% accuracy. Xiao et al. [34] investigated InceptionV3 and ResNet50 deep learning models to detect the 1000 plant species. This study utilized both the PlantCLEF and Oxford Flower datasets. For comparison, the InceptionV3 model was 69.5% accurate on the PlantCLEF dataset and 92.8% correct on the Oxford Flower dataset. PlantCLEF achieved 68%, and the Oxford Flower dataset obtained 92.4% accuracy with the ResNet50 deep learning model.

Research conducted by Pereira et al. [35] investigated the AlexNet model based on a convolutional neural network to classify the different varieties of grapes. The researchers used two datasets for the research. The first dataset was developed by merging the DRVG and DRGV2018 datasets and Flavia Leaf as the second dataset. The merger dataset DRVG obtained 77.30% accuracy and 89.75% on the Flavia Leaf dataset.

Mahmudul Hassan et al. [36] proposed a CNN to identify the different plant species. The researchers used Flavia, MK, and LeafSnap datasets. The MK consisted of MK-D1 and MK-D2. The proposed method achieved 99.15% accuracy on MK-D1 and 99.43% on the MK-D2 dataset. The Flavia dataset gained 99.75% accuracy, while the LeafSnap dataset obtained 89.17% accuracy. Erkan et al. [3] proposed an ODC model based on the artificial bee colony method to detect the different plant species and handwritten digit classification. The researchers used the Folio Leaf image dataset for plant species identification and the MNIST digit image dataset for handwritten digit classification. The bilateral CNN was studied by Pearline et al. [37] using machine learning classifiers. DenseNet-121, MobileNet, and Xception deep learning models extracted the features. Regarding accuracy, the suggested technique attained 98.2% on the Folio Leaf Image dataset and 99.21% on the MNIST dataset. Then different machine learning classifiers such as Support Vector Machine (SVM), Multi-Layer Perceptron (MLP), Behavioral Cloning (BC), Random Forest (RF) classifier, Classification and Regression Tree (CART), K-Nearest Neighbors (KNN), Naïve Bayes (NB), Multiple Linear Regression (MLR), and Latent Dirichlet Allocation (LDA) [38–45] were used. It was observed that the MLR classifier performed better compared to other classifiers. MLR attained 98.71% accuracy on Flavia, 96.38% on Folio, 99.14% on Swedish Leaf, and obtained 99.39% accuracy on a self-created dataset.

Leaf segmentation with a complex background was also a challenging and complicated task. Few researchers worked on different plant leaves segmentation, such as Wang et al. [46], which combines pre-segmentation and morphological operations. After leafstalk removal, seven Hu geometric moments and sixteen Zernike moments are retrieved as shape characteristics from segmented binary pictures. A self-collected dataset of 1200 images were used, with 20 classes including 60 leaf samples for each class with complex background. They segmented the willow, plum, London plane tree, China redbud, chestnut, laurel, rose bush, hazel, panicled goldrain tree, sweetgum, honeysuckle, donglas fir, maple, arrowwood, tulip tree, ginkgo, photinia, camphor tree, seating, and Chinese allspice and got 92.6% accuracy. Research conducted by Itakura et al. [47] in which automatic leaf segmentation and parameters of plant structure were retrieved using point cloud 3D images. The leaves were segmented automatically with 3D models combined with 2D and 3D point-cloud processing techniques. The researchers used small plants such as Japanese sacandra, umbellate, kangaroo vine, council tree, dwarf schefflera, hydrangea, and pothos. The proposed 3D model achieved 86.9% accuracy. Research conducted by Yang et al. [48] in which 15 species segmentation was performed using Mask RCNN deep learning model used VGG16 deep learning model. They used a dataset of fifteen species and two thousand and five hundred images. The species included gardenia jasminoides, callisia fragrans, psidium littorale, Osmanthus fragrans, bixa Orellana, ficus macrocarpa, calathea makoyana, rauvolfia verticillate, Ardisia quinquegona, baccaurea ramiflora, synesepalum dulcificum, Hydnocarpus anthelminthic, daphne odora, dracaena surculose, and mussaenda pubesens. The proposed method achieved 91.5% accuracy. Nikbakhsh et al. [49] developed a new unsupervised approach based on G-mutual information for a fusion of plant leaf image segmentation. They evaluated the proposed method by generating a new leaf image dataset with natural scenes having different orientations and positions. The researchers used k-means, Self-Organizing Map (SOM), c-means, and fuzzy leaf segmentation methods to fust the results.

The literature on plant leaf species identification and segmentation needs to be improved with difficulties. The first issue with the available research is that only healthy plant leaf species were utilized for categorization and segmentation. Diseased plant leaf species were not included in these investigations. As a result, it became necessary to design a method for distinguishing between healthy and diseased leaves of the same plant species. The second issue was that the guava, java plum, and potato leaf species could not be uniquely identified or segmented by any one study in the literature. Most studies focus on the leaves of houseplants or the division of very small plants. The lack of information for guava, java plum, and potato leaf species meant that researchers had to develop their methods for classifying and segmenting these plants. Datasets were urgently required to classify and segment potato, guava, and java plum leaves. These datasets will be the potential for further studies, including the identification of plant species and the segmentation of both healthy and damaged leaves. The last major issue in the literature was that the efficiency of existing approaches needed to be enhanced to achieve desired results. This research aimed to provide solutions to the issues mentioned above.

We created two distinct datasets using PLSSD for MobileNetV2-UNET training and Plant Species Dataset (PLSD) for PSD-SE-DLM training. A hybrid approach was trained and tested on two datasets during the present investigation. The datasets are described in depth in the following section:

Images from the PLSD and PLSSD were pre-processed to ensure more consistent classification results and improved feature extraction. Images were extracted using the python code from the plant video footage. The photos were then resized to 224 × 224 pixels using python scripts that clipped out the excess background.

Training, validation, and testing sets were each created from the full PLSD dataset. We trained a PDDCNN model with the training dataset and then used validation and test datasets to assess the model’s efficacy. The datasets were divided into a training set of 79.23%, a validation set of 10.56%, and a testing set of 10.21%. As can be seen in Table 3, the PLSD dataset contains 4500 images, 600 of which were utilized for training and validation and 580 for testing. The current research used training photos labeled guava, potato, and java plum while keeping the same 79.23% image ratios. Regarding PLSD, the remaining 20.77% of fresh photos were divided into a validation and testing group with ratios of 10.55% and 10.21%. The proposed Plant Species Detection using Stacking Ensemble Deep Learning Model (PSD-SE-DLM) was trained on a training set to classify and predict the class label of each training image.

We split the Plant Leaf Species Segmentation Dataset (PLSSD) into three parts: training, validation, and testing. The total number of images utilized in the PLSSD training set was 2,898; 966 were assigned to each of the three classes (guava, java plum, and potato). In addition, there were 726 images in the validation set, with 242 representing each class. However, there were 363 inference samples in the testing set, with 121 examples in each category. As can be seen in Table 2, the PLSSD contained 3987 samples across all classes.

The proposed models were trained and tested using a Google Colab Pro account with powerful Graphical Processing Units (GPUs) without configuration requirements. The transfer learning deep learning models were used for training. We conducted four experiments in which three base models were used then a stacking ensemble model was developed using the output of the base models. All experiments of proposed PSD-SE-DLM used Adam optimizer with a learning rate of 0.0001, and SparseCategoricalCrossentropy loss functions were used for compiling all the models. We used 16 batch size, early stopping, saved the best val_loss model, and 5 epochs to train all the models, as shown in Table 4. The proposed MobileNetV2-UNET model used 4 batch size, 50 epochs, early stopping, and saved the best val_loss model, as depicted in Table 5.

A classifier’s accuracy is determined by dividing the total number of correct classifications by the number of proper categories overall.

(1)Accuracy=(TP+TN)(TP+TN+FP+FN)

When evaluating a model’s performance, it is important to remember that classification accuracy is not always an accurate indicator. Because the model is not learning anything and all samples are expected to be of the highest quality, expecting a high accuracy rate makes no sense. When the distribution of classes is unbalanced, this is one possibility. Following on the heels of this definition, precision refers to the inconsistencies found when you repeatedly measure the same object with the same equipment. One of these measurements is precision, which is defined as:

(2)Precision=TP(TP+FP)

Another critical parameter is recall, the proportion of input samples from a class that the model correctly predicts. Calculating the recall is as simple as:

(3)Recall=TP(TP+FN)

Precision and recall can be measured by using a statistic known as the F1 score:

(4)F1Score=2×(Precision×Recall)(Precision+Recall)

Classifiers are plotted on a receiver operating characteristic curve (ROC) based on the work of their cutoff limit. An outstanding model’s cutoff threshold can be determined using the well-known ROC curve. It appears that the TPR against the FPR for various limit values.

Fig. 7a depicts the proposed model’s training and validation set loss and accuracy performance, as shown in the performance graph. A batch size of 04 was used, with 50 training epochs and employed an Adam optimizer. After 28 epochs, the training loss dropped from 10.30 percent to 5.91 percent. Validation losses begin at 6.45 percent and gradually drop to 5.95 percent at the end of the era. Fig. 7b shows the proposed MobileNetV2-UNET training and validation accuracy. During the last period, the training accuracy was 89.47 percent, while the validation accuracy was 96.53 percent and gradually climbed until it reached 96.74 percent at the end. Fig. 8 shows the results of the MobileNetV2-UNET segmentation model prediction on the test set. The proposed method obtained 96.38% accuracy on the test set, as shown in Table 6. The proposed segmentation method performed excellent segmentation of the complex and diseased leaves of guava, java plum, and potato. All metrics indicate that the MobileNetV2-UNET plant leaf segmentation technology performs exceptionally well.

The suggested MobileNetV2-UNET model accurately segmented healthy and diseased guava, potato, and java plum leaves (96.36%). As shown in Fig. 9, the suggested method successfully segmented damaged leaves from their outside boundaries but failed to appropriately segment the hole, fold over, and the damaged leaf from its interior. The suggested model performed exceptionally well when segmenting healthy and diseased leaves from potatoes, java plums, and potatoes.

We also examined the proposed Plant Species Detection performance utilizing the Stacking Ensemble Deep Learning Model (PSD-SE-DLM). As shown in Fig. 10, the training accuracy began at 94.97% after the first epoch and achieved its maximum (100%) following the final epoch of the algorithm. During the start to final epoch, PSD-SE-DLM training loss dropped from 39.18 to 0.52, and PSD-SE-DLM validation loss dropped from 14.0 to 2.40. At the end of the last epoch, the validation accuracy dropped from 99.32% to 99.16%.

Table 9 shows the PSD-SE-DLM model’s precision, recall, F1 score, and accuracy based on the test set’s unseen data. Table 9 shows that the PSD-SE-DLM model performed exceptionally well on the test set. The guava plant leaf species got 100% recall and 99% precision and recall. The java plum classes had 100% precision, recall, and f1 scores, while the potato class had 100% precision, 98% recall, and 99% f1 scores. The proposed model received 100% accuracy in guava and java plum classes, 98.5% on potato leaf species, and 99.48% overall accuracy in all categories. We found that the proposed PSD-SE-DLM model outperformed state-of-the-art methods in every category we tested.

As shown in Fig. 11, the performance of the PSD-SE-DLM model was also validated using the confusion matrix on the test set. All test photos of guavas and java plums were successfully identified, demonstrating the efficacy of the stacking ensemble model. In comparison, 197 out of 200 photos for potato leaves were appropriately identified, while three were incorrectly labeled as guava. The confusion matrix depicted that the PSD-SE-DLM model achieved 99.48% accuracy and 0.52% misclassification accuracy. The confusion matrix of the proposed model reflected the excellent classification accuracy of all the plant species classes.

As demonstrated in Fig. 12, the performance of the PSD-SE-DLM base model may also be evaluated using the ROC curve. By contrasting the true positive rate (TPR) with the false positive rate (FPR) at specific cutoffs, we may see the receiver operating characteristic (ROC). According to [60], if your classification problem has an area under the curve (AUC) of 80–90, you are doing well; if it has an AUC of 90–100, you are doing well. Classes like guava and java plum had about 100%, while the potato class got 99% area under the curve, demonstrating the superior performance of the PSD-SE-DLM model.

All the evaluation measures, such as accuracy, precision, recall, F1 score, and roc curve, depicted that the proposed PSD-SE-DLM model had performed exceptionally well in classifying test data.

No study has been found in the literature that worked on guava or potato or java plum, or a combination of three plant species. Therefore, a comparison of this research is not possible. We compare our model with the other plant leaf species. The study presented in [17] proposed an ensemble model which can detect multiple plant leaf species and got 92.60% accuracy, as illustrated in Table 10. Szededy et al. [30] got 92.8% accuracy on InceptionV3, and the ResNet50 model obtained 92.4% accuracy. Gogul and Kumar [33] studied CART, KNN, NB, MLR, and LDA classifiers and obtained 98.71% accuracy using the MLR classifier. The suggested ensemble model achieved an accuracy of 99.48% for identifying the leaves of guava, potato, and java plum plants using the stack ensemble technique. The proposed ensemble model used the stack ensemble technique for the guava, potato, and java plum leaf species and achieved 99.48% accuracy, as shown in Table 10. The results showed that the proposed PSD-SE-DLM model achieved higher accuracy than the existing studies.

The proposed PSD-SE-DLM model was put to the test on the PLSD dataset by combining it with the MobileNetV2 [51], ResNet50 [57], GoogLeNet [30], and AlexNet [32] models using transfer learning. Thus, there was necessary to be consistent between studies concerning the environment and the methods used to enhance the data. The accuracy of modern deep learning methods is displayed in Table 11. As shown in Table 11, the accuracy of the MobileNetV2 model was 95.52%, ResNet50 was 96.10%, GoogLeNet was 92.35%, and AlexNet was 95.87%, and that of the suggested PSD-SE-DLM model was 99.48%. As shown in Table 11, the findings demonstrated that the proposed PDDCNN model achieved the best accuracy (99.75%) compared to other state-of-the-art models.

Plant leaf segmentation is a difficult task. Despite progress, this area has several limits. This study limited to images of plant leaves of guava, java plum, and potatoes. Plant canopy leaves overlap, making them hard to distinguish. Occlusion can misclassify or partially segment leaves, causing segmentation errors. Leaf forms, textures, and curves vary. The proposed method struggles with leaves with irregular edges, complicated patterns, or fine structures. Complex leaf structures may challenge the proposed. Segmentation depends on the lighting. Uneven lighting, shadows, and reflections can make leaves hard to identify from the background. The proposed method may struggle with illumination differences between photos or locales. Some leaves have colors or textures that match their surroundings. Due to this resemblance, the proposed MobileNetV2-UNET method may misclassify or merge leaves with the background, resulting in mistakes. Sensor noise, picture acquisition circumstances, and image compression can cause distortions and flaws in plant leaf photographs. Noise sources might skew the proposed segmentation findings. Accurate segmentation models require lots of labeled training data. However, collecting high-quality annotated data for plant leaf segmentation takes much work. More diversified and annotated datasets may hamper segmentation algorithm development and generalization. Leaf forms, sizes, hues, and textures vary per species. Due to leaf features, building a universal segmentation model that works across plant species takes much work. High-resolution photos and big datasets demand computationally intensive segmentation techniques. It can hinder these algorithms’ real-time or efficient deployment, especially in resource-constrained applications.

Though promising, the PSD-SE-DLM approach has limitations. The proposed leaf species identification system may need more training data. Same-species leaves vary in form, size, color, texture, and other characteristics. PSD-SE-DLM may misclassify or under-classify intra-class variability if it cannot capture it. This variability demands a robust model architecture and a diversified training dataset with many leaf variants within each species. However, various plants may have similar leaves. The approach may need help differentiating closely related species with comparable leaf features. Higher-level contextual information or multi-modal data sources may be needed to distinguish comparable species.