Multispectral imaging is a new and emerging area with diverse application domains such as agriculture, health and sciences, geo monitoring, and environmental changes. Recent research trends indicate that the synergy of multispectral images with other fields has generally proven effective in urban and regional planning [27]. This section discusses some of the recent research around multispectral images. One crucial problem is soil analysis, particularly soil salinization, where soil degradation is a prime issue. Hu et al. [6] attempted to solve this problem with an unmanned aerial vehicle (UAV) borne multispectral imager. Remote sensing is very useful for collecting data for agriculture applications. Zheng et al. [3] used different modeling algorithms, including parametric and non-parametric algorithms, for estimating leaf nitrogen content (LNC) in winter wheat. This was done with the help of multispectral images using UAVs. With the help of classification algorithms, including linear regression (LR), random forests (RF), and support vector machine (SVM) alongside UAV multispectral images, the prediction of canopy nitrogen weight in Corn was discussed by Lee et al. [28]. They reported that RF and SVM are better than LR for the problem. In weed monitoring, the main issue is determining weed amount and location. For this problem, three methods are proposed by Osorio et al. [29] using deep learning and multispectral images with the comparison of visual estimations from the experts. The proposed methods are SVM, You Only Look Once Version 3 (YOLOV3), and Region-based convolutional neural network (R-CNN). The F1 scores were 88%, 94%, and 94%, respectively. Another area where multispectral images are used is related to oil pollution, a severe environmental issue. Ozigis et al. [30] worked on detecting oil pollution and its impact on biological elements. They used multispectral and multi-frequency images, fuzzy logic, and RF methods. An overall accuracy of 75% for dense areas of vegetation was obtained. Furthermore, the Cropland and grassland areas had an accuracy of 59.4% and 65% respectively. Lima et al. [31] developed a system for early-stage supervision of the status of organic fertilization on tomato plants. The proposed system was developed using a multispectral camera with five lenses including green, red, red edge, near-infrared, and RGB alongside with computation image processing mechanism. One of the successful mechanisms of monitoring the characteristics of vegetation by exploiting the Normalized Difference Vegetation Index (NDVI). With the help of multispectral images, Cao et al. [32] identified that using wide-dynamic-range vegetation index (WDRVI) derived from UAV multispectral image monitoring of sugar beet growth indicator is very useful as sugar beet has very large ground biomass.

Monitoring several crops (cultivated areas) is very challenging since identifying different tops is very costly. Sicre et al. [33] attempted to find different ways to use satellite images (both optical and radar) to classify the land covers. Their suggested method was dependent on the wavelength and penetration depth of the signal providing the images. Lei Ma et al. [23] reviewed the land cover object-based classification in different research publications. The findings indicate that remote sensing-based imagery was the dataset most frequently used.

UAV can monitor leaf area index (LAI). Qi et al. [34] proposed new ways to find the high-precision LAI prediction system with the help of multispectral images. Multispectral images are also useful in classifying potato defects [35–37]. In addition, detecting bruises on apples is also a very challenging classification task when using multispectral images. Additionally, classifying bruises is very important when discussing the automatic apple classification system. With the help of principal component analysis (PCA), hyperspectral imaging and multispectral imaging systems for selective bands were developed by Huang et al. [16], which achieved 90% accuracy. Powdery mildew (PM) is a fungal disease that can cause powdery growth on the surface of plants, leaves, and fruits. Early detection of powdery mildew is very crucial. Chandel et al. [17] studied using multispectral images for PM detection and mapping, achieving 77% accuracy. Bhargava et al. proposed a binary classifier [18] for sorting mono and bi-color apples. They achieved different accuracy on different datasets using multispectral imaging. Feng et al. showed the results in [19] that indicate that fruit recognition using multispectral images can be very effective. The current state of the art and new applications of remote sensing in agriculture were discussed by Khanal et al. [38]. Applications related to plant disease detection, yield estimation, water stress monitoring, etc., were discussed.

The hyperspectral image consists of several bands that have unique classifications. Each segment of the pixel can be treated as a unique label that identifies a specific target class. After image acquisition, a hypercube is formed, which contains information regarding both spectral and spatial data. As the hypercube has three dimensions, the image’s resolution helps in two dimensions, while several bands assist in-depth, which is the 3^rd dimension. Ranjan et al. [39] used principal component analysis (PCA) for feature extraction for hyperspectral image classification, and with the help of K-means clustering, they formed different clusters. After that, all clusters were trained using SVM. The accuracy of this approach was recorded to be 90%, 77%, and 88% for three different types of classes. They concluded that accuracy was better when data was initially divided into different groups before classification. Research on crop height for maize biomass by Zhang [40] used UAV hyperspectral imagery. Prediction of Aboveground biomass (ABG) using stepwise regression and XGBoost regression model with the highest accuracy of R-Squared (R²) of 0.81 and Root Mean Square Error (RMSE) of 0.27 was obtained, showing that hyperspectral imagery can play a vital role in the estimation of maize aboveground biomass with better accuracy. Rubio-Delgado et al. [41] proved that hyperspectral images effectively estimate the determination coefficient between spectral data and leaf nitrogen content. They used a full spectrum range for vegetation indices (VI) and Partial Least Square Regression (PSLR) models and achieved a large R² value. Results can be improved by using temporal information on leaf nitrogen content. Messina et al. provided a comprehensive research summary of remote sensing’s contribution to the growth of olive farm management in the last two decades [42].

Yao et al. provided a complete review of remote sensing applications using UAVs [43]. Selected applications were from precision agriculture and vegetation, urban environment and management, and disaster hazards and rescue. The advantages and selected areas were also highlighted in the conclusion. Gautam et al. studied the water status of horticultural crops using different remote sensing applications [44]. Remote measurements of water status, such as soil moisture, canopy 3D structure, etc., remote sensing is the best tool for planning and managing different applications for larger areas. Yang et al. [45] found that airborne hyperspectral imagery is used in mapping cotton yield. With stepwise regression’s help, 61% and 69% variability for two yield fields were observed. They concluded that due to narrow bands, hyperspectral imagery is much better than multispectral imagery, which uses broad bands. Zhang et al. [46] used hyperspectral images to observe water quality and proposed that a deep learning model could monitor water quality concentration. Mean absolute percentage error (MAPE) was 8.78% and 12.36%, and R² was 0.81 to 0.93. They suggested using cloud stations for a reduction in time and costs. Coverage using UAVs can also be much broader so that more data can be collected quickly. It is very important to use an optical sensor to detect crop stress, which can specifically find spectral wavelength affected by stress factors. It is also essential to consider which spectral resolution is required to prevent it. Mewes et al. [47] discovered that the problem is reducing the spectral resolution and spectrum angle mapper. The proposed method for this problem was SVM. The results show that only a few hyperspectral bands are enough to detect fungal infection in wheat. In contrast, the authors also suggested using the derived result for feature selection in agriculture. Cen et al. [48] examined the internal defects of pickling cucumbers using hyperspectral images. The performance measure was minimum redundancy–maximum relevance (MRMR) with 95% and 94% accuracy on two different conveyor belt speeds. They primarily worked on identifying several optimal wavebands for PCA for fast and efficient algorithms and light sources. Further study can be done in discrimination between slightly and severely defective labels.

Machine Learning is a branch of Artificial Intelligence that deals with problems related to prediction and classification. The phrase ‘learning’ relates to the amount of data given to an algorithm, and more data would mean more learning. For any task (T), if the algorithm’s performance (P) increases with respect to experience (E), then the algorithm is said to be a machine learning algorithm. Machine Learning is fundamentally classified into three types: (1) Supervised Learning, (2) Unsupervised Learning, and (3) Reinforcement Learning. In supervised learning, the model is trained with labelled data and predicts the labels from new data. In supervised learning, learning is done from a dataset, making predictions iteratively, and with each iteration, the parameters of the algorithms are adjusted accordingly. Supervised learning includes LR, Logistic Regression, Nearest Neighbors, Decision Tree, RF, SVM, etc. In unsupervised learning, the model is trained with unlabeled data and groups the data based on feature similarity. Unsupervised learning tries to find the patterns in the dataset without any pre-existing target labels. It helps in finding insights into data. Sometimes, unsupervised learning performs complex tasks better than supervised learning. Clustering, association, and dimensional reduction are the main approaches that use unsupervised learning [49]. In reinforcement learning, the algorithm performs actions that maximize performance in particular conditions. Fig. 1 shows the three types of machine learning and their differences.

Fig. 2 explains the overall flow of a machine-learning algorithm. In the first phase, the training phase, the data or training examples go to the feature extractor for feature extraction.

After getting features from training examples, the model for machine learning is trained; once the training model is ready for the prediction, the system takes a new test example and classifies/predicts the output.

Deep learning is a subset of machine learning inspired by the human brain, consisting of billions of neurons. Deep learning uses Artificial Neural Network algorithms. The word deep refers to the number of layers in a network. If the number of layers is unknown, the system automatically adjusts the weights using an optimization function. The layer consists of several neurons. Since it mimics the human brain, deep learning also requires much data for better accuracy [50]. Although deep learning evolved from machine learning, the two have some differences. The main difference is that deep learning requires large amounts of data, and features are not extracted before applying the deep learning model. Deep learning can automatically extract the features that further process the data for classification. The concept of deep learning is illustrated in Fig. 3 for further clarification.

The databases used in the search were Springer, Science Direct, IEEE Xplore, and ACM. The timeline for the manuscripts was between 2017 and 2022 because of the most recent publication in multispectral and hyperspectral images. We have used four search strings for the survey.

Search String 1. (Multispectral Images) AND (Agriculture) AND (Deep Learning)

Search String 2. (Multispectral Images) AND (Agriculture) AND (Machine Learning)

Search String 3. (Hyperspectral Images) AND (Agriculture) AND (Deep Learning)

Search String 4. (Hyperspectral Images) AND (Agriculture) AND (Machine Learning)

The exclusion and inclusion criteria were straightforward. We selected the research from 2017 onwards and did not include any articles before 2017 in our primary studies. The reason for selecting the last five years is to provide the recent developments in hyperspectral and multispectral images. All the selected articles are peer-reviewed, and all articles are searched using the mentioned string. All articles were written in English, and articles in other languages that appeared during the search were not considered. Fig. 4 depicts the inclusion and exclusion criteria.

We search the articles from the following databases: SPRINGER, SCIENCEDIRECT, IEEE XPLORE, and ACM. Surprisingly, we did not find papers in IEEE Xplore despite trying the search with each of the four strings. Fig. 5 shows the well-known publishers from where articles are selected.

Fig. 6 shows the initial frequency comparison of all the research articles from selective publishers. We found that Google scholars have comparatively large numbers of papers compared to other databases.

The search strategy involved an automated search, as depicted in Fig. 7. The automated search proved instrumental in locating and providing us with 30,100 research articles for the study, out of which 27,900 papers were obtained from Google Scholar. Consequently, we excluded the articles sourced from Google Scholar to avoid redundancy.

Following the exclusion process, we extracted 2,200 articles for this study. Subsequently, we applied inclusion and exclusion criteria based on titles, abstracts, year of publication, and keywords, resulting in 185 articles remaining after the screening process. In the next phase, we applied the selection criteria to the full-text studies, and we were left with 44 primary studies. Our selection criteria aim to include multispectral or hyperspectral images or images with deep learning or machine learning in agriculture.

Fig. 8 shows the distribution of selected/primary studies for selected years after applying all the mentioned criteria and selection mechanisms.

RQ1. In what diverse research areas within agriculture have Multispectral or Hyperspectral images found applications?

The selection of primary studies focused on hyperspectral and multispectral applications in agriculture. To address our first research question (RQ1), we categorized all relevant agricultural works into four main classes: grains, fruits, vegetables, and plant issues. The distribution of these classes is illustrated in Fig. 10 using a Pie chart, revealing that 43% of the selected studies were related to fruits.

These classes encompass a range of diverse subclasses. Under the grains class, we included research studies on rice and various types of seeds. In the plant class, we focused on issues such as citrus greening detection, coffee ripeness, and yield monitoring, identification of apple and rose flowers, detection of different leaf diseases and deficiencies, crop water detection, mapping of agriculture, and vine disease detection. The summarized structure is presented in Table 3. The vegetable class comprised primary studies on defect detection and grading of potatoes and tomatoes and yield forecasting of carrots and cucumbers. However, our primary emphasis was on fruits, which involved the detection and grading of apples, estimation of mangoes’ maturity, identification of strawberry diseases and ripeness, early detection of decayed oranges, assessment of pineapple quality, and detection of pears crown bruises, along with bruise detection of various fruits and recognition of on-branch fruits.

RQ2.Which Machine Learning and Deep Learning Techniques have been employed to tackle agricultural problems using Multispectral and Hyperspectral images? To address RQ2, we thoroughly examined all the selected articles, which involved 17 distinct algorithms. Notably, the most frequently employed techniques were from deep learning, utilized in 27 instances, followed by SVM, used 13 times.

Fig. 11 illustrates all the machine learning and deep learning techniques employed with multispectral and hyperspectral images. Certain techniques were applied to address specific agricultural problems, and researchers recommended their continued use for similar challenges due to algorithmic and data limitations. We have categorized these techniques into five groups: Deep Learning techniques, SVM, K Nearest Neighbors, and Logistic Regression, and the remaining methods were grouped under “Other.” Fig. 12 illustrates the occurrence frequency of each technique in our selected studies. Additionally, Table 4 provides a study-wise classification of Machine and Deep Learning algorithms.

Deep Learning, particularly Convolutional Neural Networks, has shown remarkable performance in various spectral classification problems [53,55,61,62]. Studies suggested that Convolutional Neural Networks are very well-suited for spectral image classification because they can automatically learn discriminative features directly from the input data without the need for explicit feature engineering [74–76,83,88–92]. They can capture complex patterns and relationships in spectral images, leading to accurate classification results. Deep Learning models can handle large amounts of data and effectively learn from diverse spectral bands, making them suitable for hyperspectral and multispectral image analysis [52,58].

Conversely, in chosen studies, SVMs are extensively employed owing to their capacity to manage high-dimensional feature spaces and effectively handle nonlinear data [57]. SVMs seek to find the optimal hyperplane that maximally separates different classes in the feature space [69,70]. They can handle the curse of dimensionality by utilizing the kernel trick, which implicitly maps the input data to a higher-dimensional feature space, allowing for better separation of classes. SVMs have a strong theoretical foundation, work well with small to medium-sized datasets, and have been successfully applied to spectral image classification tasks [93]. Deep Learning was the most preferred choice of researchers when working with multispectral and hyperspectral images in the agriculture domain.

Regarding different types of tasks, classification is the most widespread problem while working with spectral images. We find that there are several reasons for the popularity. One of the primary objectives in agricultural applications is to identify and classify different plants, diseases, or crops. Classification provides specific interpretable results so agriculture experts can easily understand and treat them accordingly. While regression techniques can be used for crop yield estimation, classification can be a precursor to yield prediction. Identification of weeds and their management are essential parts of agriculture. Weed detection can reduce its competition with crops. Thus, different classification models on spectral images can be trained to assess the health of crops. The health of crops can include stress signs, different deficiencies, or any disease that can affect crop growth. We have already stated that SVMs and deep learning models have proven more effective while working in spectral image analysis in the agriculture sector.

RQ3: What are the prevalent datasets extensively utilized for spectral classification purposes?

Since multispectral and hyperspectral imagery is an emerging field, many datasets can be found in the literature. Here are some of the most extensively utilized datasets. The Xuzhou dataset was collected in 2014 at the Xuzhou peri-urban site. This dataset consists of 500 × 260 pixels with 9 available classes. The Salinas dataset is a widely used benchmark dataset for hyperspectral image analysis. It represents an agricultural area in Salinas Valley, California, and contains hyperspectral imagery acquired by an airborne sensor. The dataset consists of 512 × 217 pixels with 224 spectral bands of 16 classes. The Pavia Centre imagery dataset is collected at the University of Pavia, Italy, using a reflective optics system imaging spectrometer (ROSIS). Nine distinct land cover classes are accessible, with a spatial resolution of 1096 × 1096. The Pavia University dataset was also captured in the University of Pavia in 2001, having a spatial resolution of 610 × 640 and a total of 9 classes of urban environmental construction. Houston dataset consists of 380–1050 nm spectral wavelength with 48 bands. This data was provided by the IEEE GRSS data fusion contest and captured by the University of National Center of Airborne Laser mapping on the premises of the University of Houston. The Kennedy Space Centre dataset was collected in 1996 at Kennedy Space Center, Florida. In Table 5, we have compiled a summary of the dataset, including the number of available classes, spectral size, and spatial size.

This data is collected with the help of JPL’s Airborne imaging spectrometer, which has 13 classes and a spatial resolution of 512 × 614. The Hekla dataset has a spatial resolution of 2084 × 614 and a spectral size of 157. There are t22 classes available in this dataset. The Trento dataset was collected with the help of the AISA eagle sensor. The total number of classes is six, and the spatial resolution is 600 × 166. Botswana data was collected using a Hyperion sensor over the Okavango Delta in Botswana, South Africa, in 2001. This consists of 14 land cover labels with a spatial resolution of 1476 × 256. Indian Pines dataset was obtained by AVIRIS sensor in northwest Indiana, United States of America. This dataset has 16 labels of land covers and 220 spectral bands in the range of 0.2 to 2.4 µm, having a spatial resolution of 145 × 145 pixels [95–97].

Indian Pines, Salinas, and Houston datasets are commonly utilized in various studies. Additionally, the Pavia University dataset and the Pavia Centre and Kennedy Centre datasets hold a prominent position among the most widely employed datasets in spectral imagery [98–100].

RQ4: What are the challenges, limitations, and potential future developments concerning Hyperspectral and Multispectral images in conjunction with Deep Learning and Machine Learning Models?

Some primary studies suggest employing the same images with alternative classification algorithms to compare results. Within our selected primary studies, various issues were addressed using limited data, prompting the recommendation for larger datasets to achieve improved accuracy. However, due to environmental and camera bandwidth constraints, certain limitations persist. Interestingly, some research asserts that multispectral images are suitable for classifying various types of fruits and vegetables, while others advocate their use for flowers. To provide a comprehensive overview, we have summarized these findings, including limitations and challenges, in Table 6. These findings pave the way for future research in the expansive domain of multispectral and hyperspectral imagery.

Some models display consistent errors across all features, emphasizing the necessity for improving feature selection techniques. When conducting a comparative analysis of classification algorithms for a particular problem, it becomes evident that the learning process tends to be sluggish. However, this can be accelerated by isolating the feature selection process and applying the method to other fruits. Enriching the dataset with additional disease samples and testing it on various deep-learning architectures can help overcome the challenge of limited training samples.

The small size of datasets is indeed a standard limitation when working with spectral images in the context of agricultural fruits [68,77,87]. Obtaining and annotating large-scale datasets specific to agricultural fruits can be challenging due to various factors [73]. Here are some of the few reasons why the datasets related to spectral images were often small.

These factors can impact the quality and representativeness of the datasets. Here is a closer look at these limitations.

Data scarcity can pose significant challenges, particularly in machine learning. These are some major aspects and issues that arise while facing data scarcity.

KNN was used for the classification of the rose plant in [59] and citrus greening detection in [69] and in [70] for the regression assessment of ready-to-eat pineapple quality. The classification accuracy was 65% and 81%, and the R-square value was 0.42, respectively. The challenge is to test different algorithms on the same type of problems at different stages of growth of the particular plant.

Citrus greening detection [66–69], pineapple quality [70], strawberry ripeness [77], tomato grading [65], and high-quality watermelon seeds classification were done by SVM with accuracy ranging from 79% to 98%. Testing new algorithms, refining feature selection, and more diverse sample images at various ripeness stages can enhance results. It is noted that accuracy may decrease as the number of target classes increases. Nevertheless, SVM holds potential for broader applications, including analyzing different seeds and crops, making it a promising avenue for future research and development.

Deep Learning and different types of neural networks were used for the detection of fruit diseases [83], vine disease detection [68], detection of green vegetation cover [84], detection of growth stages of rice [72], fruit recognition [73], apple defects [74], blossom detection in apple trees [75], tomato’s external defects [76] and strawberry disease classification [94]. The accuracy achieved was in the range of 65% to 99%. One of the main issues was the slow learning process, which can be enhanced by choosing the meta-heuristic techniques for feature selection. The other main challenge is adaptability. We can test the same model on the new dataset. We can enhance the adaptability by updating the dataset with new models on different agriculture problems.