Data acquisition was conducted on 07 January 2024 in Thu Thua, Long An Province, Vietnam, as shown in Fig. 1.

Seventeen days after sowing, both the seed rate and number of tillers were carefully measured. To capture this data, a Phantom Pro 4 drone equipped with a DJI-FC6310S camera (capturing images at 4864 × 3648 pixels) was employed. Although video recording is possible, autonomous flight paths are preferred to ensure consistent image quality across varying environmental conditions. The FieldAgent software was used to plan and execute these autonomous flights. The ground sample distance (GSD) was calculated manually for each altitude, and is detailed in Table 1.

Image data was acquired at 10:15 AM on the designated observation day. During data collection, strong winds were present, blowing in the opposite direction of the drone’s flight path. Favorable lighting conditions ensured optimal image capture. To account for variations in environmental backgrounds, data collection sites were randomly selected within the field. These conditions included challenging scenarios such as backlighting, reflections on water surfaces, dark backgrounds, and the presence of objects that could be difficult to identify.

The experimental field was spanned approximately 2 ha, with a seeding rate of 40 kg/ha. A wider spacing of 10 × 15 cm² was implemented, with approximately three seedlings planted per hill, and fertilizer was applied at a depth of 7 cm. During the vegetative stage, young rice plants primarily develop leaves and stems. The timing of tillering can vary depending on rice variety, nutrient availability, and prevailing weather conditions. In this study, a sowing machine was used to precisely plant IR4625 rice seeds one day after germination.

To complement the advanced imaging techniques used in this study, traditional methods were also employed to assess rice seed density 14 days after sowing (Fig. 2) [33]. Four distinct polyvinyl chloride (PVC) squares, each measuring 50 cm × 50 cm, were strategically placed at the four corners of the paddy field. These squares, aligned with the tractor’s plowing path as shown in Fig. 3, serve as visual landmarks. The images can be accurately divided into smaller manageable sections by identifying and locating these squares within the aerial images. It is important to note that the seed sowing machine used in this study created evenly spaced holes, with a predetermined distance of 10–12 cm between each hole. Therefore, knowing the precise coordinates of these PVC landmarks can provide valuable clues for identifying the locations of individual seed holes and clusters within images.

The proposed system is illustrated in Fig. 4. Aerial images were converted into binary images, and a Hough line transformation was applied to detect the four edges of each landmark. Finally, the center point of each landmark was calculated.

To detect seedling positions, a dot map was generated, and labels were assigned to each cluster center. In this study, the objects were categorized into three classes: single rice seedlings, clusters of rice seedlings, and undefined objects. For each dot representing a potential seedling or cluster, a square window of size 35 × 35 pixels was defined, with the dot coordinates serving as the window’s center. This window was used to extract a feature vector from the image.

To accomplish this, the study primarily employed spatial descriptors, specifically Gabor filters, and histograms of color ratios. Gabor filters are steerable filters that are sensitive to both orientation and frequency. They act as linear filters, analyzing the presence of specific frequency components within the image in different directions around the point of interest. Research in computer vision suggests that the frequency and orientation responses of Gabor filters closely resemble those of the human visual system. This makes them particularly well-suited for representing and distinguishing textures. Mathematically, Gabor filters are recognized for their optimal joint resolution in both spatial and frequency domains. In the discrete domain, two-dimensional Gabor filters are defined as Eqs. (1) and (2): (1)Gc[i,j]=Be(−(i2+j2)2σ2)cos⁡(2πf(icos⁡θ+jsin⁡θ)) (2)Gs[i,j]=Ce(−(i2+j2)2σ2)cos⁡(2πf(icos⁡θ+jsin⁡θ))

In the Gabor filter equation, B and C represent the normalization factors determined during the process. Parameter f defines the specific frequency being analyzed within the texture. By adjusting the values of θ (orientation) and σ (scale), The Gabor filter can be tuned to detect textures with different orientations and spatial extents by adjusting the θ (orientation) and σ (scale) values. For this study, a Gabor filter bank was designed with the following parameters: four frequency levels, six orientations, maximum frequency of 0.327, and half-octave frequency ratio. The design of this filter bank is based on the considerations outlined in Ref. [34]. For each color channel in the image, the mean and standard deviation of the absolute values within the transformed images were extracted as textural features. This resulted in 144 monochrome features (4 frequencies × 6 orientations × 2 [mean and standard deviation] × 2 channels). Throughout this study, the Gabor filter configuration remains consistent. In addition, the color histograms of the RGB channels were utilized as features. For each channel, the image was initially quantized into 64 distinct colors. Subsequently, three color-ratio histograms were calculated for each pixel, and the color value was compared to the values of its neighboring pixels. These color ratios are invariant to changes in the surface orientation, viewpoint, and illumination direction, making them robust features for image analysis.

To enhance the precision of locating the center point during the window-sliding process, GoogLeNet was employed to predict the initial center point. Although landmarks can be identified and initial estimates of sowing hole locations can be made, a comprehensive view of the entire cluster is necessary for an accurate density assessment. During the inference stage, the window slides towards the center point, as predicted by GoogLeNet [34]. The predicted center point was then subjected to a re-evaluation process. If the predicted center falls within an acceptable margin of error from the actual center of the window, a feature vector is extracted. This feature vector was subsequently used as an input for the classifier to assign labels. The loss function employed to train GoogLeNet is defined by Eq. (3) providing by [35]. (3)L=(xi^−xp)2+(yi^−yp)2d2where (xpi,ypi) is the given center of the data image i. (xi^,yi^) is the predicted center of the network. d is the window size. Furthermore, to enhance the accuracy of the final classifier, U-Net [36] was trained for background removal and cluster segmentation. In our experiment, the cluster class had a strong correlation with the cluster segment area. A parallel pipeline makes the final classifier more robust and reliable.

To train the proposed system, three different datasets were prepared and created regarding 3 different stages [33]. The first dataset comprises images in which cluster centers, that is, dots, are located at various positions. In this set, the images of undefined objects have center coordinates of (0, 0). In this study, we employ the U-Net deep learning network. U-Net offers several advantages in image segmentation, particularly for remote sensing and biological applications. Additionally, the U-Net model demonstrates superior efficiency in training with limited data compared to other convolutional neural networks (CNNs). Moreover, U-Net is well-suited for applications with small batch sizes, which is particularly relevant to this study, as the size of rice plants is relatively small. For Unet training, a set of raw and mask images were created, and some typical samples are shown in Fig. 5.

To enrich the dataset and improve its suitability for training, various augmentation techniques have been applied, including image transformations, such as rotation, blurring, denoising, and blending. These transformations were iteratively applied to increase the diversity of the dataset. The resulting augmented dataset was more comprehensive and robust, encompassing a wider range of visual characteristics and complexities. This enhanced dataset improves the ability of the model to generalize and perform well during training and evaluation. A detailed breakdown of the class distribution within the augmented dataset is presented in Table 2 with 80% for training and 20% for testing.

Following the training phase, key performance metrics such as accuracy, precision, recall, and F1-score, as defined in Eqs. (4)–(7) were calculated using the confusion matrix to assess the model’s performance: (4)accuracy=TPn+TNnTPn+FPn+TNn+FNn (5)precision=TPnTPn+FPn (6)recall=TPnTPn+FNn

(7)F1_score=2 ∗ precison ∗ recallprecision+recallwhere TPn/TNn represents the true positive and true negative classes, FPn/FNn represents the false positive and false negative classes, and n represents the count.

The model was trained on a personal computer equipped with an Intel Core i5-14400F processor and 16 GB of RAM. Using MATLAB software, the training process was conducted on a single CPU for approximately 133 min. The training procedure consisted of 30 epochs, with 339 iterations per epoch. The performance of the proposed model was evaluated using a confusion matrix Fig. 6 and a set of statistical metrics (Table 3). The proposed model demonstrated a significantly high accuracy. Misclassifications were minimal, with 15 of 130 samples being incorrectly classified between “single rice seedlings” and “undefined objects.”

Extensive experimentation has determined that the optimal altitude for UAV operation is six meters above ground level. Fig. 7 illustrates the results of the proposed model, where the red dots indicate the center of each seedling. These findings demonstrate that our method effectively estimates the number of seedlings in each aerial image, despite variations in plant size across the field. However, challenges arise in the precise location of individual rice seeds within certain image regions. These challenges are particularly evident when dust or other unidentified objects obscure the view of the rice plants or when the seeds are very small and may be mistakenly classified as background features by conventional analysis techniques.

Fig. 8 shows the results of classifying and counting rice plants from the aerial images. The results show that the proposed method can detect and classify nearly all rice plants in the image frame. The green rectangle represents a single rice seedling, whereas the yellow rectangle represents a cluster rice seedling. The detailed results indicate that the proposed method can classify single rice plants even under backlit conditions. Under good lighting conditions, the proposed method could classify almost all single and clustered rice seedlings.

To evaluate the effectiveness of UAV technology compared with traditional methods, we initially focused on two key parameters: “hill density per square meter” and “tiller number per square meter.” However, because of the overlapping canopy caused by germination, only the “hill density per square meter” could be reliably measured.

In traditional method, the hill density per square meter, five sample plots, each measuring 50 × 50 cm², were randomly placed at the four corners and the center of the field (Fig. 1). Within each plot, the number of single seedlings and cluster seedlings per square was recorded. Using data from these plots, the hill density was extrapolated to estimate the total density for the entire paddy field.

In our method, samples were collected from UAVs at designated locations. For each area, images were randomly selected for analysis, and the values derived from these images were averaged and compared with the manually calculated results. Since the image dimensions could be measured (as shown in Table 1), the average value for the sampled areas was determined.

Table 4 presents the results obtained from the five different images captured at an altitude of 6 m.

At an altitude of 6 m, the proposed UAV-based method and the traditional ground-based approach for estimating hill density showed minimal differences, typically ranging from 2 to 4 hills per square meter. Additionally, it is important to note that valuable information about rice germination can be obtained by analyzing the average number of clusters and individual rice seedlings per square meter.

To compare our research findings with those of several state-of-the-art studies on seedling detection and counting. Zhang et al. [12] achieved 93% accuracy in segmenting rice plants using LW-SegNet and LW-Unet with a multi-spectral camera mounted on a UAV at a 30-m flight altitude. Tseng et al. [32] and Yang et al. [37] leveraged the UAV Open Dataset to detect transplanted rice plants at a 40-m altitude, achieving accuracies between 99% and 100% using various deep learning methods. Bai et al. [38] focused on reproductive stages and introduced RiceNet, a network that improved rice plant counting accuracy, achieving the lowest mean absolute error (MAE) of 8.6 and root mean square error (RMSE) of 11.2 compared to other networks.

Previous studies have also employed deep learning networks to count rice plants in the field. However, these studies primarily focused on accuracy evaluation and were conducted on transplanted rice plants, which had already reached a considerable size. Moreover, the germination period of the rice plants in those studies was relatively long. Our research focuses on the stage when rice plants are about to begin tillering, at which point the germination rate in the field can be observed. We classified them in different groups, including one rice seedling and a cluster of seedlings, and calculated the number of hill/m². This is highly significant in supporting farmers during the later stages of the growing season.