Vehicle recognition technologies are essential for effective traffic management and law enforcement, enabling regulated vehicle flow and controlled access to restricted zones. Modern systems typically rely on three core components: Vehicle Make and Model Recognition (VMMR), Automatic Number Plate Recognition (ANPR), and vehicle color detection. VMMR enhances security by assisting in vehicle identification, supporting enforcement and surveillance tasks [1]. ANPR enables real-time tracking and precise license plate recognition, while color detection systems improve traffic monitoring and public safety through automated color extraction.

Recent advances in deep learning have demonstrated promising results by integrating VMMR and ANPR [1]. Building on these developments, our framework employs MobileNetV2 and YOLO for initial vehicle detection, while YOLOv4-tiny, PaddleOCR, and SVTR-tiny enhance ANPR accuracy under varying environmental conditions. Our approach refines existing methods, prioritizing lightweight architecture and operational efficiency for real-world deployment.

Despite progress in each domain, standalone systems face significant limitations in practical scenarios. VMMR systems often struggle with environmental noise, such as poor lighting, adverse weather, and congested traffic [2]. Similarly, ANPR performance degrades under low-light conditions, non-standard camera angles, and regional regulatory variations. Vehicle color detection is also susceptible to lighting fluctuations and weather-related distortions, leading to misclassifications. Without addressing these challenges, systems fail to provide comprehensive, actionable vehicle data.

Advanced deep learning models like EfficientNet [3] and Vision Transformers [4] have improved feature extraction and pattern recognition. EfficientNet achieves high accuracy through compound scaling, while Vision Transformers use attention mechanisms to capture fine-grained details. However, both demand substantial computational resources, limiting their use in resource-constrained environments.

While individual subsystems have matured, a critical gap remains in developing unified, lightweight frameworks that integrate VMMR, ANPR, and color detection. Our work bridges this gap by combining the strengths of each subsystem to compensate for their weaknesses. We anchor our framework in MobileNetV2, chosen for its efficiency and low computational overhead, making it ideal for real-time, edge-level intelligent transportation applications.

Despite extensive research, few holistic systems reliably integrate these capabilities under diverse conditions. This paper contributes the following:

We propose an integrated system combining VMMR, ANPR, and color detection in a modular, extensible architecture. By merging adaptive image processing with deep learning, our solution delivers efficient, multifunctional recognition.

By consolidating these functionalities into a single framework, our solution eliminates the need for separate systems, offering a more efficient and reliable approach to real-world vehicle recognition.

Vehicle Make and Model Recognition (VMMR), Automatic Number Plate Recognition (ANPR), and car color detection have been the focus of extensive research aimed at enhancing recognition performance across diverse real-world scenarios. While notable advancements have been made in each of these areas, most existing systems have been developed independently, resulting in fragmented solutions that often fail to deliver comprehensive performance in dynamic and varied environments.

VMMR, in particular, poses significant challenges due to high intra-class variance arising from vehicle modifications, inconsistent lighting conditions, and varying viewing angles. For instance, the study in [5] presents a deep learning-based VMMR model that demonstrates high accuracy under controlled settings. By leveraging convolutional neural networks (CNNs) for feature extraction, the system achieved impressive recognition rates. However, its performance declined significantly in real-world scenarios involving inconsistent backgrounds and fluctuating lighting conditions. Another approach in [5] employed a hybrid method combining CNNs and classical image processing techniques. Although effective in structured environments, the model’s reliance on high computational power and large volumes of labeled data limits its scalability and real-time application.

Recent developments in ANPR systems have also garnered attention. Research cited in [8] highlighted the success of traditional image processing techniques, particularly during daytime conditions. However, our analysis reveals a sharp performance drop during nighttime or under adverse weather conditions. Enhanced systems, as described in [8], integrated image preprocessing with deep learning to improve visibility robustness. Despite these improvements, they struggled with unusual license plate fonts and partially obscured characters, indicating room for further refinement.

Color recognition, too, remains sensitive to environmental conditions. The work in [9] introduced a technique that combined histogram equalization with image analysis to compensate for lighting inconsistencies. While this method achieved satisfactory results in stable lighting, it was less effective during rapid illumination changes, such as those caused by natural weather transitions. Further developments, including the machine learning algorithm proposed in [10], improved performance under such dynamic conditions. However, the model faced limitations when differentiating between similar dark shades like navy blue and black, often confusing them.

Integrated vehicle recognition systems combining VMMR, ANPR, and color detection are still relatively rare. Álvarez-Bazo et al. [11] proposed a joint VMMR and ANPR system optimized for well-lit conditions, but it lacked support for color detection. Another notable work by Guerrero-Ibez et al. [12] combined VMMR, ANPR, and facial recognition using SIFT and OCR techniques, achieving 75% accuracy on a toy car dataset. However, the system’s limited scalability and modest accuracy prevent its practical deployment. While several integrated models have demonstrated improved accuracy by using CNNs for VMMR and ANPR, their computational demands render them unsuitable for edge devices or real-time use. Moreover, most existing solutions overlook challenges related to environmental robustness, computational efficiency, and region-specific needs, such as accommodating the variations in Pakistani license plates.

Addressing these gaps, our framework integrates proven methods into a unified architecture that is both efficient and robust. It achieves a significantly improved overall accuracy of 93.3% (as discussed in Section 4), making it a strong candidate for real-world deployment.

Additionally, scalability to large datasets posed a challenge for the system. The findings of existing work are summarized in Table 1.

The proposed vehicle recognition and color detection framework employs advanced deep-learning models and image processing techniques to identify cars, license plates, manufacturers, and colors, as shown in Fig. 1. The process begins with image import from CCTV, dashcams, or other sources, followed by checks to ensure images meet acceptable formats, such as .jpg, .jpeg, and .png. Resizing, normalization, noise reduction, and contrast enhancement improve image quality and uniformity.

Using grid-scanning, YOLO (You Only Look Once) predicts bounding boxes and class probabilities to identify vehicles and output their coordinates [18]. Vehicle parts are trimmed for processing. We have trained the MobileNetV2 algorithm to accurately classify automobile manufacturers and models. The choice of MobileNetV2 for the VMMR task was driven by its ability to balance accuracy and computational efficiency. Unlike heavier architectures, MobileNetV2 is designed to perform well on resource-constrained edge devices, ensuring real-time performance critical to intelligent transportation systems [19]. Additionally, its depth-wise separable convolutions reduce computational cost without compromising feature extraction quality for ANPR, another YOLO model detects license plates, and EasyOCR extracts text. License plate OCR begins with grayscale conversion, scaling, and enhancement.

To detect color, BGR automobile images are converted to HSV color space, where color is clustered by HSV ranges to identify the dominant color. Vehicle make, model, license plate text, and color data are used for applications such as traffic monitoring, automatic toll collection, and smart parking. To handle data volume and scale, containerization and microservices are deployed on edge devices and cloud infrastructure. Integrating these modules with a registered vehicle database allows cross- checking, improving system efficacy. A CSV file backup serves as the test database for real-time verified vehicle information. The proposed framework comprises the following modules: VMMR, ANPR, color detection, as elaborated next.

Our system combines Vehicle VMMR, Automatic Number Plate Recognition technology, and color recognition to identify vehicles fully.

To ensure robustness and generalizability, our system was validated on public and curated datasets, reflecting the integration of VMMR, ANPR, and color detection for Pakistani traffic scenarios. Pakistani Vehicles Cars Dataset, with approximately 364 images, was used alongside a curated dataset of 4000 Stanford Car images having 48 classes for training and validation. The Pakistani dataset achieved 90.63% accuracy, reflecting optimization for local vehicle types.

ANPR module was trained and validated on 364 Pakistani license plates along with YOLO, achieving 93.0% detection accuracy and 88.5% OCR accuracy, aligning with the overall 93.75% (Fig. 5). The combined system shows reliable performance in these test results.

Fig. 5 shows the accuracy rate for each module. The ANPR module achieved an accuracy of 93.7% along with varying lightening conditions. The VMMR module achieved an accuracy of 90.63% in identifying vehicle makes and models. This high accuracy indicates the module’s ability to process complex visual data, effectively distinguishing different vehicle brands and models.

Fig. 6 illustrates the training process of the VMMR system using the CNN architecture, MobileNetV2. The left graph displays the training and validation loss over 40 epochs, while the right graph shows the training and validation accuracy over the same number of epochs. These metrics are crucial for understanding the model’s performance and generalization capabilities.

The training accuracy plateaus near 0.99, indicating that the model has effectively learned from the training data. However, the validation accuracy levels out around 0.85, with slight fluctuations, suggesting the model’s generalization on new unseen data has reached a peak with the current configuration.

In Table 2, the VMMR outcomes are summarized as follows:

True Positives (TP): 20 (cases with “TRUE” for Is True Vehicle)

From the results of VMMR shown in Table 2, the accuracy, precision and recall has been calculated and is presented in Table 3.

The ANPR module demonstrated a notable accuracy of 93.75% in reading license plates, as shown in Fig. 5. This result indicates the module’s effectiveness in preprocessing and accurately extracting license plate information. Techniques such as grayscale conversion and contrast enhancement contribute significantly to the module’s ability to accurately recognize and read license plates.

The color detection module based on the HSV color space reached the accuracy of 71.87% in color classification (Fig. 5). Although this accuracy is smaller than that of VMMR (90.63%) and ANPR (93.75%) it is good enough to be useful in practice, since it adds to the rest of the modules in our single framework. The HSV-based clustering algorithm was selected because of its computational simplicity, and therefore, could be used in resource-limited edge devices, and its resistance to illumination changes due to decoupling of hue and brightness (Section 3.3). The issues such as shadows or color fading are overcome thanks to adaptive thresholding and preprocessing methods (e.g., normalization, contrast enhancement), confirmed on 32 examples (Table 4) while color detection showing 15 True Positives out of 32 examples (Table 5) and Suzuki Alto case study (92% accuracy, Table 6). In comparison to deep learning-based classifiers (e.g., CNNs) that are resource-demanding and needs a substantial amount of labelled data, HSV clustering is more efficient and performance-wise, which is why it is part of the overall 93.3% vehicle matching accuracy Table 7.

The integrated performance of the VMMR, ANPR, and Color Detection modules resulted in a vehicle matching accuracy of 93.33% when cross-referenced with the registered database. This signifies that 93.3% of the vehicles were successfully matched with the registration database, with only 6.7% unmatched. The high success rate underscores the effectiveness of combining results from multiple modules for comprehensive vehicle identification. High accuracy across individual modules, coupled with overall vehicle matching success, indicates the system’s strong potential for practical applications in vehicle recognition and management.

Table 6 presents result for a specific vehicle, the Suzuki Alto 2007, tested for system consistency and robustness. 35 variations of this vehicle’s image were generated with adjustments in factors like angles and lighting conditions. Each image was tested across all three modules—VMMR, ANPR, and Color Detection, with the following consolidated results:

VMMR: The system correctly identified the Suzuki Alto with overall VMMR accuracy of 95% across the 35 variations.

Next, we tested the system with 35 different variations of the same vehicle image to evaluate its robustness under varying conditions. These unified results for the Suzuki Alto show that the system maintains high accuracy in vehicle identification, recognition, and classification even with variations in the vehicle image data.

Table 7 shows the results, confirming that the integrated system effectively performs vehicle make and model recognition, license plate recognition, and color detection, resulting in a comprehensive vehicle identification solution. The system’s high accuracy and reliability across modules make it suitable for implementation in applications like traffic monitoring and law enforcement.

Since the system’s overall system performance relies on the accuracy of matched vehicles, the overall system performance aligns with matched vehicle accuracy. Visual results and statistical data further validate the system, showing its ability to process and analyze vehicle images under diverse conditions. From a holistic perspective, the system combines several vehicles identification features, with each module contributing to a better understanding of each vehicle. This integrated solution leverages advanced technologies to enhance recognition accuracy and offers the practical framework for real-world applications.

The combined results from the VMMR, ANPR, and Color Detection modules offer a clear view of the system’s capabilities. By merging these components, the vehicle identification system proves to be both robust and effective. As shown in Fig. 7, an example vehicle is detected with key details such as make, model, license plate, and color. This integrated approach highlights the system’s ability to compile data from different modules, forming a complete profile of the vehicle.

The system’s overall performance in matching vehicles with a registered database is illustrated in Fig. 8. The system successfully matched 93.3% of vehicles, with only 6.7% remaining unmatched. This high matching rate demonstrates the system’s effectiveness in cross-referencing data from each module for accuracy. The success rate emphasizes the system’s reliability and strength in identifying vehicles.

This study presents a unified approach that integrates Vehicle Make and Model Recognition (VMMR), Automatic Number Plate Recognition (ANPR), and color detection into a cohesive system for advanced vehicle tracking. By leveraging complementary modules, the system enhances accuracy and reliability through cross-verification, minimizing recognition errors. Pixel-level integration further refines the process, enabling more precise and efficient identification—an essential attribute for real-world vehicle recognition and verification systems.

Ghaida Saadouli et al. [13] proposed a security solution combining license plate recognition, car model identification, and facial recognition for access control. Their method utilized SIFT and DoG for vehicle detection, while OCR and the Viola–Jones algorithm were used for number plate and facial recognition. Although their system achieved a 75% success rate in identifying vehicles, the results were based on tests involving toy cars under controlled conditions, limiting the generalizability of their findings.

In contrast, our research introduces a more scalable and contemporary system employing MobileNetV2 for VMMR, YOLO for ANPR, and HSV-based clustering for color detection. Tested in diverse real-time environments, our system consistently achieved a 93.3% accuracy rate—demonstrating strong performance regardless of lighting or environmental variations. Moreover, the use of deep learning techniques, along with containerized deployment, makes our solution suitable for large-scale smart city implementations.

While the HSV clustering method for color detection yielded lower accuracy (71.87%) compared to deep learning-based classifiers, it offered a lightweight and computationally efficient alternative—particularly advantageous for deployment on edge devices. In future work, we aim to explore hybrid models that combine HSV clustering with lightweight convolutional neural networks (CNNs) to improve accuracy under extreme lighting conditions while maintaining operational efficiency.

Overall, the integrated system proved to be more robust than standalone solutions, achieving a 93.3% matching accuracy (Fig. 8). This demonstrates the effectiveness of the combined modules in delivering reliable performance suitable for real-world intelligent transportation applications.

The integration of VMMR, ANPR, and color detection within a unified framework yields a robust and reliable vehicle identification system, well-suited for traffic monitoring, law enforcement, and vehicle management—particularly in the context of Pakistan. By leveraging proven techniques—MobileNetV2 for vehicle make and model recognition, YOLO combined with EasyOCR for license plate recognition, and HSV clustering for color detection—the proposed system achieves a commendable accuracy of 93.3% (as detailed in Section 4), consistently outperforming modular or standalone solutions. Comprehensive testing on diverse datasets—including the Stanford Cars dataset for VMMR and a custom set of 364 Pakistani license plates for ANPR—demonstrates the system’s effectiveness, adaptability, and resilience under various environmental conditions.

In the future, we intend to extend the ANPR module to support multilingual and international license plates, such as those found in Chinese and European regions, using datasets like CCPD and UFPR-ALPR. Enhancing color recognition under challenging lighting scenarios is another focus, with plans to incorporate lightweight CNN-based classifiers. To improve real-time deployment, we will investigate edge computing solutions, including Docker-based containerization, to minimize latency within smart city infrastructure. In parallel, advanced data augmentation strategies will be developed to better handle occlusion and adverse weather conditions such as fog or rain. Finally, adopting a cloud-native architecture with microservices will enable horizontal scaling, making the system capable of managing large-scale urban traffic networks efficiently.These enhancements aim to reinforce the framework’s real-world applicability, enabling high-performance, real-time vehicle recognition while preserving its modularity and computational efficiency.