For a long time, the service life of pavement has been short, and it is difficult to reach the design life. The maintenance cycle of the pavement is short, which greatly wastes manpower, material, and financial resources.

The first step in checking on road conditions is to monitor the pavement, which helps us assess how well the roads are performing and whether they’re safe and comfortable for drivers. Several studies have looked into this, trying to figure out just how damaged the pavement might be [1,2]. There are different ways to monitor road surfaces, whether they’re paved or unpaved, to understand the type and extent of any damage. This is really important for deciding on the right maintenance approaches. When it comes to monitoring pavement conditions, you’ve got two main options: static monitoring and dynamic monitoring. The choice depends on the kind of equipment and measurement methods being used [1]. In static monitoring, you set up detection devices in specific spots for a certain amount of time. On the other hand, dynamic monitoring involves using devices attached to vehicles, collecting data as those vehicles drive along different road sections. This method gives a snapshot of the pavement’s condition across various parts of the road [2–5].

There are quite a few methods out there for checking how good our road surfaces are and how well our road networks are serving drivers. Some of these methods involve things like visual inspections, image analysis, checking vibrations, tools that measure roughness (like profilometers), and using Pavement Condition Indices (PCIs). In this paper, we focus on using data from pavement vibrations as a way to gauge the state of the road, which we capture with an accelerometer sensor while vehicles are on the move. This vibration data gives us insights into the condition of the road surface and any wear or damage. When the road is in bad shape, it can lead to more vibrations in the vehicle, making for a bumpy and uncomfortable ride [1,6,7].

Monitoring the condition of road pavement and predicting its deterioration go hand in hand when it comes to sustainable road management. The first step in assessing how well a road is holding up is to pinpoint the type, severity, and number of issues on its surface. With accurate data on pavement conditions, we can create reliable models to forecast how the pavement will degrade over time. These prediction models are crucial for transport agencies, as they help them understand how pavement behaves and how quickly it deteriorates under various traffic situations. Different techniques have been used to create prediction models, including Artificial Neural Networks (ANN), empirical methods, mechanistic-empirical approaches, and Machine Learning models [6,7].

There are a bunch of different technologies and techniques out there for accurately checking pavement conditions with minimal errors. Some of these include vision-based methods and vibration-based approaches. When it comes to vision-based methods, video cameras have become popular for monitoring pavement conditions [7,8]. They not only help in tracking pavement condition but also verify findings from vibration studies, giving us better insights into the actual state of the pavement [9–12]. For instance, Koch et al. [13] did a thorough study on using computer vision for detecting pavement distress. They highlighted the benefits of using a vision-based method for monitoring both concrete and asphalt pavement and suggested that video cameras are a reliable way to get accurate results. Hadjidemetriou et al. [14] worked on a technique for detecting patches using a vision-based approach. They employed two video cameras—a built-in smartphone camera and an external one—to assess road networks in Cyprus. These cameras captured video as vehicles moved, which was then processed into individual frames for analyzing the severity and extent of patches on the roads. Similarly, Shtayat et al. [1] recorded local road conditions in Melbourne, Australia, using a line-scan camera mounted on the rear of a sedan. This setup helped to document pavement surface issues while the car was on the move at various speeds, yielding valuable insights about the pavement’s actual condition, along with details on the type, severity, and location of any distress.

On the flip side, vibration-based methods have gained traction among researchers over the past decade for assessing pavement conditions and giving crucial insights into pavement health. These systems are great for revealing road pavement anomalies, surface smoothness, and overall comfort while riding [15]. Recently, monitoring pavement vibration data has proven to be effective in enhancing pavement management systems [16]. The effectiveness of using vibrations as a gauge for pavement conditions lies in its ability to identify how severe deformations are on road surfaces [17,18].

To gather pavement vibration data while vehicles are moving, dynamic monitoring systems make use of accelerometer sensors, smartphone apps, and wireless vibration meters [19]. For example, Li et al. [19] utilized an accelerometer to measure vibrations caused by vehicles passing over road defects at low speeds. This allowed them to dynamically assess the condition of the pavement surface based on the vibrations recorded during vehicle movement.

Sabir et al. [20] introduced a new way to monitor road conditions, focusing mainly on detecting potholes and speed bumps in Pakistan. Their method relies on specific ranges of vibration thresholds for both types of issues, utilizing crowdsourced data. When the vertical vibration readings surpass the chosen limits, the system flags those peaks as potential potholes. Their algorithm managed to accurately detect around 85% of potholes and about 90% of speed bumps. Meanwhile, Wu et al. [21] took a different approach by using a smartphone app placed on the back seat of a sedan to gather vibration data from urban roads in China. They analyzed the vibrations to identify various types of road distress using the Dynamic Time Warping (DTW) technique. They also created a reference dataset for potholes based on the vibration data they collected. Recognition was achieved by comparing the gathered information to these reference layouts, and their random forest (RF) method showed about 88.5% accuracy in detecting potholes.

There’s been a lot of work to improve detection rates using vibration data, with algorithms developed to incorporate different types of filters like high-pass filters, XZ-ratio filters, and z-peak filters to boost accuracy [21]. Each of these filters has unique characteristics that help manage and enhance the quality of the monitoring data, using specific thresholds as tuning parameters to improve detection accuracy. With these filters, the detection rate for potholes can reach around 92% after clustering and labeling [22,23].

When it comes to machine learning (ML), there are standard steps involved. It starts with selecting an appropriate filter to reduce random data and noise, then extracting relevant features that represent potential distresses from both time and frequency domains. After that, the features are classified to pinpoint the correct type of pavement distress. This stage helps researchers gauge the overall condition of the pavement, along with assessing the quantity and severity of any degradations [24]. The process of extracting features from thresholds is crucial for predicting pavement performance [25]. ML models serve as classification tools to identify both the type and severity of distress based on these thresholds and to forecast pavement conditions. For example, Seraj et al. [26] and Basavaraju et al. [27] used a smartphone app to gather pavement vibration data with the goal of developing an ML algorithm that could classify features from the vibration threshold data. They also extracted features using Fast Fourier Transform (FFT) and Discrete Wavelet Transform (DWT) from both time and frequency domains. Additionally, Altabey [28] and Li et al. [29] proposed a fresh approach for predicting the quantity and severity of potholes by extracting features from vibration thresholds using DWT.

We can follow the development of road damage detection approaches during the last decade in Fig. 1. By establishing an advanced and practical pavement health and safety monitoring system based on a deep learning model for the pavement, we can grasp the pavement operation status in real time and realize real-time safety alarms for the pavement service level; rationally allocate pavement maintenance resources; provide a scientific and technological basis for reducing pavement operation and maintenance costs; and ensure that the pavement inspection and maintenance strategy is targeted, timely and efficient.

As shown in Fig. 1, the pavement pothole detection approaches include traditional 2-D image processing algorithms that handle tasks like enhancing, compressing, transforming, and segmenting road camera images or depth images, all through explicit programming. On the other hand, machine and deep learning approaches tackle the challenge of detecting potholes using methods like image classification, object recognition, or semantic segmentation, often leveraging advanced convolutional neural networks (CNNs). Then you’ve got 3-D road point cloud modelling, where algorithms match a specific geometric shape—usually a flat or curved surface—to the actual road point cloud, segmenting it by comparing the observed data with the fitted models. Lastly, hybrid methods that mix two or more of these algorithm types aim to boost overall performance in pothole detection.

The schematic description of the pavement monitoring is illustrated in Fig. 2. For automatic monitoring of asphalt pavement cracks, we will use the requirements in Fig. 2 for the operation period of the pavement crack to realize the automatic acquisition of parameters such as the pavement crack and pavement pothole and to have intelligent analysis, evaluation, and critical threshold alarm for the safety status of various structures. Specifically include:

Asphalt strain monitoring: It is laid out at the bottom of the asphalt layer to obtain the dynamic bending and tensile strain data of the bottom of the asphalt mixture layer. The strain gauges are arranged horizontally, with longitudinal and transverse directions to measure the strain in the vertical direction (Fig. 3a).

Concrete strain monitoring: The surface layer of the water-stabilizing layer below the asphalt layer is arranged corresponding to the asphalt strain test position to obtain the three-dimensional stress dynamic response data of the road base (Fig. 3b).

Earth pressure monitoring: Used to measure the vertical pressure of the roadbed layer under dynamic load. The pressure cell is arranged on the top and lower surfaces of the roadbed layer, located in the center of the inner wheel track, because it is expected that the stress generated by the wheel on the pavement structure is greatest at this position (Fig. 3c).

Soil moisture monitoring: Soil moisture sensors are placed in the soil layer of the external roadbed to measure the moisture content of the soil layer (Fig. 3d).

Multi-point displacement monitoring: Measures the settlement displacement of the water-stable layer and the roadbed layer. Multiple settlement meters are arranged on the shoulders outside the carriageway to measure the displacement changes of the base layer (Fig. 3e).

Temperature monitoring: On the hard shoulder outside the lane, measurements are taken in layers from the asphalt layer to the roadbed soil layer (Fig. 3f).

Dynamic weighing: Used to measure the size and type of load. The placement of the sensor should take into account the wheel offset and vehicle type during measurement. It can detect the axle weight, total weight, wheelbase, speed, length, and passing time of vehicles at a speed of 0 to 200 km/h, automatically capture the license plate, and automatically separate the vehicle (Fig. 3g).

Weather Information: Collects various meteorological information at the monitoring points, mainly including temperature, humidity, wind speed, wind direction, rainfall, air pressure, total radiation, illuminance, and ultraviolet intensity (Fig. 4).

A Convolutional Neural Network (CNN) is a type of deep learning model particularly well-suited for processing data with grid-like topology, such as images and videos. CNNs excel at automatically learning features from raw input data, eliminating the need for manual feature extraction. They are widely used in computer vision tasks like image recognition, object detection, and image segmentation, and can also be adapted for audio and other signal processing tasks [18,19].

In this work, the mentoring system deployment technology for automatic monitoring is carried out for long-term pavement performance in the Xinjiang Autonomous Region by burying a variety of sensors in different parts of the road surface, base, roadbed, slope, etc. Aided by deep learning algorithms, a sensor monitoring network based on the Internet of Things technology is formed to achieve accurate, reliable, and continuous observation of environmental meteorology, physical state, mechanical response, structural deformation, and other indicators. Due to the large amount of data and high real-time requirements, the perception data collected from sensors such as temperature, humidity, pressure, asphalt strain, and displacement are used to train a deep learning model based on a Convolutional Neural Network (CNN) Algorithm to predict the pavement multi-point displacement to detect the pavement damage, such as asphalt cracks and asphalt potholes. Through comparison of CNN-predicted data with pavement sensors experimental data for the multi-point displacement, the CNN demonstrates significantly improved prediction quality for this case study, showing its effectiveness in accurately modeling the structural behavior. It has been found that the response of the proposed CNN achieved high rates of P%, R%, and F% equal to 87.24% , 84.12% , and 85.96% , respectively. This work highlights the potential of using a variety of sensor-aided deep learning algorithms for the monitoring of long-term pavement performance.