In the era of technology, vehicular cloud computing (VCC), Internet of vehicles (IoV) and cloud of things (CoT) are some of the promising technologies to provide better services to the users [1–6]. Many modern smart vehicles are connected to the cloud in VCC to offer various services, such as information, storage, cooperation, computation, and infotainment as a service [7,8]. Each modern smart vehicle is equipped with an on-board unit (OBU). In general, OBU contains various components, such as storage unit, processor, sensors, global positioning system (GPS), cameras, communication systems and many more, and OBU is underutilized for a considerable amount of time [3,9]. Moreover, VCC provides a platform for these underutilized vehicles’ resources, and these resources can be intelligently utilized by offering services to the users on a pay-per-use basis [10]. For instance, car drivers and passengers can benefit by obtaining services like traffic management, parking availability, emergency assistance, smooth navigation, multimedia content sharing, etc., from the VCC [1,11].

A significant number of vehicles is spending considerable amount of time at parking lot and moving within a given area (hereafter, referred as grid). These vehicles can be participated in VCC by connecting to nearest fixed roadside unit (RSU), which is further connected to cloud service provider (CSP) in order to provide backbone infrastructure to the cloud [1,10–12]. On the other hand, CSP can assign the loads to the RSUs in order to execute the same in the roadside infrastructure. Upon receiving loads from CSP, RSU can identify suitable vehicle and create a VM on that vehicle to perform load sharing as reported in [13]. However, the resources of the vehicle are not fully utilized as the created VM is handling all the resources of the vehicle. Many studies [13–22] reported that some of these VMs are overloaded and others are underutilized. Moreover, energy consumption of overloaded and under loaded vehicles are varying with respect to their utilization. Note that high utilization leads to drastic increase in energy consumption and low utilization or idle situation leads to wastage of energy. Alternatively, energy consumption increases exponentially on or above 70% of resource utilization [14,19]. Therefore, it is challenging task to manage overloaded/source and under loaded/destination vehicles and share their load to minimize the energy [15,18,23,24]. One of the solutions is to migrate/consolidate the over utilized VMs of overloaded vehicles to under loaded vehicles in order to improve resource utilization. This phenomenon motivated us to identify the overloaded vehicles and their corresponding target vehicles by incorporating threshold value and checking the suitability of the vehicles.

In this paper, we propose a resource management algorithm, called resource utilization-aware VM migration (RUVMM) that selects the appropriate source and destination vehicles, and suitable VMs hosted on the source vehicle for the process of VM migration. For this, it monitors the trend of resource utilization and selects the source and destination vehicles within a pre-determined threshold limit. It ensures that the resource utilization of any vehicles should not exceed the threshold before or after the migration process. In this way, it solves the challenges associated with VM migration in VCC. RU-VMM is simulated using MATLAB and tested using nine randomly generated datasets. The simulation results are carried out for RU-VMM algorithm and compared with two existing algorithms, namely threshold-based algorithm [14,18] and cumulative sum (CUSUM) [13] in terms of three performance metrics, namely number of source vehicles, percentage of successful VM migrations and percentage of dropped VM migrations. For the sake of comparison, we use threshold and CUSUM algorithms as other algorithms are not directly comparable to our algorithm. The comparison results show that RU-VMM algorithm performs better than other two algorithms in terms of above performance metrics. Our major contributions are as follows.

The rest of the paper is organized as follows. Section 2 discusses various studies carried out in the field of VM consolidation and migration. Section 3 presents the vehicular cloud model and problem statement followed by the proposed algorithm and performance metrics in Section 4 and Section 5, respectively. Simulation results are discussed in Section 6. We conclude in Section 7 with some notable remarks and future works.

In the recent past, many researchers [7,9,25] have explored the advancement and application of VCC for better traffic management, entertainment services, assistance in emergencies to drivers, finding parking areas, smooth navigation, managing data centers, parking lot and many more. With an increase in popularity, VCC also faces various notable problems, such as resource management, energy consumption, bandwidth, latency and many more [17,23,26]. Researchers have explored different dimensions of VCC to address the above-mentioned problems, which are briefly discussed in this section.

Hamdi et al. [18] have stated that high energy consumption makes an adverse effect on cost and environment. They have highlighted VM consolidation techniques in various dimension and discussed some important characteristics of VM consolidation techniques like varying nature of resource requirement, ideal power consumption of physical machines etc. Wu et al. [23] have proposed a power-aware scheduling algorithm, which uses the utilization threshold strategy to select host physical machines and minimum utilization gap to select VMs for the process of VM migration. Here, they used power-aware best fit decreasing to select destination vehicles. Hsieh et al. [17] have considered both current and future utilization of resources, and predicted future utilization of resources more accurately. They have determined the host overload and under load detection to perform this prediction. Li et al. [26] have proposed an energy-efficient and quality-aware VM consolidation technique, which uses four algorithms, namely multi-resource host overload detection, quality of service-aware VM selection, discrete differential evolution-based VM placement and under-loaded host detection to improve resource utilization and reduced energy consumption. Mekala et al. [16] have focused on the resource requirement rate for task classification, resource balance ranking and processing element cost to evaluate overloaded host, under loaded host and VMs for migration. Sharma et al. [27] have proposed a failure prediction technique, which is based on exponential smoothing to trigger two fault tolerance mechanisms, namely VM migration and VM check pointing. Many other studies on power and resource-aware VM placement and VM migration techniques are discussed in [3,10,11,28,29].

Andreolini et al. [14] have presented a dynamic load management approach using cumulative sum (CUSUM) algorithm to take a decision on VM reallocation. For this, it determines the abrupt change in the CPU utilization without considering any fixed thresholds. If the change is persistent, then it selects the corresponding host as a sender host and finds a suitable guest host to initiate the migration process. This algorithm achieves better performance and low overhead as it minimizes the number of migrations. However, the number of source vehicles is increased to a greater extent due to unknown thresholds over time. Alternatively, they have considered the load behavior of resources without relying on average measures. Khanna et al. [15] have monitored the performance of resources. If the performance exceeds a threshold, then it initiates the process of migration. The threshold value is relying on the service level agreement. Hence, it may vary with respect to the resources. Hsu et al. [19] have considered 70% as a threshold to perform the task consolidation. If the load exceeds 70%, then it redirects the load to other resources. However, in the case of the unavailability of resources, it keeps the load by violating the threshold. Some other studies on threshold-based algorithms are discussed in [20,21]. Most of the above studies have not considered the trend behavior of resource utilization of the vehicles while selecting source and destination vehicles in the process of VM migration. In this paper, we propose an algorithm, RU-VMM, which focuses on the trend and change in resource utilization to select source and destination vehicles and continuously monitors the utilization before and after the process of migration, such that unnecessary migrations are avoided.

The proposed resource utilization-aware VM migration (RU-VMM) algorithm is a load balancing algorithm. It is implemented in the centralized cloud in which RSUs of the grids act as intermediary between vehicles and cloud. The objective of this algorithm is to migrate the load from the VMs that are hosted on overloaded vehicles to the VMs that are hosted on under loaded vehicles. Alternatively, it balances the load of the VMs and improves the resource utilization of the VMs. This algorithm works in three phases as follows. In the first phase, it finds the set of probable source vehicles. In the second phase, it finds the VMs that are hosted on the source vehicles to initiate the process of migration. In the last phase, it finds the destination vehicles to migrate the VMs and performs the actual process of migration. The pseudo-code of these phases is shown in Algorithm 1 and Procedures 1–4. Tab. 1 shows the notations and their definition used in the pseudo-code of the proposed algorithm. An overview of the proposed algorithm is shown in Fig. 2.

RU-VMM algorithm first calculates the utilization of the vehicles for a given time window as shown in Algorithm 1 (Line 1–5). Then it calls the three procedures for selecting probable source vehicles (Procedure 1), checking their suitability (Procedure 2, which is called from Procedure 1, Procedure 3 and Procedure 4), finding VMs (Procedure 3) and destination vehicles (Procedure 4), respectively.

The Algorithm calls Procedure 1 to find a set of probable source vehicles. For this, it checks sum of the utilization of VMs that are hosted by vehicles within a time window t. Note that the sum of the utilization of the VMs should not exceed 100%. If the utilization value is above a threshold (say, 70%) at any point of time and it is persistent, then the vehicle is a potential candidate in the set of probable source vehicles (Lines 2–8 of Procedure 1). Note that the threshold is determined based on the drastic change in energy consumption and fixed at 70% in this paper as adopted in [19]. The determination of actual threshold is beyond the scope of this work. It is noteworthy to mention that the vehicles, whose utilization are above 70% for a small interval of time, are also potential candidates in the set of probable source vehicles. In order to avoid this, Procedure 1 calls the Procedure 2 to check the suitability of the source vehicles (Line 11).

Procedure 2 checks the suitability of source vehicles by determining the rolling change in utilization (Lines 2–8 of Procedure 2). The rolling change is calculated using the equation given in Line 5. Note that it calculates the utilization for every instance of time and every duration of time. Then the rolling change values are averaged (Line 7). Next, this procedure calculates the standard deviation of average values and compares the product of standard deviation and 100 with the product of standard deviation of utilization of vehicle and a system parameter, λ (Line 10). If the later value is more, then the vehicle is suitable for a source vehicle (Line 11 of Procedure 2 and Lines 12–14 of Procedure 1). Note that determination of actual value of the system parameter is beyond the scope of this work. Otherwise, the vehicle is removed from the set of probable source vehicles.

Next, the algorithm calls Procedure 3 to find a suitable VM from the final set of source vehicles (Line 7 of Algorithm 1). For this, it determines the standard deviation of VM utilization (Lines 4–6 of Procedure 3) and arranging them in descending order of their standard deviation (Line 7). Then it finds the highest standard deviation and make that VM as a potential candidate for VM migration (Lines 10–12). However, we have not used any elastic computing theory concept to find the potential VM for migration. Next, the suitability of the same vehicle is further checked (Lines 13–21). If the vehicle is still suitable, then another potential candidate is determined by following the above process (Lines 9–22). Otherwise, the vehicle is removed from the set of final source vehicles.

Next, the algorithm calls Procedure 4 to find the destination vehicles so that the VM of the source vehicle can be migrated to one of the destination vehicles. For this, it determines the average utilization of all vehicles, excluding the vehicles that are present in the set of final source vehicles and sort them in the ascending order of their average utilization (Lines 2–7 of Procedure 4). Then the vehicle with lowest average utilization is selected as a destination vehicle (Line 9) and the migration process is carried out (Line 11). The suitability of the selected vehicle is checked to ensure that the destination vehicle should not become the source vehicle after the successful migration (Lines 10–23).

Illustration

We illustrate the proposed algorithm using four vehicles (i.e., V₁, V₂, V₃, and V₄) that is present in a grid. Here, each vehicle can host one or more VMs. The proposed algorithm first calculates the resource utilization of the VMs that are hosted by the vehicles. These values are shown up to time instance t = 10 in Tab. 2, and their cumulative values with respect to vehicles and time instances are shown in Fig. 3. Then the proposed algorithm calls Procedure 1 to find the set of probable source vehicles. Here, vehicles V₁, V₂ and V₃ are the set of probable source vehicles as their resource utilization values exceed 70%.

Now, the algorithm calls Procedure 2 to check the suitability of the vehicles as source vehicle(s). For example, let us check the suitability of the vehicle V₃. For this, the algorithm calculates the rolling change (RC) in the resource utilization. For V₃, resource utilizations are 20, 25, 32, 39, 43, 69, 78, 81, 88 and 95 for time instance 1 to time instance 10, respectively. Suppose k = 1 and k′=1, RC₃ is calculated as (2520)11-1=0.2500. Note that RC is calculated for each time instance k and these values are (3225)11-1=0.2800, (3932)11-1=0.2188, 0.1026, 0.6047, 0.1304, 0.0385, 0.0864 and 0.0795, respectively. Then these RC values are averaged as 0.1990 for k = 1. Similarly, RC values are calculated for k = 2 (i.e., (3220)12-1=0.2649, 0.2490, 0.1592, 0.3301, 0.3468, 0.0835, 0.0622 and 0.0830) to k = 9 (i.e., (9520)19-1=0.1890), respectively and the average values, for k = 2 to k = 9, are 0.1973, 0.1981, 0.2048, 0.2178, 0.2037, 0.1954, 0.1925 and 0.1890, respectively.

Next, the algorithm finds standard deviation of all the averaged values as 0.0084 (i.e., SD_arc) and standard deviation of the utilization as 14.0500 (i.e., SD_ru) of the vehicle V₃. As (SD_{arc}\times 100 = 0.0084 \times 100 = 0.8400)$]]>(SDru×λ=14.0500×0.5000(Let)=7.0250)>(SDarc×100=0.0084×100=0.8400), the vehicle V₃ is suitable for a source vehicle. Note that λ is a system parameter and it is set as 0.5 for this illustration.

The vehicle V₃ hosts three VMs (i.e., VM₃₁, VM₃₂ and VM₃₃). Now, the algorithm finds the suitable VM(s) that is hosted on vehicle V₃ by taking the standard deviation of VM utilization and arranging them in descending order of their standard deviation. The standard deviation of VM₃₁, VM₃₂ and VM₃₃ is 4.4500, 3.9800 and 21.0400, respectively. As VM₃₃ contains the highest standard deviation, it is the potential candidate for VM migration. Once the successful migration of VM₃₃ is over, vehicle V₃ is no more suitable for a source vehicle to carry out further migration.

Now, the algorithm finds a destination vehicle to migrate VM₃₃. For this, it finds the average utilization of all vehicles, excluding the vehicles that are present in the set of final source vehicles and sort those vehicles in the ascending order of their average utilization. In this illustration, V₄ is only available. Note that VM₃₃ can be migrated to vehicle V₄ if the utilization of vehicle V₄ after accommodating VM₃₃ should not be a potential candidate for the set of probable source vehicles. Here, VM₃₃ is migrated to vehicle V₄ and vehicle V₃ is removed from the set of final source vehicles after this successful migration. As a result, the total number of VM candidates is one, i.e., VM₃₃.

From the set of probable source vehicles (refer Fig. 3), it is clearly visible that the change in utilization for vehicle V₃ is stable, but change in utilization for vehicle V₁ and vehicle V₂ is not stable. In this illustration, our proposed algorithm RU-VMM finds vehicle V₃ as a source vehicle. Note that apart from vehicle V₃, other vehicles, that are present in set of probable source vehicles, are not suitable for migration. We run the same illustration for the existing threshold-based algorithm [15,19–21] and CUSUM algorithm [14]. Here, threshold-based algorithm finds VM₁, VM₂ and VM₃ as source vehicles and CUSUM algorithm finds VM₂ and VM₃ as source vehicles. The detailed comparison of proposed and existing algorithms is given in the Tab. 3.

In this section, we present three performance metrics to compare the proposed and existing algorithms. Their definitions are presented in the following subsections.

We create a virtual environment using MATLAB R2017a (version 9.2), on a computer system with Intel(R) core(TM) i7-4790 CPU@3.60 GHz 3.60 GHz, 8 GB RAM, 64-bit Windows 10 operating system for the simulation of the proposed algorithm RU-VMM. We also use MATLAB R2017a to generate nine datasets by considering three different types of situation, namely low congestion, medium congestion and high congestion of vehicles. Note that these datasets are named using the total number of vehicles in a grid (e.g., 500, 1000, 1500, 5000 etc.). Each dataset further contains three instances of same size. We follow the Monte–Carlo simulation method and uniform distribution to generate the datasets [5,13]. The resource utilization is collected for one hour and the average resource utilization is determined in each five minutes for the simplicity of simulation. The detail of the parameters and their respective values is shown in Tab. 4. In addition, we use simulation of urban mobility (SUMO) traffic simulator 1.6.0 to generate the parameters, such as timestamp, lane, position, speed, vehicle acceleration/deceleration and vehicle length, and network simulator OMNeT++. We also use IEEE 802.11p with ITS band of 5.9 GHz to exchange the data between the vehicles and/or the RSUs [30]. This band enables wireless access in vehicular environments. We use a open source hybrid simulation framework, called Veins, which uses IEEE 802.11p for communication and to integrate two simulators, namely SUMO and OMNeT++ using traffic control interface (TraCI) [31]. In the simulation process, we consider a city map (Bhubaneswar, Odisha) as shown in Fig. 4, which consists of a set of lanes. The other configuration parameters are shown in Tab. 5.

The simulation results of the proposed algorithm, RU-VMM and two existing algorithms, threshold and CUSUM are compared in terms of number of final source vehicles and percentage of successful/dropped VM migration as shown in Figs. 5 and 6, respectively. Note that we present the average of three instances per dataset for the comparison of the proposed and existing algorithms. The simulation results show the better performance of the proposed algorithm RU-VMM in comparison to the existing algorithms. RU-VMM algorithm improves 16.91% than the CUSUM algorithm and 71.59% than the threshold algorithm in terms of nfsv, and 20.62% than the CUSUM algorithm and 275.34% than the threshold algorithms in terms of psvmm. The simulation results are also carried out using SUMO and OMNeT++, and compared in terms of percentage of dropped VM migration as shown in Fig. 7. Here also, the proposed algorithm outperforms the existing algorithms. The rationality behind this better performance is as follows. (1) The proposed algorithm finds the RC of set of probable vehicles to determine continuous increase in utilization. (2) The proposed algorithm uses standard deviation to monitor the abrupt change of VMs. (3) The proposed algorithm restricts the VM candidates by continuously monitoring the utilization after each successful migration.

In this paper, we have presented a novel algorithm RU-VMM for VM migration in vehicular cloud computing. The objective of the proposed algorithm is to migrate the load from the overloaded vehicles to the under loaded vehicles. For this, RU-VMM creates a set of probable sources vehicles and monitors the rolling change in resource utilization to select the final source vehicles. Then it finds a destination vehicle to migrate the VM by continuously monitoring the resource utilization of the destination vehicle and the source vehicle. The proposed algorithm has been simulated using MATLAB and compared with two existing algorithms, threshold and CUSUM using twenty-seven instances of nine dataset. The comparison has been shown in terms of three performance metrics, namely number of final source vehicles, percentage of successful migration(s) and percentage of dropped migration(s). The comparison of simulation results has shown that the proposed algorithm RU-VMM outperforms the two existing algorithms in terms of three performance metrics.

The proposed algorithm has not considered the bandwidth and delay associated with VM migration, which can be considered as a future work. Moreover, determining an appropriate threshold to prepare a set of probable source vehicles is very much interesting and beyond the scope of this paper. It can be further investigated to develop more efficient algorithm. We have not considered the parameters associated with driving strategies, such as speed, acceleration, path and many more, which can be considered as a future work. We have also not considered the road accident and emergency scenarios in our proposed model and algorithm, and not tested the proposed algorithm in the actual environment. The congestion scenarios can be generated using SUMO by taking the number of vehicles and speed of the vehicles with respect to the traffic routes. We will extend our work by incorporating these scenarios in our future work.