In this paper, we deploy an MEC environment where each base station is equipped with an edge server with a storage capacity denoted as 𝒯. Each mobile user establishes a connection with a specific MEC server, and within any given time slot, a user can only communicate with one base station. MEC servers are interconnected with each other through X2 or Xn links and connected to the cloud via backhaul links [20]. The set of base stations is denoted as BS={bs1,bs2,bs3,…,bsi}, the set of users is denoted as U={u1,u2,u3,…,uj}, and the cloud server serves as the source server for content. It is assumed that content requests are mutually independent, and each user can request only one content item within a specific time slot, with user locations remaining fixed during the given time slot. The system model comprises four key components: the cache model, transmission delay model, mobility model and fault model. This comprehensive framework is designed to address user demands in high-mobility environments, with the goals of reducing user request latency, improving cache hit rates, and optimizing MEC system performance through fault tolerance mechanisms. Reference to Table 1 is provided for annotations in this paper.

As illustrated in Fig. 1, the system architecture can be divided into three fundamental layers [21]. At the topmost tier is the cloud server layer, consisting of high-performance servers with ample storage capacity. The intermediate layer is the edge server layer, comprising various base stations, each equipped with data storage, caching capabilities, and computational resources. The bottom layer is composed of mobile vehicle users. These base stations are distributed randomly within the simulated area, interconnected through wireless networks, and linked to the cloud servers via backhaul connections. When a user initiates a content request, the request is initially transmitted to the nearby base station. The base station then searches its local cache for the requested content. If the content is available in the local cache, the base station promptly delivers it to the mobile user. In cases where the desired content is not found in the local cache, the base station attempts to request the content from neighboring base stations. Only if adjacent base stations cannot provide the requested content does the base station forward the request to the cloud center [22].

Edge servers face limitations related to storage capacity and computational resources, which necessitate the selective caching of content to ensure quality of service for users [23]. At t, the cache status of edge server bsk can be represented as:

(1)Ck(t)={Πk,1t, Πk,2t,…, Πk,mt}

where, the binary variable Πk,j is used to denote whether content cp is cached in bsk, with Πk,p defined as:

(2)Πk,pt={0,ifcpis cached inbsk1,otherwise 

Considering the constraints imposed by hardware resources and storage capacity, the cache capacity of edge servers should not exceed their total available space. Hence, edge servers must adhere to Eq. (3). This implies that, at t, if content cw is to be cached in bsk, it is necessary for the current server’s capacity to be sufficiently available, as defined by the equation:

(3)∑p=1,p≠wnΠk,pt⋅ξp+ξw≤𝒯

where, ξp represents the size of content cp and Tk represents the cache capacity of bsk.

Transmission delay refers to the time delay experienced by data during its transfer across a network. This delay can be categorized into three primary components: firstly, the delay associated with data transmission within local caches; secondly, the delay arising from collaborative transfers between various edge servers; and lastly, the delay incurred when data is transmitted from the cloud to end users. These three components collaboratively contribute to the reduction of the overall transmission delay [24]. In t, the three components of transmission delay for bsk are defined as Dk,localt, Dk,nighbort, and Dk,cloudt, and their specific mathematical expressions are as follows:

(4)Dk,localt=∑p=1n∑b=1jwbsk,ub,cpt⋅ξp⋅Πk,pt/rlocal (5)Dk,neighbort=∑p=1n∑b=1jwbsk,ub,cpt⋅ξp⋅(1−Πk,pt)⋅ΠNk,pt/rneighbor (6)Dk,cloudt=∑p=1n∑b=1jwbsk,ub,cpt⋅ξp⋅(1−Πk,pt)⋅(1−ΠNk,pt)/rcloud

where, ξp represents the size of content cp, wbsk,ub,cpt represents the content cp requested by user uj from base station bsk at time t, rlocal, rneighbor, rcloud represent the transmission rates from the local, neighbor, and central servers, respectively. Πp indicates whether content cp is cached at the current base station, and ΠNk,pt indicates whether content cp is cached at neighboring nodes.

The total transmission delay is defined as:

(7)Dt=∑k=1iDk,localt+Dk,neighbort+Dk,cloudt

In this assumption, we consider that the mobility of a mobile user exhibits an arbitrary pattern, with the direction and angle of movement varying over time. This implies that the user’s movement trajectory is not constrained to a specific pattern or path and can change dynamically based on various factors or influences. Thus the user’s path is expressed in terms of latitude and longitude as follows:

(8)Lubt={lonub(t),latub(t)}

Lubt denotes the moving trajectory of the bth user in t. Where latub(t) denotes the longitude point of the mobile trajectory of the bth user in t, and lonub(t) denotes the latitude point of the mobile trajectory of the bth user in t.

The failure probability of edge servers is a multifaceted issue that is influenced by various factors. The failure probability is affected by multiple factors, such as the quality of hardware equipment, environmental conditions, operating time, and workload status [25]. This paper assumes that the failure events of edge servers are independent and follow a Poisson process. Consequently, the failure probability of an edge server equipped with base station bsk in a single time slot, denoted as PFk(X), can be further elaborated as:

(9)PFk(X)=(eλt−1)f_kxx!eλtTk0≤PFk(X)≤1x∈{0,1,2,…,n}

where X represents the number of failure occurrences, e is the base of the natural logarithm, λ is the failure rate (the average rate of failure events), fk represents the number of failures in base station bsk, and Tk is the time during which bsk failures occur.

The probability of task processing failure TFk(X) on the edge server bsk, when a user requests resources, can be estimated using queuing theory:

(10)TFk(X)=ρCN(1−ρ)eρtTk

where ρ represents the system’s utilization rate (the ratio of task arrival rate to task processing rate), and CN denotes the capacity of the server.

When a base station fails, it ceases to accept user request tasks. However, if a failure occurs after receiving a user request, resulting in the inability to process the received task, it leads to a delay in task execution:

(11)FTkt=∑k=1iwbsk,ub,cpt⋅ξp⋅dbsk,ubCapk(1+TFk(X))

where Capk the processing capacity of the k-th base station and dbsk,ub denotes the distance of communication between the base station and the b-th user.

The content caching problem involves minimizing content delivery latency in scenarios that consider additional requirements or constraints. Our goal is to minimize latency to the greatest extent possible while meeting the time constraints and capacity limits of each MEC, all the while ensuring the effective delivery of the required content. Consequently, this problem is formally formulated as:

(12)Min⁡1T∑t=1T(Dt+FTk) s.t.C1.\quad∑p=1nΠk,p⋅ ξp≤ 𝒯C2.\quad Πk,p∈{0,1}C3.\quad0<dub≤ dmaxC4.\quadFTk<Tneighbor 

Constraint C1 indicates the size of the content requested by the user cannot exceed the capacity size of the edge server. Constraint C2 is to constrain the non-negativity and integrity of the variable. Constraint C3 indicates the maximum distance of the user request content. Constraint C4 states that the delay caused by a base station failure, leading to the inability to process an individual user request task, must be shorter than the time it takes for a user to successfully send a request to the base station.

To fulfill user requests and minimize content transmission latency, edge servers can proactively predict the content that users are likely to request and cache it locally. While highly popular content is more likely to be requested, it fails to account for users’ personal preferences [26]. Therefore, simply predicting based on popularity is not always realistic. In the context of users’ mobility, accurately predicting content that aligns with their preferences and caching the target content accordingly is a challenging problem that aims to further enhance the QoS for users. Based on this, we propose an adaptive content prediction algorithm (ACP) that takes into account user preferences. The specific algorithm is depicted in Algorithm 1.

Initially, collect the historical records of user content access within time period T. The cumulative number of content accessed by users is denoted as W. Time period T is then divided into o time slots, denoted as {t0,t1,…,to}. For each time tm (0<m<o), calculate the content fitness Pfκtm of user content access. The calculation formula is as follows:

(13)Pfκtm=Rfκ(tm)∑κRfκ(tm)

where, κ represents the category of content accessed by users, fκ represents the content accessed by users during time tm, and Rfκ(tm) indicates the number of requests made by users for content fκ(tm) during time tm. By inputting the collection of content matching degrees {Pfκt0,Pfκt1,…,Pfκtm} into an exponential smoothing prediction model, we can obtain the predicted content matching degree P^fκtm+1 for users in the next time slot. Consequently, the predicted content f^κ(tm+1) for user clusters in that time slot can be determined.

The content feature adaptation degree Sκ represents the degree of match between the predicted content f^κ(tm+1) for users in time tm+1 and their accessed content fκ(tm) in time tm. The calculation formula is as follows:

(14)Sκ=∑κθκ(Chaκ^(tm+1)−Chaκ⁡(tm))2

where θκ represents the weights of different features. Chaκ⁡(tm) denotes the corresponding feature of the accessed content fκ(tm) by users in time tm. Chaκ^(tm+1) represents the corresponding feature of the predicted content f^κ(tm+1) based on the content adaptation degree prediction value P^fκtm+1 for users in tm+1.

Based on the content feature matching, the content adaptation degree Pfκtm+1 for predicting the content of the user cluster at tm+1 is calculated using the following formula:

(15)Pfκtm+1=∑z=0ZSκ,z⋅Pfκ,ztmN

where, Sκ,z represents the content feature matching degree of the κth content category for the zth user cluster; Pfκ,ztm represents the content adaptiveness of the zth user cluster for the κth content category in tm.

DRL is a technique that merges deep learning and reinforcement learning to train intelligent agents [27]. By engaging in ongoing interactions with the environment, these agents learn optimal strategies to make decisions. Multiple DRL agents refer to the presence of multiple independent intelligent agents simultaneously in a reinforcement learning environment [28]. In traditional reinforcement learning, typically only one agent interacts with the environment and learns the optimal strategy. The objective of multiple deep reinforcement learning agents is to learn and optimize their strategies through collaboration or competition, aiming to achieve global optimality or specific goals. Each agent perceives the environmental state, selects actions, and interacts with the environment, continuously improving its strategy through trial and error and learning [29].

In this paper, we use a multi-intelligent deep reinforcement learning framework (Fig. 2), and partially incomplete observable reinforcement learning problems can be modeled as Partially Observable Markov Decision Processes (POMDP) [30] modeled as a six-tuple {S,A,R,P,ζ,δ}. S represents the set of states, A denotes the set of actions, R represents the reward function, and P corresponds to the state transition probability matrix.

P=[P11⋯P1h⋮⋱⋮Ph1⋯Phh]

In this context, ζ refers to the set of observations, while δ represents the set of observation probabilities. The state, action, and reward can be defined as follows:

1) State: at time t, the specific state of the edge cache nodes is denoted as St={S1t,S2t,⋯,SMt}. The set of agents is defined as Agt={ag1t,ag2t,…,agNt}, where N represents the number of agents. Each agent is placed on a specific edge server device and engages in interactions and competitions with other agents to enhance its strategy. Through continuous learning, the agents strive to improve their performance over time.

2) Action: The action set At={ast,aft} represents the decisions made by the agent based on its perception of the environmental state regarding whether to cache content. In this context, ast denotes the action of caching the content, while aft represents the action of discarding the content.

3) Reward: The reward function R(St,At) represents the reward obtained by the agent for selecting a particular action in the current state. The agent utilizes this reward to adjust its exploration strategy. The general procedure involves the agent first interacting with the environment to obtain the state St. Based on the input state St, the agent selects an action At and receives an immediate reward. Finally, the agent accumulates the previous immediate rewards to obtain the current cumulative reward. The primary objective of the reward function is to minimize content delivery latency. When an action meets the requirements, the reward value increases, while it decreases when the action fails to meet the requirements. The reward function is directly tied to the optimization objective and can be defined as follows:

(16)R(St,At)={e−min(1T∑t=1T(Dt+FTk)),ast−1,aft

Algorithm 2, referred to as FT−MAACC, innovatively integrates fault tolerance with multi-agent reinforcement learning to enhance collaboration and learning among agents within distributed environments, while simultaneously increasing the system’s resilience to faults and errors. In traditional multi-agent reinforcement learning settings, failures in one or more agents can significantly impact the overall performance and stability of the system. FT−MAACC addresses this challenge by incorporating fault-tolerance mechanisms that enable the system to adapt and continue learning in the face of such failures. At the heart of this algorithm is the Actor-Critic architecture, which assigns the responsibility of decision-making based on the current state to the Actor, and the task of evaluating the outcomes of actions to the Critic. This division of responsibilities ensures that each agent in the system is equipped with its Actor and Critic, thereby addressing the critical challenge of maintaining system integrity and performance amidst potential agent failures.

Understanding the computational complexity of FT−MAACC is crucial for assessing its applicability across multi-agent systems of varying scales. The complexity is primarily influenced by several key factors, including the number of neurons in each layer of the Actor and Critic networks, the number of layers, and the dimensions of the input and output layers. For instance, the total computational complexity of the Actor-network can be expressed as O(ds×h1,1+∑m=1w1−1h1,m×h1,m+1+h1,w1×ι), where ds represents the size of the input layer, h1,m denotes the number of neurons in the m-th layer, and ι is the dimension of the output layer. Similarly, the Critic network, which is tasked with evaluating the value of selected actions, follows a parallel complexity expression as O(ds×h2,1+∑m=1w2−1h2,m×h2,m+1+h2,w2). When considering n agents within the system, the overall computational complexity becomes n times the sum of the complexities of the Actor and Critic networks for all agents.

Beyond its architectural design, FT−MAACC proactively estimates the probabilities of base station malfunctions and task processing failures to ensure fault tolerance. This assessment enhances the system’s reliability and stability. By understanding the likelihood of base station outages and the risks associated with task processing errors, FT−MAACC can implement targeted strategies to mitigate potential failure scenarios. This approach contributes to reduced content delivery latencies and an improved overall user experience within edge computing environments.

To assess the FT-MAACC method and simulate content requests and user behavior in mobile environments, we integrated mobile user access event logs and mobility trajectories from the Shanghai Telecom dataset [31] with user content preferences from the Movielens 1M [32]. The Shanghai dataset comprises over 7.2 million content access event records and their corresponding mobility trajectories from 9,481 mobile users across 3,233 edge sites over six months, reflecting users’ mobility patterns and behavioral habits. By incorporating rating information from the MovieLens dataset, we attributed specific content preferences to these mobility trajectories, simulating users’ interests in movies or video content at various locations and times. This approach enabled the construction of a comprehensive model that reflects both the realities of the mobile environment and user preferences. Fig. 3 shows the distribution location of edge nodes in Shanghai. Fig. 4 is an example recording the trajectory of a taxi in the city heart of Shanghai. We use Python to implement the proposed FT−MAACC method. The neural network model is built upon a network of evaluated actors and commenters, as well as a network of target actors and commenters. The target network is updated with the Adam optimizer, with learning rates of 0.001 for the critic network and 0.0005 for the actor network, and a discount factor of 0.9. The relevant parameters are shown in Table 2.

FT−MAACC is compared against the following benchmark algorithms: Thompson sampling (TS) [33], Random Selection Algorithm (RSA) [34], Greedy Algorithm (GA) [35], MARL [12], and MAACC (The algorithm in this paper does not consider fault tolerance).

For performance analysis, we configured an environment where each base station was equipped with an edge server. Different base stations cached corresponding contents according to corresponding caching strategies to test the resource requests of mobile vehicles.

As shown in Fig. 5, different kinds of applications are represented by different shapes, such as triangles, prototypes, and squares. Different colors indicate the resources cached by different edge servers. When the vehicle moves, the resource hit rates of MAACC and MARL cache are higher than others, which means the vehicle can obtain resources from the nearest base station. On the contrary, TS/RSA/GA request resources from adjacent edge servers resulting in increased latency. Crossed-out base stations indicate a failure in the system, while dashed lines represent the inability to acquire resources from the faulty base stations. The FT−MAACC algorithm proactively detects base stations with a high probability of failure, selecting the nearest operational base station to provide resources. This approach aims to enhance the hit rate and reduce latency.

Fig. 6 shows the cache hit rates under various caching strategies and cache capacities in a fault-free base station scenario. As the capacity of the edge server increases, the cache hit ratio is observed to increase. By increasing the cache capacity of the edge server, the number of cached resources also increases. Consequently, mobile users are more likely to retrieve resources from local and adjacent edge servers, enhancing overall resource availability and accessibility. This results in a higher hit ratio. Since FT−MAACC solely incorporates fault tolerance mechanisms in comparison to MAACC, the cache hit rates of FT−MAACC and MAACC are identical in the scenario where the base station is free from faults. FT−MAACC and MAACC outperform TS/RSA/GA/MARL by 22.2%/37.1%/15.2%/12.8%, respectively. This is because FT−MAACC and MAACC utilize multi-depth reinforcement learning agents and introduce mechanisms of cooperation and competition, allowing agents to compete, cooperate, or collaborate in their learning process. This promotes mutual learning and interaction among the agents, thereby improving overall performance.

Fig. 7 showcases the cache hit rate considering different cache capacities, various cache policies, and the occurrence of a random node failure. By incorporating fault tolerance mechanisms, FT−MAACC considers the probability and impact of node failures, giving priority to resource requests from base stations that are less susceptible to failures. This approach effectively mitigates cache misses caused by node failures, leading to an improved cache hit rate. The hit rates of other algorithms showed fluctuations primarily due to the inherent randomness of cache misses when user requests need to access cache data located on a failed node. FT−MAACC outperformed MAACC/TS/RSA/GA/MARL by 8.3%/18.4%/37.4%/17%/11.9%, respectively.

Fig. 8 reveals the request latency with different caching strategies under different cache capacities in the scenario where the base station is fault-free. An increase in the cache capacity of an edge server leads to a decrease in resource transfer latency for all cache strategies. This observation suggests a direct relationship between cache capacity and the reduction of latency in resource transfers. This is because the larger the cache capacity, the higher the cache hit ratio of the edge server, the higher the possibility of mobile users obtaining resources from the local and neighboring servers, and the lower the corresponding resource transmission delay. Based on Fig. 8, it is evident that both FT−MAACC and MAACC demonstrate comparable and lower latency compared to other methods. This can be attributed to the adoption of a multi-agent mechanism in FT−MAACC and MAACC, which enhances the exploration capabilities of the training environment by incorporating diverse intelligent agents. Each agent can independently explore the environment and acquire unique strategies. This comprehensive exploration approach enables the system to discover optimal strategies and effectively reduce latency.

Fig. 9 depicts the request latency in the scenario where a random node failure occurs, considering different cache capacities and cache policies. When an edge node experiences a failure, the access latency for users to that node increases. Among the evaluated strategies, FT−MAACC exhibits the lowest latency. However, its latency is higher compared to the scenario where no failures occur. This is primarily due to the time required for detecting and diagnosing faulty nodes. In contrast, the latency of other strategies generally increases and exhibits fluctuations. This can be attributed to the possibility of user requests being directed to the failed nodes, necessitating redirection to alternative available nodes, which introduces additional delays in network communication and forwarding. Moreover, the unavailability of data stored on the faulty nodes necessitates alternative means of data retrieval, further contributing to increased access latency. In summary, FT−MAACC, with its integrated fault tolerance mechanisms, effectively mitigates latency to a certain extent, positioning it as a superior choice compared to other algorithms.