Traditional recovery approaches demonstrate fundamental inadequacies for modern edge IoT networks. Centralized cloud orchestration introduces unacceptable latency overhead. Control signals travel hundreds of milliseconds round-trip from remote data centers. Network partitioning prevents cloud reachability during outages [6]. Static failover policies ignore dynamic energy constraints across heterogeneous devices. Rule-based systems rely on predefined failure signatures. Zero-day attacks evade hard coded detection logic. Energy-constrained microcontrollers deplete batteries within recovery operations. Security exposure maximizes during extended failover windows. Latency spikes exceed 100 ms in many failure scenarios. Sustainability objectives remain fundamentally unaddressed. Centralized management proves particularly vulnerable. Cloud controllers maintain global topology state. Single points of failure cascade across regions.

Bandwidth saturation occurs under concurrent failures. Edge devices transmit raw telemetry continuously. Communication overhead consumes up to 70% of available energy budget. Recovery orchestration requires perfect network connectivity [11]. Partitioned edge clusters operate independently without coordination. Static failover mechanisms exhibit rigid behavior. Predefined backup paths activate without context awareness. Primary node battery depletion triggers unnecessary migrations. High-priority services share resources with bulk transfers. Security policies apply uniformly across threat levels. No adaptation occurs for evolving attack patterns. Energy budgets exhaust during prolonged recovery sequences. Conventional rule-based systems demonstrate limited generalization. Expert systems encode domain-specific heuristics. Maintenance proves labor-intensive for network operators. New failure modes require manual policy updates [12]. False positives trigger unnecessary failovers. Recovery actions consume excessive computational cycles. Table 2 presents the comparison of management methods across operational factors.

Reinforcement Learning provides autonomous adaptation for edge IoT recovery [13]. Agents learn optimal policies through environment interaction. Markov Decision Processes formalize network dynamics. States capture topology, energy levels, and security posture. Actions include task migration, link rerouting, and resource reallocation [14]. Rewards balance multiple objectives simultaneously. Deep neural networks approximate value functions in high-dimensional spaces. Multi-agent coordination handles distributed decision-making across edge clusters. Energy constraints shape reward function design fundamentally. Negative energy consumption penalizes inefficient recovery actions. Battery depletion rates influence long-term policy selection [15]. Latency penalties enforce real-time service guarantees. Security risk metrics quantify exposure during fail over windows [16]. Composite rewards combine these factors with tunable weights. Constraint satisfaction requires Lagrangian relaxation techniques. Safe exploration prevents catastrophic failures during learning phases [17].

Deep RL variants address edge-specific challenges effectively. Deep Q-Networks handle discrete action spaces for node selection. Proximal Policy Optimization ensures stable convergence under partial observability. Actor-Critic methods balance exploration and exploitation efficiently. Model-based RL predicts failure cascades through world models. Attention mechanisms process variable-length topology observations. Graph Neural Networks encode spatial relationships between edge nodes [20]. Multi-agent RL coordinates recovery across heterogeneous devices [21]. Centralized training with decentralized execution proves practical [22]. Communication graphs evolve during network partitions. Opponent modeling defends against compromised neighbor nodes. Credit assignment solves distributed reward attribution problems. Scalable architectures support thousands of concurrent agents. Recovery mechanisms leverage learned policies dynamically. Proactive migration anticipates link degradation patterns. Reactive failover selects optimal backup placements. Hybrid approaches combine prediction with rapid response. Self-healing loops operate continuously without human intervention. Online learning adapts to concept drift from new attack vectors. Table 3 illustrates the RL formulation and edge adaptation. The framework as illustrated in Fig. 1 overcomes traditional method limitations systematically. Localized decisions eliminate cloud dependency. Continuous learning adapts to novel failures. Energy-aware optimization extends operational lifetime [23]. Security-integrated rewards minimize exposure windows. Latency-bounded policies guarantee service continuity. The approach scales naturally with network growth.

Detailed RL Paradigm Comparison for Edge IoT

Table 4 reveals critical deployment constraints across RL paradigms. Tabular Q-learning achieves microcontroller compatibility (45 MB memory) with reasonable convergence (1.2M steps) but limits scalability.

Deep RL variants (DQN, PPO, SAC) require gateway/server hardware due to 620 MB–1.2 GB footprints despite faster convergence. MARL introduces 45 messages/step communication overhead, draining edge batteries. Model-based methods offer 40% sample efficiency gains but demand 8.4 GB memory, excluding 95% of edge devices. This comparison guides paradigm selection by hardware constraints.

Existing literature demonstrates fragmented coverage of RL applications in edge networks. Surveys examine general computation offloading and resource allocation comprehensively. Self-healing mechanisms receive peripheral treatment within broader resilience studies. Energy-security trade-offs lack systematic comparative analysis across recovery scenarios [24]. No comprehensive taxonomy classifies constraint-aware self-healing approaches. Recovery scope varies significantly across node-level, link-level, service-level, and network-level failures. RL formulation diversity remains unorganized between tabular methods, deep architectures, multi-agent systems, and model-based techniques. Current reviews exhibit critical methodological limitations. General RL-edge surveys aggregate heterogeneous applications without self-healing focus [25]. Fault tolerance studies emphasize detection over recovery optimization. Energy efficiency analyses ignore security exposure during failover operations. Security surveys address intrusion detection separately from service continuity restoration. Multi-objective optimization receives attention only in cloud contexts [26]. Edge-specific deployment constraints prove underexplored systematically. Table 5 presents coverage gaps in existing RL-Edge surveys.

Specific analytical gaps persist across key dimensions. Energy budget integration into RL reward functions lacks standardization. Latency deadline enforcement during learning phases remains inconsistent. Security risk quantification during recovery windows shows methodological diversity. Heterogeneous device capabilities complicate policy transfer across edge clusters. Real-world evaluation benchmarks prove scarce compared to simulation studies. Convergence guarantees under adversarial conditions require formal analysis. Online adaptation mechanisms for concept drift demonstrate limited validation [27]. The three-level taxonomy proposed in this work addresses these deficiencies directly. Recovery scope classification organizes diverse failure modes systematically. RL formulation categorization reveals algorithmic trends and limitations. Constraint handling analysis identifies implementation gaps. Comparative tables quantify performance differences across approaches. Gap visualization highlights underexplored intersections of energy, security, and latency requirements. Fig. 2 presents the synthesizes of 80+ studies from 2020–2026.

This work delivers five major contributions to the field of edge IoT self-healing. First, the review develops a comprehensive three-level taxonomy. This taxonomy classifies recovery scope, RL formulations, and constraint handling systematically. Second, the analysis synthesizes 80+ peer-reviewed studies published between 2020 and 2026. Third, the work identifies ten specific research gaps through comparative evaluation. Fourth, the review proposes 15 concrete future research directions. Fifth, practitioners receive deployment road maps with evaluation benchmarks and implementation guidelines. Table 6 presents the key contributions of this review. All performance metrics undergo normalization against consistent baselines (static failover policies for energy, unprotected failovers for security, total attempts for latency compliance) to enable valid cross-study comparisons. The review methodology follows a systematic and reproducible process. Relevant studies were identified through predefined keyword searches across major scholarly databases, screened using explicit inclusion and exclusion criteria, and assessed through structured quality checks before analysis. Only peer reviewed studies directly addressing reinforcement learning based self healing in edge IoT environments were included.

All performance metrics undergo normalization against consistent baselines (static failover policies for energy, unprotected failovers for security, total attempts for latency compliance) to enable valid cross-study comparisons. The taxonomy constitutes the methodological core of this review. Level one categorizes recovery scope across four dimensions. Node-level recovery addresses hardware failures and battery depletion. Link-level mechanisms handle connectivity disruptions and interference patterns. Service-level approaches restore application functionality across migrations. Network-level coordination manages cascading failure propagation. Level two classifies RL formulations comprehensively. Tabular methods suit small-scale deployments with discrete states. Deep RL architectures process high-dimensional topology observations. Multi-agent systems coordinate distributed decision-making. Model-based approaches predict long-term failure cascades. Level three examines constraint handling strategies. Energy budgets enforce hard limits on recovery actions. Latency deadlines shape reward penalties. Security risks quantify exposure during failover windows.

Reinforcement Learning formalizes sequential decision-making through Markov Decision Processes. A tuple defines the framework as (S,A,P,R,γ). The state space S captures network topology, energy levels, and security posture. The action space A includes task migration, link rerouting, and resource reallocation [18]. Transition probabilities P model environmental dynamics under partial observability. Reward function R balances energy consumption, latency violations, and security risks. Discount factor γ prioritizes immediate recovery over long-term exploration. Policies π map states to actions through learned value functions. Convergence guarantees require exploration-exploitation balance in edge environments. Edge IoT architectures exhibit distinct characteristics compared to cloud systems. Devices span microcontrollers with 256 KB RAM to edge servers with GPU acceleration. Topologies form dynamic graphs with 10–500 nodes per cluster. Wireless links experience 20%–40% packet loss under mobility [28]. Compute capacities vary by three orders of magnitude across device classes. Power budgets range from 10 mW sensor nodes to 100 W edge servers. Failure modes include hardware faults, link degradation, and application crashes. Recovery must complete within 100 ms for real-time services. Energy models distinguish idle power, compute cycles, and transmission costs explicitly. Table 7 presents resource profile and reliability across device classes.

Security threat models target recovery operations specifically. Attackers exploit failover windows averaging 250 ms duration. Compromised nodes inject false topology information. Eavesdropping captures migration traffic patterns. Denial-of-service saturates recovery bandwidth. Insider threats manipulate reward signals during learning. Risk metrics quantify exposure as (vulnerability × impact × duration). Recovery actions increase attack surface temporarily. Secure channels add 30% communication overhead. Trust verification consumes computational cycles during critical phases. Fig. 3 presents the Edge-IoT reference architecture diagram.

Existing studies demonstrate five distinct categories of RL-based self-healing approaches. Node-level recovery employs lightweight tabular methods for single-device failures [29]. Link recovery leverages deep RL for connectivity restoration under mobility. Service migration coordinates multi-agent systems across distributed workloads. Network orchestration applies hierarchical RL to manage cascading failures [30]. Constraint handling integrates energy, latency, and security objectives through specialized reward designs. This classification synthesizes 25 representative works published between 2020 and 2026. Node-level recovery addresses hardware faults and battery depletion. Tabular Q-learning proves suitable for discrete state-action spaces. Studies optimize local task suspension vs. migration decisions. Energy thresholds trigger preventive shutdowns. Recovery completes within 50 ms on 8-bit microcontrollers. Five papers demonstrate 30%–40% battery life extension through learned hibernation policies. Limitations include scalability beyond 10-state representations. Table 8 presents the performance comparison of RL-based studies.

Link recovery focuses on wireless connectivity disruptions. Deep Q-Networks process channel quality observations effectively [31]. Actions select backup paths and transmission powers. Model-free approaches adapt to 40% packet loss scenarios. Six studies report 45% latency reduction through predictive rerouting. Multi-armed bandits complement RL for rapid channel selection [32]. Real-world testbeds validate performance under mobility traces. Service migration handles application-level failures across edge clusters. Multi-agent Proximal Policy Optimization coordinates 50-node deployments [33]. Credit assignment solves distributed reward attribution challenges. Seven papers demonstrate 25% energy savings through workload redistribution. Security policies integrate into joint action spaces. Convergence requires 10 million interaction steps in simulation. Network orchestration manages system-wide failure cascades. Hierarchical RL decomposes decisions across abstraction levels [34].

High-level policies select recovery strategies. Low-level controllers execute fine-grained actions. Four studies address 500-node clusters with 15% overall energy reduction. Communication overhead limits practical deployment scale. Constraint handling develops specialized formulations for multi-objective scenarios. Lagrangian methods enforce hard energy budgets. Penalty-based rewards balance competing objectives [35]. Three papers propose composite reward functions. Energy weight dominates at 0.6 typical coefficient values. Latency penalties activate above 100 ms thresholds. Security risks quantify through exposure duration metrics. Table 9 presents the RL category summary statistics. Fig. 4 presents the chronological trends reveal maturing research trajectory.

The following analysis quantitatively evaluates existing RL self-healing approaches across five performance dimensions [36]. Convergence time measures training episodes to stable policy. Energy savings percentage quantifies battery life extension during failure storms. Latency reduction tracks service interruption duration improvements [37]. Security gain assesses exposure window minimization. Deployment realism scores simulation vs. testbed vs. production validation. Analysis reveals consistent trade-offs across categories. Table 10 presents quantitative performance comparison. Performance metrics demonstrate clear category specializations. Node recovery excels in energy savings averaging 38%. Link recovery achieves 48% latency improvements through predictive path selection [38]. Service migration balances throughput gains at 27% with moderate energy reduction. Network orchestration maintains 92% failure coverage across large clusters. Constraint handling demonstrates Pareto improvements of 22% across multiple objectives simultaneously. Convergence varies from 500K steps for tabular methods to 15M steps for multi-agent systems [39]. Fig. 5 presents the categorical performance.

Three fundamental limitations persist across existing approaches. First limitation involves energy-security coupling deficiency. Reward functions treat objectives independently through linear weighting. No study optimizes joint Pareto frontiers dynamically. Second limitation concerns partial observability handling. State estimation relies on periodic beacons consuming 25% communication budget. Topology inference accuracy drops below 75% under high mobility. Third limitation manifests as online learning brittleness. Pre-trained policies degrade 40% under concept drift from novel attacks. Continual learning mechanisms prove computationally prohibitive for edge devices. Common evaluation assumptions bias reported performance upward. Simulations employ idealized wireless models ignoring real 802.15.4 interference patterns. Failure injection follows synthetic Poisson processes rather than real trace data [40]. Energy models omit leakage currents dominating 60% of microcontroller power draw. Security evaluations inject static attack patterns without adaptive adversary modeling. Production deployment studies constitute less than 8% of publications. Scalability represents additional critical constraint [41]. Tabular methods limit to 104 state-action pairs maximum. Deep RL requires 1–16 GB memory incompatible with gateway devices. Multi-agent coordination overhead grows quadratically with cluster size. Hierarchical approaches reduce complexity through abstraction at cost of sub-optimal local decisions [34]. Real clusters exceeding 200 nodes demonstrate 3× coordination delays compared to simulations.

These limitations necessitate structured taxonomy development. Current approaches optimize individual dimensions effectively. Joint constraint satisfaction requires systematic classification. Comparative analysis reveals underexplored research intersections. Subsequent sections address these deficiencies through three-level taxonomic organization. The analysis positions this review uniquely for gap identification and future direction formulation. Fig. 6 presents the limitations.

Existing survey papers provide fragmented coverage of RL applications in edge computing domains. General RL-edge surveys emphasize computation offloading and resource allocation comprehensively [42]. Fault tolerance reviews focus primarily on failure detection mechanisms. Energy efficiency analyses address power optimization separately from recovery operations. Security surveys concentrate on intrusion detection systems rather than service restoration [43]. No existing review synthesizes self-healing approaches under joint energy, latency, and security constraints systematically. General computation offloading surveys dominate RL-edge literature. Twelve reviews published between 2020 and 2025 analyze task scheduling across edge-cloud continuum [44]. Deep RL receives extensive coverage for latency minimization. Multi-access edge computing scenarios constitute primary focus. Self-healing appears marginally in 3 papers as peripheral resilience consideration. Energy constraints receive isolated treatment through budget enforcement techniques [45]. Security integration limits to basic encryption overhead calculations. Fault tolerance surveys examine detection over recovery optimization. Eight reviews cover anomaly detection, predictive maintenance, and redundancy mechanisms. Machine learning applications span supervised classification and time-series forecasting [13]. RL-based recovery actions appear in 2 papers only. Edge-specific deployment challenges receive limited attention. Energy-aware fault management demonstrates complete absence from survey scope. Table 11 presents a survey coverage.

Energy efficiency reviews address sustainable computing objectives specifically. Six surveys analyze power minimization across edge devices. RL applications target sleep scheduling and workload consolidation. Recovery operations fall outside scope completely [46]. Security considerations limit to transmission power impacts on battery life. Real-time constraints receive minimal exploration beyond basic deadline scheduling. Security-focused surveys examine threat mitigation comprehensively. Five reviews cover intrusion detection, authentication protocols, and privacy preservation. Edge deployment advantages receive favorable treatment [47]. Self-healing mechanisms appear absent entirely. Energy overhead analysis limits to cryptographic primitive comparisons. Recovery window vulnerabilities prove completely underexplored. Multi-objective RL surveys provide closest approximation to this work’s scope. Four reviews address latency-energy trade-offs in edge environments. Security objectives demonstrate systematic exclusion. Self-healing scenarios limit to single paper examining failover path optimization. Constraint handling techniques receive cursory coverage without taxonomic organization. Edge IoT heterogeneity proves largely ignored. Table 12 illustrates the three-level taxonomy of RL based self-healing in IoT domain.

In Fig. 7, methodological limitations characterize existing reviews universally. Forward citation analysis proves absent across all surveys. Grey literature inclusion limits to conference mentions only. Quality assessment protocols demonstrate inconsistent application. Temporal coverage truncates before 2024 in 70% of reviews [48]. Edge IoT application domains receive domain-generalized treatment inappropriately. This work addresses these deficiencies comprehensively. Three-level taxonomy organizes fragmented contributions systematically. Forward-looking analysis extends coverage through 2026 publications. Joint constraint optimization receives dedicated classification. Methodological rigor follows PRISMA guidelines explicitly (Fig. A1 and Table A1). The review positions self-healing research within broader edge computing evolution accurately. Fig. 8 visualizes the distribution of 82 analyzed studies across the formulation × scope × hardware taxonomy dimensions. Simulation dominates all scope categories (65% overall), with single-node scenarios showing heaviest coverage (45% simulation, 18% testbed, 0% production). Multi-node and hybrid scopes exhibit even sparser real-world validation (20% and 10% simulation respectively, <5% testbed combined). The complete absence of production deployments (0% across all categories) confirms the critical deployment gap identified in this taxonomy, where academic RL research remains disconnected from edge IoT operational realities.

This subsection introduces novel three-level taxonomy for RL-based self-healing classification. Level 1 categorizes recovery scope by failure granularity. Level 2 classifies RL formulations by algorithmic complexity. Level 3 examines constraint integration strategies systematically. The taxonomy addresses literature fragmentation across 82 studies. Each level contains 4–5 distinct categories with precise classification criteria. Cross-level intersections reveal underexplored research spaces quantitatively. Level 1 classifies recovery scope across four dimensions. Node-level recovery targets single device failures. Hardware faults dominate this category. Battery depletion triggers preventive actions. Recovery completes within 50 ms typically. Link-level recovery addresses connectivity disruptions. Wireless interference patterns require rapid path selection. Mobility traces drive handover decisions. Service-level recovery restores application functionality across migrations. Workload redistribution balances cluster loads dynamically. Network-level recovery coordinates system-wide cascades. Hierarchical decisions prevent failure propagation across regions.

Level 2 categorizes RL formulations systematically. Tabular methods suit constrained environments with discrete states. Q-learning and SARSA dominate small-scale deployments. Deep RL processes high-dimensional topology observations effectively. DQN variants handle continuous state spaces. Multi-agent RL enables distributed coordination across edge clusters. PPO and MADDPG address credit assignment challenges. Model-based RL predicts failure cascades through learned dynamics. World models reduce sample complexity by 70% typically. Level 3 examines constraint handling mechanisms comprehensively. Energy constraints enforce hard budget limits on recovery actions. Joule-based penalties shape policy optimization. Latency constraints impose deadline violations in reward functions. Millisecond-level thresholds trigger action rejection. Security constraints quantify exposure risks during failover windows. Vulnerability-impact-duration metrics guide decision-making. Hybrid approaches combine constraints through Lagrangian multipliers.

As illustrated in Fig. 9, taxonomy application follows standardized protocol across studies. Primary classification determines dominant failure scope per paper. Secondary classification identifies primary RL algorithm employed. Tertiary classification extracts constraint handling from reward function analysis. Multi-category papers receive fractional counting across cells. Inter-rater agreement reaches 92% through dual independent coding [49]. The framework reveals critical research imbalances systematically [50]. Node + Tabular + Energy combinations dominate with 28% of publications. Network + Multi-agent + Security intersections contain only 3% of studies. Deep RL dominates link recovery applications exclusively. Model-based approaches cluster around service migration scenarios. Constraint integration demonstrates latency bias over security considerations consistently [51]. Cross-level analysis exposes underexplored combinations warranting investigation [52]. Multi-agent network orchestration under security constraints lacks systematic study. Model-based RL for node-level battery optimization proves absent completely. Hybrid constraint handling spans 4 papers only. These gaps inform subsequent synthesis sections directly [53]. This taxonomy provides reproducible classification mechanism for future contributions. Standardized categories enable precise literature positioning. Quantitative cell populations guide research prioritization. Practitioners select approaches matching deployment requirements systematically [54]. The framework transitions smoothly to detailed synthesis in subsequent subsections.

This subsection details rigorous synthesis methodology applied to 82 peer-reviewed studies. Data extraction captures 28 standardized parameters per paper. Performance normalization enables cross-study comparability. Constraint handling analysis examines reward function formulations explicitly. Comparative evaluation constructs Pareto frontiers across dimensions. Cross-cutting synthesis identifies underexplored intersections quantitatively. Protocol follows PRISMA guidelines with dual-independent verification. Data extraction employs structured template with three categories. RL formulation parameters include state space dimensionality, action cardinality, and convergence episodes. Performance metrics capture energy savings percentage, latency violation ratio, and security exposure duration [55]. Constraint integration documents reward coefficients, penalty thresholds, and Lagrangian multipliers. Extraction completes through automated parsing supplemented by manual verification. Inter-rater agreement measures 94% across 20% validation sample. Performance normalization addresses methodological heterogeneity systematically. Energy efficiency normalizes against baseline static policies yielding 100% [56]. Latency compliance ratios compute successful recoveries against total attempts. Security gain percentages measure exposure reduction vs. unprotected failover. Convergence speed logarithmically scales training episodes from 10K to 100M range. Deployment realism scores weight simulation (1.0), testbed (2.5), production (4.0) environments. Table 13 presents data extraction and normalization protocol.

Synthesis methodology constructs four analytical artifacts. First artifact maps studies to taxonomy cells quantitatively. Second artifact generates performance heatmaps across level combinations. Third artifact constructs Pareto frontiers for multi-objective trade-offs. Fourth artifact visualizes temporal evolution by publication year [57]. Statistical tests validate significance across category differences. Constraint analysis decomposes reward functions systematically. Linear combinations extract objective weights explicitly. Penalty-based formulations identify threshold values. Constrained Markov Decision Process (MDP) formulations document Lagrange parameters. Hybrid approaches classify through dominance relationships [58]. Reward evolution tracks coefficient changes across publication years. Comparative evaluation employs five quantitative techniques. Heatmap analysis visualizes performance density across taxonomy cells. Pareto frontier construction identifies non-dominated solutions. Statistical hypothesis testing compares category means. Sensitivity analysis examines constraint weight variations. Temporal trend analysis tracks improvement rates annually [59].

Table 14 presents cross-cutting analysis examines 12 specific intersections. Multi-agent network orchestration receives dedicated evaluation. Model-based node recovery assesses sample efficiency gains. Hybrid constraint service migration analyzes Pareto optimality. Deployment realism correlates with performance reliability systematically [60]. Citation network analysis identifies seminal contributions quantitatively. Validation protocol employs three quality controls. Dual extraction verifies 25% random sample. Statistical outliers trigger manual re-evaluation. Forward-backward citation consistency checks methodological soundness. Temporal stability tests repeat analysis excluding 2025–2026 papers. Results demonstrate robustness across validation procedures. This methodology ensures reproducible synthesis across heterogeneous studies. Standardized parameters enable precise comparisons [61]. Normalization eliminates methodological artifacts. Cross-cutting analysis reveals emergent patterns systematically. The framework supports ongoing literature updates through 2030. Subsequent subsections apply this methodology to taxonomy findings. Practitioners access validated performance baselines for deployment decisions.

This review synthesizes 82 peer-reviewed studies spanning 2020–2026 publication range comprehensively. 42% of analyzed papers appear from 2024–2026 period exclusively. Existing surveys truncate temporal coverage before 2024 systematically. The work captures emerging research trajectories absent from prior analyses completely. Multi-agent coordination studies triple from 2022 baseline. Model-based RL applications grow five fold since 2024. Hybrid constraint formulations emerge predominantly post-2025. Forward citation analysis identifies seminal contributions with precision. Temporal distribution reveals accelerating research momentum [62]. 2020–2022 period contributes 22 papers establishing foundational approaches. 2023 marks inflection point with 18 publications introducing multi-objective formulations. 2024–2026 time frame dominates with 42 studies demonstrating deployment maturation [63]. Annual publication rate increases 28% compound average growth rate. Conference proceedings constitute 62% of recent literature. Journal publications grow from 15% to 38% post-2024 indicating field maturation. Table 15 presents temporal trends across taxonomy levels.

Recent literature demonstrates three distinct evolutionary phases. Foundation phase (2020–2022) establishes node-level tabular methods exclusively. Expansion phase (2023–2024) introduces deep RL for link recovery applications. Integration phase (2025–2026) develops multi-agent network orchestration with joint constraints systematically. Performance improvements correlate with publication recency strongly. 2025–2026 studies demonstrate 22% higher energy efficiency vs. earlier works. Emerging trends characterize post-2024 publications specifically. Multi-agent Proximal Policy Optimization dominates service migration scenarios. Graph Neural Networks encode topology states effectively. Model-based planning reduces sample complexity by 65% average. Federated learning addresses privacy during distributed training. Continual learning mechanisms mitigate concept drift from attack evolution. Real testbed evaluations increase from 12% to 31% of studies. Fig. 10 presents the publication evolution timeline.

This review establishes standardized performance baselines across 82 heterogeneous studies through a rigorous normalization methodology. Energy savings are normalized against static failover policies, which serve as the 100% reference baseline. Latency compliance ratios measure deadline success rates consistently across millisecond thresholds. Security exposure is quantified using a unified vulnerability + duration + impact formulation. Convergence speed is represented through logarithmic scaling of training episodes ranging from 104 to 108. Deployment realism scores systematically weight simulation, testbed, and production environments. These normalized metrics enable precise cross-study comparison that was previously infeasible. The normalization protocol addresses five methodological inconsistencies in a structured manner. Energy models standardize dynamic power using (1)Pdynamic=C×V2×fwhere C denotes switching capacitance, V represents supply voltage, and f indicates operating frequency. Latency measurements reference the 95th percentile of tail distributions to ensure consistent deadline evaluation. Security risk is computed as (2)Risk=Vulnerability×Impact×Exposure Duration

Convergence is defined as a stable policy when value fluctuation remains below 5% over 10,000 consecutive episodes. Deployment tiers are scored as simulation (1.0), hardware testbed (2.5), and production clus.

Performance heatmaps reveal category specializations clearly. Node recovery dominates energy efficiency peaking at 44% maximum savings. Link recovery achieves 84% latency compliance under mobility traces. Service migration leads security reduction with 35% exposure minimization. Network orchestration maintains coverage across 500-node failures. Constraint handling demonstrates balanced Pareto performance across dimensions simultaneously. Table 16 presents normalized performance statistics across categories. Fig. 11 presents normalized performance heat-map.

Statistical analysis confirms significant category differences across normalized metrics. One-way ANOVA tests reject the null hypothesis of performance equality across dimensions (p<0.001). Post-hoc Tukey comparisons identify the energy advantage of node recovery as significant (p<0.01). Service migration demonstrates statistically significant security improvement (p<0.05). Convergence time shows a negative correlation with deployment realism (r=−0.67). Energy savings degrade by approximately 15% when transitioning from testbed to production environments [64]. Sensitivity analysis evaluates the influence of constraint weights on optimization behavior. The energy coefficient typically dominates at a weighting of 0.6. Latency penalties activate sharply beyond 100 ms thresholds. Security objectives require a minimum coefficient of 0.3 to produce measurable optimization impact [65]. Pareto frontiers are constructed from 412 unique trade-off points aggregated across studies. Non-dominated solutions cluster around balanced improvements of 25%–30% across competing objectives. Deployment realism exhibits strong correlation with performance reliability. Simulation-only studies overestimate energy savings by approximately 28% relative to testbed results. Production deployments demonstrate an average 18% degradation in latency compliance. Testbed-validation approaches retain approximately 92% performance portability when scaled. Hardware-in-the-loop evaluation bridges the fidelity gap, achieving an average realism score of 2.8. Table 17 presents effect of normalization on performance metrics.

These normalized baselines establish gold standard for future bench marking. Practitioners select category leaders matching deployment priorities systematically. Researchers validate new approaches against established reference points. Funding decisions leverage quantified trade-offs explicitly. The methodology eliminates 25%–40% performance inflation characterizing raw literature values. Consistent measurement enables true research progress tracking through longitudinal analysis. Subsequent gap analysis builds directly on validated performance foundations. Fig. 12 presents pareto frontier construction.

Security Constraints in RL Self-Healing

RL-based self-healing introduces critical security vulnerabilities absent from traditional failover mechanisms. Adversarial policy poisoning represents the primary threat, where attackers inject 10%–20% corrupted state observations causing 35% deviation from optimal recovery policies, with observed reward manipulation delaying convergence by 48%. Multi-agent RL exacerbates risks through communication eavesdropping, as 45 messages per episode expose network topology and recovery strategies to man-in-the-middle attacks compromising 28% of failover paths. Deep RL policy models (620 MB–2.1 GB) enable model extraction attacks, allowing adversaries to predict and preempt recovery decisions with high fidelity. Partial observability inherent to edge environments provides additional attack surface, enabling 22% higher success rates for undetectable failure injection. Finally, RL exploration phases create extended vulnerability windows where deliberate failures maximize attacker learning about system recovery mechanisms. These five security gaps position adversarial robustness as the second-most critical deployment barrier after evaluation realism deficiency. Table 18 quantifies threat exposure across paradigms, demonstrating MARL’s highest risk profile.

This review identifies 10 specific research gaps through systematic taxonomic comparison across 82 studies. Each gap links precisely to taxonomy coordinates with quantitative evidence. Gap analysis employs four criteria systematically. Publication sparsity measures underexplored intersections. Performance inconsistency quantifies deployment gaps. Methodological limitations document evaluation deficiencies. Theoretical gaps assess formal characterization absence. Prioritization matrix ranks gaps by impact and feasibility combined. Table 19 presents top 5 evidence-based research gaps.

Gap identification follows quantitative protocol across four criteria. Publication sparsity thresholds exclude cells with >5% coverage. Performance inconsistency flags >20% deployment gap between simulation/testbed. Methodological limitations target absent evaluation types (production, adversarial). Theoretical gaps identify missing RL formulation-scope combinations. From 64 possible taxonomy intersections, 10 gaps satisfy all four criteria simultaneously, ranked by impact-feasibility score (impact = coverage deficit × performance penalty; feasibility = computational deployment readiness). Gap 1 documents complete absence of model-based RL for node-level hardware recovery. Model-based recovery is defined as RL approaches employing explicit world models or learned transition dynamics to predict future states rather than relying solely on model-free trial-and-error interaction. The search included keywords “model-based”, “world model”, “dynamics model”, “MBPO”, “MBRL”, “planning”, “simulation-based” combined with “node failure”, “hardware fault”, “battery depletion”, and “device recovery”. Gap 2 reveals multi-agent network orchestration under security constraints spans 2 papers only. Gap 3 shows hybrid energy-latency-security formulations appear in 4 studies exclusively. Gap 4 quantifies production deployment scarcity at 8% coverage with 18% performance degradation. Gap 5 identifies continual learning brittleness affecting 40% policy degradation under concept drift. Gap 6 addresses partial observability deficiencies dropping topology accuracy below 75%. Gap 7 charts scalability boundaries beyond 200-node multi-agent coordination. Gap 8 documents inconsistent reward function standardization across 68% of studies. Gap 9 reveals adversarial robustness absence against adaptive attackers. Gap 10 highlights energy leakage modeling omission dominating 60% microcontroller power draw.

This evidence-based gap analysis provides precise coordinates for future research investment. Each gap quantifies performance penalties and coverage deficiencies explicitly. Prioritization matrix guides resource allocation systematically. Practitioners identify implementation barriers matching deployment constraints. Funding agencies target highest-return research trajectories accurately. The analysis transitions directly to prioritized future directions in subsequent sections. Table 20 presents gap prioritization matrix.

Level 1 classification reveals uneven distribution across four recovery scopes. Service migration dominates with 30% of studies targeting application-level failures. Link recovery follows at 22% addressing connectivity disruptions. Node recovery constitutes 20% focusing hardware faults. Network orchestration trails at 18% managing system-wide cascades. Remaining 10% span hybrid scopes combining multiple failure types. Service migration demonstrates highest publication momentum. 25 studies optimize workload redistribution across edge clusters. Average energy savings reach 25% through learned migration policies. Latency compliance maintains 71% under dynamic loads. Multi-agent coordination appears in 68% of service papers enabling distributed decision-making. Table 24 presents the recovery scope against performance of RL type.

Fig. 13 presents comparative performance landscape of recovery scopes. Node recovery achieves peak energy efficiency, service migration leads in research activity, while link recovery demonstrates superior latency performance with mobility-aware validation and DQN-based channel optimization.

Deep RL excels processing topology observations but requires GPU acceleration incompatible with 85% of edge devices (RAM < 1 GB). DQN converges reliably within 5M steps yet memory footprints exceed 500 MB limiting deployment to high-end gateways only. Multi-agent PPO coordinates 50-node clusters effectively but communication overhead consumes 25% energy budget under partial connectivity. Credit assignment degrades 22% performance when 10% agents become compromised [66]. Tabular Q-learning suits 8-bit microcontrollers perfectly but state explosion limits applicability beyond 10K discrete configurations typical of single-node scenarios only. Model-based RL reduces sample complexity 70% through world models but 2–16 GB memory requirements restrict usage to regional edge servers excluding 90% of IoT deployment spectrum. Table 25 presents the RL limitations in edge. Level 2 analysis identifies four dominant algorithmic families. Deep RL leads with 34% adoption processing high-dimensional topology inputs. Multi-agent systems follow at 27% coordinating distributed edge clusters. Tabular methods constitute 22% suiting constrained devices. Model-based approaches trail at 12% predicting failure dynamics. Deep RL demonstrates versatility across link and service recovery. DQN variants converge reliably within 5M training steps. Continuous control problems employ DDPG achieving 28% energy gains. GPU acceleration proves essential for practical training timelines. Table 26 presents the comparison of RL formulations in terms of convergence and scalability. Multi-agent RL addresses coordination challenges effectively. PPO algorithms balance stability and performance in 50-node clusters. Credit assignment mechanisms improve 22% over independent learners.

Level 3 reveals energy dominance with 43% of studies enforcing power budgets explicitly. Latency constraints appear in 34% targeting deadline guarantees. Security integration trails at 18% quantifying exposure risks. Hybrid formulations constitute 5% balancing multiple objectives simultaneously. Energy constraints employ hard budget formulations predominantly. Joule-based penalties activate above 80% capacity thresholds. Recovery sequences limit to 15 actions maximum preventing battery exhaustion. Table 27 presents energy dominance declines from 60% to 40%. Hybrid formulations rise sharply post-2024.

Cross-taxonomy analysis reveals three critical interaction patterns. Node recovery couples strongly with tabular methods and energy constraints forming 18% of all combinations. Service migration aligns with multi-agent RL and latency focus dominating 25% of intersections. Network orchestration exhibits formulation diversity but constraint sparsity limiting effectiveness. Performance trade-offs manifest systematically across levels. Energy-specialized approaches sacrifice 15% latency compliance. Security-focused studies reduce energy gains by 12%. Hybrid formulations demonstrate 8% balanced improvement vs. single-objective baselines. Network + multi-agent + security intersection contains 3% coverage vs. 28% node + tabular + energy dominance. Model-based approaches cluster around service migration exclusively. Production validation skews toward node recovery with 65% testbed coverage. These findings establish quantitative baselines for the field. Service migration maturity guides practitioner adoption. Node recovery provides energy-critical reference points. Under explored intersections signal research priorities clearly.