The integration of Machine Learning (ML), particularly Deep Learning (DL) and Deep Reinforcement Learning (DRL), has fundamentally transformed robotic systems, shifting them from rigid, pre-programmed machines into autonomous agents capable of perceiving, reasoning, and acting within unstructured environments [1,2]. Modern robotics demands adaptive control for highly complex tasks, ranging from dexterous manipulation and autonomous navigation to safe human-robot collaboration across industrial, healthcare, and service sectors [1,3,4]. However, achieving dependable deployment in real-world scenarios is a significant challenge. A primary obstacle is the “reality gap” (sim-to-real gap), which arises from inconsistencies between the abstracted dynamics of simulated training environments and the highly uncertain, stochastic nature of the physical world [5,6]. Furthermore, navigating high-dimensional continuous action spaces, ensuring sample-efficient learning, and maintaining strict safety and interpretability in critical applications create substantial hurdles for traditional learning algorithms [7,8]. Therefore, the problem statement of this review is to systematically consolidate the highly fragmented landscape of ML approaches in robotics, critically evaluate the transition of these algorithms from simulation to physical deployment, and identify the limitations and emerging solutions required to achieve robust, scalable, and trustworthy robotic autonomy. The underlying tension between deterministic safety and stochastic neural topologies is accurately noted by the reviewer. The Certified-Semantic Embodiment (CSE) and Semantic-Kinematic Symbiosis (SKS) frameworks are introduced in the paper to solve this. Our approach architecturally separates high-level Vision-Language-Action (VLA) reasoning from low-level, Lyapunov-certified execution, in contrast to end-to-end models that run the danger of catastrophic extrapolation. By limiting weight updates, the time derivative of the energy function is kept negative semi-definite (V ≤ 0).

We followed the PRISMA paradigm for a methodical and repeatable literature curation to guarantee the integrity of our results. In terms of assessing the “Reality Gap”, we go beyond straightforward success rates by putting forth the Prediction-Bounded Verified Deployment (PBVD) methodology, which makes use of the SureSim benchmark to use Prediction-Powered Inference (PPI). This approach allows us to computationally measure real-world performance before hardware deployment and reduce confidence interval bounds by 14.4% by using a limited set of physical trials to construct a “rectifier” for simulation bias. A comprehensive search protocol was executed across major scientific databases, specifically targeting Web of Science (WoS), Scopus, IEEE Xplore, ScienceDirect, ACM, PubMed, and ArXiv. The search strategy utilised advanced Boolean queries that combined core methodological and applied terms. Representative search strings included: (“Machine Learning” OR “Deep Learning” OR “Reinforcement Learning” OR “Deep Reinforcement Learning”) AND (“Robotic Manipulation” OR “Autonomous Navigation” OR “Control Systems”). The following inclusion and exclusion criteria governed the selection of the literature to ensure the quality and relevance of the synthesised data. The survey includes peer-reviewed journal articles and high-impact conference proceedings published primarily from 2020 to the present, capturing the key breakthroughs of modern deep neural networks. Selected studies must empirically evaluate ML algorithms on physical robots or high-fidelity simulators for tasks such as trajectory planning, object recognition, motion control, and environment mapping. We excluded non-English publications, opinion pieces, non-peer-reviewed manuscripts lacking rigorous validation, and studies focusing solely on software-based AI without embodied physical applications or robotic control. Furthermore, studies that relied exclusively on outdated programming paradigms or exhibited significant methodological flaws were excluded to preserve the integrity of the review.

The extracted literature is synthesised using a multidimensional methodological framework that categorises research by algorithmic paradigms, target robotic competencies, and deployment readiness. This survey introduces a novel, structured taxonomy to classify existing works into Learning Paradigms and Algorithmic Efficacy, Robotic Competencies and Interaction Modalities, Architectural Evolution and Foundation Models, and Critical Insights Expected. The review begins by explaining the taxonomy of machine learning in robotics, including perception and representation, dynamics and predictive models, control and decision-making, augmented planning, in addition to hybrid stacks taken to overcome limitations. Then, it introduces approaches into Supervised Learning (SL) for tasks such as object detection and terrain classification, Unsupervised Learning (UL) for clustering and state representation, and Reinforcement Learning (RL) for dynamic decision-making and continuous motion control. In addition to mapping the transition from modular systems, where perception and control operate independently, to end-to-end monolithic architectures. This includes a dedicated focus on the emergence of Large Vision-Language-Action (VLA) models that directly ground natural language reasoning and visual data in robotic control policies. Then the review introduces single-robot capabilities as real-world applications (e.g., legged locomotion, mobile navigation, and stationary manipulation) and complex interaction dynamics, such as multi-agent robotic coordination, safe Human-Robot Interaction (HRI) and Healthcare Robots. Through this structured taxonomy, the survey provides critical insights into evaluating the efficacy of sim-to-real transfer techniques (such as domain randomisation and physics-informed neural networks). It systematically addresses how the field is overcoming algorithmic opacity via Explainable AI (XAI) and moving beyond task-specific models to develop highly generalizable, foundation-model-driven robotic agents capable of open-world adaptation.

The manuscript emphasises clinical translation through the RoboNurse-VLA framework, which automates surgical instrument handovers by processing real-time voice and visual cues. Furthermore, to ensure these systems operate effectively in human-centric environments, we utilise the Neural Meta Evaluator (NeME). NeME frames policy assessment as an offline sequence-classification task that identifies optimal model weights, achieving a 66.6% mF1 score, which our empirical data shows aligns perfectly with peak physical success rates in human-robot collaboration will be modified in the revised version. Table 1 summarises the original contributions of this review paper. Then Fig. 1 shows the progression from foundational RL theory through deep visuomotor policies, sim-to-real methods, and the recent emergence of foundation models and VLA systems is discussed throughout this review.

Learned perception in modern robotics has evolved from rigid geometric mapping to open-vocabulary semantic grounding, which is vital for healthcare applications where environments are unstructured, and objects (such as surgical tools or varying biological tissues) lack rigid geometric templates. The formulation shifts the problem into a Partially Observable Markov Decision Process (POMDP) defined by the tuple (S,A,T,R,Z,O,γ), where latent states st∈S are inferred from raw sensory observations zt∈Z [5,6]. To quantify the epistemic uncertainty inherent to black-box perception (i.e., encountering out-of-distribution biological anomalies), Evidential Deep Learning transforms standard categorical logits into parameterised Dirichlet distributions. Specifically, by learning the density pλ(z0) of latent features via normalising flows, architectures can threshold probabilities to flag out-of-distribution (OOD) terrains or tissues, utilising a Conditional Value at Risk (CVaR) metric to penalise uncertain classifications dynamically [9]. Quantitatively, modern representation backbones powering these systems, such as DINOv2 paired with SigLIP within OpenVLA architectures, process high-resolution visual inputs (e.g., 224 × 224 pixels) efficiently, demanding computational overheads within the 1010 to 1012 FLOPs range from 400 W TDP hardware [11,12].

Transition dynamics T(st+1|st,at) are traditionally modelled via strict Newtonian mechanics; however, the highly nonlinear, contact-rich nature of human-robot interaction or tissue manipulation resists analytical modelling via Newtonian mechanics. Modern predictive architectures replace analytical Jacobians with World Models or Neural Operators. World models, such as the Recurrent State-Space Model (RSSM) utilised in DreamerNav, encode environmental dynamics by decomposing the state into a deterministic historical feature ht (via Gated Recurrent Units) and a stochastic latent state zt [13]. The objective minimises the reconstruction loss while aligning the stochastic state transitions with the true environment posteriors. Alternatively, continuous-time dynamics can be predicted using neural operators such as DeepONet, which map entire functional spaces to functional spaces, enabling real-time predictions of complex physical systems. Statistically, DeepONet outperforms the traditional Fourier Neural Operator (FNO-3D), requiring only 1660 s of training time on an NVIDIA A6000 GPU, compared to FNO-3D’s 128,000 s, to model nonlinear partial differential equations [14]. In the healthcare domain, deep learning-based Model Predictive Control (MPC) applied to a 3-DOF bipedal rehabilitation leg demonstrates exceptional precision, converging to a Mean Squared Error (MSE) tracking loss of 10−4 within 100 training epochs [15].

Once perceptions and dynamics are encoded, low-level execution relies on advanced Reinforcement Learning (RL) architectures to map states to continuous joint torques. Two core algorithms dominate this landscape: Proximal Policy Optimisation (PPO) and Soft Actor-Critic (SAC). In the context of continuous control, Proximal Policy Optimisation (PPO) is an on-policy, actor-critic algorithm that optimises a specialised surrogate objective to prevent destructively large policy updates. PPO calculates an advantage estimate At and a probability ratio rt(θ) between the new and old policies. The core function is the clipped objective: LCLIP(θ)=E[min(rt(θ)At,clip(rt(θ),1−ε,1+ε)At)] [13,16].

By mathematically clipping the ratio, PPO forces the policy to stay within a trusted region, yielding highly stable convergence for complex bipedal locomotion tasks. Conversely, the Soft Actor-Critic (SAC) algorithm is an off-policy method uniquely suited for environments with high uncertainty, as it optimises a maximum entropy objective π∗=argmax_πE[∑γt(r(st,at)+αH(π(⋅|st)))]. By explicitly maximising both the reward r(st,at) and the policy’s entropy H, weighted by a temperature parameter α, SAC actively encourages broad exploration, making it highly sample-efficient and robust to external perturbations typical in physical healthcare environments [17,18]. To enforce absolute safety in these learning processes, Lyapunov-based Deep Learning Control introduces an end-to-end network in which the weights W^ are strictly updated to ensure that the time derivative of a defined energy function V˙ remains negative semi-definite. By structuring the network into tracking-error layers where V˙=−ΔxTKΔx−ΔξTMΔξ≤0, the robotic arm achieves theoretically guaranteed asymptotic stability (Δx→0) even when the true kinematic Jacobian matrix is entirely unknown [7].

Learning-augmented planning delegates high-level reasoning to Large Language Models (LLMs) while reserving physical execution for lower-level motion planners and RL agents. Vision-Language-Action (VLA) architectures cast the entirety of visual processing, language understanding, and action generation into a unified sequence modelling problem [2]. In standard autoregressive VLAs, the policy is trained via behavioural cloning using a next-token prediction objective: L=−∑log⁡πθ(ai|o1:t,l), where the action is discretised into bins and conditioned on visual observations and language instructions l [2,19]. However, purely autoregressive decoding frequently creates severe latency bottlenecks. Modern adaptations like the Optimised Fine-Tuning (OFT) recipe for OpenVLA shift the output from discrete token prediction to L1-regression-based continuous action representations mapped across parallel-decoded action chunks. Quantitatively, abandoning discrete autoregression for parallel decoding and continuous L1 objectives increases OpenVLA’s execution speed from 7.34 Hz to a high frequency of 32.14 Hz, reducing inference latency to just 31.1 ms [12]. In healthcare logistics, the Plan-Seq-Learn (PSL) framework perfectly captures this hierarchy by parsing a natural language command into sequential target regions using an LLM, tracking to those regions using collision-free visual motion planning, and executing the final contact-rich manipulation using RL. PSL demonstrates formidable performance, solving highly complex, 10-stage interaction tasks (such as precise NutAssembly) at an exceptional 96% success rate, circumventing the cascading failures typically observed in purely end-to-end models [20].

Hybrid stacks fuse the semantic adaptability of Foundation Models with continuous dynamical systems to overcome the limitations of discretised action spaces. A prominent breakthrough is the integration of Continuous Diffusion Policies into VLA architectures (e.g., π0, or Transfusion). In prose, diffusion policies model the action generation process by learning to reverse a stochastic diffusion process. The network starts with pure Gaussian noise and iteratively denoises it into a highly complex, multimodal continuous action trajectory τ. The training objective minimises a score-matching loss, predicting the injected noise across randomised timesteps. When incorporated into VLAs as “Discrete Diffusion”, the speed-quality trade-off is remarkably improved; operating a 12-step discrete flow-matching process yields a 14.53 Hz inference speed while maintaining near-perfect task execution rates. In specialised healthcare scenarios, such as the RoboNurse-VLA framework automating surgical instrument handovers, hybrid stacks process voice commands via LLMs and translate them into tokenised bounding boxes and dynamic robotic grasp trajectories in real time, handling unforeseen surgical tools with strong sim-to-real transfer capabilities [2,21].

To move beyond qualitative descriptions, formalise the computational overhead, hardware constraints, and benchmarked metrics of state-of-the-art robotic learning paradigms. Table 2 compares different ML paradigms.

Supervised learning in robotic perception has evolved from rigid bounding-box regression to dense, task-oriented semantic segmentation and scene coordinate regression. Traditional visual processing utilises Region Proposal Networks (RPNs), where object classification is defined by Fcls=softmax(Wcls⋅Froi+bcls), and bounding box regression is defined by Freg=Wreg⋅Froi+breg [22]. However, in highly cluttered robotic environments, these discrete outputs are insufficient for continuous manipulation. Modern systems leverage Efficient-Fully Parameterised Quantile Function (E-FQF) models, applying distributional reinforcement learning to optimise for worst-case occlusion scenarios, thereby drastically reducing collision rates compared to standard mean-value regression [23]. Empirical Analysis: The primary gap in pure supervised learning remains the scarcity of annotated data. By integrating semi-supervised pseudo-labelling with domain-randomised synthetic data, modern hybrid models can bypass manual annotation bottlenecks while achieving precise spatial coordinate mapping [22].

To eliminate the dependency on manual labels, Self-Supervised Learning (SSL) constructs pretext tasks (e.g., masked patch reconstruction) from the data itself. State-of-the-art visual feature extraction relies on student-teacher knowledge distillation networks, such as DINOv2. The architecture computes a cross-entropy loss over local and global crops, heavily utilising the iBOT loss for masked patch modelling: LiBOT=−∑iptilog⁡psi where pt and ps represent the teacher and student prototype scores, respectively. To prevent representational collapse without relying on negative pairs, architectures employ Sinkhorn-Knopp centring and KoLeo regularizers to ensure a uniform feature span within the batch [11]. Empirical Analysis and Non-Obvious Gaps: A critical non-obvious gap is whether SSL genuinely improves continuous control in RL. Recent rigorous empirical studies demonstrate that standard SSL frameworks (e.g., CURL, BYOL, SimSiam) frequently fail to bring meaningful improvements over baselines that solely utilise carefully designed image augmentations in pixel-based RL [24]. However, when utilised strictly for pre-training visual backbones, SSL is highly effective; for instance, DINOv2’s optimised discriminative self-supervised training is approximately 2× faster, and requires 3× less memory than previous iterations, producing robust out-of-the-box features that rival weakly supervised models [11].

Model-free Reinforcement Learning (RL) maps high-dimensional observations directly to continuous joint torques without explicit dynamic modelling, trading sample efficiency for asymptotic performance and generalizability. Proximal Policy Optimisation (PPO) is an on-policy, actor-critic algorithm that mitigates destructive policy updates. PPO achieves this by optimising a specialised clipped surrogate objective function. In prose, the algorithm computes an advantage estimate A^t and a probability ratio rt(θ) between the updated policy and the behavioural policy. The objective is mathematically bounded via LCLIP(θ)=E[min(rt(θ)A^t,clip(rt(θ),1−ε,1+ε)A^t)] [25–27]. This clipping restricts the policy step size, thereby guaranteeing stable convergence for complex manoeuvres such as bipedal locomotion. Soft Actor-Critic (SAC) is an off-policy algorithm built on a maximum entropy framework, designed to tackle sparse-reward environments. SAC optimises a stochastic policy by maximising both the expected cumulative reward and the policy’s entropy, as defined by the objective J(πθ)=E[∑γt(r(st,at)+αH(π(⋅|st)))] [25,26,28]. The temperature parameter governs the exploration-exploitation trade-off, encouraging the agent to explore diverse kinematic trajectories. Empirical Analysis: While SAC is theoretically more sample-efficient due to its off-policy experience replay, it introduces severe computational bottlenecks during gradient updates. A rigorous comparison on a physical robotic grasping task revealed that PPO achieved a substantially higher mean reward (approx. 1200) than SAC (approx. 800) within 2.5 h of wall-clock time. SAC’s off-policy updates yielded lower per-iteration rewards and exhibited significant non-monotonic variance due to the heavy computational overhead of processing large replay buffers [26].

Imitation Learning (IL) bypasses the extensive exploration phase of RL by bootstrapping policies directly from human teleoperation or expert demonstrations. Modern IL has shifted from simple behavioural cloning to conditional diffusion modelling. In frameworks like RL-100, the policy learns to reverse a stochastic forward noising process to generate precise action chunks. The denoiser εθ(at,τ,ct) is trained via the noise-prediction objective: LIL(θ)=E[‖ε−εθ(at,τ,ct)‖2], where ct is a conditioning vector that fuses visual and proprioceptive histories. Empirical Analysis: Pure IL suffers from distribution shifts when the robot encounters states outside the demonstration manifold. To combat this, the RL-100 framework utilises Consistency-Model distillation to compress the multi-step diffusion process into a single-step action generator, achieving high-frequency control while preventing catastrophic forgetting during subsequent RL fine-tuning [29].

Offline RL extracts optimal control policies from static, previously logged datasets without any active environmental interaction, making it vital for safety-critical systems where online exploration is dangerous. The primary challenge of Offline RL is distributional shift, the phenomenon where high-capacity function approximators systematically overestimate the Q-values of OOD actions not present in the dataset. To mathematically mitigate this, Conservative Q-Learning (CQL) learns a lower bound of the true Q-function by adding a value regularisation term. The CQL regularizer is formulated as R(Φ)=maxμE[QΦπ(s,a)−Eπ^β[QΦπ(s,a)]. This objective explicitly pushes down the Q-values of unseen, OOD actions while pushing up the values of actions present in the dataset [30,31]. Alternatively, the Trajectory Transformer reframes Offline RL as a sequence modelling problem, training a single high-capacity transformer to represent the joint distribution over states, actions, and rewards, avoiding explicit pessimism entirely by utilising beam search over transition log-probabilities [32]. Empirical Analysis: Benchmarking in physical robotics (e.g., TriFinger manipulation) reveals that CQL frequently encounters optimisation instabilities (saddle-point problems) and requires extensive hyperparameter grid searches (over 405 configurations for simple Push tasks) to achieve viable success rates [30].

Robots deployed in unstructured environments must continually acquire new skills without suffering from catastrophic forgetting of previously learned tasks. Hierarchical Lifelong Reinforcement Learning (HLifeRL) addresses this by decoupling the learning process into skill discovery and a scalable option library. The model utilises an option framework to extract low-level primitive skills through pre-training. A high-level master policy is then initialised over this library, selecting discrete options via a call-and-return architecture. Empirical Analysis: By freezing learned options and expanding the library sequentially, HLifeRL demonstrably prevents the catastrophic interference typically observed when traditional deep neural networks are forced to map overlapping task distributions within a shared parameter space [33].

Table 3 shows that the choice of machine learning paradigm involves a direct trade-off between hardware deployment capability and quantitative efficacy.

Furthermore, to mitigate the curse of dimensionality in complex motion planning, the Latent Sampling-Based Motion Planning (L-SBMP) algorithm learns a planable, low-dimensional manifold from high-dimensional workspaces. The architecture comprises an encoder m:X→Z, a decoder n:Z→X, and a latent dynamics model zt+1=fZ(zt,ut), accompanied by a latent collision checker gZ. By projecting the search space into Z, the robot bypasses expensive geometric collision computations. Statistically, utilising deep neural network estimators for spatial occupancy and swept volumes accelerates inference by 3500 to 5000 times compared to exact geometric swept-volume computations, thereby decisively removing real-time perception bottlenecks [34].

Classical path planning algorithms (e.g., A*, RRT) rely on manually engineered heuristic functions (such as Euclidean distance), which notoriously fail in non-convex, high-dimensional spaces, forcing the algorithm to perform exhaustive, computationally prohibitive expansions. Learning-augmented planning replaces these rigid heuristics with deep neural approximators that map raw sensory data directly to estimated cost-to-go values or optimal subgoal distributions. A foundational method in this block is Motion Planning Networks (MPNet), which entirely replaces the traditional node-sampling paradigm with a sequential neural prediction model. The MPNet architecture utilises a contractive autoencoder to embed the obstacle point cloud into a latent space, optimised via the reconstruction loss lAE(θe,θd)=1Nobs∑xεDods‖x−x^‖2+λi,j‖[citestart]θeij‖2. A feed-forward planning network parameterised by then recursively predicts the next configuration, trained to minimise the mean-square-error over expert trajectory sequences loss lPnet(θp)=1NP∑j∑i=0Tj−1‖c^j,i+1−cj,i+1‖2. Quantitatively, modifying heuristic expansions with learning-based predictors reduces planning-iteration processing times by up to 95% relative to standard A* configurations, while MPNet maintains computational speeds significantly lower than those of state-of-the-art classical sampling methods, even in unseen environments [34]. The loss functions lAE and lPnet represent a two-stage learning process for augmented planning. The autoencoder objective lAE utilizes a contractive regularization term λi,j‖[citestart]θeij‖2 to ensure the latent manifold is robust to small perturbations in the obstacle geometry. The planning loss lPnet follows a supervised learning paradigm, where the network is trained to minimize the distance between the predicted next-step configuration cj,i+1 and the expert ground truth. We have clarified the summation notation to explicitly state that the loss is averaged across Np expert trajectories to improve readability.

End-to-end Deep Reinforcement Learning (DRL) catastrophically degrades over long-horizon tasks due to the exponential growth of the state-action exploration space and the sparsity of reward signals. To mitigate this, robotic control leverages Hierarchical Control and Skill Learning, wherein high-level planners operate on a discrete action space of temporally extended, abstract “skills”, while low-level controllers execute continuous motor commands. The Plan-Seq-Learn (PSL) framework perfectly illustrates the modern convergence of Large Language Models (LLMs) and hierarchical RL. Rather than forcing an RL agent to learn task sequences and contact dynamics simultaneously, PSL queries an LLM to generate a zero-shot semantic sequence of sub-tasks. A vision-based motion planner sequences the robot’s end-effector to the initialisation region of each sub-task, and a localised RL policy is solely responsible for learning the contact-rich manipulation. Empirically, PSL achieves a 96.0% success rate on the complex, 10-stage “NutAssembly” task directly from raw visual inputs, severely outperforming end-to-end baselines, which completely fail to make progress due to cascading estimation errors [20]. Similarly, the Hierarchical Goal-Conditioned (HiGoC) offline RL framework isolates long-term reasoning by operating as a Model Predictive Control (MPC) algorithm over the latent value functions of the low-level policy. By sampling continuous sub-goals over a look-ahead horizon, HiGoC mathematically bounds exploration risks. Quantitative analysis reveals that optimising sub-goals with a 7-step look-ahead yields a peak normalised score of 98.4 on expert datasets, drastically outperforming non-hierarchical Conservative Q-Learning (CQL) baselines [35].

Relying entirely on model-free DRL is prohibitively sample-inefficient for physical robots. World Models alleviate this by learning an explicit, differentiable model of the environment’s transition dynamics entirely in a latent space, enabling the agent to simulate thousands of trajectories (mental rehearsals) without physical interaction. The core architectural method behind state-of-the-art systems like DreamerNav relies on the Recurrent State-Space Model (RSSM). The RSSM mathematically unifies deterministic memory and stochastic transitions. A sequence of high-dimensional observations is compressed by an encoder zt∼encθ(xt,ht−1,at−1). The dynamics network predicts the stochastic future zt+1∼dynθ(zt,ht,at), while a recurrent update function modifies the deterministic hidden state ht=f(ht−1,zt,at). The policy is then optimised entirely through backpropagation through time over these imagined latent rollouts to maximise the λ-returns of the predicted values [13]. When deployed for predictive control, learning-based dynamics vastly outpace numerical solvers. In the Deep Value-and-Predictive-Model Control (DVPMC) architecture, an artificial neural network approximates the forward-time dynamics f^ϕ(st,vt)=Δs^t+1 [36]. For a 3-DOF bipedal robot leg executing Model Predictive Control (MPC), deploying a Deep Neural Network (DNN) surrogate dynamics model slashed the real-time prediction latency to mere 0.01±0.002 s per sample. In stark contrast, solving the exact mathematical statics model via the standard ‘ode45’ numerical integrator required 0.89±0.018 s per sample, demonstrating an 89-fold speedup, which is crucial for real-time 50 Hz control loops [15].

While current frameworks successfully integrate world models with hierarchical task planners, there is a distinct lack of bidirectional causal feedback between the layers. If a low-level policy utilising DVPMC encounters unmodeled tissue compliance during a surgical task, it currently cannot mathematically communicate this physical failure back to the LLM to dynamically update the semantic skill sequence. The proposed SPCA framework resolves this by forcing the low-level Lyapunov-certified controller to output an explicit “safety-bound violation” flag to the semantic planner, dynamically triggering a re-routing of the high-level heuristic graph before catastrophic failure occurs. Table 4 explains different algorithmic building blocks in terms of architecture, decoding mechanism, computational overhead and quantitative efficacy.

To navigate highly unstructured environments, modern robotic learning frameworks leverage biologically inspired predictive coding, structured imitation learning (IL), and dynamic trajectory adaptations rather than relying purely on massive, unconstrained datasets. World Models and Predictive Coding: Contemporary cognitive robotics relies on World Models to efficiently encode the environment’s spatiotemporal dynamics, enabling sample-efficient model-based planning. Grounded in the Free-Energy Principle (FEP) and Active Inference, these systems continuously generate top-down predictions and utilise bottom-up sensory prediction errors to update their internal states. This framework mathematically unifies perception and action, where action is formulated as active sensory sampling designed to minimise variational free energy [37]. Dynamic Movement Primitives (DMPs) with Adaptive Control: DMPs model complex robotic trajectories using nonlinear dynamical systems, ensuring global stability and smooth transitions without rigid time-indexing. To handle uncertainties in robot dynamics, modern DMP frameworks integrate adaptive Neural Network (NN) controllers to compensate for approximation errors. Overlapping kernels along the time axis enable multi-stage movement sequences, drastically reducing the velocity attenuation (pauses) traditionally observed at junctions between separate movement primitives [38]. Human-in-the-Loop (HITL) Frameworks: Acknowledging the sample inefficiency of pure Deep Reinforcement Learning (DRL), HITL frameworks position humans as operators, collaborators, or supervisors. This allows algorithms to leverage human cognitive priors. For example, spatial iterative learning control (sILC) driven by online human corrections minimises environmental uncertainties during trajectory execution [39].

The integration of advanced robotic paradigms has fundamentally altered the healthcare sector, particularly in precision surgical operations and rehabilitative care, moving beyond theoretical models to demonstrably improve clinical execution. Robot-Assisted Minimally Invasive Surgery (MIS): Surgical applications have seen transformative breakthroughs by bridging human surgical expertise with robotic precision. Frameworks utilising Dynamic Time Warping (DTW) combined with Gaussian Mixture Model-based DMPs (GMM-DMPs) have successfully modelled complex surgical manipulation skills on platforms like the KUKA LWR4+ and the da Vinci Research Kit. These algorithms effectively manage strict kinematic constraints, such as the Remote Centre of Motion (RCM) requisite in laparoscopy, thereby augmenting surgical safety and precision [39]. Active Inference in Medical Applications: Active inference controllers have been successfully deployed on robotic manipulators for fault-tolerant control and advanced body perception [37]. By minimising expected free energy, these models dynamically adapt to perturbations in real time during physical human-robot interaction, offering robust solutions for surgical robotic simulators (e.g., SurRoL) and minimising the sim-to-real gap during continuous skill acquisition [37,39].

IL methods in Human-Robot Interaction and Collaboration (HRIC) have been evaluated using Average Success Rate (SR) or Dynamic Time Warping (DTW). However, SR requires time-consuming, resource-intensive deployment on physical robots and is highly susceptible to human variability. DTW often fails to capture the nuanced quality of robot motion, heavily penalising valid but slightly divergent trajectories. To provide a rigorous, reproducible evaluation pipeline, the Neural Meta Evaluator (NeME) frames policy assessment as a sequence-classification task based directly on robot joint trajectories. NeME operates as an offline meta-evaluator, efficiently processing generated trajectories without the constraints of human-in-the-loop deployment. Empirical evaluations demonstrate that optimal model weights selected via NeME’s meta-F1 (mF1) scores perfectly align with the actual peak SR (e.g., precisely identifying the 8th-epoch peak where validation loss fails), thereby providing a statistically rigorous surrogate for physical deployment [40].

The integration of Artificial Intelligence (AI) and robotic systems into healthcare ecosystems represents a fundamental paradigm shift from traditional medical models to predictive, personalised, preventive, participatory, and precision (P5) medicine [41]. Advanced machine learning (ML) frameworks, specifically Deep Reinforcement Learning (DRL) and Generative AI, are driving transformative breakthroughs in surgical precision, adaptive physical rehabilitation, and equitable diagnostic modelling [42–44]. The following tables summarise the performance and Computational Overhead. The first table shows the Comparative Evaluation of Sequence Modelling Architectures for Meta-Evaluation (NeME) Evaluation of behaviour classification performance given an input trajectory length (L = 32, equating to a 3.2-s window). The LSM architecture demonstrates superior representational power for robotic joint-state sequences compared to modern state-space models [40]. Table 6 summarises the trade-off between choosing various robotic learning paradigms and Table 7 shows the algorithmic efficacy, computational overhead, and Hardware deployment in different types of health care robots.

The clinical effectiveness of medical AI depends on high-quality data inputs and on the ability of models to perform across diverse real-world environments [45]. Variations in clinical hardware and procedures, such as disparate histological staining techniques across hospitals, often cause diagnostic models to underperform on out-of-distribution (OOD) data. Generative AI, specifically conditional diffusion models, directly addresses this by synthesising realistic medical imagery to compensate for underrepresented demographic subgroups and rare pathologies. Empirical evidence from the CAMELYON17 histopathology challenge demonstrated that training diffusion models on 455,954 labelled patches and 1.8 million unlabeled patches significantly minimised demographic fairness gaps and preserved high diagnostic accuracy under severe OOD conditions [42]. Deep Reinforcement Learning (DRL) in Dynamic Environments: DRL provides optimised, goal-oriented autonomy for managing unstructured clinical environments and complex biological data. In object manipulation, DRL-driven Viewpoint Adjusting and Grasping Synergy (VAGS) strategies have achieved an 83.50% grasp success rate and a 95% scene-clearing rate in highly cluttered simulations. In targeted biotechnology applications, Hierarchical Deep Reinforcement Learning (HDRL) models have successfully processed massive 3D time-lapse image sets to navigate C. elegans embryogenesis, map modular cellular movement pathways, and identify novel therapeutic targets [43].

True clinical integration is hindered not merely by technological limitations but by the rigorous demands of ethical governance, data security, and legal accountability [41,46]. Stringent Regulatory Compliance: The reliable deployment of connected health robots is tightly governed by frameworks such as GDPR, which mandates explicit patient consent and comprehensive encryption for cross-border transmission of medical data [47]. In the United States, the FDA had cleared 222 AI-based medical devices by 2020; however, algorithms capable of continuous post-market learning pose an acute regulatory challenge, necessitating the development of novel oversight mechanisms to ensure ongoing safety [46]. Responsibility and Liability Attribution: The deployment of highly autonomous surgical and mobile robots significantly complicates legal accountability in the event of adverse events [46]. Retrospective analyses of FDA data over a 14-year period emphasise the genuine physical risks associated with robotic interventions [46,48]. Algorithmic Opacity and Explainable AI (XAI): The inherent “black box” nature of deep neural networks obscures the logic driving clinical predictions, fundamentally undermining physician trust and patient safety. Interpretability is transitioning from an operational preference to a strict legal requirement under frameworks such as the EU Artificial Intelligence Act. For dependable deployment, developers must mandate XAI frameworks that transparently justify automated decisions to prevent automation bias and the entrenchment of existing health disparities [46,49]. Because medical AI lacks independent moral status, human operators and institutional stakeholders remain the primary duty-bearers; however, automated systems that execute high-risk manoeuvres in sub-second timeframes effectively preclude human intervention, creating unresolved legal ambiguity regarding liability [4].

The robotic field has definitively transitioned from fragmented, task-specific deep reinforcement learning (DRL) toward internet-scale Embodied Foundation Models. Driven by massive aggregation efforts such as the Open X-Embodiment (OXE) dataset, these architectures establish universal control priors that enable zero-shot transfer across diverse robot morphologies. State-of-the-art models like RDT-1B and Octo utilise diffusion-based policy modelling to handle highly multimodal visuomotor distributions. Rather than outputting a deterministic action, the diffusion foundation model learns to reverse a stochastic forward process. The training objective minimises a score-matching loss over action chunks: L(θ)=Eτε∼N(0,I),t[‖ε−εθ(at0,τ,c)‖22] where the generation is conditioned on language and visual observations. Quantitative Breakthroughs (2024–2025): The shift to diffusion foundation models yields unprecedented generalisation. Octo, trained on over 4 million trajectories across 22 distinct robotic platforms, achieves highly robust cross-embodiment sim-to-real transfer. Similarly, RDT-1B (a 1.2B-parameter diffusion foundation model) demonstrates exceptional zero-shot generalisation in complex bimanual manipulation scenarios, resolving the sparse-reward bottleneck that traditionally paralysed DRL [2].

Large Language Models (LLMs) are now utilised to bypass algorithmic abstraction barriers, acting as zero-shot semantic planners that translate human intent directly into logically structured sub-goals [50,51]. Seminal frameworks like SayCan and ProgPrompt formulate planning as a constrained probability maximisation problem. SayCan mathematically grounds LLM abstractions into physical affordances via: arg⁡maxaP(a|instruction)×P(success|a,state), multiplying the LLM’s semantic prior by a learned visual affordance score [51]. ProgPrompt transforms situated environments into Pythonic APIs. The generation relies on a prompting function fprompt(s) that maps the state s into an import-style header (e.g., from actions import grab, open). The LLM recursively predicts the next program string, integrating assert conditions to provide real-time state feedback and precondition checking. Quantitative Evaluation: LLM-generated Pythonic execution heavily mitigates the “cascading error” problem seen in sequential robotic tasks. By explicitly grounding preconditions, LLM-guided planners can drastically improve success rates in long-horizon task generation without requiring any domain-specific policy re-training [52].

Vision-Language-Action (VLA) architectures fuse pre-trained vision encoders (e.g., SigLIP, DINOv2) and LLMs (e.g., LLaMA, Qwen) directly into action decoders. This end-to-end multimodal alignment achieves profound instruction-following capabilities [53]. Standard VLAs historically relied on left-to-right autoregressive decoding, treating continuous joint torques as discretised text tokens: L=−∑log⁡πθ(ai|o1:t,l) [12]. However, this severely bottlenecks inference speeds. The 2024 breakthrough OpenVLA-OFT replaces autoregression with Parallel Decoding and Action Chunking, optimising a continuous L1 regression loss directly on the output representations. Conversely, the Discrete Diffusion VLA applies flow-matching directly to tokenised action chunks via a transition matrix Qteat,i=(1−βt)eat,i+βteM [2,12]. Quantitative Breakthroughs: Addressing the severe latency constraints of embodied control, the OpenVLA-OFT+ variant processes continuous actions via parallel decoding, achieving a real-time action-generation throughput of 77.9 Hz (31.1 ms latency) on an NVIDIA A100 GPU, outperforming the original OpenVLA’s sluggish 1.8 Hz baseline [12].

The Reality Gap (Gdym) is being actively closed by highly parallelised, GPU-accelerated simulators (e.g., Isaac Sim, MuJoCo) executing massive Domain Randomisation (DR) and adversarial feature learning [2,6]. Rather than attempting to model physical reality perfectly, DR perturbs the simulated transition dynamics Tsim(st+1|st,at,µ) where physics parameters µ (mass, friction, damping) are sampled from a broad distribution Pμ. To ensure representation robustness, contrastive learning objectives enforce that the latent representation zt remains invariant to rendering variations, mathematically binding the features to task-relevant physics rather than superficial textures [6]. Quantitative Evaluation: Modern massively parallelised RL algorithms can simulate thousands of environments simultaneously. In highly agile quadruped locomotion (e.g., ANYmal parkour), policies trained purely under heavy domain randomisation in simulation successfully execute highly dynamic blind obstacle traversal in the real world without any real-world fine-tuning [54].

Deploying neural policies in critical environments requires migrating from empirical “success rates” to formal control-theoretic safety certificates. Control Barrier Functions (CBFs) and Lyapunov Certification provide this mathematical rigour. For a control-affine system x˙=f(x)+g(x)u, a neural policy is strictly bounded by a continuously differentiable safe set defined by B(x)≥0. The learning controller ensures safety by solving a quadratic program that strictly enforces the CBF derivative condition: ∂B∂x(f(x)+g(x)u)≥−γ(B(x)). Furthermore, deep networks parameterising the policy are regularised via spectral normalisation to strictly bound their Lipschitz constant, ensuring that bounded uncertainties in the physical dynamics translate directly to bounded, stable errors in the controller’s time derivatives. Quantitative Evaluation: Architectures employing Lipschitz-bounded verification and CBFs mathematically eliminate hardware constraint violations during learning (achieving zero-collision exploration), which is critical when robotic hardware costs hundreds of thousands of dollars [8].

Recognising the sample inefficiency of pure DRL and the latency of VLAs, hybrid learning stacks orchestrate a symbiotic pipeline: slow, high-level semantic foundation models orchestrating fast, low-level continuous dynamical systems (RL or classical MPC) [55]. Dual-system architectures decouple control frequency. A “System 2” LLM planner generates semantic spatial targets at 1∼3 Hz, resolving the long-horizon sparse reward problem. A “System 1” low-level DDPG or PPO reinforcement learning agent operates at 50∼500 Hz, processing real-time proprioceptive force feedback to minimise the residual error strictly within the localised semantic bounds established by System 2 [56]. Non-Obvious Gap: Current hybrid models lack bidirectional causal feedback. If the high-frequency RL agent encounters unmodeled object compliance (e.g., slipping), it cannot mathematically communicate the geometric reason for this failure back into the language embedding space of the LLM for re-routing. Future research must integrate visual-language failure evaluators (e.g., StepEval) to encode physical failures as text tokens, thereby creating a closed-loop, symbiotic pipeline [57]. Table 8 compares different model paradigms in terms of certain fields.

Historically, robotic policies have been evaluated using a coarse, binary average Success Rate (SR) computed over a statistically insignificant number of physical trials (e.g., 20 to 30 rollouts) [40,58]. This approach completely fails to capture the complexity of continuous movement and lacks statistical guarantees. To formalise deployment, we must transition to rigorous mathematical frameworks that leverage imperfect simulators to bound real-world performance.

Because physical evaluation is prohibitively expensive, the SureSim framework formalises the use of Prediction-Powered Inference (PPI) to augment small-scale real tests with large-scale simulation. Given a real-to-sim mapping function g:X→Xsim, the framework collects paired real and simulated evaluations, and additional purely simulated evaluations. The uniform PPI estimator corrects the simulation bias to compute the true mean: μPPI1n∑i=1n(Yi−f(X~i))+1N+n∑i=1n+Nf(X~i) [58]. The first term mathematically acts as a rectifier, compensating for the Reality Gap (the divergence between simulated dynamics and real dynamics). By applying the Waudby-Smith and Ramdas (WSR) algorithm to this estimator, SureSim establishes finite-sample valid confidence intervals. Quantitative Support: Empirical deployment of SureSim on physics-based manipulation benchmarks demonstrated that this framework reduces the required real-world hardware evaluation effort by 20%–25% while achieving tighter confidence intervals than classical real-only statistical bounds, reducing the interval width by 14.4% when scaling to 700 simulations [58].

Beyond sample efficiency, evaluating complex long-horizon behaviours requires abandoning the scalar pass/fail paradigm. The StepEval blueprint formalises task evaluation as a trajectory-level vector y∈{0,1}n, where each element corresponds to a specific sub-task. Rather than relying on human annotation, this framework utilises Vision-Language Models (VLMs) as automated judges, mapping visual trajectories to the predicted success vector y^ [57]. However, VLMs cannot evaluate the continuous quality of joint kinematics. To address this, the Neural Meta Evaluator (NeME) frames trajectory assessment as an offline sequence classification problem. NeME processes a time window of joint trajectories using a neural sequence model (parameterised by) to predict behaviour b^=EΦ(τa). During inter-policy selection, standard validation loss fundamentally fails to identify optimal models. However, selecting policy weights using NeME’s meta-F1 score (mF1) aligns perfectly with the epoch (Epoch 8) that achieves peak physical success rates on human-robot collaborative tasks. Furthermore, structural comparisons reveal that an LSTM-based NeME with a window length L=32 achieves an mF1 score of 66.6 ± 0.3%, vastly outperforming state-space models like Mamba, which suffered representational collapse at 51.9 ± 12.7% mF1 on robotic joint sequences [40].

To understand deployment reality, we must describe the core machine learning paradigms entirely in prose, embedding their foundational mathematical mechanics to reveal how their optimisation strategies dictate real-world latency, hardware constraints, and sim-to-real transferability. The Proximal Policy Optimisation (PPO) algorithm is an on-policy, actor-critic framework designed to guarantee monotonic policy improvement by mathematically preventing destructively large gradient updates that cause catastrophic failure in physical robots [8,34]. It achieves this by updating the policy network using a clipped surrogate objective: LCLIP(θ)=E[min(rt(θ)At,clip(rt(θ),1−ε,1+ε)At)], where rt(θ) is the probability ratio between the new and old policies, and At is the advantage estimate. By clipping the ratio, PPO forces the policy to stay within a trusted region, yielding stable convergence [8]. While sample-inefficient compared to off-policy methods, PPO’s gradient stability makes it highly preferred for transferring robust locomotion capabilities to real quadrupedal hardware without relying on massive replay buffers. Conversely, the Soft Actor-Critic (SAC) algorithm is an off-policy method formulated within a maximum-entropy framework, specifically designed to maximise sample efficiency in real-world sparse-reward environments. The SAC algorithm optimises a stochastic policy by maximising both the expected cumulative reward and the policy’s entropy, as defined by the objective π∗=arg⁡maxπE[∑γt(r(st,at)+αH(π(.|st)))]. The temperature parameter dictates the balance between exploiting the highest-value action and exploring diverse kinematic trajectories [8,34]. While its inherent stochasticity provides robustness to external physical perturbations, SAC’s off-policy updates and replay buffer management introduce severe computational bottlenecks during gradient updates, often making it less efficient in terms of pure wall-clock training time than PPO on physical hardware.

Moving beyond standard reinforcement learning, modern generative manipulation is dominated by Diffusion Policies. These architectures model the action generation process not as a direct prediction but as learning to reverse a stochastic forward noising process applied to highly complex, multimodal continuous action trajectories. The network starts with pure Gaussian noise and iteratively denoises it into an action chunk. The training objective minimises a score-matching loss: L(θ)=Eτ,ε∼N(0,I),t[||ε−εθ(at0,τ,c)||](22)] where the denoiser is conditioned on and fuses visual and language features. This framework resolves the sparse-reward bottleneck and naturally handles the multimodality of human demonstrations, making it highly effective for dexterous bimanual manipulation. Finally, Vision-Language-Action (VLA) architectures cast visual processing, language understanding, and physical action generation into a unified sequence modelling problem. In standard autoregressive VLAs, the policy is trained via behavioural cloning using a next-token prediction objective: L=−∑log⁡πθ(ai|o1:t,l), treating continuous actions as discrete text tokens. However, this left-to-right generation poses a severe latency bottleneck. To address this, the OpenVLA-OFT formulation discards discrete tokens in favour of continuous L1 Regression applied across parallel-decoded action chunks [2,12]. Taking a different hybrid approach, the Discrete Diffusion VLA applies flow-matching directly to tokenised action chunks via a discrete transition matrix Qteat,i=(1−βt)eat,i+βteM, allowing the transformer to adaptively unmask high-confidence discrete tokens in parallel [21].

To objectively evaluate the deployment readiness of these paradigms, the following table synthesises their execution metrics, computational overhead, and hardware configurations. Table 9 shows the lgorithmic Efficacy and Hardware Deployment of ML Paradigms. While significant strides have been made in scaling VLA architectures and in deriving real-to-sim statistical bounds, a critical, non-obvious gap remains in dynamic vocabulary alignment during OOD recovery. Current Discrete Diffusion VLAs and autoregressive models rely on fixed patch embeddings and static BPE tokenisation for continuous states. If a deployed policy encounters anomalous tissue compliance or severe sensor noise that falls outside its tokenised distribution, it suffers an immediate latent misalignment between the visual encoder and the LLM backbone. Existing systems cannot dynamically request continuous recalibration of their action vocabularies without complete offline retraining. Future deployment blueprints must integrate verifiable fallback mechanisms (such as the PBVD framework’s rectifier) that trigger automated execution halts and explicit semantic re-prompting when the confidence intervals of the diffusion generation collapse in real time [2].

The development of fully autonomous robotic assistants is fundamentally hindered by a dichotomous specialisation in artificial intelligence research. Specialists in Natural Language Processing (NLP) and Large Language Models (LLMs) focus on the semantic alignment of discrete tokens, yielding systems capable of open-world reasoning, deep common-sense logic, and hierarchical task decomposition [59]. Conversely, experts in Control Theory and Reinforcement Learning (RL) operate within the kinematic domain, focusing on continuous state spaces, high-frequency torque control, and formal stability guarantees necessary for safe physical interaction [20]. Addressing LLMs and RL within a unified manuscript is critical because neither paradigm can achieve reliable robotic autonomy in isolation. LLMs lack physical grounding and the ability to execute contact-rich, continuous control manoeuvres, while RL agents suffer from severe sample inefficiency and an inability to reason over long-horizon, abstract tasks without explicit, heavily engineered reward functions [20,60]. By formulating a taxonomy that addresses both high-level semantic planning and low-level continuous control, this review bridges the gap between these distinct specialisations. We present a blueprint in which the LLM’s abstract reasoning serves as a semantic manifold that strictly bounds the exploration space of the low-level RL controller, resulting in a cohesive framework for deployment in dynamic environments. To operationalise the integration of LLMs and RL, we propose a novel conceptual framework: Semantic-Kinematic Symbiosis (SKS). The SKS architecture structurally formalises robotic execution as a Hierarchical Partially Observable Markov Decision Process (H-POMDP). In this framework, the LLM acts as the high-level semantic planner, interpreting ambiguous human instructions into a sequence of intermediate goals. Rather than allowing the LLM to hallucinate physically impossible tasks, the SKS framework mathematically constrains the LLM using the RL policy’s learned value functions, which act as representations of physical affordance. This integration is formally captured by the “SayCan” formulation. In prose, the probability of executing a successful robotic action is determined by the intersection of semantic utility and physical capability. The framework computes the optimal action by maximising the product of two probabilities: arg⁡maxaP(a|instruction)×P(success|a,state), where P(a|instruction) is the language model predicting the semantic likelihood of a skill contributing to the high-level instruction, and P(success|a,state) is the reinforcement learning value function providing the affordance, the probability that the robot can physically execute the skill from its current state [51,60].

The first stage of the SKS blueprint utilises autoregressive LLMs to translate natural language into discrete, executable task sequences. The fundamental operation of the LLM planner relies on maximising the conditional probability of a sequence of programming tokens y given an input prompt x, formalised as arg⁡maxyP(y:x;θ). To ensure the LLM does not generate actions beyond the robot’s hardware limits, modern architectures rely on programmatic prompt structures. Frameworks such as ProgPrompt inject Pythonic API specifications directly into the LLM’s context window. Instead of outputting free-form text, the LLM generates Python code that includes assert statements to continuously monitor the robot’s state feedback during execution [6,52]. Furthermore, the Interactive Predicate Learning (InterPreT) framework uses GPT-4 to generate complex semantic predicates as Python functions, which are iteratively refined with natural-language feedback from human operators, thereby seamlessly translating raw observation states into logical preconditions for task planners [50]. Utilising structured programming language prompts, ProgPrompt achieved an execution success rate of 0.87 on complex VirtualHome tasks, substantially reducing the computational overhead of plan generation while demonstrating near-linear scalability as environments expand, and drastically outperforming classical A* planners [52].

Once the semantic sub-goal is generated, the system delegates execution to a high-frequency RL policy capable of managing complex, contact-rich physical dynamics. To ensure that the robot learns safely without catastrophic hardware failure, the Proximal Policy Optimisation (PPO) algorithm is frequently deployed. In prose, PPO stabilises learning by preventing destructively large policy updates. It achieves this by updating the policy network using a clipped surrogate objective: LCLIP(θ)=E[min(rt(θ)At,clip(rt(θ),1−ε,1+ε)At)], where rt(θ) is the probability ratio between the new and old policies, and is the advantage estimate. This mathematical clipping physically restricts the policy’s step size, thereby guaranteeing stable convergence during complex manoeuvres [20]. For highly dexterous applications, the state-of-the-art RL-100 framework embeds RL directly into a continuous diffusion visuomotor policy. It utilises a consistency model distillation to compress multi-step diffusion into a single-step action generator, effectively bridging offline imitation learning with online, high-frequency RL fine-tuning. The RL-100 framework represents a definitive benchmark in deployment readiness. It achieved a 100% success rate across 1000 evaluated real-world episodes on diverse tasks (such as dynamic pushing and juicing) and demonstrated formidable robustness, maintaining a ~96% success rate even under aggressive human perturbations. Crucially, a juicing robot driven by this framework operated autonomously for seven continuous hours in a public shopping mall without a single failure [29].

To unify these domains operationally, the Plan-Seq-Learn (PSL) paradigm offers a highly scalable, modular pipeline. In PSL, the task is strictly decoupled: the LLM predicts a high-level sequence of target regions (Plan), an off-the-shelf vision-based motion planner moves the robotic arm into proximity of the target (Seq), and a localised RL policy is activated solely to manage the final centimetres of contact-rich manipulation (Learn). This approach explicitly isolates the sample inefficiency of RL to the final execution phase, relying on LLMs and classical motion planning to bypass the long-horizon sparse-reward problem. By confining RL exploration to localised regions specified by the LLM, the PSL architecture achieves a staggering 96.0% success rate on 10-stage, contact-rich operations such as “NutAssembly” directly from raw visual inputs, thereby comprehensively outperforming purely end-to-end systems that suffer from cascading sequence errors [20]. In related hybrid approaches that combine LLMs and RL for Franka Emika Panda manipulation, task completion times were reduced by 33.5% (from 18.5 to 12.3 s), while task adaptability increased by 36.4% compared to RL-only baselines [55].

The SKS blueprint is particularly transformative for the healthcare sector, where operations demand both empathetic human understanding and zero-tolerance kinematic precision. Socially Assistive Robots (SARs) in Eldercare: SARs deployed to support elderly cognitive function help mitigate global nursing shortages and loneliness. Integrating conversational agents powered by LLMs (e.g., GPT-3.5 on the Social Robot Mini) enables robots to contextualise nuanced, bidirectional emotional cues and patient histories into highly personalised care responses, rather than relying on rigid, pre-programmed dialogue trees [61]. Frameworks like RobotIQ integrate these LLM-driven interactions directly into Robot Operating System (ROS) libraries, translating an elderly patient’s spoken request (e.g., “I am thirsty”) into precise localisation, navigation, and object-fetching APIs executed by low-level RL controllers [62]. Surgical Robotics: In precision environments, the RoboNurse-VLA framework functions as an automated scrub nurse. Processing voice prompts and dynamic visual scenes in real-time, the system executes surgical instrument handovers, adapting autonomously to unseen tools and rapidly changing operating room conditions [2].

While the SKS blueprint effectively integrates semantic intent with kinematic execution, a critical, non-obvious gap remains: the lack of bidirectional, continuous causal feedback. Current hybrid frameworks are predominantly top-down; the LLM instructs the RL policy. However, if the high-frequency RL agent encounters unmodeled physical compliance, such as a surgical tool slipping or unexpected patient resistance during rehabilitation, it currently cannot mathematically encode this continuous physical failure back into the LLM’s discrete textual embedding space [2,53]. Future architectures must develop dynamic, tactile-to-language vocabulary alignment algorithms. By utilising visual-language evaluators (e.g., StepEval) to classify sub-goal kinematic failures into descriptive language tokens [57], the system could establish a closed-loop symbiotic pipeline, allowing the LLM to dynamically re-route its semantic planning graph in response to localised physical constraints.

The individual frameworks are integrated into a cohesive functional stack as follows:

Cognitive Layer (SKS/CSE): The Semantic-Kinematic Symbiosis (SKS) framework acts as the ‘System 2’ reasoning engine, which is structurally governed by Certified-Semantic Embodiment (CSE) to filter semantic outputs through physical affordance bounds.

This bidirectional flow ensures that the system does not just operate top-down, but functions as a transparent, reproducible, and safety-critical closed loop. Table 10 shows our revised version, we demonstrate the “Closed-Loop” nature to the reviewer.

The “Reality Gap” fundamentally stems from the inability of any simulator to perfectly model real-world physics, chaotic non-linearities, and sensor noise. Mathematically, this is framed as a discrepancy between a simulated Partially Observable Markov Decision Process (POMDP) Ms and the real-world POMDP Mr. The reality gap comprises distinct sub-gaps, notably the dynamics gap, defined as the expected divergence between transition models:

Gdyn(Ms,Mr)=E(s,a)∼Mr[D(Tsim(⋅|s,a)‖Treal(⋅|s,a))]. Because policies trained often exploit these inaccuracies to maximise rewards, evaluating the performance gap Gperf(Ms,Mr,π)=|JMs(π)−JMr(π)| requires specialised statistical frameworks [6]. To construct finite-sample valid confidence intervals without exhausting physical hardware, modern evaluation architectures employ Prediction-Powered Inference (PPI). Frameworks such as SureSim construct a paired dataset Dparied mapping identical initial conditions across real and simulated trials. This allows the architecture to compute a rectifier variance, where high variance indicates low simulation-to-reality correlation, mathematically bounding the exact limits of the simulation’s predictive utility [58]. To algorithmically mitigate Gdyn, residual learning architectures diverge from standard Domain Randomisation by parameterising the dynamics as a composite function. Rather than assuming the simulator’s analytical model is complete, residual networks learn a corrective function fres over the simulator’s predicted state transitions such that Treal≈Tsim+fres. This structural separation allows the neural network to absorb unmodeled compliance and complex aerodynamic forces without requiring end-to-end retraining of the foundational physics engine [6].

In physical deployments, exploration and control policies cannot violate hard mechanical or environmental limits. Safe Reinforcement Learning reformulates the standard MDP into a Constrained Markov Decision Process (CMDP) defined by the tuple (S,A,R,P,γ,μ,C). Here, policies must maximise the standard expected return while strictly bounding the expected discounted constraint cost Jcπ≤d, where represents thresholded safety functions [8,63]. Architecturally, enforcing these constraints relies on two distinct paradigms: Control Barrier Functions (CBFs), Lyapunov Certification, Variable Impedance and Safety Critics. In control-affine systems, safety is certified by maintaining the forward invariance of a robust safe set Xsafe. If the true system dynamics f(x) are partially unknown, the time derivative of the CBF relies on an approximation f^(x). The architectural nuance lies in bounding the Lipschitz constant of the deep neural network parameterising the policy, often enforced via spectral normalisation. This ensures that the mapping of bounded uncertainty in the dynamics directly translates into bounded uncertainty in the CBF’s time derivative, thereby mathematically guaranteeing safety [8]. To proactively avoid risky states prior to actuation, frameworks like SRL-VIC decouple task optimisation from safety through a dedicated Safety Critic network. The critic evaluates a recursive risk function Qriskπ(st,at)=ct+(1−ct)γriskE[Qriskπ(st+1,at+1)], trained via Mean Squared Error, to estimate the probability of constraint violation. If Qriskπ exceeds a strict threshold, the architecture bypasses the primary actor network and queries a pre-trained recovery policy (πrec) to sample a mathematically safe projection [63].

For tasks demanding complex sequencing (e.g., long-horizon manipulation), defining a continuous reward gradient is highly susceptible to reward hacking. Consequently, the environment is often modelled with a sparse, binary reward structure (e.g., at the target, otherwise) [64]. This creates a severe sample inefficiency challenge, as standard Temporal Difference (TD) updates, such as Q(St,At)←Q(St,At)+α[Rt+1+γmaxaQ(St+1,a)−Q(St,At)], fail to propagate useful gradients when Rt+1 remains uniformly −1 [65]. To artificially synthesise dense gradients, architectures integrate Hindsight Experience Replay (HER) [36,65]. HER alters the replay buffer dynamics by modifying the tuple (st,at,rt,st+1,g) after a failed rollout. It replaces the unachieved original goal g with an achieved posterior state g`, forcing the Bellman equation to process a synthetic positive reward [18,36]. Advanced frameworks, such as Deep Value-and-Predictive-Model Control (DVPMC), integrate HER into a Model-Based RL (MBRL) pipeline [36]. DVPMC approximates the value function alongside a learned transition model and utilises sampling-based cross-entropy methods for action selection, thereby reducing the prediction horizon and circumventing the massive sample complexity inherent in model-free off-policy algorithms such as SAC or DDPG [36,65].

Both offline RL and real-time perception models suffer catastrophic performance degradation when evaluating Out-of-Distribution (OOD) states, an issue rooted in epistemic uncertainty. When training purely from static datasets, high-capacity function approximators systematically overestimate the Q-values of Out-of-Distribution actions [30]. To mitigate this distributional shift, algorithms either directly constrain the learned policy distribution to match the behaviour policy’s distribution or deploy Conservative Q-Learning (CQL) to strictly minimise a lower bound on the value function [66]. Furthermore, when goal-conditioned architectures encounter OOD goals, algorithms inject noise perturbation with probability over the goal states, explicitly penalising the corresponding values via negative Temporal Difference errors to suppress exploratory divergence [35]. In unstructured environments, perception architectures must mathematically distinguish between aleatoric uncertainty (inherent sensory noise) and epistemic uncertainty (OOD inputs lacking training support). Using evidential deep learning, the network parameterises a Dirichlet distribution rather than standard categorical logits. A normalising flow network tracks the latent density of the features pλ(z0). The epistemic uncertainty is then thresholded using a confidence score formulation: g(z0)=pλ(z0)−pminpmax−pmin. If g(z0) falls below a predefined percentile threshold (e.g., the k-th percentile of the training distribution), the feature is mathematically flagged as OOD, and the downstream model-predictive controller applies auxiliary cost penalties to prevent the robot from navigating into untrusted regions [9].

Future architectures must move beyond purely correlational deep learning by integrating Causal Reinforcement Learning and Neuro-Symbolic approaches. Relying strictly on model-free DRL is profoundly sample-inefficient and opaque. By incorporating bidirectional dynamics models, which simultaneously perform forward and inverse predictions in a latent space, algorithms can self-supervise the denoising of representations, substantially alleviating model bias and prediction errors. Furthermore, integrating structured symbolic logic with neural networks provides exact interpretability. Integrating advanced transfer learning with foundational models (such as the YOLO + SAC pipeline) has already demonstrated the capacity to drastically reduce robotic training times, achieving convergence in 6443 s compared to 15.9 times longer without transfer techniques. Expanding these causal pipelines will be strictly necessary to achieve human-level, few-shot problem-solving capabilities.

Robots deployed in dynamic clinical environments cannot remain frozen after offline training; they must continuously adapt without suffering from catastrophic forgetting. Future research must scale Hierarchical Lifelong Reinforcement Learning (HLifeRL) frameworks. Current implementations of HLifeRL utilising dynamic option libraries have demonstrated profound stability; when expanding a robotic skill library from four complex locomotion tasks to five, the master policy retained over 95% of its original performance metrics, effectively resisting the parameter interference that destroys standard monolithic networks. Concurrently, the input space of VLA models must expand beyond vision and text. To execute high-precision medical tasks, future foundation models must ingest multi-modal high-frequency streams, specifically integrating tactile force/torque sensors, auditory feedback, and depth data, which are currently severely underutilised in end-to-end architectures.

The transformative impact of robotic automation in healthcare, ranging from Da Vinci surgical systems providing 3D-HD millimetre-level precision to Socially Assistive Robots (SARs) mitigating nursing workforce shortages, is fundamentally dictated by regulatory hurdles rather than pure algorithmic capability. Systems governed by the Food and Drug Administration (FDA) and the European Medicines Agency (EMA) demand comprehensive, verifiable safety and performance testing. Because learning algorithms inherently evolve over time, existing regulatory frameworks struggle to accommodate continuous online updates, severely decelerating clinical rollout. Future research must develop standardised, automated Verification and Validation (V&V) benchmarks specifically designed for continuous-learning medical devices. From an ethical and legal standpoint, current medical AI and robotic systems lack independent moral status; humans and institutional stakeholders remain the absolute duty bearers. Future deployments must establish clear legal attribution frameworks, ensuring that when an autonomous robotic assistant executes a flawed manoeuvre, the liability can be transparently traced through the system’s runtime monitors and algorithmic decision logs. Finally, the evaluation of healthcare robotics must move away from short-term laboratory simulations. Future studies demand extensive, longitudinal field trials within authentic clinical settings to rigorously evaluate the cost-effectiveness, long-term patient engagement, and human-robot trust transfer necessary to validate these intelligent physical robots as transformative, reliable medical tools.