This study addresses the real-time visual tracking task in edge environments by proposing a robust visual servoing control system based on a higher-order sliding mode observer, enabling a quadrotor UAV to autonomously track a moving soccer ball during outdoor sports broadcasts while relying solely on a monocular camera and an inertial measurement unit, thereby eliminating any dependency on external positioning or velocity sensors such as GPS. The system adopts a hierarchical control architecture in which the observer plays a central role: operating on resource-constrained edge devices, it leverages only visual information to estimate unknown external disturbances and target motion states in real time, significantly reducing both hardware requirements and computational overhead to meet the lightweight deployment demands of edge nodes while simultaneously providing the continuous disturbance estimation necessary to sustain a low-latency control loop. Experimental validation was conducted under simulated real soccer match scenarios featuring wind disturbances and abrupt trajectory changes, and the results demonstrate that the system restores the ball to a stable position near the center of the camera’s field of view within seconds of a perturbation, thereby satisfying the stringent stability and real-time performance requirements of live broadcasting. This work confirms the feasibility of delivering low-latency, highly reliable visual intelligence services on resource-constrained edge platforms through advanced observer design, offering a technical reference for the convergence of real-time perception and control within the edge Internet of Things and providing valuable insights for future research on large-scale edge collaboration and secure, sustainable service architectures.
Image-based visual servoingadaptive controlaerial vehiclesdisturbance observerssoccer ballSchool-Level Fund Project of Guangdong University of Science and TechnologyGKY-2025BSQDW-15Introduction
The increasing demand for immersive viewing experiences in live sports broadcasting—particularly in dynamic football matches—has revealed the inherent limitations of conventional static camera systems [1]. Traditional fixed-angle cameras cannot adequately capture rapid positional transitions, such as counterattacks or long-ball trajectories, leading to fragmented spectator engagement. In contrast, unmanned aerial vehicles (UAVs), with their agile mobility and autonomous tracking capabilities [2–4], provide a disruptive solution capable of delivering cinematic-level perspectives beyond the reach of terrestrial setups. In this context, the visual-based drone tracking control method can effectively address the shortcomings of the traditional live streaming approach. Football tracking represents a fundamental task in drone-assisted live broadcasting, where the primary objective is high-precision trajectory tracking of the football’s motion path. This aligns with the broader control paradigm of visual servoing for unmanned aerial systems [5,6].
Within the domain of robotic vision-guided control, two principal methodologies have emerged: position-based visual servoing (PBVS), which relies on reconstructed 3D target pose, and image-based visual servoing (IBVS), which directly utilizes 2D image features for error regulation [7]. The latter approach circumvents the need for explicit pose estimation, thereby mitigating projection model inaccuracies and camera calibration errors that commonly degrade PBVS performance. In quadrotor systems, traditional depth estimation approaches often lead to increased computational complexity and measurement inaccuracies [8]. To overcome these challenges, alternative solutions have been proposed, including techniques based on spherical image moments [9] and virtual image moments [10]. The virtual image moments approach introduces an auxiliary reference frame that effectively isolates roll and pitch dynamics, consequently streamlining controller development. Experimental validation has confirmed its superior effectiveness in stationary positioning scenarios [11].
While visual servoing methods have demonstrated considerable success in UAV control, significant challenges persist in tracking unknown dynamic targets and mitigating external disturbances [12], particularly limiting their application in dynamic environments like live soccer broadcasting. Current research predominantly assumes perfect knowledge of target dynamics, which proves unrealistic in practice—for instance, the instantaneous velocity of a soccer ball during play cannot be predetermined. Several approaches have attempted to address these limitations: barrier Lyapunov function-based controllers [13] eliminate the need for explicit platform dynamics, while auxiliary dynamic systems [14] handle unknown target velocities by treating them as bounded disturbances, albeit with inherent steady-state tracking errors. More advanced solutions employ nonlinear tracking-differentiator observers [15] to estimate target motion through visual cues, with subsequent developments [16] extending virtual camera IBVS through velocity estimation for improved tracking of arbitrarily oriented planar targets. Nevertheless, these approaches still depend on velocity measurements from the UAV to indirectly determine the moving target’s velocity. The noisy velocity signals propagate through the control system, causing undesirable oscillations in the attitude control loop and increased power expenditure [17].
In practical applications, UAVs conducting aerial live broadcasts over open soccer fields are susceptible to environmental disturbances such as wind, which can significantly compromise tracking accuracy and consequently degrade broadcast quality. To systematically address the challenge of disturbance rejection in quadrotor visual servoing, Xie et al. have developed a series of adaptive control solutions. Their initial work [18] established adaptive control schemes to enhance servoing performance under external disturbances. Building upon this foundation, their subsequent research [19] introduced an advanced output feedback IBVS approach with improved disturbance compensation capabilities. Nevertheless, these methods capable of compensating for external disturbances, though its effectiveness against time-varying disturbances in real-world scenarios remains limited. In work [20], a nonlinear model predictive control strategy was developed through the offline design of tightened constraint sets. Work [21] introduced a saturated control scheme for rotational dynamics to counteract external disturbances. However, model-based control approaches often rely on worst-case assumptions, trading off nominal performance for enhanced robustness [22]. Observer-based methodologies have gained prominence in addressing the critical trade-off triangle of control precision, disturbance attenuation, and closed-loop stability. Work [23] introduced an enhanced extended state observer (ESO) design exhibiting improved disturbance rejection capabilities. Finite-time estimation of external disturbances was accomplished using sliding-mode observers (SMOs) in [24]. To address the high-frequency chattering issue inherent in conventional sliding mode observers, reference [25] pioneered the introduction of a higher-order sliding mode observer (HOSMO) to achieve continuous estimation. Work [26] introduced a higher-order sliding mode observer design that leverages quadrotor position and velocity state feedback to enable both continuous and finite-time disturbance estimation. Recently, work [27] further designed HOSMO and successfully achieved highly accurate synchronous estimation of the relative speed of moving targets and external disturbances. While this method demonstrates effectiveness, its practical implementation critically depends on high-accuracy measurements of both positional and velocity states.
Building upon these research findings, this study focuses on controlling a quadrotor UAV to track a moving soccer ball, addressing needs distinct from conventional fixed-view broadcasting in football matches. In this work, we decompose the UAV control system into two cascaded loops: an attitude loop and a position loop. For the position control loop, we develop a high-order sliding mode observer that utilizes visual information to estimate both external disturbances and the football’s velocity. A backstepping-based controller is then designed to actively compensate for these estimated disturbances and unknown football velocity, enabling accurate tracking of the football’s trajectory. The attitude loop derives desired angular velocities from the position loop’s thrust commands, while a torque controller ensures the UAV’s angular velocities precisely track these reference commands. In this work, the onboard sensing suite comprises only a downward-facing camera and a standard IMU; no external positioning systems or dedicated velocity sensors are required. Compared with existing literature, the main contributions of this work are summarized as follows:
The proposed control architecture incorporates a state observer to reconstruct velocity information, providing a complete solution for vision-based football tracking that operates without physical velocity sensors and relies solely on an IMU and an onboard camera.
The proposed control scheme guarantees asymptotic convergence of tracking errors for UAVs operating under time-varying external disturbances.
The corresponding torque controller is designed based on the full nonlinear quadrotor dynamics rather than employing linearization techniques, thereby significantly enhancing the dynamic responsiveness of the IBVS strategy.
This paper is organized as follows. Section 2 introduces a quadrotor dynamic and a image dynamic. Section 3 presents the controller design and provides Lyapunov-based stability proofs for the closed-loop system. Section 4 presents comprehensive performance evaluation and experimental results of the proposed method, followed by concluding remarks in Section 5.
Problem Formulation
This section formulates the rigid-body dynamic model of a quadrotor UAV in three-dimensional space and establishes a perspective projection model for the onboard camera system. By introducing a virtual camera paradigm, we derive image moment features defined on the virtual imaging plane, followed by a formal problem statement for the visual tracking task.
System Description
To characterize the kinematic relationship between the UAV and the target soccer ball, four coordinate frames are considered for the quadrotor IBVS system description. The inertial frame Fi={OiXiYiZi} is aligned with north, east, down. Fb={ObXbYbZb} represents a body-fixed coordinate system aligned with forward, right, and down axes, with its origin located at the center of mass of the quadrotor. Orign of frame B is located at p=(xyz)T with respect to the Inertia frame, its attitude is represented by three Euler angles ϕ, θ, and ψ, corresponding to roll, pitch, and yaw angles, respectively. The transformation can be represented by a rotation matrix R: Fb→Fi. For the convenience of presentation, it is assumed that the camera frame Fc={OcXcYcZc} aligns with the quadrotor body frame. Consider a virtual camera frame Fv={OvXvYvZv} that inherits the translational motion of the real quadrotor without rotation, with its origin aligned with that of the actual camera frame, and its roll and pitch angles are zero and yaw angle is the same as the actual camera frame. The geometric relationships between the four frames is shown in Fig. 1.
The geometric relationship among the camera frame, virtual image frame, and inertial frame.
Establishing the quadrotor dynamics model in the virtual camera coordinate system are as follows:p˙=v,v˙=−fRe3m+ge3+dfR˙=Rsk(Ω)JΩ˙=τ−Ω×JΩ+dτwhere the mass of the quadrotor m, gravitational acceleration g, and symmetric inertia matrix J define its physical properties. v and Ω respectively represent the linear velocity of the quadrotor within the virtual camera plane and the angular velocity in the body coordinate frame. f and τ represent the thrust force and torque inputs to the quadrotor. Lumped disturbance forces df and torques dτ act on the translational and rotational dynamics of the quadrotor, respectively, arising from both unmodeled dynamics and external disturbances. The unit vector is defined as e3=[001]T, and sk(⋅) denotes the skew-symmetric matrix operator.
To facilitate the derivation of the quadrotor dynamic model and enable disturbance identification and compensation, this paper adopts the following fundamental assumptions:
Assumption 1: The Euler angles of the quadrotor are constrained within the following ranges:ϕ∈[−π2,π2],θ∈[−π2,π2],ψ∈[−π,π].
Assumption 2: The disturbance df and dτ in dynamics are second-order differentiable, with bounded first-order and second-order derivatives.
Image Dynamics
Our target of interest is the motion of a soccer ball, which projects as a circular shape in the camera plane. We therefore derive the image dynamics equation based on the virtual imaging plane using circular image moments. For a soccer ball located at Pc=(X,Y,Z) in the camera coordinate frame, the corresponding image-plane coordinates are given by
pc=[ucnc]=fcZ[XY]with fc>0 is the focal length of the camera.
The soccer ball coordinates in the virtual camera frame are Pv=(Xv,Yv,Zv). Based on the virtual imaging plane defined in the previous section, and utilizing the quadrotor angular measurements directly obtained from the IMU, we project points from the physical imaging plane to the virtual imaging plane through rotation matrix transformation:pv=1r3p¯[r1p¯r2p¯]where pv=[uv,nv]T denote the coordinates of the soccer ball projection in the virtual imaging plane, p¯=[uc,nc,1]T∈R3, the ri∈R1×3 is the ith row of the orientation matrix R, for i=1,2,3.
Then, the circle-based image moments mij of order i+j are defined as
mij=∬uvinvjduvdnvand the image feature vector is defined by
q=[qzxg,qzyg,qz]T
The visual feature parameters are mathematically defined through image moment calculations, where xg=m10a and yg=m01a specify the centroid coordinates of the circular feature in the virtual image plane, with a=m00 representing its observed area. The normalized area metric qz=zda∗a, incorporates the expected feature area a∗ at the desired quadrotor pose, while zd>0 corresponds to the target vertical coordinate in the body-fixed frame when the UAV achieves its reference position. These parameters collectively form the basis for visual servoing control, where (xg,yg) provides positional feedback and qz serves as a scale-invariant feature measurement. The reference features are given by the constant vector qd=[xd,yd,zd], which represents the desired target in the virtual image plane at a desired height zd.
Define the image moment tracking error as
q1=q−qd.
Under the virtual quadrotor rotation-independent assumption and the kinematics, the differential relation P˙v=−(v−vd) holds, where the vd denotes the velocity of the target soccer ball. This, when integrated with the projective mapping (Eq. (5)), generates the visual dynamics as
q˙=−(v−vd).
Let us define the velocity error term as
ev=v−vd.
Then we express the complete dynamics, incorporating both the velocity error dynamics and the image feature dynamics, in the following manner:q1˙=−eve˙v=−fRe3m+ge3+df−v˙d
Remark 1: Based on the perspective projection model, a visual servoing system is constructed using image moment features to establish precise mapping between image space and Cartesian space. This feature design offers two critical advantages: geometric invariance and dynamic decoupling. The virtual image plane maintains parallel alignment with the target plane during UAV attitude variations, while the scaled area feature directly correlates with depth information, thereby forming a complete image feature state representation.
Controller Design
This section first designs a disturbance observer and linear velocity estimator based on quadrotor dynamics and image feature dynamics. Under conditions involving external disturbances and unknown target velocity, a visual feedback controller is derived using the backstepping method. Finally, a torque controller is developed to guarantee convergence of angular tracking errors, with rigorous stability proofs provided through Lyapunov theory. The control block diagram of the system is shown in the Fig. 2.
Control structure.
Disturbance and Velocity Observers Design
The designed hierarchical control framework employs a dual-loop structure: the vision-based outer-loop controller calculates the necessary thrust force F according to visual servoing requirements and generates the target angular velocity command, which functions as the reference input for the inner-loop attitude control system. The inner-loop controller maintains stable quadrotor attitude control by producing accurate torque τ outputs.
Rewrite the velocity error dynamics in a concise form:e˙v=F+DfF=−fRe3m+ge3,Df=df−v˙d
The control objective necessitates trajectory tracking of a moving soccer ball by the quadrotor. Thus, Df represents a lumped disturbance term governing the velocity error dynamics, aggregating parametric uncertainties, exogenous disturbances, and target-induced motion perturbations. Since each component of the lumped disturbance is twice differentiable with bounded second derivatives, a constant A satisfying ||Df||≤A can be deduced from Assumptions 2. In practice, an exact value of A is not required; a sufficiently large conservative estimate can be used to initialize the observer gains, which are then tuned experimentally to achieve a balance between estimation accuracy and robustness to measurement noise.
To facilitate subsequent derivations, for any real-valued vector x∈Rn, we define the element-wise function:sign(x)≜[sign(x1),sign(x2),…,sign(xn)]T,|x|≜[|x1|,|x2|,…,|xn|]T,xa≜[x1a,x2a,…,xna]T.
For each component:sign(xi)={+1if xi>00if xi=0−1if xi<0,∀i=1,2,…,n.
To estimate the external disturbance force Df and velocity error ev, the high-order sliding mode observer in the position loop are designed based on Eqs. (12) and (14) as follows:σ˙1=−σ2+z1,σ˙2=F+σ3+z2,σ˙3=σ4+z3,σ˙4=z4,e^v=σ2,D^f=σ3.
The formulation incorporates four correction terms z1−z4 and an auxiliary parameter α. The auxiliary variable α is initialized to zero for computational convenience. The correction terms are design as
z1=kz1|q1−σ1|34sign(q1−σ1),z2=kz2|z1|23sign(z1),z3=kz3|z2|12sign(z2),z4=kz4|z3|sign(z3),where the kzi,i=1,2,3,4, are some positive constant.
The error terms are given by: σ~1=q1−σ1, σ~2=ev−σ2, σ3~=Df−σ3, σ4~=Df˙−σ4. Then, the error dynamics can be written as
Ξ:{σ~˙1=−kz,1|σ~1|34sign(σ~1)−σ~2,σ~˙2=−kz,2|z1|23sign(z1)+σ~3,σ~˙3=−kz,3|z2|12sign(z2)+σ~4,σ~˙4=−kz,4sign(z3)+D¨f.
With appropriately chosen observer gains
kz,1=5A1/4,kz,2=3A1/2,kz,3=1.5A3/4,kz,4=1.1A.
Based on Lemma 8 in [25], we obtain that the system Ξ will converge to zero in finite time. After the convergence of the errors, D^f and e^v approaches to Df and ev in finite time, respectively.
Remark 2: Owing to the pronounced dynamic coupling inherent in quadrotor systems, the observer-based intermediate input necessitates availability of its first and second time derivatives. Verified herein, D^f produces smooth estimations of Df, ensuring second-order differentiability of control input. As established in [25], the HOSMO inherently exhibits finite-time convergence. This property inherently guarantees satisfaction of the separation principle, enabling decoupled design of controller and observer.
Outer-Loop Controller Design
Based on the image feature errors, the following Lyapunov function is defined:V1=12q1Tq1.
By substituting Eq. (12), the time derivative of V1 is obtained as
V˙1=−q1Tev=−q1T(e^v+e~v)
Let us consider e^v as control input, the desired value e^vd of e^v is set as e^vd=k1q1, with the constant parameter k1>0. While direct assignment of e^v as a control input is infeasible, the backstepping methodology introduces a secondary error variable:q2=q1−1k1e^v.
The incorporation of the new error term modifies the time derivatives of both (12) and (19), resulting in
q˙1=−e~v−k1(q1−q2)V˙1=−q1Te~v−q1Tk1(q1−q2).
Using Eqs. (21) and (16), time derivative of q2 is
q˙2=−e~v−k1(q1−q2)−1k1(z2+F+D^f)
Define a new Lyapunov function
V2=V1+12q2Tq2
By time-differentiating V2 yield
V˙2=−q1Te~v−k1q1T(q1−q2)−q2Te~v−k1q2T(q1−q2)−1k1q2T(z2+F+D^f)
Since the vector F serves as the actual control input, it is designed to satisfy
F=−z2−D^f+k2q2where k2 is a positive constant.
By substituting (26) into (25), V˙2 will be
V˙2=−(q1T+q2T)e~v−q1Tk1q1−q2T(k2k1−k1)q2.
Given the finite-time convergence property of the HOSMO, there exists a finite time constant T>0 such that e~v(t)=0 and D~f(t)=0 for all t≥T. During the transient phase t∈[0,T), the observer estimation errors remain bounded and can be treated as bounded disturbances in the Lyapunov analysis of the controller. By the input-to-state stability (ISS) property of the closed-loop system, these bounded observer errors lead to bounded tracking errors. After the observer converges, i.e., for t≥T, the nominal Lyapunov analysis ensures asymptotic convergence of all tracking errors. This cascaded systems approach rigorously justifies the separation between the observer and controller designs [12].
Specifically, since the velocity estimation error e~v achieves finite-time convergence to zero, there exists a finite time constant t1 such that e~v(t)=0 for all t≥t1. For t≥t1, the closed-loop dynamics of V˙2 is governed by the following:V˙2=−q1Tk1q1−q2T(k2k1−k1)q2.
By choosing the constants satisfying
k1>0,k2k1−k1>0We obtain that V˙2≤0. Analysis of Eq. (24) reveals that V˙2=0 is satisfied if and only if q1=0, q2=0. The designed controller is uniformly asymptotically stable, and the system errors q1, q2 will converge to zero. According to Eq. (20), when the terms q1=q2=0, it follows that ev=0.
Inner-Loop Attitude Controller
In the previous subsection, we derived the control input uf required for the quadrotor to complete the visual servoing task. The objective of the inner-loop attitude controller is to design the control input τ that enables the quadrotor to adjust its attitude angles and track the desired force vector. This section will focus on torque controller design based on unknown parameter and disturbance.
It follows from Assumption 2 that there exist positive constant B such that: ||d˙τ||≤B. A sliding-mode disturbance observer is designed for the external disturbances acting on the quadrotor during rotational motion. Let dτ^ represent the estimated value of dτ, d~τ=d^τ−dτ denote the estimated error, and to observe the system disturbance signals, the following auxiliary variables are designed as
erev=zrev−JΩ,z˙rev=−Ω×JΩ+τ+d^τ−λaeδ1−λbeδ2,where λa and λb are positive constant, 0<δ1<1, δ2>1.
The sliding mode surface and the reaching law are designed to make the error converge to zero in a finite time, which is shown as follows:dτ~=dτ^−dτ=e˙rev+λaeδ1+λbeδ2,d^˙τ=−c1dτ~−c2sign(dτ~).
A new Lyapunov candidate function is defined as
L1=12d~τTd~τ.
The time derivative of L1 yields
L˙1=d~τT(−c1dτ~−c2sign(d~τ)−d˙τ)=−d~τTc1d~τ−c2d~sumτ−d~τTd˙τ≤−d~τTc1d~τ−(c2−B)d~sumτ.
Vector dsumτ=(|dτ1|+|dτ2|+|dτ3|)≥0, dτi (i=1,2,3) represents the i-th component value of vector dτ. With c1>0, c2>B, hence L˙1≤0. The disturbance estimation error is guaranteed to converge to zero. We now design a torque controller for the quadrotor.
Let Ωd=[Ω1dΩ2dΩ3d] denote the desired angular velocity expressed in the body frame. Recall the expression of F in Eq. (15), the control input admits the following reformulation:F=−RfE3m+ge3
By time-differentiating (26), we can obtain the desired attitude matrix Rd=(r1d,r2d,r3d) as
{r1,d=r2,d×r3,dr2,d=r3,d×Ψ‖r3,d×Ψ‖r3,d=−F‖F‖where the Ψ=[cosψd,sinψd,0]T, ψd denotes the desired yaw angle.
For a time-varying Rd, the desired angular velocity and its derivative can be derived as
sk(Ωd)=RdTR˙d,sk(Ω˙d)=sk(Ωd)TRdTR˙d+RdTR¨d.
By solving for F˙ and F¨, the desired angular velocity Ωd and angular acceleration Ω˙d can be obtained.
Note that the intermediary input F designed in Eq. (26) is twice differentiable, ensuring the well-defined nature of Ωd and Ω˙d. However, higher-order derivatives of F cannot be reliably computed through difference methods due to image noise and external disturbances. To address this, we propose employing an HOSM differentiator to estimate F˙ and F¨. Let F^ represent the estimated value of F, define the estimated error F~=F^−F.
x˙1=ka‖F~‖23sign(F~)+x2x˙2=kb‖F~‖13sign(F~)+x3x˙3=kcsign(F~)
The positive constant gains ka>0, kb>0, kc>0 are selected, with the estimator outputs F^=x1, F^˙=x2, and F^¨=x3. As established in [28], finite-time stability guarantees F=F^, F˙=F^˙, and F¨=F^¨ within finite time. Consequently, both the desired angular velocity Ωd and its derivative Ω˙d are obtained.
The angular velocity tracking error is defined as follows:eΩ=Ω−Ωd.
A new Lyapunov function is selected as follows:V3=L1+12eΩTJeΩ.
Differentiating this function yields:V˙3=L˙1+eΩT(−Ω×JΩ+τ+dτ−JΩ˙d)
The torque controller is designed as:τ=Ω×JΩ+JΩ˙d−d^τ−k3eΩ.
Substituting Eq. (41) into (40) yields
V˙3=−d~τTc1d~τ−c2d~sumτ−d~τTd˙τ−eΩTk3eΩ+eΩTd~τ≤−d~τT(c1−12)d~τ−(c2−ξ)d~sumτ−eΩT(k3−12)eΩ
Then, the control parameters c1, c2 and k3 are selected to satisfy the following relationship:{c1>12,c2>B,k3>12.
Substituting Eq. (43) into (42), we obtain that V˙3≤0, the closed-loop system exhibits uniform asymptotic stability, satisfying that limt→∞(eΩ)=0.
Remark 3: The desired angular velocity is derived from Eq. (15). Unlike conventional PD controllers that track Ωd, the proposed torque controller directly incorporates the quadrotor’s dynamic model characteristics. This approach provides more accurate dynamic compensation while addressing model uncertainties and environmental variations, resulting in improved robustness and real-time performance compared to traditional methods. Moreover, since conventional differentiation methods introduce unacceptable errors in computing higher-order derivatives of F, a high-order sliding mode observer is employed to accurately estimate F˙ and F¨, thereby enhancing the controller’s precision.
Experimental Results and Discussion
This section experimentally validates the proposed observer-based IBVS controller under conditions of unknown ball velocity and external disturbances. Two representative disturbance scenarios are investigated: (i) constant disturbances to evaluate steady-state disturbance rejection capability, and (ii) time-varying disturbances to validate robustness under realistic environmental conditions such as wind fluctuations. Correspondingly, two distinct target motion trajectories are considered: (i) a curvilinear-to-linear ground trajectory (xy-plane motion with vz=0), modeling receiving-and-kicking maneuvers; and (ii) a parabolic aerial trajectory (xyz-axis motion), representing long-kicking situations.
Hardware Platform
A comprehensive hardware setup was implemented to experimentally validate the proposed disturbance-observer-based visual control strategy. The platform was designed to meet the specific demands of dynamic soccer ball tracking, emphasizing real-time image processing, robust communication, and precise flight control. The core components are detailed in the following subsections.
For visual perception, a downward-facing global shutter camera was mounted on a vibration-damped plate to ensure clear image capture during flight. The critical task of real-time image processing, feature extraction, and executing the high-order sliding mode observer and backstepping controller was handled by a Raspberry Pi 4B single-board computer, which served as the onboard computational unit. This choice provides a compact, low-power solution capable of meeting the processing demands of the vision-based control system. A Pixhawk flight controller managed the low-level attitude stabilization and motor control. The entire system was integrated through the Robot Operating System (ROS) framework, with the signal architecture illustrated in Fig. 3, ensuring robust and synchronized communication between all components.
Signal architecture of quadrotor control system.
Physical parameters of the quadrotor were set as: mass m=2kg, gravitational acceleration g=9.81m/s2, and moment of inertia J=diag(0.0745,0.0745,0.113)kg⋅m2/rad2. A downward-facing camera with a focal length of 1.8 mm and resolution of 640 × 480 pixels was mounted beneath the quadrotor. The reference image moment and height were specified as qd=[002]T and zd=2. Controller and observer parameters were set as: k1=2, k2=6, c1=4, c2=12, λa=λb=6, δ1=0.5, δ2=2, ka=10, kb=20, kc=30.
Image Processing and Real-Time Implementation Strategy
To ensure high-frequency visual servoing on the resource-constrained embedded platform (Raspberry Pi 4B), it is possible to implement a Region of Interest (ROI) extraction strategy based on spatio-temporal continuity.
Vision Algorithm Pipeline: The algorithm initiates with a global search in the first frame to lock onto the target soccer ball. Once the target is identified, subsequent processing is confined to a dynamic ROI window centered on the feature coordinates from the previous frame. Within this localized window, feature extraction is executed through HSV color space segmentation, morphological filtering, and the Hough Circle Transform. To strike an optimal balance between the field of view (FOV) and computational overhead, the image acquisition resolution is configured at 640 × 480 pixels.
Latency and Update Frequency: The vision processing pipeline is optimized using the OpenCV library on the Raspberry Pi, achieving an average per-frame processing time of approximately 18 ms. Coupled with a camera acquisition rate of 30 FPS, the overall update frequency of the visual control loop is maintained at a consistent 30 Hz.
Communication and Architectural Advantages: Communication between the Raspberry Pi and the Pixhawk flight controller is facilitated via a TTL serial port using the MAVLink protocol at a high baud rate of 921,600 bps. Since all visual computations are executed locally on the onboard edge device, the system entirely circumvents the bandwidth fluctuations and multi-second latencies inherent in traditional wireless image transmission. This edge-computing architecture ensures that the total end-to-end latency is strictly controlled within 50 ms, providing the necessary real-time responsiveness for tracking high-dynamic targets in outdoor environments.
Results and DiscussionCurvilinear-to-Linear Ground Trajectory
The football’s velocity is initially set to v=[−3sin(0.25t),3cos(0.25t)−3,0]⊤ m/s for the first 20 s, then switched to v=[2.8,−1.9,0]⊤ m/s from 20 to 40 s. The system is subject to constant external disturbances including a linear acceleration disturbance df=[0.6,0.5,0.3]⊤ m/s2 and an angular acceleration disturbance dτ=[0.4,0.5,0.6]⊤ rad/s2 throughout the entire duration. This configuration enables evaluation of the system’s dynamic response under changing motion conditions while maintaining persistent disturbance inputs. The initial positions expressed in the initial frame are set to [3,2,3]⊤ m for the unmanned aerial vehicle (UAV) and [0,0,0]⊤ m for the soccer ball. To evaluate the performance advantages of the proposed control scheme, the IBVS method without disturbance compensation was employed as a baseline for comparison. The test results are shown in Figs. 4–7.
The spatial trajectories of the drone and the soccer ball.
The relative position of the quadrotor with respect to the soccer ball.
The state variables of the quadrotor. (<bold>a</bold>) The velocity of quadrotor (The dashed line represents the soccer ball’s velocity, while the solid line indicates the quadrotor velocity). (<bold>b</bold>) The Euler angles of the quadrotor.
Time evolution of the Image errors. (<bold>a</bold>) Time evolution of <inline-formula id="ieqn-173"><mml:math id="mml-ieqn-173"><mml:msub><mml:mi>q</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:math></inline-formula>. (<bold>b</bold>) Time evolution of <inline-formula id="ieqn-174"><mml:math id="mml-ieqn-174"><mml:msub><mml:mi>q</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:math></inline-formula>.
Figs. 4 and 5 demonstrate that the proposed control algorithm achieves stable tracking of a moving soccer ball in the presence of constant external disturbances and unknown ball velocity, delivering satisfactory tracking performance and accuracy, thereby validating our theoretical results. To illustrate the effectiveness of the proposed approach, it is observed that the proposed method (solid orange line) achieves higher fidelity in tracking the soccer ball trajectory (blue line) compared to the baseline method (dashed green line). Starting from the same initial position, the proposed controller rapidly adjusts the UAV’s path to maintain the desired relative pose directly above the moving target. In contrast, the method without disturbance compensation exhibits a significant and persistent offset throughout the flight. As can be seen from Fig. 5, due to the sudden change in the velocity of the soccer ball, a slight chattering occurs in the drone’s relative position at t=20s. However, the amplitude of chattering is very small, and the relative position re-converges to stability within 1.5 s, demonstrating the robustness of the proposed control algorithm. The proposed method (solid lines) successfully drives the relative x and y positions to zero and the z position to the desired altitude of 2.0 m within approximately 4 s. Conversely, the comparison method (dashed lines) fails to eliminate steady-state errors, resulting in constant offsets of approximately 0.5 m in x and 0.8 m in y and a bias in altitude.
As can be observed from Fig. 6a, since the drone is initially far from the soccer ball at t=0s, it needs to accelerate to converge to the desired position directly above the ball. Upon reaching the target position, the drone successfully tracks the ball’s velocity with high precision, maintaining a constant relative position to the moving ball. Fig. 6b demonstrates that when the soccer ball’s velocity changes abruptly, the quadrotor Euler angles exhibit minor oscillations but rapidly regain convergence. This validates both the effectiveness and robustness of our proposed attitude-loop controller. Fig. 7 demonstrates the convergence of image feature errors, which ensures simultaneous convergence of both the UAV’s spatial positioning and velocity tracking errors.
Parabolic Aerial Trajectory
The football’s velocity is initially set to v=[3sin(0.25t),3−cos(0.25t),2sin(0.314t)]⊤ m/s for the first 20 s, then switched to v=[2t,t,0]⊤ m/s from 20 to 25 s. The system is subject to external disturbances including a variational disturbance df=[0.2sint,0.5cost,0.4sin(1.5t)]⊤ m/s2 and an angular acceleration disturbance dτ=[0.4sint,0.5cost,0.6sin2t]⊤ rad/s2 throughout the entire duration. This configuration enables to validate the effectiveness of the proposed algorithm under time-varying disturbances and varying soccer ball depth information. The initial positions expressed in the initial frame are set to [2,4,3]⊤ m for the unmanned aerial vehicle and [0,0,0]⊤ m for the soccer ball. To evaluate the superiority of the proposed HOSMO, we introduced a nonlinear Extended State Observer (ESO) as a baseline, which is a widely recognized benchmark for disturbance estimation. The test results are shown in Figs. 8–12.
The spatial trajectories of the drone and the soccer ball.
The relative position of the quadrotor with respect to the soccer ball.
The state variables of the quadrotor. (<bold>a</bold>) The velocity of quadrotor (The dashed line represents the soccer ball’s velocity, while the solid line indicates the quadrotor velocity). (<bold>b</bold>) The Euler angles of the quadrotor.
Time evolution of the Image errors. (<bold>a</bold>) Time evolution of <inline-formula id="ieqn-183"><mml:math id="mml-ieqn-183"><mml:msub><mml:mi>q</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:math></inline-formula>. (<bold>b</bold>) Time evolution of <inline-formula id="ieqn-184"><mml:math id="mml-ieqn-184"><mml:msub><mml:mi>q</mml:mi><mml:mn>2</mml:mn></mml:msub></mml:math></inline-formula>.
Error convergence situation. (<bold>a</bold>) Position-loop disturbance estimation error (The solid line represents HOSMO, and the dotted line represents ESO). (<bold>b</bold>) Evolution of the Euclidean norm of tracking errors over time.
As shown in Figs. 8 and 9, the proposed control strategy successfully maintains accurate tracking of the motion of the soccer ball despite time-varying external disturbances and variations in ball altitude, demonstrating satisfactory tracking precision. Unlike the conventional ESO, which only provides asymptotic convergence, the proposed HOSMO guarantees finite-time convergence of the estimation errors. This allows the observer to reconstruct the lumped disturbances and target velocities with a significantly superior convergence rate.
Fig. 9 reveals a transient response at t=20s following the velocity of the ball discontinuity, where the drone exhibits minor positional chattering. The system demonstrates rapid recovery, achieving restabilization within 2.5s-evidence of the disturbance rejection capability of the controller. Fig. 10a demonstrates the drone’s convergence from an initial offset position at t=0s, achieving precise velocity tracking and stable relative positioning above the moving ball. Fig. 10b confirms the attitude controller’s robustness through Euler angle dynamics during velocity transitions, with transient oscillations dampening within 1s. Notably, Fig. 11 establishes the image feature error convergence, which guarantees simultaneous stabilization of both spatial positioning and velocity tracking errors through the visual servoing framework.
As shown in Fig. 12a, the disturbance estimation errors of the HOSMO (solid lines) exhibit a significantly faster convergence rate compared to the ESO (dotted lines). At the initial phase and during the disturbance occurrence at t=20s, the HOSMO-based observer demonstrates its finite-time stability by reconstructing the unknown perturbations almost instantaneously. Fig. 12b illustrates the evolution of the Euclidean norm of tracking errors ||ep||. The proposed method (solid blue line) achieves a rapid settling time of approximately 4 s, while the ESO-based method (dashed purple line) requires nearly 7 s to reach a comparable steady state.
Conclusion and Discussion
This research delivers a robust vision-based solution for quadrotor tracking of soccer balls during high-dynamic matches, directly addressing sports filming challenges. The virtual camera projection enables consistent ball imaging regardless of drone attitude changes, while moment features decouple depth and position sensing—ensuring the target remains centered during free kicks or aerial passes. The integrated control architecture compensates for unpredictable ball motions and environmental disturbances through finite-time disturbance observers and adaptive backstepping, achieving sub-meter tracking precision (as quantified in Table 1) even in parabolic flight tests with time-varying wind interference. Crucially, the system operates using only desired flight height and yaw orientation, simplifying deployment in stadium environments.
Quantitative comparison of tracking performance (Unit: m).
RMSE
Proposed Method
Comparison Method
Experiment 1
Experiment 2
Experiment 1
Experiment 2
x (m)
0.0787
0.1780
0.5864
0.2274
y (m)
0.0501
0.3876
0.8068
0.7586
z (m)
0.0008
0.0035
0.4977
0.0631
Future work will focus on computational optimization for onboard vision systems and multi-drone cooperative filming strategies to expand coverage for large-scale sporting events. Additionally, a comparative evaluation of resource utilization (e.g., memory footprint, power consumption) across different edge platforms will be conducted to further validate the lightweight deployment capability.
Not applicable.
Funding Statement
This research was supported by the School-Level Fund Project of Guangdong University of Science and Technology (No. GKY-2025BSQDW-15).
Author Contributions
The authors confirm contribution to the paper as follows: Conceptualization, Yao-Bo Long; methodology, Yao-Bo Long, Yu-Ke Ouyang; software, Yu-Ke Ouyang; validation, Yu-Ke Ouyang, Bo Zhuang, Ao-Qi Liu; formal analysis, Yao-Bo Long, Yu-Ke Ouyang; investigation, Yao-Bo Long; resources, Yao-Bo Long; data curation, Yao-Bo Long; writing—original draft preparation, Yu-Ke Ouyang; writing—review and editing, Ao-Qi Liu; visualization, Bo Zhuang; supervision, Ao-Qi Liu; project administration, Yao-Bo Long. All authors reviewed and approved the final version of the manuscript.
Availability of Data and Materials
Data available within the article.
Ethics Approval
Not applicable.
Conflicts of Interest
The authors declare no conflicts of interest.
ReferencesDingR, ZhuY, FengX, ChaiB, JiR. A multi-objective assignment algorithm for UAVs in multiple task with different workloads. In: Proceedings of the 2023 Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC); 2023 Apr 14–16; Dalian, China. p. 354–9.HulensD, GoedeméT. Autonomous flying cameraman with embedded person detection and tracking while applying cinematographic rules. In: Proceedings of the 2017 14th Conference on Computer and Robot Vision (CRV); 2017 May 16–19; Edmonton, AB, Canada. p. 56–63.PueyoP, DendarietaJ, MontijanoE, MurilloA, SchwagerM. CineMPC: a fully autonomous drone cinematography system incorporating zoom, focus, pose, and scene composition. IEEE Trans Robot. 2024;40:1740–57.MademlisI, MygdalisV, NikolaidisN, MontagnuoloM, NegroF, MessinaA, et al. High-level multiple-UAV cinematography tools for covering outdoor events. IEEE Trans Broadcast. 2019;65(3):627–35. doi:10.1109/tbc.2019.2892585.LinJ, MiaoZ, WangY, WangH, WangX, FierroR. Vision-based safety-critical landing control of quadrotors with external uncertainties and collision avoidance. IEEE Trans Control Syst Technol. 2024;32(4):1310–22. doi:10.1109/tcst.2024.3363372.ChenY, WuY, LanL, ZhongH, MiaoZ, ZhangH, et al. Dynamic target tracking of unmanned aerial vehicles under unpredictable disturbances. Engineering. 2024;35(5):74–85. doi:10.1016/j.eng.2023.05.017.Janabi-SharifiF, DengL, WilsonW. Comparison of basic visual servoing methods. IEEE/ASME Trans Mechatron. 2011;16(5):967–83. doi:10.1109/tmech.2010.2063710.LuM, ChenH, LuP. Perception and avoidance of multiple small fast moving objects for quadrotors with only low-cost RGBD camera. IEEE Robot Autom Lett. 2022;7(4):11657–64. doi:10.1109/lra.2022.3205114.GuoD, LeangK. Image-based estimation, planning, and control for high-speed flying through multiple openings. Int J Robot Res. 2020;39(9):1122–37. doi:10.1177/0278364920921943.ZhangX, FangY, ZhangX, JiangJ, ChenX. A novel geometric hierarchical approach for dynamic visual servoing of quadrotors. IEEE Trans Ind Electron. 2020;67(5):3840–9. doi:10.1109/tie.2019.2917420.ZhengD, WangH, WangJ, ChenS, ChenW, LiangX. Image-based visual servoing of a quadrotor using virtual camera approach. IEEE/ASME Trans Mechatron. 2017;22(2):972–82. doi:10.1109/tmech.2016.2639531.LinJ, WangY, MiaoZ, FanS, WangH. Robust observer-based visual servo control for quadrotors tracking unknown moving targets. IEEE/ASME Trans Mechatron. 2023;28(3):1268–79. doi:10.1109/tmech.2022.3217034.AslH, YoonJ. Bounded‐input control of the quadrotor unmanned aerial vehicle: a vision‐based approach. Asian J Control. 2017;19(3):840–55. doi:10.1002/asjc.1420.AslH. Robust vision-based tracking control of VTOL unmanned aerial vehicles. Automatica. 2019;107:425–32.CaoZ, ChenX, YuY, YuJ, LiuX, ZhouC, et al. Image dynamics-based visual servoing for quadrotors tracking a target with a nonlinear trajectory observer. IEEE Trans Syst Man Cybern Syst. 2020;50(1):376–84. doi:10.1109/tsmc.2017.2720173.LiJ, XieH, LowK, YongJ, LiB. Image-based visual servoing of rotorcrafts to planar visual targets of arbitrary orientation. IEEE Robot Autom Lett. 2021;6(4):7861–8. doi:10.1109/lra.2021.3101878.ChenY, WuY, ZhangZ, MiaoZ, ZhongH, ZhangH, et al. Image-based visual servoing of unmanned aerial manipulators for tracking and grasping a moving target. IEEE Trans Ind Inform. 2023;19:8889–99.XieH, LowK, HeZ. Adaptive visual servoing of unmanned aerial vehicles in GPS-denied environments. IEEE/ASME Trans Mechatron. 2017;22(6):2554–63. doi:10.1109/tmech.2017.2755669.XieH, LynchA, LowK, MaoS. Adaptive output-feedback image-based visual servoing for quadrotor unmanned aerial vehicles. IEEE Trans Control Syst Technol. 2020;28(3):1034–41. doi:10.1109/tcst.2019.2892034.ZhangK, ShiY, ShengH. Robust nonlinear model predictive control based visual servoing of quadrotor UAVs. IEEE/ASME Trans Mechatron. 2021;26(2):700–8. doi:10.1109/tmech.2021.3053267.Jabbari AslH, YoonJ. Robust image-based control of the quadrotor unmanned aerial vehicle. Nonlinear Dyn. 2016;85(3):2035–48. doi:10.1007/s11071-016-2813-2.ChenW, YangJ, GuoL, LiS. Disturbance-observer-based control and related methods—an overview. IEEE Trans Ind Electron. 2015;63(2):1083–95. doi:10.1109/tie.2015.2478397.HuaH, FangY, ZhangX, LuB. A novel robust observer-based nonlinear trajectory tracking control strategy for quadrotors. IEEE Trans Control Syst Technol. 2021;29(5):1952–63. doi:10.1109/tcst.2020.3024805.LiB, GongW, YangY, XiaoB, RanD. Appointed fixed time observer-based sliding mode control for a quadrotor UAV under external disturbances. IEEE Trans Aerosp Electron Syst. 2022;58(1):290–303. doi:10.1109/taes.2021.3101562.LevantA. Higher-order sliding modes, differentiation and output-feedback control. Int J Control. 2003;76:924–41. doi:10.1080/0020717031000099029.ZhaoZ, CaoD, YangJ, WangH. High-order sliding mode observer-based trajectory tracking control for a quadrotor UAV with uncertain dynamics. Nonlinear Dyn. 2020;102:2583–96. doi:10.1007/s11071-020-06050-2.HuS, WangQ, WangF, LiY. Finite-time dynamic visual servo control for quadrotor tracking unknown motion target. Nonlinear Dyn. 2025;113(7):6959–77. doi:10.1007/s11071-024-10713-9.MorenoJ. A Lyapunov approach to output feedback control using second-order sliding modes. IMA J Math Control Inf. 2012;29(3):291–308. doi:10.1093/imamci/dnr036.