Abstract
Flying quadcopters with suspended payloads presents a significant challenge due to the complex, hybrid dynamics introduced by the swinging mass. Traditional control methods often require precise knowledge of physical parameters, such as cable length and payload mass, which are difficult to estimate in real-time. This paper presents a model-based meta-reinforcement learning (MB-MRL) framework that enables quadcopters to adapt to unknown suspended payloads during flight. By augmenting a dynamics model with latent variables updated via variational inference, the system can identify the physics of a new payload within seconds. Experimental results demonstrate that this approach significantly outperforms non-adaptive controllers and generalizes effectively to unseen payload configurations.
Introduction
While quadcopter control is a well-studied field, the addition of a payload attached via a flexible cable transforms the vehicle into a complex system with high-order dynamics. The payload introduces oscillations and forces that vary drastically depending on the cable's length and the object's mass. Standard control strategies, such as PID or fixed-model System Identification, typically rely on a priori knowledge of these parameters. When a drone encounters a payload with an unknown weight or a different string length, the resulting model mismatch often leads to poor tracking performance or catastrophic instability.
The core challenge addressed in this work is rapid online adaptation. To operate effectively in dynamic environments, a robot must be able to identify how a new payload affects its flight characteristics and adjust its motor commands almost instantaneously without manual recalibration.
Methodology
The proposed framework utilizes a model-based meta-reinforcement learning approach, building upon the Probabilistic Ensembles with Trajectory Sampling (PETS) algorithm. The architecture consists of three primary components:
1. Latent Dynamics Model
The researchers trained a neural network to predict the next state ($s_{t+1}$) based on the current state and action. To account for varying physical properties, they introduced a stochastic latent variable ($z$) as an input. This $z$ variable serves as a "context" or "style" embedding that captures the specific dynamics of the current payload, such as its mass or the tether length, without requiring these values to be explicitly measured.
2. Meta-Training
During the training phase, the robot is exposed to a variety of payloads. The model learns a shared set of weights ($\theta$) that represent the general physics of flight, while simultaneously learning to map specific flight trajectories to the appropriate latent variable $z$. This "learning to learn" phase ensures the model is prepared for a wide distribution of possible dynamics.
3. Online Adaptation via Variational Inference
At test time, the robot maintains a small buffer of recent flight data. Every few milliseconds, the system performs variational inference to estimate the posterior distribution of $z$. By performing gradient descent on the Evidence Lower Bound (ELBO), the model asks: "Given the movements I just made and the resulting states, what kind of payload must I be carrying?" This updated $z$ is then fed into a Model Predictive Controller (MPC) to plan optimal future actions.
Experimental Evaluation
The system was evaluated using a DJI Tello quadcopter. The training dataset consisted of approximately 16,000 data points (roughly 1.1 hours of flight time) collected by a human pilot. The training data included payloads with tether lengths of 18cm and 30cm and masses ranging from 10 to 15 grams.
The evaluation tasks required the quadcopter to follow circle, square, and figure-8 trajectories in a vertical plane. To test the robustness of the meta-learning approach, the researchers also evaluated the model on unseen tether lengths (21cm and 27cm) to determine if the system could interpolate between the dynamics learned during training.
Results and Discussion
Tracking Performance
The MB-MRL approach achieved an average tracking error of approximately 24 pixels. In comparison, standard non-adaptive MBRL methods frequently failed to complete the tasks, often drifting out of the camera's field of view. The meta-learning framework also outperformed manually tuned PID controllers, which struggled to compensate for the payload's oscillations.
The Value of Latent Representations
A key insight from the study was that the model performed better when allowed to "discover" its own latent variables ($z$) rather than being provided with explicit physical parameters like tether length. This suggests that the latent space captures complex, unmodeled nuances—such as air resistance or magnetic gripper interactions—that simple physical constants fail to represent.
Rapid Adaptation and Generalization
Visualization of the latent variable $z$ during flight showed that it converges to a stable value almost immediately after the drone begins moving. As $z$ stabilizes, the tracking error drops significantly, demonstrating a direct link between the model's understanding of the payload and its flight accuracy. Furthermore, the system successfully handled sudden discontinuities, such as picking up or dropping a payload, by flushing the data buffer and re-adapting to the new state within seconds.
Conclusion
This work demonstrates that model-based meta-reinforcement learning is a powerful tool for the control of complex, variable robotic systems. By using latent variables and online variational inference, quadcopters can adapt to the unpredictable dynamics of suspended payloads in real-time.
Future work will focus on moving beyond external camera tracking to onboard sensing (such as IMUs or computer vision), optimizing the computational overhead of the MPC planner for onboard deployment, and testing the system's resilience to external environmental factors like wind.
Summary updated