Sharma, Ankit (Indian Institute of Technology Roorkee, Hitachi India Pvt. Ltd.), Chakrabarty, Sohom (Indian Institute of Technology Roorkee)

Curriculum-Based Proximal Policy Optimization for Hovering UAVs: Controlling Slack-To-Taut Dynamics During Suspended Payload Release

Scheduled for presentation during the Regular Session "Aerial and Space Robots" (WeBT4), Wednesday, July 16, 2025, 14:00−14:20, Room 108

Joint 10th IFAC Symposium on Mechatronic Systems and 14th Symposium on Robotics, July 15-18, 2025, Paris, France

This information is tentative and subject to change. Compiled on July 16, 2025

Abstract

Designing a unified controller for a UAV carrying a suspended payload presents significant challenges, particularly during transitions when the cable shifts between slack and taut states. These transitions generate abrupt jerks and introduce highly nonlinear dynamics. Conventional approaches typically depend on multiple specialized controllers and rely heavily on mathematical modelling, which complicates implementation. To address these issues, we propose a Curriculum-Based Proximal Policy Optimization (C-PPO) controller that employs a single integrated policy capable of managing both slack and taut cable conditions without requiring explicit system modelling. By leveraging model-free reinforcement learning, the controller learns directly from dynamic interactions within a MuJoCo virtual environment, encompassing UAV dynamics and payload movements. We demonstrate theoretically that the C-PPO’s learned policy is both unique and robust, and effective in mitigating instabilities caused by sudden jerks. Empirically, it was also shown that C-PPO maintains stable hovering performance even under previously unseen conditions, including variations in payload mass, rope length, and wind profiles, by conducting experiments within the MuJoCo environment.