Deep Reinforcement Learning-Based Control of Snake Robot Joints

Baysal Aslanhan Y. A., Altas I. H.

ARABIAN JOURNAL FOR SCIENCE AND ENGINEERING, cilt.50, ss.1-26, 2025 (SCI-Expanded, Scopus)

Özet

Snake robots exhibit great potential for navigating and performing tasks in complex and dynamic environments. However, controlling them in such environments is challenging due to their many degrees of freedom, making precise joint control essential for their effective operation. Classical control strategies can only provide satisfactory performance for specific conditions. Therefore, more sophisticated control methods are required to manage them under changing environments. This paper introduces a novel approach for controlling the wheel-less snake robot joints on different ground conditions by developing four controllers based on deep reinforcement learning (DRL) algorithms. Unlike classical methods, DRL algorithms offer learning flexibility and robust generalization capabilities, allowing the robots to adapt to varying environmental conditions more effectively. The environmental changes are simulated by adjusting gait parameters. The findings reveal that DRL-based controllers demonstrated robust adaptability and high accuracy in joint control across varying conditions, showcasing their potential for managing the complexities of snake robot dynamics. Among these, the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, combined with transfer learning (Transfer-TD3) achieved the best performance, maintaining an average integral square error of 0.0327 across training and testing datasets. Additionally, it demonstrated a convergence speed 76% faster than the standard TD3 algorithm and reduced position tracking error by approximately 8.41% compared to the classical PD controller. A Wilcoxon Signed-Rank Test (p = 0.01687) confirmed the statistically significant improvements of Transfer-TD3 in generalization capability. This study demonstrates the potential of DRL algorithms to enhance the adaptive capabilities of snake robots in simulated environments, laying the groundwork for future research and potential real-world applications.