Biologically Inspired Reinforcement Learning for Locomotion: A Central-Pattern Generator Approach

Abstract

Bipedal locomotion, such as walking or running, involves complex coordination of rhythmic and cyclic movements that must be stable, smooth, and adaptable to varying terrains, which is challenging to achieve in robotic and simulated environments. Current reinforcement learning (RL) approaches often fail to generate stable and natural locomotion patterns due to a lack of inherent rhythmic control, resulting in jerky, unstable, and inefficient gaits. These methods typically do not incorporate the biological principles of rhythmicity and adaptability, which are crucial for achieving natural bipedal locomotion. This study presents a novel approach integrating Central Pattern Generators (CPG) with multiple RL algorithms, including Maximum a Posteriori Policy Optimization, Deep Deterministic Policy Gradient, and Soft Actor-Critic (SAC), using Matsuoka Oscillators to generate rhythmic patterns. By comparing these RL-CPG hybrid methods, the research demonstrates improvements in energy efficiency and synchronization for SAC+CPG in controlled environments. While other algorithms may have advantages in different conditions, SAC+CPG showed the most stable, and rhythmic gait, while minimising energy usage under the tested parameters. This study highlights the first multi-algorithm application of RL combined with CPGs for rhythmic control in bipedal locomotion, contributing to the future of robotics and cyber-physical systems.