Crafting a robotic swarm pursuit–evasion capture strategy using deep reinforcement learning

In this paper we study the multi-agent pursuit–evasion problem, and present an extension of the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) deep reinforcement learning algorithm. Previous pursuit–evasion advancements with MADDPG have focused on training capture strategies dependent on the restriction of evader movement with environmental features. We demonstrate a method to train pursuer agents to collaboratively surround and encircle an evader for reliable capture without a strategy rooted in environment entrapment (i.e. cornering). Our method utilizes a novel two-stage, variable-aggression, continuous reward function based on geometrical inscribed circles (incircles), along with a corresponding observation space, with agents operating in an entrapment-disadvantaged environment. Our results show reliable capture of an intelligent, superior evader by three trained pursuers in open space with our encircling strategy. A key novelty of our work is demonstrating the ability to transition behaviors learned using deep reinforcement learning from a simulated robotic system with imperfect world assumptions to a real-world robotic agents.