Voronoi-Based Multi-Robot Autonomous Exploration in Unknown Environments via Deep Reinforcement Learning

@article{Hu2020VoronoiBasedMA,
  title={Voronoi-Based Multi-Robot Autonomous Exploration in Unknown Environments via Deep Reinforcement Learning},
  author={Junyan Hu and Hanlin Niu and Joaqu{\'i}n Carrasco and Barry Lennox and Farshad Arvin},
  journal={IEEE Transactions on Vehicular Technology},
  year={2020},
  volume={69},
  pages={14413-14423},
  url={https://api.semanticscholar.org/CorpusID:228989788}
}
A novel cooperative exploration strategy is proposed for multiple mobile robots, which reduces the overall task completion time and energy costs compared to conventional methods and enables the control policy to learn from human demonstration data and thus improve the learning speed and performance.
View on IEEE
doi.org
270 Citations
Highly Influential Citations
10
Background Citations
95
Methods Citations
52
Results Citations
3

Figures from this paper

270 Citations

Voronoi-GRU-Based Multi-Robot Collaborative Exploration in Unknown Environments

This paper proposes a novel collaborative exploration strategy for multiple mobile robots, aiming to quickly realize the exploration of entire unknown environments with a significantly reduced exploration time, and investigates a hierarchical control architecture.

Multi-Robot Exploration in Unknown Environments via Multi-Agent Deep Reinforcement Learning

A multi-agent deep reinforcement learning (MADRL) based method with the centralized training and decentralized execution (CTDE) architecture was proposed and extensive experimental results show that the method significantly improves the multi-robot exploration performance in unknown environments.

Communication-Efficient Multi-Robot Exploration Using Coverage-Biased Distributed Q-Learning

The proposed CQLite method uses ad hoc map merging, and selectively shares updated Q-values at recently identified frontiers to reduce communication costs significantly and outperformed cutting-edge multi-robot exploration techniques like Rapidly-exploring Random Trees and Deep Reinforcement Learning.

Bio-inspired Decentralized Multi-robot Exploration in Unknown Environments

A novel multi-agent reinforcement learning approach called Multi-Agent Proximal Policy Optimization-Pheromone Interaction Mechanism (MAPPO-PIM) to realize self-adaptive and self-organized exploration to achieve efficient collaboration in communication-limited scenarios.

MARVEL: Multi-Agent Reinforcement Learning for constrained field-of-View multi-robot Exploration in Large-scale environments

This work proposes MARVEL, a neural framework that leverages graph attention networks, together with novel frontiers and orientation features fusion technique, to develop a collaborative, decentralized policy using multi-agent reinforcement learning (MARL) for robots with constrained FoV, and introduces a novel information-driven action pruning strategy.

Efficient Autonomous Exploration and Mapping in Unknown Environments

A Local-and-Global Strategy (LAGS) algorithm that combines a local exploration strategy with a global perception strategy, which considers and solves the regional legacy issues in the autonomous exploration process to improve exploration efficiency is proposed.

Decentralised Multi-Robot Exploration Using Monte Carlo Tree Search

This work proposes an approach based on the Monte Carlo Tree Search (MCTS) algorithm to guide robots during exploration missions, and allows for the coordination of the robots' movements in a decentralized manner, relying on point-to-point communication.

Multi-Robot Environmental Coverage With a Two-Stage Coordination Strategy via Deep Reinforcement Learning

A Two-Stage Coordination (TSC) strategy, which consists of a high-level leader module and a low-level action executor that provides the robots with the topology and geometry of the environment, which are crucial for robots to learn “where” they should go and avoid invalid coverage.

Multi-Robot Collaborative Exploration on Communication-Constrained Environments

Experimental results demonstrate that CMICE outperforms the state-of-the-art models in both ideal and communication-constrained environments.

Efficient deep reinforcement learning-based multirobot path planning

This work proposes an efficient reinforcement learning-based path planning policy for multi-robot systems using distributed multi-agent deep deterministic policy gradients (MADDPG), and introduces the TEM-MRDDPG algorithm, which builds on MR-MADDPG by integrating trajectory optimization and centralized training with distributed execution, leading to improved adaptability and efficiency.
...

42 References

Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning

This work presents a decentralized sensor-level collision avoidance policy for multi-robot systems, which directly maps raw sensor measurements to an agent's steering commands in terms of movement velocity and demonstrates that the learned policy can be well generalized to new scenarios that do not appear in the entire training period.

Deep Reinforcement Learning Robot for Search and Rescue Applications: Exploration in Unknown Cluttered Environments

This letter presents the first use of deep learning to address the robot exploration task in USAR applications, and uniquely combine the traditional approach of frontier-based exploration with deep reinforcement learning to allow a robot to autonomously explore unknown cluttered environments.

Bayesian Reinforcement Learning for Multi-Robot Decentralized Patrolling in Uncertain Environments

A scalable decentralized online learning and planning algorithm is proposed by extending the Monte Carlo tree search method, which computes patrols that are bounded optimal and is able to achieve high performance in typical decentralized patrolling scenarios.

Coordinated multi-robot exploration

This paper presents an approach for the coordination of multiple robots, which simultaneously takes into account the cost of reaching a target point and its utility and describes how this algorithm can be extended to situations in which the communication range of the robots is limited.

Deep Reinforcement Learning Supervised Autonomous Exploration in Office Environments

An algorithm is presented that utilizes deep reinforcement learning (DRL) to learn exploration knowledge over office blueprints, which enables the agent to predict a long-term visiting order for unexplored subregions and an exploration architecture that integrates a DRL model, a next-best-view (NBV) selection approach and a structural integrity measurement to further improve the exploration performance.

Autonomous Robotic Exploration by Incremental Road Map Construction

An efficient path planning framework is introduced to reduce the path length and exploration time and a target selection mechanism helps the robot determine the next best target to explore and the proposed trajectory optimization algorithm helps in reducing the path cost.

Deep reinforcement learning with successor features for navigation across similar environments

This paper proposes a successor-feature-based deep reinforcement learning algorithm that can learn to transfer knowledge from previously mastered navigation tasks to new problem instances and substantially decreases the required learning time after the first task instance has been solved.

Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation

It is shown that, through an asynchronous deep reinforcement learning method, a mapless motion planner can be trained end-to-end without any manually designed features and prior demonstrations.

The Sensor-based Random Graph Method for Cooperative Robot Exploration

A roadmap of the explored area, with the associate safe region, is built in the form of a sensor-based random graph (SRG) by a team of mobile robots equipped with range finders by using a randomized local planner that automatically realizes a tradeoff between information gain and navigation cost.

Efficient Autonomous Exploration Planning of Large-Scale 3-D Environments

This letter presents a method that combines both approaches, with FEP as a global exploration planner and RH-NBVP for local exploration, and presents techniques to estimate potential information gain faster, to cache previously estimated gains and to exploit these to efficiently estimate new queries.