Authors

Eduardo Sebastián, Thai Duong, Nikolay Atanasov, Eduardo Montijano, Carlos Sagüés

Abstract

This paper presents LEMURS, an algorithm for learning scalable multi-robot control policies from cooperative task demonstrations. We propose a port-Hamiltonian description of the multi-robot system to exploit universal physical constraints in interconnected systems and achieve closed-loop stability. We represent a multi-robot control policy using an architecture that combines self-attention mechanisms and neural ordinary differential equations. The former handles time-varying communication in the robot team, while the latter respects the continuous-time robot dynamics. Our representation is distributed by construction, enabling the learned control policies to be deployed in robot teams of different sizes. We demonstrate that LEMURS can learn interactions and cooperative behaviors from demonstrations of multi-agent navigation and flocking tasks.

Citation

  • Journal: 2023 IEEE International Conference on Robotics and Automation (ICRA)
  • Year: 2023
  • Volume:
  • Issue:
  • Pages: 7713–7719
  • Publisher: IEEE
  • DOI: 10.1109/icra48891.2023.10161328

BibTeX

@inproceedings{Sebasti_n_2023,
  title={{LEMURS: Learning Distributed Multi-Robot Interactions}},
  DOI={10.1109/icra48891.2023.10161328},
  booktitle={{2023 IEEE International Conference on Robotics and Automation (ICRA)}},
  publisher={IEEE},
  author={Sebastián, Eduardo and Duong, Thai and Atanasov, Nikolay and Montijano, Eduardo and Sagüés, Carlos},
  year={2023},
  pages={7713--7719}
}

Download the bib file

References

  • Yang, F. & Matni, N. Communication Topology Co-Design in Graph Recurrent Neural Network based Distributed Control. 2021 60th IEEE Conference on Decision and Control (CDC) 3619–3626 (2021) doi:10.1109/cdc45484.2021.9683779 – 10.1109/cdc45484.2021.9683779
  • Blankenstein, G., Ortega, R. & Van Der Schaft, A. J. The matching conditions of controlled Lagrangians and IDA-passivity based control. International Journal of Control vol. 75 645–665 (2002) – 10.1080/00207170210135939
  • Tolstaya, E., Paulos, J., Kumar, V. & Ribeiro, A. Multi-Robot Coverage and Exploration using Spatial Graph Neural Networks. 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 8944–8950 (2021) doi:10.1109/iros51168.2021.9636675 – 10.1109/iros51168.2021.9636675
  • Olfati-Saber, R. Flocking for Multi-Agent Dynamic Systems: Algorithms and Theory. IEEE Transactions on Automatic Control vol. 51 401–420 (2006) – 10.1109/tac.2005.864190
  • furieri, Distributed neural network control with dependability guarantees: a compositional port-hamiltonian approach. Learning for Dynamics and Control Conference (0)
  • ramachandran, Searching for activation functions. ArXiv Preprint (2017)
  • Gama, F., Li, Q., Tolstaya, E., Prorok, A. & Ribeiro, A. Synthesizing Decentralized Controllers With Graph Neural Networks and Imitation Learning. IEEE Transactions on Signal Processing vol. 70 1932–1946 (2022) – 10.1109/tsp.2022.3166401
  • Butcher, J. C. Numerical Methods for Ordinary Differential Equations. (2016) doi:10.1002/9781119121534 – 10.1002/9781119121534
  • long, Evolutionary population curriculum for scaling multi-agent reinforcement learning. International Conference on Learning Representations (0)
  • vaswani, Attention is all you need. Advances in neural information processing systems (2017)
  • tolstaya, Learning decentralized controllers for robot swarms with graph neural networks. Conference on Robot Learning (0)
  • chen, Neural ordinary differential equations. Advances in neural information processing systems (0)
  • khan, Graph policy gradients for large scale robot control. Conference on Robot Learning (0)
  • Li, Q., Lin, W., Liu, Z. & Prorok, A. Message-Aware Graph Attention Networks for Large-Scale Multi-Robot Path Planning. IEEE Robotics and Automation Letters vol. 6 5533–5540 (2021) – 10.1109/lra.2021.3077863
  • Tian, Y. et al. Kimera-Multi: Robust, Distributed, Dense Metric-Semantic SLAM for Multi-Robot Systems. IEEE Transactions on Robotics vol. 38 2022–2038 (2022) – 10.1109/tro.2021.3137751
  • Atanasov, N., Le Ny, J., Daniilidis, K. & Pappas, G. J. Decentralized active information acquisition: Theory and application to multi-robot SLAM. 2015 IEEE International Conference on Robotics and Automation (ICRA) 4775–4782 (2015) doi:10.1109/icra.2015.7139863 – 10.1109/icra.2015.7139863
  • Shi, G., Honig, W., Yue, Y. & Chung, S.-J. Neural-Swarm: Decentralized Close-Proximity Multirotor Control Using Learned Interactions. 2020 IEEE International Conference on Robotics and Automation (ICRA) 3241–3247 (2020) doi:10.1109/icra40945.2020.9196800 – 10.1109/icra40945.2020.9196800
  • Han, R., Chen, S. & Hao, Q. Cooperative Multi-Robot Navigation in Dynamic Environment with Deep Reinforcement Learning. 2020 IEEE International Conference on Robotics and Automation (ICRA) (2020) doi:10.1109/icra40945.2020.9197209 – 10.1109/icra40945.2020.9197209
  • Semnani, S. H., Liu, H., Everett, M., de Ruiter, A. & How, J. P. Multi-Agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learning. IEEE Robotics and Automation Letters vol. 5 3221–3226 (2020) – 10.1109/lra.2020.2974695
  • Long, P. et al. Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learning. 2018 IEEE International Conference on Robotics and Automation (ICRA) 6252–6259 (2018) doi:10.1109/icra.2018.8461113 – 10.1109/icra.2018.8461113
  • qu, Scalable reinforcement learning of localized policies for multi-agent networked systems. Learning for Dynamics and Control (2020)
  • Zhou, S., Phielipp, M. J., Sefair, J. A., Walker, S. I. & Amor, H. B. Clone Swarms: Learning to Predict and Control Multi-Robot Systems by Imitation. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 4092–4099 (2019) doi:10.1109/iros40897.2019.8967824 – 10.1109/iros40897.2019.8967824
  • Wang, B., Xie, J. & Atanasov, N. DARL1N: Distributed multi-Agent Reinforcement Learning with One-hop Neighbors. 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 9003–9010 (2022) doi:10.1109/iros47612.2022.9981441 – 10.1109/iros47612.2022.9981441
  • yang, Mean field multi-agent reinforcement learning. International Conference on Machine Learning (0)
  • dasari, Robonet: Large-scale multi-robot learning. Conference on Robot Learning (0)
  • Zhu, H., Claramunt, F. M., Brito, B. & Alonso-Mora, J. Learning Interaction-Aware Trajectory Predictions for Decentralized Multi-Robot Motion Planning in Dynamic Environments. IEEE Robotics and Automation Letters vol. 6 2256–2263 (2021) – 10.1109/lra.2021.3061073
  • Bogert, K. & Doshi, P. Multi-robot inverse reinforcement learning under occlusion with estimation of state transitions. Artificial Intelligence vol. 263 46–73 (2018) – 10.1016/j.artint.2018.07.002
  • van der Schaft, A. & Jeltsema, D. Port-Hamiltonian Systems Theory: An Introductory Overview. Foundations and Trends® in Systems and Control vol. 1 173–378 (2014) – 10.1561/2600000002
  • ng, Algorithms for inverse reinforcement learning. International Conference on Machine Learning (0)
  • galimberti, Hamiltonian deep neural networks guaranteeing non-vanishing gradients by design. ArXiv Preprint (2021)
  • Jiahao, T. Z., Pan, L. & Hsieh, M. A. Learning to Swarm with Knowledge-Based Neural Ordinary Differential Equations. 2022 International Conference on Robotics and Automation (ICRA) 6912–6918 (2022) doi:10.1109/icra46639.2022.9811997 – 10.1109/icra46639.2022.9811997
  • Heintzman, L., Hashimoto, A., Abaid, N. & Williams, R. K. Anticipatory Planning and Dynamic Lost Person Models for Human-Robot Search and Rescue. 2021 IEEE International Conference on Robotics and Automation (ICRA) 8252–8258 (2021) doi:10.1109/icra48506.2021.9562070 – 10.1109/icra48506.2021.9562070
  • Bloembergen, D., Tuyls, K., Hennes, D. & Kaisers, M. Evolutionary Dynamics of Multi-Agent Learning: A Survey. Journal of Artificial Intelligence Research vol. 53 659–697 (2015) – 10.1613/jair.4818
  • Pierson, A. & Schwager, M. Bio-inspired non-cooperative multi-robot herding. 2015 IEEE International Conference on Robotics and Automation (ICRA) 1843–1849 (2015) doi:10.1109/icra.2015.7139438 – 10.1109/icra.2015.7139438
  • Kan, X., Thayer, T. C., Carpin, S. & Karydis, K. Task Planning on Stochastic Aisle Graphs for Precision Agriculture. IEEE Robotics and Automation Letters vol. 6 3287–3294 (2021) – 10.1109/lra.2021.3062337
  • Sebastian, E., Montijano, E. & Sagues, C. Adaptive Multirobot Implicit Control of Heterogeneous Herds. IEEE Transactions on Robotics vol. 38 3622–3635 (2022) – 10.1109/tro.2022.3183537
  • Sebastián, E., Montijano, E. & Sagüés, C. Multi-robot Implicit Control of Massive Herds. Lecture Notes in Networks and Systems 448–459 (2022) doi:10.1007/978-3-031-21065-5_37 – 10.1007/978-3-031-21065-5_37