Human-Robot Cooperative Distribution Coupling for Hamiltonian-Constrained Social Navigation

Authors

Weizheng Wang, Chao Yu, Yu Wang, Byung-Cheol Min

Abstract

Navigating in human-filled public spaces is a critical challenge for deploying autonomous robots in real-world environments. This paper introduces NaviDIFF, a novel Hamiltonian-constrained socially-aware navigation framework designed to address the complexities of human-robot interaction and socially-aware path planning. NaviDIFF integrates a port-Hamiltonian framework to model dynamic physical interactions and a diffusion model to manage uncertainty in human-robot cooperation. The framework leverages a spatial-temporal transformer to capture social and temporal dependencies, enabling more accurate spatial-temporal environmental dynamics understanding and port-Hamiltonian physical interactive process construction. Additionally, reinforcement learning from human feedback is employed to fine-tune robot policies, ensuring adaptation to human preferences and social norms. Extensive experiments demonstrate that NaviDIFF outperforms state-of-the-art methods in social navigation tasks, offering improved stability, efficiency, and adaptability11The experimental videos and additional information about this work can be found at: https://sites.google.com/view/NaviDIFF.

Citation

• Journal: 2025 IEEE International Conference on Robotics and Automation (ICRA)
• Year: 2025
• Volume:
• Issue:
• Pages: 10808–10815
• Publisher: IEEE
• DOI: 10.1109/icra55743.2025.11128561

BibTeX

@inproceedings{Wang_2025,
  title={{Human-Robot Cooperative Distribution Coupling for Hamiltonian-Constrained Social Navigation}},
  DOI={10.1109/icra55743.2025.11128561},
  booktitle={{2025 IEEE International Conference on Robotics and Automation (ICRA)}},
  publisher={IEEE},
  author={Wang, Weizheng and Yu, Chao and Wang, Yu and Min, Byung-Cheol},
  year={2025},
  pages={10808--10815}
}

Download the bib file

References

• Zheng, Diffusion-based planning for autonomous driving with flexible guidance. The Thirteenth International Conference on Learning Representations (2025)
• Chi C, Xu Z, Feng S, Cousineau E, Du Y, Burchfiel B, Tedrake R, Song S (2024) Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research 44(10–11):1684–1704. https://doi.org/10.1177/0278364924127366 – 10.1177/02783649241273668
• Shamsah A, Gu Z, Warnke J, Hutchinson S, Zhao Y (2023) Integrated Task and Motion Planning for Safe Legged Navigation in Partially Observable Environments. IEEE Trans Robot 39(6):4913–4934. https://doi.org/10.1109/tro.2023.329952 – 10.1109/tro.2023.3299524
• Yao W, de Marina HG, Lin B, Cao M (2021) Singularity-Free Guiding Vector Field for Robot Navigation. IEEE Trans Robot 37(4):1206–1221. https://doi.org/10.1109/tro.2020.304369 – 10.1109/tro.2020.3043690
• Devo A, Mezzetti G, Costante G, Fravolini ML, Valigi P (2020) Towards Generalization in Target-Driven Visual Navigation by Using Deep Reinforcement Learning. IEEE Trans Robot 36(5):1546–1561. https://doi.org/10.1109/tro.2020.299400 – 10.1109/tro.2020.2994002
• Fan T, Long P, Liu W, Pan J (2020) Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios. The International Journal of Robotics Research 39(7):856–892. https://doi.org/10.1177/027836492091653 – 10.1177/0278364920916531
• Kretzschmar H, Spies M, Sprunk C, Burgard W (2016) Socially compliant mobile robot navigation via inverse reinforcement learning. The International Journal of Robotics Research 35(11):1289–1307. https://doi.org/10.1177/027836491561977 – 10.1177/0278364915619772
• Biswas J, Veloso MM (2013) Localization and navigation of the CoBots over long-term deployments. The International Journal of Robotics Research 32(14):1679–1694. https://doi.org/10.1177/027836491350389 – 10.1177/0278364913503892
• Newman P, Sibley G, Smith M, Cummins M, Harrison A, Mei C, Posner I, Shade R, Schroeter D, Murphy L, Churchill W, Cole D, Reid I (2009) Navigating, Recognizing and Describing Urban Spaces With Vision and Lasers. The International Journal of Robotics Research 28(11–12):1406–1433. https://doi.org/10.1177/027836490934148 – 10.1177/0278364909341483
• Pereira GAS, Pimenta LCA, Fonseca AR, Corrêa L de Q, Mesquita RC, Chaimowicz L, de Almeida DSC, Campos MFM (2009) Robot Navigation in Multi-terrain Outdoor Environments. The International Journal of Robotics Research 28(6):685–700. https://doi.org/10.1177/027836490809757 – 10.1177/0278364908097578
• Wang W, Wang R, Mao L, Min B-C (2023) NaviSTAR: Socially Aware Robot Navigation with Hybrid Spatio-Temporal Graph Transformer and Preference Learning. 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 11348–1135 – 10.1109/iros55552.2023.10341395
• Wang W, Mao L, Wang R, Min B-C (2024) Multi-Robot Cooperative Socially-Aware Navigation Using Multi-Agent Reinforcement Learning. 2024 IEEE International Conference on Robotics and Automation (ICRA) 12353–1236 – 10.1109/icra57147.2024.10611322
• Mavrogiannis C, Knepper RA (2021) Hamiltonian coordination primitives for decentralized multiagent navigation. The International Journal of Robotics Research 40(10–11):1234–1254. https://doi.org/10.1177/0278364921103773 – 10.1177/02783649211037731
• Wang C, Chen X, Li C, Song R, Li Y, Meng MQ-H (2023) Chase and Track: Toward Safe and Smooth Trajectory Planning for Robotic Navigation in Dynamic Environments. IEEE Trans Ind Electron 70(1):604–613. https://doi.org/10.1109/tie.2022.314875 – 10.1109/tie.2022.3148753
• Trautman P, Ma J, Murray RM, Krause A (2015) Robot navigation in dense human crowds: Statistical models and experimental studies of human–robot cooperation. The International Journal of Robotics Research 34(3):335–356. https://doi.org/10.1177/027836491455787 – 10.1177/0278364914557874
• Mavrogiannis CI, Knepper RA (2018) Multi-agent path topology in support of socially competent navigation planning. The International Journal of Robotics Research 38(2–3):338–356. https://doi.org/10.1177/027836491878101 – 10.1177/0278364918781016
• Mavrogiannis C, Alves-Oliveira P, Thomason W, Knepper RA (2022) Social Momentum: Design and Evaluation of a Framework for Socially Competent Robot Navigation. J Hum-Robot Interact 11(2):1–37. https://doi.org/10.1145/349524 – 10.1145/3495244
• Samavi S, Han JR, Shkurti F, Schoellig AP (2025) SICNav: Safe and Interactive Crowd Navigation Using Model Predictive Control and Bilevel Optimization. IEEE Trans Robot 41:801–818. https://doi.org/10.1109/tro.2024.348463 – 10.1109/tro.2024.3484634
• Liu S, Chang P, Huang Z, Chakraborty N, Hong K, Liang W, McPherson DL, Geng J, Driggs-Campbell K (2023) Intention Aware Robot Crowd Navigation with Attention-Based Interaction Graph. 2023 IEEE International Conference on Robotics and Automation (ICRA) 12015–1202 – 10.1109/icra48891.2023.10160660
• Sun M, Baldini F, Trautman P, Murphey T (2021) Move Beyond Trajectories: Distribution Space Coupling for Crowd Navigation. Robotics: Science and Systems XVI – 10.15607/rss.2021.xvii.053
• Cao C, Trautman P, Iba S (2019) Dynamic Channel: A Planning Framework for Crowd Navigation. 2019 International Conference on Robotics and Automation (ICRA) 5551–555 – 10.1109/icra.2019.8794192
• Du Toit NE, Burdick JW (2012) Robot Motion Planning in Dynamic, Uncertain Environments. IEEE Trans Robot 28(1):101–115. https://doi.org/10.1109/tro.2011.216643 – 10.1109/tro.2011.2166435
• Thompson S, Horiuchi T, Kagami S (2009) A probabilistic model of human motion and navigation intent for mobile robot path planning. 2009 4th International Conference on Autonomous Robots and Agents 663–66 – 10.1109/icara.2000.4803931
• Vasquez D, Okal B, Arras KO (2014) Inverse Reinforcement Learning algorithms and features for robot navigation in crowds: An experimental comparison. 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems 1341–134 – 10.1109/iros.2014.6942731
• Kim B, Pineau J (2015) Socially Adaptive Path Planning in Human Environments Using Inverse Reinforcement Learning. Int J of Soc Robotics 8(1):51–66. https://doi.org/10.1007/s12369-015-0310- – 10.1007/s12369-015-0310-2
• Knepper RA, Rus D (2012) Pedestrian-inspired sampling-based multi-robot collision avoidance. 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication 94–10 – 10.1109/roman.2012.6343737
• Chen YF, Liu M, Everett M, How JP (2017) Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. 2017 IEEE International Conference on Robotics and Automation (ICRA) 285–29 – 10.1109/icra.2017.7989037
• Liu S, Chang P, Liang W, Chakraborty N, Driggs-Campbell K (2021) Decentralized Structural-RNN for Robot Crowd Navigation with Deep Reinforcement Learning. 2021 IEEE International Conference on Robotics and Automation (ICRA) 3517–352 – 10.1109/icra48506.2021.9561595
• Xu B, Gao F, Yu C, Zhang R, Wu Y, Wang Y (2024) OmniDrones: An Efficient and Flexible Platform for Reinforcement Learning in Drone Control. IEEE Robot Autom Lett 9(3):2838–2844. https://doi.org/10.1109/lra.2024.335616 – 10.1109/lra.2024.3356168
• Yu C, Yang X, Gao J, Yang H, Wang Y, Wu Y (2022) Learning Efficient Multi-agent Cooperative Visual Exploration. Lecture Notes in Computer Science 497–51 – 10.1007/978-3-031-19842-7_29
• Ibarz J, Tan J, Finn C, Kalakrishnan M, Pastor P, Levine S (2021) How to train your robot with deep reinforcement learning: lessons we have learned. The International Journal of Robotics Research 40(4–5):698–721. https://doi.org/10.1177/027836492098785 – 10.1177/0278364920987859
• Kalashnikov, Scaling up multi-task robotic reinforcement learning. Conference on Robot Learning (2022)
• Xu, Prediction-guided multi-objective reinforcement learning for continuous robot control. International conference on machine learning (2020)
• Haarnoja T, Moran B, Lever G, Huang SH, Tirumala D, Humplik J, Wulfmeier M, Tunyasuvunakool S, Siegel NY, Hafner R, Bloesch M, Hartikainen K, Byravan A, Hasenclever L, Tassa Y, Sadeghi F, Batchelor N, Casarini F, Saliceti S, Game C, Sreendra N, Patel K, Gwira M, Huber A, Hurley N, Nori F, Hadsell R, Heess N (2024) Learning agile soccer skills for a bipedal robot with deep reinforcement learning. Sci Robot 9(89). https://doi.org/10.1126/scirobotics.adi802 – 10.1126/scirobotics.adi8022
• Rudin, Learning to walk in minutes using massively parallel deep reinforcement learning. Conference on Robot Learning (2022)
• Raffin, Smooth exploration for robotic reinforcement learning. Conference on Robot Learning (2022)
• Dalal, Accelerating robotic reinforcement learning via parameterized action primitives. Advances in Neural Information Processing Systems (2021)
• van der Schaft A (2007) Port-Hamiltonian systems: an introductory survey. Proceedings of the International Congress of Mathematicians Madrid, August 22–30, 2006 1339–136 – 10.4171/022-3/65
• Angerer M, Music S, Hirche S (2017) Port-Hamiltonian based control for human-robot team interaction. 2017 IEEE International Conference on Robotics and Automation (ICRA) 2292–229 – 10.1109/icra.2017.7989264
• Groothuis SS, Stramigioli S, Carloni R (2017) Modeling Robotic Manipulators Powered by Variable Stiffness Actuators: A Graph-Theoretic and Port-Hamiltonian Formalism. IEEE Trans Robot 33(4):807–818. https://doi.org/10.1109/tro.2017.266838 – 10.1109/tro.2017.2668385
• Altawaitan A, Stanley J, Ghosal S, Duong T, Atanasov N (2024) Hamiltonian Dynamics Learning from Point Cloud Observations for Nonholonomic Mobile Robot Control. 2024 IEEE International Conference on Robotics and Automation (ICRA) 16937–1694 – 10.1109/icra57147.2024.10610395
• Sebastián E, Duong T, Atanasov N, Montijano E, Sagüés C (2023) LEMURS: Learning Distributed Multi-Robot Interactions. 2023 IEEE International Conference on Robotics and Automation (ICRA) 7713–771 – 10.1109/icra48891.2023.10161328
• Song Z, Antsaklis PJ, Lin H (2024) Port-Hamiltonian-Based Geometric Control for Rigid Body Platoons With Mesh Stability Guarantee. IEEE Control Syst Lett 8:2805–2810. https://doi.org/10.1109/lcsys.2024.351667 – 10.1109/lcsys.2024.3516672
• Massaroli S, Poli M, Califano F, Faragasso A, Park J, Yamashita A, Asama H (2019) Port–Hamiltonian Approach to Neural Network Training. 2019 IEEE 58th Conference on Decision and Control (CDC) 6799–680 – 10.1109/cdc40024.2019.9030017
• Desai SA, Mattheakis M, Sondak D, Protopapas P, Roberts SJ (2021) Port-Hamiltonian neural networks for learning explicit time-dependent dynamical systems. Phys Rev E 104(3). https://doi.org/10.1103/physreve.104.03431 – 10.1103/physreve.104.034312
• Mao W, Xu C, Zhu Q, Chen S, Wang Y (2023) Leapfrog Diffusion Model for Stochastic Trajectory Prediction. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 5517–552 – 10.1109/cvpr52729.2023.00534
• Neary, Compositional learning of dynamical system models using port-hamiltonian neural networks. Learning for Dynamics and Control Conference (2023)
• Shi G, Honig W, Yue Y, Chung S-J (2020) Neural-Swarm: Decentralized Close-Proximity Multirotor Control Using Learned Interactions. 2020 IEEE International Conference on Robotics and Automation (ICRA) 3241–324 – 10.1109/icra40945.2020.9196800
• Schulman, Prox-imal policy optimization algorithms. arXiv preprint (2017)
• Thrun S, Bennewitz M, Burgard W, Cremers AB, Dellaert F, Fox D, Hahnel D, Rosenberg C, Roy N, Schulte J, Schulz D MINERVA: a second-generation museum tour-guide robot. Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C) 3:1999–200 – 10.1109/robot.1999.770401
• Burgard W, Cremers AB, Fox D, Hähnel D, Lakemeyer G, Schulz D, Steiner W, Thrun S (1999) The Museum Tour-Guide Robot RHINO. Informatik aktuell 245–25 – 10.1007/978-3-642-60043-2_29
• Mavrogiannis C, Baldini F, Wang A, Zhao D, Trautman P, Steinfeld A, Oh J (2023) Core Challenges of Social Robot Navigation: A Survey. J Hum-Robot Interact 12(3):1–39. https://doi.org/10.1145/358374 – 10.1145/3583741
• Johnson JK (2020) The Colliding Reciprocal Dance Problem: A Mitigation Strategy with Application to Automotive Active Safety Systems. 2020 American Control Conference (ACC – 10.23919/acc45564.2020.9147351
• Turnwald A, Wollherr D (2018) Human-Like Motion Planning Based on Game Theoretic Decision Making. Int J of Soc Robotics 11(1):151–170. https://doi.org/10.1007/s12369-018-0487- – 10.1007/s12369-018-0487-2
• Chen C, Liu Y, Kreiss S, Alahi A (2019) Crowd-Robot Interaction: Crowd-Aware Robot Navigation With Attention-Based Deep Reinforcement Learning. 2019 International Conference on Robotics and Automation (ICRA) 6015–602 – 10.1109/icra.2019.8794134
• Chen C, Hu S, Nikdel P, Mori G, Savva M (2020) Relational Graph Learning for Crowd Navigation. 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 10007–1001 – 10.1109/iros45743.2020.9340705
• Everett M, Chen YF, How JP (2018) Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learning. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 3052–305 – 10.1109/iros.2018.8593871
• Ortega R, van der Schaft A, Castanos F, Astolfi A (2008) Control by Interconnection and Standard Passivity-Based Control of Port-Hamiltonian Systems. IEEE Trans Automat Contr 53(11):2527–2542. https://doi.org/10.1109/tac.2008.200693 – 10.1109/tac.2008.2006930
• van der Schaft A, Jeltsema D (2014) Port-Hamiltonian Systems Theory: An Introductory Overview. Foundations and Trends® in Systems and Control 1(2–3):173–378. https://doi.org/10.1561/260000000 – 10.1561/2600000002
• Ortega R, van der Schaft A, Castanos F, Astolfi A (2008) Control by Interconnection and Standard Passivity-Based Control of Port-Hamiltonian Systems. IEEE Trans Automat Contr 53(11):2527–2542. https://doi.org/10.1109/tac.2008.200693 – 10.1109/tac.2008.2006930
• Sprangers O, Babuska R, Nageshrao SP, Lopes GAD (2015) Reinforcement Learning for Port-Hamiltonian Systems. IEEE Trans Cybern 45(5):1017–1027. https://doi.org/10.1109/tcyb.2014.234319 – 10.1109/tcyb.2014.2343194
• Blankenstein G, Ortega R, Van Der Schaft AJ (2002) The matching conditions of controlled Lagrangians and IDA-passivity based control. International Journal of Control 75(9):645–665. https://doi.org/10.1080/0020717021013593 – 10.1080/00207170210135939
• Achiam, Gpt-4 technical report. arXiv preprint (2023)
• Lee, B-pref: Benchmarking preference-based reinforcement learning. Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1) (2021)
• Haarnoja, Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. International conference on machine learning (2018)
• Schulman, High-dimensional continuous control using generalized advantage estimation. International Conference on Learning Representations (ICLR) (2016)
• van den Berg J, Guy SJ, Lin M, Manocha D (2011) Reciprocal n-Body Collision Avoidance. Springer Tracts in Advanced Robotics 3–1 – 10.1007/978-3-642-19457-3_1
• Helbing D, Molnár P (1995) Social force model for pedestrian dynamics. Phys Rev E 51(5):4282–4286. https://doi.org/10.1103/physreve.51.428 – 10.1103/physreve.51.4282
• Bertoni L, Kreiss S, Alahi A (2019) MonoLoco: Monocular 3D Pedestrian Localization and Uncertainty Estimation. 2019 IEEE/CVF International Conference on Computer Vision (ICCV) 6860–687 – 10.1109/iccv.2019.00696
• Vats A, Anastasiu DC (2023) Enhancing Retail Checkout through Video Inpainting, YOLOv8 Detection, and DeepSort Tracking. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) 5530–553 – 10.1109/cvprw59228.2023.00585