Feedback Nash Equilibrium Solutions of Two-Player LQ Differential Games: Synthesis and Analysis via a State/Costate Interpretation

Authors

M. L. Scarpa, B. Nortmann, M. Sassano, T. Mylvaganam

Abstract

Linear quadratic differential games and their feedback Nash equilibrium (F-NE) solutions are considered. First, it is shown that F-NE strategies can be derived from the restriction to an invariant subspace of a system that is reminiscent of the state/costate dynamics arising in the context of open-loop NE solutions. Second, in terms of synthesis, it is shown that the equilibrium subspace can be rendered externally stable via virtual inputs without modifying the underlying F-NE strategies. Building upon these findings, we propose a gradient descent algorithm to determine a solution of the coupled Algebraic Riccati Equations associated with F-NE, which are generally challenging to solve. Finally, in terms of analysis, we show that the F-NE strategy of each player can be interpreted as the output of a passive Port-Controlled Hamiltonian system, and that the behaviour of the original system under the action of the F-NE strategies can be interpreted as an interconnection of these.

Citation

• Journal: IEEE Control Systems Letters
• Year: 2024
• Volume: 8
• Issue:
• Pages: 1451–1456
• Publisher: Institute of Electrical and Electronics Engineers (IEEE)
• DOI: 10.1109/lcsys.2024.3410630

BibTeX

@article{Scarpa_2024,
  title={{Feedback Nash Equilibrium Solutions of Two-Player LQ Differential Games: Synthesis and Analysis via a State/Costate Interpretation}},
  volume={8},
  ISSN={2475-1456},
  DOI={10.1109/lcsys.2024.3410630},
  journal={IEEE Control Systems Letters},
  publisher={Institute of Electrical and Electronics Engineers (IEEE)},
  author={Scarpa, M. L. and Nortmann, B. and Sassano, M. and Mylvaganam, T.},
  year={2024},
  pages={1451--1456}
}

Download the bib file

References

• Başar, Dynamic Noncooperative Game Theory (1999)
• Başar, T. & Bernhard, P. H∞-Optimal Control and Related Minimax Design Problems. (Birkhäuser Boston, 2008). doi:10.1007/978-0-8176-4757-5 – 10.1007/978-0-8176-4757-5
• Mylvaganam, T. & Astolfi, A. A Nash Game Approach to Mixed H2/H∞ Control for Input-Affine Nonlinear Systems. IFAC-PapersOnLine 49, 1024–1029 (2016) – 10.1016/j.ifacol.2016.10.302
• LaValle, S. M. & Hutchinson, S. Game theory as a unifying structure for a variety of robot tasks. Proceedings of 8th IEEE International Symposium on Intelligent Control 429–434 doi:10.1109/isic.1993.397675 – 10.1109/isic.1993.397675
• Cappello, D. et al. A Hybrid Controller for Multi-Agent Collision Avoidance via a Differential Game Formulation. IEEE Trans. Contr. Syst. Technol. 29, 1750–1757 (2021) – 10.1109/tcst.2020.3005602
• Dockner, E. J., Jorgensen, S., Long, N. V. & Sorger, G. Differential Games in Economics and Management Science. (2000) doi:10.1017/cbo9780511805127 – 10.1017/cbo9780511805127
• Wishart, D. Differential Games. A Mathematical Theory with Applications to Warfare and Pursuit, Control and Optimization. Phys. Bull. 17, 60–60 (1966) – 10.1088/0031-9112/17/2/009
• Ho, Y. C. Differential games, dynamic optimization, and generalized control theory. J Optim Theory Appl 6, 179–209 (1970) – 10.1007/bf00926600
• Engwerda, LQ Dynamic Optimization and Differential Games (2005)
• Sassano, M., Mylvaganam, T. & Astolfi, A. (Cyclo-Passive) Port-Controlled Hamiltonian dynamics in LQ differential games. 2021 American Control Conference (ACC) 704–709 (2021) doi:10.23919/acc50511.2021.9483326 – 10.23919/acc50511.2021.9483326
• Sassano, OL-NE for LQ differential games: A port-controlled hamiltonian system perspective and some computational strategies (2024)
• Boyd, S. & Vandenberghe, L. Convex Optimization. (2004) doi:10.1017/cbo9780511804441 – 10.1017/cbo9780511804441
• Ruder, An overview of gradient descent optimization algorithms. arXiv:1609.04747 (2016)
• Santra, S., Hsieh, J.-W. & Lin, C.-F. Gradient Descent Effects on Differential Neural Architecture Search: A Survey. IEEE Access 9, 89602–89618 (2021) – 10.1109/access.2021.3090918
• Starr, A. W. & Ho, Y. C. Nonzero-sum differential games. J Optim Theory Appl 3, 184–206 (1969) – 10.1007/bf00929443
• Sassano, M., Mylvaganam, T. & Astolfi, A. Model-Based Policy Iterations for Nonlinear Systems via Controlled Hamiltonian Dynamics. IEEE Trans. Automat. Contr. 68, 2683–2698 (2023) – 10.1109/tac.2022.3199211
• Engwerda, J. Algorithms for computing Nash equilibria in deterministic LQ games. CMS 4, 113–140 (2006) – 10.1007/s10287-006-0030-z
• Freiling, G., Jank, G. & Abou-Kandil, H. On global existence of solutions to coupled matrix Riccati equations in closed-loop Nash games. IEEE Trans. Automat. Contr. 41, 264–269 (1996) – 10.1109/9.481532
• Li, T.-Y. & Gajic, Z. Lyapunov Iterations for Solving Coupled Algebraic Riccati Equations of Nash Differential Games and Algebraic Riccati Equations of Zero-Sum Games. New Trends in Dynamic Games and Applications 333–351 (1995) doi:10.1007/978-1-4612-4274-1_17 – 10.1007/978-1-4612-4274-1_17
• van der Schaft, A. & Jeltsema, D. Port-Hamiltonian Systems Theory: An Introductory Overview. (2014) doi:10.1561/9781601987877 – 10.1561/9781601987877
• Faulwasser, T. & Kellett, C. M. On continuous-time infinite horizon optimal control—Dissipativity, stability, and transversality. Automatica 134, 109907 (2021) – 10.1016/j.automatica.2021.109907