Authors

Stefano Massaroli, Michael Poli, Federico Califano, Jinkyoo Park, Atsushi Yamashita, Hajime Asama

Abstract

We introduce optimal energy shaping as an enhancement of classical passivity-based control methods. A promising feature of passivity theory, alongside stability, has traditionally been claimed to be intuitive performance tuning along the execution of a given task. However, a systematic approach to adjust performance within a passive control framework has yet to be developed, as each method relies on few and problem-specific practical insights. Here, we cast the classic energy-shaping control design process in an optimal control framework; once a task-dependent performance metric is defined, an optimal solution is systematically obtained through an iterative procedure relying on neural networks and gradient-based optimization. The proposed method is validated on state-regulation tasks.

Citation

  • Journal: SIAM Journal on Applied Dynamical Systems
  • Year: 2022
  • Volume: 21
  • Issue: 3
  • Pages: 2126–2147
  • Publisher: Society for Industrial & Applied Mathematics (SIAM)
  • DOI: 10.1137/21m1414279

BibTeX

@article{Massaroli_2022,
  title={{Optimal Energy Shaping via Neural Approximators}},
  volume={21},
  ISSN={1536-0040},
  DOI={10.1137/21m1414279},
  number={3},
  journal={SIAM Journal on Applied Dynamical Systems},
  publisher={Society for Industrial & Applied Mathematics (SIAM)},
  author={Massaroli, Stefano and Poli, Michael and Califano, Federico and Park, Jinkyoo and Yamashita, Atsushi and Asama, Hajime},
  year={2022},
  pages={2126--2147}
}

Download the bib file

References

  • Abadi M., 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16) (2016)
  • Arimoto S., Proceedings of Robotics Research, 1st International Symposium on Robotics Research (1984)
  • Arulkumaran, K., Deisenroth, M. P., Brundage, M. & Bharath, A. A. Deep Reinforcement Learning: A Brief Survey. IEEE Signal Processing Magazine vol. 34 26–38 (2017) – 10.1109/msp.2017.2743240
  • Baydin A. G., J. Mach. Learn. Res. (2017)
  • Berkenkamp F., Advances in Neural Information Processing Systems (2017)
  • Bertsekas D. P., Dynamic Programming and Optimal Control (1995)
  • Bradbury J., JAX: Composable Transformations of Python+NumPy Programs (2018)
  • Buşoniu, L., de Bruin, T., Tolić, D., Kober, J. & Palunko, I. Reinforcement learning for control: Performance, stability, and deep approximators. Annual Reviews in Control vol. 46 8–28 (2018) – 10.1016/j.arcontrol.2018.09.005
  • Byrnes, C. I., Isidori, A. & Willems, J. C. Passivity, feedback equivalence, and the global stabilization of minimum phase nonlinear systems. IEEE Transactions on Automatic Control vol. 36 1228–1240 (1991) – 10.1109/9.100932
  • Chen R. T., Advances in Neural Information Processing Systems (2018)
  • Cranmer M., Lagrangian Neural Networks, preprint, https://arxiv.org/abs/2003.04630 (2020)
  • Dimeas, F. & Aspragathos, N. Online Stability in Human-Robot Cooperation with Admittance Control. IEEE Transactions on Haptics vol. 9 267–278 (2016) – 10.1109/toh.2016.2518670
  • Dormand, J. R. & Prince, P. J. A family of embedded Runge-Kutta formulae. Journal of Computational and Applied Mathematics vol. 6 19–26 (1980) – 10.1016/0771-050x(80)90013-3
  • Duindam, V., Macchelli, A., Stramigioli, S. & Bruyninckx, H. Modeling and Control of Complex Physical Systems. (Springer Berlin Heidelberg, 2009). doi:10.1007/978-3-642-03196-0 – 10.1007/978-3-642-03196-0
  • Dulac-Arnold G., Challenges of Real-World Reinforcement Learning, preprint, https://arxiv.org/abs/1904.12901 (2019)
  • Glorot X., Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (2010)
  • Grathwohl W., FFJORD: Free-Form Continuous Dynamics for Scalable Reversible Generative Models, preprint, https://arxiv.org/abs/1810.01367 (2018)
  • Greydanus S., Advances in Neural Information Processing Systems (2019)
  • Groothuis S. S., Front. Robotics AI (2018)
  • Innes M., Fashionable Modelling with Flux, preprint, https://arxiv.org/abs/1811.01457 (2018)
  • Kidger P., Neural Controlled Differential Equations for Irregular Time Series, preprint, https://arxiv.org/abs/2005.08926 (2020)
  • Kingma D. P., Adam: A Method for Stochastic Optimization, preprint, https://arxiv.org/abs/1412.6980 (2014)
  • Kölsch L., Optimal Control of Port-Hamiltonian Systems: A Time-Continuous Learning Approach, preprint, https://arxiv.org/abs/2007.08645 (2020)
  • Lutter M., Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learning, preprint, https://arxiv.org/abs/1907.04490 (2019)
  • Massaroli S., 1st IFAC Workshop on Robot Control (WROCO) (2019)
  • Massaroli S., Stable Neural Flows, preprint, https://arxiv.org/abs/2003.08063 (2020)
  • Massaroli, S. et al. Port–Hamiltonian Approach to Neural Network Training. 2019 IEEE 58th Conference on Decision and Control (CDC) 6799–6806 (2019) doi:10.1109/cdc40024.2019.9030017 – 10.1109/cdc40024.2019.9030017
  • Massaroli S., Dissecting Neural ODEs, preprint, https://arxiv.org/abs/2002.08071 (2020)
  • Nageshrao, S. P., Lopes, G. A. D., Jeltsema, D. & Babuska, R. Port-Hamiltonian Systems in Adaptive and Learning Control: A Survey. IEEE Transactions on Automatic Control vol. 61 1223–1238 (2016) – 10.1109/tac.2015.2458491
  • Ortega, R., van der Schaft, A., Castanos, F. & Astolfi, A. Control by Interconnection and Standard Passivity-Based Control of Port-Hamiltonian Systems. IEEE Transactions on Automatic Control vol. 53 2527–2542 (2008) – 10.1109/tac.2008.2006930
  • Putting energy back in control. IEEE Control Systems vol. 21 18–33 (2001) – 10.1109/37.915398
  • Paszke A., Advances in Neural Information Processing Systems (2019)
  • Poli M., TorchDyn: A Neural Differential Equations Library, preprint, https://arxiv.org/abs/2009.09346 (2020)
  • Pontryagin L. S., The Mathematical Theory of Optimal Processes, translated from the Russian by K. N. Trirogoff, edited by L. W. Neustadt (1962)
  • Robbins, H. & Monro, S. A Stochastic Approximation Method. The Annals of Mathematical Statistics vol. 22 400–407 (1951) – 10.1214/aoms/1177729586
  • Secchi C., STAR 29 (2007)
  • Smyrlis, G. & Zisis, V. Local convergence of the steepest descent method in Hilbert spaces. Journal of Mathematical Analysis and Applications vol. 300 436–453 (2004) – 10.1016/j.jmaa.2004.06.051
  • Sprangers, O., Babuska, R., Nageshrao, S. P. & Lopes, G. A. D. Reinforcement Learning for Port-Hamiltonian Systems. IEEE Transactions on Cybernetics vol. 45 1017–1027 (2015) – 10.1109/tcyb.2014.2343194
  • Stramigioli S., Modeling and IPC Control of Interactive Mechanical Systems—A Coordinate-Free Approach (2001)
  • Stramigioli S., Mathematical Control Theory I
  • Sutton R. S., Reinforcement Learning: An Introduction (2018)
  • A. J. Van der Schaft, $L_{2}$-Gain and Passivity Techniques in Nonlinear Control (2000)
  • Vos, E., Scherpen, J. M. A., Schaft, A. J. van der & Postma, A. Formation Control of Wheeled Robots in the Port-Hamiltonian Framework. IFAC Proceedings Volumes vol. 47 6662–6667 (2014) – 10.3182/20140824-6-za-1003.00394
  • Zhong Y. D., Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control, preprint, https://arxiv.org/abs/1909.12077 (2019)
  • Zhong Y. D., Benchmarking Energy-Conserving Neural Networks for Learning Dynamics from Data, preprint, https://arxiv.org/abs/2012.02334 (2020)
  • Zhong Y. D., Dissipative SymODEN: Encoding Hamiltonian Dynamics with Dissipation and Control into Deep Learning, preprint, https://arxiv.org/abs/2002.08860 (2020)
  • Zhong Y. D., Advances in Neural Information Processing Systems (2020)
  • Zhuang J., Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODE, preprint, https://arxiv.org/abs/2006.02493 (2020)