Port-Hamiltonian Systems in Adaptive and Learning Control: A Survey

Authors

Subramanya P. Nageshrao, Gabriel A. D. Lopes, Dimitri Jeltsema, Robert Babuska

Abstract

Port-Hamiltonian (PH) theory is a novel, but well established modeling framework for nonlinear physical systems. Due to the emphasis on the physical structure and modular framework, PH modeling has become a prime focus in system theory. This has led to a considerable research interest in the control of PH systems, resulting in numerous nonlinear control techniques. General nonlinear control methodologies are classified in a spectrum from model-based to model-free, where adaptation and learning typically lie close to the end of the range. Various articles and monographs have provided a detailed overview of model-based control techniques on PH models, but no survey is specifically dedicated to the learning and adaptive control methods that can benefit from the PH structure. To this end, we provide a comprehensive review of the current learning and adaptive control methodologies that have been adapted specifically to PH systems. After establishing the required theoretical background, we elaborate on various general machine learning, iterative learning, and adaptive control techniques and their application to PH systems. For each method we highlight the changes from the general setting due to PH model, followed by a detailed presentation of the respective control algorithm. In general, the advantages of using PH models in learning and adaptive controllers are: i) Prior knowledge in the form of PH model speeds up the learning. ii) In some instances new stability or convergence guarantees are obtained by having a PH model. iii) The resulting control laws can be interpreted in the context of physical systems. We conclude the paper with notes on open research issues.

Citation

• Journal: IEEE Transactions on Automatic Control
• Year: 2016
• Volume: 61
• Issue: 5
• Pages: 1223–1238
• Publisher: Institute of Electrical and Electronics Engineers (IEEE)
• DOI: 10.1109/tac.2015.2458491

BibTeX

@article{Nageshrao_2016,
  title={{Port-Hamiltonian Systems in Adaptive and Learning Control: A Survey}},
  volume={61},
  ISSN={1558-2523},
  DOI={10.1109/tac.2015.2458491},
  number={5},
  journal={IEEE Transactions on Automatic Control},
  publisher={Institute of Electrical and Electronics Engineers (IEEE)},
  author={Nageshrao, Subramanya P. and Lopes, Gabriel A. D. and Jeltsema, Dimitri and Babuska, Robert},
  year={2016},
  pages={1223--1238}
}

Download the bib file

References

• frueh, Iterative learning control with basis functions (2000)
• Fujimoto, K., Sakurama, K. & Sugie, T. Trajectory tracking control of port-controlled Hamiltonian systems via generalized canonical transformations. Automatica vol. 39 2059–2069 (2003) – 10.1016/j.automatica.2003.07.005
• Wang, Z. & Goldsmith, P. Modified energy-balancing-based control for the tracking problem. IET Control Theory & Applications vol. 2 310–322 (2008) – 10.1049/iet-cta:20070124
• álvarez, Port controller Hamiltonian synthesis using evolution strategies. Dynamics Bifurcations and Control (2002)
• Fujimoto, K. & Koyama, I. Iterative Feedback Tuning for Hamiltonian Systems. IFAC Proceedings Volumes vol. 41 15678–15683 (2008) – 10.3182/20080706-5-kr-1001.02651
• Sprangers, O., Babuska, R., Nageshrao, S. P. & Lopes, G. A. D. Reinforcement Learning for Port-Hamiltonian Systems. IEEE Transactions on Cybernetics vol. 45 1017–1027 (2015) – 10.1109/tcyb.2014.2343194
• Dirksz, D. A. & Scherpen, J. M. A. Power-based adaptive and integral control of standard mechanical systems. 49th IEEE Conference on Decision and Control (CDC) 4612–4617 (2010) doi:10.1109/cdc.2010.5717079 – 10.1109/cdc.2010.5717079
• Wei, A. & Wang, Y. Adaptive control of uncertain port-controlled Hamiltonian systems subject to actuator saturation. International Journal of Control, Automation and Systems vol. 9 1067–1073 (2011) – 10.1007/s12555-011-0606-4
• Wang, Y., Feng, G. & Cheng, D. Simultaneous stabilization of a set of nonlinear port-controlled Hamiltonian systems. Automatica vol. 43 403–415 (2007) – 10.1016/j.automatica.2006.09.008
• Dirksz, D. A. & Scherpen, J. M. A. Structure Preserving Adaptive Control of Port-Hamiltonian Systems. IEEE Transactions on Automatic Control vol. 57 2880–2885 (2012) – 10.1109/tac.2012.2192359
• Sun, Y. Z., Liu, Q. J., Song, Y. H. & Shen, T. L. Hamiltonian modelling and nonlinear disturbance attenuation control of TCSC for improving power system stability. IEE Proceedings - Control Theory and Applications vol. 149 278–284 (2002) – 10.1049/ip-cta:20020399
• Eiben, A. E. & Smith, J. E. Introduction to Evolutionary Computing. Natural Computing Series (Springer Berlin Heidelberg, 2003). doi:10.1007/978-3-662-05094-1 – 10.1007/978-3-662-05094-1
• Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers. IEEE Control Systems vol. 32 76–105 (2012) – 10.1109/mcs.2012.2214134
• Kober, J. & Peters, J. Reinforcement Learning in Robotics: A Survey. Adaptation, Learning, and Optimization 579–610 (2012) doi:10.1007/978-3-642-27645-3_18 – 10.1007/978-3-642-27645-3_18
• Busoniu, L., Babuska, R., De Schutter, B. & Ernst, D. Reinforcement Learning and Dynamic Programming Using Function Approximators. (2017) doi:10.1201/9781439821091 – 10.1201/9781439821091
• Vrabie, D. & Lewis, F. Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems. Neural Networks vol. 22 237–246 (2009) – 10.1016/j.neunet.2009.03.008
• Grondman, I., Busoniu, L., Lopes, G. A. D. & Babuska, R. A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) vol. 42 1291–1307 (2012) – 10.1109/tsmcc.2012.2218595
• Beard, R. W., Saridis, G. N. & Wen, J. T. Approximate Solutions to the Time-Invariant Hamilton–Jacobi–Bellman Equation. Journal of Optimization Theory and Applications vol. 96 589–626 (1998) – 10.1023/a:1022664528457
• Grondman, I., Vaandrager, M., Busoniu, L., Babuska, R. & Schuitema, E. Efficient Model Learning Methods for Actor–Critic Control. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) vol. 42 591–602 (2012) – 10.1109/tsmcb.2011.2170565
• Nageshrao, S. P., Lopes, G. A. D., Jeltsema, D. & Babuška, R. Passivity-based reinforcement learning control of a 2-DOF manipulator arm. Mechatronics vol. 24 1001–1007 (2014) – 10.1016/j.mechatronics.2014.10.005
• vrabie, Optimal Adaptive Control and Differential Games by Reinforcement Learning Principles (2013)
• van der Schaft, A. & Jeltsema, D. Port-Hamiltonian Systems Theory: An Introductory Overview. Foundations and Trends® in Systems and Control vol. 1 173–378 (2014) – 10.1561/2600000002
• Longman, R. W. & Huang, Y.-C. The Phenomenon of Apparent Convergence Followed by Divergence in Learning and Repetitive Control. Intelligent Automation & Soft Computing vol. 8 107–128 (2002) – 10.1080/10798587.2002.10644210
• wen, Bridging learning control and repetitive control using basis functions. Proceedings of the AAS/AIAA Space Flight Mechanics Meeting (0)
• j åström, Adaptive Control (2013)
• khalil, Nonlinear Systems (2002)
• ioannou, Robust Adaptive Control (2012)
• Iterative Learning Control. (Springer US, 1998). doi:10.1007/978-1-4615-5629-9 – 10.1007/978-1-4615-5629-9
• slotine, Applied nonlinear control (1991)
• Wang, Y., Gao, F. & Doyle, F. J., III. Survey on iterative learning control, repetitive control, and run-to-run control. Journal of Process Control vol. 19 1589–1600 (2009) – 10.1016/j.jprocont.2009.09.006
• A survey of iterative learning control. IEEE Control Systems vol. 26 96–114 (2006) – 10.1109/mcs.2006.1636313
• Beyer, H.-G. & Schwefel, H.-P. Evolution strategies – A comprehensive introduction. Natural Computing vol. 1 3–52 (2002) – 10.1023/a:1015059928466
• Hjalmarsson, H. Iterative feedback tuning—an overview. International Journal of Adaptive Control and Signal Processing vol. 16 373–395 (2002) – 10.1002/acs.714
• Fujimoto, K. & Scherpen, J. M. A. Balanced Realization and Model Order Reduction for Nonlinear Systems Based on Singular Value Analysis. SIAM Journal on Control and Optimization vol. 48 4591–4623 (2010) – 10.1137/070695332
• Satoh, S., Fujimoto, K. & Hyon, S.-H. Gait Generation for a Hopping Robot Via Iterative Learning Control Based on Variational Symmetry. Lecture Notes in Control and Information Sciences 197–208 doi:10.1007/978-3-540-73890-9_15 – 10.1007/978-3-540-73890-9_15
• A˚ström, K. J., Aracil, J. & Gordillo, F. A family of smooth controllers for swinging up a pendulum. Automatica vol. 44 1841–1848 (2008) – 10.1016/j.automatica.2007.10.040
• Iterative feedback tuning: theory and applications. IEEE Control Systems vol. 18 26–41 (1998) – 10.1109/37.710876
• Hjalmarsson, H., Gunnarsson, S. & Gevers, M. A convergent iterative restricted complexity control design scheme. Proceedings of 1994 33rd IEEE Conference on Decision and Control vol. 2 1735–1740 – 10.1109/cdc.1994.411185
• Li Cuiyan, Zhang Doagchun & Zhuang Xianyi. A survey of repetitive control. 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566) vol. 2 1160–1166 – 10.1109/iros.2004.1389553
• Kiyasu, Y., Fujimoto, K. & Sugie, T. ITERATIVE LEARNING CONTROL OF NONHOLONOMIC HAMILTONIAN SYSTEMS: APPLICATION TO A VEHICLE SYSTEM. IFAC Proceedings Volumes vol. 38 1–6 (2005) – 10.3182/20050703-6-cz-1902.00221
• Fujimoto, K., Horiuchi, T. & Sugie, T. Optimal control of Hamiltonian systems with input constraints via iterative learning. 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475) 4387–4392 doi:10.1109/cdc.2003.1272188 – 10.1109/cdc.2003.1272188
• fujimoto, Optimal control of Hamiltonian systems via iterative learning. Proc SICE Annu Conf (0)
• Satoh, S., Fujimoto, K. & Hyon, S.-H. Biped gait generation via iterative learning control including discrete state transitions. IFAC Proceedings Volumes vol. 41 1729–1734 (2008) – 10.3182/20080706-5-kr-1001.00296
• Koopman, J. & Jeltsema, D. Casimir-Based Control Beyond the Dissipation Obstacle. IFAC Proceedings Volumes vol. 45 173–177 (2012) – 10.3182/20120829-3-it-4022.00046
• Putting energy back in control. IEEE Control Systems vol. 21 18–33 (2001) – 10.1109/37.915398
• fujimoto, On iterative learning control of nonholonomic Hamiltonian systems. Proc 16th Math Theory Netw Syst Symp (0)
• Ortega, R., van der Schaft, A., Maschke, B. & Escobar, G. Interconnection and damping assignment passivity-based control of port-controlled Hamiltonian systems. Automatica vol. 38 585–596 (2002) – 10.1016/s0005-1098(01)00278-3
• Ortega, R. & García-Canseco, E. Interconnection and Damping Assignment Passivity-Based Control: A Survey. European Journal of Control vol. 10 432–450 (2004) – 10.3166/ejc.10.432-450
• Fujimoto, K. & Sugie, T. Iterative learning control of hamiltonian systems: I/O based optimal control approach. IEEE Transactions on Automatic Control vol. 48 1756–1761 (2003) – 10.1109/tac.2003.817908
• Jacob, B. & Zwart, H. J. Linear Port-Hamiltonian Systems on Infinite-Dimensional Spaces. (Springer Basel, 2012). doi:10.1007/978-3-0348-0399-1 – 10.1007/978-3-0348-0399-1
• Willems, J. C. Dissipative dynamical systems part I: General theory. Archive for Rational Mechanics and Analysis vol. 45 321–351 (1972) – 10.1007/bf00276493
• strang, Introduction to Linear Algebra (2003)
• Dirksz, D. A. & Scherp, J. M. A. Adaptive tracking control of fully actuated port-Hamiltonian mechanical systems. 2010 IEEE International Conference on Control Applications 1678–1683 (2010) doi:10.1109/cca.2010.5611301 – 10.1109/cca.2010.5611301
• landau, Adaptive Control Algorithms Analysis and Applications (2011)
• van der Schaft, A. L2 - Gain and Passivity Techniques in Nonlinear Control. Communications and Control Engineering (Springer London, 2000). doi:10.1007/978-1-4471-0507-7 – 10.1007/978-1-4471-0507-7
• sutton, Reinforcement Learning An Introduction (1998)
• schaft, Port-Hamiltonian systems: An introductory survey. Proc Int Congress Math (0)
• van der schaft, Port-Hamiltonian systems: Network modeling and control of nonlinear physical systems. Adv Dyn Control Struct Mach CISM Courses Lectures (2004)
• Fujimoto, K. & Sugie, T. Canonical transformation and stabilization of generalized Hamiltonian systems. Systems & Control Letters vol. 42 217–227 (2001) – 10.1016/s0167-6911(00)00091-8
• Duindam, V., Macchelli, A., Stramigioli, S. & Bruyninckx, H. Modeling and Control of Complex Physical Systems. (Springer Berlin Heidelberg, 2009). doi:10.1007/978-3-642-03196-0 – 10.1007/978-3-642-03196-0
• scherpen, On adjoints and singular value functions for nonlinear systems. Proc Conf Inform Sci Syst (0)
• Ortega, R., van der Schaft, A., Castanos, F. & Astolfi, A. Control by Interconnection and Standard Passivity-Based Control of Port-Hamiltonian Systems. IEEE Transactions on Automatic Control vol. 53 2527–2542 (2008) – 10.1109/tac.2008.2006930
• Nijmeijer, H. & van der Schaft, A. Nonlinear Dynamical Control Systems. (Springer New York, 1990). doi:10.1007/978-1-4757-2101-0 – 10.1007/978-1-4757-2101-0
• Fujimoto, K. & Sugie, T. ON ADJOINTS OF HAMILTONIAN SYSTEMS. IFAC Proceedings Volumes vol. 35 7–12 (2002) – 10.3182/20020721-6-es-1901.00251
• Fujimoto, K. & Sherpen, J. M. A. Eigenstructure of nonlinear hankel operators. Lecture Notes in Control and Information Sciences 385–397 doi:10.1007/bfb0110228 – 10.1007/bfb0110228
• Fujimoto, K. & Satoh, S. Repetitive control of Hamiltonian systems based on variational symmetry. Systems & Control Letters vol. 60 763–770 (2011) – 10.1016/j.sysconle.2011.04.025
• Yuqian Guo & Daizhan Cheng. Stabilization of time-varying Hamiltonian systems. IEEE Transactions on Control Systems Technology vol. 14 871–880 (2006) – 10.1109/tcst.2006.879979
• Slotine, J.-J. E. & Li Weiping. Adaptive manipulator control: A case study. IEEE Transactions on Automatic Control vol. 33 995–1003 (1988) – 10.1109/9.14411
• Yamakita, M. & Furuta, K. Iterative Generation of Virtual Reference for a Manipulator. Robotica vol. 9 71–80 (1991) – 10.1017/s0263574700015587
• tan, Extremum seeking from 1922 to 2010. 29th Chinese Control Conf (CCC) (0)