Designing of robust adaptive passivity-based controller based on reinforcement learning for nonlinear port-Hamiltonian model with disturbance

Authors

A. Gheibi, A. R. Ghiasi, S. Ghaemi, M. A. Badamchizadeh

Abstract

ABSTRACT The passivity-based control (PBC) is not robust and it relies upon the system model. Moreover, partial differential equations (PDE) are encountered during its designing process which are difficult to be solved and in some cases unfeasible. In this article, reinforcement learning (RL) designs the PBC parameters via solving PDE online. RL and adaptive control are employed in order to make the nonlinear closed-loop system robust against the disturbance and model uncertainty. Through the utilisation of adaptive control technique, the passivity-based controller design along with learning could be executed as though the disturbance within the system could also be eliminated. The simulations and the comparison made with the previous methods manifest the greater advantage and superiority of the proposed method.

Citation

• Journal: International Journal of Control
• Year: 2020
• Volume: 93
• Issue: 8
• Pages: 1754–1764
• Publisher: Informa UK Limited
• DOI: 10.1080/00207179.2018.1532607

BibTeX

@article{Gheibi_2018,
  title={{Designing of robust adaptive passivity-based controller based on reinforcement learning for nonlinear port-Hamiltonian model with disturbance}},
  volume={93},
  ISSN={1366-5820},
  DOI={10.1080/00207179.2018.1532607},
  number={8},
  journal={International Journal of Control},
  publisher={Informa UK Limited},
  author={Gheibi, A. and Ghiasi, A. R. and Ghaemi, S. and Badamchizadeh, M. A.},
  year={2018},
  pages={1754--1764}
}

Download the bib file

References

• Al-Tamimi, A., Lewis, F. L. & Abu-Khalaf, M. Model-free Q-learning designs for linear discrete-time zero-sum games with application to H-infinity control. Automatica vol. 43 473–481 (2007) – 10.1016/j.automatica.2006.09.019
• Aracil J.. Proceedings World Automation Congress (2004)
• Astolfi, A., Chhabra, D. & Ortega, R. Asymptotic stabilization of some equilibria of an underactuated underwater vehicle. Systems & Control Letters vol. 45 193–206 (2002) – 10.1016/s0167-6911(01)00176-1
• Barto, A. G., Sutton, R. S. & Anderson, C. W. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man, and Cybernetics vol. SMC-13 834–846 (1983) – 10.1109/tsmc.1983.6313077
• Bertsekas, D. P. Neuro-Dynamic Programming. Encyclopedia of Optimization 2555–2560 (2008) doi:10.1007/978-0-387-74759-0_440 – 10.1007/978-0-387-74759-0_440
• Byrnes, C. I., Isidori, A. & Willems, J. C. Passivity, feedback equivalence, and the global stabilization of minimum phase nonlinear systems. IEEE Transactions on Automatic Control vol. 36 1228–1240 (1991) – 10.1109/9.100932
• Chang, D. E. On the method of interconnection and damping assignment passivity-based control for the stabilization of mechanical systems. Regular and Chaotic Dynamics vol. 19 556–575 (2014) – 10.1134/s1560354714050049
• Dirksz, D. A. & Scherpen, J. M. A. Structure Preserving Adaptive Control of Port-Hamiltonian Systems. IEEE Transactions on Automatic Control vol. 57 2880–2885 (2012) – 10.1109/tac.2012.2192359
• Duindam V.. Modeling and control of complex physical systems: The port-Hamiltonian approach (2014)
• Ernst, D., Glavic, M., Geurts, P. & Wehenkel, L. Approximate Value Iteration in the Reinforcement Learning Context. Application to Electrical Power System Control. International Journal of Emerging Electric Power Systems vol. 3 (2005) – 10.2202/1553-779x.1066
• An experimental comparison of several nonlinear controllers for power converters. IEEE Control Systems vol. 19 66–82 (1999) – 10.1109/37.745771
• Grondman, I., Vaandrager, M., Busoniu, L., Babuska, R. & Schuitema, E. Efficient Model Learning Methods for Actor–Critic Control. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) vol. 42 591–602 (2012) – 10.1109/tsmcb.2011.2170565
• Jiao, X., Shen, T. & Tamura, K. Passivity-based robust feedback control for non-linear systems with input dynamical uncertainty. International Journal of Control vol. 77 517–526 (2004) – 10.1080/00207170410001682498
• Khan, S. G., Herrmann, G., Lewis, F. L., Pipe, T. & Melhuish, C. A Novel Q-Learning Based Adaptive Optimal Controller Implementation for a Humanoid Robotic Arm*. IFAC Proceedings Volumes vol. 44 13528–13533 (2011) – 10.3182/20110828-6-it-1002.02232
• Konidaris G.. AAAI conference on artificial intelligence (2011)
• Liu X.. Proceedings of the American control conference (2000)
• Wei Liu, Ying Tan & Qinru Qiu. Enhanced Q-learning algorithm for dynamic power management with performance constraint. 2010 Design, Automation & Test in Europe Conference & Exhibition (DATE 2010) 602–605 (2010) doi:10.1109/date.2010.5457135 – 10.1109/date.2010.5457135
• Maschke B. M.. IFAC symposia series (1993)
• Mojallizadeh, M. R. & Badamchizadeh, M. A. Adaptive Passivity-Based Control of a Photovoltaic/Battery Hybrid Power Source via Algebraic Parameter Identification. IEEE Journal of Photovoltaics vol. 6 532–539 (2016) – 10.1109/jphotov.2016.2514715
• Nageshrao, S. P., Lopes, G. A. D., Jeltsema, D. & Babuška, R. Interconnection and Damping Assignment Control via Reinforcement Learning. IFAC Proceedings Volumes vol. 47 1760–1765 (2014) – 10.3182/20140824-6-za-1003.01705
• Nageshrao, S. P., Lopes, G. A. D., Jeltsema, D. & Babuška, R. Passivity-based reinforcement learning control of a 2-DOF manipulator arm. Mechatronics vol. 24 1001–1007 (2014) – 10.1016/j.mechatronics.2014.10.005
• Ortega, R., Loría, A., Nicklasson, P. J. & Sira-Ramírez, H. Passivity-Based Control of Euler-Lagrange Systems. Communications and Control Engineering (Springer London, 1998). doi:10.1007/978-1-4471-3603-3 – 10.1007/978-1-4471-3603-3
• Putting energy back in control. IEEE Control Systems vol. 21 18–33 (2001) – 10.1109/37.915398
• Ortega, R. & Spong, M. W. Adaptive motion control of rigid robots: A tutorial. Automatica vol. 25 877–888 (1989) – 10.1016/0005-1098(89)90054-x
• DOI not foun – 10.1002/(sici)1099-1115(199802)12:1<63::aid-acs467>3.0.co;2-#
• Sprangers, O., Babuska, R., Nageshrao, S. P. & Lopes, G. A. D. Reinforcement Learning for Port-Hamiltonian Systems. IEEE Transactions on Cybernetics vol. 45 1017–1027 (2015) – 10.1109/tcyb.2014.2343194
• Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction. IEEE Transactions on Neural Networks vol. 9 1054–1054 (1998) – 10.1109/tnn.1998.712192
• Van Der Schaft, A. J. & Maschke, B. M. On the Hamiltonian formulation of nonholonomic mechanical systems. Reports on Mathematical Physics vol. 34 225–233 (1994) – 10.1016/0034-4877(94)90038-8
• Yang, X., Liu, D. & Wang, D. Reinforcement learning for adaptive optimal control of unknown continuous-time nonlinear systems with input constraints. International Journal of Control vol. 87 553–566 (2013) – 10.1080/00207179.2013.848292