Proposal of Deterministic Policy Gradient Method for Port-Hamiltonian Systems Using Eligibility Trace and Verification by Numerical Experiment
Authors
Shuichi FUKUNAGA, Ryota KOKUBO
Abstract
This paper proposes a deterministic policy gradient method for port-Hamiltonian systems using an eligibility trace. The deterministic policy gradient method commonly uses one of two types of algorithms, either the on- or off-policy method. The proposed algorithm employs the off-policy method to perform a probabilistic search. In addition, we introduce an eligibility trace to the method to speed up the learning process. A numerical simulation shows the effectiveness of the proposed method.
Citation
- Journal: Transactions of the Society of Instrument and Control Engineers
- Year: 2023
- Volume: 59
- Issue: 4
- Pages: 232–234
- Publisher: The Society of Instrument and Control Engineers
- DOI: 10.9746/sicetr.59.232
BibTeX
@article{FUKUNAGA_2023,
title={{Proposal of Deterministic Policy Gradient Method for Port-Hamiltonian Systems Using Eligibility Trace and Verification by Numerical Experiment}},
volume={59},
ISSN={1883-8189},
DOI={10.9746/sicetr.59.232},
number={4},
journal={Transactions of the Society of Instrument and Control Engineers},
publisher={The Society of Instrument and Control Engineers},
author={FUKUNAGA, Shuichi and KOKUBO, Ryota},
year={2023},
pages={232--234}
}References
- Nian, R., Liu, J. & Huang, B. A review On reinforcement learning: Introduction and applications in industrial process control. Computers & Chemical Engineering 139, 106886 (2020) – 10.1016/j.compchemeng.2020.106886
- Sprangers, O., Babuska, R., Nageshrao, S. P. & Lopes, G. A. D. Reinforcement Learning for Port-Hamiltonian Systems. IEEE Trans. Cybern. 45, 1017–1027 (2015) – 10.1109/tcyb.2014.2343194
- 3) T. Morimura, E. Uchibe and K. Doya: Utilizing the natural gradient in temporal difference reinforcement learning with eligibility traces, Proc. 2nd Int. Symp. Information Geometry and its Applications, 256/263 (2005)
- 4) 福永,岩本:自然勾配を用いたポートハミルトン系のための強化学習の高速化,計測自動制御学会論文集,59-2, 70/76 (2023)
- 5) D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra and M. Riedmiller: Deterministic Policy Gradient Algorithms, Proc. 31st Int. Conf. Mach. Learn., 387/395 (2014)