Improved Performance for the DC-AC Converters Control System Based on PCH Controller and Reinforcement Learning Agent

Authors

Abstract

Starting from the classical structure of a three-phase voltage DC-AC converter whose basic controller is designed based on the PI-type control law, this article shows the structure of a DC-AC converter control system (CCS) based on the Port Controlled Hamiltonian (PCH) controller, along with the improvement of DC-AC CCS performance by means of machine learning (ML) strategy. Among these strategies, the most suitable for process control is reinforcement learning (RL), and the RL Twin-Delayed Deep Deterministic Policy Gradient (TD3) agent was chosen from the concrete implementations. The control structures and the synthesis of the PCH control law based on passivity theory are presented, and, in addition, the creation and training of an RL-TD3 agent is presented. Through numerical simulations it is proved the improvement in the DC-AC CCS performance in case of using the RL-TD3 agent in terms of the performance indicators of the control systems, of which we mention: response time, steady-state error, ripple, but also in terms of the quality of electricity according to the Total Harmonic Distortion (THD) analysis.

Citation

Journal: 2022 4th Global Power, Energy and Communication Conference (GPECOM)
Year: 2022
Volume:
Issue:
Pages:
Publisher: IEEE
DOI: 10.1109/gpecom55404.2022.9815661

BibTeX

@inproceedings{Nicola_2022,
  title={{Improved Performance for the DC-AC Converters Control System Based on PCH Controller and Reinforcement Learning Agent}},
  DOI={10.1109/gpecom55404.2022.9815661},
  booktitle={{2022 4th Global Power, Energy and Communication Conference (GPECOM)}},
  publisher={IEEE},
  author={Nicola, Marcel and Nicola, Claudiu-Ionel},
  year={2022}
}

Download the bib file

References

Bansal, H. et al. Port-Hamiltonian formulation of two-phase flow models. Systems & Control Letters 149, 104881 (2021) – 10.1016/j.sysconle.2021.104881
Zhao, Y., Yu, H. & Wang, S. Development of Optimized Cooperative Control Based on Feedback Linearization and Error Port-Controlled Hamiltonian for Permanent Magnet Synchronous Motor. IEEE Access 9, 141036–141047 (2021) – 10.1109/access.2021.3119625
Liu, X., Yu, H., Yu, J. & Zhao, Y. A Novel Speed Control Method Based on Port-Controlled Hamiltonian and Disturbance Observer for PMSM Drives. IEEE Access 7, 111115–111123 (2019) – 10.1109/access.2019.2934987
Shchur, I., Lis, M. & Biletskyi, Y. Passivity-Based Control of Water Pumping System Using BLDC Motor Drive Fed by Solar PV Array with Battery Storage System. Energies 14, 8184 (2021) – 10.3390/en14238184
Reis, T. & Willems, J. C. A balancing approach to the realization of systems with internal passivity and reciprocity. Systems & Control Letters 60, 69–74 (2011) – 10.1016/j.sysconle.2010.10.009
Belkhier, Y. et al. Robust interconnection and damping assignment energy-based control for a permanent magnet synchronous motor using high order sliding mode approach and nonlinear observer. Energy Reports 8, 1731–1740 (2022) – 10.1016/j.egyr.2021.12.075
User’s Guide Matlab and Simulink MathWorks Natick MA USA (2020)
Brandimarte, P. From Shortest Paths to Reinforcement Learning. EURO Advanced Tutorials on Operational Research (Springer International Publishing, 2021). doi:10.1007/978-3-030-61867-4 – 10.1007/978-3-030-61867-4
sutton, Reinforcement learning An introduction Second edition (2018)
Wu, H., Jia, Y., Yang, F., Zhu, L. & Xing, Y. Two-Stage Isolated Bidirectional DC–AC Converters With Three-Port Converters and Two DC Buses. IEEE J. Emerg. Sel. Topics Power Electron. 8, 4428–4439 (2020) – 10.1109/jestpe.2019.2936145
Nayak, P. & Rajashekara, K. An Asymmetrical Space Vector PWM Scheme for a Three Phase Single-stage DC-AC Converter. 2019 IEEE Energy Conversion Congress and Exposition (ECCE) 635–639 (2019) doi:10.1109/ecce.2019.8912742 – 10.1109/ecce.2019.8912742
Sayed, M. A., Takeshita, T. & Kitagawa, W. Advanced PWM Switching Technique for Accurate Unity Power Factor of Bidirectional Three-Phase Grid-Tied DC–AC Converters. IEEE Trans. on Ind. Applicat. 55, 7614–7627 (2019) – 10.1109/tia.2019.2919596
Kobayashi, N., Hayashi, Y., Iyasu, S. & Handa, Y. Fast Current Control of the Single-phase DC-AC Converter Using Digital Peak Current Mode Control. 2019 21st European Conference on Power Electronics and Applications (EPE ’19 ECCE Europe) P.1-P.7 (2019) doi:10.23919/epe.2019.8914814 – 10.23919/epe.2019.8914814
MATLAB Central File Exchange (2021)
Kobayashi, N., Hayashi, Y., Iyasu, S. & Handa, Y. Digital Peak Current Mode Control Method for the Single-phase Bi-directional DC-AC Converter. 2021 23rd European Conference on Power Electronics and Applications (EPE’21 ECCE Europe) P.1-P.8 (2021) doi:10.23919/epe21ecceeurope50061.2021.9570656 – 10.23919/epe21ecceeurope50061.2021.9570656
AC–DC Converters. Real‐Time Electromagnetic Transient Simulation of AC–DC Networks 301–375 (2021) doi:10.1002/9781119819035.ch8 – 10.1002/9781119819035.ch8
abu-rub, AC–DC–AC Converters for Distributed Power Generation Systems. Power Electronics for Renewable Energy Systems Transportation and Industrial Applications (2014)
Serra, F. M., Fernández, L. M., Montoya, O. D., Gil-González, W. & Hernández, J. C. Nonlinear Voltage Control for Three-Phase DC-AC Converters in Hybrid Systems: An Application of the PI-PBC Method. Electronics 9, 847 (2020) – 10.3390/electronics9050847