Authors

Shuopeng Wang, Rixin Wang, Yanhui Liu, Ying Zhang, Lina Hao

Abstract

Pneumatic artificial muscles (PAMs), as typical soft actuators characterized by hysteresis and nonlinearity, pose a challenging task in modeling and control. This paper proposes a Deep Lagrangian Networks Reinforcement Learning (DeLaNRL) controller that combines deep Lagrangian networks (DeLaN) with reinforcement learning to achieve precise motion control of PAMs. By leveraging the DeLaN model, the dynamic model is constrained to adhere to the Lagrangian first principle, enhancing the model’s compliance with physical constraints. Furthermore, to improve the generality and adaptability of the model to various input data, the Self-scalable tanh (Stan) function is employed as the activation function within the DeLaN model. To validate the effectiveness of the proposed modeling approach, the model is tested on both sampled and unknown motions. The results demonstrate the effectiveness and generalization capability of the DeLaN model with the Stan activation function. Subsequently, the reinforcement learning controller is applied to the learned dynamics model, resulting in control strategies capable of precise motion control. To further demonstrate the effectiveness of the proposed controller, experiments are conducted on both simulation and the experiment platform for reaching and tracking tasks. The simulation results indicate that the control error is less than 0.91 millimeters, while on the experimental platform, the control error is less than 3.7 millimeters. These results confirm that the proposed DeLaNRL controller exhibits high control performance.

Keywords

Physics-informed neural networks; Lagrangian dynamics; Pneumatic artificial muscles; Reinforcement learning; Deep Lagrangian networks

Citation

  • Journal: Engineering Applications of Artificial Intelligence
  • Year: 2025
  • Volume: 148
  • Issue:
  • Pages: 110406
  • Publisher: Elsevier BV
  • DOI: 10.1016/j.engappai.2025.110406

BibTeX

@article{Wang_2025,
  title={{Dynamic modeling and control of pneumatic artificial muscles via Deep Lagrangian Networks and Reinforcement Learning}},
  volume={148},
  ISSN={0952-1976},
  DOI={10.1016/j.engappai.2025.110406},
  journal={Engineering Applications of Artificial Intelligence},
  publisher={Elsevier BV},
  author={Wang, Shuopeng and Wang, Rixin and Liu, Yanhui and Zhang, Ying and Hao, Lina},
  year={2025},
  pages={110406}
}

Download the bib file

References

  • Alessi, C., Bianchi, D., Stano, G., Cianchetti, M. & Falotico, E. Pushing with Soft Robotic Arms via Deep Reinforcement Learning. Advanced Intelligent Systems vol. 6 (2024) – 10.1002/aisy.202300899
  • Antonelo, E. A. et al. Physics-informed neural nets for control of dynamical systems. Neurocomputing vol. 579 127419 (2024) – 10.1016/j.neucom.2024.127419
  • Armanini, C., Boyer, F., Mathew, A. T., Duriez, C. & Renda, F. Soft Robots Modeling: A Structured Overview. IEEE Transactions on Robotics vol. 39 1728–1748 (2023) – 10.1109/tro.2022.3231360
  • Cao, J. et al. Prediction of Chemical Looping Hydrogen Production Using Physics-Informed Machine Learning. Energy & Fuels vol. 38 19929–19938 (2024) – 10.1021/acs.energyfuels.4c02988
  • Carvalho, F. de C. T., Nath, K., Serpa, A. L. & Karniadakis, G. E. Learning characteristic parameters and dynamics of centrifugal pumps under multiphase flow using physics-informed neural networks. Engineering Applications of Artificial Intelligence vol. 138 109378 (2024) – 10.1016/j.engappai.2024.109378
  • Centurelli, A. et al. Closed-Loop Dynamic Control of a Soft Manipulator Using Deep Reinforcement Learning. IEEE Robotics and Automation Letters vol. 7 4741–4748 (2022) – 10.1109/lra.2022.3146903
  • Chen, X. The computational design of a fractal-inspired soft robotic. Alexandria Engineering Journal vol. 84 37–46 (2023) – 10.1016/j.aej.2023.10.056
  • Cui, Y., Matsubara, T. & Sugimoto, K. Pneumatic artificial muscle-driven robot control using local update reinforcement learning. Advanced Robotics vol. 31 397–412 (2017) – 10.1080/01691864.2016.1274680
  • Cuomo, S. et al. Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next. Journal of Scientific Computing vol. 92 (2022) – 10.1007/s10915-022-01939-z
  • Delcey, (2024)
  • Faria, A data-driven tracking control framework using physics-informed neural networks and deep reinforcement learning for dynamical systems. Eng. Appl. Artif. Intell. (2024)
  • Gnanasambandam, R., Shen, B., Chung, J., Yue, X. & Kong, Z. Self-Scalable Tanh (Stan): Multi-Scale Solutions for Physics-Informed Neural Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence vol. 45 15588–15603 (2023) – 10.1109/tpami.2023.3307688
  • Goto, T. et al. Characteristics verification of pneumatic artificial muscles for compressive loading. Advanced Robotics vol. 37 887–899 (2023) – 10.1080/01691864.2023.2226200
  • Han, Robust learning-based control for uncertain nonlinear systems with validation on a soft robot. IEEE Trans. Neural Netw. Learn. Syst. (2023)
  • Hong-Gui Han, Xiao-Long Wu & Jun-Fei Qiao. Real-Time Model Predictive Control Using a Self-Organizing Neural Network. IEEE Transactions on Neural Networks and Learning Systems vol. 24 1425–1436 (2013) – 10.1109/tnnls.2013.2261574
  • Hošovský, A. et al. Dynamic characterization and simulation of two-link soft robot arm with pneumatic muscles. Mechanism and Machine Theory vol. 103 98–116 (2016) – 10.1016/j.mechmachtheory.2016.04.013
  • Huang, K. et al. Knowledge-Informed Neural Network for Nonlinear Model Predictive Control With Industrial Applications. IEEE Transactions on Systems, Man, and Cybernetics: Systems vol. 54 2241–2253 (2024) – 10.1109/tsmc.2023.3341031
  • Kothera, C. S., Jangid, M., Sirohi, J. & Wereley, N. M. Experimental Characterization and Static Modeling of McKibben Actuators. Journal of Mechanical Design vol. 131 (2009) – 10.1115/1.3158982
  • Li, L. et al. Deep Reinforcement Learning in Soft Viscoelastic Actuator of Dielectric Elastomer. IEEE Robotics and Automation Letters vol. 4 2094–2100 (2019) – 10.1109/lra.2019.2898710
  • Liang, Dynamic modeling and analysis for dual pneumatic artificial muscle actuated manipulators. (2019)
  • Liang, D. et al. Energy-Based Motion Control for Pneumatic Artificial Muscle Actuated Robots With Experiments. IEEE Transactions on Industrial Electronics vol. 69 7295–7306 (2022) – 10.1109/tie.2021.3095788
  • Liu, Physics-informed neural networks to model and control robots: A theoretical and experimental investigation. Adv. Intell. Syst. (2024)
  • Liu, M., Liang, L. & Sun, W. A generic physics-informed neural network-based constitutive model for soft biological tissues. Computer Methods in Applied Mechanics and Engineering vol. 372 113402 (2020) – 10.1016/j.cma.2020.113402
  • Lutter, M. & Peters, J. Combining physics and deep learning to learn continuous-time dynamics models. The International Journal of Robotics Research vol. 42 83–107 (2023) – 10.1177/02783649231169492
  • Naughton, N. et al. Elastica: A Compliant Mechanics Environment for Soft Robotic Control. IEEE Robotics and Automation Letters vol. 6 3389–3396 (2021) – 10.1109/lra.2021.3063698
  • Nazeer, M. S., Laschi, C. & Falotico, E. RL-Based Adaptive Controller for High Precision Reaching in a Soft Robot Arm. IEEE Transactions on Robotics vol. 40 2498–2512 (2024) – 10.1109/tro.2024.3381558
  • Qiu, J., Sun, K., Wang, T. & Gao, H. Observer-Based Fuzzy Adaptive Event-Triggered Control for Pure-Feedback Nonlinear Systems With Prescribed Performance. IEEE Transactions on Fuzzy Systems vol. 27 2152–2162 (2019) – 10.1109/tfuzz.2019.2895560
  • Raissi, M., Perdikaris, P. & Karniadakis, G. E. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics vol. 378 686–707 (2019) – 10.1016/j.jcp.2018.10.045
  • Relano, Gaussian process regression for forward and inverse kinematics of a soft robotic arm. Eng. Appl. Artif. Intell. (2023)
  • Sanyal, S. & Roy, K. RAMP-Net: A Robust Adaptive MPC for Quadrotors via Physics-informed Neural Network. 2023 IEEE International Conference on Robotics and Automation (ICRA) 1019–1025 (2023) doi:10.1109/icra48891.2023.10161410 – 10.1109/icra48891.2023.10161410
  • Schegg, P. et al. SofaGym: An Open Platform for Reinforcement Learning Based on Soft Robot Simulations. Soft Robotics vol. 10 410–430 (2023) – 10.1089/soro.2021.0123
  • Song, C., Xie, S., Zhou, Z. & Hu, Y. Modeling of pneumatic artificial muscle using a hybrid artificial neural network approach. Mechatronics vol. 31 124–131 (2015) – 10.1016/j.mechatronics.2015.04.021
  • Thuruthel, T. G., Falotico, E., Renda, F. & Laschi, C. Model-Based Reinforcement Learning for Closed-Loop Dynamic Control of Soft Robotic Manipulators. IEEE Transactions on Robotics vol. 35 124–134 (2019) – 10.1109/tro.2018.2878318
  • Tondu, B. Modelling of the McKibben artificial muscle: A review. Journal of Intelligent Material Systems and Structures vol. 23 225–253 (2012) – 10.1177/1045389x11435435
  • Wang, Online incremental dynamic modeling using physics-informed long short-term memory networks for the pneumatic artificial muscle. IEEE Robot. Autom. Lett. (2024)
  • Yang, X., Du, Y., Li, L., Zhou, Z. & Zhang, X. Physics-Informed Neural Network for Model Prediction and Dynamics Parameter Identification of Collaborative Robot Joints. IEEE Robotics and Automation Letters vol. 8 8462–8469 (2023) – 10.1109/lra.2023.3329620
  • Yang, J., Su, J., Li, S. & Yu, X. High-Order Mismatched Disturbance Compensation for Motion Control Systems Via a Continuous Dynamic Sliding-Mode Approach. IEEE Transactions on Industrial Informatics vol. 10 604–614 (2014) – 10.1109/tii.2013.2279232
  • Yao, J. et al. Adaptive Actuation of Magnetic Soft Robots Using Deep Reinforcement Learning. Advanced Intelligent Systems vol. 5 (2023) – 10.1002/aisy.202200339
  • Yoon, T. et al. Kinematics-Informed Neural Networks: Enhancing Generalization Performance of Soft Robot Model Identification. IEEE Robotics and Automation Letters vol. 9 3068–3075 (2024) – 10.1109/lra.2024.3362644
  • Zhang, J., Chen, X., Stegagno, P. & Yuan, C. Nonlinear Dynamics Modeling and Fault Detection for a Soft Trunk Robot: An Adaptive NN-Based Approach. IEEE Robotics and Automation Letters vol. 7 7534–7541 (2022) – 10.1109/lra.2022.3184034
  • Zhang, Y., Liu, H., Ma, T., Hao, L. & Li, Z. A comprehensive dynamic model for pneumatic artificial muscles considering different input frequencies and mechanical loads. Mechanical Systems and Signal Processing vol. 148 107133 (2021) – 10.1016/j.ymssp.2020.107133
  • Zhang, Disturbance preview-based output feedback predictive control for pneumatic artificial muscle robot systems with hysteresis compensation. IEEE- ASME Trans. Mechatronics (2024)
  • Zhao, L., Liu, X. & Wang, T. Trajectory tracking control for double-joint manipulator systems driven by pneumatic artificial muscles based on a nonlinear extended state observer. Mechanical Systems and Signal Processing vol. 122 307–320 (2019) – 10.1016/j.ymssp.2018.12.016