Authors

Truong Ngoc Cuong, Hwan-Seong Kim, Le Ngoc Bao Long, Sam-Sang You, Nguyen Duy Tan

Abstract

This study explored the challenge of multi-equipment collaborative scheduling and management within container terminals, focusing on essential equipment such as reach stackers, automated guided vehicles (AGVs), quay cranes, and yard cranes. The job shop scheduling problem (JSSP) in container handling was modeled using a quadratic unconstrained binary optimization (QUBO) formulation. A novel scheduling approach is developed based on the synergistic combination of an artificial intelligence (AI) algorithm and quantum computing (QC) (or AI-QC scheduler). This strategy integrates deep reinforcement learning (DRL) and a quantum approximate optimization algorithm (QAOA) to minimize the makespan (total completion time) with accurate coordination of port equipment. Based on operating scenarios in port environments, the proposed AI-QC scheduler offers significant advantages over traditional methods such as first-in-first-out (FIFO), shortest processing time (SPT), and genetic algorithms (GA), particularly in handling stochastic disturbances in processing times. The hybrid technique based on deep learning, quantum computing, and (DRL-QAOA) has self-optimization capabilities, adjusting to changes in market demand in port operations. The AI-QC scheduler can outperform benchmark algorithms in allocating and scheduling port resources, achieving a lower makespan and enhanced operational efficiency across various scenarios. Robust performance indicates optimizing equipment utilization and enhancing terminal throughput, particularly in unpredictable scenarios. Hence, deep-learning AI-driven quantum optimization strategies can improve the competitiveness and efficiency of container terminal operations in a rapidly evolving global market.

Keywords

Job scheduling; Multiple equipment; Container terminal; Deep learning; Quantum optimization algorithm

Citation

  • Journal: Engineering Applications of Artificial Intelligence
  • Year: 2025
  • Volume: 148
  • Issue:
  • Pages: 110431
  • Publisher: Elsevier BV
  • DOI: 10.1016/j.engappai.2025.110431

BibTeX

@article{Cuong_2025,
  title={{Deep learning-enhanced quantum optimization for integrated job scheduling in container terminals}},
  volume={148},
  ISSN={0952-1976},
  DOI={10.1016/j.engappai.2025.110431},
  journal={Engineering Applications of Artificial Intelligence},
  publisher={Elsevier BV},
  author={Cuong, Truong Ngoc and Kim, Hwan-Seong and Bao Long, Le Ngoc and You, Sam-Sang and Tan, Nguyen Duy},
  year={2025},
  pages={110431}
}

Download the bib file

References

  • Ajagekar, A. & You, F. Quantum computing for energy systems optimization: Challenges and opportunities. Energy vol. 179 76–89 (2019) – 10.1016/j.energy.2019.04.186
  • Ansere, Quantum deep reinforcement learning for 6G mobile edge computing-based IoT systems. (2023)
  • Ansere, Quantum deep reinforcement learning for dynamic resource allocation in mobile edge computing-based IoT systems. IEEE Trans. Wireless Commun. (2023)
  • Applegate, D. & Cook, W. A Computational Study of the Job-Shop Scheduling Problem. ORSA Journal on Computing vol. 3 149–156 (1991) – 10.1287/ijoc.3.2.149
  • Arulkumaran, K., Deisenroth, M. P., Brundage, M. & Bharath, A. A. Deep Reinforcement Learning: A Brief Survey. IEEE Signal Processing Magazine vol. 34 26–38 (2017) – 10.1109/msp.2017.2743240
  • Azad, U., Behera, B. K., Ahmed, E. A., Panigrahi, P. K. & Farouk, A. Solving Vehicle Routing Problem Using Quantum Approximate Optimization Algorithm. IEEE Transactions on Intelligent Transportation Systems vol. 24 7564–7573 (2023) – 10.1109/tits.2022.3172241
  • Cheng, B., Xie, T., Wang, L., Tan, Q. & Cao, X. Deep reinforcement learning driven cost minimization for batch order scheduling in robotic mobile fulfillment systems. Expert Systems with Applications vol. 255 124589 (2024) – 10.1016/j.eswa.2024.124589
  • Da Col, Industrial-size job shop scheduling with constraint programming. Oper Res Perspect (2022)
  • Coronato, A., Naeem, M., De Pietro, G. & Paragliola, G. Reinforcement learning for intelligent healthcare applications: A survey. Artificial Intelligence in Medicine vol. 109 101964 (2020) – 10.1016/j.artmed.2020.101964
  • Cuong, T. N., Kim, H.-S., Long, L. N. B. & You, S.-S. Seaport profit analysis and efficient management strategies under stochastic disruptions. Maritime Economics & Logistics vol. 26 212–240 (2023) – 10.1057/s41278-023-00271-z
  • Cuong, T. N., Kim, H.-S., You, S.-S. & Nguyen, D. A. Seaport throughput forecasting and post COVID-19 recovery policy by using effective decision‐making strategy: A case study of Vietnam ports. Computers & Industrial Engineering vol. 168 108102 (2022) – 10.1016/j.cie.2022.108102
  • Denkena, B., Schinkel, F., Pirnay, J. & Wilmsmeier, S. Quantum algorithms for process parallel flexible job shop scheduling. CIRP Journal of Manufacturing Science and Technology vol. 33 100–114 (2021) – 10.1016/j.cirpj.2021.03.006
  • Fattahi, P., Saidi Mehrabad, M. & Jolai, F. Mathematical modeling and heuristic approaches to flexible job shop scheduling problems. Journal of Intelligent Manufacturing vol. 18 331–342 (2007) – 10.1007/s10845-007-0026-8
  • Fitzek, (2021)
  • Fösel, (2021)
  • García-Pérez, G., Rossi, M. A. C. & Maniscalco, S. IBM Q Experience as a versatile experimental testbed for simulating open quantum systems. npj Quantum Information vol. 6 (2020) – 10.1038/s41534-019-0235-y
  • Glover, F., Lewis, M. & Kochenberger, G. Logical and inequality implications for reducing the size and difficulty of quadratic unconstrained binary optimization problems. European Journal of Operational Research vol. 265 829–842 (2018) – 10.1016/j.ejor.2017.08.025
  • Guo, (2024)
  • Kaweegitbundit, P. Evaluation Dispatching Rules for Two-Stage Hybrid Flow Shop Scheduling with Parallel Machines. Applied Mechanics and Materials vols 152–154 1487–1491 (2012) – 10.4028/www.scientific.net/amm.152-154.1487
  • Ku, W.-Y. & Beck, J. C. Mixed Integer Programming models for job shop scheduling: A computational analysis. Computers & Operations Research vol. 73 165–173 (2016) – 10.1016/j.cor.2016.04.006
  • Kurowski, K. et al. Application of quantum approximate optimization algorithm to job shop scheduling problem. European Journal of Operational Research vol. 310 518–528 (2023) – 10.1016/j.ejor.2023.03.013
  • Li, H., Peng, J., Wang, X. & Wan, J. Integrated Resource Assignment and Scheduling Optimization With Limited Critical Equipment Constraints at an Automated Container Terminal. IEEE Transactions on Intelligent Transportation Systems vol. 22 7607–7618 (2021) – 10.1109/tits.2020.3005854
  • Mohammadbagherpoor, (2021)
  • Neumann, N. M. P., de Heer, P. B. U. L. & Phillipson, F. Quantum reinforcement learning. Quantum Information Processing vol. 22 (2023) – 10.1007/s11128-023-03867-9
  • Ng, W. C. & Mak, K. L. Yard crane scheduling in port container terminals. Applied Mathematical Modelling vol. 29 263–276 (2005) – 10.1016/j.apm.2004.09.009
  • Pei, B., Xu, Y. & Wu, J.-L. Stochastic averaging for stochastic differential equations driven by fractional Brownian motion and standard Brownian motion. Applied Mathematics Letters vol. 100 106006 (2020) – 10.1016/j.aml.2019.106006
  • Qing-dao-er-ji, R. & Wang, Y. A new hybrid genetic algorithm for job shop scheduling problem. Computers & Operations Research vol. 39 2291–2299 (2012) – 10.1016/j.cor.2011.12.005
  • Rashidi, H. & Tsang, E. P. K. A complete and an incomplete algorithm for automated guided vehicle scheduling in container terminals. Computers & Mathematics with Applications vol. 61 630–641 (2011) – 10.1016/j.camwa.2010.12.009
  • Saidi-Mehrabad, M., Dehnavi-Arani, S., Evazabadian, F. & Mahmoodian, V. An Ant Colony Algorithm (ACA) for solving the new integrated model of job shop scheduling and conflict-free routing of AGVs. Computers & Industrial Engineering vol. 86 2–13 (2015) – 10.1016/j.cie.2015.01.003
  • Schworm, P., Wu, X., Glatt, M. & Aurich, J. C. Solving flexible job shop scheduling problems in manufacturing with Quantum Annealing. Production Engineering vol. 17 105–115 (2022) – 10.1007/s11740-022-01145-8
  • Singh, How are reinforcement learning and deep learning algorithms used for big data based decision making in financial industries–A review and research agenda. Int J Inf Manag Data Insights (2022)
  • Venturelli, (2015)
  • Wang, L. et al. Dynamic job-shop scheduling in smart manufacturing using deep reinforcement learning. Computer Networks vol. 190 107969 (2021) – 10.1016/j.comnet.2021.107969
  • Wang, Z., Zhang, J. & Yang, S. An improved particle swarm optimization algorithm for dynamic job shop scheduling problems with random job arrivals. Swarm and Evolutionary Computation vol. 51 100594 (2019) – 10.1016/j.swevo.2019.100594
  • Wauters, M. M., Panizon, E., Mbeng, G. B. & Santoro, G. E. Reinforcement-learning-assisted quantum optimization. Physical Review Research vol. 2 (2020) – 10.1103/physrevresearch.2.033446
  • Xin, J., Negenborn, R. R. & Lodewijks, G. Energy-aware control for automated container terminals using integrated flow shop scheduling and optimal control. Transportation Research Part C: Emerging Technologies vol. 44 214–230 (2014) – 10.1016/j.trc.2014.03.014
  • Xue, Z., Zhang, C., Miao, L. & Lin, W.-H. An ant colony algorithm for yard truck scheduling and yard location assignment problems with precedence constraints. Journal of Systems Science and Systems Engineering vol. 22 21–37 (2013) – 10.1007/s11518-013-5210-0
  • Yu, T., Huang, J. & Chang, Q. Optimizing task scheduling in human-robot collaboration with deep multi-agent reinforcement learning. Journal of Manufacturing Systems vol. 60 487–499 (2021) – 10.1016/j.jmsy.2021.07.015
  • Zhou, Quantum approximate optimization algorithm: performance, mechanism, and implementation on near-term devices. Phys. Rev. X (2020)