Hybrid Control Based on Deep Reinforcement Learning and Dynamic Window Approach for Robotic Navigation

https://doi.org/10.58571/CNCA.AMCA.2025.038

Resumen: This work presents a hybrid mobile robot navigation system for simulated environments. The system integrates the Dynamic Window Approach (DWA) with a deep reinforcement learning (DRL) using an actor-critic architecture. Initially, the robot uses DWA, while concurrently, a DRL agent generates alternative control speeds using actor and critic networks. An entropy-based mechanism dynamically weights the DWA and DRL speeds, transitioning gradually to DRL control. The system's performance is evaluated using success rate and trajectory length, comparing the hybrid approach to DWA and DRL alone. Results demonstrate improved robustness and performance, particularly in complex scenarios.

¿Cómo citar?

Vasquez-Jalpa, C., Nakano, N. & Velasco-Villa, M. (2025). Hybrid Control Based on Deep Reinforcement Learning and Dynamic Window Approach for Robotic Navigation. Memorias del Congreso Nacional de Control Automático 2025, pp. 221-226. https://doi.org/10.58571/CNCA.AMCA.2025.038

Palabras clave

Robotic navigation, Deep reinforcement learning, Entropy, Dynamic window approach, Hybrid control, Mobile robot.

• An, Z., Weixiang, W., Wenhao, B., Zhanjun, B. (2024). A path planing method based on deep reinforcement learning for AUV in complex marine environment. Ocean Engineering, 313, 119354. https://doi.org/10.1016/j.oceaneng.2024.119354
• Bodong, T. and Jae-Hoon, K. (2024). Deep reinforcement learning-based local path planning in Dynamic environments for mobile robot. Journal of King Saud University – Computer and Information Sciences¸ 36, https://doi.org/10.1016/j.jksuci.2024.102254
• Ebrahim, S., Said, A., Hitham, A., Safwan, A., Alawi, A., Mohammed, R., Suliman, F. (2024). Deep deterministic policy gradient algorithm: A systematic review. Heliyon, 10. https://doi.org/10.1016/j.heliyon.2024.e30697
• Fanfan, S., Bofan, Y., Jun, Z., Chao, X., Yong, C., Yanxiag, H. (2024). TD3-based trajectory optimization for energy consumption minimization in UAV-assisted MEC system. Computer Networks, 255, 110882. https://doi.org/10.1016/j.comnet.2024.110882
• Haisen, G., Zhigang, R., Jialun, L., Zongze, W., Shengli, X. (2023). Optimal navigation for AGVs: A soft actor -critic based reinforcement learning approach with composite auxiliary rewards. Engineering Applications of Artificial Intelligence, 124, 106613. https://doi.org/10.1016/j.engappai.2023.106613
• Husman, A., Oscar, A. (2024). Optimized TD3 algorithm for robust autonomous navigation in crowded and Dynamic human-interaction environments. Results in Engineering, 24, 102874. https://doi.org/10.1016/j.rineng.2024.102874
• Jinding, Z., Kai, Z., Zhongzheng, W., Wensheng, Z., Chen, L., Liming, Z., Xiaopeng, M., Piyang, L., Ziwei, B., Jinzheng, K., Yongfei, Y., Jun, Y. (2024). A latent space method with maximum entropy deep reinforcement learning for data assimilation. Geoenergy Science and Engineering, 243. https://doi.org/10.1016/j.geoen.2024.213275
• Laiyi, Y., Jing, B., Haitao, Y. (2022). Dynamic path planning for mobile robots with deep reinforcement learning. IFAC PapersOnLine, 55-11. 10.1016/j.ifacol.2022.08.042
• Liang, G., Te, S., Xudong, L., Ke, L., Natalia, D., David, F., Zhengfeng, Z., Junping, Z. (2020). Demostration Guided Acctor-Critic Deep Reinforcement Learning for Fast Teaching of Robots in Dynamic Environments. IFAC PapersOnLine, 53-5, 271-278. 10.1016/j.ifacol.2021.04.227
• Shuhuan, W., Yili, S., Ahmad, R., Zeteng W., Zhengzheng, G., Simeng, G. (2025). A deep residual reinforcement learning algorithm based on Soft Actor-Critic for autonomous navigation. Expert Systems Whit Applications, 259. https://doi.org/10.1016/j.eswa.2024.125238
• Xiaoyu, G., Jiayu, Y., Shuai, L., Hengwei, L. (2022). Actorcritic with familiarity-based trajectory experience replay. Information Sciences, 582, 633-647. https://doi.org/10.1016/j.ins.2021.10.031
• Yanjie, C., Norzalilah, N. (2024). An improved Dynamic window approach algorithm for dynamic obstacle avoidance in mobile robot formation. Decision Analytics Journal, 11. https://doi.org/10.1016/j.dajour.2024.100471