Home⇛Isabela State University Linker: Journal of Engineering, Computing and Technology⇛vol. 2 no. 1 (2025)

HadouQen: Adaptive AI Agent Using Reinforcement Learning in Street Fighter II: Special Champion Edition

Isaiah Phil Pangilinan | Neo Alaric B. Villanueva | Irish Paulo R. Tipay | Audrey Lyle D. Diego

Discipline: Artificial Intelligence

 

Abstract:

This study presents the development of an AI agent trained using Proximal Policy Optimization (PPO) to compete in Street Fighter II: Special Champion Edition. The agent learned optimal combat strategies through reinforcement learning, processing visual input from frame-stacked grayscale observations (84 × 84 pixels) obtained through the OpenAI Gym Retro environment. Using a convolutional neural network architecture with carefully tuned hyperparameters, the model was trained across 16 parallel environments over 100 million timesteps. The agent was tested against M. Bison, the game's final boss and most challenging opponent, across 1,000 consecutive matches to evaluate performance. Results showed exceptional performance with a 96.7%-win rate and an average reward of 0.912. Training metrics revealed a healthy learning progression, showing steady improvement in average reward per episode, decreased episode length indicating more efficient victories, and stable policy convergence. The findings also demonstrate the effectiveness of PPO-based reinforcement learning in mastering complex fighting game environments and provide a foundation for future research in competitive game-playing agents capable of human-level performance in fast-paced interactive scenarios.



References:

  1. Aamir, A., Tamosiunaite, M., & Wörgötter, F. (2021). Caffe2Unity: Immersive visualization and interpretation of deep neural networks. Electronics, 11(1), 83. https://doi.org/10.3390/electronics11010083
  2. Almeida, P., Carvalho, V., & Simões, A. (2024). Reinforcement learning as an approach to train multiplayer first-person shooter game agents. Technologies, 12(3), 34. https://doi.org/10.3390/technologies12030034
  3. Alonso, E., Peter, M., Goumard, D., & Romoff, J. (2021). Deep reinforcement learning for navigation in AAA video games. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence (IJCAI-21) (pp. 2133–2139). International Joint Conferences on Artificial Intelligence Organization. https://doi.org/10.24963/ijcai.2021/294
  4. Andrychowicz, M., Raichuk, A., Stańczyk, P., Orsini, M., Girgin, S., Marinier, R., Hussenot, L., Geist, M., Pietquin, O., Michalski, M., Gelly, S., & Bachem, O. (2021). What matters for on-policy deep actor-critic methods? A large-scale study. OpenReview. https://openreview.net/forum?id=nIAxjsniDzg
  5. Ashktorab, Z., Liao, Q. V., Dugan, C., Johnson, J., Pan, Q., Zhang, W., Kumaravel, S., & Campbell, M. (2020). Human-AI collaboration in a cooperative game setting. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW2), 1–20. https://doi.org/10.1145/3415167
  6. Berner, C., Brockman, G., Chan, B., Cheung, V., Debiak, P., Dennison, C., Farhi, D., Fischer, Q., Hashme, S., Hesse, C., Józefowicz, R., Gray, S., Olsson, C., Pachocki, J., Petrov, M., De Oliveira Pinto, H. P., Raiman, J., Salimans, T., Schlatter, J., & Zhang, S. (2019). Dota 2 with large scale deep reinforcement learning. arXiv. https://doi.org/10.48550/arxiv.1912.06680
  7. Clifton, J., & Laber, E. (2020). Q-Learning: Theory and applications. Annual Review of Statistics and Its Application, 7(1), 279–301. https://doi.org/10.1146/annurev-statistics-031219-041220
  8. Dong, S., Wang, P., & Abbas, K. (2021). A survey on deep learning and its applications. Computer Science Review, 40. https://doi.org/10.1016/j.cosrev.2021.100379
  9. Gallotta, R., Todd, G., Zammit, M., Earle, S., Liapis, A., Togelius, J., & Yannakakis, G. N. (2024). Large language models and games: A survey and roadmap. arXiv. https://arXiv.org/abs/2402.18659
  10. Goldwaser, A., & Thielscher, M. (2020). Deep reinforcement learning for general game playing. In Proceedings of the AAAI Conference on Artificial Intelligence, 34(2), 1701–1708. https://doi.org/10.1609/aaai.v34i02.5533
  11. Halina, E., & Guzdial, M. (2022). Diversity-based deep reinforcement learning towards multidimensional difficulty for fighting game AI. arXiv. https://arXiv.org/abs/2211.02759
  12. Hazra, T., & Anjaria, K. (2022). Applications of game theory in deep learning: A survey. Multimedia Tools and Applications, 81(6), 8963–8994. https://doi.org/10.1007/s11042-022-12153-2
  13. Hu, C., Zhao, Y., Wang, Z., Du, H., & Liu, J. (2023). Games for artificial intelligence research: A review and perspectives. arXiv. https://arXiv.org/abs/2304.13269
  14. Janiesch, C., Zschech, P., & Heinrich, K. (2021). Machine learning and deep learning. Electronic Markets, 31(3), 685–695. https://doi.org/10.1007/s12525-021-00475-2
  15. Jeyakumar, J. V., Noor, J., Cheng, Y., Garcia, L., & Srivastava, M. (2020). How can I explain this to you? An empirical study of deep neural network explanation methods. In Advances in Neural Information Processing Systems (NeurIPS).
  16. Joo, H., & Kim, K. (2019). Visualization of deep reinforcement learning using Grad-CAM: How AI plays Atari games? In 2019 IEEE Conference on Games (CoG) (pp. 1–8). IEEE. https://doi.org/10.1109/cig.2019.8847950
  17. Li, S. E. (2023). Deep reinforcement learning. In Reinforcement Learning for Sequential Decision and Optimal Control (pp. 227–255). Springer. https://doi.org/10.1007/978-981-19-7784-8_10
  18. Osborn, J. C., Lederle-Ensign, D., Wardrip-Fruin, N., & Mateas, M. (2023). Combat in games. eScholarship. https://escholarship.org/uc/item/9zj6r5wz
  19. Samek, W., Montavon, G., Lapuschkin, S., Anders, C. J., & Müller, K. (2021). Explaining deep neural networks and beyond: A review of methods and applications. Proceedings of the IEEE, 109(3), 247–278. https://doi.org/10.1109/jproc.2021.3060483
  20. Shao, K., Tang, Z., Zhu, Y., Li, N., & Zhao, D. (2019). A survey of deep reinforcement learning in video games. arXiv. https://arXiv.org/abs/1912.10944
  21. Simonov, A., Zagarskikh, A., & Fedorov, V. (2019). Applying behavior characteristics to decision-making process to create believable game AI. Procedia Computer Science, 156, 404–413. https://doi.org/10.1016/j.procs.2019.08.222
  22. Taherdoost, H. (2023). Deep learning and neural networks: Decision-making implications. Symmetry, 15(9), 1723. https://doi.org/10.3390/sym15091723
  23. Wang, D., Weisz, J. D., Muller, M., Ram, P., Geyer, W., Dugan, C., Tausczik, Y., Samulowitz, H., & Gray, A. (2019). Human-AI collaboration in data science: Exploring data scientists’ perceptions of automated AI. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), Article 211, 1–24. https://doi.org/10.1145/3359313
  24. Yin, Q., Yang, J., Huang, K., Zhao, M., Ni, W., Liang, B., Huang, Y., Wu, S., & Wang, L. (2023). AI in human-computer gaming: Techniques, challenges and opportunities. Machine Intelligence Research, 20(3), 299–317. https://doi.org/10.1007/s11633-022-1384-6

Tools


Copyright © 2025  KITE Digital Educational Solutions |  Exclusively distributed by CE-Logic