RSUSSH 2020
IN20-140 Developing a Dynamic Decision Making of an Enemy Character in a Tower Defense Game using Reinforcement Learning Technique in Unity ML-Agents
Presenter: Ekapong Nopawong
Rangsit University, Thailand
Abstract
A tower defense game refers to a game that player have to protect a tower and attack an enemy character. The enemy characters will response to fight the players. However, the characters cannot decide to choose a new direction movement when occurring in a new situation. Thus, the objective of this research is to develop the dynamic decision making for an enemy character in a tower defense game using reinforcement learning technique in the Unity ML-Agents. The learning is designed as the lesson learning from the lesson zero to the lesson three called the curriculum learning. Therefore, agents need to learn from the curriculum learning by receiving a cumulative reward score and deleting scores from punishment when learning errors occur. The results show that the learning of lesson zero to lesson three has the growth rate from the cumulative reward score. Furthermore, the wrong decision making of the agents has the regressive rate between lesson zero to lesson three from the value loss score. These two scores mean that the enemy characters can dynamically decide to choose direction movement in an environment.
Citation format:
Nopawong, E., & Praditsangthong, R.. (2020). Developing a Dynamic Decision Making of an Enemy Character in a Tower Defense Game using Reinforcement Learning Technique in Unity ML-Agents. Proceeding in RSU International Research Conference, May 1, 2020. Pathum Thani, Thailand.QUESTIONS & ANSWERS
Thank you Ajarn Ekapong for your interesting presentation. Your game looks really fun! And I think that curriculum learning is a good strategy for ML. I have a few questions:
1) What are the feedbacks of the players after you have implemented the ML with curriculum learning?
2) Can you change the level of learning according to the level of difficulty in the game?
3) Can it be possible that the ai agent becomes very intelligent and therefore, invincible for the players?
I was wondering if you could explain more details about how "the curriculum learning" comes from? That's because of my understanding, ML requires massive data sets to train on, and those should be inclusive and unbiased. Do you follow this way?
Answer for the participant
1) What are the feedbacks of the players after you have implemented the ML with curriculum learning?
ANS. We have no feedback yet because we are in the process of developing a game that uses results from ml agents to replace the original AI.
2) Can you change the level of learning according to the level of difficulty in the game?
ANS. t is possible to change variables that determine the character's ability to perform at different levels, for example walking speed, health, armor etc.
3) Can it be possible that the ai agent becomes very intelligent and therefore, invincible for the players?
ANS. Although the ml agent can perform a limited number of actions, depending on what is possible or not. But there is a chance that it will find a way to do anything under that limitation to overcome us.
Answer for the visitor
I was wondering if you could explain more details about how "the curriculum learning" comes from? That's because of my understanding, ML requires massive data sets to train on, and those should be inclusive and unbiased. Do you follow this way?
ANS. In this research, we use the Unity ML Agent, which uses reinforcement learning in conjunction with curriculum learning to help the ML Agent learning become more efficient and faster. When we want to apply the ML Agent to solve complex problems, we may not need to use the curriculum learning with the ML Agent. However, we want the ML Agent to solve problems that are straightforward or with few steps, such as walking to a destination. But when walking to a destination on the way, there are many obstacles to overcome; we will use curriculum learning to gradually increase the difficulty of not having any obstacles on the way. Too many obstacles, so that the ML Agent can learn slowly.
Answer for the participant
1) What are the feedbacks of the players after you have implemented the ML with curriculum learning?
ANS. We have no feedback yet because we are in the process of developing a game that uses results from ml agents to replace the original AI.
2) Can you change the level of learning according to the level of difficulty in the game?
ANS. t is possible to change variables that determine the character's ability to perform at different levels, for example walking speed, health, armor etc.
3) Can it be possible that the ai agent becomes very intelligent and therefore, invincible for the players?
ANS. Although the ml agent can perform a limited number of actions, depending on what is possible or not. But there is a chance that it will find a way to do anything under that limitation to overcome us.
Answer for the visitor
I was wondering if you could explain more details about how "the curriculum learning" comes from? That's because of my understanding, ML requires massive data sets to train on, and those should be inclusive and unbiased. Do you follow this way?
ANS. In this research, we use the Unity ML Agent, which uses reinforcement learning in conjunction with curriculum learning to help the ML Agent learning become more efficient and faster. When we want to apply the ML Agent to solve complex problems, we may not need to use the curriculum learning with the ML Agent. However, we want the ML Agent to solve problems that are straightforward or with few steps, such as walking to a destination. But when walking to a destination on the way, there are many obstacles to overcome; we will use curriculum learning to gradually increase the difficulty of not having any obstacles on the way. Too many obstacles, so that the ML Agent can learn slowly.