>>On one hand, this is a testament to the power of contemporary machine learning methods and the latest computer chips to process vast amounts of data. On the other, it’s a reminder of how fundamentally unintelligent AI agents are.
forgive any ignorance, but how do changes to the game affect AI behavior? E.g. New characters, new maps, balance changes, bug fixes, etc. Do the AI have to be re-trained in a similar amount time, or is it expedited? How adaptive are the A.I. systems?
What I'm getting at is - in a real world scenario (like the city transportation example from the article), how quickly can the A.I. respond to things unexpected changes (natural disasters, accidents, etc)? Will they be able to handle the unexpected in a safe time frame?
Depends on the program, of course. You're asking how robust the trained model is to outliers and how effective it is at extrapolation. Different training algorithms have different behaviors.
A Reinforcement Learning machine would have to be re-trained for changes such as that, but not from scratch. Depending on the size of the change, it would take much less time to adapt to such a change.
Several notes:
-The article notes that a lot of the hard parts of human video-game playing have been done 'for' the AI by hardcoding. It doesn't actually have to look at the screen to parse information--it has direct access to game variables that player would have to access through menus. More relevant to your question, a lot of the strategic decisions like item purchasing, ward placement, and probably character placement were just picked by the programmers or set to be ignored. The whole thing is less impressive than the headline sounds. I think it was just learning--where to run on the map and when to attack things?
-RL is based on deep learning, and there are fundamental issues with deep learning's ability to adapt to genuinely new scenarios. None of these system can presently adapt to something genuinely unprecendeted in a time frame you would consider 'safe'. They need at least several opportunities to observe how the world works after the changes and the consequences of their actions. To try to make this concrete--they can't reason about what implications a flood/power outage/landslide has for their traffic management. They can only learn from trial and error <-(the important part) how traffic behaves during a disaster.
False. Reinforcement Learning is a category of algorithm. There are many possible implementations, not only "deep learning". Further, you might be surprised at how well a generalized, trained model might mimic "reasoning".
A great achievement even for a game which is relatively simplistic like a MOBA. Still waiting for AI to become competitive in Starcraft which i think is a much greater challenge.
An impossible to beat Starcraft AI is already doable without all this AI training. Just by controlling every individual unit seperatly in impossible to counter maneuvers far beyond any human actions per minute.
You could basically get an AI to win every single time with a handful of zerglings in a few minutes this way.
You could train an AI for 1 billion years a day to play pong or you could hardcode it to never miss the ball.
Mostly speculation and observations of many hours in Starcraft. I don't think anyone has really set out to create an AI where the only goal is to never lose. I can imagine a program that can take one unit and program it to always remain 1 pixel farther than the maximum distance any enemy can shoot it unless for some reason it knows with 100% certainty it will win an engagement.
I'm not aware of any public api or hooks into controlling Starcraft or it would have probably been done by now. It would also be used for cheating so they can't make that anyway.
I like your example of pong; because it illustrates how a simple problem of 'AI winning' can spiral out of control into a nightmare of work.
On the other hand. Pong is simple with 1 rule. Starcraft, or in this case Dota, have multiple facets to win.
Plus, most SC players just gg out of the game when they know they've lost. I'd think that is a more apt use for AI. Like chess, see out X moves; if it's a lost cause; call it, and get to the queue faster for another game.
Somewhat misleading title: they're playing a restricted set of DotA 2 (only 5 heroes instead of 100+, some mechanics turned off), and some important decisions are handled by hard-coded rules rather than the main AI.
13 comments
[ 85.8 ms ] story [ 368 ms ] threadforgive any ignorance, but how do changes to the game affect AI behavior? E.g. New characters, new maps, balance changes, bug fixes, etc. Do the AI have to be re-trained in a similar amount time, or is it expedited? How adaptive are the A.I. systems?
What I'm getting at is - in a real world scenario (like the city transportation example from the article), how quickly can the A.I. respond to things unexpected changes (natural disasters, accidents, etc)? Will they be able to handle the unexpected in a safe time frame?
Several notes:
-The article notes that a lot of the hard parts of human video-game playing have been done 'for' the AI by hardcoding. It doesn't actually have to look at the screen to parse information--it has direct access to game variables that player would have to access through menus. More relevant to your question, a lot of the strategic decisions like item purchasing, ward placement, and probably character placement were just picked by the programmers or set to be ignored. The whole thing is less impressive than the headline sounds. I think it was just learning--where to run on the map and when to attack things?
-RL is based on deep learning, and there are fundamental issues with deep learning's ability to adapt to genuinely new scenarios. None of these system can presently adapt to something genuinely unprecendeted in a time frame you would consider 'safe'. They need at least several opportunities to observe how the world works after the changes and the consequences of their actions. To try to make this concrete--they can't reason about what implications a flood/power outage/landslide has for their traffic management. They can only learn from trial and error <-(the important part) how traffic behaves during a disaster.
False. Reinforcement Learning is a category of algorithm. There are many possible implementations, not only "deep learning". Further, you might be surprised at how well a generalized, trained model might mimic "reasoning".
You could basically get an AI to win every single time with a handful of zerglings in a few minutes this way.
You could train an AI for 1 billion years a day to play pong or you could hardcode it to never miss the ball.
I think a smart player could know the player was going Zerg, and wall in.. and make some decisions that make the micro of Zerg less useful.
I'm not aware of any public api or hooks into controlling Starcraft or it would have probably been done by now. It would also be used for cheating so they can't make that anyway.
On the other hand. Pong is simple with 1 rule. Starcraft, or in this case Dota, have multiple facets to win.
Plus, most SC players just gg out of the game when they know they've lost. I'd think that is a more apt use for AI. Like chess, see out X moves; if it's a lost cause; call it, and get to the queue faster for another game.