Looks like they are going to limit the APM of the AI. I wonder how they are going to decide the limit? I've never played StarCraft, but from what I understand very high APM is needed to play the game at the highest levels.
I've followed the professional StarCraft 2 scene for a long time now. There were a few players who were famous for having high APMs, here is an over-the-sholder view of one of them [1]. I don't have a reference but believe he used to play in the ~500APM range. That would probably be the upper limit of a human's ability.
I wouldn't be surprised if a 500APM player is making closer to 100 meaningful actions per minute. It looks like most of the actions are spam that may register on the APM meter but not affect the outcome of the game.
An AI limited to even 200 decision-driven actions per minute would probably have a significant advantage over a "500APM" player.
probably limit it to human levels. 400 to 600 apm is around the limit for brief periods of time. It makes for an interesting optimization problem with how to use them though.
High apm is required, yes. But most people push it by pushing it through spamming 1-2-3 (groups of your army). There was a pro player who didnt do it and he was around +- 70 APM.
It would be interesting to see what percentage of a pro player's APM is actually useful. Like you said, most of the time they are spamming buttons to keep their momentum up.
There is a replay analyzer that attempts to figure this out, generally the effective APM will go up during a fight, but the players will keep the APM fairly high even when the eAPM is not high just because it is easier to go from many useless actions to many useful actions than few useful actions to many useful actions.
There's probably no reason to restrict the computer the below human pro players (200-300 APM). My guess is the engine just wasn't designed with superhuman APM in mind and could be exploited. If that's the case then the AI would in some sense not really be playing the same game as the humans.
Buts it's intentionally dumbed down to keep the game fun, loosing all the time would get frustrating real quick. Theres no ROI for AI devs to kick the humans ass (except for maybe now with our AI/Cloud/VR/ craze).
Superhuman APM makes it a really different game. Current Starcraft AIs can do thousands of commands per second. That means that essentially each unit has its own AI and can act independently of all the other units. There's almost no delay between different unit commands.
Unlimited APM makes AI much less interesting from the perspective of game theory.
Think of it like a robotic boxer, except instead of an android, they build two 30 foot long walls of spring-loaded boxing gloves that close in on the human boxer. Yes, the robot punches the guy a lot, so the problem seems "solved" but it isn't really.
There was a Starcraft brood war AI competition that did not have a limit on APM.
The top contestants were employing strategies that required APM an order of magnitude higher than what players could do. For example, in brood war, SCVs can repair any Terran building or vehicle but it is not worth the effort to repair goliaths or tanks for a player.
However, the AI could easily manage repairing it's tanks which makes the Terran army much more cost effective.
In SCII there is auto repair and other improved UX that probably limit the ability of novel strategies like repairing vehicles.
If you slow it down, it would be apparent to you too. High-level human players do this kind of thing to, just not this fast. But effectively, it's watching where the attacking unit is facing and it knows when the next attack is coming, so it knows what the viable targets are and can react when it becomes obvious.
(it's also possible the system is receiving information directly from the game engine, but it's definitely possible to figure this out from nothing but the information on screen)
Because the tanks are using the default unit AI, which AFAIK amounts to "when your weapon is ready to fire, shoot the closest enemy in range". The zergling AI knows the attack range and rate of each tank so can ensure that the closest zergling is always on its own when a tank is ready to fire.
What would be really interesting is putting a similar smart AI on the tanks, which could make decisions like shooting at the highest concentration of zerglings instead of the closest one, or holding its fire if one lone zergling is in range, but a clump of them is about to enter range.
Human players already do both optimizations with widow mines, which are a unit that takes several seconds to prepare its shot so it's feasible to retarget. It's not feasible for humans to do this with tanks though as they fire immediately and automatically.
You could even see counterevolution as two AI algorithms become better at predicting what the other will do.
Game theory dictates that you shouldn't anticipate what your opponent is likely to do, but rather assume that they will chose the best choice available. Given that you need to move the zerglings before the tank fires the tank should be able to retarget and always hit the highest concentration. That being said organizing your lings to run in a formation to never expose any to splash shouldn't be too hard.
Back when I played SC2, you really didn't need a high APM to succeed, unlike SC1. SC2 offered some helpful mechanics that SC1 did not, such as the ability to select infinite units at once, or give commands to multiple structures at one. I think Sheth, a high level Zerg player, was famous for having under 100 APM.
That may be an ok definition for you, but it most certainly isn't for SC2. I wasn't anywhere near the pro level and I would peak at 300 apm and would spend most of the game around 200. 12 APM is borderline not playing.
Consider it a statement about what I'm looking for in a game. I'm not really interested in competing over who can work their mouse button harder, or in driving myself to excel in giving myself repetitive motion injuries. Those are things that take away from starcraft; they don't enhance it.
I would strongly encourage you to go against your preconceived idea of what StarCraft is (a game where people compete on who's the fastest to bash their keys) and see a professional game and all the strategy it entails, such as in the semi-finals and grand final that will be on today.
Pro players do have valid reasons for having such a high APM (even if a portion of it, maybe 30%, is just "spamming" to keep theirselves active and ready), but you can be an excellent player with rather low APM (~120 maybe?). Why would you need to do 2 action per second? A lot of it is simply building units periodically (e.g pressing 1 then "S" every 10 seconds) but then APM really gets high when there is a battle and you need to give actions to multiple sets of units to get a good engagement.
Sheth did not have under 100 apm. He was infamous for not having a high apm in the early game (people like to spam keys in early game for many reasons). He still had over 200 apm once the game got going. This is on the low end for pros in sc2.
> We’re particularly pleased that the environment we’ve worked with Blizzard to construct will be open and available to all researchers next year.
This is awesome. I've only ever reached the Platinum league in Starcraft II (1v1), but I'd almost feel more driven to create bots to (hopefully) surpass that skill level, than actually playing the game.
It's going to happen anyway. I think we are better off doing such things in the open and having some visibility into it rather than be surprised when something happens.
The problem is we is not everyone... it's only those who agree with you. Everyone else will just keep on pushing ahead, hence the less-bad option of not relegating that advancement to dark corners.
How is 'saying no' going to stop the military of this or any other country from doing it? How do you measure the disadvantage we'd be at by not allowing our military to do it, but letting other militaries (over which we have no control) do it?
So, basically another arms race to the bottom, similar to nuclear warheads ? 1940s thinking with 2016 technology. I hear what you're saying but not all progress is good progress.
The limits of our ability to think a global scale are an issue, we still have a tribe mentality, this discussion demonstrates that. "The Chinese, The British etc"
We still think in terms of tribes, but we have technology change the world. Until we evolve some more we're probably not fit to wield more and more powerful weapons.
It would be good if we could stop worrying about weaponry all together and focus on real issues like deforestation, poverty, pollution of the sea, climate change etc.
I get it, and I wholeheartedly support your position on the issue. The reality, unfortunately, is that we are horrid little creatures always scheming and fighting for power. Stopping nation X from doing, will only encourage nations Y and Z to develop it faster as an advantage.
The only way I can see an agreement to limit the escalation of AI warfare is on the same context we saw the START treaty: once the weapons are developed by at least two key players as "deterrent" there's an incentive to reach an agreement to stop further development.
Call me cynical, but if this election has shown me anything is how tiny-minded people still command an inordinate amount of power in this country.
What are you willing to sacrifice to say no? How about your immediate comfort and free time. Thats the real cost. Its not "when it happens we'll stabd togethor", Its "youtube videos or protecting the world from a potential future threat that everyone will think Im crazy for worrying about"
I'd be curious to hear more of your argument. The world is a complex place. How do you protect the one's you love when others exist who wouldn't say no?
Starcraft isn't troops on the battlefield, in the same way that Risk, the boardgame isn't troops on the battlefield.
Just because the 1s and 0s or game pieces represent "troops" to humans, doesn't mean that the underlying game mechanics have anything at all to do with a real war.
For all we know, the problems solved by a monopoly AI, are more applicable to a real war than those solved by a Risk or starcraft AI.
You must admit that there is an underlying aggressive theme in games like StarCraft. The visuals show clear resemblance of people being killed by projectiles and by other war technology. The AI will be rewarded for skillfully decimating the enemy, for protecting the units it has control over and for coming up with strategies like ambush, raids, patrolling etc. I am more convinced by the idea that an AI will be able generalize to the real world from playing StarCraft than from playing Monopoly. However, I think it will be quite a leap to the real world either way.
> You must admit that there is an underlying aggressive theme in these games. The visuals show clear resemblance of people being killed by projectiles and by other war technology.
An AI doesn't know or care about any of this.
Literally its applying statistics to optimize a cost function or assign labels to input states.
These numbers could represent number of kills or number of dollars; the computer doesn't care.
It's just absentmindedly optimizing a cold value function! No more humanity! More more external input! Just cold machine logic!
We've doomed ourselves!!!!
But seriously. No, there is nothing inherent about an AI maximizing wins in StarCraft than one maximizing profits on the stock exchange. Although I feel like there's a bunch of Sci-Fi where this obviously goes awry.
I do research in "Deep RL" for a living. The same algorithms (and advances related to them) that can be used in such a scenario can be used to create machine learning solutions for social good.
Similarly, algorithms used to detect cancer in medical imaging can be used to do automatic targeting on killer drones. Should we stop doing research altogether?
There are better war simulators if that were the purpose. There also wouldn't be a need to publicize it. Finally, big data/A.I is already used by the military. It probably isn't "decision making" levels, but it doesn't need to be. "Decision helping" is a good enough advantage.
If you are going to do war you should do it well. An AI commander could be programmed to maneuver troops in a way to reduce civilian casualties, reduce damage on infrastructure, and even maximize the chances of the enemy surrendering rather than having them killed.
Of course, I doubt that an AI commander would replace a human making the actual decisions in the foreseeable future. It would just be another intelligence tool for military command. From this point of view you are essentially arguing for our military to be ignorant, I don't think that is a good bet.
Legends trace the origin of the game to the mythical Chinese emperor Yao (2337–2258 BC), who was said to have had his counselor Shun design it for his unruly son, Danzhu, to favorably influence him.[64] Other theories suggest that the game was derived from Chinese tribal warlords and generals, who used pieces of stone to map out attacking positions.[65][66]
This worry and the implication (control use of AI) is similar to the development of cryptography. Try to stop people from doing evil with it is a completely fruitless task.
the press is going to make a huge clickbait deal about this, even if the complexity of starcraft 2, even if huge, is extremely small compared to the complexity of the world
This is so exciting. I've always wanted to program bots to play online games -- mainly for learning purposes. (Can I make a bot that plays better than me?)
But I've never done it because of the risk of bans. I'm glad that Blizzard has opened it up for people to experiment with this. I wonder how it will interact with any sort of anti-cheat systems in place, etc.
This has been around for awhile [1]. They give you an API to use and then you upload your JAR. They have a Twitch channel [2] that is constantly streaming AIs playing against each other in their tournament.
You should give Adventure Land [1] a try. Its a browser MMO where you can code a bot to play for you. Its still in alpha but it has been pretty stable for the past few days that I have been playing it.
I suspect this will eventually lead to AI as a service for games. Rather than build a terrible AI that delays a game by months, approaching a company that can build a decent AI initially which gets better overtime would probably be ideal and create better experiences.
Is it harder to make an AI that is mediocre at playing the game vs one that is good? That is the main issue with making good AIs for games now. We want AIs (if you can call them that) that play kinda like humans but worse then the player as the player wants to win.
Making an AI for game isn't about making a good AI. It is about making an AI that loses in a convincing manner. This is especially try for games like Starcraft (RTS and strategy games in general. For FPS games it really isn't so important as you can just increase or decrease the accuracy and hp of the enemies)
That would be funny. The difficulty slider changes to "How many hours of playtime" instead of easy, medium, hard. Actually it could even just scale it automatically based on a metric like steam hours played. that would be awesome.
I don't think so. Many of the fun things AIs do in games is attack at different stages of the game. They send a small force at the beginning, and then a little larger force a few minutes later, and then a big attack at the end. No AI trying to win would end up at this strategy. You would have to define the fun goals and have the AI train on that for it's success criteria.
Probably not, to be honest. For a game like chess, the challenge is that there are so few things you can actually do. Inefficiencies aren't really a thing, every single move has a semi-discrete value.
Starcraft however has all kinds of minor inefficiencies you can build in. You could inconsistently build workers, you could occasionally supply cap, your army comps can vary in sophistication level, etc.
Well, starcraft AIs aren't particularly good. Any player who has completed the single player campaign could beat all the AIs that came with the game that didn't cheat.
And a player in the top 20 percent could beat any AI that has ever been created.
It's like saying that a Go AI would have a huge advantage because it can move instantly, giving the opponent no time to think. Perhaps it contributes, but strategy is far more important.
This essay reminds me very much of the "Playing to Win" series by David Sirlin: http://www.sirlin.net/ptw/
It's also worth noting that AlphaGo optimizes its perceived chance of winning the game, rather than its score. It "gets complacent" and mostly plays very conservatively the moment it's ahead on points, because it doesn't care about margins, only about win/loss.
Definitely, but my guess is that Starcraft AI's are really good at Macro play (building the right units and tech trees), but at the upper levels of SC players win or lose in Micro play (specific positioning of units), which is much harder for an AI to accomplish well, since there are a lot of little tricks. Basically, Macro can be programmed easily, but Micro requires the player to respond creatively.
It's actually the other way around, I'd say. Individual unit control follows very naturally when you have unlimited APM. And most of the time you can define simple and good criteria on how to position the units (e.g. concave, or exactly the right distance apart to avoid splash damage, or one unit a tiny bit more towards the enemy to be the first one targeted by incoming fire, ...). Build orders can be programmed, yes, but transitioning between strategies properly based on observations and guessing or judging the enemy's strategy is quite hard.
The ones I saw in the competitions lost against humans on such micro rules since the humans quickly intuited their blind spots. The ability to bluff and screw with AI's, a human strong point, is what made me predict Starcraft was going to be way harder than Go. I'm still betting on the human in this one.
And the last line you said is also true to make things even more difficult. :)
It does, but the AI has to do useful things first before doing them quickly is a benefit.
Additionally, humans can use economic reasoning (player built 3 barracks to they can't be building x, y and z). This can lead to AI being excessively safe (economically inefficient).
I think it's really important to realize is that a game AI isn't really there to try to win, or maybe even to "convincingly" beat you the same way a human is. Perhaps you could train an AI to make the game more "fun", but that's pretty hard to measure and people are so different. It's a really hard optimization process.
One commonality, it seems, is that players want to get better at the game. Perhaps, an AI that TEACHES the user to improve their play, and reduce errors would be an easier problem. So far we mostly do this using heuristics of gradually increasing difficulty, but it should really depend on what the player is weakest at... and what the user can improve at fastest! That is something nicely measurable, which you might use to train an AI.
Then, if/when people play against other human players they are as strong as they can be... often an FPS goal.
You could use some kind of library of example "mistakes" made by very-high-level players to identify when lower-rank players are making similar errors; then point out to the user how everyone else who made that mistake 'updated' their behavior.
>an AI that TEACHES the user to improve their play
Wouldn't you be able to do mutual training that way? By mutating the AI's "strategy" only after losses, the player would continuously need to figure out a weakness in the AI's game-plan. And the AI tries to evolve its strategy every time it loses.
> Perhaps you could train an AI to make the game more "fun"
This is actually a way more interesting problem to solve, for the gaming industry.
Beating games is cool for industries where gaming is not the final problem (autonomous cars, various decision making, etc). But if AIs are to be exploited by gaming industry, it needs to learn things like : "is the user bored?", "what usually triggers new interest?", "how should difficulty be adjusted given current way the user is playing?", etc.
I think it's about options for players, some are strong others fresh and both want to choose an AI to their likings. Sometimes they are in the mood to get beat maybe to learn new tactics and occasionally just a victim to fight frustration.
What wrong with a strength option? I'd include an unbeatable entry in any case.
First person shooters can have super accurate AIs that are really frustrating to play against. Try beating the last "boss" in quake 3 on nightmare difficulty. A game like Starcraft relies much more on decision making that raw 3d math calculations and that makes making a decent AI pretty difficult without giving it advantages like extra resources or vision. That said, even though a game like Quake is easier to write an AI for, top players can still be top level AIs with superior reasoning about using the map and timing power up spawns and such.
> First person shooters can have super accurate AIs that are really frustrating to play against.
The Worms series is also notorious for this. Their AI targeting seems to flip between "clueless" and "godlike", with very little middle ground. E.g. https://youtu.be/gqPITW04vRQ?t=1m20s
It depends. AIs can do some things much better than nearly all humans. Twiddly little micromanagement for example. Total Annihilation (and it's successors like Supreme Commander) do this. The AI will send out a constant stream of micromanaged harassers constantly order to annoy the human opponent to death.
Where the AIs tend to be very bad is in long term thinking and overall strategy.
Blizzard's AI in Starcraft generally plays like a slightly more aggressive but novice human. It's more fun than the TA AI, but not terribly strong once you understand the game mechanics.
Some AIs for the Spring RTS Engine had this too. The funny thing is though- harrassment doesent scale. In these Exponential Economy type of games, the harassment only works as long as the macro-strategy is unable to take off. After a certain treshold, you can micro all the funny dances in the world- but the wave of enemys is going to carry this surfer to the shores end.
I bet you could run an analysis of "move quality" for the human player, find out where the moves he is making would rank in a list of possible moves sorted by value of the AI's quality function, and select the same-percentile moves for the computer AI to make (from its available moves).
> Making an AI for game isn't about making a good AI. It is about making an AI that loses in a convincing manner. This is especially try for games like Starcraft (RTS and strategy games in general. For FPS games it really isn't so important as you can just increase or decrease the accuracy and hp of the enemies)
It is actually about making a good AI. Civ 6 or SC2, in both cases they simply "cheat" by providing multipliers to the AI in order to solve its limitations. The basic function is largely the same. Providing the AI with cheat codes is not "good" AI.
A good AI design would remove the need for such arbitrary advantages that are out of context for human players and leads to weird issues.
I'm not sure that accuracy/HP are the right things to tweak. I've always wanted to be "surprised," in an unscripted way, by AI enemies in FPS and RTS games.
One game experimented with smart enemy units that could sneak up on a player and attack them from behind, or outflank them. The players hated it and assumed the computer was cheating and just spawning enemies behind them.
The same is sort of true in RTS games. The new rerelease of age of empires 2 has vastly improved AI that was developed by modders. Lots of players thought it was just cheating.
But in general it has been very well received. I think players can get used to good AI. They've just been trained to expect stupid AI that cheats.
For the vast majority of players it's probably irrelevant whether the AI cheats or not. The goal is not to play fair and beat the human usually, but to put up a good fight and make things challenging and, most importantly, fun for the human. Of course, competitive and professional players have different expectations or requirements in that regard.
Still, AI in games is mostly a trade-off. It usually can't take too much resources (because it usually has to run on the same machine the human is playing on), it has to be believable and fun to play against. This usually rules out too fancy algorithms and approaches with dubious returns.
I think the majority of players would prefer AIs that are challenging through better strategy and tactics, rather than just being bullet sponges or having better aim. People also get frustrated when the computer wins through cheating, rather than actually being better.
AI is terrible in games because it's literally such a hard problem. Thinking like a human, working in a group, overcoming obstacles and reacting to dynamic behaviour from the human player. This is far harder that Go which has 100x simpler rules. Additionally they had a mega ton of training data to feed into the Go model to begin with, which a game won't have as the data structures will all be different under the hood.
The reason they picked this challenge is to work on problems where the whole state of the game is not immediately accessible (games of partial information, as opposed to chess and Go). Compared to AlphaGo, it is much more similar to the situations humans encounter in daily life.
I thought a neat way to make NPCs less dumb in games would be to let other players (internet connection required) to be the NPCs. They would have limited coverage area maybe set tasks and restrictions and let them fight a real person to move onto the next part of the game.
Basically set up a queue of player and what NPC character to play and when a user gets to that part of the game where NPC needs to load, you jump in and play. When the NPC dies you the game (scene) and get put back in the queue of playable NPCs.
You're storming the beaches of Normandy. You duck behind cover. Suddenly a shot rings out. You're dead. The kill cam reveals you were killed by a human-controlled enemy, standing on a seagull in the skies above. [1]
Unfortunately, humans will find advantages that shouldn't be used in any sort of game that tries to build a believable world.
That isn't to say it's a bad idea. You'd just have to design the game with human enemies in mind. I was very interested in The Crossing [2], which incorporated that of mechanic. Unfortunately, it was canceled. :(
I've been to the Game Developer's Conference (once) before. Believe me, there have been startups that have provided A.I. services for years (it was kind of like the equivalent of the Havok engine for physics, you had to implement it into your game). I don't know how well used they are, but I remember them being mentioned at one of the lectures about A.I. that I attended. They had tables and swag, too. I don't remember their names.
In previous years I've seen a lot of AI tech for action-games -- not neccessiarly FPS but shooter in a flat world with obstacles. Eg. marking objects for cover, marking positions to defend from etc.
There are a lot of problems, but mainly I think it boils down to game people, like a lot of programmers, wanting to write something themselves. You'll always understand it better, and it'll be perfectly tailored to your problem. Unless the problem is REALLY hard and takes a long time coding tedious things, they'd rather just do it themselves, since coding something is always more fun than figuring out someone else's code.
Also, out of my experience with many different fields of programming, game programmers seem to want to write more of the stuff themselves than most. This would be another huge subject to go into, but I think the biggest reason is that historically you couldn't ever really punt on performance in games, and you had to tweak everything to your problem. If your game ran slowly or looked bad, people just wouldn't buy it.
Most games don't need this. Smoke and mirrors work a lot better than honest AI to create an illusion of intelligence, and it's much easier to change game mechanics and balance with a simpler AI.
That's so cool! I wish they could start doing AI for team based competitive games like League of Legends where meta-play and team decision making is important.
Is that too complicated to tackle yet?
Are you imagining a single AI playing all 5 heroes on a team, or 5 independent AIs each playing a single hero with limited communication between them, or a single AI controlling a single hero on a team with 4 humans? I would say the first scenario (1 AI controlling entire team) is roughly equivalent to the the RTS case. It's just fewer units with more abilities, and character progression instead of base building. The other two cases are probably a lot more complicated.
Either one could work. If the single AI that controls 5 heroes is easier to program, let's imagine that.
I have the feeling that it's more complicated than a 1vs1 RTS game, but I don't know why...
Controlling five units is a lot easier than playing an RTS. In an RTS you don't just control five units, you control potentially hundreds of units, and you have base-building and an economy to manage.
Honestly all these scenarios are interesting research avenues.
Inter-agent communication, especially in a machine learning context, is an unsolved problem, exploring it in such a scenario would be nice.
Agent-human communication is probably as hard, and also largely unsolved, although there is a push right now for dialog systems to achieve human level performance.
I can't wait to see the first team of 5 separate AI learn to insult each other. It'll be a watershed moment in history when the first vote to kick another AI occurs.
AI controls one champion, maybe a bit weaker than player controlled ones.... and it controls the minions. every x wave, they can go into jungle or do other things rather than push lane
Wow that's a very interesting idea. Several models working together. Plus you limit their method of communication by being able to use the chat only. Inventing and learning new languages made by other models!
restricting controls while typing is an arbitrary restriction considering they can communicate with audio asynchronously.
this brings up another interesting topic all together- which is how inefficient human language is at communicating complex ideas, quickly, with people you don't know.
that is- just like how AI has an obvious APM advantage, it also has an obvious communication advantage (raw serialization and deserialization. there is no room for ambiguity or misinterpretation).
basically, all information known by one agent will be known by all other agents in near-realtime. something humans can't do yet.
Best would be if they had a language model, but no memory of communicating together in the past, so each game the AI would have the additional hurdle of building up communication.
depends on the goal, right? considering they want to limit APM in the SCII model- we can assume that goal is something like "beat a human under human conditions". in that case yes, communication needs some limitation. but this seems uninspiring. "assuming" a no-limitations AI can beat a human in RTS, the question seems more like "how many inefficiencies must we introduce before the AI is 'fair' game".
but, there is a different use case where you have an AI team try to beat another AI team under optimal conditions, which i think is more interesting, as it has the chance to completely change the metagame (like what Deep Blue and DeepMind can do for Chess/Go respectively).
> restricting controls while typing is an arbitrary restriction considering they can communicate with audio asynchronously.
On the other hand, if four people are simultaneously speaking to you, it's hard to understand all of them at once (especially if you're also talking at the same time); of course, an AI would presumably be able to communicate information much more quickly to other AIs than humans can to other humans, so limiting one to "speaking" at a time would probably not be that detrimental to them
The Berkeley Overmind was designed for a 2010 Brood War competition. The website (http://overmind.cs.berkeley.edu/), although no longer updated, still contains many videos and links to articles that describe the internals of the AI and the design process. The Ars Technica article in particular goes quite in depth.
This is really good news!
Lets hope DeepMind can improve even further on their Differentiable neural computers (DNC) which seems like an requirement for this kind of AI to work (exploiting long-term dependencies).
I also hope that other research/industry teams will join on the competition to create competing AIs. Very exciting!
I can't wait to see how far DeepMind can go in this area. I was initially skeptical that AlphaGo could defeat top human players, but then it happened. Who knows, perhaps one day AI can compete against progamers in GSL!
Competitive AI in broodwar already can destroy the best humans. AI can use strategies and tactics that human can't effectively do. Most notable is perfect splitting which changes the game quite a bit.
>Competitive AI in broodwar already can destroy the best humans
Who do you mean by "the best humans"? I've never heard of top-tier progamers such as Flash, Jaedong, Bisu, etc. losing to AI. In fact, I doubt any B-teamer or above would lose to an AI in a Bo3.
It would be cool to see a project like this for an open source game, such as OpenTransportTycoonDeluxe (http://www.openttd.org/). The AI developed by interacting with the OpenTTD economy might even prove useful for urban planning of real geographic regions.
That's great.
AIs on SC1 relied on many hacks. Initially I thought that DeepMind was going to create a bot for the original SC.
I hope some of the advances in SC2 AI can be integrated into the in-game AI. e.g: a trained neural network that plays better than the "hard" AI, but can run on a consumer box and not on a massive cluster.
This would be so much more fun with a turn-based game where speed isn't a variable, like Civilization. I'd love to play against several AIs that were better than me because of code, not because they get a bunch of extra in-game bonuses. With a nice AI API, you could have online competitions where AIs battled every month.
This will be good for games moving forward due to the meta changing for players as the AI adapts to their tactics and vice versa. Lessons learned from this can then be applied to other areas. And as an added bonus it creates more interest in AI research.
I'd love to see a e-sport league where the teams are AI human hybrids (Centaur teams). We know that AI human hybrid teams are great at chess [1],
and I'd love to see rts games played by 'Centaur' teams. In the same way the innovations made in F1 often trickle down to consumer cars, can you imagine the advances that could be made in human-machine interactions in the crucible of a real-time Centaur gaming league?
I'd rather that the coding was something that was left to the player or to the third party market. Right now programming games are kind of boring, more like a way of prettying up homework than anything else. A really fun game where programming could give you a competitive edge would be something to see.
The closest thing I can think of is add-ons for MMOs, but there the game developers specifically try to prevent add-ons that are too performance enhancing instead of making the coding an integral part of the game.
Typically they train the AI against itself. That gives it a continuous skill gradient. Otherwise it would lose all of the games without learning anything. Since there are no successes to learn from. Or it would win all the games after it figures out how to exploit the built-in bot. With strategies that probably won't apply to humans.
For anyone looking for more information about Starcraft 2, the world championship is on this weekend and the broadcast for today has just started (16:30EST)
Pretty soon AI for games will be so good that we won't have to play online multiplayer for a really good authentic experience. Once we can't tell the difference between the AI and an actual human playing.
Training data? After all, AlphaGo trained on a database of over 30M expert human moves. I suspect one championship round from Team EnVy is worth billions of iterations of random exploration ;)
Kudos to both Blizzard and DeepMind. Anticipating a lot of fun with this. StarCraft 2 could indeed become the AI pedagogy standard.
343 comments
[ 2.6 ms ] story [ 289 ms ] thread[1] https://youtu.be/WHcl6Cs1FAM?t=7m57s
An AI limited to even 200 decision-driven actions per minute would probably have a significant advantage over a "500APM" player.
The point is to build ai that compete with humans using human strategies. Not to cheese with borked game mechanic exploits.
People were making custom AIs for starcraft 2 since it was in beta. They can't beat any human worth their salt.
Unlimited APM makes AI much less interesting from the perspective of game theory.
Think of it like a robotic boxer, except instead of an android, they build two 30 foot long walls of spring-loaded boxing gloves that close in on the human boxer. Yes, the robot punches the guy a lot, so the problem seems "solved" but it isn't really.
The top contestants were employing strategies that required APM an order of magnitude higher than what players could do. For example, in brood war, SCVs can repair any Terran building or vehicle but it is not worth the effort to repair goliaths or tanks for a player.
However, the AI could easily manage repairing it's tanks which makes the Terran army much more cost effective.
In SCII there is auto repair and other improved UX that probably limit the ability of novel strategies like repairing vehicles.
https://www.youtube.com/watch?v=IKVFZ28ybQs
(it's also possible the system is receiving information directly from the game engine, but it's definitely possible to figure this out from nothing but the information on screen)
What would be really interesting is putting a similar smart AI on the tanks, which could make decisions like shooting at the highest concentration of zerglings instead of the closest one, or holding its fire if one lone zergling is in range, but a clump of them is about to enter range.
Human players already do both optimizations with widow mines, which are a unit that takes several seconds to prepare its shot so it's feasible to retarget. It's not feasible for humans to do this with tanks though as they fire immediately and automatically.
You could even see counterevolution as two AI algorithms become better at predicting what the other will do.
This is awesome. I've only ever reached the Platinum league in Starcraft II (1v1), but I'd almost feel more driven to create bots to (hopefully) surpass that skill level, than actually playing the game.
The commenter I was responding to has strong ideas about it however, so I wanted to provide the opportunity for an explanation.
How do you propose that ban be enforced?
Having a spine has nothing to do with it.
The limits of our ability to think a global scale are an issue, we still have a tribe mentality, this discussion demonstrates that. "The Chinese, The British etc"
We still think in terms of tribes, but we have technology change the world. Until we evolve some more we're probably not fit to wield more and more powerful weapons.
It would be good if we could stop worrying about weaponry all together and focus on real issues like deforestation, poverty, pollution of the sea, climate change etc.
The only way I can see an agreement to limit the escalation of AI warfare is on the same context we saw the START treaty: once the weapons are developed by at least two key players as "deterrent" there's an incentive to reach an agreement to stop further development.
Call me cynical, but if this election has shown me anything is how tiny-minded people still command an inordinate amount of power in this country.
Starcraft isn't troops on the battlefield, in the same way that Risk, the boardgame isn't troops on the battlefield.
Just because the 1s and 0s or game pieces represent "troops" to humans, doesn't mean that the underlying game mechanics have anything at all to do with a real war.
For all we know, the problems solved by a monopoly AI, are more applicable to a real war than those solved by a Risk or starcraft AI.
An AI doesn't know or care about any of this. Literally its applying statistics to optimize a cost function or assign labels to input states. These numbers could represent number of kills or number of dollars; the computer doesn't care.
It's just absentmindedly optimizing a cold value function! No more humanity! More more external input! Just cold machine logic!
We've doomed ourselves!!!!
But seriously. No, there is nothing inherent about an AI maximizing wins in StarCraft than one maximizing profits on the stock exchange. Although I feel like there's a bunch of Sci-Fi where this obviously goes awry.
Similarly, algorithms used to detect cancer in medical imaging can be used to do automatic targeting on killer drones. Should we stop doing research altogether?
Of course, I doubt that an AI commander would replace a human making the actual decisions in the foreseeable future. It would just be another intelligence tool for military command. From this point of view you are essentially arguing for our military to be ignorant, I don't think that is a good bet.
Legends trace the origin of the game to the mythical Chinese emperor Yao (2337–2258 BC), who was said to have had his counselor Shun design it for his unruly son, Danzhu, to favorably influence him.[64] Other theories suggest that the game was derived from Chinese tribal warlords and generals, who used pieces of stone to map out attacking positions.[65][66]
But I've never done it because of the risk of bans. I'm glad that Blizzard has opened it up for people to experiment with this. I wonder how it will interact with any sort of anti-cheat systems in place, etc.
[1] http://sscaitournament.com
[2] https://www.twitch.tv/certicky
I personally am willing to pay $40 for an environment to test my ai on for a couple months.
[1] http://adventure.land/
Im curious if a startup can be built from this.
Making an AI for game isn't about making a good AI. It is about making an AI that loses in a convincing manner. This is especially try for games like Starcraft (RTS and strategy games in general. For FPS games it really isn't so important as you can just increase or decrease the accuracy and hp of the enemies)
Starcraft however has all kinds of minor inefficiencies you can build in. You could inconsistently build workers, you could occasionally supply cap, your army comps can vary in sophistication level, etc.
And a player in the top 20 percent could beat any AI that has ever been created.
AIs are really in an abysmal state for RTSes.
See "The Marginal Advantage," an essay from the famous Starcraft player Day9: http://www.teamliquid.net/forum/brood-war/64514-competitive-...
I meant to say that strategy trumps mechanics. Mechanical proficiency is necessary but not sufficient.
It's also worth noting that AlphaGo optimizes its perceived chance of winning the game, rather than its score. It "gets complacent" and mostly plays very conservatively the moment it's ahead on points, because it doesn't care about margins, only about win/loss.
http://wiki.teamliquid.net/starcraft/Micro_and_Macro
And the last line you said is also true to make things even more difficult. :)
Additionally, humans can use economic reasoning (player built 3 barracks to they can't be building x, y and z). This can lead to AI being excessively safe (economically inefficient).
One commonality, it seems, is that players want to get better at the game. Perhaps, an AI that TEACHES the user to improve their play, and reduce errors would be an easier problem. So far we mostly do this using heuristics of gradually increasing difficulty, but it should really depend on what the player is weakest at... and what the user can improve at fastest! That is something nicely measurable, which you might use to train an AI.
Then, if/when people play against other human players they are as strong as they can be... often an FPS goal.
That's how MOOCs should function, too.
Wouldn't you be able to do mutual training that way? By mutating the AI's "strategy" only after losses, the player would continuously need to figure out a weakness in the AI's game-plan. And the AI tries to evolve its strategy every time it loses.
This is actually a way more interesting problem to solve, for the gaming industry.
Beating games is cool for industries where gaming is not the final problem (autonomous cars, various decision making, etc). But if AIs are to be exploited by gaming industry, it needs to learn things like : "is the user bored?", "what usually triggers new interest?", "how should difficulty be adjusted given current way the user is playing?", etc.
What wrong with a strength option? I'd include an unbeatable entry in any case.
First person shooters can have super accurate AIs that are really frustrating to play against. Try beating the last "boss" in quake 3 on nightmare difficulty. A game like Starcraft relies much more on decision making that raw 3d math calculations and that makes making a decent AI pretty difficult without giving it advantages like extra resources or vision. That said, even though a game like Quake is easier to write an AI for, top players can still be top level AIs with superior reasoning about using the map and timing power up spawns and such.
The Worms series is also notorious for this. Their AI targeting seems to flip between "clueless" and "godlike", with very little middle ground. E.g. https://youtu.be/gqPITW04vRQ?t=1m20s
Where the AIs tend to be very bad is in long term thinking and overall strategy.
Blizzard's AI in Starcraft generally plays like a slightly more aggressive but novice human. It's more fun than the TA AI, but not terribly strong once you understand the game mechanics.
It is actually about making a good AI. Civ 6 or SC2, in both cases they simply "cheat" by providing multipliers to the AI in order to solve its limitations. The basic function is largely the same. Providing the AI with cheat codes is not "good" AI.
A good AI design would remove the need for such arbitrary advantages that are out of context for human players and leads to weird issues.
The same is sort of true in RTS games. The new rerelease of age of empires 2 has vastly improved AI that was developed by modders. Lots of players thought it was just cheating.
But in general it has been very well received. I think players can get used to good AI. They've just been trained to expect stupid AI that cheats.
Still, AI in games is mostly a trade-off. It usually can't take too much resources (because it usually has to run on the same machine the human is playing on), it has to be believable and fun to play against. This usually rules out too fancy algorithms and approaches with dubious returns.
it's necessary anyway to prevent it from having godlike micro.
https://www.youtube.com/watch?v=DkbNzzZEU9g
Basically set up a queue of player and what NPC character to play and when a user gets to that part of the game where NPC needs to load, you jump in and play. When the NPC dies you the game (scene) and get put back in the queue of playable NPCs.
Unfortunately, humans will find advantages that shouldn't be used in any sort of game that tries to build a believable world.
That isn't to say it's a bad idea. You'd just have to design the game with human enemies in mind. I was very interested in The Crossing [2], which incorporated that of mechanic. Unfortunately, it was canceled. :(
[1] Not a made-up problem. https://youtu.be/UfZFHdywYLA
[2] https://en.wikipedia.org/wiki/The_Crossing_(video_game)
How would you scale for games with different inputs and outputs?
Doesn't quite apply to a mario style game
Also, out of my experience with many different fields of programming, game programmers seem to want to write more of the stuff themselves than most. This would be another huge subject to go into, but I think the biggest reason is that historically you couldn't ever really punt on performance in games, and you had to tweak everything to your problem. If your game ran slowly or looked bad, people just wouldn't buy it.
Inter-agent communication, especially in a machine learning context, is an unsolved problem, exploring it in such a scenario would be nice.
Agent-human communication is probably as hard, and also largely unsolved, although there is a push right now for dialog systems to achieve human level performance.
this brings up another interesting topic all together- which is how inefficient human language is at communicating complex ideas, quickly, with people you don't know.
that is- just like how AI has an obvious APM advantage, it also has an obvious communication advantage (raw serialization and deserialization. there is no room for ambiguity or misinterpretation).
basically, all information known by one agent will be known by all other agents in near-realtime. something humans can't do yet.
but, there is a different use case where you have an AI team try to beat another AI team under optimal conditions, which i think is more interesting, as it has the chance to completely change the metagame (like what Deep Blue and DeepMind can do for Chess/Go respectively).
On the other hand, if four people are simultaneously speaking to you, it's hard to understand all of them at once (especially if you're also talking at the same time); of course, an AI would presumably be able to communicate information much more quickly to other AIs than humans can to other humans, so limiting one to "speaking" at a time would probably not be that detrimental to them
Who do you mean by "the best humans"? I've never heard of top-tier progamers such as Flash, Jaedong, Bisu, etc. losing to AI. In fact, I doubt any B-teamer or above would lose to an AI in a Bo3.
http://overmind.cs.berkeley.edu/
I hope some of the advances in SC2 AI can be integrated into the in-game AI. e.g: a trained neural network that plays better than the "hard" AI, but can run on a consumer box and not on a massive cluster.
[1] http://bloomreach.com/2014/12/centaur-chess-brings-best-huma...
The closest thing I can think of is add-ons for MMOs, but there the game developers specifically try to prevent add-ons that are too performance enhancing instead of making the coding an integral part of the game.
https://www.twitch.tv/starcraft
Kudos to both Blizzard and DeepMind. Anticipating a lot of fun with this. StarCraft 2 could indeed become the AI pedagogy standard.