If I'm interested in Neural Networks, and I've mainly been working in web development (I know Java, C, and Python). Where would be a good place for me to start so that I can eventually make something like this project?
Thanks! I'm really enjoying seeing the huge variety of domains people have applied char-rnn to. I haven't anticipated response at this scale, to both the original blog post and the code release. (I try to keep a somewhat up to date list of responses on the bottom of the blog - cooking recipes, music, obama speeches, magic cards...)
In terms of learning, I would also encourage people to try the materials from our CS231n class - this is a class I taught at Stanford last quarter with my adviser. It's technically about Convolutional Networks, but most of the class material is building up generic Neural Networks, backprop, and so on. You can also try our IPython Notebook assignments.
This is from one of the people who helped write the toolbox in Matlab. There is a page limit on the Amazon publishing service, so that is why it's posted here[1]. Chapter 14 describes notation for the recurrent networks and explains their training.
Believe me, it's been thought of (it's been in my head ever since my colleague Frank Seide introduced me to RNNs 4 years ago, I'm sure I wasn't the first!). It is one thing to generate code that looks like code, quite another to generate code that is useful. Character-by-character probably isn't good enough for this task.
The old Jungloid technique for Java factory automation/type conversions (I have an A, I want a B) seems like a possible use for this kind of thing. http://dl.acm.org/citation.cfm?id=1065018
If a prediction engine can consistently predict your next line of code, wouldn't that be a sign that you're doing way too much copying and pasting instead of abstracting common patterns?
Magic is an interesting game just from an text analytics perspective. Using a source like mtgjson you get a reasonably sized corpus of text that looks natural enough for a human to comfortably grok but sufficiently templated to apply even naive methods and acquire awesome results.
I built a tool that uses n-grams to find cards similar to each other. The result is comparable to manual suggestions done by users but takes around a minute rather than months to cleanly update after a new set is released!
I wrote it around a year ago when I was still getting a good feel for go so its a fairly gross ~500 line go package. However, it exports a moderately sane QueryableSimilarityData with the ability to Query that data for individual names. A query result is both the cards that were found to match and how confident it was. It caches after computing the entire dataset's results so it is fast to query after closed.
If you want to muck around with it, you'll need to use a relative import because its just easier for a single file package.
Just stick an AllCards-x.json from mtgjson.com in the same directory as binary the package was used in and it should expose the interface I described above to anything trying to use it.
As a note, its non-permissively licensed but message me on reddit under this username and I'll re-license that component. Also, feel free to message me if you want a hand or have any ideas.
Are you the creator of mythicspoiler? I know they wanted to do something like that. As of a few sets ago the 'Similar Cards' in their spoilers were chosen manually.
I am not but curiously enough I got the idea from a short response the creator of mythicspoiler had made on a reddit thread asking for an api to find similar cards or some such. He mentioned that he had played with n-grams but found it to be a dead end.
I ran with the idea and found a version weighted on matched n-gram length gets reasonable results.
> Whenever you cast a spell, you may return target creature card from your graveyard to your hand.
> 2/2
That's actually a fairly plausible card, in terms of mechanics. It's thematically appropriate for black, it's about the right cost, it's powerful but specialized (returns creature cards only, and only to hand rather than to play), and it's legendary to avoid multiplying its effect. This would work well in a deck with many creatures with "enters the battlefield" effects (common in black, as well as blue and red which are often used together with black), as well as many ways to sacrifice cards. Would even work as the general in an EDH/Commander deck.
The only thing slightly odd is the flavor: it's a flying cat. And with the right art (and perhaps a "black cat crossing your path" joke in the flavor text), that could work quite well.
Duplicate it a few times over via any number of duplication mechanisms (common enough in red or blue), and playing one spell would get you multiple creatures back. Since those creatures themselves are spells, it'd be too easy to set up a loop or combo. Making it legendary means you can't easily end up with "play one spell, get back N creatures".
For that matter, creatures are cards, and there are effects powered by discarding cards or cards going to the graveyard, so if you can play one spell to get 5 more cards in your hand, and then discard 5 cards to get an effect...
My neural network is telling me that it's actually too good at 4 mana.
Don't forget about potential combos like having a bunch of zero cost creatures in your graveyard + anything else that triggers when you play a spell or when a creature enters the battlefield. But even without that, this is a card advantage machine.
It's rather good at 4 mana, though that could be balanced in the metagame by making it rare or mythic.
What zero-cost creatures did you have in mind? Those aren't incredibly common. And in any case, a combo that requires 3-4 specific cards is not overpowered; one that only requires 2 specific cards often is.
Magic is not a game that considers all combos a bug, and there are any number of ways to get to infinite mana (or infinite many other things) with less than four cards.
You cannot balance a card [in constructed] by making it rarer. You just make it so fewer people can run the deck based around it in a given tournament.
It wouldn't be particularly broken in draft anyway - which is the only meta where rarity does impact balance to some extent.
These are recurrent neural networks, not recursive. That's a different model, also used for NLP, but not so readily applicable for generating text AFAIK.
Wow, this has to be my favorite HN post. I couldn't finish reading it at work due to tear filled laughter.
I think it is less funny if you don't know anything about MTG. Which is interesting - if you don't, or don't know it well, these cards seem fairly indistinguishable from real cards. The differences are hillarious.
I was very excited to learn about recursive neural networks and how they are different from recurrent neural networks. I began imagining self-similar fractal topologies and automatic convolution layer creation etc... Disappointed to see it was just a typo :-)
The author remarks that most of these are nonsensical.
As a non-magic player who lives in a house that sometimes hosts magic games (and thus clicks links re:mtg), I found every line to be entirely plausible overheard banter.
This is a very interesting idea. I'd like to see it played with 100% of the cards being generated through a system like this. 4 players, 4 automated decks of random (possibly nonsensical cards), that would be _so much fun_!
Reminds me: it feels to me like MTG card text (excluding flavor text) obeys a formal grammar of some sort. Does it?
If it does, the NN's could possibly be made a lot more powerful if you put a "parser" in front of its input end (so it was being fed ASTs instead of text) and a "code generator" on the output end (to convert confabulated ASTs back into text).
I'm not sure if it's officially a formal grammer, but there are definitely correct and incorrect phrasings for certain effects, and those are changed over time with official rewordings posted for old cards.
It's one of the ways that unofficial card spoilers are judged. If there are phrasings that sound "out-of-style" for the effect, its usually judged as a fake from the first pass.
My understanding (based on some old blog posts[1]) is that while there's no (published) formal grammar there's definitely a set of internal rules that WoTC uses for determining how things are phrased. What is considered "technically correct" has changed somewhat over the years. I believe Sixth Edition was the first attempt to standardize and make consistent card text. Generally though cards that do the same or very similar things are going to follow consistent word ordering.
For example take Shrewd Hatchling[2] and Coral Reef[3]. Both are fairly different cards, but notice the wording of the first part of their respective card text:
> Shrewd Hatchling enters the battlefield with four -1/-1 counters on it.
and
> Coral Reef enters the battlefield with four polyp counters on it.
Cards with that sort of ability are (almost) always going to have it phrased in a way that fits the pattern of "<card name> enters the battlefield with <n> <type> counters on it."
Lots of other sorts of common mechanics are the same. Additionally, there are the so called "Golden Rules"[4] which are important to take note of when parsing the text of a card.
While Googling for links for this reply I found Gleemin[5], a "rules engine that aims to fully and correctly implement the game's tournament rules, an AI player programmable with expert knowledge and an interpreter for the game's language (the rules text on the cards)"! Seems you're not the first person to think about this!
Which suggests that it's the name of the in-progress second set of a block of custom user-created cards the person is working on. Quick glance at the posts the signature links to gives me the impression it was chosen simply as a generic "storm is coming" fantasy-setting name.
And for people who know a bit about Magic: seems a lot like "Scars block without the Phyrexians", even down to the user posting a variation on Koth.
Wonder what MaRo and the gang thinks about this. I am pretty surprised at the results so far from that post. They mostly don't make sense but I'm sure if you went through say 100 of them you're bound to find one or two legitimate ideas in there.
A question I have about this, and I'm new to neural networks so this might be entirely wrong, but would it be possible to create another neural network that returns whether or not a card is overpowered or not and run that on all the cards generated by this neural network? Then you could get a set of fairly balanced cards even if a few might be somewhat nonsensical
I am enamoured that geeks can find amazement in computer generated content. I wonder if Shakespeare scholars felt the same with the recent neural net creating Shakespeare-like content. What is to come, when cgc starts to become even more amazing? A individualized, computer generated TV-shows that hit all your favorite subjects?
Would be interesting to do this for other other games (that interest me more ;)).
Netrunner ICE generation would be interesting. I'd also like to run some algorithm over all Star Wars LCG cards and have it generate new arrangements of objective sets.
[for those that don't know the game you don't build your deck by picking cards like in Magic but rather predefined bundles of 6 cards (objective sets)]
68 comments
[ 3.1 ms ] story [ 141 ms ] threadAnother card nerfed by the removal of mana burn! (http://archive.wizards.com/magic/magazine/article.aspx?x=mtg... section 3, "Mana Pools and Mana Burn")
This thing is really cool. Also, he's using mtgjson.com, which I've used in the past and it's really awesome.
The cards do get very "real" very fast. Love it
He also has a fantastic article specifically about recurrent neural networks here: http://karpathy.github.io/2015/05/21/rnn-effectiveness/
In terms of learning, I would also encourage people to try the materials from our CS231n class - this is a class I taught at Stanford last quarter with my adviser. It's technically about Convolutional Networks, but most of the class material is building up generic Neural Networks, backprop, and so on. You can also try our IPython Notebook assignments.
Course notes: http://cs231n.github.io/ Syllabus with slides too: http://cs231n.stanford.edu/syllabus.html
Another good pointer is Andrew Ng's Coursera class - that's a thorough introduction as well.
There is a book that I can link if you are interested.
[1]:http://hagan.okstate.edu/NNDesign.pdf
I also recommend the tutorials on http://deeplearning.net/tutorial/ (Python / Theano)
And Andrej Karpathy's blog is also a great resource for explanations of deep learning concepts in simple terms: https://karpathy.github.io/
Add static analysis and it can actually be fairly useful. E.g. val name = object.[maybe you wwant to use getFirstName]?
http://www.darpa.mil/program/probabilistic-programming-for-a...
Anyways, I talk about how to use probability (but not compute it!) in http://research.microsoft.com/pubs/179363/mcdirmid12.pdf
I built a tool that uses n-grams to find cards similar to each other. The result is comparable to manual suggestions done by users but takes around a minute rather than months to cleanly update after a new set is released!
I wrote it around a year ago when I was still getting a good feel for go so its a fairly gross ~500 line go package. However, it exports a moderately sane QueryableSimilarityData with the ability to Query that data for individual names. A query result is both the cards that were found to match and how confident it was. It caches after computing the entire dataset's results so it is fast to query after closed.
If you want to muck around with it, you'll need to use a relative import because its just easier for a single file package.
Everything you need apart from the dataset is in https://github.com/Everlag/goPricesBeta/blob/master/utilitie...
Just stick an AllCards-x.json from mtgjson.com in the same directory as binary the package was used in and it should expose the interface I described above to anything trying to use it.
As a note, its non-permissively licensed but message me on reddit under this username and I'll re-license that component. Also, feel free to message me if you want a hand or have any ideas.
I ran with the idea and found a version weighted on matched n-gram length gets reasonable results.
> 2BB
> Legendary Creature - Cat
> Flying
> Whenever you cast a spell, you may return target creature card from your graveyard to your hand.
> 2/2
That's actually a fairly plausible card, in terms of mechanics. It's thematically appropriate for black, it's about the right cost, it's powerful but specialized (returns creature cards only, and only to hand rather than to play), and it's legendary to avoid multiplying its effect. This would work well in a deck with many creatures with "enters the battlefield" effects (common in black, as well as blue and red which are often used together with black), as well as many ways to sacrifice cards. Would even work as the general in an EDH/Commander deck.
The only thing slightly odd is the flavor: it's a flying cat. And with the right art (and perhaps a "black cat crossing your path" joke in the flavor text), that could work quite well.
Flying cats are actually already in the game. Back in the Planar Chaos days I used this one in a really trollsy black deck:
http://gatherer.wizards.com/Pages/Card/Details.aspx?name=mir...
http://gatherer.wizards.com/Pages/Search/Default.aspx?action...
For that matter, creatures are cards, and there are effects powered by discarding cards or cards going to the graveyard, so if you can play one spell to get 5 more cards in your hand, and then discard 5 cards to get an effect...
My neural network is telling me that it's actually too good at 4 mana.
Don't forget about potential combos like having a bunch of zero cost creatures in your graveyard + anything else that triggers when you play a spell or when a creature enters the battlefield. But even without that, this is a card advantage machine.
What zero-cost creatures did you have in mind? Those aren't incredibly common. And in any case, a combo that requires 3-4 specific cards is not overpowered; one that only requires 2 specific cards often is.
FWIW, it's about the same cost as Enduring Renewal.
Even the synergy with Fume Spitter is pretty great. :)
It wouldn't be particularly broken in draft anyway - which is the only meta where rarity does impact balance to some extent.
I think it is less funny if you don't know anything about MTG. Which is interesting - if you don't, or don't know it well, these cards seem fairly indistinguishable from real cards. The differences are hillarious.
Mointainspalk.
I think the mechanic is hard to generate a deck that utilize these mechanic and build synergy and combo out of these decks.
If it does, the NN's could possibly be made a lot more powerful if you put a "parser" in front of its input end (so it was being fed ASTs instead of text) and a "code generator" on the output end (to convert confabulated ASTs back into text).
It's one of the ways that unofficial card spoilers are judged. If there are phrasings that sound "out-of-style" for the effect, its usually judged as a fake from the first pass.
For example take Shrewd Hatchling[2] and Coral Reef[3]. Both are fairly different cards, but notice the wording of the first part of their respective card text:
> Shrewd Hatchling enters the battlefield with four -1/-1 counters on it.
and
> Coral Reef enters the battlefield with four polyp counters on it.
Cards with that sort of ability are (almost) always going to have it phrased in a way that fits the pattern of "<card name> enters the battlefield with <n> <type> counters on it."
Lots of other sorts of common mechanics are the same. Additionally, there are the so called "Golden Rules"[4] which are important to take note of when parsing the text of a card.
While Googling for links for this reply I found Gleemin[5], a "rules engine that aims to fully and correctly implement the game's tournament rules, an AI player programmable with expert knowledge and an interpreter for the game's language (the rules text on the cards)"! Seems you're not the first person to think about this!
[1]: http://archive.wizards.com/Magic/Magazine/Article.aspx?x=mtg...
[2]: http://gatherer.wizards.com/Pages/Card/Details.aspx?multiver...
[3]: http://gatherer.wizards.com/Pages/Card/Details.aspx?multiver...
[4]: http://mtgsalvation.gamepedia.com/Magic_Golden_Rules
[5]: https://github.com/stassa/Gleemin
Well.
And for people who know a bit about Magic: seems a lot like "Scars block without the Phyrexians", even down to the user posting a variation on Koth.
Stops people suing Wizards if they print a card identical or similar to a homebrew card.
Netrunner ICE generation would be interesting. I'd also like to run some algorithm over all Star Wars LCG cards and have it generate new arrangements of objective sets. [for those that don't know the game you don't build your deck by picking cards like in Magic but rather predefined bundles of 6 cards (objective sets)]