First, all images could've been painted by a human.
Second, if I always click the left one, it says Your result is 5/10. A turing test is to test "a machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human". If I – presumably a human being – score 5 out of 10, the test is not a Turing test after all, because humans aren't tested in a Turing test.
Edit: My observation was obviously wrong. Thanks for pointing it out.
You are not being tested, you are the judge. now, if all the judges do no better than chance, the system has passed. I got 8/10, and knowing the deepart pictures from before, I was pretty confident in some of the judgments, so I don't think they pass.
As a 10/10er, I think the fact that people tend to score less goes to show that:
1. The photos are low-res enough to hide the most ridiculous artifacts produced by neural nets
2. The machine-generated images are trying to emulate the likes of van Gogh, not the likes of Leonardo (again hiding the extent of NNs complete inability to understand what they're doing)
3. Most people simply neither paint nor appreciate visual art, so this is not as powerful as the Turing test! (One point of the Turing test is that a human easily passes it, because all humans speak; they don't all paint.)
It's much simpler than that. This is NOT a Turing test in that the computer generated images have been selected beforehand from the best results. What users are grading here is not the quality of the algorithm that generated the pictures; it's the quality of the choice made by the human who selected these specific pictures among all the examples of computer-generated art.
9/10. The low res made it harder to tell which one was computer generated, but it was still pretty clear. Now if they could do it full size, I'd start to be interested.
this is great work kidzik. looking forward to see more about it. hi-res are quite good, that even if in the uncanny they can be quite a success for deepart lovers.
I got 10/10 (no Facebook or Twitter account to share this, unfortunately.) Update: my wife got 10/10, too.
Right now neural nets do pretty poorly with "loose/messy" styles (you get noise instead of brushstrokes; if they posted high-resolution images there, I cannot believe anyone would score less than 10/10.) Neural nets do exceptionally poorly with "neat" styles (try copying the style of the Mona Lisa into your photo and it'll paint an eye into your mouth.)
However, I'm pretty sure that an algorithm hand-crafted to replicate an artist's style might fool me (just like a hand-crafted chess algorithm wipes the floor with a neural net plus some tree search; not so for Go, I know - I don't even play Go so I don't have a firm opinion on that one.)
I must say that this whole "photo + style" thing really gets my goat - not the research as much as the sort of press coverage it gets. I project that a stock market crash will improve such coverage significantly.
100% on this, and found it pretty easy to tell the difference. The main 'tell' was that the ape images had individual brushstrokes that were erratic in thickness/pressure/amount of paint, whereas the individual strokes in the artist images were generally more consistent.
Note that it is much easier for a computer to create a human-like painting than to behave like a human. The Turing test requires a two-way interaction medium.
Exactly right - the Turing test as it was originally conceived is very hard to implement (some argue impossible). To use the popular phrase, this is "not even wrong".
Got 8/10. Of the choices, I was instantly sure that this one wasn't computer-generated from a photo because of the unnatural dark area on the top: http://turing.deepart.io/t/6.jpg. I think (even if photos weren't involved) the fact that the dark area is out-of-place but still aesthetically congruent may suggest something about human creativity that would be hard to replicate using a computer.
Yes, exactly. While the background in the picture of the baseball player doesn't look photographic, at least the proportions are realistic. But the proportions of the female face are completely unrealistic. How could it be based on a photograph of a real human?
Are these computer-generated images based on photographs of real subjects, or on photographs of stylized paintings?
I don't think this test is very interesting until we can see the images the computer-generated ones were based on.
This is not a Turing test. Please correct me if I am wrong, but I bet that none of this was drawn by a computer. Most likely, those that were supposedly "drawn by computers" were pictures drawn by humans but applied various type of filtering (e.g. those by the deep learning algorithms).
Think what would really count as a "visual turing test" is if the computer-generated images were really "generated" by a computer, not rendered from a photo. Still, it's an interesting exercise in how much the rendering can make the photo feel "opinionated"—as if you can sense the painter behind it; that's the criteria I used for choosing, but I got a 7/10.
> what would really count as a "visual turing test"
I disagree - nobody would say drawings/paintings (by humans) based on photos or observations by eye are less legitimate than scenes completely imagined.
How about the following: The examiner can provide any picture of their choosing to two subjects A and B, where one is a human and the other a machine. At the end of a reasonable period, both subjects respond to the examiner simultaneously with one drawing each, derived from that same picture. In creating each drawing, A and B are both trying to convince the examiner that they are human. The examiner then should identify which of the two subjects is a computer and which is human. To better reflect the traditional Turing test, the examiner should be able to repeat this process with different pictures (keeping the identities of A and B unknown, but fixed) before making their determination.
Interestingly, this is a Turing test that, when applied purely to humans, doesn't require said humans to share a common language.
I have to quibble though that the Turing test is meant to be a test of intelligence, and this sort of task seems pretty different, although it may actually be visual-AI-hard (to invent a new term). So may deserve a different name.
You can call it a test of human-level aesthetic sensibility if you'd think a highly intelligent alien species would be unlikely to share our aesthetics. Although, arguably, a highly intelligent alien would be able to understand our aesthetics to the point where they pass the test regardless...
I think you are teasing me, but my answer would be: no, it doesn't count. The Turing test is about "understanding". The filtering of those images has no understanding of what the images are about.
The Turing test is not about understanding, it's simply about distinguishability. The idea is measuring understanding is hard, but it's easy to try to distinguish the production of a machine in several parameters by doing a blind test. So a computer may actually "understand more" of something but be blatantly computer-like (by being much better than a human could ever produce) -- a chess expert may identify a top level chess AI that way -- and thus fail the Turing test. So it's more of a sufficient but not necessary intelligence test.
Those photo modifications kind of fit into that theme: if you can't distinguish the filters from real art, they passed an indistinguishability test.
"one is painted by a human and another one is generated by artificial intelligence based on a photo and a style of a painter."
I find this to be easily "beatable" by simply judging which one is most likely to have its origin in a photo (vanity, and such) with a fallback on the one with a lot of repeating patterns.
If I wouldn't know that some of them were generated by ANN, I wouldn't question that like 90-95% of them were painted by a human. And I consider myself pretty alright at painting.
I'd be interested to see the original photos for those two. I expect a lot of the "art" we sense in those came from the photographer, not the algorithm.
does anybody know a way to run an algorithm with similar results on my own pictures? or how kids are calling it: "is there an open source version of this?"
10/10. My partner got 5/10, which he claims is because he's "more like a computer" (he's autistic). There were features in all of them that struck me as expressive in a very human way.
It's amazing to live in a time when questions about human identity and expression are more than just hypothetical.
Knowing that it was a turing test actually made it easier for me. (8/10) While I did take into account the painting style (the random rainbow look was suspect), I mostly ignored images where the subject looked like a photograph. This one threw me off on that basis though: http://turing.deepart.io/t/4.jpg.
I was only 3/10 - so I think it takes an eye for art to beat it. In my mind this is like a chess program getting to 1200. Not enough to beat an expert, but it's on to something.
I thought it was easier to tell which ones were computer generated rather than which were painted by a human. The computer art all seem rather abstract yet uniform.
also the high-fidelity, near-photographic paintings are always human.
Note that all of the computer generated pieces were created by this impressive deep net algorithm (paper prepublished in 8/2015): http://arxiv.org/abs/1508.06576
Somehow I cannot believe how fast they got the idea to create profit from it: http://deepart.io/
I work at DeepSense, and with these guys, but I didn't participate in this team. I was shocked as well; and now I try to learn from them as much as possible. :)
I wasn't very successful, but I tried to use "uniformity" criteria - if it looked like there are portions of the picture that were being omitted or abstracted away, I assumed human and vice versa.
I wonder if the algorithm could be improved if it had a sense of what is "important" in the picture, and then choose a different algorithm to process important and unimportant portions.
I got 6/10. I guessed that most of the things that looked "too real" were based on photos with filters applied to make it look like a painting, and that the abstract stuff was generated by humans. E.g. I was very surprised a computer produced this: http://turing.deepart.io/f/0.png or this http://turing.deepart.io/f/4.png
But the opposite heuristic doesn't work either, because not all the computer paintings are abstract. It mimics a lot of different styles and I am extremely impressed.
>I guessed that most of the things that looked "too real" were based on photos with filters applied to make it look like a painting, and that the abstract stuff was generated by humans.
I too used the same method and 5/10. Looking at the other comments, it seems people have better idea what art is than I do.
I'd be very interested to see the source images before processing, to see how the resulting images differ from the originals. FWIW, http://turing.deepart.io/t/8.jpg has a URL visible in the middle.
One thing I did find interesting is that I did the test initially and got 4/10 (most felt like guesses to be honest). I repeated the test before looking at the answers (though obviously knowing the score introduces some bias), but zoomed the page so the images were about 2-3x bigger than the default. My score increased to 9/10, and I was much more confident of my answers whilst selecting them.
9/10, but I was already familiar with the algorithm. Useful features to find the nature of the "artist" for me included the following:
- Is the style of the painting applied uniformly over the whole image? Humans will, for effect, leave parts of a painting less emphasized and developed, whereas this algorithm will generally apply a style to the entirety of an image.
- If a painting has a surreal style, the subject is generally also surreal. Human painters distort shapes and forms. This is not done by this algorithm.
- Humans will add contrast to make objects stand out, even if colours are similar to their background. This is something the computers haven't yet completely figured out.
87 comments
[ 3.7 ms ] story [ 192 ms ] threadFirst, all images could've been painted by a human.
Second, if I always click the left one, it says Your result is 5/10. A turing test is to test "a machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human". If I – presumably a human being – score 5 out of 10, the test is not a Turing test after all, because humans aren't tested in a Turing test.
Edit: My observation was obviously wrong. Thanks for pointing it out.
1. The photos are low-res enough to hide the most ridiculous artifacts produced by neural nets
2. The machine-generated images are trying to emulate the likes of van Gogh, not the likes of Leonardo (again hiding the extent of NNs complete inability to understand what they're doing)
3. Most people simply neither paint nor appreciate visual art, so this is not as powerful as the Turing test! (One point of the Turing test is that a human easily passes it, because all humans speak; they don't all paint.)
Right now neural nets do pretty poorly with "loose/messy" styles (you get noise instead of brushstrokes; if they posted high-resolution images there, I cannot believe anyone would score less than 10/10.) Neural nets do exceptionally poorly with "neat" styles (try copying the style of the Mona Lisa into your photo and it'll paint an eye into your mouth.)
However, I'm pretty sure that an algorithm hand-crafted to replicate an artist's style might fool me (just like a hand-crafted chess algorithm wipes the floor with a neural net plus some tree search; not so for Go, I know - I don't even play Go so I don't have a firm opinion on that one.)
I must say that this whole "photo + style" thing really gets my goat - not the research as much as the sort of press coverage it gets. I project that a stock market crash will improve such coverage significantly.
I guess as of today T-1000 would get his bitch ass kicked.
Are these computer-generated images based on photographs of real subjects, or on photographs of stylized paintings?
I don't think this test is very interesting until we can see the images the computer-generated ones were based on.
- obvious artifacts in the neural networks
- humans paint light differently than how a camera captures them, evident even after heavy processing.
I disagree - nobody would say drawings/paintings (by humans) based on photos or observations by eye are less legitimate than scenes completely imagined.
Interestingly, this is a Turing test that, when applied purely to humans, doesn't require said humans to share a common language.
I have to quibble though that the Turing test is meant to be a test of intelligence, and this sort of task seems pretty different, although it may actually be visual-AI-hard (to invent a new term). So may deserve a different name.
Those photo modifications kind of fit into that theme: if you can't distinguish the filters from real art, they passed an indistinguishability test.
"one is painted by a human and another one is generated by artificial intelligence based on a photo and a style of a painter."
I find this to be easily "beatable" by simply judging which one is most likely to have its origin in a photo (vanity, and such) with a fallback on the one with a lot of repeating patterns.
I don't think programs written to imitate a craft, or even to learn to imitate a craft from examples, count as AI, no matter how impressive.
If I wouldn't know that some of them were generated by ANN, I wouldn't question that like 90-95% of them were painted by a human. And I consider myself pretty alright at painting.
These 2 are especially awesome:
http://turing.deepart.io/f/0.png
http://turing.deepart.io/t/9.jpg
It's amazing to live in a time when questions about human identity and expression are more than just hypothetical.
I am convinced intelligence programming will tackle these issues some day. But apparently not yet.
I suspect that we can be trained to spot fakes too to make it harder. Perhaps there will be some co-evolution?
also the high-fidelity, near-photographic paintings are always human.
Somehow I cannot believe how fast they got the idea to create profit from it: http://deepart.io/
As a side note, for me the pace was one of the reasons to move from science to data science (shameless plug: http://p.migdal.pl/2015/12/14/sci-to-data-sci.html).
I wonder if the algorithm could be improved if it had a sense of what is "important" in the picture, and then choose a different algorithm to process important and unimportant portions.
But the opposite heuristic doesn't work either, because not all the computer paintings are abstract. It mimics a lot of different styles and I am extremely impressed.
I too used the same method and 5/10. Looking at the other comments, it seems people have better idea what art is than I do.
One thing I did find interesting is that I did the test initially and got 4/10 (most felt like guesses to be honest). I repeated the test before looking at the answers (though obviously knowing the score introduces some bias), but zoomed the page so the images were about 2-3x bigger than the default. My score increased to 9/10, and I was much more confident of my answers whilst selecting them.
- Is the style of the painting applied uniformly over the whole image? Humans will, for effect, leave parts of a painting less emphasized and developed, whereas this algorithm will generally apply a style to the entirety of an image.
- If a painting has a surreal style, the subject is generally also surreal. Human painters distort shapes and forms. This is not done by this algorithm.
- Humans will add contrast to make objects stand out, even if colours are similar to their background. This is something the computers haven't yet completely figured out.
Still, this is a very impressive algorithm.