Style transfer is done using neural networks, so it should be natural that the resultant image fools a neural network classifier. On the other hand, for the filled silhouette, the classifier network was pretty close to human.
Texture is area-filling, while silhouette is "just a line", so it sounds natural that texture areas would weigh more in an image? If there is both texture and contour "signal" in the labeled inputs, how to pick what get's weighted more? This is supposed to be learned by training.
I think, this is one of the reasons to use much more video-based training sets and these in time-correct frame order. A moving boundary is something quite significant, but these should possibly not exclusively inspected through a hole ('convolutinal')
> it is textures not shapes that drives recognition in CNN
This is a misinterpretation of the paper. It's not CNNs that are biased toward textures, it's the ImageNet dataset. As they prove by training a CNN biased toward shapes, using a modified dataset.
I thought the same. Captain Obvious called a second time. Of course CNNs are still heavily biased towards texture, because nothing has fundamentally changed in the past 5 years.
They are, and it is not that new; the interesting part is how to solve that.
For image classification, this bias for textures may be passable. For image generation, it would produce atrocious results, especially for faces. To the point, that people devised some methods to overcome this, especially with progressive networks (e.g. proGAN). See https://towardsdatascience.com/progan-how-nvidia-generated-i... for the proGAN explanation - it starts from training on 4x4 pixel images, to get the overall shape.
I'm curious if stereo vision and eyepoint movement has a benefit for humans in distinguishing shape vs texture, and perhaps that could be a promising approach for ML, but are there any databases for that? There are car focused ones like KITTI, but any more generalist ones?
16 comments
[ 4.2 ms ] story [ 44.0 ms ] threadhttps://github.com/minimaxir/hacker-news-undocumented
[1] “Suddenly, a leopard print sofa appears”, http://rocknrollnerd.github.io/ml/2015/05/27/leopard-sofa.ht...
This is a misinterpretation of the paper. It's not CNNs that are biased toward textures, it's the ImageNet dataset. As they prove by training a CNN biased toward shapes, using a modified dataset.
https://www.quantamagazine.org/where-we-see-shapes-ai-sees-t...
For image classification, this bias for textures may be passable. For image generation, it would produce atrocious results, especially for faces. To the point, that people devised some methods to overcome this, especially with progressive networks (e.g. proGAN). See https://towardsdatascience.com/progan-how-nvidia-generated-i... for the proGAN explanation - it starts from training on 4x4 pixel images, to get the overall shape.