Stable Diffusion 3 appears to be a repeat of the SD2 fiasco which significantly kneecapped its adoption.
Except this time, it will be even worse. SD2's lack of good human generation to correct its faults could be fixed with finetuning, but SD3's noncommmercial-unless-you-pay-us license will disincentivize any community finetuning.
FOR IMMEDIATE RELEASE - Site ---, -----, California: The SCP Foundation is proud to announce that in the interests of furthering commerce and humanity in general that it is entering into a partnership with Stability AI.
"We have unique assets and experience which we can bring to synergize with the world-class leading AI efforts that Stability AI has been bringing to market, and we are excited to bring these ------- to the common person on the street." said the Administrator, -------------------------.
"-------!" ---------.
Early testing have users shocked, ---------, and ---------.
"I see the world through new eyes now, all seven of them." reported one user.
So this looks like a reasonable-on-paper critique of AI models that ended up costing dearly.
One side question I haven't seen much (intelligent) discussion of: to what extent is the intensity of AI critiques disingenuous in origin?
Unfortunately, disingenuity discussions become instantly meaningless when they devolve into politics. Better to just assume that all politicians are disingenuous in everything and move on.
I would say there's a category of unpardonable voices, such as competition-suppressing established companies (and wishful ones like musk.ai) and dick-tators targeting AI as a proxy for Western economies. However, I'd pardon anxious content creators, the same way I'd pardon Lyft drivers critiquing self-driving cars (and cab drivers critiquing Lyft 10 years ago.)
The Goldilocks region is elusive. For example, it's unclear if SD3's failures are actually due to NSFW filtering. I suppose the disingenuity test could be based on the speaker's technical credentials, but that would probably dismiss too much speech.
Is there any evidence that this has anything to do with nudity filters? The article doesn't provide any evidence beyond "user's are blaming it on filters". Given most of the mangled parts appear to be hands (something AI already sucked at), and some of the photos generated are women with reasonable cleavage or showing a lot of legs, it doesn't seem obvious to me that's the case. Certainly there's plenty in those images more suggestive than some fingers, but I'm curious if there's any actual supporting evidence.
I don't think there is any hard evidence except people looking at correlations. SDXL, one of the preceding versions, is allegedly less censored and happens to do anatomy better than the examples in the article. Logically, it makes sense that omitting good data would lead to worse results - nudity filters or not, anatomy is very complex and humans are very sensitive to inaccuracies in human looks, so a giant dataset with as many different poses as possible is the way to go. But in terms of actual empirical tests and measurements - I haven't seen that yet.
I think it's mainly from the fact that they had the same problem in SD 2.0. In the interest of safety they removed so many NSFW images from the training set to the point that it could no longer generate coherent images of humans in general. This was resolved when they added NSFW pictures back into the training data for SD 2.1.
We won't really know for sure unless stability ai comments on it or releases the training data but it is the most likely explanation.
Someone will likely train this model on porn anyway in the next few months so we'll see if the finetuned models work better.
I knew that good morals and modesty would save us someday. It is clear now that human artists will remain gainfully employed by producing high quality erotic art. /s.
I did a little bit of testing. It definitely needs to be fine-tuned with more images of humans so that it understands basic anatomy better.
But for the prompts I actually tried, the speed and prompt adherence was a significant step up. The overall quality does not compare at all to SDXL fine-tuned.
I think overall people should appreciate their improvements to the architecture. Although the deficits are bizarre and point to a huge hole in the market for new companies to get involved in pre-training models.
But there will be fine-tuned on Civit AI.
Also, people kept pointing out how good PixArt is.
Many of the most popular finetuned models are sponsored by paid image generation services, which provide either funding or GPU time for training. Those generation services won't be able to use SD3 under the current licensing, so there will be no incentive for them to sponsor improvements.
Loras are easy to train on consumer hardware, but a fine-tune like Pony takes significantly more resources and a lot of work in terms of image tagging, curation, etc.
25 comments
[ 3.0 ms ] story [ 73.9 ms ] threadExcept this time, it will be even worse. SD2's lack of good human generation to correct its faults could be fixed with finetuning, but SD3's noncommmercial-unless-you-pay-us license will disincentivize any community finetuning.
"We have unique assets and experience which we can bring to synergize with the world-class leading AI efforts that Stability AI has been bringing to market, and we are excited to bring these ------- to the common person on the street." said the Administrator, -------------------------.
"-------!" ---------.
Early testing have users shocked, ---------, and ---------.
"I see the world through new eyes now, all seven of them." reported one user.
"------- ----- - ---------- ---------." reported another.
"We anticipate great profits and ----------" says Stability AI's CFO, shortly before sitting down, sitting down, sitting down, sitting down, sitting down, and sitting down.
---------.
One side question I haven't seen much (intelligent) discussion of: to what extent is the intensity of AI critiques disingenuous in origin?
Unfortunately, disingenuity discussions become instantly meaningless when they devolve into politics. Better to just assume that all politicians are disingenuous in everything and move on.
I would say there's a category of unpardonable voices, such as competition-suppressing established companies (and wishful ones like musk.ai) and dick-tators targeting AI as a proxy for Western economies. However, I'd pardon anxious content creators, the same way I'd pardon Lyft drivers critiquing self-driving cars (and cab drivers critiquing Lyft 10 years ago.)
The Goldilocks region is elusive. For example, it's unclear if SD3's failures are actually due to NSFW filtering. I suppose the disingenuity test could be based on the speaker's technical credentials, but that would probably dismiss too much speech.
We won't really know for sure unless stability ai comments on it or releases the training data but it is the most likely explanation.
Someone will likely train this model on porn anyway in the next few months so we'll see if the finetuned models work better.
But for the prompts I actually tried, the speed and prompt adherence was a significant step up. The overall quality does not compare at all to SDXL fine-tuned.
I think overall people should appreciate their improvements to the architecture. Although the deficits are bizarre and point to a huge hole in the market for new companies to get involved in pre-training models.
But there will be fine-tuned on Civit AI.
Also, people kept pointing out how good PixArt is.
Due to the noncommercial licensing, there will not be nearly as much finetuning as with SDXL.
And then Lora’s are small enough to do without large resources
Loras are easy to train on consumer hardware, but a fine-tune like Pony takes significantly more resources and a lot of work in terms of image tagging, curation, etc.