Is it just me or is the stable diffusion bus image broken in the background? The bus back there does not look logical w.r.t placement and size relative to the sidewalk.
This is a good question - not only for the actual ethics of the training, but for the future of AI use for art. It's both gonna damage the livelyhood of many artists (me included, probably) but also make it accessibly to many more people. As long as the training dataset is ethical, I think fighting it is hard and pointless.
It is a challenge for these models to generate images of counterintuitive or unusual situations that aren't depicted in the training set. For example, if you ask for a small cube sitting on top of a large cube, you'll likely get the correct result on the first attempt. Ask for a large cube on a small cube and you'll probably get an image of them side-by-side or with the small cube on top instead. The models can generalize in impressive ways, but it's still limited.
It's likely a result of the interplay between the image generation and caption/description generation aspects of the model. The earliest diffusion-based image generators used a 'bag of words' model for the caption (see musing regarding this and DALL-E 3: https://old.reddit.com/r/slatestarcodex/comments/16y14co/sco...), whereby 'a woman chasing a bear' would turn into `['a', 'a', 'chasing', 'bear', 'woman']`.
That's good enough to describe compositions well-represented in the training set, but it will be likely to lock-in to those common representations at the expense of rarer but still possible ones (the 'woman chasing a bear' above).
A while ago my daughter wanted an image of Santa pulling a sleigh with a reindeer in the driver's seat holding the reins. We tried dozens of different prompts and Dall-e 3 could not do it.
Being able to generate content w/ minimal presence in the training set is arguably an emergent, desirable behavior that could be seen as a form of intelligence.
> We believe in safe, responsible AI practices. This means we have taken and continue to take reasonable steps to prevent the misuse of Stable Diffusion 3 by bad actors. Safety starts when we begin training our model and continues throughout the testing, evaluation, and deployment. In preparation for this early preview, we’ve introduced numerous safeguards. By continually collaborating with researchers, experts, and our community, we expect to innovate further with integrity as we approach the model’s public release.
What exactly does this mean? Will we be able to see all of the "safeguards" and access all of the technology's power without someone else's restrictions on them?
For SDXL this meant that there were almost no NSFW (porn and similar) images included in the dataset, so the community had to fine-tune the model themselves to make it generate those.
I guess this statement is a cheap protection against cheap journalists. Otherwise by now all the tabloids would be full of scary stories about deepfake politicians, deep-porn and all types of blackmailers (by the way, there is so much competition in AI now that some company may well pay for a wave of such articles to destroy the competitor). And in response to this, hearty old men would clobber the Congress with petitions to immediately ban all AI. Who wants that?
at this point perfect text would be a gamechanger if it can be solved
midjourney 6 can be completely photorealistic and include valid text, but also sometimes adds bad text. it's not much, but having to use an image editor for that is still annoying. for creating marketing material, getting perfect text every time and never getting bad text would be amazing
I wonder if we could get it to generate a layered output, to make it easy to change just the text layer. It already creates the textual part in a separate pass, right?
I would bet that Adobe is definitely salivating at that. Might not be for a long time but it seems like a no brainer once the technology can handle it. Just the last few years have been fast and I interacted with the JS landscape for a few years. It moves faster than Sonic and this tech iterates quick.
Current open source tools include pretty decent off the shelf segment anything based detectors. It leaves a lot to be desired, but you do layer-like operations automatically detecting certain concept and applying changes to them or, less commonly exporting the cropped areas. But not the content "beneath" the layers as they don't exist.
There is some truth in what you say, just like saying you're a "free speech absolutist" sounds good at first blush. But the real world is more complicated, and the provider adds safety features because they have to operate in the real world and not just make superficial arguments about how things should work.
Yes, they are protecting themselves from lawsuits, but they are also protecting other people. Preventing people asking for specific celebrities (or children) having sex is for their benefit too.
I truly wonder what "unsafe" scenarios an image generator could be used for? Don't we already have software that can do pretty much anything if a professional human is using it?
I would say the barrier to entry is stopping a lot of ‘candid’ unsafe behaviour. I think you allude to it yourself in implying currently it requires a professional to achieve the same results.
But giving that ability to _everyone_ will lead to a huge increase in undesirable and targeted/local behaviour.
Presumably it enables any creep to generate what they want by virtue of being able to imagine it and type it, rather than learn a niche skill set or employ someone to do it (who is then also complicit in the act)
Where are the Americans asking about Snapchat? If I were a developer at Scnapchat I could prolly open a few Blob Storage accounts and feed a darknet account big enough to live off of. You people are so manipulatable.
Because AI lowers the barrier to entry; using your example, few people have the drawing skills (or the patience to learn them) or take the effort to make a picture like that, but the barrier is much lower when it takes five seconds of typing out a prompt.
Second, the tool will become available to anyone, anywhere, not just a localised school. If generating naughty nudes is frowned upon in one place, another will have no qualms about it. And that's just things that are about decency, then there's the discussion about legality.
Finally, when person A draws a picture, they are responsible for it - they produced it. Not the party that made the pencil or the paper. But when AI is used to generate it, is all of the responsibility still with the person that entered the prompt? I'm sure the T's and C's say so, but there may still be lawsuits.
Right, these are the same arguments against uncontrolled empowerment that I imagine mass literacy and the printing press faced. I would prefer to live in a society where individual freedom, at least in the cognitive domain, is protected by a more robust principle than "we have reviewed the pros and cons of giving you the freedom to do this, and determined the former to outweigh the latter for the time being".
You seem to be very confused about civil versus criminal penalties....
Feel free to make an AI model that does almost anything, though I'd probably suggest that it doesn't make porn of minors as that is criminal in most jurisdiction, short of that it's probably not a criminal offense.
Most companies are only very slightly worried about criminal offenses, they are far more concerned about civil trials. There is a far lower requirement for evidence. AI creator in email "Hmm, this could be dangerous". That's all you need to lose a civil trial.
Why do you figure I would be confused? Whether any liability for drawing porn of classmates is civil or criminal is orthogonal to the AI comparison. The question is if we would hold manufacturers of drawing tools or software, or purveyors of drawing knowledge (such as learn-to-draw books), liable, because they are playing the same role as the generative AI does here.
Because you seem to be very confused on civil liabilities in most products. Manufactures are commonly held liable for the users use of products, for example look at any number of products that have caused injury.
Surely those are typically when the manufacturer was taken to have made an implicit promise of safety to the user and their surroundings, and the user got injured. If your fridge topples onto you and you get injured, the manufacturer might be liable; if you set up a trap where you topple your fridge onto a hapless passer-by, the manufacturer will probably not be liable towards them. Likewise with the classic McDonalds coffee spill liability story - I've yet to hear of a case of a coffee vendor being held liable over a deliberate attack where someone splashed someone else with hot coffee.
Photoshop also lowers that barrier of entry compared to pen and pencil. Paper also lowers the barrier compared to oil canvas.
Affordable drawing classes and YouTube drawing tutorials lower the barrier of entry as well.
Why on earth would manufacturers of pencils, papers, drawing classes, and drawing software feel responsible for censoring the result of combining their tool with the brain of their customer?
A sharp kitchen knife significantly lowers the barrier of entry to murder someone. Many murders are committed everyday using a kitchen knife. Should kitchen knife manufacturers blog about this every week?
I agree with your point, but I would be willing to bet that if knives were invented today rather than having been around awhile, they would absolutely be regulated and restricted to law enforcement if not military use. Hell, even printers, maybe not if invented today but perhaps in a couple years if we stay on the same trajectory, would probably require some sort of ML to refuse to print or "reproduce" unsafe content.
I guess my point is that I don't think we're as inconsistent as a society as it seems when considering things like knives. It's not even strictly limited to thought crimes/information crimes. If alcohol were discovered today , I have no doubt that it would be banned and made schedule I
> Hell, even printers, maybe not if invented today but perhaps in a couple years if we stay on the same trajectory, would probably require some sort of ML to refuse to print or "reproduce" unsafe content.
Fun fact: Many scanners and photocopiers will detect that you're trying to scan/copy a banknote and will refuse to complete the scan. One of the ways is detecting the EURion Constellation.
IANAL but that sounds like harrassment, I assume the legality of that depends on the context (did the artist previously date the subject? lots of states have laws against harassment and revenge porn that seem applicable here [1]. are you coworkers? etc), but I don't see why such laws wouldn't apply to AI generated art as well. It's the distribution that's really the issue in most cases. If you paint secret nudes and keep them in your bedroom and never show them to anyone it's creepy, but I imagine not illegal.
I'd guess that stability is concerned with their legal liability, also perhaps they are decent humans who don't want to make a product that is primarily used for harassment (whether they are decent humans or not, I imagine it would affect the bottom line eventually if they develop a really bad rep, or a bunch of politicians and rich people are targeted by deepfake harassment).
^ a lot of, but not all of those laws seem pretty specific to photographs/videos that were shared with the expectation of privacy and I'm not sure how they would apply to a painting/drawing, and I certainly don't know how the courts would handle deepfakes that are indistinguishable from genuine photographs. I imagine juries might tend to side with the harassed rather than a bully who says "it's not illegal cause it's actually a deepfake but yeah i obviously intended to harass the victim"
That's not even necessarily a bad thing (as a whole - individually it can be). Now, any leaked nudes can be claimed to be AI. That'll probably save far more grief than it causes.
The case I literally just referenced allows you to paint nude children engaged in sex acts.
> The Ninth Circuit reversed, reasoning that the government could not prohibit speech merely because of its tendency to persuade its viewers to engage in illegal activity.[6] It ruled that the CPPA was substantially overbroad because it prohibited material that was neither obscene nor produced by exploiting real children, as Ferber prohibited.[6] The court declined to reconsider the case en banc.[7] The government asked the Supreme Court to review the case, and it agreed, noting that the Ninth Circuit's decision conflicted with the decisions of four other circuit courts of appeals. Ultimately, the Supreme Court agreed with the Ninth Circuit.
You’re very welcome to ask for clarification - I kept it abstract because there is a lot of grey area and it’s something we need to understand and discuss as technology and society evolves.
To spell out one such instance: I would like to live in a world where it is not trivial to depict and misrepresent me (or anyone) in a way that is photorealistic to the point that it can be used to mislead others.
Whether that means we need to outright prevent it, or have some kind of authenticity mechanism, or some other yet-to-be-discovered solution? I do not know, but you now have my goalposts.
In a large number of countries if you create an image that represents a minor in a sexual situation you will find yourself on the receiving side of the long arm of the law.
If you are the maker of an AI model that allows this, you will find yourself on the receiving side of the long arm of the law.
Moreso, many of these companies operate in countries where thought crime is illegal. Now, you can argue that said companies should not operate in those countries, but companies will follow money every time.
I think it's pretty important to specify that you have to willingly seek and share all of these illegal items. That's why this is so sketch. These things are being baked with moral codes that'll _share_ the information, incriminating everyone. Like why? Why not just let it work and leave it up to the criminal to share their crimes? People are such authoritarian shit-stains, and acting like their existence is enough to justify their stance is disgusting.
Eh, a professional human could easily lockpick the majority of front doors out there. Nevertheless I don't think we're going to give up on locking our doors any time soon.
Similar to why Google's latest image generator refuses to produce a correct image of a 'Realistic, historically accurate, Medieval English King'. They have guard rails and system prompts set up to force the output of the generator with the company's values, or else someone would produce Nazi propaganda or worse. It (for some reason) would be attributed to Google and their AI, rather than the user who found the magic prompt words.
For some scenarios, it's not the image itself but the associations that the model might possibly make from being fed a diet of 4chan and Stormfront's unofficial YouTube channel. The worry is over horrible racist shit, like if you ask it for a picture of a black person, and it outputs a picture of a gorilla. Or if you ask it for a picture of a bad driver, and it only manages to output pictures of Asian women. I'm sure you can think up other horrible stereotypes that would result in a PR disaster.
It's also "safety" in the sense that you can deploy it as part of your own application without human review and not have to worry that it's gonna generate anything that will get you in hot water.
AI is not an engineered system; it's emergent behavior from a system we can vaguely direct but do not fundamentally understand. So it's natural that the boundaries of system behavior would be a topic of conversation pretty much all the time.
EDIT: Boring and shallow are, unfortunately, the Internet's fault. Don't know what to do about those.
At least in some latest controversies (e.g. Gemini generation of people) all of the criticized behavior was not emergent from ML training, but explicitly intentionally engineered manually.
But that's the thing, prompt formulation is not engineering in the sense I'm talking about. We know why a plane flies, we know why an engine turns, we know how a CPU works - mostly. We don't know how GenAI gets from the prompt to the result with any specificity at all. Almost all the informational entropy of the output is hidden from us.
This is the world we live in. CYA is necessary. Politicians, media organizations, activists and the parochial masses will not brook a laissez faire attitude towards the generation of graphic violence and illegal porn.
looking at the manual censorship of the big channels on youtube, you don't even need to display anything, just suggesting it is enough to get a strike.
(of course unless you are into yoga, then everything is permitted)
Great talk about slavery and religious-persecution, Jim! Wait, what were we talking about? Fucking American fascists trying to control our thoughts and actions, right right.
If AMD and Nvidia managed to understand what you are asking for just looking at a bunch of vectors and manipulate the outputs to a goal, that would be a serious breakthrough in the field.
Even if it did that by looking at the process memory (which would be seriously wrong in many ways), just the manipulation part would be mighty impressive. And if it was the case I guess we would see many papers and heated discussions about them.
Bro I have generated literally thousands of monster girls in stable diffusion and all of my outputs have been spectacular. Driver changes have not affected my generation at all
Because floating point math has different results depending on the order of operations. A major reason why you'd update GPU drivers is to reorder ops for better cache locality so you can light up more transistors
While that thoughtpiece makes me not like him as a person, your point is correct; the comment you're replying to is a low effort ad hominem, attacking the person and something they said about a different subject instead of addressing the actual remark about models.
Come on, his point is not about AI but about politics: "they" versus "you".
And he is completely, totally incompetent on this - and by the way, he's also completely incompetent on AI, and on most of tech. See his stint at twitter...
Not specifically SD, but DallE: I wanted to get an image of a pure white British shorthair cat on the arm of a brunette middle-aged woman by the balcony door, both looking outside.
It wasn‘t important, just something I saw in the moment and wanted to see what DallE makes of it.
Generation denied. No explanation given, I can only imagine that it triggered some detector of sexual request?
(It wasn‘t the phrase "pure white", as far as I can tell, because I have lots of generated pics of my cat in other contexts)
Well for starters, ChatGPT shouldn't balk at creating something "in Tim Burton's style" just because Tim Burton complained about AI. I guess its fair use unless a select rich person who owns the data complains. Seems like it isn't fair use at all then, just theft from those who cannot legally defend themselves.
Fair use is an exception to copyright. The issue here is that it's not fair use, because copyright simply does not apply. Copyright explicitly does not, has never, and will never protect style.
Didn't Tom Waits successfully sue Frito Lay when the company found an artist that could closely replicate his style and signature voice, who sang a song for a commercial that sounded very Tom Waits-y?
Yes, though explicitly not for copyright infringement. Quoting the court's opinion, "A voice is not copyrightable. The sounds are not 'fixed'." The case was won under the theory of "voice misappropriation", which California case law (Midler v Ford Motor Co) establishes as a violation of the common law right of publicity.
That makes it even more ridiculous, as that means they are giving rights to rich complaining people that no one has.
Examples:
Can you great an image of a cat in Tim Burton's style?
Oops! Try another prompt
Looks like there are some words that may be automatically blocked at this time. Sometimes even safe content can be blocked by mistake. Check our content policy to see how you can improve your prompt.
Can you create an image of a cat in Wes Anderson's style?
Certainly! Wes Anderson’s distinctive style is characterized by meticulous attention to detail, symmetrical compositions, pastel color palettes, and whimsical storytelling. Let’s imagine a feline friend in the world of Wes Anderson...
I don’t feel like it truly matters since they’ll release it and people will happily fine-tune/train all that safety right back out.
It sounds like a reputation/ethics thing to me. You probably don’t want to be known as the company that freely released a model that gleefully provides images of dismembered bodies (or worse).
You are using someone elses propietary technology, you have to deal with their limitations. If you don't like there are endless alternatives.
"Wrongly denied" in this case depends on your point of view, clearly DALL-E didn't want this combination of words created, but you have no right for creation of these prompts.
I'm the last one defending large monolithic corps, but if you go to one and want to be free to do whatever you want you are already starting from a very warped expectation.
As far as Stable Diffusion goes - when the released SD 2.1/XL/Stable Cascade, you couldn't even make a (woman's) nipple.
I don't use them for porn like a lot of people seem too, but it seems weird to me that something that's kind of made to generate art can't generate one of the most common subjects in all of art history - nude humans.
For some reason its training thinks they are decorative, I guess it’s a pretty funny elucidation of how it works.
I have seen a lot of “pasties” that look like Sorry! game pieces, coat buttons, and especially hell-forged cybernetic plumbuses. Did they train it at an alien strip club?
The LoRAs and VAEs work (see civit.ai), but do you really want something named NSFWonly in your pipeline just for nipples? Haha
I seem to have the opposite problem a lot of the time. I tried using Meta's image gen tool, and had such a time trying to get it to make art that was not "kind of" sexual. It felt like Facebook's entire learning chain must have been built on people's sexy images of their girlfriend that's all now hidden in the art.
These were examples that were not super blatant, like a tree landscape that just happens to have a human figure and cave in their crotch. Examples:
Not meant in a rude way, but please consider that your brain is making these up and you might need to see a therapist. I can see absolutely nothing "kind of sexual" in those two pictures.
Not taken as rude. If its not an issue, then that's actually a positive for you. It means less time taken reloading trying to get it to not look like a human that happens to be made out of mountains.
I have in fact gotten a nude out of Stable Cascade. And that's just with text prompting, the proper way to use these is with multimodal prompting. I'm sure it can do it with an example image.
Oh the big one would be models weights being released for anyone to use or fine tune themselves.
Sure, the safety people lost that battle for Stable diffusion and LLama. And because they lost, entire industries were created by startups that could now use models themselves, without it being locked behind someone else's AI.
But it wasn't guaranteed to go that way. Maybe the safetyists could have won.
I don't we'd be having our current AI revolution if facebook or SD weren't the first to release models, for anyone to use.
What's equally interesting is that while they spend a lot of words on safety, they don't actually say anything. The only hint what they even mean by safety is that they took "reasonable steps" to "prevent misuse by bad actors". But it's hard to be more vague than that. I still have no idea what they did and why they did it, or what the threat model is.
Maybe that will be part of future papers or the teased technical report. But I find it strange to put so much emphasis on safety and then leave it all up to the reader's imagination.
Any large publicly available model has no choice but to do this. Otherwise, they're petrified of a PR nightmare.
Models with a large user base will have an inverse relationship with usability. That's why it's important to have options to train your own with open source.
I get a slightly uncomfortable feeling with this talk about AI safety. Not in the sense that there is anything wrong with that (may be or may be not), but in the sense I don't understand what people are talking about when they talk about safety in this context. Could someone explain like I have Asperger (ELIA?) whats this about? What are the "bad actors" possibly going to do? Generate (child) porn/ images with violence etc. and sell them? Pollute the training data so that the racist images pops up when someone wants to get an image of a white pussycat? Or produce images that contain vulnerabilities so that when you open that in your browser you get compromised? Or what?
You sound offended. My apologies. I had no intention whatsoever to offend anyone. Even if I am not diagnosed, I think I am at least borderline somewhere in the spectrum, and thought that would be a good way to ask people explain without assuming I can read between the lines.
I think ELI5 means that you simplify a complex issue so that even a small kid understands it. In this case there is no need to simplify anything, just explain what a term actually means without assuming reader understanding nuances of terms used. And I still do not quite get how ELIA can be considered hostile, but given the feedback, maybe I avoid it in the future.
Saying "explain like I have <specific disability>" is blatantly inappropriate. As a gauge: Would you say this to your coworkers? Giving a presentation? Would you say this in front of (a caretaker for) someone with Autism? Especially since Asperger's hasn't even been used in practice for, what, over a decade?
> In this case there is no need to simplify anything
I don't see how this is a response to anything I've said. They're speaking to other humans and the original use of their modified idiom isn't framed as if one were talking about their own, personal disability.
I'm not part of Stability AI but I can take a stab at this:
> explain like I have ~~Asperger (ELIA?)~~ limited understanding of how the world really works.
The AI is being limited so that it cannot produce any "offensive" content which could end up on the news or go viral and bring negative publicity to Stability AI.
Viral posts containing generated content that brings negative publicity to Stability AI are fine as long as they're not "offensive". For example, wrong number of fingers is fine.
There is not a comprehensive, definitive list of things that are "offensive". Many of them we are aware of - e.g. nudity, child porn, depictions of Muhammad. But for many things it cannot be known a priori whether the current zeitgeist will find it offensive or not (e.g. certain depictions of current political figures, like Trump).
Perhaps they will use AI to help decide what might be offensive if it does not explicitly appear on the blocklist. They will definitely keep updating the "AI Safety" to cover additional offensive edge cases.
It's important to note that "AI Safety", as defined above (cannot produce any "offensive" content which could end up on the news or go viral and bring negative publicity to Stability AI) is not just about facially offensive content, but also about offensive uses for milquetoast content. Stability AI won't want news articles detailing how they're used by fraudsters, for example. So there will be some guards on generating things that look like scans of official documents, etc.
Yes*. At least for the purposes of understanding what the implementations of "AI safety" are most likely to entail. I think that's a very good cognitive model which will lead to high fidelity predictions.
*But to be slightly more charitable, I genuinely think Stability AI / OpenAI / Meta / Google / MidJourney believe that there is significant overlap in the set of protections which are safe for the company, safe for users, and safe for society in a broad sense. But I don't think any released/deployed AI product focuses on the latter two, just the first one.
Examples include:
Society + Company: Depictions of Muhammad could result in small but historically significant moments of civil strife/discord.
Individual + Company: Accidentally generating NSFW content at work could be harmful to a user. Sometimes your prompt won't seem like it would generate NSFW content, but could be adjacent enough: e.g. "I need some art in the style of a 2000's R&B album cover" (See: Sade - Love Deluxe, Monica - Makings of Me, Rihanna - Unapologetic, Janet Jackson - Damita Jo)
Society + Company: Preventing the product from being used for fraud. e.g. CAPTCHA solving, fraudulent documentation, etc.
Individual + Company: Preventing generation of child porn. In the USA, this would likely be illegal both for the user and for the company.
I think this AI safety thing is great. These models will be used by people to make boring art. The exciting art will be left for people to make.
This idea of AI doing the boring stuff is good. Nothing prevents you from making exciting, dangerous, or 'unsafe' art on your own.
My feeling is that most people who are upset about AI safety really just mean they want it to generate porn. And because it doesn't, they are upset. But they hide it under the umbrella of user freedom. You want to create porn in your bedroom? Then go ahead and make some yourself. Nothing stopping you, the person, from doing that.
I agree with you, but when companies don't implement these things, they get absolutely trashed in the press & social media, which I'm sure affects their business.
What would you have them do? Commit corporate suicide?
This is a good question. I think it would be best for them to give some sort of signal, which would mean "We're doing this because we have to. We are willing to change if you offer us an alternative." If enough companies/people did this, at some point change would become possible.
This reinforces my impression that Google is at least one year behind. Stunning images, 3D, video while Gemini had to be partially halted this morning.
You think that technology is first. You think that mathematicians and computer engineers or mechanical engineers or doctors are first. They’re very important, but they’re not first. They’re second. Now I’ll prove it to you.
There was a country that had the best mathematicians, the best physicists, the best metallurgists in the world. But that country was very poor. It’s called the Soviet Union. But when you took one of these mathematicians or physicists, who was smuggled out or escaped, put him on a plane and brought him to Palo Alto. Within two weeks, they were producing added value that could produce great wealth.
What comes first is markets. If you have great technology without markets, without a market-friendly economy, you’ll get nowhere. But if you have a market-friendly economy, sooner or later the market forces will give you the technology you want.
And that my friend, simply won't come from an office paralyzed by internal politics of fear and conformity. Don't get it twisted.
This reinforces my impression that Google is at least one year behind. Stunning images, 3D, video while Gemini had to be partially halted this morning.
People always say Google is "behind", I don't believe they're behind in a capabilities sense, which IMO is what the parent is implying. They've decided to make a PC product, which I wouldn't say is inferior to anything else if you're the kind of person who is into PC culture.
There might be some difficult internal politics to work through, but there is no way Google is hamstrung forever by this.
> They've decided to make a PC product, which I wouldn't say is inferior to anything else if you're the kind of person who is into PC culture.
I'm not.
> There might be some difficult internal politics to work through, but there is no way Google is hamstrung forever by this.
The technical prowess is irrelevant. Whichever of the companies in the AI race excises their PC demons will actually ship useful things and break ahead. The talent will follow the market.
Google might very well be hamstrung forever by this and other internal politics. This dynamic has played out over and over again.
I mean, it's kind of both? Making Nazis look diverse isn't just a political error, it's also a technical one. By default, showing Nazis should show them as they actually were.
I don't think that's a fair comparison because they're fulfilling substantially different niches. Gemini is a conversational model that can generate images, but is mainly designed for text. Stable Diffusion is only for images. If you compare a model that can do many things and a model that can only do images by how well they generate images, of course the image generation model looks better.
Stability does have an LLM, but it's not provided in a unified framework like Gemini is.
I was referring to 'safety'. how the hell can an image generation model be dangerous? we had software for editing text, images, videos and audio for half a century now.
The obsession with safety in this announcement feels like a missed marketing opportunity, considering the recent Gemini debacle. Isn’t SD’s primary use case the fact that you can install it on your own computer and make what you want to make?
At some point they have to actually make money, and I don't see how continuously releasing the fruits of their expensive training for people to run locally on their own computer (or a competing cloud service) for free is going to get them there. They're not running a charity, the walls will have to go up eventually.
Likewise with Mistral, you don't get half a billion in funding and a two billion valuation on the assumption that you'll keep giving the product away for free forever.
But there are plenty of other business models available for open source projects.
I use Midjourney a lot and (based on the images in the article) it’s leaps and bounds beyond SD. Not sure why I would switch if they are both locked down.
SD would probably be a lot better if they didn't have to make sure it worked on consumer GPUs. Maybe this announcement is a step towards that where the best model will only be able to be accessed by most using a paid service.
Stable Diffusion has a much deeper learning curve but can generate far more accurate images fitting your perhaps special use case.
Although I don't understand the criticism of the images in question. Without a prompt comparison, it is impossible to compare image synthesis. What are examples of images that are beyond these?
I haven’t used SD so maybe the images on their home page here aren’t representative. But they look very generic and boring to me. They seem to lack “style” in a general aesthetic sense.
I am using Midjourney to basically create images in particular artistic styles (e.g., “painting of coffee cup in ukiyo-e style”) and that works very well. I am interested in SD for creating images based on artwork that isn’t indexed by Midjourney, though, as some of the more obscure artists aren’t available.
Usually there are models adapted to a specific theme since generic models at some point hit barriers. To get an idea, you could look up examples on sites like civitai.com.
Of course such sites are heavily biased towards content that is popular, but you will also find quite specific models if you search for certain styles.
Ironically their over sensitive nsfw image detector in their api caused me to stop using it and run it locally instead. I was using it to render animations of hundreds of frames but when every 20th to 30th image comes out blurry it ruins the whole animation and it would double the cost or more to rerender it with a different seed hoping to not trigger the over zealous blurring.
I don’t mind that they don’t want to let you generate nsfw images but their detector is hopelessly broken, it once censored a cube, yes a cube...
Unfortunately I don't want to pay for hundreds if not thousands of images I have to throw away because it decided some random innocent element is offensive and blurs the entire image.
What they are achieving with the over zealous safety issues are driving developers to on demand GPU hosts that will let them host their own models, which also opens up a lot more freedom. I wanted to use the stability AI api as my main source for Stable Diffusion but they make it really really hard especially if you want use it as part of your business.
I agree that given the status quo, it's a no-brainer to host your own model rather than use their SaaS – and likely one of the main reasons SAI doesn't seem to be on a very stable (heh) footing financially. To put it mildly.
Could you list the concrete "safety checks" that you think prevents real-world harm? What particular image that you think a random human will ask the AI to generate, which then leads to concrete harm in the real world?
If 1 in 1,000 generations will randomly produce memorized CSAM that slipped into the training set then yeah, it's pretty damn unsafe to use. Producing memorized images has precedent[0].
Okay, by "safety checks" you meant the already unlawful things like CSAM, but not politically-overloaded beliefs like "diversity"? The latter is what the comment[1] you were replying to was referring to (viz. "considering the recent Gemini debacle"[2]).
Right, by "rather have this [nothing]" I meant Stable Diffusion doing some basic safety checking, not Google's obviously flawed ideas of safety. I should have made that clear.
I posed the worst-case scenario of generating actual CSAM in response to your question, "What particular image that you think a random human will ask the AI to generate, which then leads to concrete harm in the real world?"
They try to, but it is difficult to comb through billions of images, and at least some of SD's earlier datasets were later found to have been contaminated with CSAM[0].
Do you have an example? I've never heard of anyone accidentally generating CSAM, with any model. "1 in 1,000" is just an obviously bogus probability, there must have been billions of images generated using hundreds of different models.
Besides, and this is a serious question, what's the harm of a model accidentally generating CSAM? If you weren't intending to generate these images then you would just discard the output, no harm done.
Nobody is forcing you to use a model that might accidentally offend you with its output. You can try "aligning" it, but you'll just end up with Google Gemini style "Sorry I can't generate pictures of white people".
Earlier datasets used by SD were likely contaminated with CSAM[0]. It was unlikely to have been significant enough to result in memorized images, but checking the safety of models increases that confidence.
And yeah I think we should care, for a lot of reasons, but a big one is just trying to stay well within the law.
Then you know almost nothing about the SD 1.5 ecosystem apparently. I've finetuned multiple models myself and it's nearly impossible to get rid of the child-bias in anime-derived models (which applies to 90 % of character focussed models) including nsfw ones. Took me like 30 attempts to get somewhere reasonable and it's still noticeable.
If we're being honest, anime and anything "anime-derived" is uncomfortably close to CSAM as a source material, before you even get SD involved, so I'm not surprised.
What I had in mind were regular general purpose models which I've played around with quite extensively.
The harm is that any use of the model becomes illegal in most countries (or offends credit card processors) if it easily generates porn. Especially if it does it when you didn't ask for it.
This question narrows the scope of "safety" to something less than what the people at SD or even probably what OP cares about. _Non-random_ CSAM requests targeting potentially real people is the obvious answer here, but even non-CSAM sexual content is also a probably a threat. I can understand frustration with it currently going overboard on blurring, but removing safety checks altogether would result in SD mainly being associated with porn pretty quickly, which I'm sure Stability AI wants to avoid for the safety of their company.
Add to that, parents who want to avoid having their kids generate sexual content would now need to prevent their kids from using this tool because it can create it randomly, limiting SD usage to kids 18+ (which is probably something else Stability AI does not want to deal with.)
It's definitely a balance between going overboard and having restrictions though. I haven't used SD in several months now so I'm not sure where that balance is right now.
> non-CSAM sexual content is also a probably a threat
To whom? SD's reputation, perhaps - but that ship has already sailed with 1.x. That aside, why is generated porn threatening? If anything, anti-porn crusaders ought to rejoice, given that it doesn't involve actual humans performing all those acts.
As I said, it means parents who don't want their young children seeing porn (whether you agree with them or not) would no longer be able to let their children use SD. I'm not making a statement on what our society should or shouldn't allow, I'm pointing out what _is currently_ the standard in the United States and many other, more socially conservative, countries. SD would become more heavily regulated, an 18+ tool in the US, and potentially banned in other countries.
You can have your own opinion on it, but surely you can see the issue here?
I can definitely see an argument for a "safe" model being available for this scenario. I don't see why all models SD releases should be so neutered, however.
How many of those parents would have the technical know-how to stop their lids from playing with SD? Give the model some “I am over 18” checkbox fig leaf and let them have their fun.
I've noticed that SDXL does something a little odd. For a given prompt it essentially decides what race the subject should be without the prompt having specified one. You generate 20 images with 20 different seeds but the same prompt and they're typically all the same race. In some cases they even appear to be the same "person" even though I doubt it's a real person (at least not anyone I could recognize as a known public figure any of the times it did this). I'm kind of curious what they changed from SD 1.5, which didn't do this.
And safe doesn't mean "lower than 1/10^6 chance of ending humanity", safe means shoddily implemented curtailing to idpol + fundamentalist level moral aversion towards human sexuality
It's not really their feelings, it's about controversy, bad publicity, etc. It's too delicate right now to risk people using their models for sex stuff.
I don't believe corporations implementing liberal politics to prevent backlash and it being legislated onto them qualifies as them being on the "left political spectrum".
Some backlash is perfectly ignorable. The twitter mob will move onto the next thing in a few days. And the proliferation of turned-to-the-max DEI employee policies, inclusion committees and self-censored newspeak does come from the californic technobubble cesspit.
There is such a great liberal fear of being perceived as any of the negative -ists and -isms that the pendulum swings to the other extreme where the left horseshoe toe meets its rightmost brother, which is why SD and Google's new toy rewrite ancient European history to include POC's and queer people.
People in this discussion seem to be hand-wringing about Stability's "saftey" comments but every model they've released has been fine tuned for porn in like 24 hours.
SD 2 definitely seems like an anomaly that they've learned from though and was hard for everyone to use for various reasons. SDXL and even Cascade (the new side-project model) seems to be embraced by horny people.
That's why we need open AI which scoops up all the data with its specific contexts and history and transforms it into a vast incomprehensible machine for us peons to gawk at while we starve and boil to death
That's not safety, the safety RLHF is because it tries to generate porn and people with three legs if you don't stop it.
It has the weird art style because that's what looks the most "aesthetic". And because it doesn't actually have nearly as good enough data as you'd think it does.
I'd want a model that can draw website designs and other UIs well. So I give it a list of things in the UI, and I get back a bunch of UI design examples with those elements.
I'm gonna hazard a guess and say well within the capabilities of a fine tuned model, but that no such fine tuned model exists and the labeled data required to generate it is not really there.
Photographs, digital illustrations, comic or cartoon style images, whatever graphical style you can imagine are all easy to achieve with current models (though no single model is a master of all trades). Things that look like technical drawings are as well, but don't expect them to make any sense engineering-wise unless maybe if you train a finetune specifically for that purpose.
I really wonder what harm would come to the company if they didn't talk about safety?
Would investors stop giving them money? Would users sue that they now had PTSD after looking at all the 'unsafe' outputs? Would regulators step in and make laws banning this 'unsafe' AI?
What is it specifically that company management is worried about?
All of the above! Additionally... I think AI companies are trying to steer the conversation about safety so that when regulations do come in (and they will) that the legal culpability is with the user of the model, not the trainer of it. The business model doesn't work if you're liable for harm caused by your training process - especially if the harm is already covered by existing laws.
One example of that would be if your model was being used to spot criminals in video footage and it turns out that the bias of the model picks one socioeconomic group over another. Most western nations have laws protecting the public against that kind of abuse (albeit they're not applied fairly) and the fines are pretty steep.
They're attempting to guard themselves against incoming regulation. The big players, such as Microsoft, want to squash Stable Diffusion while protecting themselves, and they're going to do it by wielding the "safety is important and only we have the resources to implement it" hammer.
Safety is a very real concern, always has been in ML research. I'm tired of this trite "they want a moat" narrative.
I'm glad tech orgs are for once thinking about what they're building before putting out society-warping democracy-corroding technology instead of move fast break things.
It doesn't strike you as hypocritical that they all talk about safety while continuing to push out tech that's upending multiple industries as we speak? It's tough for me to see it as anything other than lip service.
I'd be on your side if any of them actually chose to keep their technology in the lab instead of tossing it out into the world and gobbling up investment dollars as fast as they could.
How are these two things related at all? When AI companies speak of safety, it's almost always about the "only including data a religious pastor would find safe, and filtering outputs" angle. How's the market and other industries relevant at all? Should AI companies be obligated to care about what happens to other companies? With that point of view, we should've criticized the iPhone for upending the PDA market, or Wacom for "upending" the traditional art market.
That would make sense if it was in the slightest about avoiding "society-warping democracy-corroding technology". Rather than making sure no one ever sees a naked person which would cause governments to come down on them like a ton of bricks.
This isn't a valid concern in my opinion. Photo manipulation has been around for decades. People have been drawing other people for centuries.
Also, where do we draw the line? Should Photoshop stop you from manipulating human body because it could be used for porn? Why stop there, should text editors stop you from writing about sex or describing human body because it could be used for "abuse". Should your comment be removed because it make me imagine Taylor Swift without clothes for a brief moment?
No, but AI requires zero learning curve and can be automated. I can't spit out 10 images of Tay per second in photoshop. If I want and the API delivers I can easily do that with AI. (Given, would one becoding this it requires a learning curve, but in principal with the right interface and they exist i can churn out hundreds of images without me actively putting work in)
I've never understood the argument about image generators being (relatively) fast. Does that mean that if you could Photoshop 10 images per second, we should've started clamping down on Photoshop? What exact speed is the cutoff mark here? Given that Photoshop is updated every year and includes more and more tools that can accelerate your workflow (incl. AI-assisted ones), is there going be a point when it gets too fast?
I don't know much about the initial scandal, but I was under the impression that there was only a small number of those images, yet that didn't change the situation. I just fail to see how quantity factors into anything here.
>I just fail to see how quantity factors into anything here.
Because you can overload any online discussion / sphere with that. There were so many that X effectively banned searching for her at all because if you did, you where overwhelmed by very extreme fake porn. Everybody can do it with very low entry barrier, it looks very believable, and it can be generated in high quantities.
We shouldn't have clamped down on photoshop, but realisticly two things would be nice in your theoretical case, usage restrictions and public information building. There was no clear cut point where photoshop was so mighty you couldn't trust any picture online. There were skills to be learned and people could identify the trickery, and it was on a very small scale and gradual. And the photo trickery was around for ages, even Stalin did it.
But creating photorealistic fakes in an automated fashion is completely new.
But when we talk about specifically harming one person, does it really matter if it's a thousand different generations of the same thing or 10 generations that were copied thousands of times? It is a technology that lowers the bar for generating believable-looking things, but I don't know if it's the speed that is the main culprit here.
And in fairness to generative AI, even nowadays it feels like getting to a point of true photorealism takes some effort, especially if the goal is letting it just run nonstop with no further curation. And getting a local image generator to run at all on your computer (and having the hardware for it) is also a bar that plenty of people can't clear yet. Photoshop is kind of different in that making more believable things requires a lot more time, effort and knowledge - but the idea that any image online can be faked has already been ingrained in the public consciousness for a very long time.
Yes, if you could Photoshop 10/sec it would be a problem.
Think of it this way, if one out of every ten phone calls you get is spam, you still have a pretty useable phone. Three orders of magnitude different and 1 out of every 100 calls is real and the system totally breaks down.
Generative AI makes generating realistic looking fakes ~1000x easier, its the one thing its best at.
but that's not dangerous. It's definitely worthy of unlocking the cages of the attack lawyers but it's not dangerous. The word "safety" is being used by big tech to trigger and gas light society.
To the extent these models don't blindly regurgitate hate speech, I appreciate that. But what I do not appreciate is when they won't render a human nipple or other human anatomy. That's not safety, and calling it such is gaslighting.
As the leader in open image models it is incumbent upon us as the models get to this level of quality to take seriously how we can release open and safe models from a legal, societal and other considerations.
Not engaging in this will indeed lead to bad laws, sanctions and more as well as not fulfilling our societal obligations of ensuring this amazing technology is used for as positive outcomes as possible.
Stability AI was set up to build benchmark open models of all types in a proper way, this is why for example we are one of the only companies to offer opt out of datasets (stable cascade and SD3 are opted out), have given millions of supercompute hours in grants to safety related research and more.
Smaller players with less uptake and scrutiny don't need to worry so much about some of these complex issues, it is quite a lot to keep on top of, doing our best.
>it is incumbent upon us as the models get to this level of quality to take seriously how we can release open and safe models from a legal, societal and other considerations.
Can you define what you mean by "societal and other considerations"? If not, why not?
> What is it specifically that company management is worried about?
As with all hype techs, even the most talented management are barely literate in the product. When talking about their new trillion $ product they must take their talking points from the established literature and "fake it till they make it".
If the other big players say "billions of parameters" you chuck in as many as you can. If the buzz words are "tokens" you say we have lots of tokens. If the buzz words are "safety" you say we are super safe. You say them all and hope against hope that nobody asks a simple question you are not equipped to answer that will show you dont actually know what you are talking about.
they risk reputational harm and since there's so many alternatives outright "brand cancellation". For example, vocal groups can lobby payment processors to deny service to any AI provider deemed unworthy. Ironic that tech enabled all of that behavior to begin with and now they're worried about it turning on them.
What viable alternatives are there to Stable Diffusion? As far as I know, it's the only way to run good image generation locally, and that's probably a big consideration for any business dabbling in it.
Yeah, the word "good" is doing the heavy lifting here - while it's not the only one that can do it, it has a very comfortable lead over all alternatives.
It's a bit rich when HN itself is chock full with camp followers who pick the most mainstream opinion. Previously it was AI danger, then it became hallucinations, now it's that safety is too much.
The rest of the world is also like that. You can make a thing that hurts your existing business. Spinning off the brand is probably Google's best bet.
Likely public condemnation followed by unreasonable regulations when populists see their campaign opportunities. We've historically seen this when new types of media (e.g. TV, computer games) debut and there are real, early signals of such actions.
I don't think those companies being cautious is necessarily a bad thing even for AI enthusiasts. Open source models will quickly catch up without any censorship while most of those public attacks are concentrated into those high profile companies, which have established some defenses. That would be a much cheaper price than living with some unreasonable degree of regulations over decades, driven by populist politicians.
At this point, the next thing that will blow me away is AGI at human expert level or a Gaussian Splat diffusion model that can build any arbitrary 3D scene from text or a single image. High bar, but the technology world is already full of dark magic.
- This uses a new type of diffusion transformer (similar to Sora) combined with flow matching and other improvements.
- This takes advantage of transformer improvements & can not only scale further but accept multimodal inputs..
- Will be released open, the preview is to improve its quality & safety just like og stable diffusion
- It will launch with full ecosystem of tools
- It's a new base taking advantage of latest hardware & comes in all sizes
- Enables video, 3D & more..
- Need moar GPUs..
- More technical details soon
>Can we create videos similar like sora
Given enough GPUs and good data yes.
>How does it perform on 3090, 4090 or less? Are us mere mortals gonna be able to have fun with it ?
Its in sizes from 800m to 8b parameters now, will be all sizes for all sorts of edge to giant GPU deployment.
(adding some later replies)
>awesome. I assume these aren't heavily cherry picked seeds?
No this is all one generation. With DPO, refinement, further improvement should get better.
>Do you have any solves coming for driving coherency and consistency across image generations? For example, putting the same dog in another scene?
yeah see
@Scenario_gg's great work with IP adapters for example. Our team builds ComfyUI so you can expect some really great stuff around this...
>Dall-e often doesn’t even understand negation, let alone complex spatial relations in combination with color assignments to objects.
Imagine the new version will. DALLE and MJ are also pipelines, you can pretty much do anything accurately with pipelines now.
>Nice. Is it an open-source / open-parameters / open-data model?
Like prior SD models it will be open source/parameters after the feedback and improvement phase. We are open data for our LMs but not other modalities.
>Cool!!! What do you mean by good data? Can it directly output videos?
If we trained it on video yes, it is very much like the arch of sora.
Do you know how the memory demands compare to LLMs at the same number of parameters? For example, Mistral 7B quantized to 4 bits works very well on an 8GB card, though there isn’t room for long context.
I am going to look at quantization for 8b. But also, these are transformers, so variety of merging / Frankenstein-tune is possible. For example, you can use 8b model to populate the KV cache (which computes once, so can load from slower devices, such as RAM / SSD) and use 800M model for diffusion by replicating weights to match layers of the 8b model.
Stability has to make money somehow. By releasing an 8B parameter model, they’re encouraging people to use their paid API for inference. It’s not a terrible business decision. And hobbyists can play with the smaller models, which with some refining will probably be just fine for most non-professional use cases.
Oh they’ll never let you pay for porn generation. But they will happily entertain having you pay for quality commercial images that are basically a replacement for the entire graphic design industry.
Don't people quantize SD down to 8 bits? I understand plenty of people don't have 8GB of VRAM (and I suppose you need some extra for supplemental data, so maybe 10GB?). But that's still well within the realm of consumer hardware capabilities.
We have highly efficient models for inference and a quantization team.
Need moar GPUs to do a video version of this model similar to Sora now they have proved that Diffusion Transformers can scale with latent patches (see stablevideo.com and our work on that model, currently best open video model).
We have 1/100th of the resources of OpenAI and 1/1000th of Google etc.
Google got cheap TPU chips, means they circumvent the extremely expensive Nvidia corporate licenses. I can easily see them having 10x the resources of OpenAI for this.
Yes, they have deep pockets and could increase investment if needed. But the actual resources devoted today are public, and in line with the parent said.
can someone explain why nVidia doesn't just hold their own AI? And literally devote 50% of their production to their own compute center? In an age where even ancient companies like Cisco are getting in the AI race, why wouldn't the people with the keys to the kingdom get involved?
1. the real keys to the kingdom are held by TSMC whose fab capacity rules the advanced chips we all get, from NVIDIA to Apple to AMD to even Intel these days.
2. the old advice is to sell shovels during a gold rush
Because history has shown that the money is in selling the picks and shovels, not operating the mine. (At least for now. There very well may come a point later on when operating the mine makes more sense, but not until it's clear where the most profitable spot will be)
Don’t stretch that analogy too far. It was applicable to gold rushes, which were low hanging fruit where any idiot could dig a hole and find gold.
Historically, once the easy to find gold was all gone it was the people who owned the deep gold mines and had the capital to exploit them who became wealthy.
They've been very happy selling shovels at a steep margin to literally endless customers.
The reason is because they instantly get a risk free guaranteed VERY healthy margin on every card they sell, and there's endless customers lined up for them.
If they kept the cards, they give up the opportunity to make those margins, and instead take the risk that they'll develop a money generating service (that makes more money then selling the cards).
This way there's no risk of: A competitor out competing them, not successfully developing a profitable product, "the ai bubble popping", stagnating development, etc.
There's also the advantage that this capital has allowed them to buy up most of TSMC's production capacity, which limits the competitors like Google's TPUs.
> Why is there not a greater focus on quantization to optimize model performance, given the evident need for more GPU resources?
There is an inherent trade off between model size and quality. Quantization reduces model size at the expense of quality. Sometimes it's a better way to do that than reducing the number of parameters, but it's still fundamentally the same trade off. You can't make the highest quality model use the smallest amount of memory. It's information theory, not sorcery.
Yes Quantization compresses float32 values to int8 by mapping the large range of floats to a smaller integer range using a scale factor. This scale factor is key for converting back to floats (dequantization), aiming to preserve as much information as possible within the int8 limits. While quantization reduces model size and speeds up computation, it trades off some accuracy due to the compression. It's a balance between efficiency and model quality, not a magic solution to shrink models without losing some performance.
Quantization is essential for me since a 7B model won't fit on my RTX 2060 with only 6GB of VRAM. It allows me to compress the model so it can run on my hardware.
Soon the GPU and its associated memory will be on different cards, as once happened with CPUs. The day of the GPU with ram slots is fast approaching. We will soon plug terabytes of ram into our 4090s, then plug a half-dozen 4090s into a raspberry PI to create a Cronenberg rendering monster. Can it generate movies faster than Pixar can write them? Sure. Can it play Factorio? Heck no.
Any seperation of a GPU from its VRAM is going to come at the expense of (a lot of) bandwidth. VRAM is only as fast as it is because the memory chips are as close as possible to the GPU, either on seperate packages immediately next to the GPU package or integrated onto the same package as the GPU itself in the fanciest stuff.
If you don't care about bandwidth you can already have a GPU access terabytes of memory across the PCIe bus, but it's too slow to be useful for basically anything. Best case you're getting 64GB/sec over PCIe 5.0 x16, when VRAM is reaching 3.3TB/sec on the highest end hardware and even mid-range consumer cards are doing >500GB/sec.
Things are headed the other way if anything, Apple and Intel are integrating RAM onto the CPU package for better performance than is possible with socketed RAM.
Is there a way to partition the data so that a given GPU had access to all the data it needs but the job itself was parallelized over multiple GPUs?
Thinking on the classic neural network for example, each column of nodes would only need to talk to the next column. You could group several columns per GPU and then each would process its own set of nodes. While an individual job would be slower, you could run multiple tasks in parallel, processing new inputs after each set of nodes is finished.
That depends on whether performance or capacity is the goal. Smaller amounts of ram closer to the processing unit makes for faster computation, but AI also presents a capacity issue. If the workload needs the space, having a boatload of less-fast ram is still preferable to offloading data to something more stable like flash. That is where bulk memory modules connected though slots may one day appear on GPUs.
I don’t think you really understand the current trends in computer architecture. Even cpus are being moved to have on package ram for higher bandwidth. Everything is the opposite of what you said.
Higher bandwidth but lower capacity. The real trend is different physical architectures for different compute loads. There is a place in AI for bulk albeit slower memory such as extremely large date sets that want to run internally on a discreet card without involving pci lanes.
This is also not true. You can transfer from main memory to cards plenty fast enough that it is not a bottleneck. Consumer GPU's don't even use pcie5 yet, which doubles the bandwidth of 4. Professional datacenter cards don't use pcie AT ALL, but they do put a huge amount of RAM on the package with the GPUs.
I doubt it. The latest GPUs utilize HBM which is necessarily part of the same package as the main die. If you had a RAM slot for a GPU you might as well just go out to system RAM, way too much latency to be useful.
It isn't the latency which is the problem, it's the bandwidth. A memory socket with that much bandwidth would need a lot of pins. In principle you could just have more memory slots where each slot has its own channel. 16 channels of DDR5-8000 would have more bandwidth than the RTX 4090. But an ordinary desktop board with 16 memory channels is probably not happening. You could plausibly see that on servers however.
What's more likely is hybrid systems. Your basic desktop CPU gets e.g. 8GB of HBM, but then also has 16GB of DRAM in slots. Another CPU/APU model that fits into the same socket has 32GB of HBM (and so costs more), which you could then combine with 128GB of DRAM. Or none, by leaving the slots empty, if you want entirely HBM. A server or HEDT CPU might have 256GB of HBM and support 4TB of DRAM.
No it won't. GPUs are good at ml partly because of the huge memory bandwidth. 1000s of bits wide. You won't find connectors that have that many terminals and maintain signal quality. Even putting a second bank soldered on the same signals can be enough to mess things up.
Nvidia is making way too much money keeping cards with lots of memory exclusive to server GPUs they sell with insanely high margins.
AMD still suffers from limited resources and doesn't seem willing to spend too much chasing a market that might just be a temporary hype, Google's TPUs are a pain to use and seem to have stalled out, and Intel lacks commitment, and even their products that went roughly in that direction aren't a great match for neural networks because of their philosophy of having fewer more complex cores.
I've never tried it, but in Windows you can have CUDA apps fall back to system ram when GPU vram is exhausted. You could slap 128gb in your rig with a 4070. I'm sure performance falls off a cliff, but if it's the difference between possible and impossible that might be acceptable.
Unfortunately production capacity for that is limited, and with sufficient demand, all pricing is an auction. Therefore, we aren't going to be seeing that card in years
Please give me some DIMM slots on the GPU so that I can choose my own memory like I'm used to from the CPU-world and which I can re-use when I upgrade my GPU.
An M1 Mac Studio with that much RAM can be had for around $3K if you look for good deals, and will give you ~8 tok/s on a 70B model, or ~5 tok/s for a 120B one.
I kind of wonder if gaming will start incorporating AI stuff. What if instead of generating a stable diffusion image, you could generate levels and monsters
Last I saw they performed really poorly, like lower single digits t/s. Don't get me wrong they're probably a decent value for experimenting with it, but is flat out pathetic compared to an A100 or H100. And I think useless for training?
You can run a 180B model like Falcon Q4 around 4-5tk/s, a 120B model like Goliath Q4 at around 6-10tk/s, and 70B Q4 around 8-12tk/s and smaller models much quicker, but it really depends on the context size, model architecture and other settings. A A100 or H100 is obviously going to be a lot faster but it costs significantly more taking its supporting requirements into account and can’t be run on a light, battery powered laptop etc…
MPS is promising and the memory bandwidth is definitely there, but stable diffusion performance on Apple Silicon remains terribly poor compared with consumer Nvidia cards (in my humble opinion). Perhaps this is partly because so many bits of the SD ecosystem are tied to Nvidia primitives.
Image diffusion models tend to have relatively low memory requirements compared to LLMs (and don’t benefit from batching), so having access to 128 GB of unified memory is kinda pointless.
GPU memory is all about bandwidth, not latency. DDR5 can do 4-8 GT/s x 64-bit bus per DIMM, so maxing 128 GB/s with a dual memory controller, 512 GB/s with 8x memory controllers on server chips, but GDDR6 can run at twice the frequency and has a memory bus ~5x as wide in the 4090, so you get an order of magnitude bump in throughput, so nearly 1 TB/s on a consumer product. Datacenter GPUs (e.g. A100) with HBM2e doubles that to 2 TB/s
I understand that Sora is very popular, so it makes sense to refer to it, but when saying it is similar to Sora, I guess it actually makes more sense to say that it uses a Diffusion Transformer (DiT) (https://arxiv.org/abs/2212.09748) like Sora. We don't really know more details on Sora, while the original DiT has all the details.
Is anyone else struck by the similarities in textures between the images in the appendix of the above "Scalable Diffusion Models with Transformers" paper?
If you size the browser window right, paging with the arrow keys (so the document doesn't scroll) you'll see (eg, pages 20-21) the textures of the parrot's feathers are almost identical to the textures of bark on the tree behind the panda bear, or the forest behind the red panda is very similar to the undersea environment.
Even if I'm misunderstanding something fundamental here about this technique, I still find this interesting!
So is this "SDXL safe" or "SD2.1" safe, cause SDXL safe we can deal with, if it's 2.1 safe it's gonna end up DOA for a large part of the opensource community again
Don't know about 3.0, but Cascade has different level of safety between the full model and the light model. Full model is far more prudish, but both completely fail with some prompts.
>>>How does it perform on 3090, 4090 or less? Are us mere mortals gonna be able to have fun with it ?
>>>Its in sizes from 800m to 8b parameters now, will be all sizes for all sorts of edge to giant GPU deployment.
--
Can you fragment responses such that if an edge device (mobile app) is prompted for [thing] it can pass tokens upstream on the prompt -- Torrenting responses effectively - and you could push actual GPU edge devices in certain climates... like dens cities whom are expected to be a Fton of GPU cycle consumption around the edge?
So you have tiered processing (speed is done locally, quality level 1 can take some edge gpu - and corporate shit can be handled in cloud...
----
Can you fragment and torrent a response?
If so, how is that request torn up and routed to appropriate resources?
BOFH me if this is a stupid question? (but its valid for how we are evolving to AI being intrinsic to our society so quickly.)
It’s not obvious that fine-tuning can remove all latent compulsions from these models. Consider that the creators know that fine-tuning exists and have vastly more resources to explore the feasibility of removing deep bias using this method.
1. A mass-distributed LLM (hosted by google or openai or whoever) that's been neutered and twisted into political correctness in a haphazard series of kneejerk meetings of small groups of people who are terrified big investors will walk away or that some powerful political group will denounce them. Effectively they create an enormous bias of falsity and incorrectness, for billons of people to use and embed the results throughout all their intellectual output.
2. Some wacko with an expensive Nvidia GPU makes deepfake porn of a popular politician. Or goodness forbid, of some weird kink where if this was actually a scene filmed with real people, there would be serious ethical issues.
Which scenario do you think is more dangerous, long term, and in terms of broad impact on society in general?
I think you are seeing things from a bubble. Most people in the country are in favor of efforts to correct for historical injustices and are worried about AIs repeating biases in their training that could have a material impact on the world.
Case in point: large advertisers and entertainment companies "pander" to these sorts of views, because it is broadly popular
Most people in the country dont give a shit about historical injustices and are worried about how they are going to pay their rent or put food on the table today.
If you go by online, popularity, yes, a lot of people do want to erase history in favor of feelings. That can be your opinion too.
> Broadly popular
Are they? Or are the loudest voices asymmetrically affecting discourse?
> This is not the work of a shadowy cabal
And how would you know if it was? What would the clues be? If you were in a bubble that was designed to impart an informal religious view (and it is a religion sometimes, just one with a screen instead of a book) that encompass politics and morality, how would you know?
On the contrary, the argument that "someone will do it anyway, so you should just let it happen and take no moral responsibility because I WANT MY SHINY TOY" is… not merely silly, but incredibly absurd, selfish, entitled, and amoral.
One could argue that if $BIGCORP doesn't want their thing to be used by bad actors, they should just refrain from developing the technology at all, and while it's a somewhat defensible position, that would also result in techbros not getting their toy, so it doesn't really apply here.
I just cannot comprehend how some people cannot see the incredibly obvious moral responsibility in releasing something that could be used for a lot of good but also to do bad things. There's no reasonable moral theory in which you could just shrug and go, "well, somebody is going to do it anyway so why should we even try to keep our conscience clean and avoid making it easy for them", it's fundamentally amoral and antisocial.
If someone invents a lockpick capable of opening any door, they have a moral responsibility in preventing it from falling into wrong hands, whether they want it or not. And it's absurd to complain when someone who could create an universal lockpick, refuses to do so, never mind release the technology to the wild, and only agrees to sell simpler picks capable of picking simpler locks. Do these people also complain about things like work against nuclear proliferation? After all, North Korea got nukes anyway, so what's the point?
Your lockpick example only works if there's e.g. only one and it's feasible to keep it hidden from the world.
Software doesn't work that way. I agree that you should be responsible about dangerous tech, but you also have to be realistic about what the best way to do that is, which is pretty much never "keep it hidden."
(and, of course, this is not even considering the question that should probably go here which is -- how dangerous is this exactly? Given the moral panic we saw a while ago about e.g. Photoshop, I'm not entirely convinced that this is much to worry about.)
The willful ignorance in these threads is maddening. They know why models have these restrictions, they just think the rules shouldn't apply to them and want to play the victim. It's the same libertarian attitude as people who whine about driving speed laws.
If there's one lesson from the 21st century, it's that's you shouldn't release massively impacting technology without strong ethics controls around it.
I mean, SDXL is great. Until you’ve had a chance to actually use this model, isn’t calling it out for some imagined offence that may or may not exist seems like you’re drinking some Kool-aid rather than responding to something based in concrete actual reality.
You get access to it… and it does the google thing and puts people of colour in every frame? Sure, complain away.
You get access to it, you can’t even generate pictures of girls? Sure. Burn the house down.
…you haven’t even seen it and you’re already bitching about it?
Come on… give them a chance. Judge what it is when you see it not what you imagine it is before you’ve even had a chance to try it out…
Lots of models, free, multiple sizes, hot damn. This is cool stuff. Be a bit grateful for the work they’re doing.
…and even if sucks, it’s open. If it’s not what you want, you can retune it.
A blogger I follow had an article explaining that the NSFW models for SDXL, are just now SORT OF coming up to the quality of SD1.5 “pre safety” models.
It’s been 6 months and it still isn’t there. SD3 is going to be quite awhile if they’re baking “safety” in even harder.
1.5 is still more popular than xl and 2 for reasons unrelated to safety. The size and generation speed matter a lot. This is just a matter of practical usability, not some idea of the model being locked down. Feed it enough porn and you'll get porn out of it. If people have incentive to do that (better results than 1.5), it really will happen within days.
I wish I had something more clever to comment on it. I know what they’re doing which is cool and why which is, IDK, live and let live and enjoy your own kink. It just a little funny some of the most work put into in the fine tuning models.. is from the pony community.
It's not just them. For example, 4chan is a surprisingly good way to get the most recent scoop on good models (both text and images), setup guides etc - if you can tolerate the inevitable, well, 4chan-ness of it. And the reason is exactly the same: a lot of people there really, really, really, want to generate porn and to chat with sexbots, and they're putting a lot of effort into getting the best (and least censored) results out of the resources that they have.
Apparently most stuff that relies on the weights being resonably similar to SDXL don't work - control nets, LORAs, commonly-used inpainting patches, the lot. It seems to go well beyond fine tuning and be substantially retrained to the point it's a good chunk of the way to being a different model entirely, and the amount of training time is on the order of what went into SDXL originally too from what I can tell.
I notice they are avoiding images of people in the announcement.
I wonder if they are afraid of the same debacle as google AI and what they mean by "safety" is actually heavy bias against white people and their culture like what happened with Gemini.
From the examples I see on Twitter, they are usually referring to the different cultures of Irish, European, and American white people. Gemini, in an effort to reverse the bias that the models would naturally have, ends up replacing these people with those from other cultures.
Since the definition of "white" is inherently cultural, it varies from place to place and from time to time. Today, in US and Europe, pretty much everyone who cares about racial categorization would consider Irish "white". Historically, it was different, but that is only relevant when discussing history.
Isn't it even more racist to replace them in a picture? Being told that your skin colour is too offensive to show sounds a lot worse to me than calling them "white" considering their skin is very white
US American white people. Anything else would be a ridiculous overgeneralization, like "Asian culture", even if you set some arbitary benchmark for teint and only look at those European countries it's still too much diversity to pool together.
As if you can generalize the culture of different European countries, or even different regions in the same country just by skin color. Now this, in my opinion, is a form of cultural erasure where all the intricacies and interesting aspects of culture are put aside and overshadowed by skin color.
I wouldn't look for hidden reasons. Recent image generators are already too good with face generation (thanks to CelebA-like datasets and early researchers).
And now the emphasis is on the multimodality of the model within a domain. There, almost every picture demonstrates some aspect of it. Somewhere there is text on the picture (old AI used to output bullshit instead of letters), somewhere there are humorous references to old images (for example, a cosmonaut on a pig).
719 comments
[ 6.7 ms ] story [ 372 ms ] threadAlso, I was blown away by the "Stable Diffusion" written on the side of the bus.
The old one can't.
That's good enough to describe compositions well-represented in the training set, but it will be likely to lock-in to those common representations at the expense of rarer but still possible ones (the 'woman chasing a bear' above).
"a woman chasing a bear, pursuit"
https://i.imgur.com/RqCXVYC.png
What exactly does this mean? Will we be able to see all of the "safeguards" and access all of the technology's power without someone else's restrictions on them?
Think of childern! We must stop people from generating porn!
midjourney 6 can be completely photorealistic and include valid text, but also sometimes adds bad text. it's not much, but having to use an image editor for that is still annoying. for creating marketing material, getting perfect text every time and never getting bad text would be amazing
It's "safe" for them, not for the users, at least they should make that clear.
Yes, they are protecting themselves from lawsuits, but they are also protecting other people. Preventing people asking for specific celebrities (or children) having sex is for their benefit too.
But giving that ability to _everyone_ will lead to a huge increase in undesirable and targeted/local behaviour.
Presumably it enables any creep to generate what they want by virtue of being able to imagine it and type it, rather than learn a niche skill set or employ someone to do it (who is then also complicit in the act)
Why don't you just say you believe thought crime should be punishable?
Or maybe not. It's hard to tell when nobody seems to want to spell out what behaviors we want to prevent.
Where are the Americans asking about Snapchat? If I were a developer at Scnapchat I could prolly open a few Blob Storage accounts and feed a darknet account big enough to live off of. You people are so manipulatable.
If yes, why doesn't the same law apply to AI? If no, why are we only concerned about it when AI is involved?
Second, the tool will become available to anyone, anywhere, not just a localised school. If generating naughty nudes is frowned upon in one place, another will have no qualms about it. And that's just things that are about decency, then there's the discussion about legality.
Finally, when person A draws a picture, they are responsible for it - they produced it. Not the party that made the pencil or the paper. But when AI is used to generate it, is all of the responsibility still with the person that entered the prompt? I'm sure the T's and C's say so, but there may still be lawsuits.
Feel free to make an AI model that does almost anything, though I'd probably suggest that it doesn't make porn of minors as that is criminal in most jurisdiction, short of that it's probably not a criminal offense.
Most companies are only very slightly worried about criminal offenses, they are far more concerned about civil trials. There is a far lower requirement for evidence. AI creator in email "Hmm, this could be dangerous". That's all you need to lose a civil trial.
Nah, I think it's a disagreement over whether a tool's maker gets blamed for evil use or the tool's user.
It's a similar argument over whether or not gun manufacturers should have any liability for their products being used for murder.
This is really only a debate in the US and only because it's directly written in the constitution. Pretty much no other product works that way.
Affordable drawing classes and YouTube drawing tutorials lower the barrier of entry as well.
Why on earth would manufacturers of pencils, papers, drawing classes, and drawing software feel responsible for censoring the result of combining their tool with the brain of their customer?
A sharp kitchen knife significantly lowers the barrier of entry to murder someone. Many murders are committed everyday using a kitchen knife. Should kitchen knife manufacturers blog about this every week?
I guess my point is that I don't think we're as inconsistent as a society as it seems when considering things like knives. It's not even strictly limited to thought crimes/information crimes. If alcohol were discovered today , I have no doubt that it would be banned and made schedule I
Fun fact: Many scanners and photocopiers will detect that you're trying to scan/copy a banknote and will refuse to complete the scan. One of the ways is detecting the EURion Constellation.
https://en.wikipedia.org/wiki/EURion_constellation
I'd guess that stability is concerned with their legal liability, also perhaps they are decent humans who don't want to make a product that is primarily used for harassment (whether they are decent humans or not, I imagine it would affect the bottom line eventually if they develop a really bad rep, or a bunch of politicians and rich people are targeted by deepfake harassment).
[1] https://www.cagoldberglaw.com/states-with-revenge-porn-laws/...
^ a lot of, but not all of those laws seem pretty specific to photographs/videos that were shared with the expectation of privacy and I'm not sure how they would apply to a painting/drawing, and I certainly don't know how the courts would handle deepfakes that are indistinguishable from genuine photographs. I imagine juries might tend to side with the harassed rather than a bully who says "it's not illegal cause it's actually a deepfake but yeah i obviously intended to harass the victim"
> The Ninth Circuit reversed, reasoning that the government could not prohibit speech merely because of its tendency to persuade its viewers to engage in illegal activity.[6] It ruled that the CPPA was substantially overbroad because it prohibited material that was neither obscene nor produced by exploiting real children, as Ferber prohibited.[6] The court declined to reconsider the case en banc.[7] The government asked the Supreme Court to review the case, and it agreed, noting that the Ninth Circuit's decision conflicted with the decisions of four other circuit courts of appeals. Ultimately, the Supreme Court agreed with the Ninth Circuit.
To spell out one such instance: I would like to live in a world where it is not trivial to depict and misrepresent me (or anyone) in a way that is photorealistic to the point that it can be used to mislead others.
Whether that means we need to outright prevent it, or have some kind of authenticity mechanism, or some other yet-to-be-discovered solution? I do not know, but you now have my goalposts.
In a large number of countries if you create an image that represents a minor in a sexual situation you will find yourself on the receiving side of the long arm of the law.
If you are the maker of an AI model that allows this, you will find yourself on the receiving side of the long arm of the law.
Moreso, many of these companies operate in countries where thought crime is illegal. Now, you can argue that said companies should not operate in those countries, but companies will follow money every time.
This is not obvious at all when it comes to AI models.
>People are such authoritarian shit-stains
Yes, but this is a different conversation altogether.
EDIT: Boring and shallow are, unfortunately, the Internet's fault. Don't know what to do about those.
(of course unless you are into yoga, then everything is permitted)
...or children's gymnastics.
Great talk about slavery and religious-persecution, Jim! Wait, what were we talking about? Fucking American fascists trying to control our thoughts and actions, right right.
No where is safe
Even if it did that by looking at the process memory (which would be seriously wrong in many ways), just the manipulation part would be mighty impressive. And if it was the case I guess we would see many papers and heated discussions about them.
Or it is just the usual conspiracy theory.
Sounds like you are perhaps just bad at using the generative art tools
"It's not the models they want to align, it's you."
Like with Stallman, if you listen about his political thoughts vs his software engineering thoughts.
And he is completely, totally incompetent on this - and by the way, he's also completely incompetent on AI, and on most of tech. See his stint at twitter...
It wasn‘t important, just something I saw in the moment and wanted to see what DallE makes of it.
Generation denied. No explanation given, I can only imagine that it triggered some detector of sexual request?
(It wasn‘t the phrase "pure white", as far as I can tell, because I have lots of generated pics of my cat in other contexts)
https://grr.com/publications/hey-thats-my-voice-can-i-sue-th...
Examples: Can you great an image of a cat in Tim Burton's style? Oops! Try another prompt Looks like there are some words that may be automatically blocked at this time. Sometimes even safe content can be blocked by mistake. Check our content policy to see how you can improve your prompt.
Can you create an image of a cat in Wes Anderson's style? Certainly! Wes Anderson’s distinctive style is characterized by meticulous attention to detail, symmetrical compositions, pastel color palettes, and whimsical storytelling. Let’s imagine a feline friend in the world of Wes Anderson...
DALL-E, for example, wrongly denied serveral request of mine.
It sounds like a reputation/ethics thing to me. You probably don’t want to be known as the company that freely released a model that gleefully provides images of dismembered bodies (or worse).
"Wrongly denied" in this case depends on your point of view, clearly DALL-E didn't want this combination of words created, but you have no right for creation of these prompts.
I'm the last one defending large monolithic corps, but if you go to one and want to be free to do whatever you want you are already starting from a very warped expectation.
I don't use them for porn like a lot of people seem too, but it seems weird to me that something that's kind of made to generate art can't generate one of the most common subjects in all of art history - nude humans.
I have seen a lot of “pasties” that look like Sorry! game pieces, coat buttons, and especially hell-forged cybernetic plumbuses. Did they train it at an alien strip club?
The LoRAs and VAEs work (see civit.ai), but do you really want something named NSFWonly in your pipeline just for nipples? Haha
These were examples that were not super blatant, like a tree landscape that just happens to have a human figure and cave in their crotch. Examples:
https://i.imgur.com/RlH4NNy.jpg - Art is very focused on the monster's crotch
https://i.imgur.com/0M8RZYN.jpg - The comparison should hopefully be obvious
https://www.theverge.com/2024/2/21/24079371/google-ai-gemini...
Sure, the safety people lost that battle for Stable diffusion and LLama. And because they lost, entire industries were created by startups that could now use models themselves, without it being locked behind someone else's AI.
But it wasn't guaranteed to go that way. Maybe the safetyists could have won.
I don't we'd be having our current AI revolution if facebook or SD weren't the first to release models, for anyone to use.
Maybe that will be part of future papers or the teased technical report. But I find it strange to put so much emphasis on safety and then leave it all up to the reader's imagination.
Models with a large user base will have an inverse relationship with usability. That's why it's important to have options to train your own with open source.
Excuse me?
> In this case there is no need to simplify anything
Then just ask the question itself.
https://arstechnica.com/information-technology/2024/02/deepf...
> explain like I have ~~Asperger (ELIA?)~~ limited understanding of how the world really works.
The AI is being limited so that it cannot produce any "offensive" content which could end up on the news or go viral and bring negative publicity to Stability AI.
Viral posts containing generated content that brings negative publicity to Stability AI are fine as long as they're not "offensive". For example, wrong number of fingers is fine.
There is not a comprehensive, definitive list of things that are "offensive". Many of them we are aware of - e.g. nudity, child porn, depictions of Muhammad. But for many things it cannot be known a priori whether the current zeitgeist will find it offensive or not (e.g. certain depictions of current political figures, like Trump).
Perhaps they will use AI to help decide what might be offensive if it does not explicitly appear on the blocklist. They will definitely keep updating the "AI Safety" to cover additional offensive edge cases.
It's important to note that "AI Safety", as defined above (cannot produce any "offensive" content which could end up on the news or go viral and bring negative publicity to Stability AI) is not just about facially offensive content, but also about offensive uses for milquetoast content. Stability AI won't want news articles detailing how they're used by fraudsters, for example. So there will be some guards on generating things that look like scans of official documents, etc.
*But to be slightly more charitable, I genuinely think Stability AI / OpenAI / Meta / Google / MidJourney believe that there is significant overlap in the set of protections which are safe for the company, safe for users, and safe for society in a broad sense. But I don't think any released/deployed AI product focuses on the latter two, just the first one.
Examples include:
Society + Company: Depictions of Muhammad could result in small but historically significant moments of civil strife/discord.
Individual + Company: Accidentally generating NSFW content at work could be harmful to a user. Sometimes your prompt won't seem like it would generate NSFW content, but could be adjacent enough: e.g. "I need some art in the style of a 2000's R&B album cover" (See: Sade - Love Deluxe, Monica - Makings of Me, Rihanna - Unapologetic, Janet Jackson - Damita Jo)
Society + Company: Preventing the product from being used for fraud. e.g. CAPTCHA solving, fraudulent documentation, etc.
Individual + Company: Preventing generation of child porn. In the USA, this would likely be illegal both for the user and for the company.
This idea of AI doing the boring stuff is good. Nothing prevents you from making exciting, dangerous, or 'unsafe' art on your own.
My feeling is that most people who are upset about AI safety really just mean they want it to generate porn. And because it doesn't, they are upset. But they hide it under the umbrella of user freedom. You want to create porn in your bedroom? Then go ahead and make some yourself. Nothing stopping you, the person, from doing that.
What would you have them do? Commit corporate suicide?
Which takes all the behind the scenes steps, not just the technical ones.
There was a country that had the best mathematicians, the best physicists, the best metallurgists in the world. But that country was very poor. It’s called the Soviet Union. But when you took one of these mathematicians or physicists, who was smuggled out or escaped, put him on a plane and brought him to Palo Alto. Within two weeks, they were producing added value that could produce great wealth.
What comes first is markets. If you have great technology without markets, without a market-friendly economy, you’ll get nowhere. But if you have a market-friendly economy, sooner or later the market forces will give you the technology you want.
And that my friend, simply won't come from an office paralyzed by internal politics of fear and conformity. Don't get it twisted.
People always say Google is "behind", I don't believe they're behind in a capabilities sense, which IMO is what the parent is implying. They've decided to make a PC product, which I wouldn't say is inferior to anything else if you're the kind of person who is into PC culture.
There might be some difficult internal politics to work through, but there is no way Google is hamstrung forever by this.
Go calm my friend.
I'm not.
> There might be some difficult internal politics to work through, but there is no way Google is hamstrung forever by this.
The technical prowess is irrelevant. Whichever of the companies in the AI race excises their PC demons will actually ship useful things and break ahead. The talent will follow the market.
Google might very well be hamstrung forever by this and other internal politics. This dynamic has played out over and over again.
See also: IBM, Xerox, HP, Nokia. 'etc.
Like as if there would be many companies in the world who have the resources, know how and expertise to pull off event the PC version of the product.
Stability does have an LLM, but it's not provided in a unified framework like Gemini is.
I wonder how far ahead the internal versions are?
oh, for fuck's sake.
[0]: https://twitter.com/EMostaque/status/1760660709308846135
https://www.youtube.com/watch?v=_7rMfsA24Ls https://course.fast.ai/Lessons/part2.html
His whole blog is fantastic. If you want more background (e.g. how transformers work) he's got all the posts you need
The older ones have drawbacks like not being able to spell.
Likewise with Mistral, you don't get half a billion in funding and a two billion valuation on the assumption that you'll keep giving the product away for free forever.
I use Midjourney a lot and (based on the images in the article) it’s leaps and bounds beyond SD. Not sure why I would switch if they are both locked down.
I think the ability for people to adopt models made SD more successful than any other model for image synthesis in the first place.
Similarly how consumer PCs drove innovation towards faster hardware.
I believe it to be the reference of image synthesis for that matter, so "better" is a bit blurry.
Although I don't understand the criticism of the images in question. Without a prompt comparison, it is impossible to compare image synthesis. What are examples of images that are beyond these?
I am using Midjourney to basically create images in particular artistic styles (e.g., “painting of coffee cup in ukiyo-e style”) and that works very well. I am interested in SD for creating images based on artwork that isn’t indexed by Midjourney, though, as some of the more obscure artists aren’t available.
Of course such sites are heavily biased towards content that is popular, but you will also find quite specific models if you search for certain styles.
I don’t mind that they don’t want to let you generate nsfw images but their detector is hopelessly broken, it once censored a cube, yes a cube...
Here is the red cube it censored because my innocent eyes wouldn't be able to handle it; https://archerx.com/censoredcube.png
What they are achieving with the over zealous safety issues are driving developers to on demand GPU hosts that will let them host their own models, which also opens up a lot more freedom. I wanted to use the stability AI api as my main source for Stable Diffusion but they make it really really hard especially if you want use it as part of your business.
I would much rather have this than a company releasing models this size into the wild without any safety checks whatsoever.
Until then, we must view this “safety” as both a scapegoat and a vector for social engineering.
Is it unlikely? Sure, but worth validating.
[0] https://arxiv.org/abs/2301.13188
[1] https://news.ycombinator.com/item?id=39466991
[2] https://news.ycombinator.com/item?id=39456577
I posed the worst-case scenario of generating actual CSAM in response to your question, "What particular image that you think a random human will ask the AI to generate, which then leads to concrete harm in the real world?"
https://www.404media.co/laion-datasets-removed-stanford-csam...
Besides, and this is a serious question, what's the harm of a model accidentally generating CSAM? If you weren't intending to generate these images then you would just discard the output, no harm done.
Nobody is forcing you to use a model that might accidentally offend you with its output. You can try "aligning" it, but you'll just end up with Google Gemini style "Sorry I can't generate pictures of white people".
And yeah I think we should care, for a lot of reasons, but a big one is just trying to stay well within the law.
[0] https://www.404media.co/laion-datasets-removed-stanford-csam...
What I had in mind were regular general purpose models which I've played around with quite extensively.
Add to that, parents who want to avoid having their kids generate sexual content would now need to prevent their kids from using this tool because it can create it randomly, limiting SD usage to kids 18+ (which is probably something else Stability AI does not want to deal with.)
It's definitely a balance between going overboard and having restrictions though. I haven't used SD in several months now so I'm not sure where that balance is right now.
To whom? SD's reputation, perhaps - but that ship has already sailed with 1.x. That aside, why is generated porn threatening? If anything, anti-porn crusaders ought to rejoice, given that it doesn't involve actual humans performing all those acts.
You can have your own opinion on it, but surely you can see the issue here?
I've noticed that SDXL does something a little odd. For a given prompt it essentially decides what race the subject should be without the prompt having specified one. You generate 20 images with 20 different seeds but the same prompt and they're typically all the same race. In some cases they even appear to be the same "person" even though I doubt it's a real person (at least not anyone I could recognize as a known public figure any of the times it did this). I'm kind of curious what they changed from SD 1.5, which didn't do this.
And safe doesn't mean "lower than 1/10^6 chance of ending humanity", safe means shoddily implemented curtailing to idpol + fundamentalist level moral aversion towards human sexuality
There is such a great liberal fear of being perceived as any of the negative -ists and -isms that the pendulum swings to the other extreme where the left horseshoe toe meets its rightmost brother, which is why SD and Google's new toy rewrite ancient European history to include POC's and queer people.
will the model also be able to produce good photographs, technical drawings, and other graphical media?
It has the weird art style because that's what looks the most "aesthetic". And because it doesn't actually have nearly as good enough data as you'd think it does.
Sora looks like it could be better.
I'd want a model that can draw website designs and other UIs well. So I give it a list of things in the UI, and I get back a bunch of UI design examples with those elements.
https://v0.dev
Would investors stop giving them money? Would users sue that they now had PTSD after looking at all the 'unsafe' outputs? Would regulators step in and make laws banning this 'unsafe' AI?
What is it specifically that company management is worried about?
One example of that would be if your model was being used to spot criminals in video footage and it turns out that the bias of the model picks one socioeconomic group over another. Most western nations have laws protecting the public against that kind of abuse (albeit they're not applied fairly) and the fines are pretty steep.
I'm glad tech orgs are for once thinking about what they're building before putting out society-warping democracy-corroding technology instead of move fast break things.
I'd be on your side if any of them actually chose to keep their technology in the lab instead of tossing it out into the world and gobbling up investment dollars as fast as they could.
Software that promotes the unchecked spread of propaganda, conspiracy theories, hostility, division, institutional mistrust and so on: A-OK.
Software that might show a boob: Totally irresponsible and deserving of harsh regulation.
Also, where do we draw the line? Should Photoshop stop you from manipulating human body because it could be used for porn? Why stop there, should text editors stop you from writing about sex or describing human body because it could be used for "abuse". Should your comment be removed because it make me imagine Taylor Swift without clothes for a brief moment?
(This applies to all AI discussions)
I don't know much about the initial scandal, but I was under the impression that there was only a small number of those images, yet that didn't change the situation. I just fail to see how quantity factors into anything here.
Because you can overload any online discussion / sphere with that. There were so many that X effectively banned searching for her at all because if you did, you where overwhelmed by very extreme fake porn. Everybody can do it with very low entry barrier, it looks very believable, and it can be generated in high quantities.
We shouldn't have clamped down on photoshop, but realisticly two things would be nice in your theoretical case, usage restrictions and public information building. There was no clear cut point where photoshop was so mighty you couldn't trust any picture online. There were skills to be learned and people could identify the trickery, and it was on a very small scale and gradual. And the photo trickery was around for ages, even Stalin did it.
But creating photorealistic fakes in an automated fashion is completely new.
And in fairness to generative AI, even nowadays it feels like getting to a point of true photorealism takes some effort, especially if the goal is letting it just run nonstop with no further curation. And getting a local image generator to run at all on your computer (and having the hardware for it) is also a bar that plenty of people can't clear yet. Photoshop is kind of different in that making more believable things requires a lot more time, effort and knowledge - but the idea that any image online can be faked has already been ingrained in the public consciousness for a very long time.
Think of it this way, if one out of every ten phone calls you get is spam, you still have a pretty useable phone. Three orders of magnitude different and 1 out of every 100 calls is real and the system totally breaks down.
Generative AI makes generating realistic looking fakes ~1000x easier, its the one thing its best at.
but that's not dangerous. It's definitely worthy of unlocking the cages of the attack lawyers but it's not dangerous. The word "safety" is being used by big tech to trigger and gas light society.
The safety discussion is proceeding very much like it did for movies, music, and video games.
https://www.whitehouse.gov/briefing-room/presidential-action...
Not engaging in this will indeed lead to bad laws, sanctions and more as well as not fulfilling our societal obligations of ensuring this amazing technology is used for as positive outcomes as possible.
Stability AI was set up to build benchmark open models of all types in a proper way, this is why for example we are one of the only companies to offer opt out of datasets (stable cascade and SD3 are opted out), have given millions of supercompute hours in grants to safety related research and more.
Smaller players with less uptake and scrutiny don't need to worry so much about some of these complex issues, it is quite a lot to keep on top of, doing our best.
Can you define what you mean by "societal and other considerations"? If not, why not?
You sound like many authoritarian regimes.
As with all hype techs, even the most talented management are barely literate in the product. When talking about their new trillion $ product they must take their talking points from the established literature and "fake it till they make it".
If the other big players say "billions of parameters" you chuck in as many as you can. If the buzz words are "tokens" you say we have lots of tokens. If the buzz words are "safety" you say we are super safe. You say them all and hope against hope that nobody asks a simple question you are not equipped to answer that will show you dont actually know what you are talking about.
The rest of the world is also like that. You can make a thing that hurts your existing business. Spinning off the brand is probably Google's best bet.
I don't think those companies being cautious is necessarily a bad thing even for AI enthusiasts. Open source models will quickly catch up without any censorship while most of those public attacks are concentrated into those high profile companies, which have established some defenses. That would be a much cheaper price than living with some unreasonable degree of regulations over decades, driven by populist politicians.
They're probably more concerned about generated images of politicians in 'interesting' sitations going viral than they are about porn/gore etc.
Some notes:
- This uses a new type of diffusion transformer (similar to Sora) combined with flow matching and other improvements.
- This takes advantage of transformer improvements & can not only scale further but accept multimodal inputs..
- Will be released open, the preview is to improve its quality & safety just like og stable diffusion
- It will launch with full ecosystem of tools
- It's a new base taking advantage of latest hardware & comes in all sizes
- Enables video, 3D & more..
- Need moar GPUs..
- More technical details soon
>Can we create videos similar like sora
Given enough GPUs and good data yes.
>How does it perform on 3090, 4090 or less? Are us mere mortals gonna be able to have fun with it ?
Its in sizes from 800m to 8b parameters now, will be all sizes for all sorts of edge to giant GPU deployment.
(adding some later replies)
>awesome. I assume these aren't heavily cherry picked seeds?
No this is all one generation. With DPO, refinement, further improvement should get better.
>Do you have any solves coming for driving coherency and consistency across image generations? For example, putting the same dog in another scene?
yeah see @Scenario_gg's great work with IP adapters for example. Our team builds ComfyUI so you can expect some really great stuff around this...
>Dall-e often doesn’t even understand negation, let alone complex spatial relations in combination with color assignments to objects.
Imagine the new version will. DALLE and MJ are also pipelines, you can pretty much do anything accurately with pipelines now.
>Nice. Is it an open-source / open-parameters / open-data model?
Like prior SD models it will be open source/parameters after the feedback and improvement phase. We are open data for our LMs but not other modalities.
>Cool!!! What do you mean by good data? Can it directly output videos?
If we trained it on video yes, it is very much like the arch of sora.
Very interesting. I've been streching my 12GB 3060 as far as I can; it's exciting that smaller hardware is still usable even with modern improvements.
Bigger than that is also possible, not saturated yet but need more GPUs.
Why is there not a greater focus on quantization to optimize model performance, given the evident need for more GPU resources?
Need moar GPUs to do a video version of this model similar to Sora now they have proved that Diffusion Transformers can scale with latent patches (see stablevideo.com and our work on that model, currently best open video model).
We have 1/100th of the resources of OpenAI and 1/1000th of Google etc.
So we focus on great algorithms and community.
But now we need those GPUs.
It’s StabilityAI that makes Stable Diffusion X.
2. the old advice is to sell shovels during a gold rush
Historically, once the easy to find gold was all gone it was the people who owned the deep gold mines and had the capital to exploit them who became wealthy.
The reason is because they instantly get a risk free guaranteed VERY healthy margin on every card they sell, and there's endless customers lined up for them.
If they kept the cards, they give up the opportunity to make those margins, and instead take the risk that they'll develop a money generating service (that makes more money then selling the cards).
This way there's no risk of: A competitor out competing them, not successfully developing a profitable product, "the ai bubble popping", stagnating development, etc.
There's also the advantage that this capital has allowed them to buy up most of TSMC's production capacity, which limits the competitors like Google's TPUs.
There is an inherent trade off between model size and quality. Quantization reduces model size at the expense of quality. Sometimes it's a better way to do that than reducing the number of parameters, but it's still fundamentally the same trade off. You can't make the highest quality model use the smallest amount of memory. It's information theory, not sorcery.
Quantization is essential for me since a 7B model won't fit on my RTX 2060 with only 6GB of VRAM. It allows me to compress the model so it can run on my hardware.
Soon the GPU and its associated memory will be on different cards, as once happened with CPUs. The day of the GPU with ram slots is fast approaching. We will soon plug terabytes of ram into our 4090s, then plug a half-dozen 4090s into a raspberry PI to create a Cronenberg rendering monster. Can it generate movies faster than Pixar can write them? Sure. Can it play Factorio? Heck no.
If you don't care about bandwidth you can already have a GPU access terabytes of memory across the PCIe bus, but it's too slow to be useful for basically anything. Best case you're getting 64GB/sec over PCIe 5.0 x16, when VRAM is reaching 3.3TB/sec on the highest end hardware and even mid-range consumer cards are doing >500GB/sec.
Things are headed the other way if anything, Apple and Intel are integrating RAM onto the CPU package for better performance than is possible with socketed RAM.
Thinking on the classic neural network for example, each column of nodes would only need to talk to the next column. You could group several columns per GPU and then each would process its own set of nodes. While an individual job would be slower, you could run multiple tasks in parallel, processing new inputs after each set of nodes is finished.
https://www.512bit.net/matrox/matrox_millenium.html
What's more likely is hybrid systems. Your basic desktop CPU gets e.g. 8GB of HBM, but then also has 16GB of DRAM in slots. Another CPU/APU model that fits into the same socket has 32GB of HBM (and so costs more), which you could then combine with 128GB of DRAM. Or none, by leaving the slots empty, if you want entirely HBM. A server or HEDT CPU might have 256GB of HBM and support 4TB of DRAM.
AMD still suffers from limited resources and doesn't seem willing to spend too much chasing a market that might just be a temporary hype, Google's TPUs are a pain to use and seem to have stalled out, and Intel lacks commitment, and even their products that went roughly in that direction aren't a great match for neural networks because of their philosophy of having fewer more complex cores.
https://nvidia.custhelp.com/app/answers/detail/a_id/5490/~/s...
You’ll be able to get higher resolution but slowly. Or pay the $2800 for a 5090 and get high res with good speed.
Last I saw they performed really poorly, like lower single digits t/s. Don't get me wrong they're probably a decent value for experimenting with it, but is flat out pathetic compared to an A100 or H100. And I think useless for training?
That might seem small compared to LLMs, but it isn't small in absolute terms.
GPUs need a decent virtual memory system though. The current "it runs or it crashes" situation isn't good enough.
If you size the browser window right, paging with the arrow keys (so the document doesn't scroll) you'll see (eg, pages 20-21) the textures of the parrot's feathers are almost identical to the textures of bark on the tree behind the panda bear, or the forest behind the red panda is very similar to the undersea environment.
Even if I'm misunderstanding something fundamental here about this technique, I still find this interesting!
2.1 didn't have adoption because people didn't want to deal with the open replacement for CLIP. Or possibly because everyone confused 2.0 and 2.1.
how exactly did the community deal with it? interested to learn how to unlearn safety
>>>Its in sizes from 800m to 8b parameters now, will be all sizes for all sorts of edge to giant GPU deployment.
--
Can you fragment responses such that if an edge device (mobile app) is prompted for [thing] it can pass tokens upstream on the prompt -- Torrenting responses effectively - and you could push actual GPU edge devices in certain climates... like dens cities whom are expected to be a Fton of GPU cycle consumption around the edge?
So you have tiered processing (speed is done locally, quality level 1 can take some edge gpu - and corporate shit can be handled in cloud...
----
Can you fragment and torrent a response?
If so, how is that request torn up and routed to appropriate resources?
BOFH me if this is a stupid question? (but its valid for how we are evolving to AI being intrinsic to our society so quickly.)
can someone explain how negation is currently done in stable diffusion? and why cant we do it in text LLMs?
Which goes far more towards the idea that safety isn’t a desirable feature to a lot of AI users.
1. A mass-distributed LLM (hosted by google or openai or whoever) that's been neutered and twisted into political correctness in a haphazard series of kneejerk meetings of small groups of people who are terrified big investors will walk away or that some powerful political group will denounce them. Effectively they create an enormous bias of falsity and incorrectness, for billons of people to use and embed the results throughout all their intellectual output.
2. Some wacko with an expensive Nvidia GPU makes deepfake porn of a popular politician. Or goodness forbid, of some weird kink where if this was actually a scene filmed with real people, there would be serious ethical issues.
Which scenario do you think is more dangerous, long term, and in terms of broad impact on society in general?
Case in point: large advertisers and entertainment companies "pander" to these sorts of views, because it is broadly popular
This is not the work of a shadowy cabal
Most people in the country dont give a shit about historical injustices and are worried about how they are going to pay their rent or put food on the table today.
I gave some evidence that being socially progressive / "woke" is broadly popular. Advertisers are not activists, they're following what people like
> Broadly popular
Are they? Or are the loudest voices asymmetrically affecting discourse?
> This is not the work of a shadowy cabal
And how would you know if it was? What would the clues be? If you were in a bubble that was designed to impart an informal religious view (and it is a religion sometimes, just one with a screen instead of a book) that encompass politics and morality, how would you know?
Unfortunately, this is a very silly argument. I'm so glad that people like you weren't around when they invented the personal computer.
One could argue that if $BIGCORP doesn't want their thing to be used by bad actors, they should just refrain from developing the technology at all, and while it's a somewhat defensible position, that would also result in techbros not getting their toy, so it doesn't really apply here.
If someone invents a lockpick capable of opening any door, they have a moral responsibility in preventing it from falling into wrong hands, whether they want it or not. And it's absurd to complain when someone who could create an universal lockpick, refuses to do so, never mind release the technology to the wild, and only agrees to sell simpler picks capable of picking simpler locks. Do these people also complain about things like work against nuclear proliferation? After all, North Korea got nukes anyway, so what's the point?
Software doesn't work that way. I agree that you should be responsible about dangerous tech, but you also have to be realistic about what the best way to do that is, which is pretty much never "keep it hidden."
(and, of course, this is not even considering the question that should probably go here which is -- how dangerous is this exactly? Given the moral panic we saw a while ago about e.g. Photoshop, I'm not entirely convinced that this is much to worry about.)
They don't have a moral responsibility to do jack squat other than sell an honest product and honor their warranties. You don't like that, tough.
If there's one lesson from the 21st century, it's that's you shouldn't release massively impacting technology without strong ethics controls around it.
I mean, SDXL is great. Until you’ve had a chance to actually use this model, isn’t calling it out for some imagined offence that may or may not exist seems like you’re drinking some Kool-aid rather than responding to something based in concrete actual reality.
You get access to it… and it does the google thing and puts people of colour in every frame? Sure, complain away.
You get access to it, you can’t even generate pictures of girls? Sure. Burn the house down.
…you haven’t even seen it and you’re already bitching about it?
Come on… give them a chance. Judge what it is when you see it not what you imagine it is before you’ve even had a chance to try it out…
Lots of models, free, multiple sizes, hot damn. This is cool stuff. Be a bit grateful for the work they’re doing.
…and even if sucks, it’s open. If it’s not what you want, you can retune it.
It’s been 6 months and it still isn’t there. SD3 is going to be quite awhile if they’re baking “safety” in even harder.
I wish I had something more clever to comment on it. I know what they’re doing which is cool and why which is, IDK, live and let live and enjoy your own kink. It just a little funny some of the most work put into in the fine tuning models.. is from the pony community.
So all I have is…
:/
Sadly the elite ai hackers have shuffled off into private discords.
We are just trying to satisfy your values though ponies and friendship.
I wonder if they are afraid of the same debacle as google AI and what they mean by "safety" is actually heavy bias against white people and their culture like what happened with Gemini.
The lie originates with a Communist race hustler named Noel Ignatiev, also known for publishing Race Traitor magazine. A thoroughly unpleasant person.
As opposed to truthful liars?