Photoshop’s generative infill and expand are on an entirely different level from what Stable Diffusion can do right now. The fact that you can select an area, give it no prompt, and have it usually perfectly fill it in one of the 3 variations it gives you is magical.
Not hard to do. It's auto-filling with an image tagger. That's not a technology so much as just a feature. There's 3 pretty widely used taggers in SD and you can combine them and then inpaint the region of interest.
Putting some UX shimmer on that flow it is the feature there.
It is hard to do. I've tried it. The person/people that originally made Controlnet was pretty amazed at how well Photoshop's works.
Let's use an example of something I've done in Photoshop. Here's (roughly) how the image would be tagged: Bridal party, inside a mansion, stairs behind them. Now I try to get rid of table that's to the side that's in front of a window. What do you think will happen with those tags? It's surely not going to correctly put half of a window and then half of a wall, like it should be. I might get some bridal party, some stairs, etc.
If you just tag the area I'm trying to get rid of it's going to be something like "wood table, window, photograph." Same issue.
You can already do it with Photoshop/Krita/whatever plugins, but it won't work nearly as well. Adobe's version is incredible at understanding the photo or art, the style, and inpainting exactly what should be there without any prompt at all.
They can upgrade hardware to bleeding edge faster than consumers can get their hands on remotely comparable products. Most consumers also don't upgrade every year to stay on bleeding edge, they just tolerate what they can currently do. While this will be fine for most, those actually using generative AI for work aren't likely going to tolerate stagnation as easily.
Adobe is planning for a post-regulation, post-biggest-lawsuit world for sure. All their steps show that they'll base their offering on their own commercial data - paying training license fees to stock images and artists.
Whether the amounts they pay would make licensing your work sensible or not, Adobe is surely assuming this will ultimately end up as Napster-to-Spotify transition.
If we end up "happily" (means legally as well) running these on customer level hardware, then the question won't be about credits of computation. It'll be about credits to use licensed work.
If this is true (which I kinda doubt), is it going to matter to most people? Like you can't really tell the images used to train a model from the images it generates (if it's trained correctly), so I doubt the majority of people would care, like those who already use MJ for example. Training models on copyrighted data for academic research will be allowed, the models will be published, and good luck enforcing the licence; and here I'm talking about the worst case scenario where a court would find an image generated by AI to be derivative of another image in a pool of billions in a dataset (this goes way beyond any definition of derivative work for now).
Whether Adobe ever decides to let their model run locally or lock it forever into the cloud is a choice they will have to make. A lot of people trust Adobe products, so it's entirely conceivable that some people will always choose to pay for a pay-per-use generative solution from Adobe rather than try to run competing solutions locally. The question is probably whether it generates more revenue than negativity for Adobe. If most Adobe users are running their own models locally and avoiding the feature, then I think Adobe will be more likely to follow suit and move away from the pay-per-use cloud approach.
Long term they may offer both, with the local model consuming fewer credits per request. The new GPUs all have secure enclave remote attestation architecture, so Adobe would be able to offer this without the risk of someone jailbreaking the model and running it for free.
The Xbox One has not been hacked in the 10 years since its release. Apple already uses the same kind of tech to protect the various AI models in iPhones and MacBooks, for tasks like OCR search of stored image files and to auto-cutout people in photos.
Enclaves are rarely broken and the people that can are selling it to the CIA, not leaking it on the pirate bay.
What would the explanation for charging money here be though? The user is already paying for the license and they're using their own hardware and power to do so. It'd be like charging per use of any other advanced features in locally-run Adobe software.
> What would the explanation for charging money here be though? The user is already paying for the license and they're using their own hardware and power to do so.
To pay for the privilege of using a very advanced AI model. That's more reasonable than paying to unlock a game character skin that's already on your SSD, and that happens millions of times a day.
> It'd be like charging per use of any other advanced features in locally-run Adobe software.
>It'd be like charging per use of any other advanced features in locally-run Adobe software.
Software companies already do that. There are all kinds of locally-run advanced features that are only enabled with a more expensive subscription tier even though you already have the code and assets for them.
Sure, most are merely subscription based, but there are others that are per use.
I would have said the consumer sentiment amongst Adobe users is the exact opposite - that people don't trust Adobe products but they use them because they either have to or because they're currently the best products available.
I agree, these models are going to get smaller and more performant, the software to leverage your hardware is going to have major improvements, the hardware is going to change to prioritize processing these with more memory to specific coprocessors, and the OS is going to start having these models baked into them, with improved default models being a core feature of upgrading each OS version
people are looking at an extremely limited view of “bigger models on better hardware will always be in the cloud” when that reality simply won’t matter for most use cases
Wouldn't this require a complete reversal of course of, like, the last fifteen years of computing? Basically everyone wants to get out of selling products and into selling services.
When we reach the point when average people can easily run today’s models on their own devices, today’s models will no longer be SOTA and there will still be demand to run better models in the cloud.
Image generation parameter count is hitting diminishing returns, going by what we've seen with SD/SDXL. The tooling around them is far more important.
However, Stable diffusion already can run on mobile devices. There is already a good iOS app for it (and the dev is here on HN) but the problem seems to be that no one cares. There are 700,000 cloud imagegen apps crowding it out, because thats what's easier and more profitable to spam across the store and web.
> Image generation parameter count is hitting diminishing returns, going by what we've seen with SD/SDXL.
For image quality, sure - language understanding is still an issue. SDXL can generate a beautiful image, but if it doesn’t show exactly what you asked for in the prompt, on the first try, there is still room for improvement. The gap between LLMs and image generators in this regard is huge.
Depends if the limit on parameter count is a 'real' limit, or just a limit based on what current-technology models can effectively use.
Back in the 'Google Daydream' days, Google might have found that they didn't get any more image-generation performance by raising the parameter count - but that's just because the technology at the time couldn't effectively utilise more parameters. It's impossible to know what next-gen models might be able to use, but I suspect we will find ways to allow the models to take advantage of even higher parameter counts.
Stable diffusion can run on mobile devices, but it's painful and image generation takes a fraction of the time via cloud services.
We already run these on customer level hardware, you need a good PC but still, in the future the pool of people that can do is just going to be bigger.
> we will be happily running these on customer level hardware.
This is the stage of AI that will impress me most. If I can use your AI completely offline on my device on a spaceship orbiting Pluto, then I will say we have achieved an AI capacity that is impressive, even if its got the quirks of chatgpt today.
So you can fit all of ChatGPT into a 32GB storage phone? Because my understanding is something like ChatGPT takes terrabytes upon terrabytes of data on-disk and I can't imagine how many servers being piped together to make it churn out coherent sentences... I'm on about all of that on a phone that doesn't cost thousands of dollars. It would mean we have finally reached significant technological advances.
Is it moving a goal post if I'm just going off on a tangent from one simple remark? I'm just going on a tangent of something I find fascinating as a thought experiment, what will things look like when we can have isolated AIs in our pockets without needing connectivity? The reason I want this is because you could leave your house for a long roadtrip, and have an AI companion that never loses a step. Imagine if your AI had every road memorized and nuances about them, so if GPS signal dies, you can still keep going.
Sidenote, ChatGPT uses tens of thousands of GPUs to run its architecture, I think it'll take a little more than just some laptop.
Thank you for answering though :) I do think its an invaluable goal to have off-line first AI.
I think you are not super familiar with how AI works. It's worth reading up on.
For example - The Tesla self driving AI takes many hundreds (thousands?) of computers to build the model. Then it runs in realtime of 1 "GPU" that lives in my car. It's not sending frames in realtime to a supercomputer to process.
So for a spaceship - same thing. You don't need to send the thing that makes the model. You just send a finished model and a GPU to run it.
I think that we still need a few years 3-5 alone for better models, different architectures and more finetuning.
Those models will affect us more than today already and change how we perceive AI.
Than we will start to see AI optimized hardware (much more optimized).
And than perhaps in 10 years we all run a lot more models locally.
Nonetheless or despite this, the normal consumer doesn't run open models and will probably not do that for a very long time. Searching, keeping up-to-date and running models is still effort and the usage model makes a ton of sense. Escpecially in time of SaaS.
Im not running wikipedia locally. And none of my social circle operates infrastructure / server.
People just want to use it.
Besides that, whatever local models or open models will be able to do, AIaaS will have faster models, better models and more convinient models.
I'm just waiting to pay for google assistent if it becomes smart and can manage my emails my calendar and everything else. After all my gmail account already has access (through email and password reset) to most services i use.
I'm more curiuos when we will see AI service integration through much more system to system communication. Machine friendly apis (which partially already exist anyway)
PS: Look at how fast hardware development currently is. Not much change in Memory etc. Models will not just become 100x smaller in just a few years. We are right now at optimizing those models to be cost efficient. Alone this phase will take a few years.
> I'm sure that in the not too distant future (a few years at most) we will be happily running these on customer level hardware.
The models themselves will be hoarded as IP. Doesn't matter if they're in the cloud or on devices, they'll be licensed like commercial proprietary software with the same restrictions commercial software has.
This will be true as long as they are crazy expensive to produce. Eventually the cost of training a base model could drop to the range where crowdfunding could do it.
Or alternately someone could make a major advance in distributed training and we could all contribute cycles in a distributed effort like Folding@Home. As it stands training requires far too much bandwidth for synchronization and moving model data around. Some approach to sharding training would have to be discovered. It’s an open problem area.
Neural networks are very parallelizable and training is stochastic so my intuition is that it should be possible. Even if it were less efficient than synchronous training you could make up for that by harnessing 100X the compute from a huge crowd.
Platforms and rightsholders know what they're sitting on now, and I worry that the datasets required to train sufficient models in the future will also be hoarded as IP.
It's one thing to train on Common Crawl in 2023, but what about when you have to shell out millions of dollars just for access to data sets to train on in the future? Same thing with human reinforcement. The customers for both are willing to pay much more than a crowdfunding campaign would.
Training is expensive now, but data sets can be expensive in the future.
Collecting training data is actually a perfect for crowdsourcing. Images/videos are easier than text, and text is easier than high-quality text, but all are doable.
Only so far as you trust your crowd. It only takes one bad actor to poison an entire data set with a few pieces of copyrighted samples, and now the entire model is subject to potential future litigation.
How is it different from stock images? People could sell not their work on the stock sites, and it is pretty hard to detect.
Or impossible to detect, if you ask some AI model to repaint the image first, before putting it into the stock.
I feel like the amount of training data needed to compete with commercial models would require a large amount of automation to collect, and it's that same automated collection that platforms recently decided to gatekeep against.
have we exhausted the ways to train models already though? Feeding common crawl into a more advanced training program could result in a better model compared to today
Yes, but as time goes on, new data is created that won't be in Common Crawl. Many of the troves of data it archived from in the past are now behind expensive paywalls that are priced with their multibillion dollar clients' access in mind.
If the dataset contains text from 2022, things that happen in 2023 and later won't be in it. The model will only get you so far, and new data, events, concepts, discoveries, etc will be absent.
If we trained models on all text generated up until 1900, for example, you could get it to produce some impressive results if just generating text is the goal. If the goal is to build something that imitates a more general AI, it wouldn't "know" about antibiotic treatments for common illnesses, modern vaccines, either World War, powered flight, transistors, computers, etc. It would only be useful for so much.
I run stable diffusion XL with control nets on a laptop, albeit a high end one. It’s pretty decent.
I also run LLMs such as trains of llama2, though LLMs on commodity hardware are not as “there” yet as image generators. It’s a decent question and answer bot and summarizer but isn’t GPT-4 level. I could see another iteration approaching that but I’d probably need more RAM.
I'm unconvinced that local deployment will be the preferred way to run generative models anytime soon.
They seem like pretty much the perfect fit for cloud - burst compute which would result in very low hardware utilisation if ran locally.
Why would it be better to have a $1,500 GPU that is weak and used infrequently, when you could share a big cluster of better GPUs shared between a big group of people, and have it more heavily utilised?
There is a philosophical argument about owning your own hardware etc, however I think the economics and performance will eventually push this to the cloud for most use-cases (most people will just get better bang-for-buck in the cloud).
Firefly models are not publicly available. The integration with creative cloud apps is not customizable. You can't point it to a local stablediffusion setup. Open-source competitors will flop just like GIMP.
I mean that sounds good and cool and worth trying for the average hackernews, but not for the average adobe user. Like people are not paying for AI tech to get the 80% mostly-there version, they're paying for the 100% spit-and-polished end product that has the 80% of the work needed to achieve the last 20% of productizing. Consumers have demonstrated over and over they will pay a lot more to avoid dicking around with stuff and professionals can have an even stronger tendency to do this bc their time has a cost.
I have an M1 but I first remember thinking it was very impressive that my Mac could run these AI models…
…until I tried the same on my RTX 4070 and it made my Mac look like a joke.
For the 30 seconds my Mac would have taken for 1 result, which will probably need revising, the RTX would give me 30 results.
However the RTX was half the cost of my Mac, so it’s not a good investment if I just want to generate some images. I’d rather pay for the cloud if I didn’t have the RTX already.
Yeah my M1 Max is about as fast as my old 1070, at 24 steps per image it comes out to little over 2 images per minute. My new 4090 does an image per second.
M2 will be a joke for the AI models of the future.
The actual best image AI, midjourney, is probably a gigantic model under the hood, that takes 8 A100s to run (Aka more than 100GB VRAM). That's why their quality is leaps and bounds above stable diffusion XL, its because the model size simply allows for it.
Model sizes continuously grow to exploit the available hardware to the limit. Midjourney and GPT-4 have both proven that model quality is decisive to success and paying customers, so consumer hardware can never catchup to whatever Nvidia sells to the cloud.
> The actual best image AI, midjourney, is probably a gigantic model under the hood, that takes 8 A100s to run
Do you have any source for this speculation? In my experience image models are always much smaller than language and even the largest llama will fit in a smaller GPU machine than that.
>The actual best image AI, midjourney, is probably a gigantic model under the hood, that takes 8 A100s to run (Aka more than 100GB VRAM)
Unet are really expensive to run compare to a regular GPT model and they are compute-bound thx to convolution, a reason why no one has trained a unet that comes close to consume 100GB of VRAM during inference. I doubt that MJ is much bigger than SD XL, it's good but not revolutionary.
Not true.
Stable Diffusion locally on my 3080 is already faster than Adobe’s offering, not to mention after all my model tweaking and fine tuning much more useful and accurate.
Is it overall easier to use or setup? No, but I can run it for free over Adobe.
I honestly don't think that only 1 in every 10000 Adobe users own a modern-ish gaming-capable PC (and that's what's required to run SD as of now). Point being, the fact that you can already run these models on off-the-shelf consumer hardware is very promising, and as time goes on, more and more new hardware will be capable of running this.
I assume that they actually use the hardware /GPU they have and might not be able to run SD next to it constantly without upgrading.
But someone needs to make this possible and maintain such a solution which would cost also money.either you pay adobe what you already do or pay someone else who maintains the model, the infrastructure etc.
Sure some will run it themselves but my guess is that this is a niche group of people as most don't care .
But a 3080 is by no means a minimum requirement - not even close. A 3080 will give you a very comfortable experience, sure, but so will a midrange card from the same generation. And people have been squeezing SD on way less powerful hardware - even most cards from the GTX 10 series (that came out in 2016) can be made to run it, although some sacrifices have to be made below 6GB of VRAM.
even consumer GPUs have massive generational speedup. In the mid term, cloud will be hard pressed unless Nvidia aims at depressing consumers (leaving themselves open to attacks from everyone else on that market segment)
I also have a 3080, and there’s no way SD is faster. The best comparison as far as resolution is SDXL and in the time it takes me to generate one image at a reasonable amount of steps Adobe generates 3 options.
This is referring to the Photoshop stuff, which is way better than any type of SD inpainting for removing things from images. Firefly might be slower? I haven’t used it since it first came out.
I agree for just general image generation SD or Midjourney are better options in their own way.
I kind of doubt those numbers, considering my 12GB 3080 gets something like 3 it/s in ComfyUI - though I suppose that's usually with a LoRA. Even if they're accurate, very few people have a 3090/4090. Keep in mind the Photoshop version would have something akin to Controlnet going on as well, which would slow things down.
At 5usd for 100 operations, that thousand dollar gpu will break even very swiftly.
I used 5000 or more runs when I was learning to use stable diffusion.
In a commercial context, staff training alone will pay for the cards
My thoughts exactly. I did some guesstimates last month, the experimentation and playing I've done with my 3090 so far would have cost somewhere around $7k on the metered OPC options.
Sure we will be happily running these on customer level hardware, but there will always be a stronger version in the cloud “worth paying for,” won’t there?
We have lots of legal uncertainties on the model training side and we don't know what's the conclusion yet. If most of those "open source" models deem to be problematic with copyrights or whatsoever, then it might make more sense to use proprietary models for commercial use cases since many of those use cases are not really critical (e.g. replacement of stock images) but giving them lots of potential headaches with legal complications.
But when we will be able to run the current models on customer level hardware, won't the models available from cloud hardware so much better we would rather pay for the cloud?
The trend for big software is to charge more and more, while locking you in. Payments become more granular and gradual rather than predictable and discrete.
Are the technical requirements driving these monetization schemes or is it the other way around?
I saw a Mastodon thread by Paul Cantrell that equated CEOs and AI with children and the year's hot Christmas toy. Must have and have it NOW, not after Christmas when we find out if kids sustain interest in playing with it for past New Years. https://hachyderm.io/@inthehands/111075245991998920
How infantilizing. For those who have been able to make it work for them, it's already a load-bearing piece of infrastructure. How do CEOs and children see the new bridge on their commute to work/school?
This is absolutely not about hardware (or even software). It is about indemnification from copyright infringement claims. Adobe's main selling point for Firefly (or whatever it is called) is that is trained on Adobe's data.
I suspect many companies will do something like this - prepaid credits or tokens for AI features that have high inference costs. Inference costs per user are high, at least much higher than most traditional software costs. This way, the costs are aligned with the usage.
We’ll also see occasional subscription products, but only when it can be done in a way that is comfortably gross margin profitable for most users of a company. (Eg ChatGPT Plus, Claude Pro, Midjourney, 365 Copilot)
This will only change when the cost of inference goes down by a lot.
MidJourney claims 3.3 hours of GPU for $10.
If that's 3.3 hours of A100, at a rate of $2/hour that's $6.6 of GPU costs.
My original statement was based on MJ feeling very expensive per image compared to the huge number of images I can generate in an hour with SDXL on my 3090 (and 3090s can be rented for $0.20 an hour).
But I forgot how overpriced A100s are (I doubt MJ is running 3090s but that'd be pretty cool), and that MJ is probably 4x the size of SDXL (although surely more optimized).
My revised statement is that for their $10 plan there's a few bucks of GPU compute cost. Probably between 2x and 5x margins.
A100s aren’t $2/h at MJ scale tho. Firstly they likely own quite a few and they’re relatively cheap now at $4k a card, and secondly you can get much better deals than $2/h if you rent a large amount and pre-pay.
ERROR: 0.25oz of liquid remains in your Adobe™ x Mountain Dew™ Verification Can™. Please finish drinking to proceed using the software product. Thank you for your compliance.
At this point you could just call it microtransaction. Imagine free, but extremely bare bones implementations of photoshop where anything but one brush and eraser is hidden behind individual or package deals.
I was thinking the same thing.
They have introduced the advanced filters in Photoshop.
They display "running locally" (or something close to it)
when used.
I hope it won't be cloud only soon for even more $$$$$.
For the subscription to even use PS, and then $ for using
this or that feature.
It only makes sense for things that run in the cloud, as they need to pay for that compute. Similar to costs “per request” in other cloud services like AWS.
Although if Adobe did charge “per use” for locally run tools like brushes and filters… I’d hope they’d at least they’d make the billing completely “usage based” with no minimum monthly cost.
But this is insane and evil, I’ll stick with Gimp and Inkscape
It doesn't read too bad:
Creative Cloud and Adobe Stock paid users can keep taking generative AI actions, but the use of generative AI features may be slower.
Is adobe trained on their own library of licensed images, as opposed to scraping whole internets?
If so, even as a private individual just fooling around, I'll start using it from both legal and ethical perspective as long as it's reasonably equivalent to other models. And this from a person who's been fairly vocal against adobe's cloud subscription model ;-<. I can only imagine for anybody with a commercial need it would be an immediate no brain er - they'll have an established relationship, account and billing, they'll perceive it as integrating in their work flow, and it'll just become another part of the pipeline.
I’ll not pay for it, I’ve tried the tool in beta and it’s nothing to write home about. And if they decide to get cute about other creative pricing of existing features - I hope people will move on.
“Generative credits provide priority processing of generative AI content across features powered by Firefly in the applications that you are entitled to. Generative credit counts reset each month.”
I use Adobe Photoshop professionally, mostly to tweak technical images for educational purposes (e.g., removing extraneous visual data, like wires that partially obscure a component that the viewer needs to see).
In its current state, at least, Photoshop's generative AI requires a lot of iterations to tweak the image into what I need (due to them requiring technical accuracy). Charging credits for this would make it nonviable.
I use it for things that are not so technical so I find I get results I like with very little work. For instance after I made a print I couldn’t unsee this white splotch so I had selected it and used generative fill without a prompt to make it go away
when I put the corrected one on the wall together with a lot of prints it was not properly centered vertically (had to center the painting + shadow not just the painting) so I had generative fill draw another row of bricks at the bottom. I’m sure they don’t look exactly like the the corresponding bricks on the real wall but they are good enough for wall art
It adds up to another reason to keep my Creative Cloud subscription instead of looking for an alternative.
One thing that is t clear to most is that generative fill does not consider the contents of the area you are filling so asking it to remove something doesnt do anything - you’d have the same results by leaving the prompt blank
yep, I usually leave the prompt blank and get the result I want, I only write a prompt if I don't like what I get without the prompt.
The thing is if you want to have it add a spaceship or a pretty girl or something like that to the photo it has to make more background to go with whatever you tell it it draw so it automatically draws something consistent with the scene without a prompt.
There is a whole research area about tools that do "editing" so you could tell it to do something specific to an image, say "increase the exposure by 0.2 stops" (not that you need a.i. for that but somebody might rather do that than find the function in the UI) or "add a fourth motorcycle to the three motorcycles that are parked in a row" and that's even being done for 3-d models with NeRFs and point clouds and such but that's not what generative fill does right now.
> I use it for things that are not so technical so I find I get results I like with very little work
I mostly get nice fill results as well, but they are super aggressive about removing "unsafe results", even when I don't give it a prompt? I'm not filling in nudity or anything, so it's not something I'd pay extra for in this state, I don't need an AI nanny.
It wouldn’t just be slow, I’m guessing it wouldn’t run on everything but maybe GPUs with 24GB of VRAM. The newest version of Stable Diffusion is the same resolution and the only way they got it to work on 12GB cards was the split the model in two.
I'm sorry, I believe they said they split it for 8 GB cards. The point still remains that it's way faster than local generation.
Of course, I wouldn't mind it not using credits if I could use it locally either. Hopefully they continue to improve SD's IP Adapter type stuff so it's as good as Photoshop in that regard.
33 credits a day will be nothing, and at 5usd for 100, that's going to be $10 or more an hour for anyone doing any kind of regular workflow.
How munch would it cost you?
The stuff that they’ve added in 2023 alone is absolutely huge, as a photographer. The new Photoshop Remove tool along with the generative stuff has saved me hours upon hours of work. I’ve fixed images in seconds that would have taken 10 or 20 minutes before.
Krita's generative thing is quite good, compared to pre firefly PS. Krita has the same UI based issues Gimp has, namely they didn't have the nerve to steal PS shortcuts and tools. Just do it already.
I'm using firefly for the first time on a paid academic account and Im immediately struck at how good it is at making graphic / illustrative images, but it seems to be at the cost of photographic / photorealistic images. ie I can't proompt engineer it into non-illustrative non-stylized images with "realistic photograph of..." etc
I noticed the same thing with SDXL, DALL-E, Midjourney etc. It makes 100% sense from a business / tool standpoint but I miss the weird raw-internet vibe of Stable Diffusion <2 and other early versions of these tools. Could totally be my limited testing / imagination too.
In any case it seems to be a capable illustrator, but I'm not surprised they're setting up a credit system. They must have put an insane amount of work to be first to ship on a big AI image editing suite.
I can generate perfectly realistic “photographs” on SDXL.
You certainly don’t want to use the word ‘realistic,’ because real photos aren’t described that way. Just do “photo of blah blah,” or “blah blah, photo.”
Anything with credits is shit. It creates an artificial sense of scarcity and really sours my experience with a company. At the very least it should be something non fungible along the lines of “you can 200 generates a month”.
It isn't artificial, you are consuming resources and have to pay for it. You are not renting hardware for full month but buying some time share. It is similar to electricity from grid vs solar panels.
Adobe’s model is their IP and is trained on a bunch of images that they paid to be able to use. It’s the mode ethically produced model so far. We have no idea the size. There is zero chance they would let that model leak.
People consume resources doing a lot of things- your internet connection for example but a majority of home connections are essentially unlimited bandwidth for a fixed price. ChatGPT/Midjourney have found a way to bundle their offerings in a way that isn’t grating. It’s up to the companies to find a way to package that while still being able to make a reasonable margin so your customer base isn’t perpetually seething.
>> Pricing: Generative Fill, Generative Expand, Text to Image, Generative Recolor: 1 credit*
>> *For standard images of up to 2000 x 2000 pixels.
>> We plan to offer higher-resolution images, ..., in the future. The number of generative credits consumed for those features may be greater.
When you think that a pretty common DSLR or mirrorless camera these days shoots a 40-50mp image, a 2000 x 2000 pixel baseline for 1 credit is pretty small.
Pretty sure that "Instagram-only photographers" is not Adobe Photoshop's core audience. And even when uploading to Instagram, most editing is done on original images that are much higher resolution, which are then downscaled for use on IG.
Soundcloud does this with their new AI audio mastering SaaS, which they push hard every time you upload a track (it comes with a view promotion boost). It does the audio equivalent of the too-many-fingers problem images have (it ducks pads to the snare along with the bass).
This kind of reminds me of where we were with samplers in the late 80s, which makes me assume that we geezers will continue to complain and the new Marley Marl will appear and use the tech for something none of us can imagine right now.
Creative cloud all products plan gets 1000 credits per month. This seems pretty generous to me? I haven’t been using anywhere close to 1000 generative edits a month during the beta.
All the takes here that this is something I should be able to do locally are missing the point to go on a tangent.
The power of what Adobe is offering is generative AI in the context of proprietary photoshop tools.
Can you feed a prompt to a model outside of photoshop and ask for a particular result? Of course. But if your workflow already happens in an Adobe product and you want a highly specific and predictable result from Ai trained on a highly specific action, that’s something that you really can’t efficiently replicate elsewhere.
In general, it is necessary to generate the same image multiple times to get the AI to generate an image with the correct number and curvature of fingers. If it costs money every time, including failed images, it would be difficult to use, wouldn't it?
I really wish the FTC would do something about 'Microsoft points' and 'Nintendo shop coins' and other such non-currencies you must exchange real currency for in order to buy things. These companies should be forced to put a real dollar amount on their items.
Companies should also be fined heavily for charging 4 coins for an item, but only selling the coins in increments of 17. It's an obvious scam.
If companies wish to make their own currency, they should be treated like a bank, and comply with the relevant regulations, including whatever regulations are necessary to create a new currency.
IMHO PayPal dollars are the worst because they trick you into believing you hold currency when you don’t. PayPal may, at their discretion, allow you to convert your PayPal dollar to another currency. They also may take it away from you for no reason, without recourse - something that happens to thousands of people daily.
Scrip has just been around for forever and so ubiquitous that it would cost many billions to outlaw. It would be outrageous really if the FTC could just outlaw it with a stroke of a pen, and I doubt they have the legal ability to do so, since it should be a matter for elected representatives.
That all being said, I would celebrate the demise of loyalty points, gift cards, Robux, Chuck E. Cheese tokens, and so on. All of it is fundamentally anti-consumer. I'm not sure when countries outlawed paying your employees in scrip why they didn't go one step further since scrip is intrinsically anti-competitive.
I think you can steelman some good arguments for scrip though... both in terms of "why venerate fiat state-backed currency as having a special status" and "some people actually LIKE the inflexibility or having to spend their money on a certain companies products, such as compulsive misers"
It's also not great when they're devalued over time.
They serve a couple useful purposes though, even from a consumer's perspective (unless we fix enough other broken things about our payment infrastructure). There's an inherent conflict between convenience, security, transaction fees, and the chance of not being paid (or for online services like this, the guarantee that somebody orchestrates millions of accounts to distribute a compute load while fully expecting to never pay a dime) when dealing with very frequent or very small transactions.
Other options Adobe might have considered include:
1. Don't offer the feature (if the feature is good enough to tolerate company currency then this seems like a worse option)
2. Pay per use (high transaction costs, high friction if you don't store payment details, low security if you do, risk of default if you batch uses over time, ...)
3. Monthly subscription giving unlimited uses (not tenable for AI that has very real compute costs associated with it, you'll have users abuse the limits)
4. Monthly subscription with a max number of uses (just a different way to pay for credits, strictly worse than buying credits and having them last indefinitely)
5. Batch uses over time, bill if they exceed a threshold (lowers transaction fees, increases security, increases consumer goodwill, vulnerable to targeted attacks)
And so on. Do they actually have any good options, or are they choosing between a bunch of least-bad solutions? I ask partly because I was planning to use a similar model of charging up-front for credits for an unrelated service because I thought it would be the most consumer-friendly in that domain.
177 comments
[ 0.30 ms ] story [ 149 ms ] threadhttps://github.com/AbdullahAlfaraj/Auto-Photoshop-StableDiff...
Putting some UX shimmer on that flow it is the feature there.
Let's use an example of something I've done in Photoshop. Here's (roughly) how the image would be tagged: Bridal party, inside a mansion, stairs behind them. Now I try to get rid of table that's to the side that's in front of a window. What do you think will happen with those tags? It's surely not going to correctly put half of a window and then half of a wall, like it should be. I might get some bridal party, some stairs, etc.
If you just tag the area I'm trying to get rid of it's going to be something like "wood table, window, photograph." Same issue.
I'm sure that in the not too distant future (a few years at most) we will be happily running these on customer level hardware.
I do wander if companies working to develop these type of revenue models truly think it's a long term structure?
Unless nVidia changes their monetization model, and for example introduces an App Store for AI, with subscriptions, of course on locked down hardware.
Whether the amounts they pay would make licensing your work sensible or not, Adobe is surely assuming this will ultimately end up as Napster-to-Spotify transition.
If we end up "happily" (means legally as well) running these on customer level hardware, then the question won't be about credits of computation. It'll be about credits to use licensed work.
If this is true (which I kinda doubt), is it going to matter to most people? Like you can't really tell the images used to train a model from the images it generates (if it's trained correctly), so I doubt the majority of people would care, like those who already use MJ for example. Training models on copyrighted data for academic research will be allowed, the models will be published, and good luck enforcing the licence; and here I'm talking about the worst case scenario where a court would find an image generated by AI to be derivative of another image in a pool of billions in a dataset (this goes way beyond any definition of derivative work for now).
Whether Adobe ever decides to let their model run locally or lock it forever into the cloud is a choice they will have to make. A lot of people trust Adobe products, so it's entirely conceivable that some people will always choose to pay for a pay-per-use generative solution from Adobe rather than try to run competing solutions locally. The question is probably whether it generates more revenue than negativity for Adobe. If most Adobe users are running their own models locally and avoiding the feature, then I think Adobe will be more likely to follow suit and move away from the pay-per-use cloud approach.
Enclaves are rarely broken and the people that can are selling it to the CIA, not leaking it on the pirate bay.
To pay for the privilege of using a very advanced AI model. That's more reasonable than paying to unlock a game character skin that's already on your SSD, and that happens millions of times a day.
> It'd be like charging per use of any other advanced features in locally-run Adobe software.
I don't see anything stopping this.
Software companies already do that. There are all kinds of locally-run advanced features that are only enabled with a more expensive subscription tier even though you already have the code and assets for them.
Sure, most are merely subscription based, but there are others that are per use.
I would have said the consumer sentiment amongst Adobe users is the exact opposite - that people don't trust Adobe products but they use them because they either have to or because they're currently the best products available.
people are looking at an extremely limited view of “bigger models on better hardware will always be in the cloud” when that reality simply won’t matter for most use cases
However, Stable diffusion already can run on mobile devices. There is already a good iOS app for it (and the dev is here on HN) but the problem seems to be that no one cares. There are 700,000 cloud imagegen apps crowding it out, because thats what's easier and more profitable to spam across the store and web.
For image quality, sure - language understanding is still an issue. SDXL can generate a beautiful image, but if it doesn’t show exactly what you asked for in the prompt, on the first try, there is still room for improvement. The gap between LLMs and image generators in this regard is huge.
Back in the 'Google Daydream' days, Google might have found that they didn't get any more image-generation performance by raising the parameter count - but that's just because the technology at the time couldn't effectively utilise more parameters. It's impossible to know what next-gen models might be able to use, but I suspect we will find ways to allow the models to take advantage of even higher parameter counts.
Stable diffusion can run on mobile devices, but it's painful and image generation takes a fraction of the time via cloud services.
I doubt that is what Adobe will do. This is a new revenue stream for them, why would they remove it?
Gimp will use local generation but Adobe is using a proprietary dataset that they can keep secure in the cloud.
So yeah this is going to be sticking around.
This is the stage of AI that will impress me most. If I can use your AI completely offline on my device on a spaceship orbiting Pluto, then I will say we have achieved an AI capacity that is impressive, even if its got the quirks of chatgpt today.
> If I can use your AI completely offline on my device on a spaceship orbiting Pluto
And the answer was yes. I do not know the exact system requirement of the current ChatGPT, but I am fairly confident
1) ChatGPT no network mode could fit in a half of a server rack, maybe way less.
2) You can fit half a server work on a spaceship that can go to Pluto.
My guess is it's more like 1 server worth. Google tells me GPT3 was 1TB which is a very small laptop.
Sidenote, ChatGPT uses tens of thousands of GPUs to run its architecture, I think it'll take a little more than just some laptop.
Thank you for answering though :) I do think its an invaluable goal to have off-line first AI.
For example - The Tesla self driving AI takes many hundreds (thousands?) of computers to build the model. Then it runs in realtime of 1 "GPU" that lives in my car. It's not sending frames in realtime to a supercomputer to process.
So for a spaceship - same thing. You don't need to send the thing that makes the model. You just send a finished model and a GPU to run it.
Those models will affect us more than today already and change how we perceive AI.
Than we will start to see AI optimized hardware (much more optimized).
And than perhaps in 10 years we all run a lot more models locally.
Nonetheless or despite this, the normal consumer doesn't run open models and will probably not do that for a very long time. Searching, keeping up-to-date and running models is still effort and the usage model makes a ton of sense. Escpecially in time of SaaS.
Im not running wikipedia locally. And none of my social circle operates infrastructure / server.
People just want to use it.
Besides that, whatever local models or open models will be able to do, AIaaS will have faster models, better models and more convinient models.
I'm just waiting to pay for google assistent if it becomes smart and can manage my emails my calendar and everything else. After all my gmail account already has access (through email and password reset) to most services i use.
I'm more curiuos when we will see AI service integration through much more system to system communication. Machine friendly apis (which partially already exist anyway)
PS: Look at how fast hardware development currently is. Not much change in Memory etc. Models will not just become 100x smaller in just a few years. We are right now at optimizing those models to be cost efficient. Alone this phase will take a few years.
The models themselves will be hoarded as IP. Doesn't matter if they're in the cloud or on devices, they'll be licensed like commercial proprietary software with the same restrictions commercial software has.
I mean I guess my electricity provider gets paid per compute.
Or alternately someone could make a major advance in distributed training and we could all contribute cycles in a distributed effort like Folding@Home. As it stands training requires far too much bandwidth for synchronization and moving model data around. Some approach to sharding training would have to be discovered. It’s an open problem area.
Neural networks are very parallelizable and training is stochastic so my intuition is that it should be possible. Even if it were less efficient than synchronous training you could make up for that by harnessing 100X the compute from a huge crowd.
It's one thing to train on Common Crawl in 2023, but what about when you have to shell out millions of dollars just for access to data sets to train on in the future? Same thing with human reinforcement. The customers for both are willing to pay much more than a crowdfunding campaign would.
Training is expensive now, but data sets can be expensive in the future.
If the dataset contains text from 2022, things that happen in 2023 and later won't be in it. The model will only get you so far, and new data, events, concepts, discoveries, etc will be absent.
If we trained models on all text generated up until 1900, for example, you could get it to produce some impressive results if just generating text is the goal. If the goal is to build something that imitates a more general AI, it wouldn't "know" about antibiotic treatments for common illnesses, modern vaccines, either World War, powered flight, transistors, computers, etc. It would only be useful for so much.
I also run LLMs such as trains of llama2, though LLMs on commodity hardware are not as “there” yet as image generators. It’s a decent question and answer bot and summarizer but isn’t GPT-4 level. I could see another iteration approaching that but I’d probably need more RAM.
They seem like pretty much the perfect fit for cloud - burst compute which would result in very low hardware utilisation if ran locally.
Why would it be better to have a $1,500 GPU that is weak and used infrequently, when you could share a big cluster of better GPUs shared between a big group of people, and have it more heavily utilised?
There is a philosophical argument about owning your own hardware etc, however I think the economics and performance will eventually push this to the cloud for most use-cases (most people will just get better bang-for-buck in the cloud).
…until I tried the same on my RTX 4070 and it made my Mac look like a joke.
For the 30 seconds my Mac would have taken for 1 result, which will probably need revising, the RTX would give me 30 results.
However the RTX was half the cost of my Mac, so it’s not a good investment if I just want to generate some images. I’d rather pay for the cloud if I didn’t have the RTX already.
The actual best image AI, midjourney, is probably a gigantic model under the hood, that takes 8 A100s to run (Aka more than 100GB VRAM). That's why their quality is leaps and bounds above stable diffusion XL, its because the model size simply allows for it.
Model sizes continuously grow to exploit the available hardware to the limit. Midjourney and GPT-4 have both proven that model quality is decisive to success and paying customers, so consumer hardware can never catchup to whatever Nvidia sells to the cloud.
Do you have any source for this speculation? In my experience image models are always much smaller than language and even the largest llama will fit in a smaller GPU machine than that.
Unet are really expensive to run compare to a regular GPT model and they are compute-bound thx to convolution, a reason why no one has trained a unet that comes close to consume 100GB of VRAM during inference. I doubt that MJ is much bigger than SD XL, it's good but not revolutionary.
For me it kicks the shit out of Midjourney in flexibility and quality. I can make more images of higher quality, faster and cheaper.
An optimized XL model in the hands of an expert beats it, handily.
And Adobe sells to experts, not consumers for the most part.
Energy, partial hardware cost, setup time, fine-tuning time.
But someone needs to make this possible and maintain such a solution which would cost also money.either you pay adobe what you already do or pay someone else who maintains the model, the infrastructure etc.
Sure some will run it themselves but my guess is that this is a niche group of people as most don't care .
And designing your software to a minimum-spec of a 3080 would be pretty wild.
This is referring to the Photoshop stuff, which is way better than any type of SD inpainting for removing things from images. Firefly might be slower? I haven’t used it since it first came out.
I agree for just general image generation SD or Midjourney are better options in their own way.
xformers, 1024x1024x diffuser pipeline.
Are the technical requirements driving these monetization schemes or is it the other way around?
Plenty of us already are. SDXL is as good as anything in the cloud.
On the contrary. In a few years there wont be a lot of customer level hardware software (especially business software) without a subscription.
I'm sure the workloads will shift locally more and more, if for no better reasons than latency and privacy.
We’ll also see occasional subscription products, but only when it can be done in a way that is comfortably gross margin profitable for most users of a company. (Eg ChatGPT Plus, Claude Pro, Midjourney, 365 Copilot)
This will only change when the cost of inference goes down by a lot.
Meh, the price if MidJourney is like 2 orders of magnitude higher than their inference costs. The pricing is based on what they can get away with.
MidJourney claims 3.3 hours of GPU for $10. If that's 3.3 hours of A100, at a rate of $2/hour that's $6.6 of GPU costs.
My original statement was based on MJ feeling very expensive per image compared to the huge number of images I can generate in an hour with SDXL on my 3090 (and 3090s can be rented for $0.20 an hour).
But I forgot how overpriced A100s are (I doubt MJ is running 3090s but that'd be pretty cool), and that MJ is probably 4x the size of SDXL (although surely more optimized).
My revised statement is that for their $10 plan there's a few bucks of GPU compute cost. Probably between 2x and 5x margins.
For the subscription to even use PS, and then $ for using this or that feature.
Although if Adobe did charge “per use” for locally run tools like brushes and filters… I’d hope they’d at least they’d make the billing completely “usage based” with no minimum monthly cost.
But this is insane and evil, I’ll stick with Gimp and Inkscape
If so, even as a private individual just fooling around, I'll start using it from both legal and ethical perspective as long as it's reasonably equivalent to other models. And this from a person who's been fairly vocal against adobe's cloud subscription model ;-<. I can only imagine for anybody with a commercial need it would be an immediate no brain er - they'll have an established relationship, account and billing, they'll perceive it as integrating in their work flow, and it'll just become another part of the pipeline.
https://blog.adobe.com/en/publish/2023/03/21/responsible-inn...
If a photographer licenses a photo to Adobe Stock, they get paid every time someone pays to use the photo, right?
But if Adobe trained their AI on photos you had licensed to Adobe Stock. Do you get compensated at all?
If not, it’s not really different from what everyone else was doing in terms of ethics.
In its current state, at least, Photoshop's generative AI requires a lot of iterations to tweak the image into what I need (due to them requiring technical accuracy). Charging credits for this would make it nonviable.
https://mastodon.social/@UP8/110607460518784045
when I put the corrected one on the wall together with a lot of prints it was not properly centered vertically (had to center the painting + shadow not just the painting) so I had generative fill draw another row of bricks at the bottom. I’m sure they don’t look exactly like the the corresponding bricks on the real wall but they are good enough for wall art
It adds up to another reason to keep my Creative Cloud subscription instead of looking for an alternative.
The thing is if you want to have it add a spaceship or a pretty girl or something like that to the photo it has to make more background to go with whatever you tell it it draw so it automatically draws something consistent with the scene without a prompt.
There is a whole research area about tools that do "editing" so you could tell it to do something specific to an image, say "increase the exposure by 0.2 stops" (not that you need a.i. for that but somebody might rather do that than find the function in the UI) or "add a fourth motorcycle to the three motorcycles that are parked in a row" and that's even being done for 3-d models with NeRFs and point clouds and such but that's not what generative fill does right now.
I mostly get nice fill results as well, but they are super aggressive about removing "unsafe results", even when I don't give it a prompt? I'm not filling in nudity or anything, so it's not something I'd pay extra for in this state, I don't need an AI nanny.
Mine's only a 1060 though, so it does take minutes for a generation, whereas the newer cards can do it an order of magnitude faster.
Of course, I wouldn't mind it not using credits if I could use it locally either. Hopefully they continue to improve SD's IP Adapter type stuff so it's as good as Photoshop in that regard.
I noticed the same thing with SDXL, DALL-E, Midjourney etc. It makes 100% sense from a business / tool standpoint but I miss the weird raw-internet vibe of Stable Diffusion <2 and other early versions of these tools. Could totally be my limited testing / imagination too.
In any case it seems to be a capable illustrator, but I'm not surprised they're setting up a credit system. They must have put an insane amount of work to be first to ship on a big AI image editing suite.
You certainly don’t want to use the word ‘realistic,’ because real photos aren’t described that way. Just do “photo of blah blah,” or “blah blah, photo.”
It isn't artificial, you are consuming resources and have to pay for it. You are not renting hardware for full month but buying some time share. It is similar to electricity from grid vs solar panels.
>> *For standard images of up to 2000 x 2000 pixels.
>> We plan to offer higher-resolution images, ..., in the future. The number of generative credits consumed for those features may be greater.
When you think that a pretty common DSLR or mirrorless camera these days shoots a 40-50mp image, a 2000 x 2000 pixel baseline for 1 credit is pretty small.
This kind of reminds me of where we were with samplers in the late 80s, which makes me assume that we geezers will continue to complain and the new Marley Marl will appear and use the tech for something none of us can imagine right now.
The power of what Adobe is offering is generative AI in the context of proprietary photoshop tools.
Can you feed a prompt to a model outside of photoshop and ask for a particular result? Of course. But if your workflow already happens in an Adobe product and you want a highly specific and predictable result from Ai trained on a highly specific action, that’s something that you really can’t efficiently replicate elsewhere.
From this video it looks like functionality and results are similar to using the Adobe AI.
https://www.reddit.com/r/StableDiffusion/comments/11iuqhv/ma...
Companies should also be fined heavily for charging 4 coins for an item, but only selling the coins in increments of 17. It's an obvious scam.
IMHO PayPal dollars are the worst because they trick you into believing you hold currency when you don’t. PayPal may, at their discretion, allow you to convert your PayPal dollar to another currency. They also may take it away from you for no reason, without recourse - something that happens to thousands of people daily.
It has prices such as 0.000001$ per request (which is evil) and virtual currencies like Load Balancer Capacity Units (LCU).
Circle CI has points too.
Basically, everything so you would not figure out the total price easily.
That all being said, I would celebrate the demise of loyalty points, gift cards, Robux, Chuck E. Cheese tokens, and so on. All of it is fundamentally anti-consumer. I'm not sure when countries outlawed paying your employees in scrip why they didn't go one step further since scrip is intrinsically anti-competitive.
I think you can steelman some good arguments for scrip though... both in terms of "why venerate fiat state-backed currency as having a special status" and "some people actually LIKE the inflexibility or having to spend their money on a certain companies products, such as compulsive misers"
They serve a couple useful purposes though, even from a consumer's perspective (unless we fix enough other broken things about our payment infrastructure). There's an inherent conflict between convenience, security, transaction fees, and the chance of not being paid (or for online services like this, the guarantee that somebody orchestrates millions of accounts to distribute a compute load while fully expecting to never pay a dime) when dealing with very frequent or very small transactions.
Other options Adobe might have considered include:
1. Don't offer the feature (if the feature is good enough to tolerate company currency then this seems like a worse option)
2. Pay per use (high transaction costs, high friction if you don't store payment details, low security if you do, risk of default if you batch uses over time, ...)
3. Monthly subscription giving unlimited uses (not tenable for AI that has very real compute costs associated with it, you'll have users abuse the limits)
4. Monthly subscription with a max number of uses (just a different way to pay for credits, strictly worse than buying credits and having them last indefinitely)
5. Batch uses over time, bill if they exceed a threshold (lowers transaction fees, increases security, increases consumer goodwill, vulnerable to targeted attacks)
And so on. Do they actually have any good options, or are they choosing between a bunch of least-bad solutions? I ask partly because I was planning to use a similar model of charging up-front for credits for an unrelated service because I thought it would be the most consumer-friendly in that domain.