I believe the branch which adds support for Apple Silicon also adds support for running on Intel chips (albeit extremely slowly). I haven't tested it myself, but I've seen several people in the GitHub issues saying this.
The standard release works fine, if you tweak the code to use the CPU pytorch device instead of CUDA. It does take about an hour to generate a set of images with the standard options on my AMD 2600 CPU though!
Comment from github: "By the way, i confirmed to work on my Intel 16-in MacBook Pro via mps. GPU (Radeon Pro 5500M 8GB) usage is 70-80% and It takes 3 min where --n_samples 1 --n_iter 1. My repo https://github.com/cruller0704/stable-diffusion-intel-mac"
For comparison, my RTX 2070 takes 10 seconds for one image (512x512)
Broadly speaking, it looks like they are. The implementation of Stable Diffusion doesn't appear to be using all of those features correctly (i.e. device selection fails if you don't have CUDA enabled even though MPS (https://pytorch.org/docs/stable/notes/mps.html) is supported by PyTorch.
Similar goes for quirks of Tensorflow that weren't taken advantage of. That's largely the work that is on-going in the OSX and M1 forks.
Ah, I didn't realize. It's not very obvious what GPU you have in your Macbook, I couldn't actually find out where to find that in my System settings. On Windows it's inside the "Display" settings but on MacOS... where is it? :)
If you look at the substance of the changes being made to support Apple Silicon, they're essentially detecting an M* mac and switching to PyTorch's Metal backend.
So, yeah PyTorch is correctly serving as a 'glue'.
As mentioned in sibling comments, Torch is indeed the glue in this implementation. Other glues are TVM[0] and ONNX[1]
These just cover the neural net though, and there is lots of surrounding code and pre-/post-processing that isn't covered by these systems.
For models on Replicate, we use Docker, packaged with Cog for this stuff.[2] Unfortunately Docker doesn't run natively on Mac, so if we want to use the Mac's GPU, we can't use Docker.
I wish there was a good container system for Mac. Even better if it were something that spanned both Mac and Linux. (Not as far-fetched as it seems... I used to work at Docker and spent a bit of time looking into this...)
I mean, I'm most impressed with the gradual rise of 3d printers capable of printing entire houses.
That doesn't help with plumbing, electrical, kitchen cabinets and all the other stuff that goes into a house -- which is a majority of the cost -- but it's a gradual start.
I see very little which can't be automated, but I see a lot which would take many years of time and effort to automate and integrate.
For reference, inferencing the model on a 2070 takes 10-12 seconds for the same size at max-precision, and the 3070 can synthesize an image in almost 6 seconds.
If you extrapolate the power consumption (3070 @~300w vs M1 Pro GPU@~30-50w) the metrics make a lot of sense.
You'll still need to play with modifying some of the code to get it to run, but `dream.py` works for me. Funny enough, I got only img2img effectively working with the lstein branch; it broke txt2img for me.
Yesterday I thought I broke it too. In my case, the solution was just to make sure that the input image (from the editor or otherwise) was the same size as the output image. Hope it helps.
Depends what fork you're running... Some seem to be using CPU-based generation, others use the MPS device backend correctly which is MUCH faster. I have another comment floating around about lstein's fork, but it takes some massaging to get it to run happily. https://github.com/lstein/stable-diffusion/
I'm on a iMac M1 16gb and I can handle up to 768x768 but since it's shared memory I close out every other application and run things overnight.
The biggest issue with apple chips is that the --seed setting doesn't work. I should be able to set a seed to, for instance, 1083958 and if I re-run a command at the same resolution with that seed, I should get the same image every time. This would allow me to test different steps so I could generate a 100 images at 16 steps (which is quite fast) and pick the ones that are most promising and re-render at 64 or 128 steps.
But currently you can't do that on apple hardware because of an open issue in PyTorch. Genuinely hoping a fix comes soon, until it is this is more of a novelty than a tool on Apple hardware.
Me at the end of last year: "Should I really go for the full 64GB on this M1 Pro? What could I possibly use this for? mmbml mumble... something about unified GPU... something Deep Learning, one day..."
Me now: "a red juicy apple floating in outer space, like a planet" --H 768 --W 768
I was so geniunely happy with my 16GB iMac M1 until stable-diffusion released.
I was amazed at how fast and powerful it was. I thought this meant I could stop buying top-of-the-line Macs every 4 years and start buying bottom-of-the-line Macs every 5 years. And that would have been 100% true... if it weren't for stable-diffusion.
Is there a good set of benchmarks available for Stable Diffusion? I was able to run a custom Stable Diffusion build on a GCE A100 instance (~$1/hour) at around 1Mpix per 10 seconds. I.e, I could create a 512x512 image in 2.5 seconds with some batching optimizations. A consumer GPU like a 3090 runs at ~1Mpix per 20 seconds.
I'm wondering what the price floor of stock art will be when someone can use https://lexica.art/ as a starting point, generate variations of a prompt locally, and then spend a few minutes sifting through the results. It should be possible to get most stock art or concept art at a price of <$1 per image.
So you’re estimating over a thousand generated images an hour and less than a tenth of a cent per image using the A100. If that turns out to be accurate, it seems like some online image generation will included in the price of the stock art.
(DreamStudio is charging a bit over one cent per generated image at default settings, depending on exchange rates.)
Related: I wrote up instructions for running Stable Diffusion on GCE. I used a Tesla T4, which is probably the cheapest that can handle the original code. If you're spinning up an instance to play with, rather than to batch-process, then cheaper makes more sense because most of the machine's time is spent waiting for you to type stuff and look at the results.
Midjourney, in case you appreciate their output, has an unlimited plan for 30$ a month. The only limitation is that if you're an extremely heavy user, they may "relax" you, which means results come in a bit slower.
Note that they've been also experimenting with a --beta parameter which basically means the algorithm uses StableDiffusion's algorithm behind the scenes, or you can use any of 4 versions of MidJourney's more stylistic algorithms.
So if you don't want to tinker or don't have a high-end GPU, it's a cheap way to play around. I have StableDiffusion running locally but still prefer MidJourney. I enjoy the stylistic output but it's also a highly social way to generate art. Everybody is doing it in the open.
Anyway, the stock art part is a hairy subject. You should assume that you AI image is not copyrighted. Which begs the question why they would pay at all.
>The only limitation is that if you're an extremely heavy user, they may "relax" you, which means results come in a bit slower.
You don't have to be an extremely heavy user. I used it for about an hour every evening and it took 11 days out of a month subscription for them to put me on relax mode.
The relax mode is based on how busy the service is. If usage is low, it's the same a fast mode. But other times its really slow.
That makes it unpredictable enough that it stopped being fun for me to use it. I've barely used midjourney since I got put on relaxed mode - it stopped feeling like I can jump on and play because I might hit a busy period and then it'll take 5 minutes to generate a prompt
That said, I could buy more hours of fast mode and I think it's still way cheaper than Dall-E or Dreamstudio
Between this and efforts to add 3D dimension to 2D images, I don’t see much of a future for digital multimedia creator jobs.
Even TikTok could be an endless stream of ML models.
Fears of a tech dystopia may be overblown; the masses will just shut off their gadgets and live simpler if labor markets implode within the traditional political correct economic system we have.
Open source AI is on the verge of upending the software industry and copyright. I dig it.
I haven't installed from this link specifically, but I used one of the branches on which this is based a few days ago, so the results should be similar.
On a first-gen M1 Mac mini with 8GB RAM, it takes 70-90 minutes for each image.
I have the M1 8GB I mentioned in my first comment, and the M1 Pro 16GB I mentioned in my second component, side-by-side. However, the first one was running a Stable Diffusion branch from earlier in the week, so I replaced using the same instructions. The only difference now is the physical hardware.
The thing to understand is that the 8GB M1 has 8GB. When I run txt2img.py, my Activity Monitor shows a Python process with 9.42GB of memory, and the "Memory Pressure" graph spends time in the red zone as the machine is swapping. While the 16GB M1 Pro immediately shows PLMS Sampler progress, and consistently spends around 3 seconds per iteration (e.g. "3.29s/it" and "2.97s/it"), the 8GB M1 takes several minutes before it jumps from 0% to 2% progress, and it accurately reports "326.24s/it"
So yes, whether it's M1 vs M1 Pro, or 8GB vs 16GB, it really is that stark a difference.
Update: after the second iteration it is 208.44s/it, so it is speeding up. It should drop to less than 120s/it before it finishes, if it runs as quickly as my previous install. And yes, 186.04s/it after the third iteration, and 159.22s/it after the fourth.
Sounds entirely like a swap-constrained operation. You need ~8gb of VRAM to load the uncompressed model into memory, which obviously won't work well on a Macbook with 8gb of memory.
My first-gen M1 MacBook Air with 16GB takes just under 4 minutes per image. Running top while it's generating shows memory usage fluctuating between 10GB and 13GB, so if you're running on 8GB it's probably swapping a lot.
You clearly doing something wrong as I get about 3 minutes per image on m1 mac mini.
But yeah, at this stage most of guides are early hacks and require individual tweaking. It is quite expected that people get varying results. I assume in a week or a month situation will get much better and much more user-friendly.
Thanks for writing this up!! I enjoyed getting TensorFlow running with the M1, although a multi-headed model I was working on wouldn’t run.
I just made my Dad’s 101 year old birthday card using OpenAI’s image generating service (he loved it) and when I get home from travel I will use your instructions in the linked article.
Any advice for running Stable Diffusion locally vs. Colab Pro or Pro+? My M1 MacBook Pro only has 8G ram (I didn’t want to wait a month for a 16G model). Is that enough? I have a 1080 with 10G graphics memory. Is that sufficient?
I've been playing with Stable Diffusion a lot the past few days on a Dell R620 CPU (24 cores, 96 GB of RAM). With a little fiddling (not knowing any python or anything about machine learning) I was able to get img2img.py working by simply comparing that script to the txt2img.py CPU patch. Was only a few lines of tweaking. img2img takes ~2 minutes to generate an image with 1 sample and 50 iterations, txt2img takes about 10 minutes for 1 sample and 50 generations.
The real bummer is that I can only get ddim and plms to run using a CPU. All of the other diffusions crash and burn. ddim and plms don't seem to do a great job of converging for hyper-realistic scenes involving humans. I've seen other algorithms "shape up" after 10 or so iterations from explorations people do online - where increasing the step count just gives you a higher fidelity and/or more realistic image. With ddim/plms on a CPU, every step seems to give me a wildly different image. You wouldn't know that steps 10 and steps 15 came from the same seed/sample they change so much.
I'm not sure if this is just because I'm running it on a CPU or if ddim and plms are just inferior to the other diffusion models - but I've mostly given up on generating anything worthwhile until I can get my hands on an nvida GPU and experiment more with faster turn arounds.
> You wouldn't know that steps 10 and steps 15 came from the same seed/sample they change so much.
I don't think this is CPU specific, this happens at these very low number of samples, even on the GPU. Most guides recommend starting with 45 steps as a useful minimum for quickly trialing prompt and setting changes, and then increasing that number once you've found values you like for your prompt and other parameters.
I've also noticed another big change sometimes happens between 70-90 steps. It's not all the time and it doesn't drastically change your image, but orientations may get rotated, colors will change, the background may change completely.
> img2img takes ~2 minutes to generate an image with 1 sample and 50 iterations
If you check the console logs you'll notice img2img doesn't actually run the real number of steps. It's number of steps multiplied by the Denoising Strength factor. So with a denoising strength of 0.5 and 50 steps, you're actually running 25 steps.
Later edit: Oh and if you do end up liking an image from step 10 or whatever, but iterating further completely changes the image, one thing you can do is save your output at 10 steps, and use that as your base image for the img2img script to do further work.
With the 1.4 checkpoint, everything under 40 steps can't be used basically and you only get good fidelity with >75 steps. I usually use 100, that's a good middleground.
I've tried using this set of steps [1], but have so far not had luck, mostly because the ROCm driver setup is throwing me for a loop. Tried it with an RX 6700 XT and first was going to test on Ubuntu 22.04 but realized ROCm doesn't support that OS yet, so tried again on 20.04 and ended up breaking my GPU driver!
AMD market segmented their RDNA2 support in ROCm to the Navi21 set only (6800/6800 XT/6900 XT).
It is not officially supported in any way on other RDNA2 GPUs. (Or even on the desktop RDNA2 range at all, that only works because their top end Pro cards share the same die)
6600XT reporting in. Spent a few hours on Windows and WSL2 setup attempts, got no where. I don't run Ubuntu at home and don't want to dual boot just for this. From looking around I think I'd have a better chance on native Ubuntu.
Or wait, if its just about stable diffusion multiple people try to create onnx and directml forks of the models/scripts, which atleast in theory can work for AMD gpus in windows and wsl2
I have it working on an RX 6800, used the scripts from this repo[0] to build a docker image that has ROCm drivers and PyTorch installed.
I'm running Ubuntu 22.04 LTS as the host OS, didn't have to touch anything beyond the basic Docker install. Next step is build a new Dockerfile that adds in the Stable Diffusion WebUI.[1]
I tried getting pytorch vulkan inference working with radv, it gives me a missing dtype error in vkformat. Fp16 or normal precision have the same error. I think it's some bf16 thing.
If you consider that the training set includes all western art from the last few centuries it’s not too surprising. There’s an awful lot of nudes in that set & most of them are female.
If you open up the script txt2img and img2img scripts, there is a content filter. If your prompt generated anything that gets detected as "inappropriate" the image is replaced with Rick Astley.
Removing the censor should be pretty straightforward, just comment out those lines.
It's kind of amazing that ML can now intelligently rick roll people.
I think it would be awesome to update the rickroll feature to the following:
Auto Re-run the img2img with some text prompt: "all of the people are now Rick Astley" with low strength so it can adjust the faces, but not change the nudity!!!1
Probably another week or two. running on M1 iPad Pro needs to get out of PyTorch, possibly export the model either through TorchScript and then do onnx conversion. From what I found so far, not many of these conversions done (except the OpenVINO one maybe?).
I am having the same issue on MacOS 12.2.1 (21D62); Python 3.10.6
What did you upgrade to solve this? Thanks!
(I can get it working with `--precision full`)
Also, of course, it's entirely free. The web version is actually paid, though it's hard to tell because they're not super transparent about the fact that you're steadily eating through a quota of initial tokens.
EDIT 2: After playing around with this repo, I've found:
- It offers better UX for interacting with Stable Diffusion, and seems to be a promising project.
- Running txt2img.py from lstein's repo seems to run about 30% faster than OP's. Not sure if that's a coincidence, or if they've included extra optimisations.
- I couldn't get the web UI to work. It kept throwing the "leaked semaphor objects" error someone else reported (even when rendering at 64x64).
- Sometimes it rendered images just as a black canvas, other times it worked. This is apparently a known issue and a fix is being tested.
I've reached the limits of my knowledge on this, but will following closely as new PRs are merged in over the coming days. Exciting!
FYI: black images are not just from the safety checker.
Yes, the safety checker will zero out images but can just turn it off with an “if False:”; Mostly black images are due to a bug, especially frustrating because it turns up on high step counts and means you’ve wasted time running it.
My experience has been roughly 2-4/32 of an image batch comes back black at the default settings, regardless of the prompt.
Just stamp out images in batches and discard the black ones.
You have to download the model from the huggingface[0] site first (requires a free account). The exact steps on how to link the file are then detailed here[1].
I did this but then moved the directory. When re-linking and checking with ls for the path I thought "oh, alright, it's already there". Oh well, better check with ls -l earlier next time.
I had to manually install pytorch for the preload_models.py step to work, because ReduceOp wasn't found. Why even use anaconda if all the dependencies aren't included? Every time I touch an ML project, there's always a python dependency issue. How can people use a tool that's impossible to provide a consistent environment for?
You are completely correct that there are a lot of dependency bugs here, I would just like to pedantically complain that the issue in question is PyTorch supporting MPS, which is basically entirely a C++ dependency issue rather than a Python one. (PyTorch being mostly written in C++ despite having "py" in the name.) And yeah the state of C++ dependency management is pretty bad.
To get past `pip install -r requirements` I had to muck around with CFLAGS/LDFLAGS because I guess maybe on your system /opt/homebrew/opt/openssl is a symlink to something? On mine it doesn't exist, I just have /opt/homebrew/opt/openssl@1.1 symlinked to /opt/Cellar/somewhere.
Check my comment alongside yours, I got Conda to work but it did require the pre-requisite Homebrew packages you originally recommended before it would cooperate :)
I couldn't get the setup process working until I switched the python distro to 3.10, as the scripts were relying on typings features that were added in 3.10 even though the yml file specified 3.9. Was strange.
I was following the github issue and the CPU bound one was at 4-5 minutes, the MDS one was at 30 seconds, then 18 seconds, and people were still calling that slow.
What is it currently at now?
and I don't know what "fast" is, to compare
What are the Windows 10 with nice Nvidia chips w/ CUDA getting? Just curious whats comprehensive
I’m referring to there being a community effort to normalize performance metrics and results at all, with the M1 devices being in that list as well, so that we dont have to ask these questions to begin with
I only run 1 sample at a time (batch size 1), forgot to mention that, and that affects the step time.
It looks like each additional image in a batch is cheaper than the 1st image. For example if I reduce my resolution so I can generate more in a single batch
1 image, 50 steps, 320x320: 5s
2 images, 50 steps, 320x320: 8s
3 images, 50 steps, 320x320: 11s
4 images, 50 steps, 320x320: 14s
And the trend continues, and my reported iteration/sec goes down as well. It's not accounting for the fact that with steps=50 and batch size=4 it's actually running 200 steps, just in 4 parallel parts.
I just commented on another sibling comment (too late to edit the first one), but I forgot to mention my batch size is only 1. I think most people use batch size 4, so basically multiply my time by your batch size for a real comparison.
It was my bad, my script was still running a different fork. Seeing <10 second times with those parameters now. 13.6 seconds for an 3072 × 2048 upscaled image, which I'm particularly happy about.
It's falling back to CPU. Follow the instructions to use a GPU version - sometimes it's even a completely different repo, depending on whose instructions you're following.
I changed the dependency to 3.10.4 (tried 3.10.6 as well), installed python 3.10.4, deactivated and activated ldm environment, but it still uses python 3.9
Since I don't know how to use conda, I had to struggle a bit to learn how to recreate the environment.
Here's the commands that worked me for future reference:
I'm working on getting this running. Instead of "venv/bin/activate" I had to run "source venv/bin/activate". And I got an error installing the requirements, fixed by running "pip install pyyaml" as a separate command.
Great. FYI, the issue with pyyaml was caused by the "--pre" flag. The reason installing it separately fixes the problem is because it installs it without the "--pre" flag, and then it is already installed when you install the requirements.
I haven't been able to make sense of n-samples and n-iters. Changing the former caused generation to freeze at 0%. Changing the latter seems to generate multiple images, even though the SD docs for n-iters are "sample this often".
412 comments
[ 3.2 ms ] story [ 285 ms ] threadFor comparison, my RTX 2070 takes 10 seconds for one image (512x512)
I mean, are we going to see X on M1 Mac, for any X now in the future?
Also, weren't torch and tensorflow supposed to be this glue?
Similar goes for quirks of Tensorflow that weren't taken advantage of. That's largely the work that is on-going in the OSX and M1 forks.
So, yeah PyTorch is correctly serving as a 'glue'.
https://github.com/CompVis/stable-diffusion/commit/0763d366e...
These just cover the neural net though, and there is lots of surrounding code and pre-/post-processing that isn't covered by these systems.
For models on Replicate, we use Docker, packaged with Cog for this stuff.[2] Unfortunately Docker doesn't run natively on Mac, so if we want to use the Mac's GPU, we can't use Docker.
I wish there was a good container system for Mac. Even better if it were something that spanned both Mac and Linux. (Not as far-fetched as it seems... I used to work at Docker and spent a bit of time looking into this...)
[0] https://tvm.apache.org/ [1] https://onnx.ai/ [2] https://github.com/replicate/cog
We will still need people to do the hard yards, and get dirt between their fingernails. I am firmly in the camp of those people.
Fancy algorithms won't dig holes, or lay out rail tracks of over hundreds of miles.. or build houses all across the world.
Why is nobody impressed by the future?
[1] https://en.wikipedia.org/wiki/Loitering_munition
That doesn't help with plumbing, electrical, kitchen cabinets and all the other stuff that goes into a house -- which is a majority of the cost -- but it's a gradual start.
I see very little which can't be automated, but I see a lot which would take many years of time and effort to automate and integrate.
If you extrapolate the power consumption (3070 @~300w vs M1 Pro GPU@~30-50w) the metrics make a lot of sense.
My M1 with 8GB takes 70-90 minutes per image.
My M1 Pro with 16GB takes 3 minutes per image.
It's been a lot of fun to play with so far though!
You'll still need to play with modifying some of the code to get it to run, but `dream.py` works for me. Funny enough, I got only img2img effectively working with the lstein branch; it broke txt2img for me.
EDIT: Speed increased to 2.3s/iter after a reboot
It seems the GPU memory requirements beyond 512x512 are obscene.
The biggest issue with apple chips is that the --seed setting doesn't work. I should be able to set a seed to, for instance, 1083958 and if I re-run a command at the same resolution with that seed, I should get the same image every time. This would allow me to test different steps so I could generate a 100 images at 16 steps (which is quite fast) and pick the ones that are most promising and re-render at 64 or 128 steps.
But currently you can't do that on apple hardware because of an open issue in PyTorch. Genuinely hoping a fix comes soon, until it is this is more of a novelty than a tool on Apple hardware.
Me now: "a red juicy apple floating in outer space, like a planet" --H 768 --W 768
Uses about 27GB. 1.81s/it.
Can't do 1024x1024 yet because of some hardcoded Metal issue (https://github.com/pytorch/pytorch/issues/84039.
I was amazed at how fast and powerful it was. I thought this meant I could stop buying top-of-the-line Macs every 4 years and start buying bottom-of-the-line Macs every 5 years. And that would have been 100% true... if it weren't for stable-diffusion.
Use external upscalers like RealESRGAN, SwinIR or BSRGAN or GFPGAN (faces).
Alternatively use hacks like txt2imghd to get it to natively create 1 MP images.
I'm wondering what the price floor of stock art will be when someone can use https://lexica.art/ as a starting point, generate variations of a prompt locally, and then spend a few minutes sifting through the results. It should be possible to get most stock art or concept art at a price of <$1 per image.
(DreamStudio is charging a bit over one cent per generated image at default settings, depending on exchange rates.)
https://sowbug.com/posts/stable-diffusion-on-google-cloud/
Midjourney, in case you appreciate their output, has an unlimited plan for 30$ a month. The only limitation is that if you're an extremely heavy user, they may "relax" you, which means results come in a bit slower.
Note that they've been also experimenting with a --beta parameter which basically means the algorithm uses StableDiffusion's algorithm behind the scenes, or you can use any of 4 versions of MidJourney's more stylistic algorithms.
So if you don't want to tinker or don't have a high-end GPU, it's a cheap way to play around. I have StableDiffusion running locally but still prefer MidJourney. I enjoy the stylistic output but it's also a highly social way to generate art. Everybody is doing it in the open.
Anyway, the stock art part is a hairy subject. You should assume that you AI image is not copyrighted. Which begs the question why they would pay at all.
You don't have to be an extremely heavy user. I used it for about an hour every evening and it took 11 days out of a month subscription for them to put me on relax mode.
The relax mode is based on how busy the service is. If usage is low, it's the same a fast mode. But other times its really slow.
That makes it unpredictable enough that it stopped being fun for me to use it. I've barely used midjourney since I got put on relaxed mode - it stopped feeling like I can jump on and play because I might hit a busy period and then it'll take 5 minutes to generate a prompt
That said, I could buy more hours of fast mode and I think it's still way cheaper than Dall-E or Dreamstudio
Even TikTok could be an endless stream of ML models.
Fears of a tech dystopia may be overblown; the masses will just shut off their gadgets and live simpler if labor markets implode within the traditional political correct economic system we have.
Open source AI is on the verge of upending the software industry and copyright. I dig it.
> ERROR: Failed building wheel for onnx
I was able to resolve it by doing this:
> brew install protobuf
Then I ran pip install again, and it worked!
On a first-gen M1 Mac mini with 8GB RAM, it takes 70-90 minutes for each image.
Still feels like magic, but old-school magic.
Looks like RAM drastically affects the speed.
The thing to understand is that the 8GB M1 has 8GB. When I run txt2img.py, my Activity Monitor shows a Python process with 9.42GB of memory, and the "Memory Pressure" graph spends time in the red zone as the machine is swapping. While the 16GB M1 Pro immediately shows PLMS Sampler progress, and consistently spends around 3 seconds per iteration (e.g. "3.29s/it" and "2.97s/it"), the 8GB M1 takes several minutes before it jumps from 0% to 2% progress, and it accurately reports "326.24s/it"
So yes, whether it's M1 vs M1 Pro, or 8GB vs 16GB, it really is that stark a difference.
Update: after the second iteration it is 208.44s/it, so it is speeding up. It should drop to less than 120s/it before it finishes, if it runs as quickly as my previous install. And yes, 186.04s/it after the third iteration, and 159.22s/it after the fourth.
I had a YouTube video playing while I kicked off the exact command in the install docs, and got: 16.84s user 99.43s system 61% cpu 3:08.51 total
Next attempt, python aborted 78 seconds in! Weird.
Next attempt, with YouTube paused: 16.31s user 95.48s system 65% cpu 2:49.45 total
So around three minutes, I'd say.
It looks like memory is super-important for this (which isn't all that surprising, really...).
Apple M1 Max with 10-core CPU, 32-core GPU, 16-core Neural Engine - Takes 38 seconds as well up to 46 when it gets hotter.
Can anyone give comparison with Nvida gpu in terms of performance?
But yeah, at this stage most of guides are early hacks and require individual tweaking. It is quite expected that people get varying results. I assume in a week or a month situation will get much better and much more user-friendly.
> brew link protobuf --overwrite
Don't blindly run this command unless you understand what you're doing.
I just made my Dad’s 101 year old birthday card using OpenAI’s image generating service (he loved it) and when I get home from travel I will use your instructions in the linked article.
Any advice for running Stable Diffusion locally vs. Colab Pro or Pro+? My M1 MacBook Pro only has 8G ram (I didn’t want to wait a month for a 16G model). Is that enough? I have a 1080 with 10G graphics memory. Is that sufficient?
The real bummer is that I can only get ddim and plms to run using a CPU. All of the other diffusions crash and burn. ddim and plms don't seem to do a great job of converging for hyper-realistic scenes involving humans. I've seen other algorithms "shape up" after 10 or so iterations from explorations people do online - where increasing the step count just gives you a higher fidelity and/or more realistic image. With ddim/plms on a CPU, every step seems to give me a wildly different image. You wouldn't know that steps 10 and steps 15 came from the same seed/sample they change so much.
I'm not sure if this is just because I'm running it on a CPU or if ddim and plms are just inferior to the other diffusion models - but I've mostly given up on generating anything worthwhile until I can get my hands on an nvida GPU and experiment more with faster turn arounds.
I don't think this is CPU specific, this happens at these very low number of samples, even on the GPU. Most guides recommend starting with 45 steps as a useful minimum for quickly trialing prompt and setting changes, and then increasing that number once you've found values you like for your prompt and other parameters.
I've also noticed another big change sometimes happens between 70-90 steps. It's not all the time and it doesn't drastically change your image, but orientations may get rotated, colors will change, the background may change completely.
> img2img takes ~2 minutes to generate an image with 1 sample and 50 iterations
If you check the console logs you'll notice img2img doesn't actually run the real number of steps. It's number of steps multiplied by the Denoising Strength factor. So with a denoising strength of 0.5 and 50 steps, you're actually running 25 steps.
Later edit: Oh and if you do end up liking an image from step 10 or whatever, but iterating further completely changes the image, one thing you can do is save your output at 10 steps, and use that as your base image for the img2img script to do further work.
https://github.com/Birch-san/stable-diffusion/blob/birch-mps...
That branch (birch-mps-waifu) runs on M1 macs no problem.
ROCm support seems spotty at best, I have a 5700xt and I haven't had much luck getting it working.
[1] https://gist.github.com/geerlingguy/ff3c3cbcf4416be2c0c1e0f8...
AMD market segmented their RDNA2 support in ROCm to the Navi21 set only (6800/6800 XT/6900 XT).
It is not officially supported in any way on other RDNA2 GPUs. (Or even on the desktop RDNA2 range at all, that only works because their top end Pro cards share the same die)
HSA_OVERRIDE_GFX_VERSION=10.3.0 to force using the Navi21 binary slice.
This is totally unsupported and please don't complain if something doesn't work when using that trick.
But basic PyTorch use works using this, so you might get away with it for this scenario.
(TL;DR: AMD just doesn't care about GPGPU on the mainstream, better to switch to another GPU vendor that does next time...)
AMD just doesn't invest in their developer ecosystem. Also as you use a 6600 XT, no official ROCm support for the die that you use. Only for navi21.
I'm running Ubuntu 22.04 LTS as the host OS, didn't have to touch anything beyond the basic Docker install. Next step is build a new Dockerfile that adds in the Stable Diffusion WebUI.[1]
[0] https://github.com/AshleyYakeley/stable-diffusion-rocm [1] https://github.com/hlky/stable-diffusion-webui
How long does it take to do 50 iterations on a 512x512?
https://yulian.kuncheff.com/stable-diffusion-fedora-amd/
I did need to run the troubleshooting step too, could probably just move that up as a required step in the guide.
Any Python packaging experts know what's going on? all macOS 12, arm64, Python 3.10. Can't think it wouldn't resolve the wheel.
But yes, good idea to move up. I'll stick it next to the `pip install`.
Thank you!
EDIT: It was a false-positive (honest!) on the NSFW filter. To disable it, edit txt2img.py around line 325.
Comment this line out:
And replace it with:Change your prompt, or remove the filter from the code.
If you tweak the prompt to explicitly mention clothing, you should be OK though.
Removing the censor should be pretty straightforward, just comment out those lines.
I think it would be awesome to update the rickroll feature to the following:
Auto Re-run the img2img with some text prompt: "all of the people are now Rick Astley" with low strength so it can adjust the faces, but not change the nudity!!!1
AI alignment concerns are definitely overblown...
> expected scalar type BFloat16 but found Float
Has anyone seen this error? It's pretty hard to google for.
You can now run the Lstein fork[1] with M1 as of a few hours ago.
This adds a ton of functionality - GUI, Upscaling & Facial improvements, weighted subprompts etc.
This has been a big undertaking over the last few days, and I highly recommend checking it out. See the mac m1 readme [3]
[0] https://github.com/magnusviri/stable-diffusion
[1] https://github.com/lstein/stable-diffusion
[2] https://github.com/lstein/stable-diffusion/blob/main/README-...
EDIT: Got it working, with a couple of pre-requisite steps:
0. `rm` the existing `stable-diffusion` repo (assuming you followed OP's original setup)
1. Install `conda`, if you don't already have it:
2. Install the other build requirements referenced in OP's setup: 3. Follow the main installation instructions here: https://github.com/lstein/stable-diffusion/blob/main/README-...Then you should be good to go!
EDIT 2: After playing around with this repo, I've found:
- It offers better UX for interacting with Stable Diffusion, and seems to be a promising project.
- Running txt2img.py from lstein's repo seems to run about 30% faster than OP's. Not sure if that's a coincidence, or if they've included extra optimisations.
- I couldn't get the web UI to work. It kept throwing the "leaked semaphor objects" error someone else reported (even when rendering at 64x64).
- Sometimes it rendered images just as a black canvas, other times it worked. This is apparently a known issue and a fix is being tested.
I've reached the limits of my knowledge on this, but will following closely as new PRs are merged in over the coming days. Exciting!
--sampler k_euler
full command:
"photography of a cat on the moon" -s 20 -n 3 --sampler k_euler -W 384 -H 384
AttributeError: module 'torch._C' has no attribute '_cuda_resetPeakMemoryStats'
https://gist.github.com/JAStanton/73673d249927588c93ee530d08...
> User specified autocast device_type must be 'cuda' or 'cpu'
> Are you sure your system has an adequate NVIDIA GPU?
I found the solution here: https://github.com/lstein/stable-diffusion/issues/293#issuec...
Yes, the safety checker will zero out images but can just turn it off with an “if False:”; Mostly black images are due to a bug, especially frustrating because it turns up on high step counts and means you’ve wasted time running it.
My experience has been roughly 2-4/32 of an image batch comes back black at the default settings, regardless of the prompt.
Just stamp out images in batches and discard the black ones.
Going up to the parent repo points at a bunch of dead links or hugginface pages.
[0] https://huggingface.co/CompVis/stable-diffusion-v-1-4-origin... [1] https://github.com/lstein/stable-diffusion/blob/main/README-...
The command that finally worked for me:
We struggled to get Conda working reliably for people, which it looks like lstein's fork recommends. I'll see if we can get it working with plain pip.
Anyone got any ideas?
[0] https://github.com/bfirsh/stable-diffusion/blob/392cda328a69...
[1] https://gist.github.com/bfirsh/594c50fd9b2e6b173e31de753a842...
EDIT: https://github.com/lstein/stable-diffusion/issues/293#issuec... fixed it for me.
Requirements are "requirements-mac.txt" which'll need subbing in the guide.
We're testing this out with a few people in Discord before shipping to the blog post.
did you run
python scripts/preload_models.py
python scripts/dream.py --full_precision ?
I was following the github issue and the CPU bound one was at 4-5 minutes, the MDS one was at 30 seconds, then 18 seconds, and people were still calling that slow.
What is it currently at now?
and I don't know what "fast" is, to compare
What are the Windows 10 with nice Nvidia chips w/ CUDA getting? Just curious whats comprehensive
Are you referring to single iteration step times, or whole images? Because obviously it depends on the number of iteration steps used.
Windows 10, RTX 2070 (laptop model), lstein repo. I get about 3.2 iter/sec. A 50 step 512x512 image takes me 15 seconds.
Are you aware of any wiki or table like that?
It looks like each additional image in a batch is cheaper than the 1st image. For example if I reduce my resolution so I can generate more in a single batch
1 image, 50 steps, 320x320: 5s
2 images, 50 steps, 320x320: 8s
3 images, 50 steps, 320x320: 11s
4 images, 50 steps, 320x320: 14s
And the trend continues, and my reported iteration/sec goes down as well. It's not accounting for the fact that with steps=50 and batch size=4 it's actually running 200 steps, just in 4 parallel parts.
ImportError: cannot import name 'TypeAlias' from 'typing' (/opt/homebrew/Caskroom/miniconda/base/envs/ldm/lib/python3.9/typing.py)
stable-diffusion/src/k-diffusion/k_diffusion/sampling.py
(before)
(after) This issue is tracked in https://github.com/lstein/stable-diffusion/issues/302This worked afterwards: python scripts/inpaint.py --indir data/inpainting_examples/ --outdir outputs/inpainting_results
I haven't been able to make sense of n-samples and n-iters. Changing the former caused generation to freeze at 0%. Changing the latter seems to generate multiple images, even though the SD docs for n-iters are "sample this often".