Thanks! Can't wait to get a minute to check this out. Promising 2x performance over pytorch+xformers sounds too good to be true for the same card.
As someone with a lowly 10gb card sdxl is beyond my reach with a1111 it seems. It functions well enough in comfyui but I can't make anything but garbage with it in automatic. So I'm happy to see 1.5 gets a big boost, I know there's a million of us out there who can't quite squeeze SDXL out so the maturing of the "legacy" versions is a positive note to see.
I haven't been paying attention for a few months, is there a lot of development happening around SDXL or are people still mostly focused on 1.5? My expectation was that SDXL would probably not see wide adoption for a while because it's hard to make up for the worse training data, is that bearing out?
> I haven't been paying attention for a few months, is there a lot of development happening around SDXL or are people still mostly focused on 1.5?
There's been a lot happening on SDXL, yes.
> My expectation was that SDXL would probably not see wide adoption for a while because it's hard to make up for the worse training data, is that bearing out?
Nice! Care to share any projects/advancements/gens you've found particularly compelling if you have anything handy?
> What worse training data?
I don't know this, but I think it's a pretty safe assumption given the state of 2.0/the political climate around this stuff, along with the comparative lower diversity of gens I've seen come out of SDXL. I haven't seen anything to the contrary but I'd love to be wrong.
--no-half-vae, IIRC, is required for the stock sdxl vae model because otherwise it frequently produces errors. I believe there’s a half-precision fixed VAE floating around and baked into many SDXL-derived checkpoints that avoids this problem.
> As someone with a lowly 10gb card sdxl is beyond my reach with a1111 it seems
SDXL reportedly works at below 8GiB in A1111 with --low-vram, and at 8GiB to anything short of 16GiB with --med-vram-sdxl, and 16GiB and up with no options.
> but I can't make anything but garbage with it in automatic.
If you aren't getting out of memory errors with SDXL but the output sucks, its probably not a VRAM problem. The most likely thing, IME, is using the SDXL base model or another checkpoint without a baked VAE but still having A1111 configured to use an SD1.x VAE (or using one with a baked VAE but having A1111 configured to override it.) SDXL needs an SDXL-specific VAE, but A1111 doesn't bundle one, IIRC, you need to download it separately.
LoRA are explicitly supported, though you have to convert them, as well as the base model.
This seems likely to be more useful for people with a "production" workflow with particular checkpoint, LoRA, & resolution combo than what I get the impression is the more common hobbyist situation of having a decently diverse collection of checkpoints and LoRA and fairly freely mixing and matching.
SD is trained on static images and don't have a clue about motion. Interpolation won't help with that. See deforum videos. They don't make any sense. Fast generation will help iterate faster though.
For developers, TensorRT support also exists for the diffusers library via community pipelines. [1] It's limited, but if you're only supporting a subset of features, it can help.
In general, these insane speed boosts come at the cost of bleeding edge features.
28 comments
[ 3.4 ms ] story [ 75.7 ms ] threadAs someone with a lowly 10gb card sdxl is beyond my reach with a1111 it seems. It functions well enough in comfyui but I can't make anything but garbage with it in automatic. So I'm happy to see 1.5 gets a big boost, I know there's a million of us out there who can't quite squeeze SDXL out so the maturing of the "legacy" versions is a positive note to see.
Where do you read that SDXL has worse training data? They didn't disclose what the model was trained on like the previous models.
There's been a lot happening on SDXL, yes.
> My expectation was that SDXL would probably not see wide adoption for a while because it's hard to make up for the worse training data, is that bearing out?
What worse training data?
Nice! Care to share any projects/advancements/gens you've found particularly compelling if you have anything handy?
> What worse training data?
I don't know this, but I think it's a pretty safe assumption given the state of 2.0/the political climate around this stuff, along with the comparative lower diversity of gens I've seen come out of SDXL. I haven't seen anything to the contrary but I'd love to be wrong.
> --medvram-sdxl --no-half-vae
SDXL reportedly works at below 8GiB in A1111 with --low-vram, and at 8GiB to anything short of 16GiB with --med-vram-sdxl, and 16GiB and up with no options.
> but I can't make anything but garbage with it in automatic.
If you aren't getting out of memory errors with SDXL but the output sucks, its probably not a VRAM problem. The most likely thing, IME, is using the SDXL base model or another checkpoint without a baked VAE but still having A1111 configured to use an SD1.x VAE (or using one with a baked VAE but having A1111 configured to override it.) SDXL needs an SDXL-specific VAE, but A1111 doesn't bundle one, IIRC, you need to download it separately.
- You have to "pre-bake" each model (create optimized engines) for only a pre-defined set of resolutions
- ~~No LoRA support~~ I guess there is a LoRA support, but you also will need to convert them to TRT
It's been like this for quite a few months now, not sure what's new this time.
This seems likely to be more useful for people with a "production" workflow with particular checkpoint, LoRA, & resolution combo than what I get the impression is the more common hobbyist situation of having a decently diverse collection of checkpoints and LoRA and fairly freely mixing and matching.
On a 2060S, no launch arguments, going from ~4 it/s to ~7.5 it/s.
In general, these insane speed boosts come at the cost of bleeding edge features.
[1] https://github.com/huggingface/diffusers/blob/28e8d1f6ec82a6...