> The pipeline (bottom) shows how diverse OpenImages inputs are edited
using Nano-Banana and quality-filtered by Gemini-2.5-Pro, with failed attempts automatically retried.
Pretty interesting. I run a fairly comprehensive image-comparison site for SOTA generative AI in text-to-image and editing. Managing it manually got pretty tiring, so a while back I put together a small program that takes a given starting prompt, a list of GenAI models, and a max number of retries which does something similar.
It generates and evaluates images using a separate multimodal AI, and then rewrites failed prompts automatically repeating up to a set limit.
It's not perfect (nine pointed star example in particular) - but often times the "recognition aspect of a multimodal model" is superior to its generative capabilities so you can run it in a sort of REPL until you get the desired outcome.
That's a great website! Feature request: a button to toggle all the sliders left or right at the same time - would make it easier to glance the results without lots of finicky mouse moves.
I confess that I don't quite get the point here - is it just that they've paid the inference costs for a dataset than can be used for distillation/other research?
The license is CC BY-NC-ND - I’m not sure who is going to be able to use it given the NC-ND part… especially given the potential uncertainty over what uses count as commercial and what counts as derivative works. OTOH, given the bulk of this dataset is AI outputs, its copyrightability is an open question.
Looks like the dataset is distilled from Gemini nano-banana
Definitely very useful, but I’m so curious how the original datasets from these image editing models were created. I’m guessing a lot of it is synthetic data to construct scenes programmatically with layers
My rough guess is that they set a few workflows combining analytical and ML-based image manipulations to generate the training set. For instance, you can get a long way by having a segmentation model identify and mask various objects and then apply simple analytical manipulations to the masked areas such as changing their color, or diffusing new content into that area using masked guidance to another image diffusion model. In this way, you can create training pairs that your editing model learns to invert, such as “turn the woman’s hair into blonde hair” (start with a blonde haired woman, mask the hair, and get a diffusion model to turn it brown; this gives you the scene you can now invert as a training pair).
All the READMEs these days are such a tell. It's okay when explicitly prompted, but now thanks to reinforcement learning through people who have no clue, all the models just top off every change with some pointless documentation change.
Valid question, as they already have a partnership with OpenAI to use ChatGPT in Siri. I personally use GPT for illustrations and Nano Banana for photo edits (Midjourney for realistic photos).
As an aside, perhaps they're using GPT/Codex for coding. Did anyone else notice the use of emojis and → in their code?
AI industry: please _please_ get it together with naming. There shouldn’t be this much overlap between this, a dataset, and a massive image model which was already given a garbage name to begin with.
Don’t get me started in how “agent” is a term of art that means absolutely nothing, encompassing everything from a plain old shell script to a full language model.
Image editing model training is fascinating. One method for training image editing models involves using a second model to apply the inverse of the change you want the model to learn. Typically, the task you’re asking the second model to perform is easy, whereas the inverse task is difficult.
For example, you might ask the second model to cover the person’s face with a black square; a VLM model notes that the person is a man with brown hair and round glasses. Then, during training, the resulting image is presented along with the prompt, “Remove the black square from the man’s face. He has brown hair and round glasses.”
The model now learns how to remove black squares and replace them with a man’s face with brown hair and round glasses.
Since the training data is easily synthesized using existing models, you can generate enormous amounts of it - often very cheaply. For specialized editing tasks, this technique is really powerful. Build your training set for your special purpose task, fine tune an existing image editing model such as Qwen Image Edit to produce a new checkpoint or LoRA (often a LoRA is more than good enough) and then you have a special purpose model to perform whatever narrow editing task you need it to perform on your image data.
27 comments
[ 3.3 ms ] story [ 49.4 ms ] thread> The pipeline (bottom) shows how diverse OpenImages inputs are edited using Nano-Banana and quality-filtered by Gemini-2.5-Pro, with failed attempts automatically retried.
Pretty interesting. I run a fairly comprehensive image-comparison site for SOTA generative AI in text-to-image and editing. Managing it manually got pretty tiring, so a while back I put together a small program that takes a given starting prompt, a list of GenAI models, and a max number of retries which does something similar.
It generates and evaluates images using a separate multimodal AI, and then rewrites failed prompts automatically repeating up to a set limit.
It's not perfect (nine pointed star example in particular) - but often times the "recognition aspect of a multimodal model" is superior to its generative capabilities so you can run it in a sort of REPL until you get the desired outcome.
https://genai-showdown.specr.net/image-editing
I'm happy to see something from Apple but this seems so low-tech that it could be one of my own local ComfyUI workflows.
Definitely very useful, but I’m so curious how the original datasets from these image editing models were created. I’m guessing a lot of it is synthetic data to construct scenes programmatically with layers
https://lmarena.ai/leaderboard/image-edit
As an aside, perhaps they're using GPT/Codex for coding. Did anyone else notice the use of emojis and → in their code?
https://genai-showdown.specr.net/image-editing
Don’t get me started in how “agent” is a term of art that means absolutely nothing, encompassing everything from a plain old shell script to a full language model.
For example, you might ask the second model to cover the person’s face with a black square; a VLM model notes that the person is a man with brown hair and round glasses. Then, during training, the resulting image is presented along with the prompt, “Remove the black square from the man’s face. He has brown hair and round glasses.”
The model now learns how to remove black squares and replace them with a man’s face with brown hair and round glasses.
Since the training data is easily synthesized using existing models, you can generate enormous amounts of it - often very cheaply. For specialized editing tasks, this technique is really powerful. Build your training set for your special purpose task, fine tune an existing image editing model such as Qwen Image Edit to produce a new checkpoint or LoRA (often a LoRA is more than good enough) and then you have a special purpose model to perform whatever narrow editing task you need it to perform on your image data.