I'm a bit surprised by the examples. The original captions contain extra context rather than descriptions of the images.
Perhaps that doesn't matter, as the abstract suggests that's not the purpose.
On the other hand, training on captions they generate that are also incorrect is problematic. Perhaps they should have cherry-picked better examples without errors, such as details about the sugar and cream, reflections of the train in the water, or the location of the watermark and sign. I didn't look at the code, but perhaps each image feature can be written with a confidence score.
Yes, the "Western Kingbird" is a good example of that. That's very useful context. I've not read the paper yet, but this could be trivially solved by just appending the extended description to the original one. You could ask the model if the original description seems accurate or not as well to weed out any duds (like the picture of a cake described as a 'twin room').
> The original captions contain extra context rather than descriptions of the images
Another good example on how accessibility can even help people without disabilities.
You can see this on ALT text on Mastodon, and I find it really useful.
You cannot ablate the lack of knowledge, the vision models and projection layers used in this project usually lack any knowledge about NSFW, the model is simply unaware of what it's going on if you give them a NSFW image.
You mentioned pornography as well. And when people use "offensive" in an absolute sense, that often is synonymous with "what the HR department wouldn't want you looking at on a work computer", i.e. NSFW.
I'm quite baffled by the fact that LLMs can generate a dataset used to train other LLMs. One would think that such a feedback loop would produce utter nonsense but apparently not. This seems to work.
I feel the same way about synthetic data. Seems intuitively wrong that you can get new insights / unlock new abilities from generated data that you could not from the original data.
The new information comes from our choice in how to generate that data. We're not just blindly making synthetic data, we come up with clever way to generate synthetic data that is hopefully high quality and can improve our models (and if it doesn't, we don't use it).
If the correct labels in the original training set outweigh the incorrect ones then it is possible to reduce the number of errors by relabeling using the trained model. If you can also identify labels that are likely to be incorrect and then have humans focus on relabeling those you have a way to efficiently improve the data.
> first, we fine-tune a LLaMA-3-8B powered LLaVA-1.5 and then employ it to recaption ∼1.3 billion images from the DataComp-1B dataset
It's a bit surprising this works as well as they report.
After all, you would think if you trained LLaVA on a dataset with low quality labels, you'd get a model that could only generate low quality labels, and re-labelling the dataset with the model would give you labels no better than the original low quality ones.
I’m not convinced the paper shows this “works”. Table 3 clearly shows zero-shot image classification is worse with the recaptioned labels.
All the results that show improvement seem to be evaluations using LLMs, e.g. they are showing LLMs think the LLM-generated text is better, which is neither surprising nor expected to correlate with real downstream task performance - unless your final task is labelling for an LLM, e.g. retrieval I guess.
Some good some bad here; it looks like you can in general get better descriptions out of LLama-3+Llava-1.5 than random crappy / short / possibly wrong human captions.
The image generation models they train using the new caption system have better coherence and look a lot better / less weird. (I don’t know if they showed training detail differences between their comparisons, but let’s assume they’re not hiding the ball here).
The bad: Those captions don’t always perform better on all tests using existing infra - some of their tests perform worse against existing benchmarks.
Why they perform worse is an open question; the answer might be that the existing test data expects short / human answers, or it might be that the models are more verbose but less accurate (although that seems unlikely looking at the examples they show).
Also troubling to me is a toss-off note in Table 3 showing that the “concat” data, one in which they use original caption concatenated to the generated caption, has really bad CLIP outcomes. That … surprises me based on the rest of the results.
That said, their baseline comparison numbers in Table 1 are pretty compelling — this would definitely go into the ‘try it’ category based on the paper. But, it also goes into the ‘might fail in really surprising ways, so proceed carefully’ category for me.
EDIT: Another way to think of this model is to say “can we exfiltrate multi-model training data from open weights / closed training models?” And I think there the answer is yes.
> EDIT: Another way to think of this model is to say “can we exfiltrate multi-model training data from open weights / closed training models?” And I think there the answer is yes.
Could you elaborate on this? Do you mean to say that this work reveals some of LLaMa's training data, or that this can be used to generate training data for new models?
Since they're using LLama with Llava, they're getting lots of Llama's baked-in understanding of the world. And I believe that Llama had multimodal training data as part of its pipeline, whether or not we get access to the heads that parse that data in the open weights.
So, I'm saying that Llama has some ability to describe / complete captions of photos, and very probably that ability comes from some multimodal training done at FB, and that using it in this way to train something new that beats benchmarks is a form of pulling out the value of that training data.
p.s. In the original comment, I meant to type 'multimodal' and didn't, or got autocorrected and didn't notice - apologies
Only if you have actual humans reviewing the output while being paid livable wages (i.e., you don't get to use digital sweatshops for this).
But considering Google has had over a decade to get autogenerated captioning right and they consistently fail at it, I doubt any slopshop is going to produce worthwhile output.
The caption is like the title of a painting. It does not so much tell what the image is an image of, but what the author/painter tells us their image represents. Represents, not depicts.
Or consider a song, it's title does not tell where the sound was recorded, but what the composer tells us the song is about. Not where and when it was recorded.
I guess it depends on the purpose. As a query rewrite, these cover much more of the physical scene, but lack accuracy and specificity.
A couple of issues. First, the model completely botches the "Trike" brotherhood to "Tike", which also drops the meaning of the symbol. It's a tricycle. The seat isn't a sidecar, it's behind the driver's. Some of the numbers for the racecars also seem to be hallucinated.
It also doesn't seem accurate to describe the rewritten captions as having a "richer" vocabulary as the authors do. It's certainly more verbose and captures a lot of details about the setting, but lacks information density for the subject of the photo. You can't make up for low specificity with more high-probability words. Sometimes you need a long-tail word.
For example, the Western Kingbird caption. It's not just a grey and yellow bird. If the model doesn't know what a Western Kingbird is, you're not going to be able to draw a Western Kingbird no matter how verbose your description is.
It's the difference between a caption and a description. Both are useful, just in different settings.
A visually-impaired person reading the newspaper just wants to know that there is an image of, say, George Washington and John Adams. What do they look like? What are they wearing? What is the setting? Probably not important, it's just decoration for the article.
On the other hand, a visually-impaired person who's shopping for clothing might come across a t-shirt with a print and want to know what's on there. What colors are used? Is there any text? Are there logos? Stuff like that matters when you're choosing clothing because you don't want to end up looking like a clown.
There would be good and bad to recaptioning images.
Of course it would make finding some images easier.
Problems with AI image relabeling might include:
1. AIs overuse some terms. If you search for a term the AI does not use, you find nothing.
2. AIs can be wrong. What if the AI misclassifies all 1967 Corvettes?
3. AIs are good at finding the big picture, but will it know that that is the only 1967 Corvette ever to leave the factory with those odd tail lights? (I'm making that up as an example.)
4. AIs might give overly general or overly specific labels. Is that a bit of technical clipart, a PCB layout, or part of an ESP32-S3-DevBoardC-1 PCB layout that has pins 3 of 4 switched in an obvious error?
40 comments
[ 3.3 ms ] story [ 105 ms ] threadPerhaps that doesn't matter, as the abstract suggests that's not the purpose.
On the other hand, training on captions they generate that are also incorrect is problematic. Perhaps they should have cherry-picked better examples without errors, such as details about the sugar and cream, reflections of the train in the water, or the location of the watermark and sign. I didn't look at the code, but perhaps each image feature can be written with a confidence score.
Another good example on how accessibility can even help people without disabilities. You can see this on ALT text on Mastodon, and I find it really useful.
I'm saying these models are censored for the lowest common denominator.
https://en.wikipedia.org/wiki/Model_collapse
It's a bit surprising this works as well as they report.
After all, you would think if you trained LLaVA on a dataset with low quality labels, you'd get a model that could only generate low quality labels, and re-labelling the dataset with the model would give you labels no better than the original low quality ones.
All the results that show improvement seem to be evaluations using LLMs, e.g. they are showing LLMs think the LLM-generated text is better, which is neither surprising nor expected to correlate with real downstream task performance - unless your final task is labelling for an LLM, e.g. retrieval I guess.
The image generation models they train using the new caption system have better coherence and look a lot better / less weird. (I don’t know if they showed training detail differences between their comparisons, but let’s assume they’re not hiding the ball here).
The bad: Those captions don’t always perform better on all tests using existing infra - some of their tests perform worse against existing benchmarks.
Why they perform worse is an open question; the answer might be that the existing test data expects short / human answers, or it might be that the models are more verbose but less accurate (although that seems unlikely looking at the examples they show).
Also troubling to me is a toss-off note in Table 3 showing that the “concat” data, one in which they use original caption concatenated to the generated caption, has really bad CLIP outcomes. That … surprises me based on the rest of the results.
That said, their baseline comparison numbers in Table 1 are pretty compelling — this would definitely go into the ‘try it’ category based on the paper. But, it also goes into the ‘might fail in really surprising ways, so proceed carefully’ category for me.
EDIT: Another way to think of this model is to say “can we exfiltrate multi-model training data from open weights / closed training models?” And I think there the answer is yes.
Could you elaborate on this? Do you mean to say that this work reveals some of LLaMa's training data, or that this can be used to generate training data for new models?
So, I'm saying that Llama has some ability to describe / complete captions of photos, and very probably that ability comes from some multimodal training done at FB, and that using it in this way to train something new that beats benchmarks is a form of pulling out the value of that training data.
p.s. In the original comment, I meant to type 'multimodal' and didn't, or got autocorrected and didn't notice - apologies
But considering Google has had over a decade to get autogenerated captioning right and they consistently fail at it, I doubt any slopshop is going to produce worthwhile output.
Or consider a song, it's title does not tell where the sound was recorded, but what the composer tells us the song is about. Not where and when it was recorded.
A couple of issues. First, the model completely botches the "Trike" brotherhood to "Tike", which also drops the meaning of the symbol. It's a tricycle. The seat isn't a sidecar, it's behind the driver's. Some of the numbers for the racecars also seem to be hallucinated.
It also doesn't seem accurate to describe the rewritten captions as having a "richer" vocabulary as the authors do. It's certainly more verbose and captures a lot of details about the setting, but lacks information density for the subject of the photo. You can't make up for low specificity with more high-probability words. Sometimes you need a long-tail word.
For example, the Western Kingbird caption. It's not just a grey and yellow bird. If the model doesn't know what a Western Kingbird is, you're not going to be able to draw a Western Kingbird no matter how verbose your description is.
A visually-impaired person reading the newspaper just wants to know that there is an image of, say, George Washington and John Adams. What do they look like? What are they wearing? What is the setting? Probably not important, it's just decoration for the article.
On the other hand, a visually-impaired person who's shopping for clothing might come across a t-shirt with a print and want to know what's on there. What colors are used? Is there any text? Are there logos? Stuff like that matters when you're choosing clothing because you don't want to end up looking like a clown.
Of course it would make finding some images easier.
Problems with AI image relabeling might include: 1. AIs overuse some terms. If you search for a term the AI does not use, you find nothing. 2. AIs can be wrong. What if the AI misclassifies all 1967 Corvettes? 3. AIs are good at finding the big picture, but will it know that that is the only 1967 Corvette ever to leave the factory with those odd tail lights? (I'm making that up as an example.) 4. AIs might give overly general or overly specific labels. Is that a bit of technical clipart, a PCB layout, or part of an ESP32-S3-DevBoardC-1 PCB layout that has pins 3 of 4 switched in an obvious error?