This sounds a bit too good to be believable. Do they ask the reader to believe it's 100% reliable and their code magically just knows what CASM looks like without looking?!? All without a single working link to the original research. Sounds like the PR spin conman would push in the process of regulatory capture to force his future captive userbase be required by law to become a paying customer at any price or be branded a criminal (aka not rich and connected enough to rape kids). Sorry if I'm jaded.
> The approach, detailed in a paper presented at the International Conference on Machine Learning, achieved 100% accuracy in identifying models specialized for CSAM generation.
I know some reasons for 100% accuracy in machine learning, first of all the test set leaking into training data. Or you just accept a silly high false positive rate.
When I was an admin I liked to joke that if you guarantee more than 5 nines, then you are an insurance company and you are planning to pay the penalty instead of actually fulfilling your promise, here the principle is probably the same.
What is this spam. No link to today's MIT article[0].. from which most of the sentences and all of the quotes are taken, or the one author Arxiv entry[1]. Quotes as if they talked to the writers.
I don't consider myself a mathematician nor data scientist but...
Imagine you have a model that detects some content, but supposedly cannot generate it. Let's steelman and say 1 bit (pass/fail).
Now take some noise. Run it through the filter, get a rating. Perturb the noise a bit, get a new rating. Repeat the process enough times to get a difference vector. Apply the difference vector to your noise. Repeat enough times until you literally have the content you're supposed to be filtering.
Doesn't this technique turn any filter into a diffusion model, making TFA's claims literally impossible?
Obviously the 100% is wrong. The big concern with using this is there is no way to verify the model results. It’s basically making an assertion it can’t prove. Normally there is som accessible ground truth answer, but here it is illegal to actually prove the model can generate offending material (which is the whole point of this classifier) so there is no recourse and no explanation and no possibility of a human in the loop.
Worse, the ignorant will believe the 100% claim and equate a positive classification with truth.
7 comments
[ 3.9 ms ] story [ 29.1 ms ] threadIf I say every model is trained on CSAM, I too will correctly identify 100% of the models that were. Says little about my false positive rate though.
(yes, this is sarcasm)
I know some reasons for 100% accuracy in machine learning, first of all the test set leaking into training data. Or you just accept a silly high false positive rate.
When I was an admin I liked to joke that if you guarantee more than 5 nines, then you are an insurance company and you are planning to pay the penalty instead of actually fulfilling your promise, here the principle is probably the same.
[0]: https://news.mit.edu/2026/new-method-keeps-kids-safe-from-il... (https://news.ycombinator.com/item?id=48893301)
[1]: https://arxiv.org/abs/2604.25119
Imagine you have a model that detects some content, but supposedly cannot generate it. Let's steelman and say 1 bit (pass/fail).
Now take some noise. Run it through the filter, get a rating. Perturb the noise a bit, get a new rating. Repeat the process enough times to get a difference vector. Apply the difference vector to your noise. Repeat enough times until you literally have the content you're supposed to be filtering.
Doesn't this technique turn any filter into a diffusion model, making TFA's claims literally impossible?
Worse, the ignorant will believe the 100% claim and equate a positive classification with truth.