Sadly, the major ML/AI/NLP conferences are being inundated with AI slop papers. That will arguably have a bigger impact on the quality of research moving forward.
"I think you just need to accept the results of the competition. The winning submissions clearly provide value and had a lot of effort invested in them. I'm not really worried about a few inconsistencies or mistakes if the value is still there. Did you think another submission deserved to win over these?"
That comment is gold. Yeah, I'm not worried about hallucinated slop, just accept it was the winner folks.
I don't know about this exact competition but overall fair hackathons have been killed by AI.
It all seems fine from the outside but all the code is generated in all the projects and judging happens via AI, I have seen projects win because they prompt inject that they are the winners.
It used to be about human skill, now it's about ideas and of course insiders are the main winners.
AI is useful. But the amount of people that are simply offloading all of their thinking to AI and blindly accepting the answer is absurd. Kaggle is most likely using ai to assess the submissions and are not using any common sense by blindly accepting the results.
overall, the quality of products has been going downhill.
AI is not there yet, instead of working hard, everyone is choosing the easy way out.
AI slop wins prize, I wonder if Ai slop read it also. would not be surprised. however not to judge anyone, I think we are seeing slop everywhere, hope some things still require hard blocks for low quality.
its difficult to justify lack of attention and details
I think that a lot of software engineers are using LLMs and a lot of very popular tools are developed by, or are assisted by, LLMs. Is this not just going to be a thing going forward?
This feels akin to traditional artists getting angry at digital art winning competitions when that was a new concept.
We're simply in the early stages of a paradigm shift, no?
I think this is a good meta-lesson for Kaggle. When you have objective metrics to hill-climb towards, AI can do quite well. When you just phone it in and rely on LLM as a Judge, the results are not so great.
50 comments
[ 1.9 ms ] story [ 39.7 ms ] threadI get people want to work at an AI lab but slopping it in public in this manner is counterproductive to the original intended purpose of these places.
Given that LLMs are trained with RL && LLM-as-a-judge, is it really cheating if real competitions use the same?
Maybe the real alignment is the slop we decoded along the way
That comment is gold. Yeah, I'm not worried about hallucinated slop, just accept it was the winner folks.
The attached paper's (https://arxiv.org/pdf/2604.16009) title is "MEDLEY-BENCH: Scale Buys Evaluation but Not Control in AI Metacognition"
This is the most blatant Claude line, or as Claude would put it, the smoking gun.
It all seems fine from the outside but all the code is generated in all the projects and judging happens via AI, I have seen projects win because they prompt inject that they are the winners.
It used to be about human skill, now it's about ideas and of course insiders are the main winners.
AI is not there yet, instead of working hard, everyone is choosing the easy way out.
AI slop wins prize, I wonder if Ai slop read it also. would not be surprised. however not to judge anyone, I think we are seeing slop everywhere, hope some things still require hard blocks for low quality.
its difficult to justify lack of attention and details
This feels akin to traditional artists getting angry at digital art winning competitions when that was a new concept.
We're simply in the early stages of a paradigm shift, no?