Spot on critical analysis of the blog post "Developers reinvented" by Github Thomas Dohmke which includes such quotes as:
> Many Computer Science (CS) programs still center around problems that AI can now solve competently.
Yeah. No they do not. Competent CS programs focus on fundamentals not your ability to invert a binary tree on a whiteboard. [1]
Replacing linear algebra and discrete mathematics with courses called "Baby's First LLM" and "Prompt Engineering for Hipster Doofuses" is as vapid as proposing that CS should include an entire course on how to use git.
> The sample size is 22. According to this sample size calculator I found, a required sample size for just one thousand people would be 278
I'm all for criticizing a lack of scientific rigor, but this bit pretty clearly shows that the author knows even less about sample sizes than the GitHub guy, so it seems a bit pot calling the kettle black. You certainly don't need to sample more than 25% of any population in order to draw statistical information from it.
The bit about running the study multiple times also seems kinda random.
I'm sure this study of 22 people has a lot of room for criticism but this criticism seems more ranty than 'proper analysis' to me.
>I found, a required sample size for just one thousand people would be 278
It's interesting to note that for a billion people this number changes to a whopping ... 385. Doesn't change much.
I was curious, with 22 sample size (assuming unbiased sample, yada yada), while estimating the proportion of people satisfying a criteria, the margin of error is 22%.
While bad, if done properly, it may still be insightful.
It's a bit annoying that this article on AI is hallucinating itself:
> To add insult to injury, the image seems to have been created with the Studio Ghibli image generator, which Hayao Miyazaki described as an abomination on art itself.
He never said this. This is just false, and it seems like the author didn't even fact check if Hayao Miyazaki ever said this.
> Said person does not give a shit about whether things are correct or could even work, as long as they look "somewhat plausible".
This seems to be the fundamental guiding ideology of LLM boosterism; the output doesn't actually _really_ matter, as long as there's lots of it. It's a truly baffling attitude.
Professional statistician here. Not that I get to do any of that these days, bar read Significance magazine and get angry occasionally.
Looking at the original blog post, it's marketing copy so there's no point in even reading it. The conclusion is in the headline and the methodology is starting with what you want to say and working back to supporting information. If it was in a more academic setting it would be the equivalent of doing a meta-analysis and p-hacking your way to the pre-defined conclusion you wanted.
Applying any kind of rigour to it is pointless but thanks for the effort.
This analysis is not complete, it needs to continue with an analysis of how many people, and critically how many business owners, believe the lies and non-truths propagandized by that article and the entire marketing push of LLMs.
That article is not for developers, it's for the business owner, their management, and the investor class. If they believe it, they will try to enforce it.
This is serious, destroy our industry type of idiot logic.
One thing that seems to always get lost in the AI hype cycle is actually screening articles by the source. Maybe it's just because in this age of social media we don't have much in the way of journalism. I have zero interest in what Sam Altman, Dario Amodei, Thomas Dohmke, etc. have to say because they obviously want to sell something.
The Miyazaki quote get taken out of context and altered. It was "an insult to life itself", not "an abomination on art itself", and the context was creepy AI animations of zombies. He said that he won't use the technology, but the implied reason was because he saw a crass demo. "Whoever creates this stuff has no idea what pain is." https://en.wikiquote.org/wiki/Hayao_Miyazaki
23 comments
[ 2.4 ms ] story [ 53.3 ms ] thread> Many Computer Science (CS) programs still center around problems that AI can now solve competently.
Yeah. No they do not. Competent CS programs focus on fundamentals not your ability to invert a binary tree on a whiteboard. [1]
Replacing linear algebra and discrete mathematics with courses called "Baby's First LLM" and "Prompt Engineering for Hipster Doofuses" is as vapid as proposing that CS should include an entire course on how to use git.
[1] https://x.com/mxcl/status/608682016205344768
I'm all for criticizing a lack of scientific rigor, but this bit pretty clearly shows that the author knows even less about sample sizes than the GitHub guy, so it seems a bit pot calling the kettle black. You certainly don't need to sample more than 25% of any population in order to draw statistical information from it.
The bit about running the study multiple times also seems kinda random.
I'm sure this study of 22 people has a lot of room for criticism but this criticism seems more ranty than 'proper analysis' to me.
It's interesting to note that for a billion people this number changes to a whopping ... 385. Doesn't change much.
I was curious, with 22 sample size (assuming unbiased sample, yada yada), while estimating the proportion of people satisfying a criteria, the margin of error is 22%.
While bad, if done properly, it may still be insightful.
> To add insult to injury, the image seems to have been created with the Studio Ghibli image generator, which Hayao Miyazaki described as an abomination on art itself.
He never said this. This is just false, and it seems like the author didn't even fact check if Hayao Miyazaki ever said this.
Spot on, I think this every time I see AI art on my Linkedin feed.
This seems to be the fundamental guiding ideology of LLM boosterism; the output doesn't actually _really_ matter, as long as there's lots of it. It's a truly baffling attitude.
Looking at the original blog post, it's marketing copy so there's no point in even reading it. The conclusion is in the headline and the methodology is starting with what you want to say and working back to supporting information. If it was in a more academic setting it would be the equivalent of doing a meta-analysis and p-hacking your way to the pre-defined conclusion you wanted.
Applying any kind of rigour to it is pointless but thanks for the effort.
Is it clickbait if it’s literally quoting the author? I mean, yes, it was clickbait by Thomas Dohmke, but not by the source that used that headline.
That article is not for developers, it's for the business owner, their management, and the investor class. If they believe it, they will try to enforce it.
This is serious, destroy our industry type of idiot logic.