25 comments

[ 3.0 ms ] story [ 42.6 ms ] thread
With AI, it feels like deterministic outcomes are not valued as experience taught us it should.

The absence of means to measure outcomes of these prompt documents makes me feel like the profession is regressing further into cargo culting.

Bro science is rampant in the AI world. Every new model that comes out is the best there ever was, every trick you can think of is the one that makes all the other users unsophisticated, "bro, you are still writing prompts as text? You have to put them into images so the AI can understand them visually as well as textually".

It isn't strange that this is the case, because you'd be equally hard pressed to compare developers at different companies. Great to have you on the team Paul, but wouldn't it be better if we had Harry instead? What if we just tell you to think before you code, would that make a difference?

LLMs are the eternal September for software, in that the sort of people who couldn’t make it through a bootcamp can now be “programming thought leaders”. There’s no longer a reliable way to filter signal from noise.

Those 3000 early adopters who are bookmarking a trivial markdown file largely overlap with the sort of people who breathlessly announce that “the last six months of model development have changed everything!”, while simultaneously exhibiting little understanding of what has actually changed.

There’s utility in these tools, but 99% of the content creators in AI are one intellectual step above banging rocks together, and their judgement of progress is not to be trusted.

Do you ever filter signal from noise by the quality of the code? The code written by the Google founders was eventually rewritten by others, and it was likely worse than what a fresh grad produces today. Still, that initial search engine is the most influential thing they ever built, and it's something the modern Bay Area will probably never create again.
These hopeful incantations are a kind of cargo cult… But when applied to programming, the wild thing is that the cult natives actually built the airports and the airplanes but they don’t know what makes them fly and where the cargo comes from.
I know some people that would also benefit from these 65 lines of markdown. Even without using AI.
> But the original repository has almost 4,000 stars, and surely, 4,000 developers can’t be wrong?

This is such a negative messaging!

Let's check star history: https://www.star-history.com/#forrestchang/andrej-karpathy-s...

1. Between Jan 27th and Feb 3rd stars grew quickly to 3K, project was released at that time.

2. People star it to be on top of NEW changes, people wanted to learn more about what's coming - but it didn't come. Doesn't mean people are dumb.

3. If OP synthesized the Markdown into a single line: "Think before coding" - why did he went through this VS Code extension publishing? Why can't they just share learnings and tell the world, "Add 'Think before coding' before your prompt and Please try for yourself!"

PS: no I haven't starred this project, I didn't know about it. But I disagree with the authors "assumptions" about stars and correlating it to some kind of insight revelation

All good advice in general. Could add others, like x-y problems etc.

This feels like a handbook for a senior engineer becoming a first level manager talking to junior devs. Which is exactly what it should be.

However, this will go horribly wrong if junior devs are thus “promoted “ to eng managers without having cut their teeth on real projects first. And that’s likely to happen. A lot.

That's just how it is in the LLM world. We've just gotten started. Once upon a time, the SOTA prompting technique was "think step by step".
surely, 4,000 developers can’t be wrong

Apparently almost half of all the websites on the internet run on WordPress, so it's entirely possible for developers to be wrong at scale.

My probably incorrect, uninformed hunch is that users convinced of how AI should act actually end up nerfing its capabilities with their prompts. Essentially dumbing it down to their level, losing out on the wisdom it's gained through training.
I often wonder this as well, things are moving so quickly that unless you want to keep chasing the next best prompt/etc then you are better running as close to vanilla as you can IMHO.

Similar for MCP/Skills/Prompts, I’m not saying they can’t/don’t help but I think you can shoot yourself in the foot very easily and spend all your time trying to maintain those things and/or try to force the agent to use your Skill/MCP. That or having your context eaten up with bad MCP/Skills.

I read a comment the other day about sometime talking about Claude Code getting dumber then they went on to explain switching would be hard due to their MCP/Skills/Skill router setup. My dude, maybe _that’s_ the problem?

You know, it's good old prompt/context engineering. To be fair, markdowns actually can be useful because of LLM's (Transformer's) gullible/susceptible nature... At least that's what I discovered developing a prompting framework.

Of course it's hilarious a single markdown got 4000 starts, but it looks like just another example of how people chase a buzzing x post in tech space.

So we reached a point where the quality of a `piece` of software is decided based on stars on GitHub.

The exact same thing happened with xClaw where people where going "look at this app that got thousands of stars on GitHub in only a few days!".

How is that different than the followers/likes counts on the usual social networks?

Given how much good it did to give power to strangers based on those counts, it's hard not to think that we're going in the completely wrong direction.

(comment deleted)
Maybe I'm just really lucky but reading those instructions it's basically how I find Claude Code behaves. That repo with 4k stars is only 2 weeks old as well, so it's obviously not from a much less competent model.
Next inevitable step is LLM alchemy. People will be writing crazy-ass prompts which ununderstandable text which somehow get system work better than the straight-human-text prompts.
I found that "Make no mistakes, or you go to jail" improves claude-code's performance by about 43%
These days I genuinely can't tell if articles are satire or not.
There will come a day soon where “hello world” will be typed by sentient hands for the last time.
The document under discussion:

https://github.com/forrestchang/andrej-karpathy-skills/blob/...

I find the whole premise of writing some vague instructions, feeding them to a stochastic parrot and expecting a solid engineering process to materialize out of the blue quite ridiculous.

Any sufficiently advanced "AI" technology is indistinguishable from bullshit.

This is so incredibly depressing. As of the time of me posting this comment this has 73 upvotes. I'm sorry, but this is absurd. People put real effort in real posts that don't see half this many votes but AI Slop on top of AI Slop? Upvote!

This post is about a prompt that has 4K+ stars (that matters to people?) so they wrapped it up in an extension for Cursor and they ask you to try it out and star their repo (Please clap?).

And this "Sensation"?

> Was the result better? I’m not really sure.

I cannot even...

A Slop article about a Slop prompt that a bunch of Slop people starred, "I don't know if it helps but here is an extension that you should use", just the laziest of everything.

I'd bet that this prompt, if it is even helpful at all, will stop be helpful or be redundant in a couple weeks at most time.

I'm _far_ from anti-LLM, I use them quite a bit daily, I run multiple Claude Code instances (which I review, gasp!), but this is reaching a fever pitch and this article was the straw that broke this camel's back.

Funny. The author is not sure if his (and original) extension improves Claude output, but since the original project has 4k+ start on Github, "surely, 4,000 developers can’t be wrong". So, "Please try for yourself! Install the extension, don’t forget to star my repository and see the results". No matter if it's good, just star my repo.
13 Markdown files have erased 285 billion dollars of wealth in the stock market - https://martinalderson.com/posts/wall-street-lost-285-billio...

Why are you surprised that it takes only 65 lines of Markdown to create the next Linus Torvalds or Donald Knuth!

I’m learning Haskell by Thinking before Coding, without working through the difficult exercises. I finally understand Monads because of Thinking first.

I have been thinking a lot about the use of AI and how to use it. Part of my process has been watching others, namely the people who I thought were incompetent at their job before AI.

I have found the following, but I suspect as AI gets better this will change.

1) those who where incompetent before still are, but AI hides it.

2) those who were competent before AI do vastly more with AI. They seem to apply it in away that simply overshadows what the incompetent are doing.

3) the incompetent seem to be fascinated with things like skills, pre prompts and, setting policies and guidelines, and workshops. The competent seem to need none of this, are not going to workshops, already have their own and simply are more productive.