Not asking for feedback is the killer for me. Even most junior developers will ask for more information if they don't have enough context/confidence to complete a task.
I don't understand why people take bad coding practices and just let AI run with it and then expect nothing but poor quality code. Nothing about the AI revolution here changes how good software has always been written. Write tests, use a typed language, review code. If you have good patterns, good procedures, AI fits right in and fills in the blanks perfectly. Poor AI results tend to be the pot calling the kettle black.
Where I find AI most useful is getting it to do tasks I already know how to do, but would take time.
If you understand the problem you are trying to solve well enough to explain it to the LLM, you can get good results, you can also eyeball the outputted code and know right away if it's what you are after.
Getting it to do things you don't know how to do is where it goes off the rails IMO
My preferred approach in similar situations is to ask an LLM for an initial solution or code snippet, then take over manually - no endless prompt tweaking, just stop prompting and start coding. Finally (optionally), I let the LLM do a final pass to review my completed solution for bugs, optimizations, etc.
The key win is skipping the prompt refinement loop, which is (A) tedious and time-consuming, and (B) debilitating in the long run.
I recently spent over an hour trying to get ChatGPT to give me some pretty simple rsync commands. It kept giving me command line parameters that didn't work on the version of rsync on my mac. With ~50% of the failures, it would go down troubleshooting rabbit holes and the rest of the time it would "realize" that it was giving incorrect version responses. I tell it to validate each parameter against my version moving forward and it clearly doesn't do that. I am sure I could have figured it out on my own in 5 mins, but I couldn't stop watching the trainwreck of this zeitgeist tech wasting my time doing a simple task.
I am not a coder (much), but I have to wonder if my experience is common in the coding world? I guess if you are writing code against the version that was the bulk of its training then you wouldn't face this specific issue. Maybe there are ways to avoid this (and others) pitfall with prompting? As it is, I do not see at all how LLMs could really save time on programming tasks without also costing more time dealing with its quirks.
I don't like vibe coding as much as actual coding, but the biggest improvement in my workflow was shifting left even more.
Now I dedicate at least one session to just writing a spec file, and have it ask me clarifying questions on my requirements and based on what it finds in the codebase and online. I ask it to also break down the implementation plan in phases with a checklist for each phase.
I then start at least one new session per phase and make sure to nail down that phase before continuing.
The nice thing is if it gets annoying to vibe code it, I or someone on my team can just use the spec to implement things.
>There is never enough context. We learned quickly that the more context we provided and the smaller the issues, the better the results. However, no matter how much context we provided, the AI would still mess things up because it didn’t ask us for feedback. AI would just not understand if it didn’t have enough information to finish a task, it would assume, a lot, and fail.
Is it me or does it feels like the genie in the bottle thing. Remember a TV show where the guy and his friend sat down with the Genie like a lawyer to make sure every angle is covered (going to spare you the details here). That is what it feels like interacting to a LLM sometimes.
Well, except that (I think I know the scene you're referring to), it ultimately worked. The LLM, on the other hand, will feel no need to stick to its 'promises'.
(Really the genie is closer to the traditional sci-fi AI in that it's legalistic and rules-bound; the LLM very much isn't.)
"We just don’t think we will incorporate AI to do more than that, given the current state of things. We will, however, keep an eye in case the technology changes fundamentally."
I wonder whether LLMs are capable of doing more; probably, we need another paradigm for that; still, they are very, very useful when used right
I decided to adopt AI assisted coding for a recent project. Not sure what defines 'vibe coding' but the process I ended up was a iterative interaction at a measured pace.
I used Gemini AI studio for this and I was very pleased at the result and decided to open source it. I have completely captured and documented the development transcript. Personally it has give me considerable productivity boost. My only irritation was the unnecessarily over politeness that AI adopts in My take is
AI yields good ROI when you know exactly what you want at the end of the process and when you want to compare and contrast decision choices during the process.
I have used it for all artifacts of the project:
- Core code base
- Test cases
- Build scripts
- Documentation
- Sample apps
- Utilities
16 comments
[ 2.5 ms ] story [ 25.5 ms ] threadWhere I find AI most useful is getting it to do tasks I already know how to do, but would take time.
If you understand the problem you are trying to solve well enough to explain it to the LLM, you can get good results, you can also eyeball the outputted code and know right away if it's what you are after.
Getting it to do things you don't know how to do is where it goes off the rails IMO
The key win is skipping the prompt refinement loop, which is (A) tedious and time-consuming, and (B) debilitating in the long run.
> Our marketing director (that’d be me) said that if we don’t write something about it, we will be left behind...
Write when you have something to say. What was I supposed to learn here?
I am not a coder (much), but I have to wonder if my experience is common in the coding world? I guess if you are writing code against the version that was the bulk of its training then you wouldn't face this specific issue. Maybe there are ways to avoid this (and others) pitfall with prompting? As it is, I do not see at all how LLMs could really save time on programming tasks without also costing more time dealing with its quirks.
Indeed, self-invented abstractions are a bridge too far for AI.
You have to keep it close to the path already walked before by thousands of developers.
This makes AI more of a search engine on steroids than anything else.
Now I dedicate at least one session to just writing a spec file, and have it ask me clarifying questions on my requirements and based on what it finds in the codebase and online. I ask it to also break down the implementation plan in phases with a checklist for each phase.
I then start at least one new session per phase and make sure to nail down that phase before continuing.
The nice thing is if it gets annoying to vibe code it, I or someone on my team can just use the spec to implement things.
Is it me or does it feels like the genie in the bottle thing. Remember a TV show where the guy and his friend sat down with the Genie like a lawyer to make sure every angle is covered (going to spare you the details here). That is what it feels like interacting to a LLM sometimes.
(Really the genie is closer to the traditional sci-fi AI in that it's legalistic and rules-bound; the LLM very much isn't.)
I wonder whether LLMs are capable of doing more; probably, we need another paradigm for that; still, they are very, very useful when used right
I used Gemini AI studio for this and I was very pleased at the result and decided to open source it. I have completely captured and documented the development transcript. Personally it has give me considerable productivity boost. My only irritation was the unnecessarily over politeness that AI adopts in My take is
AI yields good ROI when you know exactly what you want at the end of the process and when you want to compare and contrast decision choices during the process.
I have used it for all artifacts of the project: - Core code base - Test cases - Build scripts - Documentation - Sample apps - Utilities
Transcript - https://gingerhome.github.io/gingee-docs/docs/ai-transcript/... Project - https://github.com/gingerhome/gingee
The proceeding without clarifying or asking questions thing really grinds my gears.