Code autocompletion. I think it works around 60% of the cases, sometimes it saves some tipping. Performance increase: small. Additionally, I don't learn the API as good as if I were to type it, considering moving back.
I stopped using autocomplete and I don't really miss it. It was really impressive for simple repetitive stuff, but it's an active nuisance when I'm working on something more complex.
Yes, I use copilot, Claude, and sometimes Zed with the built in Claude AI tooling.
It's really useful for basic stuff, scripting, boilerplate, repetitive things, SQL. For example it's great at converting types from different languages, or generate types from API reference docs.
You gotta be careful though as it is not perfect and if you do something "not typical" it will not work well. Also sometimes it is a bit outdated, produces code with different "style", or just plain wrong stuff, so you have to tweak it a bit.
It's not going to code for you yet (unless you do very basic stuff) but it's a great tool to increase productivity. I do believe I move faster.
I enjoy using the Projects feature of Claude. You have a workspace to add knowledge (file/image/text) that you are able to update and maintain throughout multiple discussions.
It's easy, you just remove the old content and add the updated files. There is Project level Workspace and Chat level Workspace. Each have their own size limit. Once one chat has accomplished my task, I add whatever relevant Artifacts it produced to the Project Workspace and move on to a new chat.
Asking ChatGPT to give me boilerplate is a huge productivity win for me still. For example, I recently asked ChatGPT to give me a Terraform file that sets up API Gateway, Lambda, and DynamoDB. The script it gave me worked after only a couple of minor tweaks. Was up and running in about 15 minutes, whereas I'm pretty sure it'd be hours if I had to dredge through the docs myself.
I must correct: also using RAG based LLM engines can provide very delirious responses. The possibilities are magnificent, but reliability is really not there yet.
ChatGPT o1 preview is producing amazing results for code and generally helpful with life. It's not well integrated to many IDE tools yet (I use Cody a bit which has a wait list for o1 preview), but then I keep going back to just using the ChatGPT interface anyway because it feels cleaner to manually select and upload the relevant files attachments anyway. I'm sure that will change as integrations improve. Claude seems really popular on here but I wonder if that's just because Sonnet is free and useful enough. I prefer the results of newer ChatGPT models currently and feel they are worth paying for.
Code completion. I’m about 50/50 on whether it’s actually helpful or I just spend more time reading the completions to make sure they are correct instead of just learning and typing it out myself.
They're fine with cloud. But... I suppose most of our most sensitive documents are in Microsoft's SharePoint anyway. Effectively sharing our code with them via Copilot is actually comparatively less problematic.
Personally, my main gripe with it are the response times. But I'm a latency junkie.
Our company is very not OK with sharing IP with the cloud to the extent that we only recently started using github. So we're banned from using these things.
Microsoft has been in the corporate game for longer than most people on this site have been alive. They understand what corporate clients want, and which guarantees they require. And they go to great lengths to provide what they guarantee.
I use GitHub Copilot at work. We were (and still are) not allowed to use our private accounts, but recently got company accounts we are allowed to use on codebases classified "confidential" or lower. We also have an internal chat interface to OpenAI's models with a similar restriction. I understand there's some extra agreements with Microsoft regarding our data.
It's great for refactorings and text manipulation -- things I used to do regex for can be done using gpt, though that depends on the scope so it is still not quite there to replace regex. Oh did I say it's great at bash too?
I use Cursor primarily with Claude 3.5 Sonnet. Overall a solid productivity increase depending on the task.
I have a few observations:
- I vastly prefer Cursor's Copilot++ UX for autocomplete compared to GitHub's in VSCode, which I used until a few months ago.
- The Composer multi-file editor (cmd+i) is easily its most powerful feature and what I use most often, even when I'm working on single files. It just works better for some reason.
- It's far more effective working in popular stacks, eg. Typescript/NextJS etc. It's rarely a time-saver when working in Elixir, for example.
- In a similar vein, the less 'conventional' your task or code is, the less useful it becomes.
- As the context increases, it gets noticeably less useful. I often find myself having to plan what context I want to feed it and resetting context often.
- It's very effective at 'translation' tasks, eg. converting a schema from one format to another. It's much less effective at generating complex business logic.
- I only find it useful to generate code I confidently know how to write myself already. Otherwise, it doesn't save me time. The times I've been tempted, it's almost always bitten me.
Pretty similar observations. Using Aider with Claude on an iOS app I’ve it can be helpful to scaffold new modules for example, if I give it some existing code and tell it to copy the conventions. But it’s virtually useless for editing or changing code where it will often produce code that doesn’t compile, has bugs and/or doesn’t solve the requirements.
Anything to do with Swift concurrency it’s completely hopeless, I assume partly because there’s not enough training data yet.
> But it’s virtually useless for editing or changing code where it will often produce code that doesn’t compile, has bugs and/or doesn’t solve the requirements.
That is the exact problem I am trying to solve: modifying code with LLMs really sucks most of the time. I am trying a solution with Abstract Syntax Trees: I have the LLM write the code that will write the code you need. That is, modify the source tree rather than the text representation.
I do have it working for some cases quite well, but there are lots of pitfalls with this approach too. It does take a lot of context and the LLMs aren't really that well-versed at writing specifically esprima code for example. BeautifulSoup does work better, I guess because more people use it and there's more data in the training set.
I'm adding one language at a time, currently have HTML, CSS, Javascript and Python all kind-of working. It's pretty neat but I'm not sure how well it scales yet to larger projects and more difficult requirements/implementations.
Pretty much the same, except I use github copilot. I have exactly the same remarks, but I also use it to generate test cases, and I use in addition Microsoft Copilot for:
- Writing docstrlings. It helps to have a template.
- rubber duck. II try to explain my issue to it. It doesn't find the solution 90% of the time, but explaining the issue helps me (and sometimes you get a hint)
* code translation - e.g. convert a self-contained implementation of a numerical algorithm from one language to another and generate test cases and property tests which make sure the implementations are equivalent. The goal is to avoid having to proof read the generated code.
* one-off scripts - any task where code design doesn't matter, the amount of code is limited to couple hundred lines (GPT-4o) and the result will be thrown away after use.
* API exploration - producing examples for APIs and languages I'm not fluent in. Reading reference documentation gives a better understanding, LLMs get the results out faster.
Not OP but aider is pretty useful because it can write files in your repo, run commands commit changes etc. It's pretty flexible and easy to give it context by just providing a path.
For common languages e.g. Python, Javascript it works reasonably well although it still seems to prefer older versions of libraries way too often. Which means you need to do refactoring straight out of the bat.
Whenever I use it with Rust, Golang, Scala etc it's not worth the effort.
I use it as a personal assistant, and as a replacement of internet search. It has been working great for things like refactoring small parts of code, generating boilerplate or working out what an arcane error message can mean. I find it useful for comparing two technologies, and to have it give me a refresher about some topic.
On a personal assistant level, it's been useful to have me remind of words that I have forgotten, or to have it rephrase a sentence in a different way, to suit some mood, etc.
Occasionally I have fun with it by having it answer in rhymes, or theatrically like a fortune teller, or like lyrics of some gangsta rap.
I don't know about performance increases. What I notice most is that I'm less annoyed with it, than with the general state of the internet. The web pages and I have different goals: I want a specific piece of information, and they want me to load all their ads, affiliates and such in return, and to spend time on the page reading their drivel. It is also horrible to search for version-specific information, for example, I appreciate that it brings up the Ruby answer for the latest version, but I'm stuck on 2.5.0, and so, I need that specifically. LLMs are usually great at this. Orders of magnitude less fluff, rich text-only answer.
I haven't liked any integrated tooling at all, but that's not such a surprise since I generally disable autocomplete and autocorrect since, like tinsel, I find them distracting.
I do really like to use the plain web browser tools, though (currently claude), for generating boilerplate code that I then review and integrate carefully into my code. It has sped up my workflow in that way.
I built a simple desktop tool [1] to streamline my coding workflow with LLMs, like integrating with local code context, managing prompts and custom instructions, etc. It supports copy/paste into ChatGPT/Claude, and sending prompts via API.
Currently I estimate it writes about 70% to 80% code (including adding features to the tool itself), and saved me hours of work per week.
Lately I've been exclusively using API since it is cheaper than paying $20 monthly subscription.
I've been using Zed with Claude 3.5 for a few weeks now, and I find it incredibly useful. Being able to be in full control what goes into the context, my workflow is usually
- add files, tabs, terminal output and IDE diagnostics into the context via slash commands
- feed in documentation or other remote web content, also via simple slash commands
- activate workflow mode, which will help you edit the files instead of having to copy things around
- then ask questions, ask for refactoring, ask for new features
Often I like to ask for the high level approach, and if I agree with it let it guide the implementation. It makes mistakes, so I always have to validate and test what it creates, but as you add more information into the context and the LLM has a good amount of stuff to work with, the output quality really improves significantly.
It's a process though, and it takes time to get familiar with the workflow, build intuition when the LLM falls on its face, when you should try a different approach etc.
Generally I can echo what others have said, it works best if you already kind of know what you want to achieve and just use the LLM as an assistant that does the grunt work for you, documents the process, provides a rubber duck etc.
Generally, I would not want to work without an integrated LLM anymore, it provides that much value to my workflow. No panacea, no silver bullet, but when used right in the right circumstances it can be incredibly useful.
A secondary usecase for me is working on repositories where tasks and todos are structured in markdown files (even things like travel planning). Letting the LLM Guide you through todos and create a documentation trail through the process, identify important todos, carry along information as you go is wonderful, I would absolutely give that a try as well.
The competition in AI editors is a bit silly at the moment. Everyone and their dog are "building" an AI assisted editor now by duct taping Ollama onto VS Code. I don't like my data being sent to untrusted parties, so I cannot evaluate most of these. On top of that, the things keep evolving as well, and editors that I dismissed a few months ago, are now all of a sudden turning into amazing productivity boosters, thanks to developments in both models as well as in editor tricks.
My money is on Cursor [1], which does not stop to amaze me, and seems to get a lot of traction. The integration is very clever, and it is scary how it figures out what I intend to do. Then again, I'm probably doing mundane tasks most of the time. For the few bright moments in my day I tend to use ChatGPT, because most of my real problems are in application domains, not in code.
I am not a firm believer in forking large open-source projects, though, as it will take a lot of effort to keep up with future diversions. This makes me a bit wary of projects such as Cursor and Void [2]. Somebody needs deep pockets to sustainably surpass the popularity of VS Code. To point out just one problem with forking: VS Code works fine in Ubuntu, but Cursor does not work out of the box there. Having to disable the sandbox is a show-stopper for most.
In that respect, the extensions might be a safer bet, and I think Sourcegraph's Cody and Continue are making the largest waves there. Hard to tell with so many waves.
I write more than I code, but I use ChatGPT a lot throughout the day.
I am integrating AI translations into my custom static site generator. I will test the outcome heavily before putting my name (and a big warning) on my translated content, but the early results look good. Getting it right is a lot harder than piping the page through ChatGPT. Everything needs to be translated from the UI strings to the URL slugs.
My work will no longer just benefit English-speaking immigrants, but also locals and immigrants who Google things in other languages. I am very excited about it.
I also use ChatGPT heavily for “what’s the word/expression for” questions and other fuzzy language queries. As a non-native speaker, I want to know if a given expression is appropriate for the context, if it’s dated, if it’s too informal and if it’s dialect-appropriate.
I also use it for coding, but mostly because it’s faster than reading python’s docs. I ask it questions I know the answer to, hoping to find better approaches. So much happened since Python 2.7, and I don’t always know what to ask for.
On occasion I treat it like I treated my mom as a child. I ask it all sorts of questions about the world I observe around me. It’s amazing to get a short, clear answer to work from, instead of sifting through barely relevant, spammy search results. This is super helpful when getting to know new tech and figuring out what it actually is.
It’s just so damn cool to have a sort of real-life Hitchhiker’s Guide slash Pokédex in my pocket. These things appeared in the span of a year, and nobody seems impressed. Well, I am mad impressed.
> It’s amazing to get a short, clear answer to work from, instead of sifting through barely relevant, spammy search results.
I like this premise but don’t like the fact that the answer could be completely false - fabricated and presented with equal confidence as the algorithm has zero understanding, just stats.
I've tried it, and I don't like it. There's too much confabulation to be useful as code completion, and any task more complicated than that results in logic errors most of the time.
Code completion was fine without LLMs, and solving problems myself usually ends up being quicker than trying to coerce an LLM into doing it correctly and then verifying that the output is actually correct.
The one time I used an LLM in my workflow to good success was using ChatGPT to automatically create an enum of every EU/EEA country and a switch statement over that enum. Those sort of "grunt work" tasks that don't require any thinking, but a lot of typing, seem to be where LLMs shine.
> create an enum of every EU/EEA country and a switch statement over that enum
Sorry but if you then have to check that the list of the Countries is correct and complete, is it not faster to get a list from somewhere safe and then do your usual "Find-Replace" around the separator?
Edit: I just did it for a test: it took less than two minutes, without rush (search engine; open page; find data; open spreadsheet; paste table; paste column in text editor; find-replace around '\n').
I only use it to get a sense of a project. Instead of looking for a similar project (and ending up in medium hell), I now ask claude some first steps. I can continue from there. It's like helping out a friend get his project doing g
77 comments
[ 2.8 ms ] story [ 144 ms ] threadMaybe some day it will be good enough but that day is not now.
It's really useful for basic stuff, scripting, boilerplate, repetitive things, SQL. For example it's great at converting types from different languages, or generate types from API reference docs.
You gotta be careful though as it is not perfect and if you do something "not typical" it will not work well. Also sometimes it is a bit outdated, produces code with different "style", or just plain wrong stuff, so you have to tweak it a bit.
It's not going to code for you yet (unless you do very basic stuff) but it's a great tool to increase productivity. I do believe I move faster.
I had some issues with both gpt4 and sonnet getting slower and doing pointless refactors as project grows
But you need to use Retrieval Augmented Generation (or bare LLMs will either invent the nonexistent or present solutions to non-proposed problems).
The downside of LLMs is that it doesn't remove your need to know your code. And writing code yourself is a very good way to know it.
But like everything you have to spec it well
Then any top model basically for duplicating work.
“Here is component B, here is component A and test A. Produce a test for B following the same pattern”
Personally, my main gripe with it are the response times. But I'm a latency junkie.
I’m a huge fan of keeping things simple by being verbose, and Copilot takes all the drudgery away by filling out all the repeating patterns.
Once in a while when I can’t be bothered to figure out Typescript types, I ask ChatGPT to write one for me.
I have a few observations:
- I vastly prefer Cursor's Copilot++ UX for autocomplete compared to GitHub's in VSCode, which I used until a few months ago.
- The Composer multi-file editor (cmd+i) is easily its most powerful feature and what I use most often, even when I'm working on single files. It just works better for some reason.
- It's far more effective working in popular stacks, eg. Typescript/NextJS etc. It's rarely a time-saver when working in Elixir, for example.
- In a similar vein, the less 'conventional' your task or code is, the less useful it becomes.
- As the context increases, it gets noticeably less useful. I often find myself having to plan what context I want to feed it and resetting context often.
- It's very effective at 'translation' tasks, eg. converting a schema from one format to another. It's much less effective at generating complex business logic.
- I only find it useful to generate code I confidently know how to write myself already. Otherwise, it doesn't save me time. The times I've been tempted, it's almost always bitten me.
Anything to do with Swift concurrency it’s completely hopeless, I assume partly because there’s not enough training data yet.
That is the exact problem I am trying to solve: modifying code with LLMs really sucks most of the time. I am trying a solution with Abstract Syntax Trees: I have the LLM write the code that will write the code you need. That is, modify the source tree rather than the text representation.
I wrote about my approach here: https://codeplusequalsai.com/static/blog/prompting_llms_to_m...
I do have it working for some cases quite well, but there are lots of pitfalls with this approach too. It does take a lot of context and the LLMs aren't really that well-versed at writing specifically esprima code for example. BeautifulSoup does work better, I guess because more people use it and there's more data in the training set.
I'm adding one language at a time, currently have HTML, CSS, Javascript and Python all kind-of working. It's pretty neat but I'm not sure how well it scales yet to larger projects and more difficult requirements/implementations.
- Writing docstrlings. It helps to have a template.
- rubber duck. II try to explain my issue to it. It doesn't find the solution 90% of the time, but explaining the issue helps me (and sometimes you get a hint)
* code translation - e.g. convert a self-contained implementation of a numerical algorithm from one language to another and generate test cases and property tests which make sure the implementations are equivalent. The goal is to avoid having to proof read the generated code.
* one-off scripts - any task where code design doesn't matter, the amount of code is limited to couple hundred lines (GPT-4o) and the result will be thrown away after use.
* API exploration - producing examples for APIs and languages I'm not fluent in. Reading reference documentation gives a better understanding, LLMs get the results out faster.
Whenever I use it with Rust, Golang, Scala etc it's not worth the effort.
On a personal assistant level, it's been useful to have me remind of words that I have forgotten, or to have it rephrase a sentence in a different way, to suit some mood, etc.
Occasionally I have fun with it by having it answer in rhymes, or theatrically like a fortune teller, or like lyrics of some gangsta rap.
I don't know about performance increases. What I notice most is that I'm less annoyed with it, than with the general state of the internet. The web pages and I have different goals: I want a specific piece of information, and they want me to load all their ads, affiliates and such in return, and to spend time on the page reading their drivel. It is also horrible to search for version-specific information, for example, I appreciate that it brings up the Ruby answer for the latest version, but I'm stuck on 2.5.0, and so, I need that specifically. LLMs are usually great at this. Orders of magnitude less fluff, rich text-only answer.
I do really like to use the plain web browser tools, though (currently claude), for generating boilerplate code that I then review and integrate carefully into my code. It has sped up my workflow in that way.
Autocompletions: GitHub Copilot, still it’s better that Mistral small code models in my opinion
Currently I estimate it writes about 70% to 80% code (including adding features to the tool itself), and saved me hours of work per week.
Lately I've been exclusively using API since it is cheaper than paying $20 monthly subscription.
[1] https://prompt.16x.engineer/
- add files, tabs, terminal output and IDE diagnostics into the context via slash commands - feed in documentation or other remote web content, also via simple slash commands - activate workflow mode, which will help you edit the files instead of having to copy things around - then ask questions, ask for refactoring, ask for new features
Often I like to ask for the high level approach, and if I agree with it let it guide the implementation. It makes mistakes, so I always have to validate and test what it creates, but as you add more information into the context and the LLM has a good amount of stuff to work with, the output quality really improves significantly.
It's a process though, and it takes time to get familiar with the workflow, build intuition when the LLM falls on its face, when you should try a different approach etc.
Generally I can echo what others have said, it works best if you already kind of know what you want to achieve and just use the LLM as an assistant that does the grunt work for you, documents the process, provides a rubber duck etc.
Generally, I would not want to work without an integrated LLM anymore, it provides that much value to my workflow. No panacea, no silver bullet, but when used right in the right circumstances it can be incredibly useful.
A secondary usecase for me is working on repositories where tasks and todos are structured in markdown files (even things like travel planning). Letting the LLM Guide you through todos and create a documentation trail through the process, identify important todos, carry along information as you go is wonderful, I would absolutely give that a try as well.
My money is on Cursor [1], which does not stop to amaze me, and seems to get a lot of traction. The integration is very clever, and it is scary how it figures out what I intend to do. Then again, I'm probably doing mundane tasks most of the time. For the few bright moments in my day I tend to use ChatGPT, because most of my real problems are in application domains, not in code.
I am not a firm believer in forking large open-source projects, though, as it will take a lot of effort to keep up with future diversions. This makes me a bit wary of projects such as Cursor and Void [2]. Somebody needs deep pockets to sustainably surpass the popularity of VS Code. To point out just one problem with forking: VS Code works fine in Ubuntu, but Cursor does not work out of the box there. Having to disable the sandbox is a show-stopper for most.
In that respect, the extensions might be a safer bet, and I think Sourcegraph's Cody and Continue are making the largest waves there. Hard to tell with so many waves.
[1] https://www.cursor.com/
[2] https://voideditor.com/
> ...
> My money is on Cursor [1]
Cursor also sends all your data who knows where
While we're on this subject, is there a simple way to ensure that VS Code extensions do not contact external servers?
Unfortunately, no idea, but it's a good question I'd like to have an answer to
I am integrating AI translations into my custom static site generator. I will test the outcome heavily before putting my name (and a big warning) on my translated content, but the early results look good. Getting it right is a lot harder than piping the page through ChatGPT. Everything needs to be translated from the UI strings to the URL slugs.
My work will no longer just benefit English-speaking immigrants, but also locals and immigrants who Google things in other languages. I am very excited about it.
I also use ChatGPT heavily for “what’s the word/expression for” questions and other fuzzy language queries. As a non-native speaker, I want to know if a given expression is appropriate for the context, if it’s dated, if it’s too informal and if it’s dialect-appropriate.
I also use it for coding, but mostly because it’s faster than reading python’s docs. I ask it questions I know the answer to, hoping to find better approaches. So much happened since Python 2.7, and I don’t always know what to ask for.
On occasion I treat it like I treated my mom as a child. I ask it all sorts of questions about the world I observe around me. It’s amazing to get a short, clear answer to work from, instead of sifting through barely relevant, spammy search results. This is super helpful when getting to know new tech and figuring out what it actually is.
It’s just so damn cool to have a sort of real-life Hitchhiker’s Guide slash Pokédex in my pocket. These things appeared in the span of a year, and nobody seems impressed. Well, I am mad impressed.
I like this premise but don’t like the fact that the answer could be completely false - fabricated and presented with equal confidence as the algorithm has zero understanding, just stats.
Code completion was fine without LLMs, and solving problems myself usually ends up being quicker than trying to coerce an LLM into doing it correctly and then verifying that the output is actually correct.
The one time I used an LLM in my workflow to good success was using ChatGPT to automatically create an enum of every EU/EEA country and a switch statement over that enum. Those sort of "grunt work" tasks that don't require any thinking, but a lot of typing, seem to be where LLMs shine.
Sorry but if you then have to check that the list of the Countries is correct and complete, is it not faster to get a list from somewhere safe and then do your usual "Find-Replace" around the separator?
Edit: I just did it for a test: it took less than two minutes, without rush (search engine; open page; find data; open spreadsheet; paste table; paste column in text editor; find-replace around '\n').