After seeing the demos my biggest question is: how will this not lead to people just accepting whatever GitHub Copilot suggests while introducing subtle yet catastrophic errors? Basically, how is this not going to become an alternative to just copy-pasting StackOverflow answers without verification? Especially given the IDE integration…
And to at least partially answer my own question, straight form the FAQ:
> Can GitHub Copilot introduce insecure code in its suggestions?
> There’s a lot of public code in the world with insecure coding patterns, bugs, or references to outdated APIs or idioms. When GitHub Copilot synthesizes code suggestions based on this data, it can also synthesize code that contains these undesirable patterns. This is something we care a lot about at GitHub, and in recent years we’ve provided tools such as Actions, Dependabot, and CodeQL to open source projects to help improve code quality. Similarly, as GitHub Copilot improves, we will work to exclude insecure or low-quality code from the training set. Of course, you should always use GitHub Copilot together with testing practices and security tools, as well as your own judgment.
Basically, they seem to hope that people will either be really careful about the suggested code or have existing code analysis workflows that would catch errors
Like any tool, you still need to understand the fundamentals to use it well. Anyone can use a table saw, but if you don't know what you're doing, it can be really easy to cut off a finger.
I use TabNine for this. It also uses a (probably smaller) language model trained on code and looks at the code in your project for completions.
The models are run local by default. They also have a cloud option for bigger models.
I'm being a little facetious here because the fact Copilot can generate something that I can complain about is truly amazing, but the suggested code for the 'calculate days between two dates' example is really horrible. It's going to go quite wrong any time the two dates cross a Daylight Savings Time boundary, or if the Date objects are using two different timezones, or if either date isn't a Date, etc.
function calculateDaysBetweenDates(date1, date2) {
var oneDay = 24 * 60 * 60 * 1000;
var date1InMillis = date1.getTime();
var date2InMillis = date2.getTime();
var days = Math.round(Math.abs(date2InMillis - date1InMillis) / oneDay);
return days;
}
It's definitely as good as the code I was writing for the first few years of my career though, and I can see that it could improve significantly with more data. I'm absolutely looking forward to seeing where it goes.
Point is that the average does in fact write code worst than that (including myself). Having this kind of stuff automated in such a way that it is somewhat "looked after by GitHub" will increase the quality of software overall.
Except for the fact that the cognitive load on an engineer reading/debugging foreign code is an order of magnitude higher than that of an engineer who wrote the code doing the debugging.
This looks cool but also feels like the wrong way to reuse code. It is a form of automated intelligent copy pasting. Good code reuse is done by using nicely packaged libraries, not by duplicating code across projects.
19 comments
[ 37.4 ms ] story [ 2437 ms ] threadAnd to at least partially answer my own question, straight form the FAQ:
> Can GitHub Copilot introduce insecure code in its suggestions?
> There’s a lot of public code in the world with insecure coding patterns, bugs, or references to outdated APIs or idioms. When GitHub Copilot synthesizes code suggestions based on this data, it can also synthesize code that contains these undesirable patterns. This is something we care a lot about at GitHub, and in recent years we’ve provided tools such as Actions, Dependabot, and CodeQL to open source projects to help improve code quality. Similarly, as GitHub Copilot improves, we will work to exclude insecure or low-quality code from the training set. Of course, you should always use GitHub Copilot together with testing practices and security tools, as well as your own judgment.
Basically, they seem to hope that people will either be really careful about the suggested code or have existing code analysis workflows that would catch errors
https://news.ycombinator.com/item?id=27676266
Hmm... this looks like it could be a useful tool but I'm wary of its use devolving into an all-purpose hammer.