This might be the "killer app" for AI-based code generation. Small command line tasks often involve ridiculous and inconsistent syntax (e.g. if you have to pipe through different CLI tools), so having some automation to help me skip the Google -> stackoverflow step would be super helpful.
I'd really like to see how this does with nontrivial examples. I'm happy to be proven wrong, but I bet the bash one isn't that effective for real world sysadmin tasks, since most bash things also require knowledge of the commands in-between the actual bash "glue", and usually requires regexes and pipelines to get the exact answer you want. Pretty much every time I look up a bash command (probably from the same sources this model trains on) I have to tweak it to do what I need.
For example, here are some things I pulled from `.bash_history`, and what I think I was meaning to do.
# Get the IP address of running lxc container ${CONTAINER_NAME}
CONTAINER_NAME=some-container; lxc-ls -f | grep ${CONTAINER_NAME} | awk '{print $5}' | sed -z 's|-.*\s||g;'
# Find the latest git tag version, excluding release candidates and development builds
git describe --tags `git rev-list --tags --exclude=*-rc* --max-count=100` | grep -vP '(\-rc|dev)' | head -1
# get the database name from parameters.yml
grep 'database_name:' parameters.yml | sed 's|.*:||g; s|\s||g;'
I bet #3 would would be feasible, but I'd be extremely surprised if it could get close on the first two.
PowerShell has the advantage of a really consistent standard library, syntax, objects, and has the .NET type system behind it, so any answer is probably already in the format you want or requires minimal manipulation. I can see it working much better there, and that's also probably why the example video only shows PowerShell too. Granted, I am probably the opposite of the target audience.
All that being said, would you really trust running some random commands you don't understand enough to write, because some remote server told you to? It should tell you the commands first rather than just executing them.
> It should tell you the commands first rather than just executing them.
From the videos, I believe you have to press Enter to actually execute the suggested command.
> why the example video only shows PowerShell too
The readme did say that it was inspired by a similar project targetting zsh[0], so I don't see why over time the model couldn't improve to suggest what you want.
> From the videos, I believe you have to press Enter to actually execute the suggested command.
Cool, that's good to hear if that's the case.
> The readme did say that it was inspired by a similar project targetting zsh[0], so I don't see why over time the model couldn't improve to suggest what you want.
Interesting, I know very little about all the cool stuff that zsh can do compared with bash, I suppose if it works well for zsh it could probably work for bash then, since zsh is more complex?
Personally I can't see myself ever using these kinds of tools, even if it fit my usecase. In part because I like to figure things out for myself, and also because I don't like outsourcing decisions to unaccountable ML models. I think these are cool ideas, but I think in the long run they will cause more problems than they solve, just like no-code development platforms. IMO all ML models need to be able to explain themselves before we can trust them with anything important.
Disclaimer - I worked on this Codex CLI prototype.
You definitely do have to press enter to execute the command - as with Copilot, you - the developer - can, and should, retain control.
I also like figuring things out myself, but find myself using this tool to help me do so. If I'm using kubectl to run some esoteric command, I'd rather not leave the shell to figure it out - my usage pattern is usually to ask once or twice, and then use that knowledge to handle it on my own the next time
That is a really good use case I didn't think about, since I'm so used to having to leave the terminal to look stuff up in a browser when I can't find something with a manpage or `--help`/`Get-Help`. I'll admit I'm probably being overly skeptical (as I am with a lot of ML things in general), but regardless I think this is really cool, and you guys did a great job on it!
I should probably actually try it instead of trying to second guess it. Is there a way to have it just echo the output instead of having the input ready for you to press enter? Only asking since I'm a clumsy person at the keyboard sometimes :)
You should try those! You can easily create your own few examples of complex behavior (like the ones you already have), add them to a prompt file, load that prompt and try some other hard queries. I've been quite surprised at what it can handle with the right few examples
As cool as this is I would be super hesitant to use this, what if it accidentally runs a command to delete the entire file system or lock me out if my own machine. Too black box.
Disclaimer - I worked on this Codex CLI prototype.
This is totally valid. We built it such that you are still in charge of whether or not you accept the command - what you see are just suggestions, not code that will automatically run. That said, there will be people who accept something they don't understand. We've been thinking about detecting dangerous commands and warning the user to proceed with caution - would this seem like a tenable approach for you?
Every generation seems to think natural language programming is a good idea and comes up with all sorts of ways of trying to do it. They then flail around for a few years before gradually finding out why it sucks and won't work. The space goes dead for a while until the next generation comes along and thinks they spotted a gap in the market.
Disclaimer - I worked on this Codex CLI prototype.
As someone who's worked on natural language interfaces for years this hits hard. The long tail of ways that people express themselves have always made it near impossible to truly build systems that understand our intent and can act on it. I don't think LLMs are a silver bullet that nullify this, but I do think they're exceptionally better than what we've had in the past and that with the right user interface can help us take strides in the right direction.
In the case of the Codex CLI - the goal isn't to eliminate PowerShell, Bash or zsh as computer interfaces, but to assist developers (new and old) as they're using them. The Codex CLI is using Codex off-the-shelf (no customization), and is already quite capable of helping me handle complex shell tasks that I would have taken me much longer to handle in the past. That it can rely on my NL->Code dialog history also gives it context about my task that StackOverflow could never have.
I think the key here will boil down to the user experience - a UX that eliminates shell languages and just takes natural language would be awful, but one that thoughtfully helps you express yourself in terse languages can be helpful.
I’ve been evaluating the Warp terminal emulator as an alternative to iTerm on macOS for a couple months now and they have included a very similar feature to turn an NLP input to a shell command [0].
While for most of my work I’ll shamelessly fall back to Python scripts, basic operations that don’t warrant a script but still require tools like awk, sed, etc that I never learned have greatly benefited from this little tool. I found it far faster than combing through 3-4 StackOverflow answers and hodgepodging my solution together
15 comments
[ 2.9 ms ] story [ 49.9 ms ] threadFor example, here are some things I pulled from `.bash_history`, and what I think I was meaning to do.
I bet #3 would would be feasible, but I'd be extremely surprised if it could get close on the first two.PowerShell has the advantage of a really consistent standard library, syntax, objects, and has the .NET type system behind it, so any answer is probably already in the format you want or requires minimal manipulation. I can see it working much better there, and that's also probably why the example video only shows PowerShell too. Granted, I am probably the opposite of the target audience.
All that being said, would you really trust running some random commands you don't understand enough to write, because some remote server told you to? It should tell you the commands first rather than just executing them.
From the videos, I believe you have to press Enter to actually execute the suggested command.
> why the example video only shows PowerShell too
The readme did say that it was inspired by a similar project targetting zsh[0], so I don't see why over time the model couldn't improve to suggest what you want.
[0]: https://github.com/tom-doerr/zsh_codex
Cool, that's good to hear if that's the case.
> The readme did say that it was inspired by a similar project targetting zsh[0], so I don't see why over time the model couldn't improve to suggest what you want.
Interesting, I know very little about all the cool stuff that zsh can do compared with bash, I suppose if it works well for zsh it could probably work for bash then, since zsh is more complex?
Personally I can't see myself ever using these kinds of tools, even if it fit my usecase. In part because I like to figure things out for myself, and also because I don't like outsourcing decisions to unaccountable ML models. I think these are cool ideas, but I think in the long run they will cause more problems than they solve, just like no-code development platforms. IMO all ML models need to be able to explain themselves before we can trust them with anything important.
You definitely do have to press enter to execute the command - as with Copilot, you - the developer - can, and should, retain control.
I also like figuring things out myself, but find myself using this tool to help me do so. If I'm using kubectl to run some esoteric command, I'd rather not leave the shell to figure it out - my usage pattern is usually to ask once or twice, and then use that knowledge to handle it on my own the next time
I should probably actually try it instead of trying to second guess it. Is there a way to have it just echo the output instead of having the input ready for you to press enter? Only asking since I'm a clumsy person at the keyboard sometimes :)
This is totally valid. We built it such that you are still in charge of whether or not you accept the command - what you see are just suggestions, not code that will automatically run. That said, there will be people who accept something they don't understand. We've been thinking about detecting dangerous commands and warning the user to proceed with caution - would this seem like a tenable approach for you?
Why will it be different this time?
As someone who's worked on natural language interfaces for years this hits hard. The long tail of ways that people express themselves have always made it near impossible to truly build systems that understand our intent and can act on it. I don't think LLMs are a silver bullet that nullify this, but I do think they're exceptionally better than what we've had in the past and that with the right user interface can help us take strides in the right direction.
In the case of the Codex CLI - the goal isn't to eliminate PowerShell, Bash or zsh as computer interfaces, but to assist developers (new and old) as they're using them. The Codex CLI is using Codex off-the-shelf (no customization), and is already quite capable of helping me handle complex shell tasks that I would have taken me much longer to handle in the past. That it can rely on my NL->Code dialog history also gives it context about my task that StackOverflow could never have.
I think the key here will boil down to the user experience - a UX that eliminates shell languages and just takes natural language would be awful, but one that thoughtfully helps you express yourself in terse languages can be helpful.
While for most of my work I’ll shamelessly fall back to Python scripts, basic operations that don’t warrant a script but still require tools like awk, sed, etc that I never learned have greatly benefited from this little tool. I found it far faster than combing through 3-4 StackOverflow answers and hodgepodging my solution together
[0]: https://docs.warp.dev/features/ai-command-search