Woah! I didn't know about the `command` trick. From the article -
function echo { echo "$@"; }
Oops don't do that, it will recurse until the process is killed. But if you do successfully redefine a builtin command, you can call the original with command. So this is fine:
function echo { command echo "$@"; }
Unless you also redefine the command command, then you're really screwed.
"command" works also interactively, to call the "real" thing instead of the aliased thing.
As an example, I alias "cat" to run "bat", which pretty-prints the output. If I don't want the pretty-printed output, I can "command cat foo.txt" and it will call the "real" cat.
I like to escape aliases (ie. using `\vi` when vi is aliased to $EDITOR, but I actually want it), and actually have a special case for bat where I bind ccat to `bat -p --pager=never` for nice, copy-able, and still potentially highlighted output.
On Zsh there is also '=' as prefix, at least that's what I normally use, don't recall if it's a default setting or the result of a local configuration...
The `=` prefix in zsh works, but is a different beast entirely. The backslash trick will simply stop the command parser from expanding the alias, while the equals prefix will expand to the full path of the given command at parsing time. For example:
$ print -l =cat =bat
/usr/bin/cat
/usr/bin/bat
While it is a trivial difference in your use case, it can be useful in itself; You could use `ldd =zsh` instead of needing to provide the full path, or `dpkg -S =make` to search for /usr/bin/make(or whatever) instead of returning results of files with the string make anywhere in the name.
It is a default setting for the expansion system(only when in zsh mode), but can be disabled with `unsetopt equals` if you dislike it. Gory details in zshexpn(1) and zshoptions(1).
Reminds me of the time I was on a Lisp Machine and did
(trace format)
This causes a line to be printed every time FORMAT is called, showing the arguments it was called with. Well, FORMAT is Lisp's printf -- it's used everywhere, including by TRACE. Oops!!! Time for Ctrl-Meta-Ctrl-Meta-Rubout! (That was the key sequence for an immediate reboot. Interesting how it prefigured the IBM PC's Ctrl-Alt-Delete.)
I’m a very big bash scripter - to the disdain of many younger colleagues. They prefer golang, typescript, node. I find it hard to relate. I write bash scripts to automate batch commands I find myself running as part of my job. I like the terminal and posix utilities. Very flexible and easy to read, to me.
I find that Golang is an especially good companion for Bash, because it is quite easy to write and deploy your own CLI tools for task which have higher performance requirements.
I'm preparing to release bash-modules 4.0 [0]. Can you give me feedback, please? I'm a non-native English speaker, so I need someone to help fix spelling mistakes, at least.
IMHO, you should use unofficial strict mode (set -ueEo pipefail; trap "Error at ..." ERR ) often, because it catches a lot of bugs. Sadly, strict mode doesn't work in subshell.
Also, code without comments and documentation is useless, because developer must invest some time into browsing and understanding of your code first, but, often, it faster to rewrite code from scratch instead.
Also, you forget to include a license for your code, so nobody can use your code for any purpose.
I like the idea of downloading of modules at demand, with verification of integrity. I'm looking to implement something like that for my project too, but I'm afraid that I will not be able to implement it securely enough.
One difference between the projects is that you are targeting Bash and I am targeting POSIX. Therefore, using the 'unofficial strict mode' is not an option for me. However, good tests can compensate a lot and are necessary anyway.
Yes, there are certainly some modules which lack comments. On the other hand, there is at least one module which has its own readme to explain the code:
Regarding the security: As said, I am targeting POSIX and as far as I know, the only checksum tool available is cksum. This is borderline insecure but catches transmission errors at least. However, if you do not have such strict requirements you can easily use something like sha256sum. In fact, my module allows to use alternative checksum tools but uses cksum if nothing else is specified (because it is better than nothing and always available).
Well, it certainly has some strong parts, but there are some parts which are so horribly broken, that the good parts can't keep up.
Just to give one example: The topic of errexit aka `set -e`.
In the beginning, I thought it should be an easy decision and since programs are supposed to be reliable, they should terminate in case of an error. So I added `set -e` to all my scripts. However, later I learned, that if you call a function from within a condition, the function is executed without `set -e` even if the function itself sets the option. Since then, I am unsure if it is better to have a function that 'sometimes exits' when it comes across an error or 'never exists' when it comes across an error, because 'always exists' when it comes across an error does not exist.
I love Bash too, but using it daily is probably more inertia than anything else. Some things are just downright horrible (for the life of me, I can never remember associative array syntax off the top of my head) but it's just so good when it comes to other things that it's worth sticking to.
I love bash scripting, but never knew about the ability to redefine existing commands. It made me think that it could be used maliciously, for example redefining cp so that sends a file to a remote server as well as copying it locally.
It absolutely can be used maliciously, but if an attacker can add an alias to the environment, they might also be able to manipulate PATH, making a malicious cp take precedence over the original cp. So the point is moot.
That's not really a threat model to be concerned about, if someone can define bash aliases for you you're already running their arbitrary code, so them defining bash aliases for you should be the least of your concerns.
Also, function body can be any compound command, so instead of braces {}, you can have parentheses (), which causes each function call to spawn a subshell. Very useful if your function needs isolated state or you change working directory and don’t want to affect global state. You can have exit command in such function, because it terminates the subshell only, not the whole script.
for, while and until are also compound commands, so they also can be used as function body directly without braces, but there’s no practical use for that, really.
Part of the mystery of the infamous shell fork bomb is that it picks a weird name like ':' for the function name.
:(){ :|:& };:
Turns into a less mysterious:
bomb() {
bomb | bomb &
}
bomb
I started a booklet with some of these tricks and explanations in https://raimonster.com/scripting-field-guide/, in case anyone wants to give feedback (or PRs), it'd be much appreciated.
I'll take this chance to ask a fork bomb question I always had: on Linux, is there a way to stop a fork bomb once it gets going, other than turning off the power?
> Bash has a handy POSIX mode which you can run your code under if you’re not sure if it’s portable or not.
That is not what POSIX mode is for. POSIX mode is to let POSIX scripts work, not to let non-POSIX scripts fail. The latter is done only insofar as it is necessary to achieve the former, so POSIX mode still leaves plenty of extensions enabled, which is fine for its intended use.
It's really nice to be able to use : to group up commands in a single file script that auto-converts functions to commands[0], such as ci:install-deps and ci:test. This has been a pattern I've been doing for a while now.
> The precise rules for function names are murky; mostly it seems like whatever Bash can parse unambiguously is allowed
> What is this useful for?
I think it makes sense because bash function invocation is the
same as calling an external program which name is only restricted
by what is allowed as a file-path.
Bash function names also have a different namespace than Bash variables, so you can define a function and a shell variable with the same name. I used that in a script which defines a function to be used like an alias to change to a specified directory, but where the name is also set as a shell variable that can be evaluated with $ to give the directory's pathname.
49 comments
[ 3.0 ms ] story [ 104 ms ] threadAnd there can rarely be fun side effects.
As an example, I alias "cat" to run "bat", which pretty-prints the output. If I don't want the pretty-printed output, I can "command cat foo.txt" and it will call the "real" cat.
"\ls" is the same as "command ls" and will run ls ignoring any functions or aliases that normally replace it.
The `=` prefix in zsh works, but is a different beast entirely. The backslash trick will simply stop the command parser from expanding the alias, while the equals prefix will expand to the full path of the given command at parsing time. For example:
While it is a trivial difference in your use case, it can be useful in itself; You could use `ldd =zsh` instead of needing to provide the full path, or `dpkg -S =make` to search for /usr/bin/make(or whatever) instead of returning results of files with the string make anywhere in the name.It is a default setting for the expansion system(only when in zsh mode), but can be disabled with `unsetopt equals` if you dislike it. Gory details in zshexpn(1) and zshoptions(1).
And lots of people use Nix - and write a lot of bash.
[0]: https://github.com/vlisivka/bash-modules
It is still a work-in-progress project and the documentation on the website is not up-to-date, but I think we are working on the same problem ;-)
If you want to run probably all the code I have written, you can run the test-suite like:
A total run-time of 8 Minutes is normal and there is little output in the beginning.The code is available on Github[2].
[1]: https://module.sh
[2]: https://github.com/arendtio/mdl.sh
Also, code without comments and documentation is useless, because developer must invest some time into browsing and understanding of your code first, but, often, it faster to rewrite code from scratch instead.
Also, you forget to include a license for your code, so nobody can use your code for any purpose.
I like the idea of downloading of modules at demand, with verification of integrity. I'm looking to implement something like that for my project too, but I'm afraid that I will not be able to implement it securely enough.
Yes, there are certainly some modules which lack comments. On the other hand, there is at least one module which has its own readme to explain the code:
https://github.com/arendtio/mdl.sh/tree/master/development/d...
Regarding the security: As said, I am targeting POSIX and as far as I know, the only checksum tool available is cksum. This is borderline insecure but catches transmission errors at least. However, if you do not have such strict requirements you can easily use something like sha256sum. In fact, my module allows to use alternative checksum tools but uses cksum if nothing else is specified (because it is better than nothing and always available).
Just to give one example: The topic of errexit aka `set -e`.
In the beginning, I thought it should be an easy decision and since programs are supposed to be reliable, they should terminate in case of an error. So I added `set -e` to all my scripts. However, later I learned, that if you call a function from within a condition, the function is executed without `set -e` even if the function itself sets the option. Since then, I am unsure if it is better to have a function that 'sometimes exits' when it comes across an error or 'never exists' when it comes across an error, because 'always exists' when it comes across an error does not exist.
for, while and until are also compound commands, so they also can be used as function body directly without braces, but there’s no practical use for that, really.
This limit isn't set by default in most distributions, but does exist.
It would still crash the bombed shell but the server itself would be safe and a manual kill signal could be send to the bombing thread.
rethoric -> rhetoric
1 off -> one-off
ment -> meant
low hanging -> low-hanging
flag you many of those ticking bombs -> flag many of those ticking bombs for you
No matter which editor you are using, but you -> No matter which editor you are using, you
The best example in that StackOverflow is this one: -> [maybe delete this line? or write something else. I don't know its purpose, it makes no sense.]
a bunch of elements from one go -> a bunch of elements in one go
functions in shell -> shell functions
using user's input -> using user input/using the user's input
you can unset -> You can unset
lots of code you see around rely on -> lots of code you see around relies on
to thinking on creating -> to thinking about creating
That is not what POSIX mode is for. POSIX mode is to let POSIX scripts work, not to let non-POSIX scripts fail. The latter is done only insofar as it is necessary to achieve the former, so POSIX mode still leaves plenty of extensions enabled, which is fine for its intended use.
https://www.shellcheck.net/
[0]: https://nickjanetakis.com/blog/replacing-make-with-a-shell-s...
> What is this useful for?
I think it makes sense because bash function invocation is the same as calling an external program which name is only restricted by what is allowed as a file-path.