Or just don't use Bash. Python is a great scripting language, and won't blow your foot off if you try to iterate through an array.
Other than that, yeah, if you must use bash, set -eu -o pipefail; the IFS is new and mildly interesting idea to me.
> The idea is that if a reference is made at runtime to an undefined variable, bash has a syntax for declaring a default value, using the ":-" operator:
Just note that defaulting an undefined variable to a value (let's use a default value of "fallback") for these examples is,
${foo-fallback}
The syntax,
${foo:-fallback}
means "use 'fallback' if foo is unset or is equal to "". (The :, specifically triggers this; there's a bunch of others, like +, which is "use alternate value", or, you'll get the value if the parameter is defined, nothing otherwise.
if [[ "${foo+set}" == "set" ]]; then
# foo is not undefined.
fi
And similarly,
${foo:+triggered}
will emit triggered if foo is set and not empty.)
See "Parameter Expansion" in the manual. I hate this syntax, but it is the syntax one must use to check for undefined-ness.
This honestly should be the default for all scripts. There are so many little annoyances in bash that would make it great if they were changed and improved. Sadly there's just no changing certain things.
My number one wishlist feature was a simple library system. Essentially just let me source files by name by searching in some standard user location. I actually wrote and submitted patches for this but it just didn't work out. Maintained my own version for a while and it was nice but not enough to justify the maintenance burden. Bash's number one feature is being the historical default shell on virtually every Linux distribution, without that there's no point.
This seems to be posted once per year (at least); however, hardly a complaint as the discussion tends to be high quality.
My personal mantra is if it's over 10~20 lines, I should arguably be using another language, like Python (and perhaps correspondingly, subprocess [2], if I'm in a hurry).
As a seasoned shell programmer, when I see set -euo pipefail, I know immediately that low quality code “batched list of commands”-type code follows.
It’s bad code smell and an anti-pattern as far as I’m concerned. Anyone who has actually has taken the time to learn POSIX/BASH shell to any degree or complexity knows that it’s fully capable of catching errors in-band and that a .sh file isn’t just a dumping ground for repeating your shell history.
The IFS part is misguided. If the author used double quotes around the array reference then words are kept intact:
vs=("a b" "c d")
for v in "${vs[@]}"
do #^♥ ♥^
echo "$v"
done
#= a b
#= c d
Whereas in their (counter-)example, with missing quotes:
vs=("a b" "c d")
for v in ${vs[@]}
do #^! !^
echo "$v"
done
#= a
#= b
#= c
#= d
To paraphrase the manual: Any element of an array may be referenced using ${name[subscript]}. If subscript is @ the word expands to all members of name. If the word is double-quoted, "${name[@]}" expands each element of name to a separate word.
Beware that `set -e`/errexit has quite a few pitfalls.
For example, it's automatically disabled in compound commands that are followed by AND-OR command chains.
This can especially bite you if you define a function which you use as a single command. The function's behavior changes based on what you call after it, masking errors that occurred inside it.
$ myfunc()(set -e; echo "BEFORE ERROR"; false; echo "AFTER ERROR")
$ myfunc
BEFORE ERROR
$ myfunc || echo "REPORT ERROR!"
BEFORE ERROR
AFTER ERROR
It also doesn't always propagate to subshells. (There might be an additional, non-portable option to make it propagate, though.)
$ (set -euo pipefail; x="$(echo A; false; echo B)"; echo "$x")
A
B
EXCEPT for when it sorta does respect the subshell status anyway:
(I think what's happening here is that the assignment statement gets the exit value of the subshell, which is the last command executed in it, without errexit. Subshell completes but top level shell exits.)
And Gods help you if you want to combine that with the `local` keyword, which also masks subshell return values:
$ f()(set -euo pipefail; local x="$(echo A; false; echo B; false)"; echo "$x"); f
A
B
$ f()(set -euo pipefail; local x; x="$(echo A; false; echo B; false)"; echo "$x"); f
<no output>
I like shell scripts. I've yet to find anything with more convenient ergonomics for quickly bashing together different tools. But they're full of footguns if you need them to be truly robust.
16 comments
[ 2.7 ms ] story [ 38.5 ms ] threadOther than that, yeah, if you must use bash, set -eu -o pipefail; the IFS is new and mildly interesting idea to me.
> The idea is that if a reference is made at runtime to an undefined variable, bash has a syntax for declaring a default value, using the ":-" operator:
Just note that defaulting an undefined variable to a value (let's use a default value of "fallback") for these examples is,
The syntax, means "use 'fallback' if foo is unset or is equal to "". (The :, specifically triggers this; there's a bunch of others, like +, which is "use alternate value", or, you'll get the value if the parameter is defined, nothing otherwise. And similarly, will emit triggered if foo is set and not empty.)See "Parameter Expansion" in the manual. I hate this syntax, but it is the syntax one must use to check for undefined-ness.
My number one wishlist feature was a simple library system. Essentially just let me source files by name by searching in some standard user location. I actually wrote and submitted patches for this but it just didn't work out. Maintained my own version for a while and it was nice but not enough to justify the maintenance burden. Bash's number one feature is being the historical default shell on virtually every Linux distribution, without that there's no point.
At least we've got shellcheck.
This seems to be posted once per year (at least); however, hardly a complaint as the discussion tends to be high quality.
My personal mantra is if it's over 10~20 lines, I should arguably be using another language, like Python (and perhaps correspondingly, subprocess [2], if I'm in a hurry).
[1] https://web.archive.org/web/20140523002853/http://redsymbol....
[2] https://docs.python.org/3/library/subprocess.html
bash introduced an option to respect rather than ignore errors within command sub processes years ago. So if you want to be safer, do something like:
That works as-is in OSH, which is part of https://oils.pub/(edit: the last time this came up was a year ago, and here's a more concrete example - https://lobste.rs/s/1wohaz/posix_2024_changes#c_9oo1av )
---
But that's STILL incomplete because POSIX mandates that errors be LOST. That is, it mandates broken error handling.
For example, there what I call the "if myfunc" pitfall
But even if you fix that, it's still not enough.---
I describe all the problems in this doc, e.g. waiting for process subs:
YSH Fixes Shell's Error Handling (errexit) - https://oils.pub/release/latest/doc/error-handling.html
Summary: YSH fixes all shell error handling issues. This was surprisingly hard and required many iterations, but it has stood up to scrutiny.
For contrast, here is a recent attempt at fixing bash, which is also incomplete, and I argue is a horrible language design: https://lobste.rs/s/kidktn/bash_patch_add_shopt_for_implicit...
I've been writing all my Bash scripts like this for ~15 years and it's definitely the way to go.
It’s bad code smell and an anti-pattern as far as I’m concerned. Anyone who has actually has taken the time to learn POSIX/BASH shell to any degree or complexity knows that it’s fully capable of catching errors in-band and that a .sh file isn’t just a dumping ground for repeating your shell history.
Why not simply end each script with true, just as it is started with a shebang and strict mode? Doesn't sound like the worst solution to me.
For example, it's automatically disabled in compound commands that are followed by AND-OR command chains.
This can especially bite you if you define a function which you use as a single command. The function's behavior changes based on what you call after it, masking errors that occurred inside it.
It also doesn't always propagate to subshells. (There might be an additional, non-portable option to make it propagate, though.) EXCEPT for when it sorta does respect the subshell status anyway: (I think what's happening here is that the assignment statement gets the exit value of the subshell, which is the last command executed in it, without errexit. Subshell completes but top level shell exits.)And Gods help you if you want to combine that with the `local` keyword, which also masks subshell return values:
I like shell scripts. I've yet to find anything with more convenient ergonomics for quickly bashing together different tools. But they're full of footguns if you need them to be truly robust.