I ran some tests some time ago, and the differences are pretty minimal unless you start doing comp-sci-y stuff in shell scripts. But for the type of thing that people typically use shell scripts for: it makes basically no meaningful difference.
For systems that I control myself I much prefer to avoid Bash/sh. They’re just to clunky. And if I need to use them, I try to do as little as possible in order to make it more robust.
Case in point: Declaring an array. IMHO, it’s just not ergonomic at all. Especially not in sh/dash.
Missing: When using "set -o pipefail" you should also catch any non-zero return codes that you want to accept, e.g., "{ grep -o pattern file || true ; } | sed pattern" to let the command continue (if desired) to execute even if pattern isn't found.
Shell scripts are great for executing a series of commands with branching and looping logic around them.
As soon as output needs to be parsed — especially when it’s being fed back into other parts of the script — it gets harder. Handling errors and exceptions is even more difficult.
Things really fall down on modularity. There are tricks and conventions: for example you can put all functions to do with x in a file called lib/x.sh, prefix them all with x_, and require that all positional parameters must be declared at the top of each function with local names.
At that point though, I would rather move to a language with named parameters, namespaced modules, and exception handling. In Python, it’s really easy to do the shell bits with:
which will let you pass in arguments with spaces and be able to access them as properly lexed arguments in $1, $2 etc in your script. You can even preprocess the script to be prefixed with all the usual set -exuo pipefail stuff etc.
'pushd' would imply that you want to 'popd' back out of it, but that's unnecessary, as the 'cd' will only affect a subshell that gets terminated at the end of the script. So for the user it makes no difference, the current directory stays the same. For the script it saves you an unnecessary 'popd'.
pushd/popd are intended for interactive use, not for use in scripts. It prints the full stack on directories and there is no option to be quiet. Of course, there is always redirecting to /dev/null but it is intentional to not have option to be quiet.
Usually there is no need to return to original directory. Change of directory is process-local (script-local) so the calling process is not affected by this 'cd' in the script.
I've rewritten a lot of shell scripts with awk. Obviously it's not a good fit for everything, but when it is a good fit I found it a very pleasant experience. In spite of using Unix systems for 20 years I only learned awk a few years ago and I really beat myself up for not learning it earlier.
Not really; just seems the same as most other dynamic languages. Awk does a lot of stuff for you (the "implied loop" your program runs in, field splitting) that's certainly possible (even easy) to replicate in Python or Ruby, but Awk it's just so much more convenient.
I use it for things like processing the Unicode data files, making some program output a bit nicer (e.g. go test -bench), ad-hoc spreadsheets, few other things. I got started with it as I needed to process some C header files and the existing script for that was in Awk; it worked pretty well for that too.
Bash extensions are cool to have in the interpreter. I really don't think we need more than the basic POSIX shell for most scripts. I once wrote a tar replacement in it, with a restricted YAML generator and parser and a state machine — don’t judge me, I think I was manic — and the result was weirdly beautiful.
The lack of arrays, dicts and local variables when trying to be POSIX compliant becomes quickly annoying when writing big programs though. There are of course workarounds to deal with those but I wish we didn't have to use them.
POSIX sucks when shells aren't implementing it correctly. POSIX says that PS1 expansion needs to support at least the bang ('!') expansion and the regular parameter expansion ('$' and '${'), but i found that several kinds of almquist shells don't support that even when they are explicitly POSIX-compliant, dash included.
Do you even ran your code in place where bash wasn't available? I held thought like you... 10 years ago but that really doesn't happen and if it does, rest of it probably won't work either...
Yes. Alpine ash shell (the default),macOS zsh shell (the default), and Oracle Solaris sh shell (the default). The systems are enterprise regulated, so a typical user cannot easily install a different shell. POSIX works great.
I see this argument a great deal here on hacker news. I think it's more important that the scripter understands their use case and develop accordingly. Posix portability is almost never a factor or the things that I develop, because we have a near monoculture in terms of operating system and version.
For me, the additional features that bash provides are much more important than a portability that I'll never need to use.
> If appropriate, change to the script’s directory close to the start of the script.
> And it’s usually always appropriate.
I wouldn't think so. You don't know where your script will be called from, and many times the parameters to the script are file paths, which are relative to the caller's path. So you usually don't want to do it.
In e.g. "Read the great Oil Shell blogpost." it's not clear there's a link there: the "blogpost" is a link but you only see that if you hover your mouse.
I agree. I make an effort to not change directory wherever possible and if a change is needed, do it in a subshell and just for the command that needs it (hardly any commands actually need it, anyway).
Edit: just had a quick look at your recommended link and spotted a "mistake" in 4.7 - using "read" without "-r" would get caught out by shellcheck.
> [[ ]] is a bash builtin, and is more powerful than [ ] or test.
Agreed on the powerful bit, however [[ ]] is not a "builtin" (whereas [ and test are builtins in bash), it's a reserved word which is more similar to if and while.
That why [[ ]] can break some rules that builtins cannot, such as `[[ 1 = 1 && 2 = 2 ]]` (vs `[ 1 = 1 ] && [ 2 = 2]` or `[ 1 = 1 -a 2 = 2 ]`, -a being deprecated).
Builtins should be considered as common commands (like ls or xargs) since they cannot bypass some fundamental shell parsing rules (assignment builtins being an exception), the main advantages of being a builtin being speed (no fork needed) and access to the current shell process env (e.g. read being able to assign a variable in the current process).
What would be the justification for 'cd "$(dirname "$0")"'? Going to the scripts directory does not seem very helpful. If I don't care about the current directory, I might just go to '/' or a temporary directory, when I do care about it I better stay in it or interpreting relative command line arguments is going to get difficult. When symbolic links are involved, dirname will also give the wrong directory.
It's sometimes a bit convenient if you want to read file from the directory the script is stored in. Overall I found it more confusing and awkward than anything else, and prefer setting it explicitly. It's still okay for a quick script though, but as general "best practices" advice: meh.
I also think so. Often script needs to access a file in actual current dir (for example a config file) or process files with relative paths supplied by user and changing working dir makes this hard.
I think an easier way is to find script's location and construct paths for accessing script dependencies, for example (works on Linux & macOS):
I agree, getting to know where a script "comes from" can be complex though. You can `readlink -f` (or equivalent) in many cases, but when implementing a library this might not be entirely practical. I have had to rely on this ugly if-statement [1] for that purpose.
To be fair, it's common to want to be in the script directory for certain classes of scripts. For example, scripts which automate some tasks in a project, and are written for a project.
But, more importantly, people will google for how to set cwd to the script directory more often then will google how to go to an absolute path. Having 'cd "$(dirname "$0")"' as reference in an article discussing best practices and the topic of changing the directory early, is a good idea.
And for certain classes of users, certainly. If I were in $prj/some/dir and called “../../script.sh foo”, I’d expect it to operate on $prj/some/dir/foo, not on $prj/foo. The latter would be a confusing practice, not even remotely the best one.
people will google for how to set cwd to the script directory more often
The answer should suggest setting $script_dir instead of chdir and refer to it when needed, explaining why chdir is a wrong shell mindset except for a very few special cases. It’s okay for personal use, but inheriting such scripts would be an awful experience, imo.
I can only agree with this. In my experience, everybody coming from a lifetime with windows has to learn that a "working directory" has a very real and everyday meaning in Linux.
Windows software just isn't designed that way because it usually has GUIs and "Open file" dialogs and doesn't use the cwd mostly. Most shortcuts even execute software in their installation directory (because it's most compatible with developers using relative paths for their assets and ignoring the existence of the cwd mechanic altogether, I guess?).
So for somebody just coming from Windows, this isn't just the wrong mindset for shell scripts, more dangerously it's a mindset they find appealing, because it matches their prior experience on Windows better.
My theory is that this same Windows experience is also how so many joined the command-with-filename-extension cargo cult. But unix and windows work differently, and the approach does NOT port.
> To be fair, it's common to want to be in the script directory for certain classes of scripts. For example, scripts which automate some tasks in a project, and are written for a project
I think it would be more correct to just use vcs to get the root of a project (or fail if you can't), instead of essentially hardcoding path to the script.
For example if you put your script in helpers/ then someone else did a refactor and moved all of the stuff into cmd/helpers/, any relative reference you put into script is now invalid and your script is doing the wrong thing
No, this is pretty meaningless unless sourcing in function.sh or something. If your script is short lived and doesn't have a work area (read/writing files), don't bother with cd. If it's long lived, the best default would be "cd /" or "cd /tmp" - sadly, since bash seem to mmap the script, this still doesn't free up the filesystem for unmounting. Python is different, and "cd /" is a good default for a long-lived program.
> Use bash. Using zsh or fish or any other, will make it hard for others to understand / collaborate. Among all shells, bash strikes a good balance between portability and DX.
I think fish is quite a bit different in terms of syntax and semantics (I'm not very familiar with it), but zsh is essentially the same as bash except without most of the needless footguns and awkwardness. zsh also has many more advanced features, which you don't need to use (and many people are unaware of them anyway), but will very quickly become useful; in bash all sorts of things require obscure incantations and/or shell pipelines that almost make APL seem obvious in comparison.
In my experience few people understand bash (or POSIX sh) in the first place, partly because everything is so difficult and full of caveats. Half my professional shell scripting experience on the job is fixing other people's scripts. So might as well use something that doesn't accidentally introduce bugs every other line.
Most – though obviously far from all – scripts tend to be run in environments you control; portability is often overrated and not all that important (except when it is of course). Once upon a time I insisted on POSIX sh, and then I realised that actually, >90% of the scripts I wrote were run just by me or run only in an environment otherwise under my control, and that it made no sense. I still use POSIX sh for some public things I write, when it makes sense, but that's fairly rare.
I think bash is really standing in the way of progress, whether that progress is in the form of fish, zsh, oil shell, or something else, because so many people conflate "shell" with "bash", similar to how people conflate "Google" with "search" or "git" with "GitHub" (to some degree).
I don't know fish, but I don't consider zsh a step in the right direction, as it tries to be just a cleaned up Bash, which is not enough.
There is a general problem in the fact that a radical evolution of glue languages wouldn't be popular because devs rather use Python, and small evolutions wouldn't be popular (ie. zsh), because they end up being confusing (since they're still close to Bash) and not bringing significant advantages.
I'm curious why there haven't been attempts to write a modern glue language (mind that languages like Python don't fit this class). I guess that Powershell (which I don't know, though) has been the only attempt.
zsh is not a "cleaned-up bash"; it's more of a clone of ksh (closed source at the time), with some csh features added in, as well as their own inventions. bash and zsh appeared at roughly the same time, many features were added in zsh first and added to bash later (sometimes much later, and often never).
This is kind of a good example of what I meant when people conflate "bash" with "shell".
As for your larger point: I kind of agree, but I think what zsh offers is the advantages of shell scripts with compatibility with existing scripts while still improving on it. That said, I believe oil also offers compatibility, but I haven't had the chance to look deeply in to it; just haven't had the time, and wanted to wait until it's stable (maybe it is now?)
Perl was initially invented as the "modern glue language" to replace shell. It's fallen a bit out of fashion these days though, and to be honest I never cared all that much for Perl myself either. Raku looks nice though. TCL also works well as a kind of "glue language", although it has some really odd behaviour at times due to everything being a string and I know some people hate it with a passion, but it always worked fairly well for me. But that has also fallen out of fashion.
I've also been told PowerShell is actually quite nice and has interesting concepts (and now also open source, and you can run it on e.g. Linux), but I could never get over the verbosity of it all. I'm an old unix greybeard and I want my obscure abbreviations dammit!
Just so you know. You can abbreviate almost anything in powershell or make your won aliases. I love Powershell, hands down best investment in my personal career was to really learn and understand Powershell.
The verbosity of PowerShell is overstated I think. You easily make POSH look as gnarly and esoteric as Bash if you so desire. That said, the majority of heavy lifting in POSH is done via methods these days (vs cmdlets). Your initial API query to snag the JSON might be via a cmdlet, but after that, you're slicing and dicing with real data structures. You can interact with them without having worry about whitespace or structure (meaning complex loops can easily be written on the command line without worrying about indentation).
It's a little more wordy if you're use to C or Bash. But hands down it's one of my favorite languages for slicing dicing data. No need for 3rd party libraries or binaries. No need to learn a bunch of weird awk/jq syntax which is only useful for those two tools (yay, lets learn 3 language instead one?). Plus, most of the structure translates over to C#, and you can integrate C# code directly into your POSH code if desired, as well as access pretty much any low level C# methods directly.
Working with strings? Pretty much any/every tool you could want to slice and dice strings.
The POSH REPL is amazing. You have far more flexibility around interacting with the command line than you do with Python. It's both a shell and a true language. As with any language, there are ISMs, but far fewer footguns than any other language I've spun up.
Cross platform as well with 6.0+
Intellisense ON the commandline (did I mention the awesome REPL?). Hands down one of the best built-in parameter/args/help parsing I've encountered across any language. Debugging? Amazing in vscode. And can be done strictly from the commandline as well (dynamic breakpoints? You've got it, drop you right into your catch block with an interactive shell so you can see the current status of any/all variables and manipulate them live and resume if desired)
Okay, I'm done shilling for POSH. It's hands down one of my favorite shells/languages for doing POC work, or writing utility functions. Treat it more like Python than bash. But realize that you can easily use that Pythonic-esque code right inside your shell.
I'm a Unix user and spent almost all of my professional career in bash, and switched to Powershell for my interactive shell a few years ago.
The nice thing with Powershell is that it's not verbose, but the arcane abbreviations are actually quite a bit easier to remember and discover than bash. What mixes people up is that in documented examples and reusable scripts, it makes sense to use the full, canonical name, which looks aesthetically different coming from a Unix background.
Here's what it might actually look like to check a JSON file that has an array of file metadata objects, and delete the ones that have been processed (this includes one user-defined alias, cfj, for "ConvertFrom-Json"):
gc queue.json | cfj | where status -eq processed | ri
That seems pretty NON-verbose to me, equivalent to how you'd approach this in bash. Do you have jq installed? If you do, perhaps:
If you don't have jq I think this gets much longer and is either really brittle (you're doing some kind of adhoc parsing that's terrible) or really heavyweight (like pulling in a pure bash JSON library--they exist) or you're using a one-liner from another programming language (Ruby, Python, something). Or maybe you'd complain to whatever was writing 'queue.json' and ask for a friendlier format, like something terrible and one-off you invented because it's easy to "parse" with awk?).
It's even better if you're dealing with network resources:
What probably makes you think Powershell is verbose is that, although that's how I type Powershell, it's not how I document it. If I were documenting it for someone else's use, or incorporating that pipeline into a script or module, I'd write it like this:
So it's not like it's verbose while you're using it, but verbosity is something you can reach for for clarity when it's desirable. Likewise, you can see the consistent relationship between these commands and their aliases: gc -> Get-Content, cfy -> ConvertFrom-Yaml, ri -> Remove-Item. So you have options for how verbose you need to be. I find it's very useful to have a spelled-out version for commands I don't use all the time, like 'Get-EC2Instance', and consistent verbs so I can make reasonable guesses at the command name for things I'm even less familiar with.
I didn't want to clutter this with too many examples but I'll reiterate that Powershell is a shell. It invokes your Unix commands just fine. For example, if you forgot what the option to Get-Date is, to get a Unix-style timestamp (it's 'get-date -uf %s' so not exactly hard to use): you can just type 'date +%x'. In the example above, I could have used 'cat' and 'xargs rm' instead of 'gc' and 'ri'. So it's not like you have to buy the whole kit and kaboodle right away, either.
zsh is, historically, a step from csh-like interactive shells in the direction of Bourne/ksh compatibility. It's easy to get the impression that zsh is a newer development than bash, but they're actually contemporary—bash rode on the popularity of GNU in the 90s, despite being a "small evolution" (frankly, a step back) compared to the ksh lineage.
tcsh was the default shell before that, and it didn't help much with its popularity, and for interactive usage tcsh can do most of the things bash can and is mostly okay (not scripting though).
I think being the de-facto default on Linux as part of "GNU plus Linux" has more to do with it.
A little personal color: I’m kind of a terminal tweak-fanatic but I’ve stuck with bash.
Ten years or so ago the cool kids were using zsh: which is in general a pretty reasonable move, it’s got way more command-line amenities than bash (at least built in).
Today fish is the fucking business: fish is so much more fun as a CLI freak.
But I guess I’ve got enough PTSD around when k8s or it’s proprietary equivalents get stuck that I always wanted to be not only functional but fast in outage-type scenarios that I kept bash as a daily driver.
Writing shell scripts of any kind is godawful, the equivalent python is the code you want to own, but it’s universality is a real selling point, like why I keep half an eye on Perl5 even though I loathe it: it may suck but it’s always there when the klaxon is going off.
The best possible software is useless if it’s not installed.
I personally really dislike fish as an interactive shell as it's just so busy. Things keep popping up, everything is in so many different colours, etc. It's great if you like that sort of stuff, but I really appreciate a "quiet" environment. This is also why I use Vim: all the IDEs I tried are just so "busy".
I was only talking about scripting; I know fish scripting is different, but I have no idea if it's any good. For interactive shells I don't care what people use: it's 100% a personal choice.
Quite a few things can't be disabled; for example AFAIK it doesn't offer a way to disable the autocomplete altogether, or the "fuzzy" matching. I really dislike these things. Fish is a great shell, but very opinionated which is great if your preferences align with that, and not-so-great if they don't. Which is fine because it makes the project better for those who do want these things, and not every project needs to cater to everyone.
Export $BASH_SILENCE_DEPRECATION_WARNING as described in the Apple web page pointed to by the nag message, or change your shell to your own version of Bash.
=name will expand to the entry in your PATH. e.g. =ls expands to /usr/bin/ls. So == expands to an executable named =, or rather, it tries to as you probably don't have = in your PATH.
[[ ]] disables expansions (e.g. [[ * = * ]] will work too) so it's not an issue there.
I feel like powershell hides too much to be used regularly. I have a dozen of small shellscripts and aliases to do basically what PS help me to do when i work on windows (and some), but at least i know how it work behind.
I had to work with Sencha/ExtJS early 2010. It was the same feeling. Yes, it is powerfull, but too much magic happen for something without a clear orientation (at the time, now it is used for data loaded frontend i think). PS i don't understand what it wants me to do.
The language is fine. The interface is now fine, but in 2015 it was the shittiest tty available on modern computers. It's okay since at least early 2021 (when i restarted using windows). I know it should be reasonably better, but i wouldn't trust any PS script written before 2021 to run on my workstation. I run bash scripts i wrote when i started coding.
Still, if you're new to the gig and don't care about free software and commons, you should learn PS (unless you want to work on baremetal or on MC, in this case, bash will be enough).
> I had to work with Sencha/ExtJS early 2010. It was the same feeling. Yes, it is powerfull, but too much magic happen for something without a clear orientation (at the time, now it is used for data loaded frontend i think). PS i don't understand what it wants me to do.
Bit off-topic, but I worked with ExtJS around the same time, and I found it one of the most confusing development experiences I ever had. "It's so easy, just add this one property to this deeply nested object you're passing to this function!" Thinking back on it, it's a really good example of "simple vs. easy". It didn't help it gave not a peep if you got one of those data structures wrong (capitalisation typo, wrong location, etc.)
Plus in hindsight the whole idea of "OS semantics in your browser!" was never a good one to start with, although that wasn't as obvious to me at the time.
Yeah, exactly, powershell is easy, but not simple!
ExtJS (on netbeans with windows server 2004) was always my worst development experience, it was my first internship too. That's probably the reason why it took so long (and a bag of money) for me to try JS, an IDE and developing on windows again (I only used Windows for CTFs and AndroidStudio).
I recently replaced a bit of code to look up locked files for a file share with SMB cmdlets to do the same.
The performance difference was night and day.
The biggest issue with PowerShell is that PowerShell Core is not yet default on Windows 10/11 and Windows Server.
That should be Microsofts highest priority for PowerShell.
Then not hobble who can run powershell scripts out of the box. Which makes it seem like a dangerous tool. Then no one wants to use it. Some form of powershell has been there since win7. Yet in one of the versions they decided 'oh only admins can use this unless you run this special command'. So it makes me have to revert to using CMD scripts for some simple things. Because I do not want to have to walk whoever it is thru enabling powershell.
It's mostly a scripting language whose commands communicate with each other, which also let's you issue commands interactively. In an actual shell, programs and commands communicate with the _user_ via textual output that the user can read and understand; and other programs can take the place of the user, reading and parsing the textual output instead. That's not what happens in PowerShell, mostly.
I tried PowerShell, hated it. The idea of manipulating objects instead of text streams is interesting, and avoids most of the footguns sh/bash have, but you also lose in flexibility.
One reason is that there are thousands of command line tools in the UNIX ecosystem that process text streams and are designed to work with shells like bash. You have much less options when you are processing PowerShell objects.
Note: I think the first thing I tried to do in PowerShell was a script that scanned a directory recursively for files containing a CRC in their name, and then check it, or something along these lines. After several hours of trying, I simply couldn't do it while it was relatively straightforward in bash, even with spaces in file names.
And that's not that I like UNIX shell scripting, in fact I hate it, so many footguns, that's why I wanted to try PowerShell, but it didn't fit my needs.
What things are you not able to do with PowerShell that you can with Bash + utilities? PowerShell literally gives you access to every way you can manipulate a string in C#. I get it, you're familiar with bash. But just because you know how to do something with something you're familiar with, doesn't mean PowerShell can't do what you wanted it to do.
Not trying to be combative, but learning any new language is going to require some dogfooding and digging in order to be as efficient as something you're already familiar with. I use bash and PowerShell daily. I can't think of any one thing I couldn't do in one or the other. Bash usually requires tying in another tool though. It's not strictly bash at that point. People saying stuff like jq. I rarely find that pre-installed. And if it's your machine, you can just as easily install something that doesn't require you to learn multiple languages (jq is a language).
I here you. I've been there. I'd lazily prod you to give POSH another shake. I've no horse in this race but I think you're missing out if you weren't able to accomplish in POSH what you were able to do in bash.
"You have much less options when you are processing PowerShell objects."
There is simply no need for the kind of extensive text processing common in Linux because every command returns an object whose fields can be directly referenced. Combined with the ConvertTo-Json command this is incredibly powerful. Honestly it seems like you are attempting to do things in PowerShell the bash way instead of the PowerShell way.
" I think the first thing I tried to do in PowerShell was a script that scanned a directory recursively for files containing a CRC in their name, and then check it"
I wrote a PowerShell script that recursively scanned every filed and folder on our fileshares and wrote the permissions to a file for later indexing.
I tried to write a PowerShell script to recursively scan and find files/folders older than a certain date, but kept hitting problems with the length of the path/filenames. As a complete PowerShell noob, I'm sure I was trying to do it the wrong way, but after a few attempts, I gave up and install cygwin instead.
Your problem here is with win32 not powershell I suspect. Your script would probably have worked with powershell on Linux, but windows you need to use UNC paths to get a 2^16 - ~20 character path limit rather than the 256 character path limit of regular paths.
Or there's some registry hacks to remove the limit from regular paths.
"Primitive" seems like the wrong word to apply. A modern Python programmer (a common example) might think that anything not OOP is archaic. But OOP is just one design pattern. And... it can easily lead to over-engineering and delight in the pattern per se.
Bash, awk, grep et alii don't do OOP. But they are close to the data and are powerful.
Compaints about the notation of these tools (e.g. "line noise") are becoming silly now that we've seen the appearance of some Powershell and Python statements. Any non-trivial notation will have to make choices. ^_^
I guess you would admit that beauty is subjective. Beauty in tooling is an even more rarefied discussion.
Code that we might deem "beautiful" may not even compile. ^_^
Does the tool do what it needs to do without getting fussy, gobbling RAM, and requiring a small army of maintainers to feed the monster? There is most definitely a place in technology for that kind of sanity.
There is something to be said for a design that doesn't bring with it a brigade of Opinions-As-A-Service proponents.
Any non-trivial notation is going to require learning. There is absolute value in learning. I think of many flexible and effective notations in tooling that has literally been the foundation of modern computing.
Maybe we are in the age of the Mono-Notation Luxury Coder. ^_^
Bash and gawk don't have direct in-language support for OOP, but can be used/abused to do things in an oop manner. aka without in-language support (abstractions/api), takes much more effort to do/understand OOP approach.
I think the article means using Bash for scripting, while the reader could use anything they want interactively. That's what I do—I use zsh, but I don't script in zsh.
I can't really stand Bash's arcane syntax, it drains my brain power (and time of consulting manual) every time I have to work with it. Switching to Fish has been a breath of fresh air for me. I think some people who want to use only Bash need to open their conservative mind. All of my personal shell scripts now are converted to Fish. If I want to run some POSIX-compatible script then I just use `bash scripts.sh`
Of course Bash is ubiquitous so I use them whenever I can in the company. A golden rule for me is: if it has more than 50 lines then I should probably write in a decent programming language (e.g. Ruby). It makes maintenance so much easier.
This battle was lost a long time ago. Bash is the standard on most UNIX systems. If you change this reality, one might even start to try to think about writing in fish or some other new shell. But I will not even consider another shell for scripts that need to be run by other people.
> The posix shell spec is missing a lot of reality. It’s not very actively maintained, unfortunately
POSIX was primarily intended a descriptive specification, rather than a prescriptive.
That is, it attempted to document and standardize the common behaviour found on many platforms during the Great Unix Wars, rather than say "hey we thought of this great new thing and released a spec, go implement it!", which is more how, say, web standards work. I does/did have some of that, but it was never the main goal.
These days "whatever Linux does" is the de-facto standard, for better or worse, and the need for POSIX is much less.
That's just feature creep, eventually they will implement all features of bash, zsh, csh, fish, elvish, oil, HolyC and all other shells that emerge in the meantime.
I agree that POSIX is needing of a few small additions;
* local keyword
* 2 or 3 variable expansion tricks (string replacement etc)
* pipefail
But with those, I think the spec isn't too bad. I've been writing (imo) high-quality (and shellcheck compliant) shell scripts for a decade, and _always_ try to be as pendantic about being posix compliant where humanly possible. Sometimes things are a _little_ harder (no sarcasm here), but it's really quite doable once you get the hang of it.
You scripts then end up being far more portable, and as stated below, MUCH easier to read. The trick is, to write readable code (which is hard enough for most people, regardless the language anyway :p)
e.g. (a very simple example), and an addition to Shrikant's post, avoid the terse double ampersant 'and' (&&) and double pipe 'or' (||) operators, in normal calls.
do
e.g.
```sh
if ! external_command; then
do something with failure
fi
```
Rather then
`external_command || do something with failure`
These can get up becoming really hard to read once they become longer, where reading a stupid "if" statement is easy to comprehend. Readability over saving number of lines.
One important hint left forgotten, always make sure the last line of your file is `exit 0` or similar. This is more a 'weak' security feature. Prevent people from appending files and executing stuff instead. Gives a known exit point.
Another addition would be to actually favor single quotes for pure strings, and use double quotes where you expect the shell to do something with your string (true for most cases, but there's plenty of strings that benefit from single quotes, and hint the reader what to expect). Also integers should never be quoted (as it would turn then into strings), which shows a 'bad' example in the trace statement, you are now comparing the strings '0' and '1' rather then the number. Best use something easier to read in that case.
One thing I will pick up from this post for sure, is the trace hint, I'm adding that to my scripts, but probably a little more tunable.
```sh
set -eu
if [ -n "${DEBUG_TRACE_SH:-}" ] && \
[ "${DEBUG_TRACE_SH:-}" != "${DEBUG_TRACE_SH#"$(basename "${0}")"}" ] || \
[ "${DEBUG_TRACE_SH:-}" = 'all' ]; then
set -x
fi
echo 'Hello World'
exit 0
```
Though I'll probably just settle for one of those keywords, I like 'all' the best atm. This would run the trace only if this special keyword is given, or if the variable contains the name of the script. I initially had `basename "${0%%.sh}"` but that makes things like `test,test1,test2` impossible, though that only helps if the extension is used ;)
While not needing be part of this list, I personally also always use functions, unless the script really is less then a handful lines, like google's bash-guide, always have a main etc.
In the end though I do disagree writing bash and bashisms. There's plenty of posts about that though; kind of like the whole C++ 'which version to use' discussion, where bash tends to be inconsistent in itself.
Bash as a language is a downright bad. Especially the mess around `,",'. Fish is better is in this regard, however the syntax of Fish's string-related functions is unbearable. (I have a growing suspicion, with string-related functions, syntactically valid expressions can be constructed, which don't compile!)
However, neither Bash nor Fish were created with Composability in mind, which is a show-stopper for me.
IMO don't use Bash if the script is longer than 20 lines and don't use Fish if it's longer than 50. Use Python. If you want to use a proper(!) language use any LISP-Dialect like Babashka, Guile Scheme, Racket, etc. If you need Types have a look at Haskell-Scripting.
EDIT: To clarify, use Fish for its bling-bling capabilities, don't use it for scripting and configuring your machine(s).
If you are going to write in a language that requires installing additional dependencies on every machine, why not something like Lua? The great thing about bash for me is that it just works on most machines without dependencies.
I use fish as an interactive shell but I don't write fish scripts. Once you accept a script has dependency there seems little reason not to go all the way to python (or usually I just go all the way to rust now, but I suspect others may disagree with me more on that than python)
I just use whatever the default is, and since that's zsh on macOS now, I just ported my ~/.profile to be functionally interchangeable between zsh and bash
Same PS1, aliases, functions, etc. but with a couple of slight variations due to syntax differences
> Most – though obviously far from all – scripts tend to be run in environments you control; portability is often overrated and not all that important (except when it is of course)
If you're at that spot, don't use shell in the first place but whatever other scripting language your team uses. Well, unless it's "pipe this to that to that", sh has no parallel here
I would add a "zero" best practice: don't. If you're thinking about writing enough shell script that it is worth putting it in a file, consider other languages or tools.
I'm not saying *never* write shell scripts, but always consider doing something else, or at least add a TODO, or issue, to write in a more robust language.
Speaking of shell, which language do you think has the best interoperatibility with shell commands. I mean, running a command, parsing the output, looping, adding user interaction etc. with the least amount of friction. Ruby used to come close for me, just put the command in backticks `` and write the main logic in Ruby, but I want to hear if there is something better.
I did that for 10+ years with perl, but I guess that these days Ruby and Python would be equally valid choices.
To be honest these days I use shell scripts, and if they get too large I'll replace with either golang or python. I don't love python, especially when dependencies are required, but it is portable and has a lot of things built-in that mean executing "standard binaries" isnt required so often.
Powershell, because it actually is a shell so it's great at easily invoking commands and using their outputs as actual return values, and because it has "programming language" constructs like dependency management in modules, etc.
It has some great tools for user interaction, too, including secure string handling for credentials, a TUI framework, easy parallelism, unit tests and lots more.
For example, using cd "$(dirname "$0")" to get the scripts location is not reliable, you could use a more sophisticated option such as:
$(dirname $BASH_SOURCE)
And THIS is the primary source of my furious hate in my toxic love-hate relationship with bash.
Guy is writing bash FOR 10 FUCKING YEARS and still apparently doing it wrong in 10 letter oneliner.
When it comes to bash search for even simplest command/syntax always ALWAYS leads to stackoverflow thread with 50 answers where bash wizards pull oneliners from sleeves and nitpick and argue about various intricancies
It's a case of knowing the wooledge website (and working with shellcheck), or not. Picking snippets on stackoverflow will probably do more harm than good, tbh.
> Use the .sh (or .bash) extension for your file. It may be fancy to not have an extension for your script, but unless your case explicitly depends on it, you’re probably just trying to do clever stuff. Clever stuff are hard to understand.
I don't agree with this one. When I name my script without extension (btw, .sh is fine, .bash is ugly) I want my script to look just like any other command: as a user I do not care what the language program is written in, I care about its output and what it does.
When I develop a script, I get the correct syntax highlight becuase of the shebang so the extension doesn't matter.
My thumb rule is no extension if the script goes to the local bin folder and `.sh` otherwise. Beyond syntax highlighting, the extension also helps for wildcard matching for file operations (`ls`, `cp`, `for` loop, etc).
And this rule has been followed the majority(if not all) interpreted and scripting languages. The likes of Ruby, python and JS have multiple examples. Whatever executable is in your $PATH it won’t have an extension.
Not sure if this convention is actually documented anywhere.
"Ugly" is subjective. If I encountered a file with that extension, I'd assume it uses Bash-specific features and that I shouldn't run this script with another shell.
Yes, I get it. But in my case I simply give u+x permissions to the script and then run "./script.sh" and then the script will be executed with the interpreter defined in the shebang.
The hashbang already specifies the shell, so also having it in the extension seems unnecessary. I don't like using '.sh' as an extension as it differs from other OS commands and I can't think where it's actually helpful.
If you download a script then running "sh script.sh" is a lot quicker and easier than a chmod followed by ./script.sh. You can of course also type "bash script.sh", but I don't always have it installed on every system, and the .bash extension just clarifies it.
For things in my PATH I drop any suffixes like that.
I see your point, but you can just as easily run "sh script" although that does imply that you already know that it's a shell script (obviously you wouldn't just run something from the internet without checking it first).
> obviously you wouldn't just run something from the internet without checking it first
This died a long time ago with the pervasive use of NPM and PIP and the likes.
Most developers probably run a lot of random unchecked shit all the time with local user privileges today without a blink.
Somehow people are ready for all this, but are still afraid to run a random shell script from the internet. I guess this fear is one of our chances to explain how NPM and PIP can be dangerous.
I don't know. I do that a lot on test servers when tinkering with new stuff but at work I'm very careful what I insert into my employer's infrastructure. If someone breaks in using a hole I should have taken care of, that's already bad. But if invite bad guys by installing a C&C for them, that's superbad.
If you're committed to being an idiot no amount of rules of thumb will save you. Not understanding what you use is one facet of being committed to that.
But people don't live in a vacuum. They live in an ecosystem, are subjected to it and contribute to it.
If I contribute to Nextcloud or write an app for it, I need to run npm. If I want to run PeerTube, I need to run npm. They both pull a shitload of dependencies I can't possibly review.
I personally avoid building anything using NPM and advocate for fewer / no dependencies, or for using dependencies packaged by reputable entities like Debian, but what can I do? I can't build everything myself.
> This died a long time ago with the pervasive use of NPM and PIP and the likes.
When a malicious package is found on NPM or PIP, it will get removed. However, it is quite unlikely that a website will be taken town for a malicious script (or only after a long time).
I really doubt that most readers of HN would run a random script unless it comes from a source they trust (trusted enough to least to remove a malicious script in a timely fashion).
The .sh extension told me it's a shell script of some kind.
I don't check everything I download from the internet; I don't think anyone does. It depends on what it is, where I'm getting it from, where I'm running it, etc. There are certainly some things I will review carefully, but other things I give just a quick check to see it's not in complete shambles, and others I barely check at all. I typically run the latest Vim from master, do I check every patch to see if after 30 years Bram finally sneaked in a crypto miner or password stealer? Do the people who package Vim for the Linux distros?
There's a difference between trusting well known software such as vim (especially when packaged by a distro) and trusting shell scripts from essentially anyone. If it's a well known resource, then I'd likely trust the script without checking (e.g. adding a docker repo to Ubuntu), but otherwise I'm going to give it a quick eyeball.
I would tend to agree that scripts for downloading from the internet should have a '.sh' extension to make it clear that it's a script as opposed to a binary.
Indeed; that was my point exactly. People complain about things like "curl https://sh.rustup.rs | sh" from the Rust homepage, but it's essentially the same as trusting "cd vim && ./configure && make && make install" (plus, if I would hide anything I'd do it in the probably quite large binary that script downloads, which is much harder to audit).
In my setup, I use aliases or functions to have short/mnemonic names for commands.
But the files on disk must always have proper extensions like .sh to quickly see what they are.
Personally, I prefer to keep the extension and add and alias in my aliases file inside "~/.bashrc.d". Redudant, perhaps, but I like to run a ls inside "~/.local/bin" (the place where I throw user wide personal executables) and be able to see at first glance what is a binary and what is a script.
There are use cases where you don't have execute privileges. In those cases the .sh-extension makes it clear that you can do `bash script.sh`. If you don't use an extension you wouldn't easily see that that was an option.
No, it doesn't. The extensions are usually too inaccurate to rely on. That could be either a Bourne or Bash script, meaning it could either fail at some arbitrary point during run if the wrong one is used, or just subtly, critically change some output. Much more true for Python scripts.
I honestly do it mostly coz IDEA is/was mighty stupid when it comes to detecting file types compared to Emacs... altho newer editions seemed to fix that problem for the most part
Now when I'm processing files with BASH, I nearly always end up copying stuff from there as it just bypasses common errors such as not handling whitespace or filenames that contain line breaks.
In my experience, the best practice is to implement all the non-trivial logic in the actual program or a Python script, and use shell script only for very straight-forward stuff like handling command-line arguments, environment variables and paths.
There is an error with the template script on a fully patched m1 macbook. $1 is unbound, unless you provide an argument. This seems to be an utterly basic oversight for a template script from someone attempting to educate on bash's best practices. Especially true for seeking a "good balance between portability and DX".
I'm not convinced about having shell scripts end with ".sh" as you may be writing a simple command style script and shouldn't have to know or worry about what language it's using.
It's standalone, so you just start a script using it as a template and delete bits that you don't want. To my mind, the best feature is having consistent logging functions, so you're encouraged to put in lots of debug commands to output variable contents and when you change LOG_LEVEL, all the extraneous info doesn't get shown so there's no need to remove debug statements at all.
The other advantage is the option parsing, although I don't like the way that options have to have a short option (e.g. -a) - I'd prefer to just use long options.
499 comments
[ 3.7 ms ] story [ 287 ms ] threadCase in point: Declaring an array. IMHO, it’s just not ergonomic at all. Especially not in sh/dash.
As soon as output needs to be parsed — especially when it’s being fed back into other parts of the script — it gets harder. Handling errors and exceptions is even more difficult.
Things really fall down on modularity. There are tricks and conventions: for example you can put all functions to do with x in a file called lib/x.sh, prefix them all with x_, and require that all positional parameters must be declared at the top of each function with local names.
At that point though, I would rather move to a language with named parameters, namespaced modules, and exception handling. In Python, it’s really easy to do the shell bits with:
which will let you pass in arguments with spaces and be able to access them as properly lexed arguments in $1, $2 etc in your script. You can even preprocess the script to be prefixed with all the usual set -exuo pipefail stuff etc.[0]: https://marketplace.visualstudio.com/items?itemName=timonwon...
Usually there is no need to return to original directory. Change of directory is process-local (script-local) so the calling process is not affected by this 'cd' in the script.
* use bats for testing * use shfmt for code formatting * use shellcheck for linting
If a shell script needs any kind of functionality beyond POSIX, then that's a good time to upgrade to a higher-structure programming language.
Here's my related list of shell script tactics:
http://github.com/sixarm/unix-shell-script-tactics
I only use it to select the n'th word in a csv-like line. Anything more than that, I need to search stackoverflow for the invocation.
Don't you find its syntax cumbersome?
Not really; just seems the same as most other dynamic languages. Awk does a lot of stuff for you (the "implied loop" your program runs in, field splitting) that's certainly possible (even easy) to replicate in Python or Ruby, but Awk it's just so much more convenient.
I use it for things like processing the Unicode data files, making some program output a bit nicer (e.g. go test -bench), ad-hoc spreadsheets, few other things. I got started with it as I needed to process some C header files and the existing script for that was in Awk; it worked pretty well for that too.
The Awk Programming Language book is pretty good. GNU Awk has a bunch of very useful extensions, but pretty much everything in the book still works and is useful today. You can get it at e.g. https://archive.org/details/awkprogrammingla00ahoa or https://github.com/teamwipro/learn_programing/blob/master/sh...
The GNU Awk docs are also pretty decent.
If it's not a CSV with quotes to allow for commas inside values (which I think AWK will also fail in), you can use `cut`.
To delimit on a comma and select the n'th field. Reads a bit easier IMO for that common AWK use case (which I used to use AWK for all the time).Bash in --posix mode does that perfectly.
For me, the additional features that bash provides are much more important than a portability that I'll never need to use.
> And it’s usually always appropriate.
I wouldn't think so. You don't know where your script will be called from, and many times the parameters to the script are file paths, which are relative to the caller's path. So you usually don't want to do it.
I collected many tips&tricks from my experience with shell scripts that you may also find useful: https://raimonster.com/scripting-field-guide/
Fixed, thanks!
Edit: just had a quick look at your recommended link and spotted a "mistake" in 4.7 - using "read" without "-r" would get caught out by shellcheck.
Agreed on the powerful bit, however [[ ]] is not a "builtin" (whereas [ and test are builtins in bash), it's a reserved word which is more similar to if and while.
That why [[ ]] can break some rules that builtins cannot, such as `[[ 1 = 1 && 2 = 2 ]]` (vs `[ 1 = 1 ] && [ 2 = 2]` or `[ 1 = 1 -a 2 = 2 ]`, -a being deprecated).
Builtins should be considered as common commands (like ls or xargs) since they cannot bypass some fundamental shell parsing rules (assignment builtins being an exception), the main advantages of being a builtin being speed (no fork needed) and access to the current shell process env (e.g. read being able to assign a variable in the current process).
I think an easier way is to find script's location and construct paths for accessing script dependencies, for example (works on Linux & macOS):
But, more importantly, people will google for how to set cwd to the script directory more often then will google how to go to an absolute path. Having 'cd "$(dirname "$0")"' as reference in an article discussing best practices and the topic of changing the directory early, is a good idea.
people will google for how to set cwd to the script directory more often
The answer should suggest setting $script_dir instead of chdir and refer to it when needed, explaining why chdir is a wrong shell mindset except for a very few special cases. It’s okay for personal use, but inheriting such scripts would be an awful experience, imo.
So for somebody just coming from Windows, this isn't just the wrong mindset for shell scripts, more dangerously it's a mindset they find appealing, because it matches their prior experience on Windows better.
I think it would be more correct to just use vcs to get the root of a project (or fail if you can't), instead of essentially hardcoding path to the script.
For example if you put your script in helpers/ then someone else did a refactor and moved all of the stuff into cmd/helpers/, any relative reference you put into script is now invalid and your script is doing the wrong thing
I think fish is quite a bit different in terms of syntax and semantics (I'm not very familiar with it), but zsh is essentially the same as bash except without most of the needless footguns and awkwardness. zsh also has many more advanced features, which you don't need to use (and many people are unaware of them anyway), but will very quickly become useful; in bash all sorts of things require obscure incantations and/or shell pipelines that almost make APL seem obvious in comparison.
In my experience few people understand bash (or POSIX sh) in the first place, partly because everything is so difficult and full of caveats. Half my professional shell scripting experience on the job is fixing other people's scripts. So might as well use something that doesn't accidentally introduce bugs every other line.
Most – though obviously far from all – scripts tend to be run in environments you control; portability is often overrated and not all that important (except when it is of course). Once upon a time I insisted on POSIX sh, and then I realised that actually, >90% of the scripts I wrote were run just by me or run only in an environment otherwise under my control, and that it made no sense. I still use POSIX sh for some public things I write, when it makes sense, but that's fairly rare.
I think bash is really standing in the way of progress, whether that progress is in the form of fish, zsh, oil shell, or something else, because so many people conflate "shell" with "bash", similar to how people conflate "Google" with "search" or "git" with "GitHub" (to some degree).
There is a general problem in the fact that a radical evolution of glue languages wouldn't be popular because devs rather use Python, and small evolutions wouldn't be popular (ie. zsh), because they end up being confusing (since they're still close to Bash) and not bringing significant advantages.
I'm curious why there haven't been attempts to write a modern glue language (mind that languages like Python don't fit this class). I guess that Powershell (which I don't know, though) has been the only attempt.
This is kind of a good example of what I meant when people conflate "bash" with "shell".
As for your larger point: I kind of agree, but I think what zsh offers is the advantages of shell scripts with compatibility with existing scripts while still improving on it. That said, I believe oil also offers compatibility, but I haven't had the chance to look deeply in to it; just haven't had the time, and wanted to wait until it's stable (maybe it is now?)
Perl was initially invented as the "modern glue language" to replace shell. It's fallen a bit out of fashion these days though, and to be honest I never cared all that much for Perl myself either. Raku looks nice though. TCL also works well as a kind of "glue language", although it has some really odd behaviour at times due to everything being a string and I know some people hate it with a passion, but it always worked fairly well for me. But that has also fallen out of fashion.
I've also been told PowerShell is actually quite nice and has interesting concepts (and now also open source, and you can run it on e.g. Linux), but I could never get over the verbosity of it all. I'm an old unix greybeard and I want my obscure abbreviations dammit!
It's a little more wordy if you're use to C or Bash. But hands down it's one of my favorite languages for slicing dicing data. No need for 3rd party libraries or binaries. No need to learn a bunch of weird awk/jq syntax which is only useful for those two tools (yay, lets learn 3 language instead one?). Plus, most of the structure translates over to C#, and you can integrate C# code directly into your POSH code if desired, as well as access pretty much any low level C# methods directly.
Working with strings? Pretty much any/every tool you could want to slice and dice strings.
The POSH REPL is amazing. You have far more flexibility around interacting with the command line than you do with Python. It's both a shell and a true language. As with any language, there are ISMs, but far fewer footguns than any other language I've spun up.
Cross platform as well with 6.0+
Intellisense ON the commandline (did I mention the awesome REPL?). Hands down one of the best built-in parameter/args/help parsing I've encountered across any language. Debugging? Amazing in vscode. And can be done strictly from the commandline as well (dynamic breakpoints? You've got it, drop you right into your catch block with an interactive shell so you can see the current status of any/all variables and manipulate them live and resume if desired)
Okay, I'm done shilling for POSH. It's hands down one of my favorite shells/languages for doing POC work, or writing utility functions. Treat it more like Python than bash. But realize that you can easily use that Pythonic-esque code right inside your shell.
The nice thing with Powershell is that it's not verbose, but the arcane abbreviations are actually quite a bit easier to remember and discover than bash. What mixes people up is that in documented examples and reusable scripts, it makes sense to use the full, canonical name, which looks aesthetically different coming from a Unix background.
Here's what it might actually look like to check a JSON file that has an array of file metadata objects, and delete the ones that have been processed (this includes one user-defined alias, cfj, for "ConvertFrom-Json"):
That seems pretty NON-verbose to me, equivalent to how you'd approach this in bash. Do you have jq installed? If you do, perhaps: If you don't have jq I think this gets much longer and is either really brittle (you're doing some kind of adhoc parsing that's terrible) or really heavyweight (like pulling in a pure bash JSON library--they exist) or you're using a one-liner from another programming language (Ruby, Python, something). Or maybe you'd complain to whatever was writing 'queue.json' and ask for a friendlier format, like something terrible and one-off you invented because it's easy to "parse" with awk?).It's even better if you're dealing with network resources:
That shows off another alias of Where-Object how you do a foreach loop in a pipeline. And bash: What probably makes you think Powershell is verbose is that, although that's how I type Powershell, it's not how I document it. If I were documenting it for someone else's use, or incorporating that pipeline into a script or module, I'd write it like this: So it's not like it's verbose while you're using it, but verbosity is something you can reach for for clarity when it's desirable. Likewise, you can see the consistent relationship between these commands and their aliases: gc -> Get-Content, cfy -> ConvertFrom-Yaml, ri -> Remove-Item. So you have options for how verbose you need to be. I find it's very useful to have a spelled-out version for commands I don't use all the time, like 'Get-EC2Instance', and consistent verbs so I can make reasonable guesses at the command name for things I'm even less familiar with.I didn't want to clutter this with too many examples but I'll reiterate that Powershell is a shell. It invokes your Unix commands just fine. For example, if you forgot what the option to Get-Date is, to get a Unix-style timestamp (it's 'get-date -uf %s' so not exactly hard to use): you can just type 'date +%x'. In the example above, I could have used 'cat' and 'xargs rm' instead of 'gc' and 'ri'. So it's not like you have to buy the whole kit and kaboodle right away, either.
I’m aware that ZSH has been the default shell since Catalina.
Started using fish a month ago and really liking it.
I think being the de-facto default on Linux as part of "GNU plus Linux" has more to do with it.
Like aliases but they expand in-place, so auto completion friendly, easily modifiable, etc. Love them
$ abbr s sudo
$ s<space> -> sudo
Ten years or so ago the cool kids were using zsh: which is in general a pretty reasonable move, it’s got way more command-line amenities than bash (at least built in).
Today fish is the fucking business: fish is so much more fun as a CLI freak.
But I guess I’ve got enough PTSD around when k8s or it’s proprietary equivalents get stuck that I always wanted to be not only functional but fast in outage-type scenarios that I kept bash as a daily driver.
Writing shell scripts of any kind is godawful, the equivalent python is the code you want to own, but it’s universality is a real selling point, like why I keep half an eye on Perl5 even though I loathe it: it may suck but it’s always there when the klaxon is going off.
The best possible software is useless if it’s not installed.
I was only talking about scripting; I know fish scripting is different, but I have no idea if it's any good. For interactive shells I don't care what people use: it's 100% a personal choice.
You have a level of control over things popping up too
See also <https://apple.stackexchange.com/questions/371997/suppressing...>. I went with the "use an updated brewed Bash" approach, which has been working well. Using `sudo chfn` means you don't need to futz around with editing /etc/shells.
=name will expand to the entry in your PATH. e.g. =ls expands to /usr/bin/ls. So == expands to an executable named =, or rather, it tries to as you probably don't have = in your PATH.
[[ ]] disables expansions (e.g. [[ * = * ]] will work too) so it's not an issue there.
I had to work with Sencha/ExtJS early 2010. It was the same feeling. Yes, it is powerfull, but too much magic happen for something without a clear orientation (at the time, now it is used for data loaded frontend i think). PS i don't understand what it wants me to do.
The language is fine. The interface is now fine, but in 2015 it was the shittiest tty available on modern computers. It's okay since at least early 2021 (when i restarted using windows). I know it should be reasonably better, but i wouldn't trust any PS script written before 2021 to run on my workstation. I run bash scripts i wrote when i started coding.
Still, if you're new to the gig and don't care about free software and commons, you should learn PS (unless you want to work on baremetal or on MC, in this case, bash will be enough).
Bit off-topic, but I worked with ExtJS around the same time, and I found it one of the most confusing development experiences I ever had. "It's so easy, just add this one property to this deeply nested object you're passing to this function!" Thinking back on it, it's a really good example of "simple vs. easy". It didn't help it gave not a peep if you got one of those data structures wrong (capitalisation typo, wrong location, etc.)
Plus in hindsight the whole idea of "OS semantics in your browser!" was never a good one to start with, although that wasn't as obvious to me at the time.
ExtJS (on netbeans with windows server 2004) was always my worst development experience, it was my first internship too. That's probably the reason why it took so long (and a bag of money) for me to try JS, an IDE and developing on windows again (I only used Windows for CTFs and AndroidStudio).
The performance difference was night and day.
The biggest issue with PowerShell is that PowerShell Core is not yet default on Windows 10/11 and Windows Server. That should be Microsofts highest priority for PowerShell.
One reason is that there are thousands of command line tools in the UNIX ecosystem that process text streams and are designed to work with shells like bash. You have much less options when you are processing PowerShell objects.
Note: I think the first thing I tried to do in PowerShell was a script that scanned a directory recursively for files containing a CRC in their name, and then check it, or something along these lines. After several hours of trying, I simply couldn't do it while it was relatively straightforward in bash, even with spaces in file names.
And that's not that I like UNIX shell scripting, in fact I hate it, so many footguns, that's why I wanted to try PowerShell, but it didn't fit my needs.
Not trying to be combative, but learning any new language is going to require some dogfooding and digging in order to be as efficient as something you're already familiar with. I use bash and PowerShell daily. I can't think of any one thing I couldn't do in one or the other. Bash usually requires tying in another tool though. It's not strictly bash at that point. People saying stuff like jq. I rarely find that pre-installed. And if it's your machine, you can just as easily install something that doesn't require you to learn multiple languages (jq is a language).
I here you. I've been there. I'd lazily prod you to give POSH another shake. I've no horse in this race but I think you're missing out if you weren't able to accomplish in POSH what you were able to do in bash.
There is simply no need for the kind of extensive text processing common in Linux because every command returns an object whose fields can be directly referenced. Combined with the ConvertTo-Json command this is incredibly powerful. Honestly it seems like you are attempting to do things in PowerShell the bash way instead of the PowerShell way.
" I think the first thing I tried to do in PowerShell was a script that scanned a directory recursively for files containing a CRC in their name, and then check it"
I wrote a PowerShell script that recursively scanned every filed and folder on our fileshares and wrote the permissions to a file for later indexing.
Or there's some registry hacks to remove the limit from regular paths.
https://learn.microsoft.com/en-us/windows/win32/fileio/maxim...
Anyways, it was a long time ago, maybe I will give PowerShell a retry at some point.
Bash, awk, grep et alii don't do OOP. But they are close to the data and are powerful.
Compaints about the notation of these tools (e.g. "line noise") are becoming silly now that we've seen the appearance of some Powershell and Python statements. Any non-trivial notation will have to make choices. ^_^
Painful, ugly, unpleasant?
Code that we might deem "beautiful" may not even compile. ^_^
Does the tool do what it needs to do without getting fussy, gobbling RAM, and requiring a small army of maintainers to feed the monster? There is most definitely a place in technology for that kind of sanity.
There is something to be said for a design that doesn't bring with it a brigade of Opinions-As-A-Service proponents.
Any non-trivial notation is going to require learning. There is absolute value in learning. I think of many flexible and effective notations in tooling that has literally been the foundation of modern computing.
Maybe we are in the age of the Mono-Notation Luxury Coder. ^_^
Bash and gawk don't have direct in-language support for OOP, but can be used/abused to do things in an oop manner. aka without in-language support (abstractions/api), takes much more effort to do/understand OOP approach.
Of course Bash is ubiquitous so I use them whenever I can in the company. A golden rule for me is: if it has more than 50 lines then I should probably write in a decent programming language (e.g. Ruby). It makes maintenance so much easier.
And OSH implements almost all of bash
That is, The posix shell spec is missing a lot of reality. It’s not very actively maintained, unfortunately
The canonical example is not having local vars, which basically every shell supports
> It’s not very actively maintained, unfortunately
Because that's how standards works. If there is enough interest a new standard will be made, but right now, bash isn't it.
POSIX was primarily intended a descriptive specification, rather than a prescriptive.
That is, it attempted to document and standardize the common behaviour found on many platforms during the Great Unix Wars, rather than say "hey we thought of this great new thing and released a spec, go implement it!", which is more how, say, web standards work. I does/did have some of that, but it was never the main goal.
These days "whatever Linux does" is the de-facto standard, for better or worse, and the need for POSIX is much less.
* local keyword * 2 or 3 variable expansion tricks (string replacement etc) * pipefail
But with those, I think the spec isn't too bad. I've been writing (imo) high-quality (and shellcheck compliant) shell scripts for a decade, and _always_ try to be as pendantic about being posix compliant where humanly possible. Sometimes things are a _little_ harder (no sarcasm here), but it's really quite doable once you get the hang of it.
You scripts then end up being far more portable, and as stated below, MUCH easier to read. The trick is, to write readable code (which is hard enough for most people, regardless the language anyway :p)
e.g. (a very simple example), and an addition to Shrikant's post, avoid the terse double ampersant 'and' (&&) and double pipe 'or' (||) operators, in normal calls. do e.g. ```sh if ! external_command; then do something with failure fi ``` Rather then `external_command || do something with failure`
These can get up becoming really hard to read once they become longer, where reading a stupid "if" statement is easy to comprehend. Readability over saving number of lines.
One important hint left forgotten, always make sure the last line of your file is `exit 0` or similar. This is more a 'weak' security feature. Prevent people from appending files and executing stuff instead. Gives a known exit point.
Another addition would be to actually favor single quotes for pure strings, and use double quotes where you expect the shell to do something with your string (true for most cases, but there's plenty of strings that benefit from single quotes, and hint the reader what to expect). Also integers should never be quoted (as it would turn then into strings), which shows a 'bad' example in the trace statement, you are now comparing the strings '0' and '1' rather then the number. Best use something easier to read in that case.
One thing I will pick up from this post for sure, is the trace hint, I'm adding that to my scripts, but probably a little more tunable.
```sh set -eu if [ -n "${DEBUG_TRACE_SH:-}" ] && \ [ "${DEBUG_TRACE_SH:-}" != "${DEBUG_TRACE_SH#"$(basename "${0}")"}" ] || \ [ "${DEBUG_TRACE_SH:-}" = 'all' ]; then set -x fi
echo 'Hello World'
exit 0 ``` Though I'll probably just settle for one of those keywords, I like 'all' the best atm. This would run the trace only if this special keyword is given, or if the variable contains the name of the script. I initially had `basename "${0%%.sh}"` but that makes things like `test,test1,test2` impossible, though that only helps if the extension is used ;)
While not needing be part of this list, I personally also always use functions, unless the script really is less then a handful lines, like google's bash-guide, always have a main etc.
In the end though I do disagree writing bash and bashisms. There's plenty of posts about that though; kind of like the whole C++ 'which version to use' discussion, where bash tends to be inconsistent in itself.
Shameless plug: See some of my older stuff here https://gitlab.com/esbs/bootstrap/-/blob/master/docker_boots...
> If I want to run some POSIX-compatible script then I just use `bash scripts.sh`
Shouldn't you be using a shebang?
Why would they use a shebang? `bash script.sh` works perfectly fine. Ever used a terminal?
However, neither Bash nor Fish were created with Composability in mind, which is a show-stopper for me.
IMO don't use Bash if the script is longer than 20 lines and don't use Fish if it's longer than 50. Use Python. If you want to use a proper(!) language use any LISP-Dialect like Babashka, Guile Scheme, Racket, etc. If you need Types have a look at Haskell-Scripting.
EDIT: To clarify, use Fish for its bling-bling capabilities, don't use it for scripting and configuring your machine(s).
Same PS1, aliases, functions, etc. but with a couple of slight variations due to syntax differences
If you're at that spot, don't use shell in the first place but whatever other scripting language your team uses. Well, unless it's "pipe this to that to that", sh has no parallel here
I'm not saying *never* write shell scripts, but always consider doing something else, or at least add a TODO, or issue, to write in a more robust language.
To be honest these days I use shell scripts, and if they get too large I'll replace with either golang or python. I don't love python, especially when dependencies are required, but it is portable and has a lot of things built-in that mean executing "standard binaries" isnt required so often.
It has some great tools for user interaction, too, including secure string handling for credentials, a TUI framework, easy parallelism, unit tests and lots more.
- `set -o errtrace`: trap errors inside functions
- `shopt -s inherit_errexit`: subprocesses inherit exit error
Unfortunately the list of Bash pitfalls is neverending, but that's a good start.
For example, using cd "$(dirname "$0")" to get the scripts location is not reliable, you could use a more sophisticated option such as: $(dirname $BASH_SOURCE)
I'm more likely to use the BashFAQ though for actual snippets: https://mywiki.wooledge.org/BashFAQ
I start scripts from the very useful template at https://bash3boilerplate.sh/
When it comes to bash search for even simplest command/syntax always ALWAYS leads to stackoverflow thread with 50 answers where bash wizards pull oneliners from sleeves and nitpick and argue about various intricancies
Also, did you mean to write...?
I don't agree with this one. When I name my script without extension (btw, .sh is fine, .bash is ugly) I want my script to look just like any other command: as a user I do not care what the language program is written in, I care about its output and what it does.
When I develop a script, I get the correct syntax highlight becuase of the shebang so the extension doesn't matter.
The rest of the post is great.
One advantage of no-extension is that you can swap the implementation language later without "breaking" shell history for people in your team.
Not sure if this convention is actually documented anywhere.
Random examples:
- https://github.com/PyCQA/isort/blob/main/pyproject.toml#L100
- https://github.com/pypa/pip/blob/main/setup.py#L78
- https://github.com/11ty/eleventy/blob/master/package.json#L1...
"Ugly" is subjective. If I encountered a file with that extension, I'd assume it uses Bash-specific features and that I shouldn't run this script with another shell.
That was the intention of my comment. Because the rest of the post (or most of it for sure) is not subjective.
For things in my PATH I drop any suffixes like that.
This died a long time ago with the pervasive use of NPM and PIP and the likes.
Most developers probably run a lot of random unchecked shit all the time with local user privileges today without a blink.
Somehow people are ready for all this, but are still afraid to run a random shell script from the internet. I guess this fear is one of our chances to explain how NPM and PIP can be dangerous.
If I contribute to Nextcloud or write an app for it, I need to run npm. If I want to run PeerTube, I need to run npm. They both pull a shitload of dependencies I can't possibly review.
I personally avoid building anything using NPM and advocate for fewer / no dependencies, or for using dependencies packaged by reputable entities like Debian, but what can I do? I can't build everything myself.
Am I committed to being an idiot?
When a malicious package is found on NPM or PIP, it will get removed. However, it is quite unlikely that a website will be taken town for a malicious script (or only after a long time).
I really doubt that most readers of HN would run a random script unless it comes from a source they trust (trusted enough to least to remove a malicious script in a timely fashion).
I don't check everything I download from the internet; I don't think anyone does. It depends on what it is, where I'm getting it from, where I'm running it, etc. There are certainly some things I will review carefully, but other things I give just a quick check to see it's not in complete shambles, and others I barely check at all. I typically run the latest Vim from master, do I check every patch to see if after 30 years Bram finally sneaked in a crypto miner or password stealer? Do the people who package Vim for the Linux distros?
I would tend to agree that scripts for downloading from the internet should have a '.sh' extension to make it clear that it's a script as opposed to a binary.
* safe ways to do things in bash: https://github.com/anordal/shellharden/blob/master/how_to_do...
* better scripting: https://robertmuth.blogspot.in/2012/08/better-bash-scripting...
* robust scripting: https://www.davidpashley.com/articles/writing-robust-shell-s...
Now when I'm processing files with BASH, I nearly always end up copying stuff from there as it just bypasses common errors such as not handling whitespace or filenames that contain line breaks.
I'm a fan of using BASH3 boilerplate: https://bash3boilerplate.sh/
It's standalone, so you just start a script using it as a template and delete bits that you don't want. To my mind, the best feature is having consistent logging functions, so you're encouraged to put in lots of debug commands to output variable contents and when you change LOG_LEVEL, all the extraneous info doesn't get shown so there's no need to remove debug statements at all.
The other advantage is the option parsing, although I don't like the way that options have to have a short option (e.g. -a) - I'd prefer to just use long options.