One of my favorite things about Shellcheck is that the error messages come with a link to a wiki page about the problem. Does anyone know other programs that have a similar approach to error messages?
they had the MSDN cd (microsoft developer network) for documentation in these days; that cd had a lot of thorough documentation for apis and developer tools.
I recall working with AngularJS, which output error messages that had links, which opened the documentation with bits from your stack trace filled in.
It was a while ago and seemed a bit gimmicky, but i really enjoyed the ability to see what could be done about an error by opening a web page (without having to manually dig through StackOverflow or other sites).
Now, if we'd get a development framework that'd couple documentation like that, with user generated comments (like PHP has, for example see the bottom of https://www.php.net/manual/en/function.strcmp.php ), then i think the developer experience would improve a bunch!
Who knows, maybe even allow users to contribute possible fixes and allow those as autocomplete solutions and you've just improved on what tools like Codota do a whole bunch!
If you're building a REST API, make sure to check out RFC7807 ( https://tools.ietf.org/html/rfc7807 ) , which recommends doing exactly this. Giving each error a type, which is a resolvable URI.
Staticcheck for Go kind of does something similar in that it outputs Googleable error codes, but doesn't actually print a link to the pages. It's also included as part of golangci-lint.
It is also a quite nice tool for checking the POSIX compatibility of shell scripts. I find it very useful when working on Solaris or some other UNIX that does not use Bash by default.
Same here, I basically learned how to write POSIX-compliant Shell with the help of ShellCheck. Shell is not a difficult language to learn but it definitely has its oddities and the different flavours (Bash etc.) only add to the confusion.
What people don't always expect is the immense portability that POSIX-compliant Shell offers you. This thing runs on pretty much everything.
I love linters and formatters. Often languages carry baggage and other “bad parts” that linters help you avoid.
For someone who’s not used to them they might feel annoying (e.g. why should I wrap every $() in quotes?! Ugh) but once you accept them your code will be a million times better and safer.
Same. Few years ago I started a little list of linters. Just kept adding tools and the list is huge now thanks to lots of contributors. https://github.com/analysis-tools-dev/static-analysis
Still love adding new linters there when I cross their path. :)
Unless you're composing very rich commands, likely not much will show up. It exceeds at finding things like interpolation gotchas, bad loop control, etc. Much more structural than just one off commands (though would likely bug about a bunch of useless `cat`s when you can use redirect operators instead (i.e. `cat foo.txt | grep` is an antipattern according to shellcheck)
This tool validates your commands at the Bash prompt using ShellCheck and refuses to evaluate them if there's a mistake. It ignores a set of rules that doesn't play well with oneliners.
Great tool that everyone should be using for shell scripts!
Then again, I've learnt more than once that you start with "I'm just capturing a few commands in a script" and the next thing you see is a mess of special characters, crude syntax and nasty error handling. Don't be like me...
I find it helps if you have a hard rewrite rule. I really love the convenience of shell scripts for small scripts, but as soon as I have one that crosses 100 lines I rewrite it in python. The python version is a bit less convenient for spawning processes and piping them together, but it's so much more maintainable that you can scale it up to quite a bit larger without as much worry. 100 lines is a nice size where it doesn't take very long to rewrite, so even though I figure I could get away with a longer shell script without it getting too bad, I'm definitely not going to want to rewrite it once it gets big enough to be a problem.
And then you get to google scale and realize using python for every config, like your entire build system, leads to a huge mess of completely non-deterministic build scripts that do wacky things like make HTTP calls to prod systems, depend on all kinds of global state stores, etc. So you build a new version of python that strips out all of the non-determinism and move your config to that (see starlark: https://docs.bazel.build/versions/master/skylark/language.ht... ).
I know I'm in the minority but I just don't care for python for sysadmin / devops type tasks. It fails in weird ways on file and directory manipulations and I just don't find it's stack traces very informative. A lot of scripts seem to be a bunch of system () calls strung together, how is that better than well written Shell code?
What I love python for is scraping web pages and dealing with JSON or xml it rocks for that.
This is one of my favorite tips for almost effortlessly improving devops productivity (along with using https://github.com/mvdan/sh for auto-formatting) — by now shellcheck has good editor support in e.g. VSCode and I have a standard pre-commit.com hook for all of my projects. I've seen so many long iterations where people flail at a complex shell script for something like a cron job which was immediately flagged by shellcheck. I generally recommend rewriting in Python but this is a good less invasive step.
One thing which might be worth considering is adding this to your personal ~/.shellcheckrc to make it more pedantic:
I went looking through the issue tracker to see if there's an issue about supporting $XDG_CONFIG_HOME and was pleasantly surprised to find out that it seems to already work. So ~/.config/shellcheckrc works just as well as ~/.shellcheckrc
Alongside ShellCheck, I also use shUnit2 as my unit testing framework. These two combined have let me write some pretty long (~1k loc) bash programs that work. Yes, you should test your bash.
I tried one or two but decided on shUnit2 because it seemed to be the most friendly and familiar. If you have suggestions of alternatives I'd love to hear them!
I really like task (http://taskfile.dev/) as a more usable layer on top of mydan/sh. Task is basically make but with a yaml syntax and built-in posix shell interpreter all in a single statically linked executable. It makes building something even on Windows a total breeze.
I just tried to use it this morning. I had an if with [[ ]] and one with [ ] so I hoped for advice. No errors or warnings. Then I didn't bother using it anymore. Furthermore, I switched the script to use deno!
shellcheck won't force you to prefer one of the two variants, hence the difference is relatively subtle to a degree that even documentation is not superhelpful either: https://www.oilshell.org/blog/2016/10/12.html
shellcheck will show you many more mistakes that are much more painful your example and also very common.
Deno is definitively an interesting language, but Shell scripting is the gateway to bootstrap so many of your beloved programming languages. If your environment allows it always prefer a proper programming language over shell scripting. Shell scripts are meant for Unix operating system operations, setting few variables, combining certain data-sources, doing filesystem operations, ...
A useful thing with conditionals is shellcheck will warn you not to write [ foo -eq 1 && bar -eq 2 ], but to write [ foo -eq 1 ] && [ bar -eq 2 ] or [[ foo -eq 1 && bar -eq 2 ]].
I do think it would be nice if shellcheck was more opinionated, or if it was possible to create rulesets that have stronger opinions. For example, I would personally like to enforce the use of double bracket conditionals in my scripts, but I know that others prefer the single bracket variety.
This is one of the best pieces of software. It's really changed the game and I don't think it would be easy to safely use shell in large teams without it.
Since the project I work on (https://spdk.io) largely produces a set of executables as output, it was most natural to write the tests in bash. There's one top level bash script that kicks off the full suite of tests and thousands and thousands of lines of tests all written as bash scripts stringing together calls to these executables.
One of these tests is to run shellcheck against all of the scripts in the repo. We don't allow any modifications to scripts without shellcheck giving them the green light now. The quality of our tests has increased dramatically since this was instituted - it's a really great tool.
lintshell is an early prototype of a shell linter based on the Morbig trustworthy static parser for POSIX shell, based on the Why3 platform for deductive program verification.
73 comments
[ 2.8 ms ] story [ 148 ms ] threadThere is also a LSP at : https://github.com/bash-lsp/bash-language-server
https://github.com/koalaman/shellcheck/wiki/Checks
https://doc.rust-lang.org/error-index.html
It was a while ago and seemed a bit gimmicky, but i really enjoyed the ability to see what could be done about an error by opening a web page (without having to manually dig through StackOverflow or other sites).
Now, if we'd get a development framework that'd couple documentation like that, with user generated comments (like PHP has, for example see the bottom of https://www.php.net/manual/en/function.strcmp.php ), then i think the developer experience would improve a bunch!
Who knows, maybe even allow users to contribute possible fixes and allow those as autocomplete solutions and you've just improved on what tools like Codota do a whole bunch!
https://github.com/hadolint/hadolint
See https://github.com/find-sec-bugs/find-sec-bugs/blob/master/f...
I agree that it’s really handy, especially when implementing the check in a CI pipeline with lots of developers.
Each rule has its own url. Tool therefore easily gives you a link.
https://staticcheck.io/docs/checks
What people don't always expect is the immense portability that POSIX-compliant Shell offers you. This thing runs on pretty much everything.
https://manpages.debian.org/checkbashisms
For someone who’s not used to them they might feel annoying (e.g. why should I wrap every $() in quotes?! Ugh) but once you accept them your code will be a million times better and safer.
I can credit Resharper for a large part of my proficiency in C# for example.
https://github.com/HenrikBengtsson/shellcheck-repl
This tool validates your commands at the Bash prompt using ShellCheck and refuses to evaluate them if there's a mistake. It ignores a set of rules that doesn't play well with oneliners.
(Disclaimer: I'm one of the authors)
Click on the site link[1] in parenthesis to the right of the title, you will see.
[1]: https://news.ycombinator.com/from?site=github.com/koalaman
It's like in this xkcd. It's always new to somebody.
https://xkcd.com/1053/
Then again, I've learnt more than once that you start with "I'm just capturing a few commands in a script" and the next thing you see is a mess of special characters, crude syntax and nasty error handling. Don't be like me...
[1] https://www.pyinvoke.org/
What I love python for is scraping web pages and dealing with JSON or xml it rocks for that.
One thing which might be worth considering is adding this to your personal ~/.shellcheckrc to make it more pedantic:
enable=all
https://github.com/kward/shunit2
https://github.com/anordal/shellharden
Lessons learned from writing ShellCheck - https://news.ycombinator.com/item?id=22279585 - Feb 2020 (46 comments)
Shellcheck: a static analysis tool for shell scripts - https://news.ycombinator.com/item?id=9001931 - Feb 2015 (46 comments)
ShellCheck: a static analysis and linting tool for sh/bash scripts - https://news.ycombinator.com/item?id=8777705 - Dec 2014 (20 comments)
ShellCheck – Online shell script analyzer - https://news.ycombinator.com/item?id=8182745 - Aug 2014 (14 comments)
shellcheck will show you many more mistakes that are much more painful your example and also very common.
Deno is definitively an interesting language, but Shell scripting is the gateway to bootstrap so many of your beloved programming languages. If your environment allows it always prefer a proper programming language over shell scripting. Shell scripts are meant for Unix operating system operations, setting few variables, combining certain data-sources, doing filesystem operations, ...
I do think it would be nice if shellcheck was more opinionated, or if it was possible to create rulesets that have stronger opinions. For example, I would personally like to enforce the use of double bracket conditionals in my scripts, but I know that others prefer the single bracket variety.
If it was
then shellheck would reportOne of these tests is to run shellcheck against all of the scripts in the repo. We don't allow any modifications to scripts without shellcheck giving them the green light now. The quality of our tests has increased dramatically since this was instituted - it's a really great tool.
All shells take the -n option to perform a basic syntax check.
Debian has a script for checking scripts for bashisms:
https://manpages.debian.org/checkbashisms
bashate is a automated style checker for bash (similar to pep8 for python):
https://opendev.org/openstack/bashate
lintshell is an early prototype of a shell linter based on the Morbig trustworthy static parser for POSIX shell, based on the Why3 platform for deductive program verification.
https://www.irif.fr/~treinen/colis/ https://github.com/colis-anr/morbig https://github.com/colis-anr/lintshell
Shellharden helps rewrite scripts for ShellCheck compliance:
https://github.com/anordal/shellharden/
shfmt is gofmt for shell:
https://github.com/mvdan/sh
ShellScriptFormatter is another formatter:
https://github.com/osalvador/ShellScriptFormatter
Some of these tools and other tools can be automatically run by check-all-the-things:
https://github.com/collab-qa/check-all-the-things/