Ask HN: What high level language for writing a command line utility?
I notice that a number of shell utilities are written in c. I have some ideas for some utilities, but I won't be learning c anytime soon. And, it would seem to introduce a level of low level concerns, such a memory management, that would be a pain for a small, simple utility.
What high level language would you recommend for writing a utility? Off the top of my head, it seems that python and ruby would be the top contenders. Bash itself seems to simple to write a decent sized utility in; I understand things like arrays are cumbersome
14 comments
[ 0.26 ms ] story [ 40.3 ms ] threadIf I'm building something a bit more robust or integrated I'll use Python or Groovy or Node.
An example would be: we have a Java web app with some REST APIs for updating content. Instead of rebuilding the models, I just pulled the domain objects into a core library and wrote a Groovy CLI around it.
I tend to save Groovy for bigger projects, or take my bash scripts that get unwieldy and move them over.
For maximum portability, you'll want to stick to POSIX shell scripts. If you want to make use of complex data structures, then you'll probably want Perl. You can emulate arrays in POSIX shell fairly easily, but not associative arrays (aka dictionaries, hash tables).
[1] If you care about BSDs, know that Bash isn't installed by default, and I've never needed to install it except on interactive shell servers where squeaky users were adamant about using Bash. Interestingly, Solaris has recently started shipping with Bash. But Perl is available natively on all these. I don't know if Perl is installed by default on AIX, but it's present on the AIX shell I have access to.
[2] IME you'll find Perl at least as often as Bash, though it's not universally the case. I never understood the recent obsession with Bash scripting. Its extensions are poorly designed (ksh is much more elegant) and it has a horridly complex implementation with subtle gotchas. The one bright point about Bash is that it has strong POSIX compliance. In any event, if you want to use Bash because of its language extensions, IMO you should just use Perl. But if you can deal without nice language constructs, you may as well stick to POSIX compliant code. Among other things, the POSIX shell documentation (the Standard Unix Specification spec is free from opengroup.org) is simple and concise and easily navigable, in stark contrast to Bash documentation. Read through the "Shell Command Language" chapter a couple of times and you'll understand shell scripting better than the vast majority of seasoned Unix hackers.
I prefer having a local HTML copy because even though the frames-based site is eminently navigable, occasionally it's useful to be able to use grep(1) to search through the standard.
Another useful resource is the bug tracker for the SUS/POSIX working group. You can browse a list of the errata and requested future changes at http://austingroupbugs.net/view_all_bug_page.php
For years Sun was the de facto steward of the working group. But as is evident, with some sleuthing, from the above database, Red Hat is the de facto steward today. Which means if you're curious where Red Hat intends to evolve (or lock down) the Linux user land, watching where the future POSIX/SUS standard is going can be revealing. Commercial unices like Solaris and AIX have begun implementing Linux API and utility extensions directly, but the BSDs tend to emphasize implementing existing and forthcoming POSIX-defined interfaces. In any event, unlike 15 or 20 years ago POSIX compliance is exceptionally strong across the board, but especially on open source platforms. If you're into systems programing, it pays to be familiar with the standard. It may seem esoteric, but IME the commitment to the POSIX standard by the community is stronger than it has ever been. Everybody is asymptotically approaching complete POSIX compliance. I always code to POSIX first and only supplement with native interfaces when there's demonstrated utility. Linux has strong backward compatibility, but native interfaces can change and are more likely to change than those required for POSIX compliance. For example, Linux deprecated the BSD sysctl(2) syscall years ago in favor of /proc, which broke a ton of applications, including Tor. Even though Linux _technically_ still supports sysctl(2) via a compile-time option, it's been removed from all modern Linux distributions AFAIK (certainly Red Hat awhile ago, more recently Ubuntu). By contrast, even if Linux doesn't support some POSIX interface currently, it's likely that support will be eventually added. Similarly, if you relied too heavily on glibc extensions 10 years ago, you would have made alot of trouble for yourself when musl libc came out.
I'm very pragmatic when it comes to maximizing portability, and I don't write strictly POSIX compliant code. In fact, for C programming I've found that the worst thing you could do for portability is define the _POSIX_C_SOURCE macro as it hides both native interfaces as well newer POSIX interfaces that are de facto portable. (Most systems support most of the SUSv4/POSIX-2008 interfaces, but nobody supports the complete suite of interfaces, so defining _POSIX_C_SOURCE=200809L can backfire, and _POSIX_C_SOURCE=200112L is needlessly restrictive.) And my "POSIX" shell scripts always use #!/bin/sh, even though on Solaris the POSIX-compliant shell is at /usr/xpg4/bin/sh, and shebang interpretation isn't even required by POSIX. (IME /bin/sh is sufficiently compliant everywhere, as is shebang inte...
Manual memory management isn't really a problem for small, simple utilities, it becomes a problem on larger scale systems. C is a rather simple language to learn.
I would want the codebase to be accessible to a wide audience.