Another example: the modern Bash shell vs the Plan 9's Rc shell. The difference in documentation is over tenfold, with the latter still being more less laden with gotchas:
$ man bash | wc -w
48189
$ 9 man rc | wc -w
3264
Having said that, I still love me some sick diff -burN-style mnemonics.
but rc doesn't have built in regex and doesn't have hash tables. My guess is that bash is trying to be a full scripting language these days, sort of like perl.
Because rc is a shell to run commands, hence the name rc, run command. For the tasks you mentioned you would use a program or language such as awk to handle those cases.
>but rc doesn't have built in regex and doesn't have hash tables
That's an advantage; Plan 9 provides awk for those use cases. There is a clear separation, and a simple yet powerful language for communication (pipes, file descriptors, and files).
>bash is trying to be a full scripting language these days
Arguably Bash over-did somewhere around the time they added support for TCP connections
exec {FD}<>/dev/tcp/news.ycombinator.com/80; echo GET / >&$FD; cat <&$FD
Actually, that's something that seems inspired by Plan 9 with its /net/tcp/xxx files. Bash just tries to make it cross platform so it cannot rely on the host OS like rc does.
Maybe there was a clear separation 30 years ago, but there no longer is. The shortcut for the posts below is that you can write a Lisp in all of shell, awk, and make (and sed too) [1]
I won't get to this in Oil (without help) but I still think it's valid. The languages have so much overlap that they should just be combined. Of course that's what Perl was (leaving out Make), but Raku and Python 3 seem to have left a hole in that design space.
If you are running scripts most of the time bash is overkill (batch files don't need things like command completion, history and all that interactive stuff). Don't believe me?
$ ls -lAh `which bash`
-rwxr-xr-x 1 root root 1.1M Jun 6 2019 /bin/bash
$ ls -lAh `which dash`
-rwxr-xr-x 1 root root 119K Jan 25 2018 /bin/dash
The entirety of plan 9 cat is just 36 lines. That's true unix philosophy. Small tools that do one thing well which when combined produce more powerful tools. The -v flag is silly bloat as cat is nothing more than a program which catenates files.
This is another reason why I'm developing an ad-hoc data representation with strong typing. JSON was almost right, but failed because there are too many fundamental data types that cannot be represented in a portable way.
A general purpose ad-hoc data format with all the fundamental types, that doesn't require endless text parsing but can be converted to a 1:1 text format for human viewing, would be helpful in so many situations, including serialization for piping between unix tools.
There is a comparison table on the linked GitHub page, with the following formats: Concise (the proposed format), XML, JSON, BSON, CBOR, Messagepack, Cap'n, Protobufs, Flatbuffers, Thrift, and ASN.1.
Although not technically a pro/con list, it highlights the evaluation criteria used to compare.
Hmm, I think your table is using a different definition of "zero-copy" than is currently common when discussing serialization formats.
In your comparison you claim Concise is zero-copy, but it does not appear to be so, at least in the sense that Cap'n Proto and FlatBuffers are. Concise appears to use variable-width integers, tag-value sequences, and 8-byte alignment, all of which make encoded messages unsuitable as in-memory data structures; it looks to me like you have to parse the message into some sort of AST before you could meaningfully operate on the content. "Zero-copy" in the Cap'n Proto / FlatBuffers sense means that there's no need to do any "parsing" because the whole message is efficiently traverseable as-is; e.g. you can mmap() in a very large message and then find and access any one value within it in O(1) time (or perhaps O(log n), if the data structure is nested). This generally requires that primitive values have fixed widths and fields have fixed offsets within their parent object.
Digging into your docs it looks like what you really mean by "zero copy" is that individual strings or byte sequences can be used directly without copying them out of the original message buffer. This is a fairly common property of any binary protocol; e.g. Protocol Buffers can do this (albeit without NUL-termination), but in your table you have indicated that it cannot. I would maybe call this "zero-copy strings" to be clearer.
On another note, in your table you have indicated that Cap'n Proto doesn't have a "String" type, but this is not true -- Cap'n Proto's "Text" type is a UTF-8 string (and is even NUL-terminated on the wire).
The comparison table doesn't seem fair, especially on the XML column. In particular, it is stated that XML doesn't support decimal and float, whereas XSD in fact defines 19 primitive data types including decimal and float. List are also supported implicitly because child nodes of an element are ordered. Moreover, the claim that XML doesn't support metadata is devious because any file can precisely be extended with any additional metadata (possibly in their own namespace), without client breaking if done well.
XML strength also lies in its extensibility, and its large number of compliant implementations, which goes way way beyond a single uncompleted implementation by a single author.
The problem is that you need an XML schema (or at least inline annotations on elements) to designate some value as decimal or float - it's not encoded directly in the data. I think that's what OP really meant.
Yes, but it's too rust-specific. For example, specifying struct names ties the document to a particular implementation, and isn't helpful for a general purpose data representation. Also, it's text only.
RON is fine for rust, but not for other languages that might not have its feature set.
I am with the author on the general point. But I think it ignores the fact discussing how things were in the '60s when the commands were written just for people ignores a fundamental change. To wit:
>the added options are convenience flags: setting the permissions of the new directory and making parent directories if they don't exist.
Obviously completely unnecessary for normal users, but those have to exist for automated processes. The "problem" is caused in part by the success of Unix, that now computers are using computers.
If you believe mkdir -p is completely unnecessary for normal users and only exists for scripting, you probably want to think about how it might fit into your workflow again. In fact, my mkdir is simply aliased to mkdir -p, and I have a self evidently-named wrapper function called mkcd which I use even more often than mkdir.
Well a script can do path parsing too, it’s just rather painful. Similar (actually more complex) real world task that does get implemented in POSIX sh: realpath(1) replacement, when it doesn’t exist.
But he doesn't ignore it... the last paragraph says:
> To be clear, I'm not saying that I or anyone else could have done better with the knowledge available in the 70s in terms of making a system that was practically useful at the time that would be elegant today. It's easy to look back and find issues with the benefit of hindsight. What I disagree with are comments from Unix mavens speaking today, with comments like McIlroy's, which imply that we just forgot or don't understand the value of simplicity, or Ken Thompson saying that C is as safe a language as any and if we don't want bugs we should just write bug-free code. These kinds of comments imply that there's not much to learn from hindsight, we were building things as well as anyone could build things in the 70s, all we have to do is go back to building systems like the original Unix mavens and all will be well. I respectfully disagree.
In other words, the real problem isn't that modern software is getting more complex (as there is a good reason for it); the problem is that we're insisting on using an environment that is not up to the task of dealing with modern complexity.
For an example of how insane this world is: consider putting your source trees in a location that contains a space in its name. How many build systems are going to completely and utterly fail to build your code? And... why should we tolerate this situation?
I think its just a reflection of how much more widely software is used, how complicated our environments are, how many admins/developers there are compared to the old days. Eg the aws command is hugely complicated, but I'm not sure breaking it out into smaller commands easily really would make anything easier, just pollute the command line namespace.
In my web-based shell implementation [0], I use a library concept, such that you have to "import" commands into the current execution environment. You can either do this kind of thing:
$ import fs
This imports all the commands in the 'fs' library, without any concern for namespaces, and if there are any already existing commands with the same name, they are not overwritten (the new ones are not imported).
But if you do this:
$ import fs as myfs
Then, all the commands in the 'fs' library are invoked like this:
Isn't it possible that, like human languages that evolve over time for multiple reasons (https://en.wikipedia.org/wiki/Language_change) such as language contact, principle of least action, and pejoration, that computer languages also experience similar shifts? All of this grumbling about how command line options have exploded or why doesn't everyone use language X or shell Y or OS Z that has existed for forever is similar in tone to me about people who grumble about Oxford commas or starting sentences with conjunctions or (heaven forbid) using emojis?
Of course some principles of speech and syntax should stay around (as they probably exist for a good reason), but it is foolhardy to grip tightly onto any language, human or computer, thinking it will always stay the same, when it plainly won't.
I'm assuming you didn't read the article, because a lot of the author's examples have nothing to do with evolution or complexity.
Also please don't compare human languages to programming languages to make a point. It's the programmer equivalent of bad managers comparing Java to JavaScript. Argument by analogy in such situations is a sign of not having a basic understanding of the problem at hand. Yes I am aware of the contradiction.
computer languages also experience similar shifts?
Partly, but less so.
Human language vocabularies approximate Huffman Coding - the more frequently used words (and phrases) tend to be proportionally shorter. This is a gradual process for spoken languages, and somewhat quantized for written languages.[1] It's possible because with human languages, "close" is "good enough"; one can read english written with ye olde orthography, if with a bit of extra effort.
Less so in the lifespans of particular programming languages. Per Postel's law we tend to be liberal with extending but conservative with breaking backward compatibility. I think the case of command line tool parameters is particularly conservative for various reasons; among them because the 'knowledge' of options is spread among multitude of separate project - many of them obscure and bespoke - and coordination would be nigh impossible.
However there are some signs of similar trends; note how the "function" and "integer" and similar keywords of older languages tend to turn into "fn" and "int" in newer languages.
--
[1] Not sure if there's a comparable process for grammars, but I guess it's a fair bet...?
I agree, that's why I mentioned principle of least action. But you also forget the power of idioms. We would connect to a file with a file handle using something like open("file"), and then have to close it with close(handle). This got tedious (and dangerous because people would forget to close files), so people would write macros that would automatically open and close file handles, and soon languages would have the "with" idiom that would handle the context for you. And this is not the only case of an idiom causing a paradigm shift in languages.
Pejoration happens too. When is the last time you used a GOTO in code? And yet, 40 years ago, you couldn't write a BASIC program without one. And the revulsion against NULL causing a shift to Maybes and optional types to the point where the concept is reified in many languages now?
All I'm saying is that there are multiple mechanisms that are changing the languages we write code with, and that it is changing far faster than people realize.
Systems grow in complexity and as a result you want to do more with them and for them, all while maintaining decades of backwards compatibility. Which is why all those basic tools presented in the article, that have existed over the past 4 decades still do the same and more. They have to cater to scenarios that didn't exist 4 decades ago but also to the ones that did.
That’s an excuse. Of course demands will increase over time—all the more reason to build a system that scales well! Unix Philosophy doesn’t demand much: just small simple composable tools, each of which does one thing well.
So why does `ls` have a recursion flag? Why does `cp` have a recursion flag? Why does `chmod` have a recursion flag? Why does `chown` have a recursion flag? Why isn’t recursion its own command, that takes as argument the command to apply at each level?
Ditto untyped, untagged streams, which mean every tool implements its own custom UI behavior when attached to a terminal, instead of outputting tagged data and letting the consumer run it through system-standard data formatters.
This is not good—or even borderline competent—systems design. This is bullshit. Because the people who slapped it all together are bums, and the people who accepted it didn’t care enough to call them on it.
But noooo, we can’t change anything now; because “backwards compatibility”.
Protip: when you build a system that can’t evolve, that grinds to a halt under its own impossible weight, it’s time to confess “yeah, we totally ballsed that one up” and start over again, lessons learnt. Because maintaining that status quo is a tyranny: dragging everyone else down to the level of its incompetence, and beating them to death with intransigence. No prizes for guessing who that benefits: the bums who built it. And who pays for it: everyone else.
It’s a scam. Perpetrated by and for those who cannot admit they are wrong.
Smash it down. Because whatever comes after it should be so much better, the only question will be “Why the frell did we wait so long?”
--
“Those who make peaceful revolution impossible will make violent revolution inevitable.”—John F Kennedy
Why did ls even need 11 parameters to begin with? Why not exactly 1? Or none? Why did tar need 12? Why did find need 14? The 28 tools in the example (excluding non-existing ones) had an average of ~3 parameters per tool in '79. Should 2017 have brought us 200 extra individual tools to provide the same additional functionality? You want another tool that does recursion for ls to save us from "incompetent bums" and "grinding to a halt"?
Everything around you grew in complexity to adapt to your needs because there's value in consolidation. It's why you are now typing from a device that is orders of magnitude more complex and integrated than the one used to write down the Unix philosophy.
You ask the question "why" but fail to answer the "why not". Why would a tool that still does exactly what it used to do in the past not be able to also do more especially when it's still within its sphere of relevance? And not the "out of principle" answer, not the handwaivy, tin foil hat answer, not the offensive and insulting answer that you provided above (bums? incompetent? bullshit? scam?).
Times change and philosophies adapt to remain relevant. "We must use time as a tool, not as a couch" - JFK.
> You want another tool that does recursion for ls to save us from "incompetent bums" and "grinding to a halt"?
Yes I absolutely do. Remember UNIX? "Many tools, each small, each doing one thing well". Adding hundreds of flags to older tools does NOT achieve that. Your point here?
> Everything around you grew in complexity to adapt to your needs because there's value in consolidation.
Please don't cross into abstract philosophy. It adds absolutely nothing to a specific technical discussion.
> It's why you are now typing from a device that is orders of magnitude more complex and integrated than the one used to write down the Unix philosophy.
I use my computers as the complex messes and a series of train-wrecks that they are, and now hear me out please, because I literally have no other choice. Be it Win10, Linux, Mac, it's honestly almost irrelevant. One example: to this day nobody can make a reliable system-wide sleep mode that is not messed up by extra hardware (like my external HDDs), on neither of those 3 major OSes. In 2020.
I can't go back to my Apple IIc. I will lose my job if I do.
What's your point here? That such a complexity is inevitable? Is that it? But who argues against that? Of course it's inevitable, we see it in every area of life -- it's one of the aspects of entropy. What's being discussed is: "isn't it time to take a firmer stance and start using newer tools that don't have these problems and protest against the old ones by not using them"?
I personally answered "yes" and I am gradually doing more and more of it -- already replaced grep, ls, find/xargs, about 13 in total IIRC. You know what? My productivity on the terminal skyrocketed. I started ditching languages with too much magic and implied behaviour and too weak/random typing (PHP/JS come to mind) and moved to others which, while not covering everything that I'd like to have (like Elixir), are still letting me sleep that much better knowing that `[] == 0` is actually false and I won't have to curse my way through logs in the next morning.
> You ask the question "why" but fail to answer the "why not".
If you actually frequent HN and aren't just looking for conflict then you'd know that literally every month we have a few rants by people who are now burned out that decry the current state of affairs. But if you really want me to answer then I can likely summarise it to:
"Because our tiny brains don't work well with so much complexity in the long run. We can cope for a few years, some even for 20 and more, but eventually we can't. Because having smaller, more-focused, and more (total count of) tools that each do something really well, has been demonstrated to work better everywhere, not only in IT."
(But I am aware this has been contended endlessly and it devolves into belief and tribalism and science and historical evidence goes out the window in the first three comments tops. Sad. Shouldn't happen to techies. We should be better than that.)
> Why would a tool that still does exactly what it used to do in the past not be able to also do more especially when it's still within its sphere of relevance?
Because it's NOT, and your parent commenter gave you an excellent example which you immediately dismissed and THEN proceeded to ask your question again, pretending he never gave an example. Are you aiming at being disrespectful on purpose? Confusing.
> And not the "out of principle" answer, not the handwaivy, tin foil hat answer
You are aware you are crossing into ad hominem and almost insults, no?
> Times change and philosophies adapt to remain relevant.
Well that's a perfect example how the same sentence is viewed in two completely different ways. This is EXACTLY the problem, dude: philosophies don't adapt at all...
I'm sure a cursory parse of the comments will help you see I haven't insulted anybody, merely literally quoted the insults underpinning OP's comment, now flagged.
Simplicity means different things to different people and at different times. You are both making the same argument for simplicity for the sake of simplicity with no objective measure for it other than a "guideline" philosophy of "does one thing well", and just dismiss the majority who doesn't fit your "burnout as proof" argument.
If you want to wave the philosophy flag you must also acknowledge that either it never applied to pretty much anything as such, or it implies you should adapt your tools to the times, the facts, the challenges.
> Again, your point?
Take full advantage of the times you live in or be left behind, burnt out, reminiscing the glorious past I guess...
> Everything around you grew in complexity to adapt to your needs
Unix provides rather limited primitives for composition, and a lot of things remain unsafe or very difficult to do right in shell. That's why we keep extending tools and doing things (rather redundantly) in a real (if outdated and limited..) programming language.
A lot of that complexity is due to poor design, not due to inherent complexity in the user's needs.
If mods have a problem with my attitude, please close my account. Because I’d rather be rude and right than polite and wrong, and these regular apologetics really get my goat.
This culture and profession is far better at making excuses than systems. That should be utterly unacceptable when these systems impact the lives of BILLIONS. Priorities, people.
Just yesterday in fact, while implementing a state machine. Except the developers of that particular language explicitly did away with GOTO, so I ended up approximating it with a SWITCH/CASE. At least the language supports fall-through...
Which is to say, people differ in judgements of various features (footguns vs swiss army knifes). Moreover, the judgements seem to oscillate as fashions, in ~10 year long cycles. Please recall how exceptions and templates used to be a hot thing back in the day. Ditto garbage collectors. Presently we're all in love with Rust, and the "JITs considered harmful" paper will drop any day now.
> the more frequently used words (and phrases) tend to be proportionally shorter
It's interesting to think about how things can be swapped dynamically in this system. Consider the evolution of 'telephone.' In the 1980's, 'phone' referred to a land-line telephone. In the 2000's, it was ambiguous; for young people, it meant 'cell phone' most likely, or we started calling them 'house phones' for land lines. Now 'phone' is mostly used for a 'smart cellular telephone'.
Another interesting example is "Playstation". That always refers to the current generation, and we retroactively apply "PS1."
It will be an interesting period for history in the future to try and decode our present-day meanings since so many words are period specific.
> Less so in the lifespans of particular programming languages.
You could probably trace a similar evolution across programming languages, just by systematizing the phrases/words used in each to accomplish the same thing.
Human language doesn't get more complex over time. Humans have limited mental capacity. We are creating software that exceeds out ability to understand it.
This bums me out as well. One of the prime examples of this is tar: whenever someone says "extract this tarball", they usually give the command as this:
tar xfvz my-source-code.1.2.3.tar.gz
What does those flags mean? Who knows! I mean, I know because I've used it enough, but for many people those are just a mysterious incantation (there's even an xkcd about it! [0])
The reason is arguably that tar has too many responsibilities here. Tar should only do two things: create tar streams and extract tar streams into files. You do that with "tar c" and "tar x", and that's everything you should need to remember (ok, fine, "tar t" for listing files is good to know as well).
Commands to create tarballs should look like this:
tar c directory | gzip > directory.tar.gz
And untar commands should look like this:
zcat directory.tar.gz | tar x
or maybe
cat directory.tar.gz | gunzip | tar x
or whatever version you prefer with "tar x" at the end. This is composable, super-easy to remember, and you can VERY easily swap out any parts for any other parts (for instance, changing the source from a file to a url with curl, or the destination for an ssh pipe or something).
One of the benefits of the "swapping out" parts are of course that you're not restricted to what's built into tar: you can use ANY compression utility you want. For instance, there's a thing called pigz[1] which parallelizes gzip decompression for increased speed. It's trivial to swap into the pipeline when you use tar that way:
cat directory.tar.gz | unpigz | tar x
Congratulations, you now have much faster expansion of tarballs, without needing to recompile tar.
We've taught a generation of programmers that "tar" is hard to use and you have to look up a magic incantation every time you need it. Ironically, that makes tar less flexible, because now you think of it as a single-purpose utility (creating and unpacking compressed tarballs) instead of a versatile unix tool which can be used in all sorts of different situations.
Modern tar can detect the format so "tar xfv" is enough. The "v" option only means "verbose" so it's better just "tar xf" and avoid spamming the terminal.
I always use -v because every so often I find a tarball that was taken from inside of the working directory instead of outside. Of course these days it's not humanly possible to ctrl-c it before it extracts thousands of files and makes cleanup a chore but at least you won't be confused as to what happened when you attempt to cd into the directory later. For that matter knowing what directory to cd into is also a good reason to use -v.
Note: If none of your files were clobbered and there are no spaces in the filenames to worry about you can do:
tar -tf [tarball] | xargs rm -d
Run it repeatedly if you also want to remove the directories.
If I want to untar a file, odds are I also want to decompress it so I like having both features in one tool. That said I also really appreciate your composable approach. I wish my early unix education had leaned in pipes a lot harder.
But what I came here to say is that I find the typical tar flags pretty intuitive. X means extract, z means using zip/unzip, v means verbose, and f for file I/O instead of stdin/stdout. For typical use cases, I don't find its interface magical but logical. I'll freely admit to needing 'man tar' anytime I stray from the typical use, but that's true of a lot of commands.
The bigger offender in my mind is rsync. I remember auvP as my typical flags but don't ask me what they do. And good luck guessing whether my destination or source path need a trailing slash on the first try...
Yeah, it's true, rsync is a real nightmare when it comes to arguments.
I remember reading that xkcd comic though and I went "yup, that's me!", and I've spent a bunch of time thinking about why that was. You list those flags there and say there intuitive, but they're unique to tar, you have to memorize them, and there's no reason that they need to be in tar when they're available right there in the shell.
Why do I need to memorize that the "f" flag is "file output/input", when I can just write "> dest-file" or "cat src-file |" (or "< src-file") in my shell? That works for every command, not just tar. Why do I need to remember that the z file is for gzip (what's the equivalent flag for bzip2?) when gzip/gunzip exists and works for anything?
So many people simply don't understand what tar does or how to use it without googling the magic formula they need, and it's because of all these damn flags that everyone insists on using instead of doing it the "unix way".
Backwards compatibility is a bitch mostly. Tar defaults to using the tape drive because that's what it was originally designed to do. Using it to create file archives was a hack added later, but nobody wanted to break backwards compatibility with old backup scripts by changing its defaults, so we're stuck with its oddball syntax today.
dd is another program where the commandline flags predate modern usage (although it does at least behave nicely with pipes by default) and look totally weird compared to everything else. But again nobody wants to touch them because it would break millions of existing scripts.
That's true, and it's nice that it works, but it's just more magic information you have to memorize about tar. And it works one way but not the other. What is the flag for creating the archive compressed with bzip2? Without looking it up? I have no idea. "b" maybe? Does the file name come before the folder name, or is it vice versa? Who knows.
This problem is universal. The solution is to use --words instead of -s single letter options. Thankfully, git has a good manual that covers all the options:
that was confusing. when i read this, it was exiftool, so i was going to reply with ffmpeg. looks like you felt it was a better example as well
anything as powerful as FFMPEG, EXIFTool, ImageMagick, x264/265, etc is going to have switches/options from hell just out of plain necessity. these are not simple little Unix-esque tools. these are full blown applications that just happen to not have a UI.
The same thing goes for ImageMagick, but that seems unfair because it's more that they have created a command-line language for describing multimedia transformation pipelines.
It's not as much that the arguments allow you to run a program under variant configurations, but rather that you create completely new programs for each run.
I'll note that video encoding GUI screens tend to be overwhelming as well. It's simply a complex problem with a whole lot of parameters you might want to tweak.
The alternative is to hide all of the complexity and hope that the defaults are good enough for your users. Which works ok for stuff like Mom encoding her cell phone video but falls down pretty quick when someone needs to do something slightly out of the box, like add in a second audio track.
For the later years, it would be interesting to see the comparison for GNU fileutils vs what's in the base of FreeBSD or OpenBSD vs what's offered by busybox. The author mentions this briefly with Darwin's ls command (which looks like it might just come directly from another BSD).
Also tar's increases might include new compressions methods like bzip2, XZ, etc.
Right, but what business does tar have compressing anything in the first place? It's a tape archiver. You can pipe its input/output to compress or gzip or bzip2. Early versions didn't have any ability to compress/decompress archives and relied on external tools, but then that functionality was absorbed, increasing the size of the codebase. And as you point out, every time a new compression method is invented, someone needs to update a bunch of tools instead of just installing one new utility.
> but what business does tar have compressing anything in the first place? It's a tape archiver.
Most of the tape devices themselves had/have compression on them so that tar did not need to do it. However, that compression was pretty weak (2:1), so giving tar the ability to use bzip2 and send the compressed data stream to the tape (bypassing the hardware compression) gave the user much more convenience. Also, older systems probably were bad at doing the compression in real-time, so it probably slowed down the write to tape. I'm just guessing, I have no experience writing to tape from such an old system. I'm not really upset by tar geting some extra features.
Seems like a natural progression given the growth in usage of these tools over the last few decades. How many people were using these tools in 1979 compared to today?
Command line options are a terrible surface for usability and more importantly, discoverability. We are at least two decades overdue for replacing them with a structured, typed, reflective API and yet every generation of hackers persists to cargo-cult the cli utility, taking pride in remembering copious minutia as if they were wizzards in command of arcane incantations.
We very much do, at least in their hats as hackers (true computer scientists never cared much about actually using computers) there was a point where an online command prompt (one you could edit in real time, as opposed to having to put in a new punch card etc.) was state of the art of interactivity and software interoperability was defined on the level of word sizes and hardware registers, but we can and should move on from that.
If you don't know, when you run `echo -n foo bar`, the shell splits that according to its syntax and runs `echo` with the arguments, `-n`, `foo`, and `bar`. The array of those 4 strings is argv.
To better expand on what fctorial is probably trying to say, argv is a simple list of strings. Plain text is a nice common ground among the myriad of programming languages that executables can be written in. In order replace options with "a structured, typed, reflective API", argv could no longer be a simple list of strings. There would have to be a common typesystem among all programming languages. It wouldn't just be needed for the understanding of argv, but also for the output, since the output of programs is used to feed the arguments of other programs. This puts heavy restrictions on the liberty programming languages can take in defining their own typesystems while retaining compatibility with such an inter-program typesystem.
This isn't to say that it can't be done, but I'm fairly certain it would suck on multiple fronts. You might be able to gain some things, but you'd certainly be giving away others.
EDIT: I don't want to spend too much time expanding what you'd be giving away, but to start
1) Having the common ground be plain text means that you get 1 form of output that can be used both for human and for machines to read. The programmer might not even intend for their output to be used by other programs, but it will always be trivial to do so. You can propose that they could have 2 forms of outputs, one plain text for humans and one typed for machines, but there would be no guarantee that they would have the same information. It's likely that one form will miss information that the other has, and you'd be left with humans reading output meant for machines or machines reading output meant for humans, so you gain nothing.
2) Less brevity. You complained about options being too detailed ("remembering copious minutia"), but that's because they're meant for interactive use. This is not a problem about the interface being untyped. It's about the interface optimizing for being quick to use. This would either also arise with a typed interface (although more difficultly so) or the CLI would simply be more tedious to use. No win there.
3) It would be more inconvenient to use interactively. For example, if you want to differenciate between a number 5 and a string "5", you're going to have to add syntax, like the quotes I used in this sentence. You can see the inconvenience by setting you shell to be something like python, for example. There would be too much syntax for interactive use.
>If you don't know, when you run `echo -n foo bar`, the shell splits that according to its syntax and runs `echo` with the arguments, `-n`, `foo`, and `bar`. The array of those 4 strings is argv.
So it does, albeit a better example of the shell doing something nontrivial would be replacing file patterns with filenames, but it doesn't help since my program could be expecting a list of urls instead and has no standardized way to tell the shell this.
>To better expand on what fctorial is probably trying to say, argv is a simple list of strings. Plain text is a nice common ground among the myriad of programming languages that executables can be written in. In order replace options with "a structured, typed, reflective API", argv could no longer be a simple list of strings. There would have to be a common typesystem among all programming languages. It wouldn't just be needed for the understanding of argv, but also for the output, since the output of programs is used to feed the arguments of other programs. This puts heavy restrictions on the liberty programming languages can take in defining their own typesystems while retaining compatibility with such an inter-program typesystem.
Yes, a language-agnostic reflective type system with a shell and supporting language tools, it's a pretty big implementation and coordination effort and an even greater one to retroactively describe the command line semantics of existing commands so not holding my breath here.
>Besides that, it would be more inconvenient to use interactively. For example, if you want to differentiate between a number 5 and a string "5", you're going to have to add syntax, like the quotes I used in this sentence. You can see the inconvenience by setting you shell to be something like python, for example. There would be too much syntax for interactive use.
A matter of preference I suppose as I have been using jupyter/ipython for all "shell" tasks for quite a while.
Brevity is a particularly false saving here if you factor in the time it takes to lookup documentation on something you don't use just often enough to have its command line options sit in your brains L1 cache.
> the time it takes to lookup documentation on something you don't use just often enough
Well, if I don't use it often, I'm not going to be looking at the documentation often. However, when I do, we're talking about the time it takes to type `man foo<enter>/bar<enter>`, where bar can be `keyword` or `--option\b`. That doesn't even take 30 secs.
thirty seconds and a context switch to a man window is a lifetime, compare with an environment where I just type utiltiy.<tab> autocomplete to 'option', have an inline documentation displayed under the line and a seamless choice box that allows me to choose from several valid values for that option while displaying inline documentation for what each means.
I don't feel that kind of friction with man, but ok. That environment you describe is also common. Though I don't use it, in zsh, if I hit tab, I get that menu with inline documentation you describe. What are you arguing for anyway?
Because it's still crude and unstructured - my program still has to operate on an array of strings, and I need to author zsh completion and matching voodoo have it distributed and do the same for everyone's shell of choice whereas I would like to be able to export a struct definition and have my program receive it populated with user input instead of argv
No, or it wouldn't have been overdue.
Microsoft powershell is probably the closest thing I can think of, which in turn is powered by the .NET/CLR type system.
and do you find powershell to be a simpler alternative? i recently had to look into a 100 or so lines powershell script and it felt crazily hard to read. Good thing I had access to the guy who wrote it so that he just told me what it was supposed to do (he gave up explaining the script)
Yes the powershell language itself is a poorly thought-out hack but using shell scripting for non-trivial tasks is a related but different problem that has to do more with a (thankfully narrowing !) skillset gap between devops/IT people and software developers. So just like a 100 or so line bash script would probably be better off written in python so would a 100 line PS script be better done in any .NET language and exposed as a cmdlet
The lack of discovery is not inherent in a command line system. The usual counter-example is something like TOPS-20 https://www.bourguet.org/v2/pdp10/users/chap2#sec_2
where the command line system was built such that for every command you could interactively in the middle of typing your command line get help about what the possible options were, what the argument for the option you'd just typed was, etc (and this was built in to the system and commands, not a half-baked add-on like bash tab-completion).
Didn't mean to imply that it was inherent, see the comment about powershell bellow - just that that this is the standard we unfortunately came to accept as adequate.
To some degree it reminds me of a change in Haskell that came about a few years back. It used to be the case that many of the basic functions operated on things like lists and functions. Tricky to learn but not too bad. It got annoying when you wanted to use those tools on generalizations of lists, so the language changed. Now those standard library functions operate on things like the Foldable type class which is must follow some laws defined in terms of endomorphisms. So simple functions like “any” or “concat” which used to operate on lists and be super newbie friendly still can operate on lists in the same way, but now do lots more, at the expense of the newbies (and complexity).
I think the change was jokingly called “pulling up the ladder behind us.”
Learning some command line commands feels like learning a whole new framework. A man page should rarely need to be a whole damned manual.
They still work on plain lists with the same syntax you could always use (that is, lists are a member of the Foldable type class), right?
If so, then this primarily seems like an issue of how to present the documentation and not the mechanics of the change itself. Instead of autogenerating docs that say concat :: Foldable t => t (t a) -> t a (or whatever) and making people think about Foldable, write docs that say that concat is still [[a]] -> [a], with a note at the bottom saying, by the way, this generalization exists, click here to see the more complicated version if you've decided you care about Foldable. Probably the text of the docs remains the same, in fact ("concatenate a list of lists together into a single list"), you just display the real prototype and explain that "list" could also mean something else. (There's a number of ways to implement this, from writing docs by hand, to extending the doc protocol to support this specific case of lists and Foldable, to extending the doc protocol to support documenting a fake function prototype and then have a link to the real function at the bottom, and using the type system to ensure that the fake prototype is just a special case of the real function.)
A corollary is that documentation presentation is an inseparable part of library / tool design.
The other problem are type errors, which would then require knowing about Foldable to operate on lists.
The same type of solution that you propose for docs can work here too, though. The compiler can trivially substitute the typeclass for any known instance.
It seems like businesses are pushing towards releasing command line tools to make their business "hacker friendly." One example I can think of is Github and their new GitHub CLI.https://github.com/cli/cli
I feel this way about optional function parameters. They have a tendency to proliferate, because you don't break existing code by adding them, and they're always optional.... until you have https://scikit-learn.org/stable/modules/generated/sklearn.sv..., where you have 15 optional parameters.
So basically tar can behave normally (at least GNU tar can) -- it's just that there's an ancient idiom that most people have in their fingers, and that's copied around in various instructions.
The nice thing about the verbose way is that the hyphen prefix enables autocomplete.
None of those SVM parameters stick out as obviously unnecessary to me. What alternative do you propose for functions that have many minor variants with a large common core, and many hyperparameters?
> that's the beauty of command line options -- unlike with a GUI, adding these options doesn't clutter up the interface
GUIs provide discoverability so often times you don't need a manual. If you don't know how to use a command line tool you must open the manual. So in that way I would consider the man pages as part of the UI and those definitely get cluttered.
I've worked with music software GUIs, these are notoriously plagued by feature-creep.
For a desktop paradigm, there is always a way to do better with GUI as you add more features; but you'll refactor / redesign a lot on the way and that has a cost in familiarity, which directly impacts user productivity. It's a trade off.
The dirty secret of discoverability is that only a tiny subset of features is relevant at any time in the UX, and identifying that subset + triggering user awareness about it is the "art" part. We learned a lot with mobile in that regard because the harsh limit on real estate becomes incentive to identify the essentials.
Discoverability doesn't have to suffer from complexity of the software, but it's not just UI design, it's UX.
And search, something that Ubuntu HUD/Unity (now on the extinction path) solved in a very elegant way. Hundred of menu options nested in remote menu folders made available by typing names. Best of both worlds, if you ask me.
If I were snarky I'd tell you that UX on both mobile and desktop in general is hurtingly bad compared to what it could be.
There are great domain apps (e.g. medical, games, professional audio, amateur astronomy to name a few I've seen) that really break open the format:
- feels like a tricorder from Star Trek, where small is good because it doesn't try to be bigger than it is, rather embraces its limitations are features not bugs (and to his credit, Jobs + Ive successfully translated these ideas into the iPod and later on iPhone early iterations).
- feels like a great videogame UI where despite the complexity, once internalized it's a breeze (even a joy, physically) to manipulate. It takes a lot of testing and feedback and is admittedly one huge QA cost in video game making, so there's why most companies don't do it for 'normal' products whose UI/UX is 'good enough' (or so they think, for now).
- clever integration with features, e.g.: slider on a phone with a tiny motor feedback when you cross a unit bar or even for magnitude; a bigger-deeper 'womp' around the center coupled with a UI magnet that makes you 'feel' and 'see' the slider fix itself on the 0 position. This all designed so you interact while looking at something else, just like a physical button.
I don't know. I feel like we've made steps of giants in the 2000s and then progressively dropped the ball when we realized the simplest UX on mobile yielded the highest engagement (think twitter, insta, how "simple" they are, how limitations actually enhance user activity; in a very questionable way but that's besides the point in terms of profit thus research).
I sincerely hope someday a disruptor kills everyone else with awesome UX (just like Apple did in the 2000s) and forces the market to re-think its abysmal UX standards, and let's not talk about ads because that would be extra-snarky.
If I were snarky I'd tell you that. But I'm not, so I'll leave these remarks for someone else to make.
Fortunately, I'm in a very snarky mood today. I absolutely agree with those things you didn't tell me. And I do blame the "when we realized the simplest UX on mobile yielded the highest engagement" thing. I believe this to be the cause - people figured out that the ROI on shiny is better than ROI on building useful things, so most wide-audience software is now just garbage (and quite often a vehicle to scam users out of their personal data).
The market is allowed to exist because profit is a quite decent proxy for utility. But whenever you are optimizing using an imperfect proxy, it's possible to overoptimize - make the proxy metric look better, at the expense of the actual goal it was supposed to stand for. I believe software (along with much of other market sectors) have crossed that point a while ago.
Indeed. The underlying economics (the harshness of no business model, i.e. no profit = wrong idea insofar as reality/markets won't have it) make it a very difficult proposition — sometimes I'm in awe at the fact that we could pull off "open source", that the idealism of that survived collision with reality long enough to demonstrate its value.
The economy to me is a vast "optimization space", so I can only agree. It's the 'trap' of a local minima I suppose, wherein the friction of moving away to find another possible (but uncertain) minima (hopefully overall lower or more acceptable ethically) is too high. So you've indeed overoptimized your way into a local pit (e.g. the ad model now taking over the news like a virus, resulting in this new "hybrid" object sometimes called "informercial" or "infotainment").
It seems that only "disruption" (of a magnitude worthy of the name) gives enough momentum to escape the steepness of a local historical/economical minima. Unless you've got some new S-curve (which quite literally may represent the math surface of this "escape path"), history tells us we remain stuck and move in circles about the center of 'gravity' of this local minima.
Dunno if that makes sense to you, pardon the loose physical analogies.
That's what differentiates a tool from a toy. A tool is designed to maximize user's productivity in regular use. A toy has to look easy and shiny in the first few minutes of use, so that it secures a subscription.
That ability to search is helpful, but it's particularly helpful because the organization is terrible...and maybe searchability eliminates the incentive to fix the organization. In theory, it should allow rearranging things with less friction.
Hierarchies suck. That's why early web "search engines" failed, the old hierarchical portals, and Google won (once it added a good sorting algorithm to the what the initial search engines returned).
The UI is one input and the "computer" figures out the rest.
Regular users don't want/understand hierarchies, they get easily confused. They also don't want to organize things themselves, see the many failed attempts at tagging files, websites, images, etc. manually.
Every time this is mentioned, people say "shut up shut up shut up, nobody wants hierarchies". Obviously it's dogma. But if you have to keep saying it...
I am probably not a "regular user", but I got a job about a year ago defined by and intended for regular users, and one of the major parts of it was managing emails in a departmental account, using (you guessed it) hierarchical folders, and another was managing project documents in SharePoint, using (what appear to be) hierarchical folders.
I'm aware that people will tell you not to do the latter in SharePoint, that it's not really designed to be used like a filesystem, just use lists, etc. But purely from an anthropological viewpoint, I see how people shove everything on a shared drive, and then they try to move it to something like SharePoint and recreate a folder structure.
So, you know, it's not about what I want, but it's an objective fact people do want hierarchies, even when other people don't want them to. Google is pervasive, but it's not all of computing.
The other incentive with mobile is to reduce the functionality of either the entire app, or any particular view presented by the app. That's not actually the same as identifying the essentials, anymore than saying that "cutting is the essential action" really allows you to distill the _essential_ between an axe, a pair of scissors, a chainsaw and a cooking knife.
I prefer the command-line over the GUI most times - it's a standardized interface for most interactions, and it's always scriptable.
But the same could be said of CLIs tbh. After a couple dozen options, man pages, or `--help` lists become a dense read, where as a simple form on a webpage might be easier to grok. YMMV
Yes, and so does the --help page or man pages of a CLI program, the more options you add. Soon, --help takes additional arguments for which category of options you want to list. Often, options are listed alphabetically, which means if you don't know exactly what you want to do, you have to read the entire list to find the option you actually wanted.
A GUI can much more easily emphasize the most common options, it can visually group related options, and it can use common GUI widgets to, for example, show that some options are mutually exclusive with each other.
I would argue that's untrue. Limited screen space discourages the proliferation of obscure GUI options and pushes the UX towards simple packaged tasks optimised for graphic representations - or at least representations with a strong graphic organisation, even if there's plenty of text involved.
The trend in the command line is in an orthogonal direction - more and more complex function-like options with minimal mnemonic labels, optimised for text representations and text processing.
Which is a problem of its own, in a way. Reading a man page top to bottom is few minutes. Reading a user guide/Info pages (for the good software that still comes with one) for a more complex tool (e.g. gdb) takes couple of hours. If you're going to use the application more frequently or for more than an hour in total, the time spent reading the manual will pay for itself - in reduced frustration, much quicker discovery, and much less StackOverflow searches (and getting confused by wrong/misguided answers).
Reading manuals top to bottom: an ancient, forgotten superpower.
On balance, I prefer this. Extra command flags neither pick my pocket nor break my leg.
The alternative (presuming the extra functionality is useful to someone, somewhere) would seem to be more standard commands, and that choice would use up more of the precious global namespace for small yet memorable lower-case command names.
I prefer as much of that as possible be reserved for userspace. I'll admit my mind boggles a bit when I ask myself what tar could possibly do with 139 flags, and it makes me a bit nervous about the surface area the program exposes.
As an exercise, try to sort a directory by size or date, and pass the result to xargs, while supporting any valid filename. I eventually just gave up and made my script ignore any filenames containing \n.
Here you go: sort all files in the current directory by modification time, whitespace-in-filenames-safe.
The `printf (od -> sed)' construction converts back out of null-separated characters into newline-separated, though feel free to replace that with anything accepting null-separated input. Granted, `sort --zero-terminated' is a GNU extension and kinda cheating, but it's even available on macOS so it's probably fine.
If you're running this under zsh, you'll need to prefix it with `command' to use the system executable: zsh's builtin printf doesn't support printing octal escape codes for normally printable characters, and you may have to assign the output to a variable and explicitly word-split it.
This is all POSIX as far as I know, except for the sort.
Sorts by size, can just change -k1,1 to -k2,2 to sort by date. Tested on fedora 31 with gnu find, sed, sort and tr.
And really the answer given at the stackexchange link is quite on point:
> However ls is really a tool for direct consumption by a human, and in that case further processing is less useful. For futher processing, find(1) is more suited.
Find is the right solution, find gives you an enormous amount of control, to re-implement find in ls seems counter productive.
(Python3 does try to make this hard on us — Unix path names are just bags of bytes, but Python insists that printing must have some known encoding. I get around that by abusing os.write(1, ...) and by adding the dummy b"." argument to os.listdir(), which forces the results to be in bytes format.)
> I eventually just gave up and made my script ignore any filenames containing \n
I think we can all do that. Did anyone ever came across a
legitimate reason for filenames with newlines? All I can think
of are intentionally bad-named files to exploit bugs. Those
filenames should be avoided in the first place.
I have wondered how it is that my Mac Plus became a machine with almost no noticeable delays when I upgraded to 4mb ram, and now I have 8gb of ram, and a less responsive system. I realize my system is far more dynamic today. I just wonder if one day we'll be using systems with 4tb of ram that would be snappier if upgraded to 8tb or 16tb.
Given the nature of open source, I'd be surprised if there isn't a alternative for less dynamic, less resource req, systems. Is there a system out there that still uses fewer commands?
I don't know how to ask this next question, so please bare with me.. Are there any modern systems that don't require modern hardware, but are just more efficient versions of past systems?
I have been happy with the way I could use my computer for the last 20 years (probably earlier, too).. so are there any efforts to keep the simplicity of older systems that were adequate for most, make them more efficient, and either have "super hardware", or I suppose very cheap hardware?
Thanks for anyone who made it through my ignorant ramblings.
I know that some people are only as efficient as they're forced to be. I hope there are efforts to take what is needed, and make it more efficient. There are efforts like Commander X16 to make a new 8 bit computer, and others where people are rewriting things in Rust..
BTW, I have a first gen iPad sitting around that worked very the first two years or so of its existence, and it barely functions today. I know part of that is Apple "protecting" older batteries by slowing down older stuff so we go out and buy new things, but I would like to believe that we're capable or creating an OS for that hardware today that would run better than it did originally, rather than barely function at all.
"I have wondered how it is that my Mac Plus became a machine with almost no noticeable delays when I upgraded to 4mb ram"
I remember when I ran the original MacPaint (I assume it was in hand optimized assembler?) on a 16 MHz 68030 and it was amazing - everything seemed to simply be instantaneous. It put in perspective how when the 68000 came out, it was a "minicomputer on a chip" and the 68030 was conceived as a "mainframe on a chip".
In the unix-compatible world, you could take a look at openbsd as an example of OS that is not growing exponentially. Yet, the web browsers are the same two beasts that you find elsewhere in linuxes.
For a more radical approach to what you ask, look at plan9. There are some modern forks that run on modern hardware.
Ouch, the /usr/bin/tar team beat me. I only added 63 options to /bin/ps from Slackware 3.1 (1996) to Ubuntu 12 (2015). They added 81 options. If I'd known about the contest, I could have added enough options to win.
The true winner is probably gcc though, which was unfairly excluded from competition. Over 1000 options is excellent work.
Anecdote from yesterday. Me staring at merge-pathnames and wondering why everything was backwards from the physical layout of the paths. Ah, it is because the default is optional because of the direction in which the function is usually used. If I didn't take the 5 minutes to wonder about what I was missing I would have implemented a macro to do it the way I thought was most sensible. In the context of the op, I can only imagine that the impulse to pause and reflect on the question of why, or what motivates the desired for a new option has fallen victim to the fact that there are thousands of people for whom the flag solves a real and tangible problem immediately.
GNU coreutils has been very careful when adding new options. We're also careful to provide constructive feedback on why it might be best _not_ to add an option, and have summarized many of these discussions
177 comments
[ 3.0 ms ] story [ 207 ms ] threadAnother example: the modern Bash shell vs the Plan 9's Rc shell. The difference in documentation is over tenfold, with the latter still being more less laden with gotchas:
Having said that, I still love me some sick diff -burN-style mnemonics.That's an advantage; Plan 9 provides awk for those use cases. There is a clear separation, and a simple yet powerful language for communication (pipes, file descriptors, and files).
>bash is trying to be a full scripting language these days
Arguably Bash over-did somewhere around the time they added support for TCP connections
I won't get to this in Oil (without help) but I still think it's valid. The languages have so much overlap that they should just be combined. Of course that's what Perl was (leaving out Make), but Raku and Python 3 seem to have left a hole in that design space.
Shell, Awk, and Make Should Be Combined http://www.oilshell.org/blog/2016/11/13.html
Example Code in Shell, Awk, and Make http://www.oilshell.org/blog/2016/11/14.html
[1] https://github.com/kanaka/mal/tree/master/impls
perl -lane # process input by line, split into fields, print outputs with new lines, execute code
netstat -planet # show process, extended info, all socket types
ss -pimento # basically every piece of info about live connections
iostat -txyz # extended, omit first report (the since-boot one), omit devices with no activity, log datetime
(The second E does nothing but complete the word)
A general purpose ad-hoc data format with all the fundamental types, that doesn't require endless text parsing but can be converted to a 1:1 text format for human viewing, would be helpful in so many situations, including serialization for piping between unix tools.
https://github.com/kstenerud/concise-encoding#example
I was a bit of a wuss and had to use x.409 dumps, my boss could look at the raw data packets and debug problems.
Although not technically a pro/con list, it highlights the evaluation criteria used to compare.
[1] https://github.com/kstenerud/concise-encoding#comparison-to-...
[2] https://github.com/kstenerud/concise-encoding/blob/master/de...
In your comparison you claim Concise is zero-copy, but it does not appear to be so, at least in the sense that Cap'n Proto and FlatBuffers are. Concise appears to use variable-width integers, tag-value sequences, and 8-byte alignment, all of which make encoded messages unsuitable as in-memory data structures; it looks to me like you have to parse the message into some sort of AST before you could meaningfully operate on the content. "Zero-copy" in the Cap'n Proto / FlatBuffers sense means that there's no need to do any "parsing" because the whole message is efficiently traverseable as-is; e.g. you can mmap() in a very large message and then find and access any one value within it in O(1) time (or perhaps O(log n), if the data structure is nested). This generally requires that primitive values have fixed widths and fields have fixed offsets within their parent object.
Digging into your docs it looks like what you really mean by "zero copy" is that individual strings or byte sequences can be used directly without copying them out of the original message buffer. This is a fairly common property of any binary protocol; e.g. Protocol Buffers can do this (albeit without NUL-termination), but in your table you have indicated that it cannot. I would maybe call this "zero-copy strings" to be clearer.
On another note, in your table you have indicated that Cap'n Proto doesn't have a "String" type, but this is not true -- Cap'n Proto's "Text" type is a UTF-8 string (and is even NUL-terminated on the wire).
(I'm the author of Cap'n Proto.)
XML strength also lies in its extensibility, and its large number of compliant implementations, which goes way way beyond a single uncompleted implementation by a single author.
RON is fine for rust, but not for other languages that might not have its feature set.
>the added options are convenience flags: setting the permissions of the new directory and making parent directories if they don't exist.
Obviously completely unnecessary for normal users, but those have to exist for automated processes. The "problem" is caused in part by the success of Unix, that now computers are using computers.
Sure it’s convenient for writing scripts too but a script could just as well include some boilerplate for the functionality if required and it if not.
This is the argument between CISC and RISC.
> To be clear, I'm not saying that I or anyone else could have done better with the knowledge available in the 70s in terms of making a system that was practically useful at the time that would be elegant today. It's easy to look back and find issues with the benefit of hindsight. What I disagree with are comments from Unix mavens speaking today, with comments like McIlroy's, which imply that we just forgot or don't understand the value of simplicity, or Ken Thompson saying that C is as safe a language as any and if we don't want bugs we should just write bug-free code. These kinds of comments imply that there's not much to learn from hindsight, we were building things as well as anyone could build things in the 70s, all we have to do is go back to building systems like the original Unix mavens and all will be well. I respectfully disagree.
In other words, the real problem isn't that modern software is getting more complex (as there is a good reason for it); the problem is that we're insisting on using an environment that is not up to the task of dealing with modern complexity.
For an example of how insane this world is: consider putting your source trees in a location that contains a space in its name. How many build systems are going to completely and utterly fail to build your code? And... why should we tolerate this situation?
But if you do this:
Then, all the commands in the 'fs' library are invoked like this: [0] https://dev.lotw.xyz/shell.osOf course some principles of speech and syntax should stay around (as they probably exist for a good reason), but it is foolhardy to grip tightly onto any language, human or computer, thinking it will always stay the same, when it plainly won't.
Also please don't compare human languages to programming languages to make a point. It's the programmer equivalent of bad managers comparing Java to JavaScript. Argument by analogy in such situations is a sign of not having a basic understanding of the problem at hand. Yes I am aware of the contradiction.
Partly, but less so.
Human language vocabularies approximate Huffman Coding - the more frequently used words (and phrases) tend to be proportionally shorter. This is a gradual process for spoken languages, and somewhat quantized for written languages.[1] It's possible because with human languages, "close" is "good enough"; one can read english written with ye olde orthography, if with a bit of extra effort.
Less so in the lifespans of particular programming languages. Per Postel's law we tend to be liberal with extending but conservative with breaking backward compatibility. I think the case of command line tool parameters is particularly conservative for various reasons; among them because the 'knowledge' of options is spread among multitude of separate project - many of them obscure and bespoke - and coordination would be nigh impossible.
However there are some signs of similar trends; note how the "function" and "integer" and similar keywords of older languages tend to turn into "fn" and "int" in newer languages.
--
[1] Not sure if there's a comparable process for grammars, but I guess it's a fair bet...?
Pejoration happens too. When is the last time you used a GOTO in code? And yet, 40 years ago, you couldn't write a BASIC program without one. And the revulsion against NULL causing a shift to Maybes and optional types to the point where the concept is reified in many languages now?
All I'm saying is that there are multiple mechanisms that are changing the languages we write code with, and that it is changing far faster than people realize.
So why does `ls` have a recursion flag? Why does `cp` have a recursion flag? Why does `chmod` have a recursion flag? Why does `chown` have a recursion flag? Why isn’t recursion its own command, that takes as argument the command to apply at each level?
Ditto untyped, untagged streams, which mean every tool implements its own custom UI behavior when attached to a terminal, instead of outputting tagged data and letting the consumer run it through system-standard data formatters.
This is not good—or even borderline competent—systems design. This is bullshit. Because the people who slapped it all together are bums, and the people who accepted it didn’t care enough to call them on it.
But noooo, we can’t change anything now; because “backwards compatibility”.
Protip: when you build a system that can’t evolve, that grinds to a halt under its own impossible weight, it’s time to confess “yeah, we totally ballsed that one up” and start over again, lessons learnt. Because maintaining that status quo is a tyranny: dragging everyone else down to the level of its incompetence, and beating them to death with intransigence. No prizes for guessing who that benefits: the bums who built it. And who pays for it: everyone else.
It’s a scam. Perpetrated by and for those who cannot admit they are wrong.
Smash it down. Because whatever comes after it should be so much better, the only question will be “Why the frell did we wait so long?”
--
“Those who make peaceful revolution impossible will make violent revolution inevitable.”—John F Kennedy
Why did ls even need 11 parameters to begin with? Why not exactly 1? Or none? Why did tar need 12? Why did find need 14? The 28 tools in the example (excluding non-existing ones) had an average of ~3 parameters per tool in '79. Should 2017 have brought us 200 extra individual tools to provide the same additional functionality? You want another tool that does recursion for ls to save us from "incompetent bums" and "grinding to a halt"?
Everything around you grew in complexity to adapt to your needs because there's value in consolidation. It's why you are now typing from a device that is orders of magnitude more complex and integrated than the one used to write down the Unix philosophy.
You ask the question "why" but fail to answer the "why not". Why would a tool that still does exactly what it used to do in the past not be able to also do more especially when it's still within its sphere of relevance? And not the "out of principle" answer, not the handwaivy, tin foil hat answer, not the offensive and insulting answer that you provided above (bums? incompetent? bullshit? scam?).
Times change and philosophies adapt to remain relevant. "We must use time as a tool, not as a couch" - JFK.
Yes I absolutely do. Remember UNIX? "Many tools, each small, each doing one thing well". Adding hundreds of flags to older tools does NOT achieve that. Your point here?
> Everything around you grew in complexity to adapt to your needs because there's value in consolidation.
Please don't cross into abstract philosophy. It adds absolutely nothing to a specific technical discussion.
> It's why you are now typing from a device that is orders of magnitude more complex and integrated than the one used to write down the Unix philosophy.
I use my computers as the complex messes and a series of train-wrecks that they are, and now hear me out please, because I literally have no other choice. Be it Win10, Linux, Mac, it's honestly almost irrelevant. One example: to this day nobody can make a reliable system-wide sleep mode that is not messed up by extra hardware (like my external HDDs), on neither of those 3 major OSes. In 2020.
I can't go back to my Apple IIc. I will lose my job if I do.
What's your point here? That such a complexity is inevitable? Is that it? But who argues against that? Of course it's inevitable, we see it in every area of life -- it's one of the aspects of entropy. What's being discussed is: "isn't it time to take a firmer stance and start using newer tools that don't have these problems and protest against the old ones by not using them"?
I personally answered "yes" and I am gradually doing more and more of it -- already replaced grep, ls, find/xargs, about 13 in total IIRC. You know what? My productivity on the terminal skyrocketed. I started ditching languages with too much magic and implied behaviour and too weak/random typing (PHP/JS come to mind) and moved to others which, while not covering everything that I'd like to have (like Elixir), are still letting me sleep that much better knowing that `[] == 0` is actually false and I won't have to curse my way through logs in the next morning.
> You ask the question "why" but fail to answer the "why not".
If you actually frequent HN and aren't just looking for conflict then you'd know that literally every month we have a few rants by people who are now burned out that decry the current state of affairs. But if you really want me to answer then I can likely summarise it to:
"Because our tiny brains don't work well with so much complexity in the long run. We can cope for a few years, some even for 20 and more, but eventually we can't. Because having smaller, more-focused, and more (total count of) tools that each do something really well, has been demonstrated to work better everywhere, not only in IT."
(But I am aware this has been contended endlessly and it devolves into belief and tribalism and science and historical evidence goes out the window in the first three comments tops. Sad. Shouldn't happen to techies. We should be better than that.)
> Why would a tool that still does exactly what it used to do in the past not be able to also do more especially when it's still within its sphere of relevance?
Because it's NOT, and your parent commenter gave you an excellent example which you immediately dismissed and THEN proceeded to ask your question again, pretending he never gave an example. Are you aiming at being disrespectful on purpose? Confusing.
> And not the "out of principle" answer, not the handwaivy, tin foil hat answer
You are aware you are crossing into ad hominem and almost insults, no?
> Times change and philosophies adapt to remain relevant.
Well that's a perfect example how the same sentence is viewed in two completely different ways. This is EXACTLY the problem, dude: philosophies don't adapt at all...
Simplicity means different things to different people and at different times. You are both making the same argument for simplicity for the sake of simplicity with no objective measure for it other than a "guideline" philosophy of "does one thing well", and just dismiss the majority who doesn't fit your "burnout as proof" argument.
If you want to wave the philosophy flag you must also acknowledge that either it never applied to pretty much anything as such, or it implies you should adapt your tools to the times, the facts, the challenges.
> Again, your point?
Take full advantage of the times you live in or be left behind, burnt out, reminiscing the glorious past I guess...
https://news.ycombinator.com/newsguidelines.html
Unix provides rather limited primitives for composition, and a lot of things remain unsafe or very difficult to do right in shell. That's why we keep extending tools and doing things (rather redundantly) in a real (if outdated and limited..) programming language.
A lot of that complexity is due to poor design, not due to inherent complexity in the user's needs.
https://news.ycombinator.com/newsguidelines.html
https://news.ycombinator.com/newsguidelines.html
This culture and profession is far better at making excuses than systems. That should be utterly unacceptable when these systems impact the lives of BILLIONS. Priorities, people.
Just yesterday in fact, while implementing a state machine. Except the developers of that particular language explicitly did away with GOTO, so I ended up approximating it with a SWITCH/CASE. At least the language supports fall-through...
Which is to say, people differ in judgements of various features (footguns vs swiss army knifes). Moreover, the judgements seem to oscillate as fashions, in ~10 year long cycles. Please recall how exceptions and templates used to be a hot thing back in the day. Ditto garbage collectors. Presently we're all in love with Rust, and the "JITs considered harmful" paper will drop any day now.
It's interesting to think about how things can be swapped dynamically in this system. Consider the evolution of 'telephone.' In the 1980's, 'phone' referred to a land-line telephone. In the 2000's, it was ambiguous; for young people, it meant 'cell phone' most likely, or we started calling them 'house phones' for land lines. Now 'phone' is mostly used for a 'smart cellular telephone'.
Another interesting example is "Playstation". That always refers to the current generation, and we retroactively apply "PS1."
It will be an interesting period for history in the future to try and decode our present-day meanings since so many words are period specific.
You could probably trace a similar evolution across programming languages, just by systematizing the phrases/words used in each to accomplish the same thing.
The reason is arguably that tar has too many responsibilities here. Tar should only do two things: create tar streams and extract tar streams into files. You do that with "tar c" and "tar x", and that's everything you should need to remember (ok, fine, "tar t" for listing files is good to know as well).
Commands to create tarballs should look like this:
And untar commands should look like this: or maybe or whatever version you prefer with "tar x" at the end. This is composable, super-easy to remember, and you can VERY easily swap out any parts for any other parts (for instance, changing the source from a file to a url with curl, or the destination for an ssh pipe or something).One of the benefits of the "swapping out" parts are of course that you're not restricted to what's built into tar: you can use ANY compression utility you want. For instance, there's a thing called pigz[1] which parallelizes gzip decompression for increased speed. It's trivial to swap into the pipeline when you use tar that way:
Congratulations, you now have much faster expansion of tarballs, without needing to recompile tar.We've taught a generation of programmers that "tar" is hard to use and you have to look up a magic incantation every time you need it. Ironically, that makes tar less flexible, because now you think of it as a single-purpose utility (creating and unpacking compressed tarballs) instead of a versatile unix tool which can be used in all sorts of different situations.
[0]: https://xkcd.com/1168/
[1]: https://zlib.net/pigz/
Modern tar can detect the format so "tar xfv" is enough. The "v" option only means "verbose" so it's better just "tar xf" and avoid spamming the terminal.
Note: If none of your files were clobbered and there are no spaces in the filenames to worry about you can do:
tar -tf [tarball] | xargs rm -d
Run it repeatedly if you also want to remove the directories.
But what I came here to say is that I find the typical tar flags pretty intuitive. X means extract, z means using zip/unzip, v means verbose, and f for file I/O instead of stdin/stdout. For typical use cases, I don't find its interface magical but logical. I'll freely admit to needing 'man tar' anytime I stray from the typical use, but that's true of a lot of commands.
The bigger offender in my mind is rsync. I remember auvP as my typical flags but don't ask me what they do. And good luck guessing whether my destination or source path need a trailing slash on the first try...
I remember reading that xkcd comic though and I went "yup, that's me!", and I've spent a bunch of time thinking about why that was. You list those flags there and say there intuitive, but they're unique to tar, you have to memorize them, and there's no reason that they need to be in tar when they're available right there in the shell.
Why do I need to memorize that the "f" flag is "file output/input", when I can just write "> dest-file" or "cat src-file |" (or "< src-file") in my shell? That works for every command, not just tar. Why do I need to remember that the z file is for gzip (what's the equivalent flag for bzip2?) when gzip/gunzip exists and works for anything?
So many people simply don't understand what tar does or how to use it without googling the magic formula they need, and it's because of all these damn flags that everyone insists on using instead of doing it the "unix way".
dd is another program where the commandline flags predate modern usage (although it does at least behave nicely with pipes by default) and look totally weird compared to everything else. But again nobody wants to touch them because it would break millions of existing scripts.
> z means using zip/unzip
gzip/gunzip, doesn't look like tar supports zip/unzip, which is a whole different algorithm.
Perhaps you mean that you've looked it up. Others could do so, too; it's not that hard (even if a web comic makes light-hearted fun of the subject)!
Anyway, here's a fun fact: tar is smarter than you give it credit. The commands below do just work:
How about:
I didn't have to look any of that up.https://news.ycombinator.com/item?id=22478692
https://git-man-page-generator.lokaltog.net/
https://ffmpeg.org/ffmpeg.html
anything as powerful as FFMPEG, EXIFTool, ImageMagick, x264/265, etc is going to have switches/options from hell just out of plain necessity. these are not simple little Unix-esque tools. these are full blown applications that just happen to not have a UI.
It's not as much that the arguments allow you to run a program under variant configurations, but rather that you create completely new programs for each run.
See https://ffmpeg.org/ffmpeg-all.html
The alternative is to hide all of the complexity and hope that the defaults are good enough for your users. Which works ok for stuff like Mom encoding her cell phone video but falls down pretty quick when someone needs to do something slightly out of the box, like add in a second audio track.
Also tar's increases might include new compressions methods like bzip2, XZ, etc.
Most of the tape devices themselves had/have compression on them so that tar did not need to do it. However, that compression was pretty weak (2:1), so giving tar the ability to use bzip2 and send the compressed data stream to the tape (bypassing the hardware compression) gave the user much more convenience. Also, older systems probably were bad at doing the compression in real-time, so it probably slowed down the write to tape. I'm just guessing, I have no experience writing to tape from such an old system. I'm not really upset by tar geting some extra features.
To better expand on what fctorial is probably trying to say, argv is a simple list of strings. Plain text is a nice common ground among the myriad of programming languages that executables can be written in. In order replace options with "a structured, typed, reflective API", argv could no longer be a simple list of strings. There would have to be a common typesystem among all programming languages. It wouldn't just be needed for the understanding of argv, but also for the output, since the output of programs is used to feed the arguments of other programs. This puts heavy restrictions on the liberty programming languages can take in defining their own typesystems while retaining compatibility with such an inter-program typesystem.
This isn't to say that it can't be done, but I'm fairly certain it would suck on multiple fronts. You might be able to gain some things, but you'd certainly be giving away others.
EDIT: I don't want to spend too much time expanding what you'd be giving away, but to start
1) Having the common ground be plain text means that you get 1 form of output that can be used both for human and for machines to read. The programmer might not even intend for their output to be used by other programs, but it will always be trivial to do so. You can propose that they could have 2 forms of outputs, one plain text for humans and one typed for machines, but there would be no guarantee that they would have the same information. It's likely that one form will miss information that the other has, and you'd be left with humans reading output meant for machines or machines reading output meant for humans, so you gain nothing.
2) Less brevity. You complained about options being too detailed ("remembering copious minutia"), but that's because they're meant for interactive use. This is not a problem about the interface being untyped. It's about the interface optimizing for being quick to use. This would either also arise with a typed interface (although more difficultly so) or the CLI would simply be more tedious to use. No win there.
3) It would be more inconvenient to use interactively. For example, if you want to differenciate between a number 5 and a string "5", you're going to have to add syntax, like the quotes I used in this sentence. You can see the inconvenience by setting you shell to be something like python, for example. There would be too much syntax for interactive use.
So it does, albeit a better example of the shell doing something nontrivial would be replacing file patterns with filenames, but it doesn't help since my program could be expecting a list of urls instead and has no standardized way to tell the shell this.
>To better expand on what fctorial is probably trying to say, argv is a simple list of strings. Plain text is a nice common ground among the myriad of programming languages that executables can be written in. In order replace options with "a structured, typed, reflective API", argv could no longer be a simple list of strings. There would have to be a common typesystem among all programming languages. It wouldn't just be needed for the understanding of argv, but also for the output, since the output of programs is used to feed the arguments of other programs. This puts heavy restrictions on the liberty programming languages can take in defining their own typesystems while retaining compatibility with such an inter-program typesystem.
Yes, a language-agnostic reflective type system with a shell and supporting language tools, it's a pretty big implementation and coordination effort and an even greater one to retroactively describe the command line semantics of existing commands so not holding my breath here.
>Besides that, it would be more inconvenient to use interactively. For example, if you want to differentiate between a number 5 and a string "5", you're going to have to add syntax, like the quotes I used in this sentence. You can see the inconvenience by setting you shell to be something like python, for example. There would be too much syntax for interactive use.
A matter of preference I suppose as I have been using jupyter/ipython for all "shell" tasks for quite a while.
Well, if I don't use it often, I'm not going to be looking at the documentation often. However, when I do, we're talking about the time it takes to type `man foo<enter>/bar<enter>`, where bar can be `keyword` or `--option\b`. That doesn't even take 30 secs.
Does this better alternative exist?
I think the change was jokingly called “pulling up the ladder behind us.”
Learning some command line commands feels like learning a whole new framework. A man page should rarely need to be a whole damned manual.
If so, then this primarily seems like an issue of how to present the documentation and not the mechanics of the change itself. Instead of autogenerating docs that say concat :: Foldable t => t (t a) -> t a (or whatever) and making people think about Foldable, write docs that say that concat is still [[a]] -> [a], with a note at the bottom saying, by the way, this generalization exists, click here to see the more complicated version if you've decided you care about Foldable. Probably the text of the docs remains the same, in fact ("concatenate a list of lists together into a single list"), you just display the real prototype and explain that "list" could also mean something else. (There's a number of ways to implement this, from writing docs by hand, to extending the doc protocol to support this specific case of lists and Foldable, to extending the doc protocol to support documenting a fake function prototype and then have a link to the real function at the bottom, and using the type system to ensure that the fake prototype is just a special case of the real function.)
A corollary is that documentation presentation is an inseparable part of library / tool design.
The same type of solution that you propose for docs can work here too, though. The compiler can trivially substitute the typeclass for any known instance.
This makes it easier to remember the syntax for create and list too:
That's how I write the install instructions for Oil:https://www.oilshell.org/release/0.8.pre2/doc/INSTALL.html
-----
So basically tar can behave normally (at least GNU tar can) -- it's just that there's an ancient idiom that most people have in their fingers, and that's copied around in various instructions.
The nice thing about the verbose way is that the hyphen prefix enables autocomplete.
GUIs provide discoverability so often times you don't need a manual. If you don't know how to use a command line tool you must open the manual. So in that way I would consider the man pages as part of the UI and those definitely get cluttered.
For a desktop paradigm, there is always a way to do better with GUI as you add more features; but you'll refactor / redesign a lot on the way and that has a cost in familiarity, which directly impacts user productivity. It's a trade off.
The dirty secret of discoverability is that only a tiny subset of features is relevant at any time in the UX, and identifying that subset + triggering user awareness about it is the "art" part. We learned a lot with mobile in that regard because the harsh limit on real estate becomes incentive to identify the essentials.
Discoverability doesn't have to suffer from complexity of the software, but it's not just UI design, it's UX.
For some reason, designers and some users don't like it, but a hierarchy (folders, menus, etc) is something many people find useful and effective.
But it seems pretty common to declare people can only handle a few options plus search.
There are great domain apps (e.g. medical, games, professional audio, amateur astronomy to name a few I've seen) that really break open the format:
- feels like a tricorder from Star Trek, where small is good because it doesn't try to be bigger than it is, rather embraces its limitations are features not bugs (and to his credit, Jobs + Ive successfully translated these ideas into the iPod and later on iPhone early iterations).
- feels like a great videogame UI where despite the complexity, once internalized it's a breeze (even a joy, physically) to manipulate. It takes a lot of testing and feedback and is admittedly one huge QA cost in video game making, so there's why most companies don't do it for 'normal' products whose UI/UX is 'good enough' (or so they think, for now).
- clever integration with features, e.g.: slider on a phone with a tiny motor feedback when you cross a unit bar or even for magnitude; a bigger-deeper 'womp' around the center coupled with a UI magnet that makes you 'feel' and 'see' the slider fix itself on the 0 position. This all designed so you interact while looking at something else, just like a physical button.
I don't know. I feel like we've made steps of giants in the 2000s and then progressively dropped the ball when we realized the simplest UX on mobile yielded the highest engagement (think twitter, insta, how "simple" they are, how limitations actually enhance user activity; in a very questionable way but that's besides the point in terms of profit thus research).
I sincerely hope someday a disruptor kills everyone else with awesome UX (just like Apple did in the 2000s) and forces the market to re-think its abysmal UX standards, and let's not talk about ads because that would be extra-snarky.
If I were snarky I'd tell you that. But I'm not, so I'll leave these remarks for someone else to make.
The market is allowed to exist because profit is a quite decent proxy for utility. But whenever you are optimizing using an imperfect proxy, it's possible to overoptimize - make the proxy metric look better, at the expense of the actual goal it was supposed to stand for. I believe software (along with much of other market sectors) have crossed that point a while ago.
The economy to me is a vast "optimization space", so I can only agree. It's the 'trap' of a local minima I suppose, wherein the friction of moving away to find another possible (but uncertain) minima (hopefully overall lower or more acceptable ethically) is too high. So you've indeed overoptimized your way into a local pit (e.g. the ad model now taking over the news like a virus, resulting in this new "hybrid" object sometimes called "informercial" or "infotainment").
It seems that only "disruption" (of a magnitude worthy of the name) gives enough momentum to escape the steepness of a local historical/economical minima. Unless you've got some new S-curve (which quite literally may represent the math surface of this "escape path"), history tells us we remain stuck and move in circles about the center of 'gravity' of this local minima.
Dunno if that makes sense to you, pardon the loose physical analogies.
ZBrush feels like that to me. Most people think that's it looks like a dogs breakfast at first view, and that it's desperately in need of an overhaul.
Yet when you just want to sculpt 3D clay, it feels remarkably intuitive once you know where everything is.
You mean like Apple did in the 1980s? They regressed in the 2000s.
The UI is one input and the "computer" figures out the rest.
Regular users don't want/understand hierarchies, they get easily confused. They also don't want to organize things themselves, see the many failed attempts at tagging files, websites, images, etc. manually.
I am probably not a "regular user", but I got a job about a year ago defined by and intended for regular users, and one of the major parts of it was managing emails in a departmental account, using (you guessed it) hierarchical folders, and another was managing project documents in SharePoint, using (what appear to be) hierarchical folders.
I'm aware that people will tell you not to do the latter in SharePoint, that it's not really designed to be used like a filesystem, just use lists, etc. But purely from an anthropological viewpoint, I see how people shove everything on a shared drive, and then they try to move it to something like SharePoint and recreate a folder structure.
So, you know, it's not about what I want, but it's an objective fact people do want hierarchies, even when other people don't want them to. Google is pervasive, but it's not all of computing.
But the same could be said of CLIs tbh. After a couple dozen options, man pages, or `--help` lists become a dense read, where as a simple form on a webpage might be easier to grok. YMMV
A GUI can much more easily emphasize the most common options, it can visually group related options, and it can use common GUI widgets to, for example, show that some options are mutually exclusive with each other.
You don't read a man page top to bottom, you skim and skip around, discovering the information you need.
The trend in the command line is in an orthogonal direction - more and more complex function-like options with minimal mnemonic labels, optimised for text representations and text processing.
Which is a problem of its own, in a way. Reading a man page top to bottom is few minutes. Reading a user guide/Info pages (for the good software that still comes with one) for a more complex tool (e.g. gdb) takes couple of hours. If you're going to use the application more frequently or for more than an hour in total, the time spent reading the manual will pay for itself - in reduced frustration, much quicker discovery, and much less StackOverflow searches (and getting confused by wrong/misguided answers).
Reading manuals top to bottom: an ancient, forgotten superpower.
I personally don’t use most of these options and their existence just clutter up the manual.
The alternative (presuming the extra functionality is useful to someone, somewhere) would seem to be more standard commands, and that choice would use up more of the precious global namespace for small yet memorable lower-case command names.
I prefer as much of that as possible be reserved for userspace. I'll admit my mind boggles a bit when I ask myself what tar could possibly do with 139 flags, and it makes me a bit nervous about the surface area the program exposes.
https://unix.stackexchange.com/questions/112125/is-there-a-r...
As an exercise, try to sort a directory by size or date, and pass the result to xargs, while supporting any valid filename. I eventually just gave up and made my script ignore any filenames containing \n.
The `printf (od -> sed)' construction converts back out of null-separated characters into newline-separated, though feel free to replace that with anything accepting null-separated input. Granted, `sort --zero-terminated' is a GNU extension and kinda cheating, but it's even available on macOS so it's probably fine.
If you're running this under zsh, you'll need to prefix it with `command' to use the system executable: zsh's builtin printf doesn't support printing octal escape codes for normally printable characters, and you may have to assign the output to a variable and explicitly word-split it.This is all POSIX as far as I know, except for the sort.
And really the answer given at the stackexchange link is quite on point:
> However ls is really a tool for direct consumption by a human, and in that case further processing is less useful. For futher processing, find(1) is more suited.
Find is the right solution, find gives you an enormous amount of control, to re-implement find in ls seems counter productive.
(Python3 does try to make this hard on us — Unix path names are just bags of bytes, but Python insists that printing must have some known encoding. I get around that by abusing os.write(1, ...) and by adding the dummy b"." argument to os.listdir(), which forces the results to be in bytes format.)
I think we can all do that. Did anyone ever came across a legitimate reason for filenames with newlines? All I can think of are intentionally bad-named files to exploit bugs. Those filenames should be avoided in the first place.
What if the largest file on your drive has that filename and no tool can show it?
Given the nature of open source, I'd be surprised if there isn't a alternative for less dynamic, less resource req, systems. Is there a system out there that still uses fewer commands?
I don't know how to ask this next question, so please bare with me.. Are there any modern systems that don't require modern hardware, but are just more efficient versions of past systems?
I have been happy with the way I could use my computer for the last 20 years (probably earlier, too).. so are there any efforts to keep the simplicity of older systems that were adequate for most, make them more efficient, and either have "super hardware", or I suppose very cheap hardware?
Thanks for anyone who made it through my ignorant ramblings.
Can't find but feel there's a quote like "Programs will grow to fill memory"
BTW, I have a first gen iPad sitting around that worked very the first two years or so of its existence, and it barely functions today. I know part of that is Apple "protecting" older batteries by slowing down older stuff so we go out and buy new things, but I would like to believe that we're capable or creating an OS for that hardware today that would run better than it did originally, rather than barely function at all.
That is a ball we set in motion when we introduced virtual memory, particularly with overcommit.
Give a developer the option to write calls to malloc that never fail, and boy will they make use of that! Performance be damned.
I remember when I ran the original MacPaint (I assume it was in hand optimized assembler?) on a 16 MHz 68030 and it was amazing - everything seemed to simply be instantaneous. It put in perspective how when the 68000 came out, it was a "minicomputer on a chip" and the 68030 was conceived as a "mainframe on a chip".
Back before clear, I used 'echo ^V<esc>c', but that's not portable to different termcap/curses/ncurses display devices.
The true winner is probably gcc though, which was unfairly excluded from competition. Over 1000 options is excellent work.
that said, I live in it.
https://www.gnu.org/software/coreutils/rejected_requests.htm...