If students can't deal with the bullshittery on the command line, how do they hope to deal with the even deeper bullshittery found in the libraries of most programming frameworks?
He's not suggesting that they can't. He's suggesting that doing so is an obstacle to doing research (it clearly is) and that the obstacle isn't intrinsic to the research process itself.
They should add a class to take before his class called Intro to POSIX 101 on how to use a Unix-type system and the command line options.
My 16 year old son taught himself how to program in order to mod his own video games. I wanted to teach him, but he figured it out from Youtube Videos and Google searches. He had to learn how command line switches worked, etc in order to use free and open source programming tools.
I learned Unix at a university in 1986 as part of a computer science class. I got into Linux in 1995 using Slackware on a 486DX clone. I remember a time before the GUI was used where everything was command line based in DOS and writing batch files to do things. Nobody taught me how to do that, I sort of learned it from trial and error. A lot of my DOS programming classes I had to use command line switches to compile my programs and edit text files with a Word Processor or the edit.com in DOS 5.0 or something. Things sure have changed now with IDEs and source code editors that highlight syntax in color. I learned HTML using Windows Notepad to make HTML text files. Things would have been easier if there was a class on this stuff that was required before taking the classes that needed it.
I worked in a computer lab in 1990-1994 at a college and taught the command line options to students just learning how to program.
When I learned Visual BASIC, everything was GUI based and it even generated forms I could paint controls on. It changed the way programming worked for me.
But free and open source tools are still command line based, and someone has to train the students how to use them. I sort of wish I still worked in a computer lab and training students, but those days are gone.
You can call command line bullshit, but at the end of the day it has the least amount of abstraction meaning more visibility in what's actually happening. Visibility is always nice, especially when something breaks.
Say there was this nice GUI that built and setup lot's of projects for you. Who's responsibility is it to make sure that everything it supports stays supported?
If you argue that supporting the GUI is an undue burden, you're merely shifting that burden onto everyone who has to perform the same steps manually every single time. Automating or away may be more up front work, but it only needs to be done once.
right, but it needs to be done again by the single person maintaining the automation tools, who presumably is more of the "programmer" than "user" inclination. everyone else in the lab benefits.
For long established, regularly used tools, there is a case for funding this activity somehow.
For the many specialized tools? I'm not sure how this would actually happen, unless the person who creates it decides to maintain it. When their career focus doesn't value that tool, we can't really expect anything.
Worse, let's say some established systems exist. They have a barrier to entry in that new software needs to be compliant and potentially integrated on some level. People already are focused away from this stuff. Raising that bar means fewer overall contributions.
Now, it's often: I wrote this. It's open. You can use it.
And they move on.
If, suddenly, I wrote this, and it's open gets followed up by, "you need to make sure this works with all this other stuff..." what happens?
i meant, if there is an organisation (the OP's example involves a university department, but this holds for other places too) where the majority of people simply want to use computers to get their work done, it is well worth having one person (or a team, as things scale up) whose job is to automate all the incidental details of working with the operating system and environment, so that people don't have to context switch from a "user" to a "programmer" perspective all the time.
Seems to me those people, or person would also have to be pretty great generalists.
A lot of research type tools get written and then left laying a out. Just testing and packaging, integrating, etc... would be a considerable project.
Could be the basis of some sort of course or career though. I'm thinking of the sysadmin types who manage software and research teams outside academia. The really good ones I know used to hold a position, or have some personal interest sufficient to make these sorts of activities make sense.
in general, it's a great productivity booster across the board to have a team doing this; the main problem is getting buy-in from orgs, because the time and effort wasted when you don't have one are so pervasive as to be largely invisible.
Invariably, unless it's a major software product with a lavish budget, the GUI based setup program is less intelligible than the command line, and is accompanied by instructions consisting of page after page of screen shots, "with circles and arrows, and a paragraph on the back of each one," to quote the balladeer. And when the next version of the OS comes out, or a new version changed the installation slightly, the instructions are invalid, but were too costly to update.
After years of that, "sudo apt-get install" was like a breath of fresh air.
To help those who are reflexively clicking "add comment" in anger, you really should read the article first. You're near certain to argue against a strawman otherwise.
I've always had people who wanted to learn programming become so bogged down in the tools around programming. It led me to thinking that it might be better to teach the fundamentals with pen/paper/whiteboard. To get the "this is why programming is important/powerful" message across. Anyone tried this/had any success with it?
I haven't pushed it on any family-members yet, but "Code: The Hidden Language of Computer Hardware and Software" does a pretty good job building up from the basics, although my eyes started to glaze over a bit when it came to some of the circuit-diagrams.
In the last couple of years I've had few people ask me 'how can I write a program with a GUI.' And I give the advice, 'Download the free version of Microsoft Visual Studio for C# and select new project, windows forms' and then start mucking around.
All of them were convinced by others to use qt and C++
All of them gave up after a week or two of no progress.
I start with FoxPro DOS (2.6). After the end of the 3 months course, a full app with menus, forms, reports, etc. The same with Acces. Later Delphi (the best RAD ever!). In contrast, Visual Studio is clumsy.. but all others tools fall so far below than people truly imagine html+js is easy!
Sort of. I taught middle-school students programming using Scratch (and a whiteboard) for three summers, and found that its interface of dragging and dropping blocks of code worked well for teaching programming fundamentals (conditionals, loops, functions, events, etc) without the hassle of a command-line.
This follow up article is definitely worth reading. But just to put things in perspective, I got my degree in physics, and I overcame bullshitteries in programming, plumbing, machining, electronics, mechanics, bike repair, and so forth.
I think that part of the value, if not the major part, is that at the PhD level, you're given a problem that is ill defined, that your advisor doesn't know how to solve, and whose solution doesn't necessarily come entirely from within your field. Though I was a physics student, solving my thesis problem required developing skills beyond my advisor's own expertise, in areas such as electronics and programming. I had to invent a solution that he had not imagined. It helped a lot that those things were also my hobbies.
Which brings us to this reality. One day, and I pretty much guarantee it will happen, your student will download something from the Web and they will get stuck.
You don't know stuck until it's actually on fire. ;-)
I actually love the term he used. I started my programming in the Windows world and wasn't exposed to POSIX command line stuff until later. I'm comfortable with it nowadays, yet I know exactly what he's talking about with the 'wall'. To me, the 'bullshittery' would mean the inconsistency of it all. There's no standard for command-line tools, so they all tend to be slightly different. The end result is having to learn and memorize a ton of (what are in the end useless) variations of syntax.
Please note I'm not saying anything is better in the Windows world; far from it. I'm more talking about the transition from GUI tools, where feeling your way around in the dark is much less daunting to the uninitiated, to the command line.
Yeah but the shell actually provides a mechanism that allows you to FORGET all the bullshit -- functions.
I look up commands on the Internet, and then put them into single line shell script functions with a name. And then I write a comment about how it works, and check it into a git repo.
Then I never have to remember the exact invocation. I just type my nice function name, which is consistently named, and describes in my own words what the command does.
It's odd to me to see my coworkers and others very slowly coming up with a long command line, getting it wrong, very slowly correcting it, etc. I don't have time for that. I just grep my shell scripts directory for something close, copy and paste, edit it in my editor, and then run it.
It's very quick and puts no strain on the mind. Actually I find enjoyable to put my own names on arcane command line syntax.
Having read the article I half-agree with the author. Yes, learning arcane and inconsistent APIs is not the work the students signed up for and studying command-line tools doesn't materially help them in the near-term.
However, the things that make the command-line so difficult also make it easy to package tools together underneath simpler apis. I suspect the students are struggling with having to use tools that aren't supported by their department or industry community. Otherwise they'd have a shared script that they all maintain that turns a bare PC/Mac/*nix box into one that's capable of doing great work.
This happens with every single engineering team. Those who understand the tools wrap them up into something solid so new folks have a great experience on their first day.
If you want a mostly nice, mostly well-documented development toolchain, you can either:
1. Install some proprietary, GUI-based development tools and color inside the lines,
2. Figure out what toolchain you want your students to use, and package it up using apt or docker or whatever (and tell your students to get a VM if they're on a different OS), or
3. Give your students remote logins on shared computers.
Any of these will allow you to keep the "bullshittery" to a tolerable minimum, and get on with the subject of the course.
But in the long run, unfortunately, if you want to be a programmer, you've got to learn to manage your own tools. This might mean being able to figure out command-line "bullshittery", or it may mean knowing how to unit test ASP.NET REST controllers. Even in the days of MacOS 8, back when there was no command line, there were still plenty of unholy, mind-eating horrors hiding in the dev stack. (The Component Manager, for instance.) And let's not even talk about the complexities of the web stack.
You can choose your flavor of "bullshittery", but if you want to be halfway good at this job, you've eventually got learn how to deal with whatever tools you choose.
The Python crowd has worked out what I think is a decent compromise, which is to create some packaged installers that are good enough for most of us. I use WinPython. I don't know what it would take to do something like that for other languages.
I consider "programming" to be a religious journey:
* "You're on your own path" you get errors that the person next to you doesn't.
* "To learn is to invent; you must come to realizations by yourself" different solutions in stackexchange in 2012, '13, and '14 don't work for you.
* I don't have a cliche-quip for this, but if religion is how you find your complex relationship with the world, programming is how you find your complex relationship with computation. I emphasize "how" because its a process and a practice.
* It's mostly bullshit. Because we don't have physics-for-everything, we have religion. Because we don't have math-for-everything[0], we have programming. I'm speaking loosely and rhetorically.
* Related to the above, religion is a bunch of workarounds, like programming. This ecumenical counsel said such, but that ecumenical counsel revised it.
[0] In this instance, I'll define "programming" to be "mostly failed attempts at precise description." Math-for-everything would be "very precise descriptions."
I'm a researcher in the same basic field, and I too am pretty good at this "bullshittery."
However, Philip trivializes the "10,000 hours" spent learning this stuff as merely learning how to cope with this interface. This is so far off it hurts. Those countless hours spent messing with free software is how I learned to use other people's work, compile it, read it, fix bugs in it, and learn how other people think and write software. This is an incredibly important educational experience and there's sometimes no substitute for time and patience.
He's not trivializing anything, he's merely expressing a rather obvious (and frankly sophomoric) statement - software can be made more user friendly. And making software more user friendly actually isn't that hard. What is hard is to figure out which software to make more user friendly (given finite resources). That is a very very challenging problem.. and what's more is a sign of your intellectual worth in solving it.
I took something slightly different. My reading is that many people refuse to accept that there are obvious deficiencies in the dominant paradigm, and instead they cling to it religiously as if it were a be-all-end-all rather than a local maximum embedded with strong network effects.
Think about it this way: in 1995, you'd have spent those 10,000 hours reading other people's C code, reading the same dumb hash table or linked list implemented 400 different ways, the same goofy 4-5 line loops implementing "split" with strtok() over and over. But you don't anymore, because most research isn't done using C.
In 1995 you'd have said "learning to understand what a strtok string split routine looks like is learning how other people think and write software". And you'd have been right. But you would also have been describing bullshittery that we are all glad not to have to deal with anymore.
That's the overall feeling I get from this piece: there is more bullshittery to eliminate than the 400 different versions of "xmalloc" that we had to deal with in 1995.
Also: that because understanding how to iterate over the tokens of a colon-delimited string with strtok() isn't actually intrinsic to much computer science research, it's very important that we don't judge aptitude or capability by how conversant people are with strtok, or whatever it is strcspn does.
I actually disagree with this. I came to computing from the reverse direction (I did theoretical math in grad school and then steadily needed to become better at programming as I needed it in industrial jobs). At first I thought a lot of stuff classified as 'bullshittery' and that with convenience tools like MATLAB, most of it was obsolete and taking time away from focusing on the supposedly more fundamental "real work" of some domain science.
But as I have gone down rabbit holes with Haskell, C, LLVM, and some other tools, I have discovered that seeing those 400 different implementations of basic data structures has been extremely helpful. Reading and re-reading decades-old tutorials on finer points of gdb and memory layout and books on what NUMA-aware memory architectures are like has been extremely educational and important. I would now regard MATLAB, and even the idea of wanting a prototyping platform like MATLAB, as bullshittery.
I used to think that progress, so to speak, in computing was supposed to mean less and less need to be proficient with low-level tools, and a constant push to abstract away the low level and use languages or tools that offer the same things in a high level, black box manner.
But I totally don't think this any more. Now I want to be a power user of all the low level things and I wish that the surface area of tech and science wasn't growing so fast that it forces me to be a wimpy abstraction user in a lot of domains that I don't have time for.
Anyway, I'm just adding my perspective that the attitude that seemingly cumbersome low-level stuff that was formerly instructional but now abstracted is a nuisance is wrong. The world overall doesn't have nearly enough competent technicians and engineers who really know (intuition in the fingertips) that low level stuff, and we're allowing the superficial pleasantries of limiting abstractions to raise up generations of programmers who not only don't know this stuff at all, but also aren't even curious about how it works. They can do jobs when the black box abstractions are working, but as soon as something breaks, they don't know how to fix it.
Maybe one way to say it is that we're breeding programmers who have shifted their curiosity away from how fundamental atomic parts aggregate to make functioning tools and towards how to permute a series of ready-made black box abstractions for some social-level end goal (e.g. how do I permute Heroku and Postgres and Spark ML to make a recommender system) -- but it's not actually clear to me that it's better in a utilitarian sense that we're supposedly "freed" from our low level chains to think about these one-level-up problems.
I'm curious - what's your motivation for being interested in tech? Have you given some thought about what drives you?
I've found, personally, that my motivation has changed fairly dramatically over the years, and with it, my perspective on high-level vs. low-level programming. When I was in high school and college, my motivation was that I wanted to feel competent - I wanted a skill that was all my own, that I could master without anyone telling me I had to. Immediately after college, through my first job & startup, my motivation was to get rich, or at least develop a financial position better than my peers. While I was at Google, my motivation was to earn the respect of my peers, and now that I'm out on my own trying out startup ideas, my motivation is to test my intuitions about the software industry.
When I wanted to feel competent or earn the respect of my peers, I've usually gravitated toward low-level technologies and concepts, things that were hard conceptually or close to the machine but didn't have much practical application. Haskell, lambda calculus, compiler design, blazingly fast code, the bits & bytes of network protocols. When I wanted to get rich or test out my ideas, I've gravitated toward very high-abstraction, productivity-enhancing, flavor-of-the-month technologies. Javascript (in 2007, before it became hot), frontend dev, Polymer, React, native mobile development, Parse & Firebase, etc. I currently care very little about tech, because I've learned that in the grand scheme of things it will usually not make-or-break a business idea.
When I read your post, I'm curious why you use terms like "wimpy abstraction user" or "superficial pleasantries" or "generations of programmers who not only don't know this stuff". That seems to imply that you don't respect these programmers. Why? Frontend devs frequently make more than backend devs, and have many more job opportunities to choose from. The people who make the most (founding CEOs) often have the wimpiest coding skills of all. What makes for a status hierarchy where the closer to the machine you are, the higher the status? Not saying it's wrong - I've certainly felt like this at times too. But given that I'm now in my "study reality and poke it in certain ways to see why it behaves like it does" phase, I'm curious.
FWIW, paradoxically I've found that each time I've moved from a core motivation, it's because I've found that I can never achieve what was motivating me. So my "feel competent" phase ended when I started digging deeper into all the subfields of computer science and realized I could never master it all. My "get rich" phase ended when I realized that I already had enough and no matter how much money I earned, there would always be something else I might want, if I let myself. My "respect of my peers" phase ended when I realized that there will always be some people who liked you and some people who hated you and that which side they fell on depended more on how you treat them than on anything you accomplish yourself, so really all I needed was the respect of myself. Maybe the "test my intuitions" phase will end when I decide that no matter how much information I have or what I experience, there will always be an alternate explanation, but I suspect that will trigger an existential crisis that I'm not quite ready for. There's probably some brilliant Nietzschean revelation in there somewhere, but I'll settle for some observations on status & respect in programming communities. ;-)
I don't see it as an issue of respecting other developers based upon their preferences. I don't care that much about mere preferences. But I do think there are consequentialist downsides to promoting a situation where people are incentivized to stop feeling curious about reductionist understanding of many things. I also think it's fairly obvious at this point in history that endeavors which are correlated with good financial outcomes are often not correlated to outcomes that are widely agreed to be "normatively good." You can make even more than your example front-end devs if you're willing to sling 25-year-old C++ to incrementally maintain some monstrous business reporting and compliance system in an overtly corrupt bank. It doesn't mean we should (or shouldn't) see such a career choice as a generally "good" thing, or that the market forces of the world superimpose to produce that as a lucrative employment option at all should be seen as a good thing about the world's market forces.
I think your question is a bit too vague for me to be very interested in it. "What motivates me" probably changes based on how hungry I am, and there are super basic answers like "I happened to be good at math" that can better explain most of my motivation than some wistfully self-reflective assessment.
But, if I someone pulled a gun on me in a darkened alley and made me answer, I would say that my motivation to be interested in tech is that you can create so much with it, and I like working in a creative/design-space capacity, and I don't have any natural talent for other forms of compositional creativity like writing or music.
There are certain other people whose respect and thoughtful opinions I value (I doubt any of them would choose to work for Google, at least not modern Google), but I want to retain independence of thought too and I don't mind if pushes me slightly along the Martin Gardner "hermits and cranks" spectrum.
I do care about financial success, but only because I've been dealt a harsh set of external responsibilities in life and I don't feel I have much choice but to use whatever talents I have to claw out of that. That being said, though, in the two most lucrative and status-gaining positions I've had, I couldn't stomach the office politics and the way that arbitrary political and status signaling consistently overruled evidence-based pragmatism, and I quit those jobs despite putting myself in extremely risky financial situations both times. I can't say for sure, but I think if I had to face those situations again, knowing what I know now, I still would quit, and possibly would have quit even sooner.
But if I had to say one specific thing that is the motivation, I would say it is that I want to personally feel a sense of pride and craftsmanship in my work. When I solve a business problem by writing software, I want to feel like the way I solved it was full of craftsmanship, in addition to feeling proud of the final product. The manner in which I work is very important to me, perhaps much more important than the actual tasks that comprise the work itself.
Most tech jobs are horrible from this point of view because the only aspect of the job that can be consistently micromanaged from an outside point of view is the manner in which the work is completed (use this tech stack, log progress in this Agile/Scrum software, etc). You can't yet jack into a programmer's brain and micromanage their approach, from a creative point of view, so middle managers settle for micromanaging the outward appearances of how work gets done.
I could speculate too much, and too incoherently, but I think my comments about how we're teaching people to fundamentally value thinking about component abstractions more than reductionist understanding of atomic units relate to this, and go along with efforts to commoditize software labor. My preference is that this should be stopp...
>The people who make the most (founding CEOs) often have the wimpiest coding skills of all. What makes for a status hierarchy where the closer to the machine you are, the higher the status?
Continuing your analogy a bit further, we can say that many politicians (dictators) make much much more than most founding CEOs and they have the wimpiest of wimpiest coding skills of all! I mean, we must not confuse between the ability to make more with ability to tackle technical problems.
Regarding the status hierarchy, we can see that Nicola Tesla was certainly very closer to machine as compared many people, surely he has a very high position in the status hierarchy than most of the run of the mill founding CEOs with wimpiest coding skills.
> Frontend devs frequently make more than backend devs, and have many more job opportunities to choose from.
Out of curiosity, how true is this statement? I hear from my friends that most frontend devs start with higher salary, but have a hard time climbing the corporate ladder, especially if the frontend is not the core competency of the company. I wonder which one is a more accurate depiction of reality.
Both statements are true. The market for frontend devs is both competitive and very liquid: there are many job opportunities and many frontend developers. As a result, starting salaries tend to be high, and you can very quickly "rise to what you're worth": if you don't like your boss or your company is underpaying you, jump ship.
However, frontend tech tends to expire every 5 years. This is part of the reason why starting salaries are high: junior developers can compete effectively with senior ones. If you go backend, you tend to amass a steady stream of domain & infrastructural knowledge about the company which makes you significantly more valuable to them over time. Raises and promotions follow, even if that knowledge is of limited usefulness on the open market.
Whether you'd prefer one or the other partially depends on how you would like to manage your career. The OP in this subthread indicates that he's drawn to the craftsmanship, the idea of steadily making something better over time, and this style tends to suit backend dev better. I tend to be drawn toward discovery, the process of finding loopholes & gaps in the market and filling them with new solutions, so for me frontend & entrepreneurship is more useful. (Frontend dev tends to be a more useful skill for founders, because almost every startup needs an appealing frontend but most can get by with a half-assed backend until they hire more engineers. Witness DropBox vs. Tarsnap - they came out at the same time and both use AWS for storage, but DropBox built a beautiful frontend and a minimally-functional backend, while Tarsnap barely built a frontend at all but has the most sophisticated secure backend I've seen in a product.) For me, the idea of climbing the corporate ladder sounds like hell, while to other engineers on this site, the idea of retraining in a different technology every 5 years probably sounds like hell.
Honestly it comes down to whether or not you want to understand how something really works. If you are 99% of users then someone has pre-anticipated everything you will ever do with a computer and so there are nice pretty GUI tools to do whatever you want to do that someone has worked very hard and possibly been paid a great deal of money to create to make your life easier.
If you are going to be building things or programming or creating new things you have to understand how things actually work. I'm sorry if that means you need to know that python on OSX is different from python on Windows but you do.
If you are doing research by definition your needs can never be anticipated so you will never have pretty GUI tools to satisfy your needs. So I know the command line sucks for most people but if you want to do real work you're going to have to learn how to use it since it drops all of the leaky abstractions that prevent you from doing what you need to do when you're doing something novel.
The command-line is also an abstraction. I'm not anti-command line (for non-user stuff), but it irks me that we see the command line as some kind of god-given fundamental bloc, rather than just another abstraction that just happen to be at a level most programmers are happy with. I think the command line will persist as long as keyboards are the primary data entry tool, but nothing makes the command line inherently less leaky than other interfaces.
The command line isn't an abstraction: a particular shell is, sure. Directly inputting machine instructions is a form of a command line. At its most basic the computer is executing an ordered sequence of commands and you need a way to tell it to execute the commands you wish. Machines run on a sequence of function-argument instructions -- this is fundamentally what the command line is. I suppose you could invent a GUI that lets you click on the opcodes and memory addresses you want to execute but that's bordering on silly. It's always going to be very awkward to capture fundamental computation with a GUI unless the way the machines work changes in a big way.
Every machine in existence has a command line interface. Some of them have GUIs built on top of that. Sounds pretty fundamental to me.
>Every machine in existence has a command line interface.
No they don't. Embedded computers do not have command lines, for example. Embedded computers don't give a shit about text. Most computers do not accept commands at all. As for those who do, some don't accept it in text form, for example gaming consoles or punch-card antiquities.
Command lines are the consequence of keyboards, and whenever you see a computer on which a keyboard cannot be attached or emulated (phones) then you're looking at a computer without a command line.
Okay, a trivial and meaningless nitpick but a technically accurate one. Every general purpose computing machine that you can interact with and reprogram can be done by command line. Sometimes you have to plug another machine and use ITS command line to send communications to the original machine. I'll agree with you that microwaves do not have command line interfaces but calling them "computers" is quite a stretch.
No, it's not a meaningless nitpick! Yes, general purpose computers need to be programmed, but the reason we do it with a command line is because we prefer keyboards for data entry. When we stop using keyboards we will stop using command lines, just as we've stopped using punch cards. Everything that's not manually and physically setting voltages on memory is an abstraction. It's important to note that the command line is more an UI than an abstraction, though. Where the command line is on the abstraction level depends on implementation. Which one UI we use depends on convenience.
We prefer keyboards for data entry because keyboards are what enter the actual data we need to enter: we prefer typing text over the 'point and grunt' method of GUIs, where some interface takes our clicks and tries to intelligently convert them to command line instructions using context and guesswork.
Once we've deleted the written word from human existence and no longer have need of text, have transcended proper known computational structure and can express computations soley by thought and avoid programming altogether certainly we will move beyond the command line.
The command line remains the simplest, cleanest, most leak-free abstraction we have to interact with a computer effectively. I'll agree with you that it's possible there's a better one, but we've been trying for 40 years and haven't even come close -- all we've done is written some translations which let people enter command line instructions by pointing and clicking and having the machine guess how to interpret it instead of typing what they actually mean. I think it's the best we're ever going to get though I'm happy to leave room for optimism if you have any. In the meantime, those of us who have to get work done (I assume you are one of our cohort) will live in the real world and use it without complaint.
The only reason the command line is complicated is because computers are complicated. If you want to use computers in a deep way you can't hide behind the GUI and run away from actually dealing with the machine forever.
I am honestly almost speechless at that. The command line is complicated, because people are terrible at designing console apps and parsers. They end up with two or three ways of expressing anything, and give little or no clue (usage? really?) as to what they mean or how they differ.
Clean? I could agree with 'concise' but not clean. Some apps take multiple switches behind a single cue character (is it '/'? is it '-'? '--'? something else?) and some can only parse one at a time. Some take switches before text and filenames, some after, some have a mix.
Command line apps are the sort that only a mother could love. They are ugly, opaque, have an enormous learning curve, and even experts tend to type them into files because they would hate to have to craft them all over again each time they need them. Macros; cheat sheets; scripts are all crutches to escape the tyranny of command lines.
Shrug I'd agree you can definitely find fault with many of the programs that currently exist on the command line though every single one is still miles and miles better than a GUI -- if you're going to do something one time, it's great to click a checkbox like 'show hidden files'. If you're going to do it regularly, it's worth looking up '-a'. If you want to do something unusual it takes a minute or so looking up the command's help. Whereas on the GUI if you want to do something unusual, the answer is generally "whelp, sorry, learn CLI peasant you just can't do it since I didn't anticipate your weird need and spend time rendering a menu choice and 4 interface screens with click boxes so you wouldn't have to type anything".
If GUIs had as many options as command line programs did they would be completely unusable trash, with an enormous unintelligible control panel, and you'd be looking iconography up in a reference manual instead of switches (have fun designing all those icons and having different ones express different subtle alterations in how a command will execute)
There are a lot of options because the tools are flexible and capable of expressing MUCH more than a GUI. I'll give you there are some consistency issues, but they are historical not fundamental to the CLI. I'm honestly surprised someone hasn't written a wrapper to standardize every app in a shell so they all use the same format (ie have a 'one true format' for argument passing and just convert as necessary for every CLI app, parsing their manpages as they are installed if necessary), it wouldn't be very difficult. It never occured to me (or anyone else likely) because it just isn't a problem. You try to use a program one time, it doesn't work the way you expect, you look it up, it takes 4 seconds, if you use it a couple more times then you just know.
Scripts don't run away from the command line, they embrace it . Scripts are the reason you use the command line; it's just text, so once you can do something once you can do it as many times as you want and share it with others instantly. Have you seen a macro to automate a GUI process? That moves the mouse around and clicks on stuff? And breaks constantly? Probably not, because nobody uses them.
Once you get into a command that's long enough to type into a file you've already written a program and sure if it's worth saving people will save it. The CLI blends using a machine and programming it and IMO that's a good thing -- it promotes literacy instead of encouraging people to point and grunt at shiny menu boxes and to refuse to learn how their machine actually works. If you're doing research and your main tool is a computer, it pays to learn how it works. It's not exciting but it's the same in every field. If you want to become an expert you need to understand your tools on a deep level.
Sure GUIs can be bad (just like clis). But GUIs can be good, which is pretty much impossible with command lines. You have wizards that take you step by step; you have tabbed control panels with profiles you can save for next time (just like scripts) e.g. Beyond Compare. Everything can be labeled in front of you, reminding you what's possible and grouped sensibly. All impossible with command line tools.
I think you have it backwards. Someone has pre-anticipated 99% of everything you will ever do with a computer, no matter who you are. And if you don't take the time to learn the things that let you do these things in one minute flat each:
...then you are going to spend an hour writing each one of them in Java or C++ or some nonsense like that and then debugging it. With Python you'll get it down to fifteen minutes each, maybe.
Now, I'm not going to claim that the command line is the only way to do these things (except for the last one). SQL or Excel or probably REXX would work too. But take advantage of the massive body of software that's already out there that can solve the 99% of your problems you have in common with everybody else, so you can spend your time mostly on the 1% that is your research.
No, nobody, say, anticipated that I would try to upgrade Ubuntu while using Spotify. Otherwise it wouldn't have broken the app. Nobody anticipated that my friend would have upgraded to Windows 10 in whatever machine state he did it otherwise it would not have deleted and reset his user profile and all his permissions. Both these issues are unexpected and require handling on a deeper (commandline) level than 99% of uses, not because the users are trying to do something weird but because the systems are so complex that even normal use can lead to wildly unexpected situations.
You can use the shortcuts and use GUIs and they are excellent and of course I use them as everyone does but if they are all you learn the moment anything unexpected happens (which happens a lot when you are using or building experimental software) you are dead in the water because you haven't developed a mental model of how your system actually works.
Hmm, I guess I see what you mean. I don't think that's mostly the kind of thing pgbovine was talking about, though. He was talking about the hurdles of getting to "hello, world", or whatever the equivalent is in terms of modifying some random piece of research software they've downloaded. Those hurdles are what cost his students productivity when they don't know how to navigate them, and how he teaches them to leap over them.
Now, it's also true that when things break at one level of abstraction, you have to debug them at the next level of abstraction down, which means you have to understand that level of abstraction. But I don't think he's talking about that, because (among other things) that rarely has to do with the command line as such.
Can't upvote this enough, and I urge people also to read his other article on this topic, as well as the rebuttal linked within.
I got into computers in the early 1980s when built-in BASIC interpreters were considered Terribly Clever and graphics were something you got by loading/constructing an alternate character set to fit on the text grid. Like many others I stumbled onwards through Vax terminals and onto PCs, where I was lucky enough to obtain a copy of AT&T System V/386, heh heh. So that's a fairly decent 35 years of experience with command-line interfaces...
...and about 15-20 years of dismay at people continuing to reinvent the damn things. Look, you know the only reason for commands like mv and cp was that one time we had 300 baud modems hooked up to dumb terminals and there were good reasons to worry about minimizing character count? It doesn't mean that the resulting obscurantism was a good thing in itself. It's not. There are no good reasons to make people go through typing everything and to perpetuate ASCII graphics fetishism and so on. If anything, it's become an unspoken set of rules to keep people away from our nerd treehouse. Actually typing out all that shit is a huge waste of time.
Writing shell scripts and makefiles by hand is insane, we buiilt computers precisely to save us from this sort of pointless tedium. I find it really depressing that UI innovation on Linux has basically ground to a hatl and that KDE, Gnome and so on are have basically not evolved at all over the last 15 years. Back around 1999 I was running Gnome and E.16 (or was it 15?) and I had a hugely customizable UI that was may more powerful and flexible than anything on Mac/Win (admittedly this was less true for applications, but still. Nowadays when I boot into Linux it mostly feels old and tired.
As I pointed out, I've been using CLIs for a good long time. I got my first copy of Linux from comp.os.minix because there were no distributions as such at the time. It's not like I can't figure this stuff out. But I'm sick of it! There's a reason most consumer software is built around GUIs - they are better. In a well-designec GUI, you can find everything you need within the user interface, instead of having to accumulate a heap of arcane CLI knowledge (starting with the question 'what do I type?!').
It's true that the CLI offers greater flexibility of you know exactly what you want to do. It's also true that for physical items, you can cast or 3d print something that is absolutely perfectly shaped if you know what you're doing. But if you have kids and you want them to discover the joys of engineering, you should probably buy them a big box of Lego instead of a set of carving tools or a CAD workstation.
Oh I still like E. What I mean is, I'm disappointed that UI experimentation and development on Linux seems to have stalled. I'm having trouble thinking of the last time I saw something that suprised/impressed me there, maybe Compiz?
In my similarly long experience, I have discovered that it's OK if the tools that you use nearly every day are "arcane", whether they have a GUI or not. Any time invested in getting to learn the tools will be paid back in efficiency later down the road.
As a simple example; an avid gimp user will not click through menus for most of the common actions, but will rather use keyboard shortcuts. It is no different for any CLI environment where you need to memorize a bunch of programs.
It certainly would be nice if your very first early start would focus more on the actual programming, and less on the weird abstract environment that most of us work in. But once you're hooked on programming, and that probably happens quite fast if you are into it, I find it hard to believe that you would not be willing to spend time to optimize your tool usage. The CLI does not get in your way there at all IMHO. The old unix philosophy of having a bunch of very simple tools that do one thing well and can be combined, is very useful in the day to day working environment.
I think it is ineffective to only focus on command line "bullshittery" and not discuss API bullshittery. I think the problem of learning commands pales to the problem of learning poorly-designed APIs. One extreme example of this is the Windows Event Tracing API [1], but there are many more along the same vein.
While I completely agree with the gist and point of the the article, the word "bullshittery" is a bit over-the-top.
I grew up in the suburbs of Detroit and in middle school and high school there was an abudance of shop classes; wood shop, metal shop, electronics, auto shop. In first-year wood shop, we started out with a little coping saw and a file and made golf tee holders and bird houses out of soft pine. After a while, our projects got bigger and we moved on to using stuff like planer and jigsaw. After we demonstrated proficiecy with those we moved on to more complex/dangerous tools like the lathe, table saw and drill presses. Stuff that could easily take a finger off, if you weren't careful.
Throwing someone unexperienced at a lathe or a bandsaw is simply _irresponsible_. I think that one need to learn the basics, to grasp the the mechanics, with simplier/less dangerous tool. However, one proficiency is gained there is a new class of tool available to a skilled craftsperson/professional.
Yes, its important to learn how to use a coping saw, but that doesn't make a table-saw "bullshit". Just a different class of tool, that could seriously hurt you if you don't know what you are doing.
I had the same response. I can't imagine hearing a carpenter go on and on with "Its absurd that i have to set the depth of my dadoes. Its so hard to be me and have to sharpen my chisels. Someone like me should not have to set the depth of my planes or adjust the width of the opening. Damn you Lie and Neilsen you are ruining my brilliance. If thats not enough I have to constantly explain these things to these wretched apprentices that are always bothering me." It sounds like entitled whining to me. It is a job, not a recreational activity. Soldiers have to clean their rifles and ballerinas need to break in their ballet shoes. I have to imagine that even individuals at the pinnacle of their professions occasionally have to deal with things they would rather avoid.
This analogy falls flat when you compare the levels of Unix 'bullshittery' to a shop. In a shop you have a limited number of already working machines that are readily visible. In Unix, you not only have to know how to use the machines, but how to install them, how to repair them, how to select similar, yet slightly different ones, how to copy your machines and make them run the same way in a different location. And on top of that to a novice all of the machines are hidden, and you have to read manuals/tutorials/forums to even know that they exist.
Unix tools can sometimes be horribly obtuse to use when first starting out. As you get use to them, you may discover the elegance of simple programs with pipable output, but their beauty is not readily obvious. Going from Windows installers to apt-get to git clone/make/setup, is a long process. Expecting students to already know this eats up time.
You're talking about a mythical shop which already has all the tools - very few shops have them all. Similarly, there are plenty of physical tools which are not obvious to use; my housemate tinkers in the shed with wood and he's got a variety of tools, some of which I'm nonplussed in how to use. Plenty of the tools are hidden, whether 'hidden' is defined as 'not present' or 'unclear usage'.
Likewise methodology. As a carpenter you might wonder how to curve wood. You might even hear about steam being involved. But unless you're finding out from someone who's already done it, it's not that simple to just steam a bit of wood and usefully bend it from first principles.
Hell, even just using a chisel for a novice is easier if you talk to someone first - especially a metalworking chisel.
Of course, the analogy falls down–like all analogies–at some point. However, there is a whole realm of industrial fabrication beyond what is available at a simple/hobby shop.
To put it a different way: there are tools that I want to use when I am making one of something. Then there are different tools when I am making 10 of something. Again at 100, 1,000, and 1,000,000.
I've been a teacher, and I think there is a inherent conflict: As a teacher, I want my students to learn _how_ to do something, but I also want them to do things "right". Those two goals are often at odds. Sometimes the best way to learn is to do it "wrong"; preferably in a safe environment where doing it wrong won't kill/maim you. I think the injustice is trying to teach students how to use the "for a million" tools (which they will _need_ later), when they really struggling with the idea of making one.
>>I can find, so it's profoundly stupid to disproportionately filter out entire demographics based on bogus criteria such as prior familiarity with incantations like “nohup tar -jxvf giant.tar.bz2 2> cmd.errs &”
There's some merit in what the professor is saying. A lot of the minutiae get in the way when you want to get something done. Who wants to wrestle with some obscure option to some kernel module to get ones graphics drivers working before starting work on a serious graphics project?
However, having grown up in an era when GNU/Linux wasn't friendly at all, I think I learnt a lot of stuff that's valuable to me even today. Much of it was concrete factoids which I no longer need or use but the valuable parts were some kind of "meta" skills (which Eytan[1] discusses much more eloquently in his rebuttal) which serve me very well even though I don't wrangle with the nitty gritty of getting obscure software working on my computer. Some examples I can think of are.
a. I can decipher man pages and other unfriendly but mostly technically complete documentation.
b. I can speak the language of the programmer fairly well which makes my work much more smoother than if I had to speak through a UX interpreter.
c. Given some kind of task, I have the confidence that once I learn the domain specific stuff, the computer and software can't really cripple me and prevent me from getting work done.
d. In case of an emergency, I know where the metaphorical life jackets are and what to do. I also have the confidence to get and use them.
e. I'm conversant with the ideas of pipes and other such tools that allow me to automate away annoying tasks which would have otherwise bothered me.
f. Even when resources are scarce (e.g. over a slow net connection on some remote machine), I know that I get work done without the fancy interfaces.
This makes me generally more efficient and I don't think I'd trade all those years of dealing with these arcane tools for anything else. I mentor students these days and a thumb rule I use to evaluate them is to see if the've managed to install a "non-mainstream" Gnu/Linux distro on their machines. If they've managed, it's almost surely a sign that they're good learners.
This is not an argument for making command line interfaces hard or keeping them that way. It's just that there's a school of thought that'll sacrifice ease of use for power and there's some value in that even for people not directly working on the tools themselves.
He's 100% right. It's all "bullshittery". Unfortunately, that's a rather trivial observation. Kinda like saying 2 + 2 = 4. The next part is the hard part - which to fix first?
I think what he really wants is a suri like AI that just does it all for him. Of course, at that point, I suspect skynet will have taken over.
That we still revolve around static, line-based interpreters from the VT100 era (over more intelligent CLIs or hybrid UIs like Oberon) is indeed rather quaint, and the proliferation of underdesigned programming tools also, but I strongly disagree with the author's claim that all these Unix programs are "[bullshit] not intellectually interesting in any way". Perhaps I'm one of the few people who actually likes to read the source code to the nodes in their /var/log/packages, but potentially getting involved with userspace system programming is not to be dismissed.
He makes a good point. I once tried teaching my two teen-age kids Python. I was surprised to discover that before we could get into any Python I had to explain all kinds of things I took for granted (or, perhaps I don't remember how I learned them) - what is terminal (this is OSX), what is the command line, why it's called "shell" and what is the kernel, what is the current directory, what are text files, how to edit them, why there is vi, vim and emacs, etc, etc. All in all it's many many hours of explanation of the environment which you need to understand before you can do any programming. We spent a couple of weeks on this stuff, got to some basic Python and then the summer was over :(
It's especially curious that one of these kids took a class in high school called "Computer Science" and they were using Java as the learning language. Somehow they managed to write programs that performed cool animations without knowing what the command line is.
I recall my high school computer science courses. They consisted of us writing stuff in DrJava, using a framework our teacher made to abstract some Java-specific things away. We ran the programs we made within it, without going into the command line. It was quite a simple IDE, but it allowed us to get things done without using a command line.
I'm in my fourth year of a computer science degree, and there are students graduating with me in a few months who have never written code outside of eclipse.
They've even had us 'deploy' a website with filezilla. It obviously works well enough for the size of projects we were dealing with, but I can't help but be a little put off by how unfamiliar/uncomfortable most people (in most of my classes) are with the command line.
> before we could get into any Python I had to explain all kinds of things I took for granted ...
It's possible to start writing Python without really knowing any of the things you list with an IDE like PyCharm -- which is probably a similar environment to the one they used for Java in "Computer Science."
It still too tricky to do things that make people, that don't know much about computers, excited about programming. Things like the C64, DOS or Qbasic were much "simpler" and of course the expectations were already lower. I would recommend something like Scratch or Processing instead.
Admittedly I didn't cut my teeth with C64 or DOS, but I have to imagine that it was only simpler because the barrier to entry was so much higher for those machines than it is today. And I would argue that expectations were higher in the past. Looking through some older 'entry level' computer science textbooks recently, I've been amazed at how complicated the material is and how quickly it covers things that we've only begun discussing as I'm about to graduate.
In those days, we also had magazines like Byte, Family Computing, Enter, etc. that included sections of type-in programs. My mom and I spent years doing that together!
Java (along with XCode) is unique in that there are nicely-packaged IDEs that let you do professional-level coding just by downloading one program.
My first professional experience was in Java. I worked for a year for a startup, working on 3-4 different programs and doing the entire UI for their product. After 2 days trying to compile things with Notepad, one of the other devs suggested "Go use JBuilder, we have a site license", and I never looked back. I didn't have to touch the command-line once. I would hit "Run program" and my program would run. I'd select "Add library JAR" by right-clicking on the project file and selecting the library. I could check out files from the VCS with the SourceSafe plugin just by right-clicking on a file and selecting a menu item.
Then I got to college, and, as some unpaid volunteer work, tried to rewrite a quickly-growing Harry Potter fansite as a database-backed PHP webapp. And in the course of getting that to work, I had to learn all these crazy UNIX commands. I had to learn about CVS (this was before Subversion and git) and how its command-line worked. I had to learn vi, because that's what I had to edit files on the server. I had to learn how to unpack tarballs. I had to learn ssh and scp because our server-admin guy was like "No way in hell am I opening up telnet on this box." I had to learn Apache configs and PHP.ini and all sorts of other magic incantations.
I'm really glad I learned them - my Linux & vim skills have actually aged better than my Java skills have, and I use vim & UNIX commands every day now. But damn, it was a lot of bullshit when I just wanted to get some text up on a website.
This is why I prefer Racket (and Dr Racket). It even comes with a great book: http://www.htdp.org/2003-09-26/Book/ . Although I'm not really sure if the material / exercises there can really spark the imagination of a child, it is fairly accessible: unlike SICP, it doesn't assume knowledge of other advanced engineering and math topics.
Yes, and algebra is the "command-line bullshitery" of doing physics.
I struggled for many years over this (my username is an in-joke with myself). There's basically two days a month that i can actually comprehend algebra, the rest of the time those little symbols merge and splurge themselves across the page until i'm dizzy.
The beginnings of a breakthrough happened when I started writing a program (algorithm) for every bit of algebra I couldn't understand. Now I am starting to see that the equals sign is just not for me. I think in processes, not static equalities. And indeed, there is a whole branch of mathematics with this philosophy: category theory.
167 comments
[ 3.0 ms ] story [ 224 ms ] threadMy 16 year old son taught himself how to program in order to mod his own video games. I wanted to teach him, but he figured it out from Youtube Videos and Google searches. He had to learn how command line switches worked, etc in order to use free and open source programming tools.
I learned Unix at a university in 1986 as part of a computer science class. I got into Linux in 1995 using Slackware on a 486DX clone. I remember a time before the GUI was used where everything was command line based in DOS and writing batch files to do things. Nobody taught me how to do that, I sort of learned it from trial and error. A lot of my DOS programming classes I had to use command line switches to compile my programs and edit text files with a Word Processor or the edit.com in DOS 5.0 or something. Things sure have changed now with IDEs and source code editors that highlight syntax in color. I learned HTML using Windows Notepad to make HTML text files. Things would have been easier if there was a class on this stuff that was required before taking the classes that needed it.
I worked in a computer lab in 1990-1994 at a college and taught the command line options to students just learning how to program.
When I learned Visual BASIC, everything was GUI based and it even generated forms I could paint controls on. It changed the way programming worked for me.
But free and open source tools are still command line based, and someone has to train the students how to use them. I sort of wish I still worked in a computer lab and training students, but those days are gone.
Say there was this nice GUI that built and setup lot's of projects for you. Who's responsibility is it to make sure that everything it supports stays supported?
Things change. It's a balance between the pay off and the ongoing investment to maintain the automation.
For long established, regularly used tools, there is a case for funding this activity somehow.
For the many specialized tools? I'm not sure how this would actually happen, unless the person who creates it decides to maintain it. When their career focus doesn't value that tool, we can't really expect anything.
Worse, let's say some established systems exist. They have a barrier to entry in that new software needs to be compliant and potentially integrated on some level. People already are focused away from this stuff. Raising that bar means fewer overall contributions.
Now, it's often: I wrote this. It's open. You can use it.
And they move on.
If, suddenly, I wrote this, and it's open gets followed up by, "you need to make sure this works with all this other stuff..." what happens?
A lot of research type tools get written and then left laying a out. Just testing and packaging, integrating, etc... would be a considerable project.
Could be the basis of some sort of course or career though. I'm thinking of the sysadmin types who manage software and research teams outside academia. The really good ones I know used to hold a position, or have some personal interest sufficient to make these sorts of activities make sense.
in general, it's a great productivity booster across the board to have a team doing this; the main problem is getting buy-in from orgs, because the time and effort wasted when you don't have one are so pervasive as to be largely invisible.
After years of that, "sudo apt-get install" was like a breath of fresh air.
If they can't figure that out, how will they ever learn how to figure out real world software engineering?
0: http://www.pgbovine.net/two-cultures-of-computing.htm
We used a whiteboard to draw a flowchart of the logical steps needed to make Tetris work, then wrote pseudocode for those modules.
In the last couple of years I've had few people ask me 'how can I write a program with a GUI.' And I give the advice, 'Download the free version of Microsoft Visual Studio for C# and select new project, windows forms' and then start mucking around.
All of them were convinced by others to use qt and C++ All of them gave up after a week or two of no progress.
Bogged down in an editor and a compiler?
When first starting out, there are two tools.
I think that part of the value, if not the major part, is that at the PhD level, you're given a problem that is ill defined, that your advisor doesn't know how to solve, and whose solution doesn't necessarily come entirely from within your field. Though I was a physics student, solving my thesis problem required developing skills beyond my advisor's own expertise, in areas such as electronics and programming. I had to invent a solution that he had not imagined. It helped a lot that those things were also my hobbies.
Which brings us to this reality. One day, and I pretty much guarantee it will happen, your student will download something from the Web and they will get stuck.
You don't know stuck until it's actually on fire. ;-)
Reads like a line from the "things i won't work with" section of In the Pipeline.
Kind of.
It's pretty nice once you get used to it.
Please note I'm not saying anything is better in the Windows world; far from it. I'm more talking about the transition from GUI tools, where feeling your way around in the dark is much less daunting to the uninitiated, to the command line.
I look up commands on the Internet, and then put them into single line shell script functions with a name. And then I write a comment about how it works, and check it into a git repo.
Then I never have to remember the exact invocation. I just type my nice function name, which is consistently named, and describes in my own words what the command does.
It's odd to me to see my coworkers and others very slowly coming up with a long command line, getting it wrong, very slowly correcting it, etc. I don't have time for that. I just grep my shell scripts directory for something close, copy and paste, edit it in my editor, and then run it.
It's very quick and puts no strain on the mind. Actually I find enjoyable to put my own names on arcane command line syntax.
This happens with every single engineering team. Those who understand the tools wrap them up into something solid so new folks have a great experience on their first day.
1. Install some proprietary, GUI-based development tools and color inside the lines,
2. Figure out what toolchain you want your students to use, and package it up using apt or docker or whatever (and tell your students to get a VM if they're on a different OS), or
3. Give your students remote logins on shared computers.
Any of these will allow you to keep the "bullshittery" to a tolerable minimum, and get on with the subject of the course.
But in the long run, unfortunately, if you want to be a programmer, you've got to learn to manage your own tools. This might mean being able to figure out command-line "bullshittery", or it may mean knowing how to unit test ASP.NET REST controllers. Even in the days of MacOS 8, back when there was no command line, there were still plenty of unholy, mind-eating horrors hiding in the dev stack. (The Component Manager, for instance.) And let's not even talk about the complexities of the web stack.
You can choose your flavor of "bullshittery", but if you want to be halfway good at this job, you've eventually got learn how to deal with whatever tools you choose.
* "You're on your own path" you get errors that the person next to you doesn't.
* "To learn is to invent; you must come to realizations by yourself" different solutions in stackexchange in 2012, '13, and '14 don't work for you.
* I don't have a cliche-quip for this, but if religion is how you find your complex relationship with the world, programming is how you find your complex relationship with computation. I emphasize "how" because its a process and a practice.
* It's mostly bullshit. Because we don't have physics-for-everything, we have religion. Because we don't have math-for-everything[0], we have programming. I'm speaking loosely and rhetorically.
* Related to the above, religion is a bunch of workarounds, like programming. This ecumenical counsel said such, but that ecumenical counsel revised it.
[0] In this instance, I'll define "programming" to be "mostly failed attempts at precise description." Math-for-everything would be "very precise descriptions."
However, Philip trivializes the "10,000 hours" spent learning this stuff as merely learning how to cope with this interface. This is so far off it hurts. Those countless hours spent messing with free software is how I learned to use other people's work, compile it, read it, fix bugs in it, and learn how other people think and write software. This is an incredibly important educational experience and there's sometimes no substitute for time and patience.
In 1995 you'd have said "learning to understand what a strtok string split routine looks like is learning how other people think and write software". And you'd have been right. But you would also have been describing bullshittery that we are all glad not to have to deal with anymore.
That's the overall feeling I get from this piece: there is more bullshittery to eliminate than the 400 different versions of "xmalloc" that we had to deal with in 1995.
Also: that because understanding how to iterate over the tokens of a colon-delimited string with strtok() isn't actually intrinsic to much computer science research, it's very important that we don't judge aptitude or capability by how conversant people are with strtok, or whatever it is strcspn does.
But as I have gone down rabbit holes with Haskell, C, LLVM, and some other tools, I have discovered that seeing those 400 different implementations of basic data structures has been extremely helpful. Reading and re-reading decades-old tutorials on finer points of gdb and memory layout and books on what NUMA-aware memory architectures are like has been extremely educational and important. I would now regard MATLAB, and even the idea of wanting a prototyping platform like MATLAB, as bullshittery.
I used to think that progress, so to speak, in computing was supposed to mean less and less need to be proficient with low-level tools, and a constant push to abstract away the low level and use languages or tools that offer the same things in a high level, black box manner.
But I totally don't think this any more. Now I want to be a power user of all the low level things and I wish that the surface area of tech and science wasn't growing so fast that it forces me to be a wimpy abstraction user in a lot of domains that I don't have time for.
Anyway, I'm just adding my perspective that the attitude that seemingly cumbersome low-level stuff that was formerly instructional but now abstracted is a nuisance is wrong. The world overall doesn't have nearly enough competent technicians and engineers who really know (intuition in the fingertips) that low level stuff, and we're allowing the superficial pleasantries of limiting abstractions to raise up generations of programmers who not only don't know this stuff at all, but also aren't even curious about how it works. They can do jobs when the black box abstractions are working, but as soon as something breaks, they don't know how to fix it.
Maybe one way to say it is that we're breeding programmers who have shifted their curiosity away from how fundamental atomic parts aggregate to make functioning tools and towards how to permute a series of ready-made black box abstractions for some social-level end goal (e.g. how do I permute Heroku and Postgres and Spark ML to make a recommender system) -- but it's not actually clear to me that it's better in a utilitarian sense that we're supposedly "freed" from our low level chains to think about these one-level-up problems.
I've found, personally, that my motivation has changed fairly dramatically over the years, and with it, my perspective on high-level vs. low-level programming. When I was in high school and college, my motivation was that I wanted to feel competent - I wanted a skill that was all my own, that I could master without anyone telling me I had to. Immediately after college, through my first job & startup, my motivation was to get rich, or at least develop a financial position better than my peers. While I was at Google, my motivation was to earn the respect of my peers, and now that I'm out on my own trying out startup ideas, my motivation is to test my intuitions about the software industry.
When I wanted to feel competent or earn the respect of my peers, I've usually gravitated toward low-level technologies and concepts, things that were hard conceptually or close to the machine but didn't have much practical application. Haskell, lambda calculus, compiler design, blazingly fast code, the bits & bytes of network protocols. When I wanted to get rich or test out my ideas, I've gravitated toward very high-abstraction, productivity-enhancing, flavor-of-the-month technologies. Javascript (in 2007, before it became hot), frontend dev, Polymer, React, native mobile development, Parse & Firebase, etc. I currently care very little about tech, because I've learned that in the grand scheme of things it will usually not make-or-break a business idea.
When I read your post, I'm curious why you use terms like "wimpy abstraction user" or "superficial pleasantries" or "generations of programmers who not only don't know this stuff". That seems to imply that you don't respect these programmers. Why? Frontend devs frequently make more than backend devs, and have many more job opportunities to choose from. The people who make the most (founding CEOs) often have the wimpiest coding skills of all. What makes for a status hierarchy where the closer to the machine you are, the higher the status? Not saying it's wrong - I've certainly felt like this at times too. But given that I'm now in my "study reality and poke it in certain ways to see why it behaves like it does" phase, I'm curious.
FWIW, paradoxically I've found that each time I've moved from a core motivation, it's because I've found that I can never achieve what was motivating me. So my "feel competent" phase ended when I started digging deeper into all the subfields of computer science and realized I could never master it all. My "get rich" phase ended when I realized that I already had enough and no matter how much money I earned, there would always be something else I might want, if I let myself. My "respect of my peers" phase ended when I realized that there will always be some people who liked you and some people who hated you and that which side they fell on depended more on how you treat them than on anything you accomplish yourself, so really all I needed was the respect of myself. Maybe the "test my intuitions" phase will end when I decide that no matter how much information I have or what I experience, there will always be an alternate explanation, but I suspect that will trigger an existential crisis that I'm not quite ready for. There's probably some brilliant Nietzschean revelation in there somewhere, but I'll settle for some observations on status & respect in programming communities. ;-)
I think your question is a bit too vague for me to be very interested in it. "What motivates me" probably changes based on how hungry I am, and there are super basic answers like "I happened to be good at math" that can better explain most of my motivation than some wistfully self-reflective assessment.
But, if I someone pulled a gun on me in a darkened alley and made me answer, I would say that my motivation to be interested in tech is that you can create so much with it, and I like working in a creative/design-space capacity, and I don't have any natural talent for other forms of compositional creativity like writing or music.
There are certain other people whose respect and thoughtful opinions I value (I doubt any of them would choose to work for Google, at least not modern Google), but I want to retain independence of thought too and I don't mind if pushes me slightly along the Martin Gardner "hermits and cranks" spectrum.
I do care about financial success, but only because I've been dealt a harsh set of external responsibilities in life and I don't feel I have much choice but to use whatever talents I have to claw out of that. That being said, though, in the two most lucrative and status-gaining positions I've had, I couldn't stomach the office politics and the way that arbitrary political and status signaling consistently overruled evidence-based pragmatism, and I quit those jobs despite putting myself in extremely risky financial situations both times. I can't say for sure, but I think if I had to face those situations again, knowing what I know now, I still would quit, and possibly would have quit even sooner.
But if I had to say one specific thing that is the motivation, I would say it is that I want to personally feel a sense of pride and craftsmanship in my work. When I solve a business problem by writing software, I want to feel like the way I solved it was full of craftsmanship, in addition to feeling proud of the final product. The manner in which I work is very important to me, perhaps much more important than the actual tasks that comprise the work itself.
Most tech jobs are horrible from this point of view because the only aspect of the job that can be consistently micromanaged from an outside point of view is the manner in which the work is completed (use this tech stack, log progress in this Agile/Scrum software, etc). You can't yet jack into a programmer's brain and micromanage their approach, from a creative point of view, so middle managers settle for micromanaging the outward appearances of how work gets done.
I could speculate too much, and too incoherently, but I think my comments about how we're teaching people to fundamentally value thinking about component abstractions more than reductionist understanding of atomic units relate to this, and go along with efforts to commoditize software labor. My preference is that this should be stopp...
Continuing your analogy a bit further, we can say that many politicians (dictators) make much much more than most founding CEOs and they have the wimpiest of wimpiest coding skills of all! I mean, we must not confuse between the ability to make more with ability to tackle technical problems. Regarding the status hierarchy, we can see that Nicola Tesla was certainly very closer to machine as compared many people, surely he has a very high position in the status hierarchy than most of the run of the mill founding CEOs with wimpiest coding skills.
Out of curiosity, how true is this statement? I hear from my friends that most frontend devs start with higher salary, but have a hard time climbing the corporate ladder, especially if the frontend is not the core competency of the company. I wonder which one is a more accurate depiction of reality.
However, frontend tech tends to expire every 5 years. This is part of the reason why starting salaries are high: junior developers can compete effectively with senior ones. If you go backend, you tend to amass a steady stream of domain & infrastructural knowledge about the company which makes you significantly more valuable to them over time. Raises and promotions follow, even if that knowledge is of limited usefulness on the open market.
Whether you'd prefer one or the other partially depends on how you would like to manage your career. The OP in this subthread indicates that he's drawn to the craftsmanship, the idea of steadily making something better over time, and this style tends to suit backend dev better. I tend to be drawn toward discovery, the process of finding loopholes & gaps in the market and filling them with new solutions, so for me frontend & entrepreneurship is more useful. (Frontend dev tends to be a more useful skill for founders, because almost every startup needs an appealing frontend but most can get by with a half-assed backend until they hire more engineers. Witness DropBox vs. Tarsnap - they came out at the same time and both use AWS for storage, but DropBox built a beautiful frontend and a minimally-functional backend, while Tarsnap barely built a frontend at all but has the most sophisticated secure backend I've seen in a product.) For me, the idea of climbing the corporate ladder sounds like hell, while to other engineers on this site, the idea of retraining in a different technology every 5 years probably sounds like hell.
It deletes the new line from the end of the string that fgets() insists on including, even though nobody anywhere has ever wanted it. :)
http://www.cplusplus.com/reference/cstring/strcspn/
foo[strlen(foo)-1] = 0;
But I don't think anyone would say strlen modifies the string.
If you are going to be building things or programming or creating new things you have to understand how things actually work. I'm sorry if that means you need to know that python on OSX is different from python on Windows but you do.
If you are doing research by definition your needs can never be anticipated so you will never have pretty GUI tools to satisfy your needs. So I know the command line sucks for most people but if you want to do real work you're going to have to learn how to use it since it drops all of the leaky abstractions that prevent you from doing what you need to do when you're doing something novel.
Every machine in existence has a command line interface. Some of them have GUIs built on top of that. Sounds pretty fundamental to me.
No they don't. Embedded computers do not have command lines, for example. Embedded computers don't give a shit about text. Most computers do not accept commands at all. As for those who do, some don't accept it in text form, for example gaming consoles or punch-card antiquities.
Command lines are the consequence of keyboards, and whenever you see a computer on which a keyboard cannot be attached or emulated (phones) then you're looking at a computer without a command line.
If you're going to be that pedantic, then you should go all the way and note that there are computers that have neither memory nor use electricity.
Once we've deleted the written word from human existence and no longer have need of text, have transcended proper known computational structure and can express computations soley by thought and avoid programming altogether certainly we will move beyond the command line.
The command line remains the simplest, cleanest, most leak-free abstraction we have to interact with a computer effectively. I'll agree with you that it's possible there's a better one, but we've been trying for 40 years and haven't even come close -- all we've done is written some translations which let people enter command line instructions by pointing and clicking and having the machine guess how to interpret it instead of typing what they actually mean. I think it's the best we're ever going to get though I'm happy to leave room for optimism if you have any. In the meantime, those of us who have to get work done (I assume you are one of our cohort) will live in the real world and use it without complaint.
The only reason the command line is complicated is because computers are complicated. If you want to use computers in a deep way you can't hide behind the GUI and run away from actually dealing with the machine forever.
Clean? I could agree with 'concise' but not clean. Some apps take multiple switches behind a single cue character (is it '/'? is it '-'? '--'? something else?) and some can only parse one at a time. Some take switches before text and filenames, some after, some have a mix.
Command line apps are the sort that only a mother could love. They are ugly, opaque, have an enormous learning curve, and even experts tend to type them into files because they would hate to have to craft them all over again each time they need them. Macros; cheat sheets; scripts are all crutches to escape the tyranny of command lines.
If GUIs had as many options as command line programs did they would be completely unusable trash, with an enormous unintelligible control panel, and you'd be looking iconography up in a reference manual instead of switches (have fun designing all those icons and having different ones express different subtle alterations in how a command will execute)
There are a lot of options because the tools are flexible and capable of expressing MUCH more than a GUI. I'll give you there are some consistency issues, but they are historical not fundamental to the CLI. I'm honestly surprised someone hasn't written a wrapper to standardize every app in a shell so they all use the same format (ie have a 'one true format' for argument passing and just convert as necessary for every CLI app, parsing their manpages as they are installed if necessary), it wouldn't be very difficult. It never occured to me (or anyone else likely) because it just isn't a problem. You try to use a program one time, it doesn't work the way you expect, you look it up, it takes 4 seconds, if you use it a couple more times then you just know.
Scripts don't run away from the command line, they embrace it . Scripts are the reason you use the command line; it's just text, so once you can do something once you can do it as many times as you want and share it with others instantly. Have you seen a macro to automate a GUI process? That moves the mouse around and clicks on stuff? And breaks constantly? Probably not, because nobody uses them.
Once you get into a command that's long enough to type into a file you've already written a program and sure if it's worth saving people will save it. The CLI blends using a machine and programming it and IMO that's a good thing -- it promotes literacy instead of encouraging people to point and grunt at shiny menu boxes and to refuse to learn how their machine actually works. If you're doing research and your main tool is a computer, it pays to learn how it works. It's not exciting but it's the same in every field. If you want to become an expert you need to understand your tools on a deep level.
Makes me think of the Altair 8800 front panel...
Now, I'm not going to claim that the command line is the only way to do these things (except for the last one). SQL or Excel or probably REXX would work too. But take advantage of the massive body of software that's already out there that can solve the 99% of your problems you have in common with everybody else, so you can spend your time mostly on the 1% that is your research.
You can use the shortcuts and use GUIs and they are excellent and of course I use them as everyone does but if they are all you learn the moment anything unexpected happens (which happens a lot when you are using or building experimental software) you are dead in the water because you haven't developed a mental model of how your system actually works.
Now, it's also true that when things break at one level of abstraction, you have to debug them at the next level of abstraction down, which means you have to understand that level of abstraction. But I don't think he's talking about that, because (among other things) that rarely has to do with the command line as such.
I got into computers in the early 1980s when built-in BASIC interpreters were considered Terribly Clever and graphics were something you got by loading/constructing an alternate character set to fit on the text grid. Like many others I stumbled onwards through Vax terminals and onto PCs, where I was lucky enough to obtain a copy of AT&T System V/386, heh heh. So that's a fairly decent 35 years of experience with command-line interfaces...
...and about 15-20 years of dismay at people continuing to reinvent the damn things. Look, you know the only reason for commands like mv and cp was that one time we had 300 baud modems hooked up to dumb terminals and there were good reasons to worry about minimizing character count? It doesn't mean that the resulting obscurantism was a good thing in itself. It's not. There are no good reasons to make people go through typing everything and to perpetuate ASCII graphics fetishism and so on. If anything, it's become an unspoken set of rules to keep people away from our nerd treehouse. Actually typing out all that shit is a huge waste of time.
Writing shell scripts and makefiles by hand is insane, we buiilt computers precisely to save us from this sort of pointless tedium. I find it really depressing that UI innovation on Linux has basically ground to a hatl and that KDE, Gnome and so on are have basically not evolved at all over the last 15 years. Back around 1999 I was running Gnome and E.16 (or was it 15?) and I had a hugely customizable UI that was may more powerful and flexible than anything on Mac/Win (admittedly this was less true for applications, but still. Nowadays when I boot into Linux it mostly feels old and tired.
As I pointed out, I've been using CLIs for a good long time. I got my first copy of Linux from comp.os.minix because there were no distributions as such at the time. It's not like I can't figure this stuff out. But I'm sick of it! There's a reason most consumer software is built around GUIs - they are better. In a well-designec GUI, you can find everything you need within the user interface, instead of having to accumulate a heap of arcane CLI knowledge (starting with the question 'what do I type?!').
It's true that the CLI offers greater flexibility of you know exactly what you want to do. It's also true that for physical items, you can cast or 3d print something that is absolutely perfectly shaped if you know what you're doing. But if you have kids and you want them to discover the joys of engineering, you should probably buy them a big box of Lego instead of a set of carving tools or a CAD workstation.
Why don't you keep using it? I'm still using sawfish.
It's incredibly fast, and the cost of customizing anything else at this point would be huge. (I've tried many times...)
As a simple example; an avid gimp user will not click through menus for most of the common actions, but will rather use keyboard shortcuts. It is no different for any CLI environment where you need to memorize a bunch of programs.
It certainly would be nice if your very first early start would focus more on the actual programming, and less on the weird abstract environment that most of us work in. But once you're hooked on programming, and that probably happens quite fast if you are into it, I find it hard to believe that you would not be willing to spend time to optimize your tool usage. The CLI does not get in your way there at all IMHO. The old unix philosophy of having a bunch of very simple tools that do one thing well and can be combined, is very useful in the day to day working environment.
[1] http://mollyrocket.com/casey/stream_0029.html
I grew up in the suburbs of Detroit and in middle school and high school there was an abudance of shop classes; wood shop, metal shop, electronics, auto shop. In first-year wood shop, we started out with a little coping saw and a file and made golf tee holders and bird houses out of soft pine. After a while, our projects got bigger and we moved on to using stuff like planer and jigsaw. After we demonstrated proficiecy with those we moved on to more complex/dangerous tools like the lathe, table saw and drill presses. Stuff that could easily take a finger off, if you weren't careful.
Throwing someone unexperienced at a lathe or a bandsaw is simply _irresponsible_. I think that one need to learn the basics, to grasp the the mechanics, with simplier/less dangerous tool. However, one proficiency is gained there is a new class of tool available to a skilled craftsperson/professional.
Yes, its important to learn how to use a coping saw, but that doesn't make a table-saw "bullshit". Just a different class of tool, that could seriously hurt you if you don't know what you are doing.
Unix tools can sometimes be horribly obtuse to use when first starting out. As you get use to them, you may discover the elegance of simple programs with pipable output, but their beauty is not readily obvious. Going from Windows installers to apt-get to git clone/make/setup, is a long process. Expecting students to already know this eats up time.
Likewise methodology. As a carpenter you might wonder how to curve wood. You might even hear about steam being involved. But unless you're finding out from someone who's already done it, it's not that simple to just steam a bit of wood and usefully bend it from first principles.
Hell, even just using a chisel for a novice is easier if you talk to someone first - especially a metalworking chisel.
To put it a different way: there are tools that I want to use when I am making one of something. Then there are different tools when I am making 10 of something. Again at 100, 1,000, and 1,000,000.
I've been a teacher, and I think there is a inherent conflict: As a teacher, I want my students to learn _how_ to do something, but I also want them to do things "right". Those two goals are often at odds. Sometimes the best way to learn is to do it "wrong"; preferably in a safe environment where doing it wrong won't kill/maim you. I think the injustice is trying to teach students how to use the "for a million" tools (which they will _need_ later), when they really struggling with the idea of making one.
Is this the best you can do?
However, having grown up in an era when GNU/Linux wasn't friendly at all, I think I learnt a lot of stuff that's valuable to me even today. Much of it was concrete factoids which I no longer need or use but the valuable parts were some kind of "meta" skills (which Eytan[1] discusses much more eloquently in his rebuttal) which serve me very well even though I don't wrangle with the nitty gritty of getting obscure software working on my computer. Some examples I can think of are.
a. I can decipher man pages and other unfriendly but mostly technically complete documentation. b. I can speak the language of the programmer fairly well which makes my work much more smoother than if I had to speak through a UX interpreter. c. Given some kind of task, I have the confidence that once I learn the domain specific stuff, the computer and software can't really cripple me and prevent me from getting work done. d. In case of an emergency, I know where the metaphorical life jackets are and what to do. I also have the confidence to get and use them. e. I'm conversant with the ideas of pipes and other such tools that allow me to automate away annoying tasks which would have otherwise bothered me. f. Even when resources are scarce (e.g. over a slow net connection on some remote machine), I know that I get work done without the fancy interfaces.
This makes me generally more efficient and I don't think I'd trade all those years of dealing with these arcane tools for anything else. I mentor students these days and a thumb rule I use to evaluate them is to see if the've managed to install a "non-mainstream" Gnu/Linux distro on their machines. If they've managed, it's almost surely a sign that they're good learners.
This is not an argument for making command line interfaces hard or keeping them that way. It's just that there's a school of thought that'll sacrifice ease of use for power and there's some value in that even for people not directly working on the tools themselves.
Footnotes: [1]: https://medium.com/@eytanadar/on-the-value-of-command-line-b...
I think what he really wants is a suri like AI that just does it all for him. Of course, at that point, I suspect skynet will have taken over.
It's especially curious that one of these kids took a class in high school called "Computer Science" and they were using Java as the learning language. Somehow they managed to write programs that performed cool animations without knowing what the command line is.
P.S.: Granted, it's not always going to be the best tool as a project grows, but it drastically lowers the barrier to one-off experiments and lessons.
They've even had us 'deploy' a website with filezilla. It obviously works well enough for the size of projects we were dealing with, but I can't help but be a little put off by how unfamiliar/uncomfortable most people (in most of my classes) are with the command line.
It's possible to start writing Python without really knowing any of the things you list with an IDE like PyCharm -- which is probably a similar environment to the one they used for Java in "Computer Science."
My first professional experience was in Java. I worked for a year for a startup, working on 3-4 different programs and doing the entire UI for their product. After 2 days trying to compile things with Notepad, one of the other devs suggested "Go use JBuilder, we have a site license", and I never looked back. I didn't have to touch the command-line once. I would hit "Run program" and my program would run. I'd select "Add library JAR" by right-clicking on the project file and selecting the library. I could check out files from the VCS with the SourceSafe plugin just by right-clicking on a file and selecting a menu item.
Then I got to college, and, as some unpaid volunteer work, tried to rewrite a quickly-growing Harry Potter fansite as a database-backed PHP webapp. And in the course of getting that to work, I had to learn all these crazy UNIX commands. I had to learn about CVS (this was before Subversion and git) and how its command-line worked. I had to learn vi, because that's what I had to edit files on the server. I had to learn how to unpack tarballs. I had to learn ssh and scp because our server-admin guy was like "No way in hell am I opening up telnet on this box." I had to learn Apache configs and PHP.ini and all sorts of other magic incantations.
I'm really glad I learned them - my Linux & vim skills have actually aged better than my Java skills have, and I use vim & UNIX commands every day now. But damn, it was a lot of bullshit when I just wanted to get some text up on a website.
[1] http://blockpy.com
I struggled for many years over this (my username is an in-joke with myself). There's basically two days a month that i can actually comprehend algebra, the rest of the time those little symbols merge and splurge themselves across the page until i'm dizzy.
The beginnings of a breakthrough happened when I started writing a program (algorithm) for every bit of algebra I couldn't understand. Now I am starting to see that the equals sign is just not for me. I think in processes, not static equalities. And indeed, there is a whole branch of mathematics with this philosophy: category theory.