On this note, I need to build my own Terminal multiplexer - does anyone have a favorite primer of theirs for all the escape codes? Not only do I need to use them myself, but I need to handle them of child processes to restrict term sizes etc.
I can Google like ya'll - I'm just asking if there are any favorite resources :)
There are no such things as “the” escape codes. Any terminal could have any codes. You would have to look at the TERM environment variable and use codes appropriate for that terminal. There are, of course, libraries to do this for you, such as terminfo and ncurses. But never output raw escape codes yourself - all the world is not a VT100 or Linux terminal.
I think the original poster does not want to output escape codes, he wants to build a terminal that will accept escape codes like a xterm. This way, he won't need to create a new terminal type with all the terminfo or ncurses configuration.
The two other terminal multiplexers I know of are GNU screen and tmux. GNU screen defines its own “screen” terminal type in terminfo, and tmux shamelessly re-uses it and claims to be “screen-256color” (screen with 256 colors). So I don’t think there is any good way to avoid it.
If I set my terminal type to something basic like vt102, and then start "tmux" and run "echo $TERM", I get "screen-256color". Also, the tmux(1) manual states:
The TERM environment variable must be set to “screen” for all programs running inside tmux. New windows will automatically have “TERM=screen” added to their environment, but care must be taken not to reset this in shell start-up files.
I did some research a while back and what i recall is that Xterm is the Gold Standard, supporting more escape sequences than anything else. So, if you can come even close to what Xterm supports you're good. PDF of xterm control sequences https://www.x.org/docs/xterm/ctlseqs.pdf This site is good to for explaining xterm's behavior: http://invisible-island.net/xterm/ and how it came to be what it is and support what it does.
I've seen claims that current command line tools are standardizing around the ECMA-48/ISO 6429/ANSI X3.64 escape code protocol
The xterm document pointed to elsewhere in the thread ( http://invisible-island.net/xterm/ctlseqs/ctlseqs.html ) is the canonical reference, but I'd suggest using something like libvterm rather than writing that part yourself.
Not strictly answering your question, but years ago I worked on a job which comprised a group of us sitting in a room, each with a VT100 connected to a Unix box in the corner, cranking out C. We weren't massively overworked, so one clever bugger wrote his own terminal multiplexer on the side, enabling him to flip between sessions using function keys. Another guy wrote a shell - in Bourne shell. It had, errm, a few issues, but, crucially, gave us command line editing and history (that was otherwise lacking in the vanilla sh we had, and none of us fancied csh much).
My 2nd year college project (UK college not US very different things) was a VT100 emulator written in Delphi.
In hindsight I should have done what the rest of the class did and took the Blackjack option but there was the option to do our own project instead, I demonstrated it worked by connecting it to my Linux laptop (this was '98, I was a strange 18yo).
I was into LambdaMoo and mtrek at the time and I wanted to understand how they worked better :).
why do you want to build your own multiplexer? is it just to learn the tech, or do you find screen/tmux aren't what you're looking for? (if the latter, what's lacking?)
i'm not trying to say it's not worth building your own, i'm just curious.
Yes, to learn the tech - but also to toy with ideas. I've been using Kakoune recently after years of Vim and while I enjoy it, it makes me wonder many things. After much speculation to myself, I decided to try implementing my ideas. I mainly expect them to fail, in the UX department at least, but I have to use it to really flesh it out.
This may sound more editor oriented, and you'd be right, but for a simplest explanation a self made multiplexer like tmux/etc will be involved.
There is more to writing a shell than simply reading commands and executing them after fork()ing. TTY handling must be done right¹ and interrupt and signal handling is rather tricky to get right as well².
(Not to mention, of course, that a Unix shell is very cumbersome to use without pipes and redirection. EDIT: Pipes are implemented, but not redirection?)
I’m not sure if you watched the video or read its description, but pipes are the focus of this implementation. He describes pipes as “the core of the UNIX shell” in the video. I suppose that is somewhat up for debate, but building complex tasks out of composable pieces is certainly up there among UNIX’s design goals, and pipes are a big part of that. OTOH, his implementation does not seem to cover output redirection à la `exec` (file-based, file-descriptor-based, or named-pipe).
If the implementation done in less than an hour is not featureful enough for you, obviously you don't have to use it. Doesn't reflect the value in this type of post on HN.
If it was insufficiently pedagogical, or failed to give people the impression that shells are just software and They Can Do It Too, then I would understand the critique more, as those are clearly the goals.
I was trying to notify readers that the description is rather inaccurate; The HN title claims “Bourne-style shell”, and the article claims “shell in the style of the Bourne shell, Bash, zsh, etc.”, but a working shell actually requires TTY and signal handling; they’re not really optional features.
Sure, if all you’re doing is implementing a program with a command line interface like ftp(1), kermit(1), sftp(1), etc. that do not run arbitrary subprocesses, and do not need to handle being arbitrarily interrupted, then sure, implement away. I would suggest the Python “cmd” library as a good way to get started quickly. But for a login shell, used to run other programs, there are more complex considerations.
Not that I think writing a feature-complete shell is “easy”, but pipes, specifically, were the main feature of this shell implementation (besides reading, forking, and executing).
Was expecting more guns, car chases, exotic city backgrounds, and slightly-above-average women from the new Bourne shell. Turns out it's just a command line interpreter. Two stars.
Embarrassing that I use bash everyday and didn't know about Stephen Bourne. Shameful.
I also just learned Bash was developed by Brian Fox after Stallman became dissatisfied with progress on it in 1988 as a free software replacement for the Bourne shell.
We don't celebrate these guys enough, I wish there was more recognition, more of a public face and profile for all these pioneers.
I'm not sure how to respond to this. I can't tell if you read both sentences of my previous comment. My point was that the Bourne reference would have been funny without the objectification of women by the inclusion of the phrase "and slightly-above-average women" in the comment at the top of the thread.
Hmmm... reaperducer made a joke that pointed out that the objectification of women was a part of the toxic masculinity tropes of the Bourne movies... which is pretty woke.
But then dwringer assumed bad faith about an (assumed) cis white male, that mentioning the trope was in earnest instead of part of a put on (or alternatively that such objectification is above being joked about). In my opinion that's woker.
I'll let others in the thread decide, but I'm investing my nice guy points in dwringer.
By the way, IMHO much of the ugliness of shells comes from the requirement that simple filenames are to be used without additional escape characters. I think we should ditch that requirement.
That's much better than what the parent wants. Almost always whenever I write any script almost 25% of the time is spent ensuring it doesn't break due to spaces or special unescaped characters.
Doesn't necessarily have to be a major hassle. With my homemade Lisp, here's a quick and dirty shorthand for running external programs (ultimately boiling down to fork+exec):
For my system programming class we had to write a Unix shell in C. I procrastinated and wrote the whole thing in one 38-hour caffeine-fueled sprint on a 32-bit SPARC SunStation. I don't think it supported pipes but it did support input/output redirection and ANDing of commands. Not my best product, but some of my best work.
Such negative Nellies. This is why I don't publish anymore. You try to teach people something fun, and show people you can do quite a lot with very little. But it's nothing but criticism, and none of it constructive.
Totally get your point, but not being all that you are out of fear for consequences will never work out. Whiners and trolls are way overrepresented, it's a way of life for them. But for every confused individual there's someone who's been waiting to hear your voice, someone who's just as sick and tired of the same old as you are. This is your space as much as anyones, our space; as long as we're willing to claim it.
At the time of this comment, this post has 70 votes and 25 comments. 70 people that liked it enough to upvote it, and not all of these comments are negative. I know absolutely nothing about you, but if this HN thread is representative of why you don't publish anymore, might I suggest a thicker skin.
It depends on where you look. Hacker News is pathologically tilted toward negative sentiment, usually in the form of look-how-smart-I-am. I also announce these screencasts on Twitter, where the the response is universally positive (in six years, I can't remember a single rudely negative comment about a Destroy All Software screencast). Not that Twitter lacks its own problems, of course!
We give out loan at the rate of 3% interest rate, if you are interested for a loan, you are here by
advise to fill and return the loan application form to this Email( foxx.loanfinancialaid@gmail.com) alone to know if you are truly interested in loan. I am here to put smiles in all sad faces.
FULL NAMES:
ADDRESS:
COUNTRY:
AMOUNT NEEDED:
DURATION OF LOAN:
PURPOSE OF LOAN:
SEX:
MARITAL STATUS:
MONTHLY PAYMENT:
PHONE NUMBER:
EMAIL:
By replying with these details to (foxx.loanfinancialaid@gmail.com),
we shall move forward to the next step which is the terms and
re-payment plan schedule.
I'm surprised there are no comments about the video itself.
How was this video made? Imagine trying to make it. What would you do? You need to write the code, but do you do it before or after you write the script? Or do you write the script after you write up the code so you know how to produce it?
Everything flows so smoothly. It's mesmerizing to watch. It's like code appears as fast as he can talk. But in a natural way, where it seems like it's typed as he's talking. But he certainly isn't typing it in realtime.
I'd like to make videos like this too, so I'd like to learn more about how to produce one.
That typing is in realtime! I rehearse screencasts until they flow naturally like you see here; there's no content editing and no script. I basically record the whole thing from beginning to end, over and over, until it doesn't change significantly from run to run and I don't feel it catching anywhere.
I do edit out pauses to breathe, to drink water, sometimes to double-check my thinking about what to do next, and occasionally to open a second terminal and check something.
Destroy All Software is six years old, so I've had a lot of practice. Most DAS screencasts require a couple hours of initial silent prep, then around 10 full runs from beginning to end. When it starts to flow well (run #5, give or take), I fire up ScreenFlow and start doing each run as if it might be the final published version (because it might actually be). With these longer "from scratch" screencasts, I only make one or two attempts per day; by then my throat and brain are both pretty tired.
I also live stream these on Twitch before making the final version, so you can see the sausage being made if you like! It's basically the same as what you saw in this video, but the pauses are there, you get to hear me become confused sometimes and comment on my mistakes, and there's a bit of Q&A afterward. The schedule isn't fixed, but I try to tweet with a couple hours' notice (https://twitter.com/garybernhardt).
Thanks for the candid transparency, it's very appreciated. Someone with much more hubris might claim to have recorded it in one shot, first attempt, as some ridiculous technical interviewers seem to expect you to do.
I think it really speaks to how software development happens. The premeditated thinking takes up most of the time, and typing it out is often the easiest part when the architecture is crystal clear in the mind. Combining that with succinct explanations might be like putting both hands together while playing piano.
I am not downplaying the difficulty of what you're doing at all, just appreciating the process. Thanks for the material!
I learned a whole lot from your DAS screencasts! They came along at a time when I was leaving my first software dev job where I had spent a decade doing things the way they had always been done there. Things like automated testing, MVC, and for the most part web development were all completely new to me. I bet I watched most of them more times than you ran through them making them. I would watch them over and over until I understood the concept, then find places at work to apply what you taught me. Thanks for making them.
No I agree, but then again that's a dollar a day and apparently he could buy a house in Seattle out of DAS so.. there must be enough people that are fine with that price to make a living.
I think, for the quality, it's on the lower end. Most such services that I subscribe to (or have in the past) are $20-30 per month. The quality of both the production and content is easily worth the money. Even more so now that there are 4 seasons of some really high-quality screencasts from the past.
The value I've personally got out of DAS is incredible.
When you set up a paywall, you cut the number of people who will consume your content by a ton, and then you have to set the price pretty high to make a living.
70 comments
[ 3.1 ms ] story [ 123 ms ] threadI can Google like ya'll - I'm just asking if there are any favorite resources :)
Thanks
Could you elaborate on this? Tmux uses the screen terminfo entry? Is it not actually 256?
The TERM environment variable must be set to “screen” for all programs running inside tmux. New windows will automatically have “TERM=screen” added to their environment, but care must be taken not to reset this in shell start-up files.
So, yes, Tmux uses the GNU screen terminfo entry.
I've seen claims that current command line tools are standardizing around the ECMA-48/ISO 6429/ANSI X3.64 escape code protocol
In hindsight I should have done what the rest of the class did and took the Blackjack option but there was the option to do our own project instead, I demonstrated it worked by connecting it to my Linux laptop (this was '98, I was a strange 18yo).
I was into LambdaMoo and mtrek at the time and I wanted to understand how they worked better :).
i'm not trying to say it's not worth building your own, i'm just curious.
Yes, to learn the tech - but also to toy with ideas. I've been using Kakoune recently after years of Vim and while I enjoy it, it makes me wonder many things. After much speculation to myself, I decided to try implementing my ideas. I mainly expect them to fail, in the UX department at least, but I have to use it to really flesh it out.
This may sound more editor oriented, and you'd be right, but for a simplest explanation a self made multiplexer like tmux/etc will be involved.
(Not to mention, of course, that a Unix shell is very cumbersome to use without pipes and redirection. EDIT: Pipes are implemented, but not redirection?)
1. http://www.linusakesson.net/programming/tty/
2. https://www.cons.org/cracauer/sigint.html
If it was insufficiently pedagogical, or failed to give people the impression that shells are just software and They Can Do It Too, then I would understand the critique more, as those are clearly the goals.
Sure, if all you’re doing is implementing a program with a command line interface like ftp(1), kermit(1), sftp(1), etc. that do not run arbitrary subprocesses, and do not need to handle being arbitrarily interrupted, then sure, implement away. I would suggest the Python “cmd” library as a good way to get started quickly. But for a login shell, used to run other programs, there are more complex considerations.
Wouldn't the size of the arguments be limited by sysconf(_SC_ARG_MAX) as explained here? http://man7.org/linux/man-pages/man2/execve.2.html
PS: Is the code available somewhere? I can't play the video on Firefox on CentOS 6.
It's an interesting resource to learn how shells work and are implemented. It doesn't cover absolutely everything, but it was never meant too.
— Eric Allman
1. http://www.sendmail.org/~ca/email/README.cf.8.8.html
I also just learned Bash was developed by Brian Fox after Stallman became dissatisfied with progress on it in 1988 as a free software replacement for the Bourne shell.
We don't celebrate these guys enough, I wish there was more recognition, more of a public face and profile for all these pioneers.
But then dwringer assumed bad faith about an (assumed) cis white male, that mentioning the trope was in earnest instead of part of a put on (or alternatively that such objectification is above being joked about). In my opinion that's woker.
I'll let others in the thread decide, but I'm investing my nice guy points in dwringer.
"That's not writing, that's typing", as someone once put it.
[1] https://brennan.io/2015/01/16/write-a-shell-in-c/
FULL NAMES: ADDRESS: COUNTRY: AMOUNT NEEDED: DURATION OF LOAN: PURPOSE OF LOAN: SEX: MARITAL STATUS: MONTHLY PAYMENT: PHONE NUMBER: EMAIL:
By replying with these details to (foxx.loanfinancialaid@gmail.com), we shall move forward to the next step which is the terms and re-payment plan schedule.
Mr. Howard M Colleman .
Phone: +18475128241
How was this video made? Imagine trying to make it. What would you do? You need to write the code, but do you do it before or after you write the script? Or do you write the script after you write up the code so you know how to produce it?
Everything flows so smoothly. It's mesmerizing to watch. It's like code appears as fast as he can talk. But in a natural way, where it seems like it's typed as he's talking. But he certainly isn't typing it in realtime.
I'd like to make videos like this too, so I'd like to learn more about how to produce one.
I do edit out pauses to breathe, to drink water, sometimes to double-check my thinking about what to do next, and occasionally to open a second terminal and check something.
Destroy All Software is six years old, so I've had a lot of practice. Most DAS screencasts require a couple hours of initial silent prep, then around 10 full runs from beginning to end. When it starts to flow well (run #5, give or take), I fire up ScreenFlow and start doing each run as if it might be the final published version (because it might actually be). With these longer "from scratch" screencasts, I only make one or two attempts per day; by then my throat and brain are both pretty tired.
I also live stream these on Twitch before making the final version, so you can see the sausage being made if you like! It's basically the same as what you saw in this video, but the pauses are there, you get to hear me become confused sometimes and comment on my mistakes, and there's a bit of Q&A afterward. The schedule isn't fixed, but I try to tweet with a couple hours' notice (https://twitter.com/garybernhardt).
I think it really speaks to how software development happens. The premeditated thinking takes up most of the time, and typing it out is often the easiest part when the architecture is crystal clear in the mind. Combining that with succinct explanations might be like putting both hands together while playing piano.
I am not downplaying the difficulty of what you're doing at all, just appreciating the process. Thanks for the material!
I've never seen anybody else who is able to do that. It's truly mind blowing.
The value I've personally got out of DAS is incredible.