Segfaults are the ideal case for memory errors, and those are the most easily caught and fixed, so you're least likely to see them. But, often those memory errors result in silent corruption which can be exploited, and that's harder to detect, especially if it relies on very specific corner cases. `curl` has had a number of these vulnerabilities over the last several years, for example.
Something as simple as `ls` is probably so small and battle tested that it's not an issue, but if you're writing an all-new, not-battle-tested tool, why wouldn't you want stronger guarantees? The new tool is being written for the features, but it's not fun to write vulnerabilities into something that should be simple and "just work."
Languages like Go and Ruby are also mostly memory safe, so those are generally fine picks here too, but every language has trade offs. In this case, the author clearly cares about performance, which Ruby does not care about.
Rust also has a built-in testing harness, which is a lot more convenient IMHO than using whatever C testing framework you might have a predisposition towards.
Regarding segfaults, when I was interviewing devs for C++ roles, I'd ask questions about a simple function like this:
std::string foo(bool flag)
{
if (flag)
return "true";
}
Questions I'd ask:
* Is the function well formed (Yes - functions need not return a value on all paths due to C ancestory, even if the return type has a non trivial actor - not sure if this is still considered well formed, but I think it was as least to C++11)
* what happens if 'foo' is called with true? (Returns "true" as one would expect)
* what happens if 'foo' is called with false? (Undefined or implementation defined behavior, but generally nothing nice - segfault, acces violation, etc)
* if it crashes, where, when and why does it crash? (Technically since its undefined, nearly any answer here suffices, if it can be backed up. Since practically, most optimizing compilers assume UB can never happen, when you return nothing from a non-void function, the compiler will attempt to invoke the destructor of a non existent object instance (assuming non POD) and boom.)
I asked this because it was a distilled example of a real world rare crash that was extremely difficult bug to track down because the crash location is often know where near the offending function.
I remember getting into a heated argument with a coworker when I claimed it should have been a compilation error. IIRC, he claimed it to be a halting problem and that the compiler couldn't determine that all paths didnt return a value. I called BS, citing at the time (circa 2004) that the new compiler on the block for C# could reliably emit errors when not all return paths returned a value.
I also like it that in C++, it's a very rare example of a very terse example dealing with a number of topics such as undefined/implementation defined behavior, debugging, compilation settings (warning levels, etc) all in a mere 4 lines of codes. With 4 LOC, which is straight forward and simple for the candidate to mentally parse, I can gleam a lot about their understanding of the language (and it's potential pitfalls).
Sorry if this got a little long winded and ranting.
Of course you can decide the type a function returns, to overkill this problem, just run any type inference algorithm powered by unification (e.g. hindley milner). Give undeclared variables the type NULL and you're all set since you can decide whether the function returns type NULL.
Your friend was confused because you cannot decide whether a certain line in your program will be executed. Since this is equivalent to the halting problem. Let's prove this! Assume we have a blackbox B(P,x,N) decides whether line N of program P will be executed given input x. Here, we can solve the halting problem:
```pseudocode
def Helper(sourcecode P, input x):
[0] P(x)
[1] print "What Do We Say to the God of Death?"
def Halts(sourcecode P, input x):
if B(Helper,(P,x),1):
return true
else:
return false
```
This means, given programs like your interview question, compiler cannot decide -- in general -- whether the program will crash or work. Of course, it's not too hard to find "good enough" heuristics that'll catch some cases and/or restricting your language to make it "harder" to encode such programs.
> Is the function well formed (Yes - functions need not return a value on all paths due to C ancestory, even if the return type has a non trivial actor - not sure if this is still considered well formed, but I think it was as least to C++11)
In C falling off the end of a function without returning is legal, as long as you don't use the return value. In C++ this is illegal.
> he claimed it to be a halting problem and that the compiler couldn't determine that all paths didnt return a value.
Theoretically we can't determine whether a function will return a value or not. In practice, heuristics get 99% of the way and the last 1% you can make the programmer put in a possibly redundant return statement.
There are two different questions: "do all paths lead to returning a value?" and "will the function return a value?"
Answering the second in the general case is equivalent to solving the halting problem. But answering the first question is much simpler. Static analyzers aren't using a heuristic approach to the second question, they're solving a completely different problem.
That's due to no shortage of effort by the maintainers, however: one small bug can easily lead to memory safety problems.
Rust is great in this sense because you don't have to think about those issues.
That said, I'd pick rust over C for most projects simply because it's a nicer language to write IMHO. It feels like a modern high-level language with things like iterators, traits, etc, and it has a great package manager too. In comparison, I find writing C to be a huge chore, especially if you're doing much string processing
Very importantly, Rust's safety guarantees help with "fearless concurrency", and better multithreading accounts for most of the performance gain we see in these new tools.
You can have fearless concurrency in C too if you use async queues and handover data completely without sharing references to data between threads (other than via the queues) - including using thread safe functions.
Rust will not help you with other multi-threadding issues, that arise from how OS implements processes/threads/signals/syscalls etc.
So I wouldn't call it fearless. Concurrent data access is just one issue you have to tak care around.
Well, they certainly have for me. As an example, GNU awk has some rather easy to the trigger segfaults which do not seem like they are going to be fixed soon.
Because it's a low level system's programming language with Haskell influences (makes it good for modelling a well-defined problem domain) with the "rewrite it in Rust" meme behind it. It's meant to be a replacement for C, so it makes sense that rustaceans would want to write replacements for programs written in C.
I think its worth mentioning that rust has the gold standard argument parsing library now:
https://clap.rs
That library is used by most(?) rust cli tools, so they all have a similar feel. In addition, there is a library built on top of clap called struct-opt that makes bringing up a new cli exceedingly easy:
https://docs.rs/structopt/0.2.15/structopt/
Seriously, checkout struct-opts page even if you don't know any rust. I think their demonstration on the front page summarizes a lot of things I like about idiomatic rust.
I haven't needed it in a long time, but I still think trollop for ruby is magical. It seems it's been renamed to optimist now: http://manageiq.github.io/optimist/
I'd say Rust is a productive language for providing competitive CLIs.
- Compiled so easy to distribute compared to Python, JS, etc
- Cargo makes it easy to pull in libraries compared to C/C++
- Smart people made some great libraries that make it trivial to get performance and/or a nice UX. e.g. Everything in ripgrep is available for reuse, down to how it walks the filesystem.
Even though of course nothing beats the power of a set of standardized and ubiquitous tools that you can always count on whenever you find yourself on a new machine, I must admit that I'm favorably inclined toward this trend (or what I perceive as one) of rewriting classic CLI tools with sometimes radical performance improvement promises, but most importantly (for me at least) better UIs: `grep` has always been acceptable for me, but I prefer `ag` these days, and I recently discovered `fd`, which seems like it will replace `find`, as I could never remember its weird syntax, even for the most basic cases.
I remember posting a bug to WSL about supposedly incorrect behavior with find, only to realize that I had incorrect syntax. I forgot to include the escape on the terminating semicolon (also, conveniently never shown the man pages), or maybe it was the enclosing parens around an or for name pattern matching.
Anyways, Id been away from *nix for a few years at the time, so I was a bit rusty and soon felt like a moron and closed the bug once I realized my error.
Never managed to switch to ripgrep over ag. Performance difference is not that much of a dealbreaker in practice, and rg is not totally backwards-compatible to ag. I don't exactly remember, what was the problem. I guess it has this annoying practice to ignore everything that's in .gitignore, while for ag I could set up an additional .agignore per-project.
For ripgrep, if you want to use PCRE2 regexes (assuming that's what you meant by backwards compatible), then the -P flag will do that.
Otherwise, ripgrep's support for .gitignore should be substantially better than what's in ag. You can use .rgignore just like .agignore. Both ripgrep and ag also support .ignore. These can override whatever is in your .gitignore.
> Performance difference is not that much of a dealbreaker in practice
This is true for smallish inputs. But the difference should become larger for bigger inputs. Hell, sometimes ag just refuses to search inputs that are too big. e.g., For a ~9.5GB file:
$ time rg '\w+\s+Sherlock Holmes' OpenSubtitles2016.raw.en -c
3003
real 1.608
user 1.105
sys 0.496
maxmem 9473 MB
faults 0
$ time ag '\w+\s+Sherlock Holmes' OpenSubtitles2016.raw.en -c
ERR: Skipping OpenSubtitles2016.raw.en: pcre_exec() can't handle files larger than 2147483647 bytes.
real 0.016
user 0.004
sys 0.011
maxmem 6 MB
faults 0
What I mean, it doesn't seem to be possible to make it forget about .gitignore files completely. For me, it doesn't make any sense for my grep tool to pay any attention to .gitignore files (let alone info/exclude and such), these are 2 completely separate tools. I might occasionally want for some project to put into .rgignore the same line as I put in .gitignore, but that's an explicit choice. Actually, most of the time I have a few global ignores (which is possible to do with ripgrep) and an occasional .agignore file, while pretty much every git directory has .gitignore of some sorts. This is not something I can get around with, since while .agignore (.rgignore) is totally personal, .gitignore affects everyone who works on the project and serves explicitly to indicate that you ought not commit these files.
If I put "--no-ignore" in my .ripgreprc, it skips both .gitignore and .rgignore files, which isn't obviously what I'm trying to achieve.
One more thing (even thought it it possible to get around it by writing my own bash function that would invoke rg) is that unlike the ag, rg doesn't seem have "--pager" option, which I always use heavily, by setting half a dozen of parameters for `less` when grepping.
So these are what I consider a deal breakers and reasons why I didn't switch to rg: while it's true that in some cases I would appreciate a better performance, it's extremely rare that I grep something over a 9.5GB file. And conveniently search for a substring in a project is something that I'm in need multiple times a day.
BTW, noticed one more minor quirk right now: export RIPGREP_CONFIG_PATH="~/.ripgreprc" doesn't work (but RIPGREP_CONFIG_PATH="/home/$USER/.ripgreprc" does, so it's probably almost never an issue).
You want --no-ignore-vcs. There are several other --no-ignore-* flags. ripgrep is a lot more flexible than ag there. And its implementation of the gitignore matching rules should work a lot better than ag. (Search ag's issue tracker to see just how many bugs there are. I'd be surprised if you weren't hitting any of them!)
As for a pager, that's pretty easy to solve: `rg foo | less -F`. Add `-R` to make colors work.
You don't need to search a 9.5GB file to make ag upset. All it takes is ~2GB. A key advantage of ripgrep over ag is that you don't have to just use it for code searching. It actually works just as well any place you'd use standard grep. ag just doesn't have an implementation that's good enough to be that robust.
Yeah, that's what I meant by writing my own wrapper, since I wouldn't want to type that in every time I search for something. And since I use bash, $@ in the middle of an alias is not possible, so: rg() { /usr/bin/rg $@ | less -LFXRqn }
It doesn't seem to be working as expected for me without explicitly adding --pretty though (obviously, rg sees it doesn't operate in a terminal and tries to be smart). End even then it's... weird. I don't know if that's `less` issue of `rg` issue, I'm guessing something about it being multi-threaded: it sometimes doesn't display some found items at the top of the terminal, but if I scroll screen down and up again -- it's there! Cannot reproduce with ag. Gonna play with this for some more time later, but it surely doesn't work as solidly with a pager, as ag --pager='whatever' does.
If it's convenient and you could try it in a different environment (perhaps a different terminal emulator or a different shell), then that would be great.
Also, this doesn't sound like a multi-threading issue to me, but if you want to rule that out, then `-j1` will force ripgrep into single threaded mode.
Is there a way to get the results inline? e.g. first filename, then line number then the matched string? I don't want the filename, newline, matches syntax.
Show results with every match on its own line, including line numbers and
column numbers. With this option, a line with more than one match will be
printed more than once.
> While sed does a whole lot more, sd focuses on doing just one thing and doing it well.
I'm sure you already know this, but sed is meant to provide the features that ed does, so sd is quite a bit more limited. Still, I can see how this can be useful, as most invocations of sed end up only really using a handful of its features.
I don't know if it's because I'm so used to find, but I had a bad UX with fd. This is my thought process each time I use it: "Why separating the extension from the filename? Is this a default? Can we change it? What is the option, -e? Or -e is the option to 'exec'? What's the flag to include hidden files? Or are they already included? Does the pattern comes before the directory? Does it need a directory or does it search it the current one? man fd, man fd, man fd."
I guess my brain just refuses to remember these for some reason.
Right, there's always going to be some additional cognitive overhead with every new tool you adopt. The question is how much you value its benefits for your use case, and is it worth those costs. If you're so used to find, and it works great for you, then keep using it! It's indeed more powerful and standard than fd, which I think is well-suited for vast majority of simple use cases (casual use).
However, here's my reasoning for the first of your questions, as a happy fd user who switched from GNU find. Maybe it can give you insight into why the UX is designed as it is.
The extension is distinguished from the filename because it's a common use case (-e jpg less keystrokes than -iname '*.jpg'), it's semantically separate from the filename (IMO) and therefore should not be treated as part of it. And unlike filenames, extensions most often do not need to be pattern matched. If you want to match multiple extensions, just repeat the -e flag e.g. -e jpg -e png.
If you feel that there is anything we could to to improve fd's UX, it would be great if you could share it on GitHub: https://github.com/sharkdp/fd
> Why separating the extension from the filename? Is this a default
It does not need to be! You can just also just use "fd README.md". Admittedly, the "." will actually search for any character (regex), so if you want to be precise, you need to escape it.
wow this is amazing... I love the file icons and it's beautiful... ls --tree ran almost instantly on a laravel folder which has a huge number of packages....
Every time I see a project using icon font sets (this particular project recommends Nerd Font [1]), I feel something is very wrong. We already have a sort of way (or two) to represent an image in the terminal, but icon fonts are a different beast because it essentially supports a vector representation. Code point collision is already happening (Nerd Font is already coping with this [2]), and in a long run this seems unsustainable. So... is there any way or proposal to encode a vector image in the terminal? (And no, iTerm doesn't seem to support SVG in its inline image support [3].)
Unfortunately, the Unix terminal is a gigantic pile of hacks accumulated since the late 1960s. It's a miracle the software stack manages to stay upright at all; adding features and getting the entire ecosystem to support them would be tough. Introducing the concept of an external resource that has to be fetched doesn't seem impossible to me, but there are a lot of issues that would have to be thought through.
Anyway, one nice thing about TrueType/OpenType fonts is that the vector representation is extremely well compressed. SVG or similar would be an efficiency loss. In this light, fonts are actually a fairly decent and practical way to represent vector symbols that will be cached and reused. Maybe a minimal practical solution might be to add an ANSI extension that requests an appropriate font for a particular area of the screen, perhaps with a URL from which a specific webfont can be fetched.
> Introducing the concept of an external resource that has to be fetched doesn't seem impossible to me, but there are a lot of issues that would have to be thought through.
I agree that an external resource is a lot of hurdles. What I had in mind while writing the reply was a stripped-down version of TrueType, with only a single `glyf` table allowed (probably with a minimal version of the `head` table as well). Judging from how icon fonts are used today, this might even be possible without a persistent font state.
>Unfortunately, the Unix terminal is a gigantic pile of hacks accumulated since the late 1960s.
Agreed, and I don't understand why most of the things still done in the Unix terminal have not transitioned to newer, less hacky environments.
I managed to replace the vast majority of the things I was using the terminal for with uses of about 400 lines of Emacs Lisp code that I wrote about 5 years ago.
A large fraction of my uses of the terminal since then have been for the purpose of my getting my Emacs environment and my data files installed on a fresh install of my OS and for recovering from those times when I inadvertently introduced a bug into my Emacs environment that my regression tests did not catch.
The 400 lines of Emacs Lisp code submits a string I typed to bash with the -c flag then asynchronously (i.e., without blocking Emacs's UI) waits for output from the bash process. I also added a way for the user to arrange for a bell to ring when the process finishes. Notably, there is no provision for the user to cause anything to be sent to the process's stdin after the process is started, which allows the 400 lines of code to use a pipe instead of a pty (which is relevant to this comment thread because if OSes didn't need to support the pty abstraction, they could be significantly simpler). In the rare case when I want to send a string to the stdin of some process I use the shell to set things up:
echo "here is the string" | some process
That whole command line becomes one argument to a call to bash. The 400 lines of code I wrote cannot be used to interact with a REPL. E.g., if you use my 400 lines of code to run "python" by itself as a command line, the python process waits for user input, which will never come because there is no way to write to the python process's stdin.
The closest analog in standard Gnu Emacs to the 400 lines I wrote is "shell mode" (shell.el) which consists of about 3 or 4 thousand lines of Emacs Lisp code. My code re-uses shell mode's code for interpreting (ignoring in my case) the escape sequences for ANSI color and for interpreting the escape sequences for "carriage control", but (like shell mode) does not handle the escape sequences for cursor addressing. IIRC the main reason I was dissatisfied with shell mode was that it contains hacky code to keep the bash process's "opinion" on what the current working directory is in sync with the shell mode buffer's "opinion" on what it is. In contrast, in my code, the buffer's "opinion" is the only one that matters (because every command line is processed by a fresh bash process on that bash process inherits its current working directory from the Emacs process when it is created).
If I ever were to start to need to issue a great many command lines on a variety of remote Unix servers, I would probably take the time to write a replacement for ssh along lines similar to what I just described.
I've been wanting to prototype the idea of an HTML-based terminal. Basically take the graphical user interface of the web and connect it to the system administration capabilities of the Unix command line. I'd probably draw inspiration from PowerShell: have programs produce some kind of data structure as output that can be rendered into HTML and displayed using stylesheets.
It would not be a trivial amount of work to build this since, to take good advantage of the UI capabilities, you'd need an entire new CLI OS - something to replace GNU Core Utilities. I haven't been sure where to start since the choice of programming language or expression language will significantly affect the overall experience.
Great so now in addition to having to support everything the terminal does (the whole pile of garbage hacks that were added since the 60s) you have this electron crap running which on its own is probably more complex than 99% of software out there. This basically took the terminal and made it a hundred times worse.
OTOH, if you do what your parent suggests and start from scratch (no terminal support) you have a platform that isn't supported by anything yet, so unless you port or reimplement everything you need you have to keep using both in parallel which sounds just cumbersome.
Add an out of band channel to ssh and your favorite ssh client/terminal emulator. Teach it that information in the out of band channel is used to render html, jupyter notebooks, or whatever. On the remote side, have your sshd set an environment variable that lets apps know the side channel is available, and then df can produce cute little pie charts or historic usage graphs. Ls can produce thumbnails if run in a directory with that kind of media and the appropriate args are used.
Honestly, it's likely that nothing special is needed on the ssh part of things as simple port forwarding and environment variables would do the trick.
No. Specifically, if the user issues the command line "cd ./foo; bar" a bash process is created that arranges for the wd to be ./foo during the execution of bar, but that cd command will have no effect on any subsequent command line (which will be evaluated by a different bash process).
If the target audience for this code were users other than just me, then it might not pay to arrange things that way, but all the motivation I personally needed to remember to treat the cd command differently while using the code I wrote was the very pleasant thought that my giving up on maintaining normal behavior of the cd command means that for my usage patterns, which again do not including using the code I wrote to interact with REPLs[1], no state need be maintained by any code whatsoever outside of the Emacs process except for the usual, expected state of the file system and any state internal to any bash processes and any of their child processes that are currently executing a command line.
Standard Gnu Emacs contains a M-x cd command, and that does modify the wd in a way that persists through the issuance of multiple command lines. (Again, I arranged for Emacs alone to keep track of the wd.)
[1]Standard Gnu Emacs contains the command M-x run-scheme, M-x run-python, etc, for interacting with a Scheme REPL, a Python REPL, etc. I do not happen to use any of them, but if I did start to use one of them, I probably would not feel tempted to re-write it the way I re-wrote shell mode (a.k.a. M-x shell) because I can think of no easy way to arrange things so that no state persists inside the REPL between command lines. What is special about interactive shells is that there is this single dinky piece of state, the wd, that if treated differently, makes the interface between the terminal or terminal-like thing and the shell process "connectionless" in the way that HTTP is connectionless: namely, after a command line finishes, the shell process and the terminal-like thing can just forget about each other.
I would imagine that this arrangement would be even more worthwhile for running shell command lines on lots of remote machines, but haven't written any code to effect it because I personally don't do that often.
ADDED. If I found myself writing "VAR=3 foo command" more than twice I would probably arrange for "VAR=3" to be permanently added the appropriate emacs variable (namely, process-environment IIRC). Basically, in my life the only environment variables that matter can be set once in a config file somewhere and forgotten.
We have some standardized emoji like . Seems like the partial 2019 solution would be to use emoji that should work in any modern font (you could make a more UNIXy-feeling font if you want everything to look less skeuomorphic and more stripped-down). This doesn't cover every programming-specific use case, but it's a nice compromise I haven't seen mentioned here.
Indeed, there is a WG2/UTC proposal to officially add Powerline symbols to Unicode [1]. The problem of this approach is that those standards are very strongly (and probably rightly) governed with a lot of crufts and limitations, for example you can't put Github symbol there because it is trademarked.
the partial 2019 solution would be to use emoji that should work in any modern font
This usually works not by having fonts define emojis (most don't) - it's just that systems reach in and render emojis from a dedicated emoji-defining font when they encounter them. So it's not like the world is full of fonts that define 'standard emojis' or funky powerline symbols.
> So... is there any way or proposal to encode a vector image in the terminal?
For raster images there are sixels, and for vector there are the tektronix codes. These are standard in xterm since forever.
Of course, they could be modernized, using real raster pixels and something like a subset of svg. I guess before standarization it would be better to have a reference implementation included in a major standard terminal.
Personally I always found using icons as defeating the whole point of using a terminal; One of the primary reasons I prefer terminal UIs over GUIs is that I don't have to decipher a bunch of strange hieroglyphs.
One major showstopper I have with this utility is that it doesn't obey my terminal color scheme and instead hardcodes some colors that probably just look good on author's terminal. [1]
Would love to see this resolved, cause it seems rather nice otherwise.
I think they were referring to the fact that when a new popular language arises, people rush to build things in that language and then act as if the language itself is sufficient to make the resulting product good instead of any inherent merits.
This project becomes pretty ridiculous when you realize you can already accomplish the same features with regular ls, if you just bother to read a bit. That means the target audience for this is very junior engineers.
Go to your dircolors file in /etc/dircolors, if it’s not there create it first with the dircolors command. Make sure your terminal is using a nerdfont patched font. Set colors in dircolors for various file types to whatever you want and append or prefix whatever icons you want to use. Done. Define granular rules for other extensions if you wish.
This has been my experience with the majority of shell toys and dev ecosystem tools that have come out in the past 20+ years; if people simply knew Unix better, they'd be able to do them all with the same tools that have been around since the 80s/early-90s. There's nothing wrong with coming up with these alternatives, but they are almost universally minor improvements over what's available by simply becoming better at Unix.
There's also nothing wrong with trying to rethink the interface to be more simple and intuitive. UX is important and can be improved (or worsened), even though we may sometimes pretend it's not so.
This is pretty and all, and there's nothing wrong with it, but...why? Do people run ls enough that they need something more than a plain ASCII list of filenames? Are peoples' directories so disorganized that this adds enough value to justify it over /bin/ls?
I probably run ls a few dozen times a day, but it's usually just to answer "what did I call that file again?" or "did I remember to copy that over?"
I can't imagine a scenario where I'd require too many extra visual cues. I use color ls, mostly because I find the optional symbols like '/' to be distracting, and differentiating files and directories via color is about all I care about.
There's overhead in becoming too attached to non-standard tools: you have to go out of your way to install them, your muscle memory needs retraining, etc.
Most importantly, tools like this break a bunch of core command line standards, like doing one thing and doing it well, and producing clean output that can be pipeline-filtered with all the other Unix standards.
One need only consider that Orthodox File Managers have been in existence for decades to realize that yes indeed people do like to work with more than plain lists of filenames.
Playing the "doing one thing" card when it comes to ls is rather risky, as ls, with its options for doing things like sorting its output, is one of the favourite examples that people point to of tools that do not adhere to this philosophy in practice.
131 comments
[ 4.3 ms ] story [ 211 ms ] thread1. People predict Rust will be more popular in the future, so they're trying to learn/master the language by practicing it.
2. People think speed and memory safety are important guarantees for unix programs.
3. People are testing whether Rust deserves the fame it has
I don't think any of the typical unix tools ever segfaulted on me in the last 15 years I use Linux.
Something as simple as `ls` is probably so small and battle tested that it's not an issue, but if you're writing an all-new, not-battle-tested tool, why wouldn't you want stronger guarantees? The new tool is being written for the features, but it's not fun to write vulnerabilities into something that should be simple and "just work."
Languages like Go and Ruby are also mostly memory safe, so those are generally fine picks here too, but every language has trade offs. In this case, the author clearly cares about performance, which Ruby does not care about.
Rust also has a built-in testing harness, which is a lot more convenient IMHO than using whatever C testing framework you might have a predisposition towards.
* if it crashes, where, when and why does it crash? (Technically since its undefined, nearly any answer here suffices, if it can be backed up. Since practically, most optimizing compilers assume UB can never happen, when you return nothing from a non-void function, the compiler will attempt to invoke the destructor of a non existent object instance (assuming non POD) and boom.)
I asked this because it was a distilled example of a real world rare crash that was extremely difficult bug to track down because the crash location is often know where near the offending function.
I remember getting into a heated argument with a coworker when I claimed it should have been a compilation error. IIRC, he claimed it to be a halting problem and that the compiler couldn't determine that all paths didnt return a value. I called BS, citing at the time (circa 2004) that the new compiler on the block for C# could reliably emit errors when not all return paths returned a value.
I also like it that in C++, it's a very rare example of a very terse example dealing with a number of topics such as undefined/implementation defined behavior, debugging, compilation settings (warning levels, etc) all in a mere 4 lines of codes. With 4 LOC, which is straight forward and simple for the candidate to mentally parse, I can gleam a lot about their understanding of the language (and it's potential pitfalls).
Sorry if this got a little long winded and ranting.
Your friend was confused because you cannot decide whether a certain line in your program will be executed. Since this is equivalent to the halting problem. Let's prove this! Assume we have a blackbox B(P,x,N) decides whether line N of program P will be executed given input x. Here, we can solve the halting problem:
```pseudocode
def Helper(sourcecode P, input x):
def Halts(sourcecode P, input x): ```This means, given programs like your interview question, compiler cannot decide -- in general -- whether the program will crash or work. Of course, it's not too hard to find "good enough" heuristics that'll catch some cases and/or restricting your language to make it "harder" to encode such programs.
In C falling off the end of a function without returning is legal, as long as you don't use the return value. In C++ this is illegal.
Theoretically we can't determine whether a function will return a value or not. In practice, heuristics get 99% of the way and the last 1% you can make the programmer put in a possibly redundant return statement.
Answering the second in the general case is equivalent to solving the halting problem. But answering the first question is much simpler. Static analyzers aren't using a heuristic approach to the second question, they're solving a completely different problem.
Rust is great in this sense because you don't have to think about those issues.
That said, I'd pick rust over C for most projects simply because it's a nicer language to write IMHO. It feels like a modern high-level language with things like iterators, traits, etc, and it has a great package manager too. In comparison, I find writing C to be a huge chore, especially if you're doing much string processing
Rust will not help you with other multi-threadding issues, that arise from how OS implements processes/threads/signals/syscalls etc.
So I wouldn't call it fearless. Concurrent data access is just one issue you have to tak care around.
That library is used by most(?) rust cli tools, so they all have a similar feel. In addition, there is a library built on top of clap called struct-opt that makes bringing up a new cli exceedingly easy: https://docs.rs/structopt/0.2.15/structopt/
Seriously, checkout struct-opts page even if you don't know any rust. I think their demonstration on the front page summarizes a lot of things I like about idiomatic rust.
- Compiled so easy to distribute compared to Python, JS, etc - Cargo makes it easy to pull in libraries compared to C/C++ - Smart people made some great libraries that make it trivial to get performance and/or a nice UX. e.g. Everything in ripgrep is available for reuse, down to how it walks the filesystem.
It’s also a lot faster, according to the benchmarks.
https://github.com/ogham/exa/pull/368
Now I feel like all my hard earned cli-fu is a lie.
Anyways, Id been away from *nix for a few years at the time, so I was a bit rusty and soon felt like a moron and closed the bug once I realized my error.
> Now I feel like all my hard earned cli-fu is a lie.
fd is also a lie. If you want a decent interface to find just use "find | grep ..."
https://github.com/BurntSushi/ripgrep
Otherwise, ripgrep's support for .gitignore should be substantially better than what's in ag. You can use .rgignore just like .agignore. Both ripgrep and ag also support .ignore. These can override whatever is in your .gitignore.
See the guide for more details: https://github.com/BurntSushi/ripgrep/blob/master/GUIDE.md#a...
> Performance difference is not that much of a dealbreaker in practice
This is true for smallish inputs. But the difference should become larger for bigger inputs. Hell, sometimes ag just refuses to search inputs that are too big. e.g., For a ~9.5GB file:
What I mean, it doesn't seem to be possible to make it forget about .gitignore files completely. For me, it doesn't make any sense for my grep tool to pay any attention to .gitignore files (let alone info/exclude and such), these are 2 completely separate tools. I might occasionally want for some project to put into .rgignore the same line as I put in .gitignore, but that's an explicit choice. Actually, most of the time I have a few global ignores (which is possible to do with ripgrep) and an occasional .agignore file, while pretty much every git directory has .gitignore of some sorts. This is not something I can get around with, since while .agignore (.rgignore) is totally personal, .gitignore affects everyone who works on the project and serves explicitly to indicate that you ought not commit these files.
If I put "--no-ignore" in my .ripgreprc, it skips both .gitignore and .rgignore files, which isn't obviously what I'm trying to achieve.
One more thing (even thought it it possible to get around it by writing my own bash function that would invoke rg) is that unlike the ag, rg doesn't seem have "--pager" option, which I always use heavily, by setting half a dozen of parameters for `less` when grepping.
So these are what I consider a deal breakers and reasons why I didn't switch to rg: while it's true that in some cases I would appreciate a better performance, it's extremely rare that I grep something over a 9.5GB file. And conveniently search for a substring in a project is something that I'm in need multiple times a day.
BTW, noticed one more minor quirk right now: export RIPGREP_CONFIG_PATH="~/.ripgreprc" doesn't work (but RIPGREP_CONFIG_PATH="/home/$USER/.ripgreprc" does, so it's probably almost never an issue).
As for a pager, that's pretty easy to solve: `rg foo | less -F`. Add `-R` to make colors work.
You don't need to search a 9.5GB file to make ag upset. All it takes is ~2GB. A key advantage of ripgrep over ag is that you don't have to just use it for code searching. It actually works just as well any place you'd use standard grep. ag just doesn't have an implementation that's good enough to be that robust.
Oh, thanks, I've missed it somehow.
> rg foo | less -F
Yeah, that's what I meant by writing my own wrapper, since I wouldn't want to type that in every time I search for something. And since I use bash, $@ in the middle of an alias is not possible, so: rg() { /usr/bin/rg $@ | less -LFXRqn }
It doesn't seem to be working as expected for me without explicitly adding --pretty though (obviously, rg sees it doesn't operate in a terminal and tries to be smart). End even then it's... weird. I don't know if that's `less` issue of `rg` issue, I'm guessing something about it being multi-threaded: it sometimes doesn't display some found items at the top of the terminal, but if I scroll screen down and up again -- it's there! Cannot reproduce with ag. Gonna play with this for some more time later, but it surely doesn't work as solidly with a pager, as ag --pager='whatever' does.
EDIT: actually found a bug report that seems to describe exactly the same problem. https://github.com/BurntSushi/ripgrep/issues/513
If you're willing, I'd definitely appreciate it if you could update #513 with your environment. In particular, by answering these questions: https://github.com/BurntSushi/ripgrep/issues/513#issuecommen...
If it's convenient and you could try it in a different environment (perhaps a different terminal emulator or a different shell), then that would be great.
Also, this doesn't sound like a multi-threading issue to me, but if you want to rule that out, then `-j1` will force ripgrep into single threaded mode.
Show results with every match on its own line, including line numbers and column numbers. With this option, a line with more than one match will be printed more than once.
I'm sure you already know this, but sed is meant to provide the features that ed does, so sd is quite a bit more limited. Still, I can see how this can be useful, as most invocations of sed end up only really using a handful of its features.
Although personally, I've only really used sed for find & replace so sd solved my gripes with sed perfectly.
I guess my brain just refuses to remember these for some reason.
However, here's my reasoning for the first of your questions, as a happy fd user who switched from GNU find. Maybe it can give you insight into why the UX is designed as it is.
The extension is distinguished from the filename because it's a common use case (-e jpg less keystrokes than -iname '*.jpg'), it's semantically separate from the filename (IMO) and therefore should not be treated as part of it. And unlike filenames, extensions most often do not need to be pattern matched. If you want to match multiple extensions, just repeat the -e flag e.g. -e jpg -e png.
If you feel that there is anything we could to to improve fd's UX, it would be great if you could share it on GitHub: https://github.com/sharkdp/fd
> Why separating the extension from the filename? Is this a default
It does not need to be! You can just also just use "fd README.md". Admittedly, the "." will actually search for any character (regex), so if you want to be precise, you need to escape it.
1. It’s written in the author’s preferred language
[1] https://github.com/ryanoasis/nerd-fonts
[2] https://github.com/ryanoasis/nerd-fonts/wiki/Codepoint-Confl...
[3] https://www.iterm2.com/documentation-images.html
https://github.com/saitoha/libsixel
By high-resolution I mean you can display JPEGs using sixels. You can run OpenGL programs using sixels.
Anyway, one nice thing about TrueType/OpenType fonts is that the vector representation is extremely well compressed. SVG or similar would be an efficiency loss. In this light, fonts are actually a fairly decent and practical way to represent vector symbols that will be cached and reused. Maybe a minimal practical solution might be to add an ANSI extension that requests an appropriate font for a particular area of the screen, perhaps with a URL from which a specific webfont can be fetched.
I agree that an external resource is a lot of hurdles. What I had in mind while writing the reply was a stripped-down version of TrueType, with only a single `glyf` table allowed (probably with a minimal version of the `head` table as well). Judging from how icon fonts are used today, this might even be possible without a persistent font state.
Agreed, and I don't understand why most of the things still done in the Unix terminal have not transitioned to newer, less hacky environments.
I managed to replace the vast majority of the things I was using the terminal for with uses of about 400 lines of Emacs Lisp code that I wrote about 5 years ago.
A large fraction of my uses of the terminal since then have been for the purpose of my getting my Emacs environment and my data files installed on a fresh install of my OS and for recovering from those times when I inadvertently introduced a bug into my Emacs environment that my regression tests did not catch.
The 400 lines of Emacs Lisp code submits a string I typed to bash with the -c flag then asynchronously (i.e., without blocking Emacs's UI) waits for output from the bash process. I also added a way for the user to arrange for a bell to ring when the process finishes. Notably, there is no provision for the user to cause anything to be sent to the process's stdin after the process is started, which allows the 400 lines of code to use a pipe instead of a pty (which is relevant to this comment thread because if OSes didn't need to support the pty abstraction, they could be significantly simpler). In the rare case when I want to send a string to the stdin of some process I use the shell to set things up:
That whole command line becomes one argument to a call to bash. The 400 lines of code I wrote cannot be used to interact with a REPL. E.g., if you use my 400 lines of code to run "python" by itself as a command line, the python process waits for user input, which will never come because there is no way to write to the python process's stdin.The closest analog in standard Gnu Emacs to the 400 lines I wrote is "shell mode" (shell.el) which consists of about 3 or 4 thousand lines of Emacs Lisp code. My code re-uses shell mode's code for interpreting (ignoring in my case) the escape sequences for ANSI color and for interpreting the escape sequences for "carriage control", but (like shell mode) does not handle the escape sequences for cursor addressing. IIRC the main reason I was dissatisfied with shell mode was that it contains hacky code to keep the bash process's "opinion" on what the current working directory is in sync with the shell mode buffer's "opinion" on what it is. In contrast, in my code, the buffer's "opinion" is the only one that matters (because every command line is processed by a fresh bash process on that bash process inherits its current working directory from the Emacs process when it is created).
If I ever were to start to need to issue a great many command lines on a variety of remote Unix servers, I would probably take the time to write a replacement for ssh along lines similar to what I just described.
It would not be a trivial amount of work to build this since, to take good advantage of the UI capabilities, you'd need an entire new CLI OS - something to replace GNU Core Utilities. I haven't been sure where to start since the choice of programming language or expression language will significantly affect the overall experience.
OTOH, if you do what your parent suggests and start from scratch (no terminal support) you have a platform that isn't supported by anything yet, so unless you port or reimplement everything you need you have to keep using both in parallel which sounds just cumbersome.
Honestly, it's likely that nothing special is needed on the ssh part of things as simple port forwarding and environment variables would do the trick.
If the target audience for this code were users other than just me, then it might not pay to arrange things that way, but all the motivation I personally needed to remember to treat the cd command differently while using the code I wrote was the very pleasant thought that my giving up on maintaining normal behavior of the cd command means that for my usage patterns, which again do not including using the code I wrote to interact with REPLs[1], no state need be maintained by any code whatsoever outside of the Emacs process except for the usual, expected state of the file system and any state internal to any bash processes and any of their child processes that are currently executing a command line.
Standard Gnu Emacs contains a M-x cd command, and that does modify the wd in a way that persists through the issuance of multiple command lines. (Again, I arranged for Emacs alone to keep track of the wd.)
[1]Standard Gnu Emacs contains the command M-x run-scheme, M-x run-python, etc, for interacting with a Scheme REPL, a Python REPL, etc. I do not happen to use any of them, but if I did start to use one of them, I probably would not feel tempted to re-write it the way I re-wrote shell mode (a.k.a. M-x shell) because I can think of no easy way to arrange things so that no state persists inside the REPL between command lines. What is special about interactive shells is that there is this single dinky piece of state, the wd, that if treated differently, makes the interface between the terminal or terminal-like thing and the shell process "connectionless" in the way that HTTP is connectionless: namely, after a command line finishes, the shell process and the terminal-like thing can just forget about each other.
I would imagine that this arrangement would be even more worthwhile for running shell command lines on lots of remote machines, but haven't written any code to effect it because I personally don't do that often.
ADDED. If I found myself writing "VAR=3 foo command" more than twice I would probably arrange for "VAR=3" to be permanently added the appropriate emacs variable (namely, process-environment IIRC). Basically, in my life the only environment variables that matter can be set once in a config file somewhere and forgotten.
[1] https://www.unicode.org/L2/L2019/19068-powerline-syms.pdf
This usually works not by having fonts define emojis (most don't) - it's just that systems reach in and render emojis from a dedicated emoji-defining font when they encounter them. So it's not like the world is full of fonts that define 'standard emojis' or funky powerline symbols.
The only more or less widely used solutions are iTerms image support (PNG images, no vector), or Sixel (xterm and some other exotic ones).
There is also ReGIS (https://en.wikipedia.org/wiki/ReGIS), which is a vector format, which might be supported by xterm (not sure).
For raster images there are sixels, and for vector there are the tektronix codes. These are standard in xterm since forever.
Of course, they could be modernized, using real raster pixels and something like a subset of svg. I guess before standarization it would be better to have a reference implementation included in a major standard terminal.
A standard xterm supports vt340 if you start it with the option "-ti vt340" which supports ReGIS as well.
[1] http://41j.com/hterm/
https://github.com/alebcay/awesome-shell
Only for rust but not a bad list I think.
https://lib.rs/command-line-utilities
Some blog post introducing cli tools.
https://darrenburns.net/posts/tools/
https://github.com/agarrharr/awesome-cli-apps
Would love to see this resolved, cause it seems rather nice otherwise.
https://github.com/Peltoche/lsd/issues/90
However it's written in our lord and savior RUST and has lots of colors so this shit is definitely "next gen."
0: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/ls...
So you're using this as an opportunity to bash an entire language ecosystem? The merits of this project have no bearing on Rust or its adoption.
I think the parent was bashing the attendant hype and the resulting reinventions but worse.
There was a blog I read about the Rust Zealot Brigade (or similar name) that would reliably descend on any HN conversation.
So please drop the dismissive patronizing tone.
It’s obviously not a matter of needing, but of aesthetics and usability, and that’s very much a valid pursuit.
How does dircolors add support to nerdfont?
How does it add different colors to user, group, permissions in the equivalent of ls -l?
Also, dircolors is not included in OS X.
I probably run ls a few dozen times a day, but it's usually just to answer "what did I call that file again?" or "did I remember to copy that over?"
I can't imagine a scenario where I'd require too many extra visual cues. I use color ls, mostly because I find the optional symbols like '/' to be distracting, and differentiating files and directories via color is about all I care about.
There's overhead in becoming too attached to non-standard tools: you have to go out of your way to install them, your muscle memory needs retraining, etc.
Most importantly, tools like this break a bunch of core command line standards, like doing one thing and doing it well, and producing clean output that can be pipeline-filtered with all the other Unix standards.
Playing the "doing one thing" card when it comes to ls is rather risky, as ls, with its options for doing things like sorting its output, is one of the favourite examples that people point to of tools that do not adhere to this philosophy in practice.
* https://news.ycombinator.com/item?id=4023265
* https://news.ycombinator.com/item?id=9673975
* https://news.ycombinator.com/item?id=8484718
... and so on.
https://the.exa.website
A modern replacement for ls, written in Rust. https://the.exa.website/
ls with coloring and icons https://github.com/Electrux/ls_extended