22 comments

[ 3.9 ms ] story [ 43.6 ms ] thread
I love the term "Higgs Bugson". Its much better than what I usually do which is just call a system haunted.
> A higgs-bugson is a bug that is reported in practice but difficult to reproduce

This was the first time I heard of "higgs-bugson". The term sounded so forced that I had to know how it differed from Heisenbug. In short, it doesn't[1].

Then why did it even exist?

The term somehow made it to the "Heisenbug"'s Wikipedia page[1], so I checked the sources. There were two and both end up at the same site: Jeff Atwood's blog post[2] quoting some StackOverflow answers to a poll-like question ("what's a programming term you coined?") because he wanted to remove lighthearted content from the site as he thought it clashed with SO's mission of educating people and advancing their skills[3].

There was a proposal on Meta StackExchange about undeleting that question with the answers, but it was refused by Jeff Atwood again because it invited "made up stuff"[4] among other reasons.

So, Wikipedia in the end, has this term in Heisenbug page because someone just blurted out something in 2010, it was copy-pasted to a blog, and then got scooped up by some news outlet. There are no other sources. Kagi doesn't find any instances of the term before it was coined on StackOverflow in 2010. For all we know, "gingerbreadboy" from England invented it.

The irony is that the term somehow made it to the literature -hence the blog post here- because someone was just having fun at StackOverflow. It obviously either sounded good, or just clicked that others started using it. StackOverflow deleted the content that actually made a small part of computer science history because it wasn't "serious".

In other words, StackOverflow cut off one of its history-making parts because it had an incomplete and simplistic view of useful. I think it might be possible to draw a line from their understanding of communities and societal dynamics to the downfall of StackOverflow after the emergence of AI[5].

[1] https://en.wikipedia.org/wiki/Heisenbug

[2] https://blog.codinghorror.com/new-programming-jargon/

[3] https://stackoverflow.blog/2010/01/04/stack-overflow-where-w...

[4] https://meta.stackexchange.com/questions/122164/can-we-un-de...

[5] https://blog.pragmaticengineer.com/stack-overflow-is-almost-...

"The normal timeout logic can take care of retransmission in the unlikely case that one is needed."

NFS can be run over TCP or UDP. Does the retransmission occur when using UDP?

With millions of LoCs, it is no surprise there are bugs.

Worse yet, the kernel runs in supervisor mode.

This kernel design is bankrupt. There's much better available, such as seL4+Genode.

Didn't know jane street did tech writeups
I'd like to highlight this:

>NFS with Kerberos

secure, simple, battle tested. no crazy architecture

works so well a bug showed up in the kernel :-)

Calling this a "Higgs-Bugson" doesn't make a lot of sense. There's nothing uncertain or difficult to reproduce about the Higgs.

The reason that it took so long to find was that the cross-section of production is very low, the decay signatures are hard to separate from the background, the specific energy scale it existed at was not well-defined, and building the LHC was (to put it mildly) difficult and expensive.

Roughly, if you'll forgive a bad analogy from a long-lapsed physicist, it was the equivalent of trying to find a very weak glow from a specific type of bug hiding at an unknown location in a huge field of corn. Except that your vision was very bad, so you had to invent a new type of previously-unimaginably excellent eyeglasses to see the thing. Also before you could even start looking you had to expend a painful amount of time and money building a flashlight so incredibly huge that it needed new types of cryogenic cooling inventing, just to stop it from melting when you switched it on.

If you had a software bug that you were almost certain was there, but you needed half of the world's GPU clusters for three years to locate and prove it, then that would be a Higgs-Bugson.

Regarding NFS, I've always loved this quote from the CTO at a hedge fund I once worked at:

"NFS is lot like heroin:

at first, it seems amazing.

But then it ruins your life"

(This is a place that did almost EVERYTHING via NFS including different applications communicating via shared files on NFS mounts. It also had the weird setup of using BOTH Linux AND Windows permissions on NFS mounts shared between user desktops [windows] an servers [linux])

anyone else feel like the linux kernel release quality has come down a bit here in the 2020s? i feel like it hasn't been this bad since the mid 90s. anecdotally in the past couple years, i've experienced a data corruption bug in xfs, wonky wifi firmware/kernel regressions, graphics artifacts and hard crashes in amdgpu. my experience with mainline releases before 2020 has been that they're rock solid. i'd doubt myself before i doubted the kernel. i say all this with a deep appreciation for everyone and the work they're doing... my intuition says that the complexity of it all is reaching a tipping point that is finally overwhelming the ages old release engineering processes.
My experience is the opposite. I run mainline starting with -rc1 on my gaming pc, its been literally flawless for me since I switched to an AMD RX 7900 XT about two years ago. Ten years ago there was a 50/50 chance -rc1 would fail to boot on at least one of my machines, I can't remember the last time that happened.

Look at all the syzbot automation on the kernel mailing list, as an example of how the process continues to improve.

IMHO the best Linux experience really is to run bleeding edge versions of everything. I wasted a great deal more time backporting patches before I started doing that, than I have spent chasing new bugs since.

Kerberos is 'fun'. I had to manage a system which used Kerberos to provide authentication between a Rubik's cube of various Windows flavours with various crypto standards, Linux machines of various versions and Java versions and apps of various maturity. It was an ever present source of weird behaviour and I had to bury myself in the innards of all these systems.

I know, not directly related to the article. Just needed to vent bitterly.

I always thought this was rather called a 'Heisenbug'?
NFS is usually only used in mixed linux/windows environments. The easiest fix is to avoid NFS and esp. Windows. NFS alone is nightmare enough, Windows is just insanity.
I think most of the places I've seen NFS used are all UNIX/Linux shops. Why do you think Windows is commonly involved?
The article is well-written and I even learned a few things. I'm glad for Nikhil's persistence troubleshooting it and fixing the bug upstream. Thanks, Nikhil!