68 comments

[ 4.9 ms ] story [ 79.3 ms ] thread
While Clang is definitely good, GCC is quickly developing last years. Since they want to move everything into Rust, why they care about C++ parts right now?
I don't think they will stop using c++ anytime soon.
Because Firefox is huge, complexe, and porting it aller to rust is à several years long project.
Glad that decisions such as this get questioned, even if they seem like the inevitably correct decision. Sometimes engineering choices require uneasy decisions; ideally you can move forward with those decisions while keeping the code base cross compiler safe enough to switch to another compiler if needed in the future.
Monocultures like this interest me. We spent a ton of time in grade school learning about how monocultures were terrible long term strategies. And yet, it seems they are actively pursued everywhere. It is easy to see why this is a problem for biological things. Just look at how close we are to losing the current bananas most in the US know. I'm pretty sure avocados are the same.

In technology, though, we seem to embrace the monoculture for things. Languages, implementations, everything. They all seem to converge to a single one.

I think I understand why this happens in many cases, but I can't help but shake a feeling that it is off. Indeed, I agree with this blog post, but I still feel that something is off. Am I alone in that?

Everyone wants one grand unifying tool/standard/instruction set/architecture/theory/system to handle all use cases because it seems elegant to do so. (See obligatory xkcd which I won't post, but you know which one I mean.) It feels right to harmonize everything, even when it actually isn't, or it stifles innovation, or may lead to long-term technical debt.
> Monocultures like this interest me.

Just because they call it a monoculture, doesn't mean it has anything to do with social behavior per se. It's a technical choice. Might as well ask why every function isn't implemented in X languages or X styles. Using a singular tool in a build is the default. It's a surprise to me, that they still were trying to support a bunch of disparate compilers. Let someone else handle that, while you run the race of feature implementation, which is hard enough.

Your first sentence is confusing to me. Monoculture is not, necessarily, a social thing. Hence why I mentioned bananas and avocados. :)

That said, I think I agree with your overall point. With the amusing counter that most new languages do go way the hell out of their way to start implementing everything. In large to keep you in that language. Even in the web services world, it amuses me to see so many folks try and cram everything into a single service where they can.

The flip side, of course, is you do have a finite budge of energy to make things happen. It makes little sense to spend that budget solving problems you don't have. Worse, you could rathole into solving technical challenges that you may never actually have a need to solve. All the while leaving your customers dry.

So, yes, there is a tradeoff. I fully agree with that. Which is why this topic is interesting to me. All of the incentives seem to be to embrace monoculture thinking in most that you do. I can't help but think that is off in the long view, though. :(

There is also economics, where they have this scale from perfect competition over oligopoly to monopoly. What happens in reality differs very much: many competitors for farmers, few competitors for car manufacturers, no competition for government institutions.

Currently, Firefox works in a compiler oligopoly. It seems they would prefer a LLVM monopoly for better efficiency.

Btw the Linux kernel only supports GCC, so Mozilla is not alone.

This isn't really different across the industries, though; is it? Even farmers only do monocultures because of the money. Specifically, it is in their interest to only farm the crop that makes them the most money, if they can. They only diversify if they have to for survival. Again, see the banana and avocado market.

The kernel is an interesting case. It specifically used the free compiler. More, it specifically does not target running on platforms that historically required another compiler.

Though, again, going back to the original point. It is cheaper to build the monoculture. To that point, the risks associated with doing so are not the same as a biological case. Well, they somewhat are, but there is also the risk of development being too expensive not to do this. It all comes back to tradeoffs and this being a decision to make.

> In technology, though, we seem to embrace the monoculture for things. Languages, implementations, everything.

I would say there's a pretty straightfoward explanation for that: monocultures are, in general, far easier. Usually (unlike the instant case, where it wasn't an option in the first place), it's even the path of least resistance.

The alternative, diversity, true dirversity, diversity of thought is very difficult. Since it, presumably inevitably, results in conflicting viewpoints, which leads to.. conflict. There doesn't seem to be a good way of handling conflict in collaboritve environments.

Consider the (rhetorical?) question, "Can't we all just get along?"

That explains why we create monocultures a lot. I'm specifically intrigued that we typically argue for them. At the least, I would expect us to feel somewhat off about doing them, but deliberately choose to do so for cost reasons. (That make sense?)
I think I understand what you're asking, and (I hope), that's also what I'm suggesting as an answer.

That is, even though having a monoculture may feel vaguely off, for most people, conflict feels much more intensely off.

In essence, arguing for monocultures is often synonymous with arguing for conflict avoidance or "just getting along".

This makes sense. I'm not sure how I feel about it, right off.

Are you asserting this is the majority of it? Or just a large portion? Other reasons I could see: a) It is easier to try and reimplement something than to try and build a bridge between old and new. b) Personal preference convinces people their choice in an implementation detail matters. (I mean, it definitely matters at a personal level. But to the end customer? Do they even know?) c) tooling is often very monoculture focused. (Though, this really just goes back to difficulty.)

I'm asserting that conflict-avoidance is why, in general, monocultures might either actively be argued in favor of, or, more specifically, why nobody actively argues for (what I consider) the true opposite, which is dirversity of thought.

I'm not necessarily saying that such a thought process would factor in, if it's never consciously brought up.

> a) It is easier to try and reimplement something than to try and build a bridge between old and new.

I've heard this described as "more fun" rather than "easier", though usually that's because the assertion is being used as a criticism.

Still, what do you mean here, by "easier"? If the difficulty in the bridge-building is largely (or entirely) due to a inexpertness, discomfort, or (less charitably) laziness with dealing with different/diverse/old thought, my point still stands.

I don't think I meant it completely in those terms. Rather, I am pushing "easier" in a term that is more in line with "cheaper." Consider, it is cheaper to reimplement a framework into javascript than it is to port the language demonstrating the framework into the web platform. (I'm thinking of the general "reactive" methodologies here.)

That said, I agree your point still stands. I was not trying to disprove it, but give it some complimentary points. That is, to help myself understand the point in the fine print, not just at the broad level. That make sense?

> Consider, it is cheaper to reimplement a framework into javascript than it is to port the language demonstrating the framework into the web platform.

I'm not sure I follow what you mean here, though I admit I'm quite inexpert in this area.

My primary exposure/interest (especially in reference to the original topic) is that of a sysadmin who's done building and porting of mostly open source software on diverse Unix platforms and some of the integration of build tools (including compilers) into existing processes... long long ago.

> That make sense?

It does. I don't think we're really disagreeing, either, just exploring the ideas around this.

I think multiple implementations is more or less a C/C++ thing. If you look at other platforms (Ruby, Python, PHP, Haskell, Java, Go, Rust, Scala, Lua, Perl) there's basically _the_ version and maybe some ancillary implementations.

In fairness, it's hard enough to build a language people use that solves real problems. Asking someone to do it more than once, or to do it once but create specific enough documentation and tests so someone else can also do it -- and then get someone else to do it -- is practically impossible.

The difficulty, huge though it is, is secondary to the lack of utility. Even with C, for a long time there was only a single viable, mainstream compiler on any given platform. Clang being the main exception today.
> Even with C, for a long time there was only a single viable, mainstream compiler on any given platform.

I don't think this is even remotely true.

On Windows in the 90's we had Borland, Watcom, Microsoft compilers. Today there is at least MSVC and icc.

On Mac there was Think C, Metrowerks Code Warrior, and others.

For embedded systems there are multiple commercial compiler vendors, as well as the open source tools.

It's been a minute since Borland or Watcom were mainstream C compilers on Windows. Consider the year between ~2000-2015. Based on my recollection, it seems like MSVC was used at least 75% of the time, with the other compilers each having a small slice of the remaining quarter. Today, very seldom do you encounter a program that was compiled with anything other than MSVC, although with Clang that's starting to change.

I don't know too much about the history of Mac compilers. You may be right.

Back in the days, it was easier to make a competitive C compiler, because the accretion of advantages to the dominant compilers was not so large. Today, the moat is too wide and too deep. There's also no reason to build a new compiler, because nobody is going to pay you for it unless it's substantially better than the free ones, which by this point are pretty damn good.

I use many programs compiled with GCC on Windows. MinGW works.
I'm sure you can find someone who uses many programs compiled with TCC. That tells me little about the actual prevalence of programs compiled using various compilers.
I believe NumPy Windows binary is built with MinGW. NumPy is used enough that it is more than "very seldom".
Python C extensions are (supposed to be) compiled with the same compiler as the Python interpreter you are building against was, which in case of the official python.org release is MSVC, see https://wiki.python.org/moin/WindowsCompilers.
The existence of Clang (ironically the very compiler discussed in the headlined article) undermines your argument, based as it is upon there being "no reason to build a new compiler".
It may not have been true on Wintel PC's, but on embedded platforms, it's very common to there be only one C compiler.
> On Mac there was

Wasn't Apple using their own MPW C compiler?

Don't forget that JavaScript also has multiple fairly popular implementations.
Ehhhhhhhh kind of. Wake me up when node (or something like it) runs on SpiderMonkey, or Electron runs on JavaScriptCore, or you can pick Chakra on Lambda. The whole point is "there's a major version most people use, and sometimes there are competing versions that are significantly different that significantly fewer people use". That's definitely the case w/ JavaScript.
You have been able to run ChakraCore on Node for a bit now, with potentially more engines to follow. That can presumably later also have ramifications later for Electron and Lambda, both of which are built around Node.
Hey I don't disagree that options exist. But I think Chakra on Node is extremely niche and almost no one uses it (despite being possible for over 2 years now). My entire thesis is that multiple equivalent equally (more or less) popular implementations is primarily (maybe only) a C/C++ thing. There are niche languages where this is the case, but they're very niche. There are also very niche implementations of popular languages or platforms, but no one uses them.

We extol the virtues of multiple implementations because it makes the ecosystem healthier, but is that even really true? Is Python's ecosystem unhealthy because theres no full alternative to CPython? Is Java's? Is serverside JS?

I generally think we haven't thought through the implications of the virtues of multiple implementations -- mostly I think the opportunity costs are bananas. Platforms that at all approach C/C++'s progress on this issue have longstanding standards committees and corporate-backed engineering teams. It's simply infeasible and a hugely bad decision to build multiple implementations if you don't have that, which, it turns out, almost no language devs do.

Hmm, I guess you have a point there. That said, as a primarily-frontend-web-developer, multiple implementations of Javascript definitely is a thing, and I would never optimise solely for V8. (Although unfortunately (but it is unfortunate!), I see that as a trend that is starting to appear more and more. But I guess that's the point of the post as well.)
It is on the Server. But V8, SpiderMonkey, JavaScriptCore, and Chakra are all running the same code on the web, and most people target all four.
OP spends a whole paragraph describing the effort Mozilla expends to ensure devs are able to target at least Firefox in addition to Chrome:

> We have a Tech Evangelism bugzilla component for outreach to sites who use techniques that don’t translate across browsers. When new sites appear that deliberately block Firefox (whether because the launch team took the time to test with Firefox and determine the user experience wouldn’t be acceptable, or because cross-browser compatibility was an explicit non-goal), Firefox engineers go find the performance cliffs and fix them. Mozilla has a long-history of promoting the benefits of multiple implementations of the web platform; some of the old guard might remember “Works best in all browsers” campaigns and the like. If you squint properly, you can even see this promotion in the manifesto (principles 2, 5, 6, 7, and 9, by my reckoning).

I think it ends up that sites happen to mostly work in most JS engines because most sites don't do a lot of difficult stuff, and if they do they're using libraries that work pretty hard on cross-browser compatibility. I definitely don't think significant effort is spent targeting Chakra, for example. I think most FEs would laugh at the mere mention. Most I know push back against even considering Firefox compatibility.

Common Lisp has at least four high-quality implementations (Allegro, Lispworks, SBCL, Clozure), and this is a fairly large language (thousand something pages of the ANSI specification + two chapters of AMOP) that is about as popular as Haskell if not less.
The days of having multiple implementations does seem to have waned heavily. In large, it seems companies taking ownership of languages destroyed this mindset. Though, I confess I have not made an official study of the topic to back that claim up.
Do they all have a differentiator?
Allegro and Lispworks are proprietary with product support provided by the vendors, SBCL and Clozure are opensource. Allegro and Lispworks have their own IDEs and some proprietary libraries bundled in (e.g. CAPI which is probably the best option for cross-platform GUI for this language), while SBCL and Clozure both rely on SLIME and Quicklisp. SBCL produces fastest x86-64 machine code, as far as I know. Clozure has superior macOS support, as it used to be owned by Apple before they spun it off and was generally Mac-centric throughout its history.
for lua, there is reference lua, luajit, https://www.havok.com/wp-content/uploads/Havok_Brochures/Hav... and possibly some other.
Lua has a special place in my heart for several reasons, but their suggested usage is to just include the source code in your application because they don't guarantee compatibility at all between minor versions. As far as they're concerned, Lua 5.0 is completely different from Lua 5.1. It has a certain charm and I personally think it's pretty cool, but each different Lua release is like the perfect case of, "there's really only the one implementation of this, use others completely at your own risk, p.s. they almost certainly aren't fully compatible". Hell LuaJIT might be more popular than Lua 5.3, and it _definitely_ isn't compatible with it.
Smalltalk had multiple competing implementations.
For Rust and Go, they're very new languages, so multiple implementations are going to be hard to come by. Although there is gccgo. Some of those you listed though have a lot of different implementations.

Ruby has the Ruby MRI, JRuby, among others. [0] Additionally it spawned a lot of Ruby-like languages on different platforms.

Python has the standard Python (Pick 2 or 3), PyPy, MicroPython, unladen_swallow, Stackless Python, IronPython, Jython, among others [1].

PHP has had the reference implementation, Zend Engine, HipHop and HHVM (Both from Facebook), among others [2].

As another comment mentioned, Javascript has V8, SpiderMonkey, Chakra, and JavascriptCore, all with their own quirks. [3]

[0] https://github.com/cogitator/ruby-implementations/wiki/List-... [1] https://wiki.python.org/moin/PythonImplementations [2] https://en.wikipedia.org/wiki/PHP#Implementations [3] https://en.wikipedia.org/wiki/JavaScript_engine#JavaScript_e...

None of these (maaaaaybe with the exception of the JS engines) are the same as clang vs. GCC (vs. MSVC), which are mostly equivalent. There are significant compatibility differences between, for example, JRuby and YARV, or Python and PyPy. You can argue compiler extensions and so on, but I think that's out of scope. As far as spec'd C/C++, it's hard to find significant incompatibilities... actually are there any?
This article specifically references not having to deal with cross-compiler differences as a monoculture advantage. So there are _differences_, perhaps not incompatibilities.
OP is pretty light on details, but GCC and clang keep up with standards amazingly well (who knows about MSVC). I'm sure there are bugs, but what even are they? Is there a list? It's definitely not at all like the difference between Python and Jython, or even Python 2 and 3.

Or if you want an even better example, look at JS benchmarks for different kinds of loops. Every engine is different. How completely bananas is that. Differences like these are a whole other universe compared to differences in the major C++ compilers.

The post addresses this. To quote:

> A fair amount of work has been done to deal with peculiar bugs in all three compilers: you can go search the source code and/or Bugzilla to find hacks that were needed for one reason or another.

So go search the source code and/or Bugzilla. Treasures await!

So I searched [1] Bugzilla and found 51 issues, none of which are from the last 2 years. There really just isn't an issue with GCC/clang incompatibility with the standard.

The issues that are still open [2] are practically all either a dev didn't test with GCC and got away with things they shouldn't have in clang, or dealing with Apple's ancient version of GCC.

I think this "monoculture" thing is really about MSVC being not that great, which I sympathize with, and which everyone who's ever used MSVC for C or C++ has discovered. But that shouldn't be an excuse to stop using GCC, which is an excellent compiler and has been for generations.

[1]

Bat-Ulzii Luvsanbat claims to know about MSVC.

* https://blogs.msdn.microsoft.com/vcblog/2018/05/07/announcin...

Clang and GCC will generally compile standard C/C++ code just fine, but they add their own extensions to the language. I'd bet a lot of C or C++ projects assume `void` has a size of 1 in pointer arithmetic, for example. Try compiling some projects with -Wpedantic.

Now, Clang tries to implement the various GNU extensions, but there are some relatively significant differences. For example, GCC has no problem with a (conditional) goto to a label pointer even if you never took the address of a label anywhere in that function, while Clang treats that as an error, which can be a problem when using macros. Clang also treats _Pragma in macro arguments as you'd expect, while GCC puts all the pragmas before the rest of the code in the macro argument.

Those are just differences I persinally have struggled with, I'm sure there are more.

While there are differences between Clang and GCC language extensions, most of them are compatible or at least can be worked around with some relatively trivial hacks.

Compare this with trying to support MSVC, which supports none of the extensions and has a poor track record of getting even the standard stuff straight (especially in the C, not C++, compiler).

By dropping MSVC, you gain a lot of useful language extensions that work with Clang and GCC. For example you get portable SIMD, atomic operations and cache prefetching and a lot of other __builtin features which are unambiguously useful.

The alternative is to support MSVC but then have to maintain some kind of middleware layer to work around compiler incompatibilities. Since the platforms supported by MSVC are already covered by other compilers, you don't gain a lot.

Pascal thing, Lisp thing, POSIX shell thing, Awk thing, Javascript thing, Vi thing, ...
> Whereas Firefox engineering is the only “consumer” of the compiler(s), and so we should optimize for that single consumer

Seems really shortsighted view. What about distro package maintainers? Downstream derivatives? Porters to obscure platforms? Random developers/potential contributors?

It would, in effect, make gcc (and MSVC) into "obscure platforms," or at least non-tier-1.

I remember when Mozilla had trouble when they ran out of virtual memory space in the linker (see https://bugzilla.mozilla.org/show_bug.cgi?id=709193 , but this wasn't the first time that happened) and caused serious churn for the Windows build because Visual Studio was their only option. To be sure, this was exacerbated because VS was and is a proprietary product, but if you're going to bet that clang doesn't suffer some similar critical failure in the future, maybe you shouldn't bet every single tier-1 build of your core product.

The alternative is betting that none of [clang or gcc or MSVC etc] experience similar critical failures.

If your plan for MSVC causing issues is to let it be broken with MSVC –– then MSVC is already not a tier-one platform.

And if you're going to avoid failures caused by issues in MSVC, the existence of other compiler options doesn't change the work you have to do to achieve that.

Is Mozilla thinking about also using the same C++ standard library (libc++) on all platforms? Is it even feasible to use libc++ on Windows yet? IIRC, clang-cl uses the VC++ libraries.
It's weird how gcc and msvc end up in the same basket here. Mozilla say they're all about openness. gcc is a free and open source compiler, while msvc is proprietary and closed source. I can see the case for keeping gcc supported, but if you don't have to support msvc, why would Mozilla give it more attention than for example icc?
open != copyleft.

go whine about your communist utopia products losing relevence elsewhere.

What often happens with standards, is that politics rears its ugly head and they become technically compromised. Some of them become absolutely toothless! C++ is sufficiently complicated, to the point where the abstractions will leak and devils will creep in with the few details of the myriads that sneak past, despite the best efforts of standard writers. Smalltalk is small enough to be succinctly describable, but politics saddled it with a completely toothless standard that basically allowed anything.

What's needed are clear, succinct, moderately opinionated standards on clear, succinct, minimalist technologies.

C++ was first standardized in 1998, but it was created in 85. The standard is still changing, today in 2018.

It's easy enough to say what's needed, but is it even possible to have that for languages with a potential life span of half a century or more?

The Ada95 standard was covering a language far more rich than C++98 (it included all multitasking and did not rely on C standard). Its level of clarity and concision is incredible compared to C++98. Both languages were comparable at that time (same age, same usage).
It's easy enough to say what's needed, but is it even possible to have that for languages with a potential life span of half a century or more?

Back in the early 2000's Camp Smalltalk folks actually came up with a way to have a "toothy" Smalltalk standard, by using homomorphisms. The idea was that to be an official Smalltalk, you didn't have to have exactly X, Y, and Z, but you needed to have something that could map 1-to-1 to X, Y, and Z. The intention was that "standard" Smalltalks would be able to be automatically translated between one another.

One of factors why such an exercise was plausible, is the extremely minimalist nature of Smalltalk. C++ is huge. The surface area for standard-breaking problems is partly decided by the size of the language, as well as its age and the number of "vendors." We cannot stop time. It's not certain that limiting "vendors" is all good. We can also choose to minimize the size of the language.