> OS will survive without Big tech, I'm not sure it works other way around.
Survive, maybe, but probably not thrive. Big Tech has been a major source of funding of OSS foundations, provider of infrastructure, and employer of many major contributors for decades.
I've been in big tech. They rarely use anything from OSS and usually have their own internal version long before OSS has an offering. Big tech will have no problem thriving without OSS. Their business models and execution aren't reliant on it at all. They only use it for recruiting
Linux, PyTorch, TensorFlow, literally every programming language used everywhere except perhaps MATLAB and CUDA (and even there most of the libraries are OSS, and those languages aren't FAANG property anyways)
I think there's a kernel of truth to what you're saying -- namely that big tech would be fine without oss and mostly builds out their own alternatives whenever necessary or desirable -- but it's definitely not true that BigTech "doesn't use OSS", "has internal versions", or "only use it for recruiting"
Lol at thinking that pytorch and tensorflow are oss. I mean they're definitely free but have you ever tried submitting a patch for something that doesn't matter to FB/Google? Those projects function as free bug finding programs for their respective sponsors.
They also exist so that infra/hardware is developed to support them specifically. The more people that use each, the more pressure exists for 3p infra/hw to build around them in pursuit of $$.
Meta uses its own php called hack. Nothing is shared with php oss. Google and Meta both have their forked versions of Cpp and python. Plus if they're closed source they will just buy it.
Pytorch and tensorflow are from FANG. Without OSS being a thing, they just keep using it without sharing it to the world.
Your examples are actually proving the opposite of your point :)
> Without OSS being a thing, they just keep using it without sharing it to the world.
But OSS is a thing and they did share it with the world.
> Your examples are actually proving the opposite of your point :)
How so?
Despite the fact that Big Tech doesn't need OSS, it's not true that BigTech:
1. "doesn't use OSS", or
2. "has internal versions" [of any relevant OSS], or
3. "only use it for recruiting".
Hack isn't a counter-example; it's open source. The face that TensorFlow started a google isn't relevant to my point, nor is the fact that FB took the healm of Torch from academia a while ago.
Companies engage with OSS strategically, not altruistically. In other news, grass is green :) But there are many strategic reasons for having pieces of infra that are (1) OSS, (2) not duplicated internally, and (3) have motivations other than recruiting.
I think you're misunderstanding my comment because you aren't reading it in context of the parent. Those are all points why big tech will be fine without open source software. I.E. if the entirety of the OSS community disappeared tomorrow, big tech would be minimally impacted. Many of the major OSS projects came from then to begin with and they already have massive efforts to maintain parallel versions. They open source tf, hack, and pytorch because it's good for recruiting. Not because it gets commits and reduces workloads. So if the OSS movement disappeared tomorrow, big tech would be the best positioned companies to keep chugging along.
Although even if all OSS software that we've discussed here disappeared tomorrow, I know at least Google and Facebook are fine because they don't use any of it. They have their own internal versions that are better and aren't shared with the world.
They also use it to manage the pace of innovation in the rest of the industry by open sourcing stuff that is similar but inferior to what they use internally. Hadoop and Kubernetes being prime examples.
The dotcom boom happened just fine even though open source adoption by businesses was pretty limited at the time. Big tech can afford to pay for software or build their own because they're big; they just like using FOSS because it saves them money.
Do you think that Google, the startup of decades ago would have scaled as rapidly if they had to license Solaris and Oracle? Same question for Twitter and Facebook without PHP and Ruby and MySQL and everything else they bootstrapped from?
Obviously, yes. I just pointed out the dotcom boom companies as examples of that.
Free server hardware would have helped Google, Twitter, Facebook, etc. to scale up faster too but they managed to grow just fine having to pay for servers. Their business models were so incredibly successful that they could have afforded to pay for software too without drastically lowering their growth.
The dot com boom largely happened because of OS, as it allowed the fast creation/scaling of businesses in a way that would have been impossible otherwise.
Big businesses didnt survive the dot com boom and bust because they had bigger budgets. If anything many overextended themselves (e.g. Sun) or missed the boat altogether and were obviated by FOSS (SGI, DEC, et al)
I'd argue the boom and also the bust happened for that exact reason. And that something similar is happening now with AI.
Without OSS we might have seen a more tepid period in the late 90s (smaller boom), as only those who could licenses IBM-whatever or MS-whatever webservers and DBs could make it to market. But also a correspondingly smaller bust, because only those with $$$$ would be able to make the investment, and due to the big barrier to entry likely would've won out their categories.
The fact that capabilities were open sourced confused the heck investors who'd been used to the MS/IBM world. They didn't understand how competition would work in the space. Hence terrible investment decisions that in retrospect look hilariously stupid.
Don't get me wrong: I much prefer the OSS world. But it's an instructive historical example. Today in AI, any old huckster with a naive nephew who knows how to finetuning things can come to market with something that looks magical to the folks holding the money bags, who likely can't differentiate between trivial hacks on top of open models and a deeper tech moat.
Sun cannibalized itself as Linux cut into its entire server line except for the very top end, and their middle to high tiers were showing up on eBay by firms that went belly-up, further cutting their margins. It didn't help that McNealy followed himself with Zander(ok, but far from McNealy's abilities) and then the utterly useless Schwartz.
Many of the others simply could not move fast enough after years of ossified businesses that were lucrative but becoming more limited.
Yes, investors were confused (disclosure: I was at VA Linux Systems when it IPO'd) but that's never been a barrier to funding the right thing (anymore than being smart guaranteed you'd pick a winner...if VC's were half as good as they portray themselves, you'd think their win-rate would climb into double digits).
The guys holding the moneybags could hire people to do the vetting, but they mostly pick insiders that maybe hit once or have otherwise sucked up with the appropriate fealty to Sand Hill "principles" than much else.
When folks spend more effort fine tuning a PPT pitch than an actual business plan, it eventually shows.
Really despise these hackneyed and obviously prejudiced narratives. Open source AI has been the norm all the while. The transformer architecture was by Google but published publicly and this just follows the culture that google, meta and co have been built on. Big tech is not "handing" out anything
Unfortunately it feels a little inevitable that big tech manages to corner this market.
H100 cards aren’t exactly enthusiast level gear.
On a cold hard cash side the open source model just doesn’t work in that context. Not unless a way is found to distribute training. broadly and efficiently like BOINC / torrent sort of style
29 comments
[ 1.6 ms ] story [ 70.4 ms ] threadSurvive, maybe, but probably not thrive. Big Tech has been a major source of funding of OSS foundations, provider of infrastructure, and employer of many major contributors for decades.
I think there's a kernel of truth to what you're saying -- namely that big tech would be fine without oss and mostly builds out their own alternatives whenever necessary or desirable -- but it's definitely not true that BigTech "doesn't use OSS", "has internal versions", or "only use it for recruiting"
If BSD isn't open source then I don't know what is.
> I mean they're definitely free but have you ever tried submitting a patch for something that doesn't matter to FB/Google?
Well, you can always fork.
> Those projects function as free bug finding programs for their respective sponsors.
Yes, most companies engage with OSS strategically and not purely altruistically. But also:
1. Big tech uses OSS.
2. Big tech does not have internal versions of many OSS projects.
3. Big tech uses OSS for reasons other than just recruiting.
Pytorch and tensorflow are from FANG. Without OSS being a thing, they just keep using it without sharing it to the world.
Your examples are actually proving the opposite of your point :)
But OSS is a thing and they did share it with the world.
> Your examples are actually proving the opposite of your point :)
How so?
Despite the fact that Big Tech doesn't need OSS, it's not true that BigTech:
1. "doesn't use OSS", or
2. "has internal versions" [of any relevant OSS], or
3. "only use it for recruiting".
Hack isn't a counter-example; it's open source. The face that TensorFlow started a google isn't relevant to my point, nor is the fact that FB took the healm of Torch from academia a while ago.
Companies engage with OSS strategically, not altruistically. In other news, grass is green :) But there are many strategic reasons for having pieces of infra that are (1) OSS, (2) not duplicated internally, and (3) have motivations other than recruiting.
Although even if all OSS software that we've discussed here disappeared tomorrow, I know at least Google and Facebook are fine because they don't use any of it. They have their own internal versions that are better and aren't shared with the world.
Free server hardware would have helped Google, Twitter, Facebook, etc. to scale up faster too but they managed to grow just fine having to pay for servers. Their business models were so incredibly successful that they could have afforded to pay for software too without drastically lowering their growth.
Big businesses didnt survive the dot com boom and bust because they had bigger budgets. If anything many overextended themselves (e.g. Sun) or missed the boat altogether and were obviated by FOSS (SGI, DEC, et al)
Without OSS we might have seen a more tepid period in the late 90s (smaller boom), as only those who could licenses IBM-whatever or MS-whatever webservers and DBs could make it to market. But also a correspondingly smaller bust, because only those with $$$$ would be able to make the investment, and due to the big barrier to entry likely would've won out their categories.
The fact that capabilities were open sourced confused the heck investors who'd been used to the MS/IBM world. They didn't understand how competition would work in the space. Hence terrible investment decisions that in retrospect look hilariously stupid.
Don't get me wrong: I much prefer the OSS world. But it's an instructive historical example. Today in AI, any old huckster with a naive nephew who knows how to finetuning things can come to market with something that looks magical to the folks holding the money bags, who likely can't differentiate between trivial hacks on top of open models and a deeper tech moat.
Many of the others simply could not move fast enough after years of ossified businesses that were lucrative but becoming more limited.
Yes, investors were confused (disclosure: I was at VA Linux Systems when it IPO'd) but that's never been a barrier to funding the right thing (anymore than being smart guaranteed you'd pick a winner...if VC's were half as good as they portray themselves, you'd think their win-rate would climb into double digits).
The guys holding the moneybags could hire people to do the vetting, but they mostly pick insiders that maybe hit once or have otherwise sucked up with the appropriate fealty to Sand Hill "principles" than much else.
When folks spend more effort fine tuning a PPT pitch than an actual business plan, it eventually shows.
Like, do you want big tech to control all AI tech? Meta releasing Llama was much better than the alternative.
H100 cards aren’t exactly enthusiast level gear.
On a cold hard cash side the open source model just doesn’t work in that context. Not unless a way is found to distribute training. broadly and efficiently like BOINC / torrent sort of style