This is just a smart move by Satya Nadella after the non-standard drama that occurred with OpenAI a few months back where it nearly imploded and then didn't.
You want both a backup for OpenAI as well as negotiating leverage if OpenAI gets too powerful and this achieves both.
I totally agree it’s also like the move where Microsoft is at least supporting Linux on their systems and cloud as not a backup but to just close you into their ecosystem . Honestly I could see Microsoft buying Huggingface.
Yes, Microsoft doesn't have to pick the sole winner in AI, but rather they could just start eating the AI ecosystem bit by bit so that they win by default. It is what large players can do. May open themselves up to some scrutiny for too many acquisitions and reducing competition though, but that is a separate issue.
"Microsoft recommends OpenAI as your default overlord. Did you know it can do everything your current AI can do, sometimes better, but always more profitably for us? [Switch now] [Ask me again in 30 seconds]"
This is how microsoft has been doing data for at least 10 years (See databricks).
Step 1: Get the industry leaders to be purchasable via Azure.
Step 2: Slowly build your own clone and start stealing user share even though your offering is still worse.
Would you mind elaborating why? I'm not super experienced in the AI world, and barely use Hugging Face. Frankly, the name makes it difficult to take it seriously.
Hugging Face is very supportive of the open source machine learning community, both in the work they do with the transformers library, as well going above and beyond in developer and community relations to build an all around great product offering and user experience. Microsoft does the opposite of all of those things and has only made GitHub worse and more unstable since acquiring them.
> Nadella [in December 2022] abruptly cut off Lee midsentence, demanding to know how OpenAI had managed to surpass the capabilities of the AI project Microsoft’s 1,500-person research team had been working on for decades. “OpenAI built this with 250 people,” Nadella said, according to Lee, who is executive vice president and head of Microsoft Research. “Why do we have Microsoft Research at all?”
> At the same time, even as the company began weaving OpenAI into the fabric of Microsoft’s products, Nadella decided not to abort Microsoft’s own research efforts in AI. During the tense exchange at the December meeting between the Microsoft CEO and Lee, other executives spoke up to defend the work of Microsoft’s researchers, including Mikhail Parakhin, who oversees Microsoft’s Bing search and Edge browser groups, Lee said. After grilling Lee in the meeting, Nadella called him privately, thanking him for the work Microsoft Research had done to understand and implement OpenAI’s work in a way that passed muster for corporate customers. Nadella said he saw Lee’s group as a “secret weapon.”
While this is entirely speculation, it's easy to imagine that there are many levels of PR magic going on here, to share a quote that on the surface feels "leaked" and "explosive" but, among investors and clients who read beyond the (very good) paywall, actually shores up a narrative that Microsoft has a capability that significantly augments OpenAI's, and allows the existence of MSR to become headline news without even needing a product release.
The Mistral deal feels like yet another step in this direction. Microsoft is not afraid of seeming "messy" in the press as long as it can control the narrative around its value-add to customers in the context of its partnerships. By contrast, the rest of FAANG's more consumer-facing positioning makes it a lot harder for them to maneuver in a similar way.
> Nadella [in December 2022] abruptly cut off Lee midsentence, demanding to know how OpenAI had managed to surpass the capabilities of the AI project Microsoft’s 1,500-person research team had been working on for decades. “OpenAI built this with 250 people,” Nadella said, according to Lee, who is executive vice president and head of Microsoft Research. “Why do we have Microsoft Research at all?”
The answer to that is till Google released the Attention is All You Need paper in 2017 there were no breakthroughs allowing models as we have now to be built, OpenAI being a small and nible team picked up on which direction the wind is blowing with LLMs and quickly brought a product to market whilst MS just did what corps do - move slowly (same for Google etc).
Microsoft research has also been not solely devoted in AI I have seen much in quantum computing and programming language research and general computer science .
I think the main move would be some type of true AGI that leads to a hard takeoff scenario, but it isn't clear we are close to that or not.
Basically something that is more than just another bump in the scorecard for GPT 5 over GPT 4. Otherwise it is still just a horse race between relatively interchangeable GPT engines.
There are no consequences for Microsoft. It owns a 49% stake in OpenAI, so the only action that OpenAI could take to hurt Microsoft would be to deliberately destroy its own value.
> I guess it has a cost, though? I presume OpenAI didn’t like this move. If that’s the case, what might be the consequences?
Until OpenAI releases GPT 5 and it blows everyone away, OpenAI's leverage is constantly decreasing as the gap between their best model and everyone else's best model decreases.
There doesn't seem to be moats right now in this industry except for pure model performance.
Maybe someone should as ChatGPT what OpenAI should do to maintain long-term leadership in this industry?
Meh, I don't think it's worth much. In a few years that'll be like claiming that so-and-so had name brand recognition for transistors. Most people don't need to care who manufactures their transistors.
Unless your market is direct to end user, end user brand name recognition doesn't matter. In the case of AI, at least so far, the primarily income won't be from end-users directly, but rather via enterprise integrations into existing tools that already have end user market share (e.g. Microsoft Office, Microsoft Windows, VS Code, Notion, etc.)
It’s not necessarily a bad thing. Most people don’t know that TSMC exists, or what Microsoft does beyond Windows and Xbox (which are a small fraction of its business).
Most people don't need to care who manufactures their transistors.
They might, in an upside-down world where the Shockley Semiconductor board tried to fire Shockley, and where the Traitorous Eight not only didn't bail out but took his side.
Brands can change quickly, but they do matter in the short term. I've witnessed customer support teams use Firefox to say they only supported Internet Explorer and government ministers who thought it was "good" that IE was the "only" web browser, and weirdly a phone company whose customer support person thought their SIM cards worked better on Android than iPhone and that their web chat wouldn't work with a Mac even though they were talking to me on a Mac at the time.
And when I was a kid, it seemed like all the teachers thought it would be a waste of time to learn MacOS because "Apple would be bankrupt soon". (Given how much all the app UIs changed, right decision for the wrong reason).
All of these examples are end-products. "AI" itself will not be. The winner in AI will be whoever permeates other products/brands most successfully, and end-user brand familiarity doesn't matter much for that. Familiarity among engineering and product leaders is what matters.
Maybe, but maybe AI will become front and center of consumer and productivity IT products and their premier brand ambassadors will be anthropomorphized AI agents. Hello Clippy, this time for real.
Sure that helps with the consumer market, but most people will use AI integrated into other products and not directly.
Those integrated AI solutions will usually be done via enterprise deals where brand name is not quite as important. It will be done by people who care about cost, reliability and ease of use.
Think of nginx's dominance in web servers even though it has no name recognition among the general population. Or Stripe's payment system.
Yes, however it's increasingly likely that the GPT in ChatGPT will not be limited to OpenAI (in the US), so I'm not sure how much ChatGPT will be worth with countless other platforms containing GPT in their names.
The thing is that there is almost no lock in in the models. So brand recognition doesn't help much as people look into the benchmarks and price sometime in the future, if not when just starting out.
If I had to pick one player who wanted to win the AI race and was willing to be ruthless to do it, I'd pick Nvidia. Computation is the excludable bottleneck, and Nvidia is the essentially the singular company who makes AI computers.
Hire Ilya, get him to hire as many of the best folks he can.
Stop selling GPUs. Hoard them. Introduce some subtle bug into the drivers that dramatically increases their rate of burn out.
Figure out some reasonable way to give attribution to original content creators, approximately solve the content ID problem of the AI age. Cut the content creators into the rev share in proportion to their data importance to the model. Make the content creators incredibly pissed off that their work is being stolen by big AI companies unfairly and encourage to them to sue the other big AI firms. Their content share multiplier increases if they get injunctions against LLM firms.
Convince politicians that the AI firms have performed an intellectual heist of epic proportions, and that they must not be allowed to even generate synthetic training data from poisoned models. With the content creators united behind you, convince congress that poisoned models must be destroyed, that even using synthetic training data from poisoned models must be illegal. Make them start over from a clean room with no copyrighted data.
don't stop just raise the margin slightly and limit the number available of the higher end chip using the proceeds to self fund building their own datacneter
do runs of cards for themselves with higher core counts and clock speed that they dont release to others.
> Make them start over from a clean room with no copyrighted data.
And when such models become popular[0], all the artists now have no job and no way to get compensation for being unable to work through no fault of their own.
I don't think that's really a winning condition. It might make you feel better about the world, but the end result is still all the artists being out of work.
[0] some models are already trained that way, although I assume you're using the word "copyrighted" in the conventional sense of "neither public domain nor an open license", as e.g. all my MIT licensed stuff is still copyrighted but it's fine to use.
In my hypothetical future, at least the people who create the content used to train the models can get "training royalties", which they aren't getting now.
There is still also money to be made in producing physical art or performances, even when AI can produce amazing digital works.
"Make them start over from a clean room with no copyrighted data." makes "the people who create the content used to train the models" the empty set.
> There is still also money to be made in producing physical art or performances, even when AI can produce amazing digital works.
Perhaps, but it may be akin to the way there is still money to be made from horse drawn carriages in city centres, even when cars displaced them over a century ago — a rare treat for special occasions, to demonstrate wealth.
Sure, though I suspect "art" is the human version of a peacock tail — the difficulty is the point, it how we signal our worth to others, cheapening it breaks that signal — which would suggest that making all forms of art easy messes with (many of) us at a deep, essentially automatic, level.
More specifically, I was responding to the idea that "compensating creators whose works are used to train the models" would actually solve anything; to use your examples, it would be as if the literal luddites were suggesting passing laws saying that "all textile machines that work like humans need to compensate the humans they displace, and also you need to make your new machines from scratch without talking to any textile workers to make sure you don't cheat", and my response would be analogous to saying "there's already machines which don't work like humans, so you're going to be out of work and have no compensation".
The Luddite movement preceded The Communist Manifesto by about 30 years. Everything's sped up since then, so I'd be surprised if we have to wait 30 years for a political shift which is to AI what Communism was to industrialisation. I'm just hoping we don't get someone analogous to Stalin or Pol Pot this time.
>If I had to pick one player who wanted to win the AI race and was willing to be ruthless to do it, I'd pick Nvidia. Computation is the excludable bottleneck, and Nvidia is the essentially the singular company who makes AI computers.
I've thought the same thing. NVIDIA getting into AI seriously is a vertical integration play and they often do that -- like NVIDIA trying to buy ARM.
If google benchmarks are to be believed, gemini 1.5 will be better than gpt and they use their own chips (Google TPU), no nvidia involved. There is also Groq. I don't see Nvidia keeping their lead and profit margins forever.
Eh, all this talk of "moats" etc. feels weird when just a few years ago it seemed like everyone was complaining they'd rearranged their corporate structure to include a fully-owned profit-making subsidiary to attract investments, and all the loud voices seemed to think a cap of x100 return on investment was so large it was unlikely to be reached.
And then OpenAI tripped and fell over a magic money printing factory, and the complaints are now in the set ["it's just a stochastic parrot", "it's so good it's a professional threat to $category", "they've lobotomised it", "they don't have a moat", "they're too expensive"].
As the saying goes, "Prediction is very difficult, especially if it’s about the future!"
MSFT needs companies like OpenAI to give Azure credits to for their valuation to continue soaring. The deferred revenue on their balance sheet from the unspent Azure credits they give as investment are worth much more to their market cap than $80B.
Uh, sorry, but this seems pretty consistent with trying to co-opt and kill open source AI competition:
> [EEE] describe its strategy for entering product categories involving widely used standards, extending those standards with proprietary capabilities, and then using those differences in order to strongly disadvantage its competitors.
> "The US tech giant will provide the 10-month-old Paris-based company with help in bringing its AI models to market. Microsoft will also take a minor stake in Mistral although the financial details have not been disclosed"
Where are the "widely used standards"? Where are the "extending the standards with proprietary capabilities"? Where is the "strongly disadvantaging competitors"?
Mistral is the most used and fine-tuned open source model by a mile, close to the standard for open models, they’ve locked them down into offering their models behind an API and in Azure. The Azure offering sets them up for be the most safe, GDPR compliant offering for enterprises in Europe, where Microsoft already has a huge reach and customer base, bolstered by Mistral being a homegrown brand.
We should just automete these comments:
MS: Oh No! Embrace Extend Extinguish !
Google: Oh No! killedbygoogle !
Meta: Oh No! So much Ads !
Apple: Oh No! Evil App Store policy !
Diversifying their AI bets definitely makes total sense. If this wasn't their strategy originally, it almost certainly became so the moment the OpenAI board fired Sam Altman.
It's easy to make simplistic judgements from the outside, but with the limited information we have, it does seem like Satya Nadella came out of this OpenAI debacle looking pretty competent.
It's hard to reconcile the fact that the Microsoft that handled the unexpected OpenAI issue so well is the same Microsoft that seems intent on literally setting fire to their flagship product! (Windows)
Don't get used to it being around. Google is probably killing them off soon:
>@searchliaison
>is the cache link in the search results gone forever?
>Hey, catching up. Yes, it's been removed. I know, it's sad. I'm sad too. It's one of our oldest features. But it was meant for helping people access pages when way back, you often couldn't depend on a page loading. These days, things have greatly improved. So, it was decided to retire it.
>Personally, I hope that maybe we'll add links to @internetarchive
from where we had the cache link before, within About This Result. It's such an amazing resource. For the information literacy goal of About The Result, I think it would also be a nice fit -- allowing people to easily see how a page changed over time. No promises. We have to talk to them, see how it all might go -- involves people well beyond me. But I think it would be nice all around.
>You're going to see cache: go away in the near future, too. But wait, I hear you ask, what about noarchive? We'll still respect that; no need to mess with it. Plus, others beyond us use it.
Is gpt4 as good in non-English uses? It's not clear to me that it would be particularly important or advantageous, but does Mistral being based in Europe and polyglot first make it interesting vs. gpt4 in some dimension?
I guess it might depend on language, but as a Spanish speaker who sometimes uses LLMs in Spanish, I'd say the gap between GPT-4 and most of the competition (Mistral included) is actually larger in Spanish than in English.
In my experience it’s not such a simple question. If you want to be able to speak in nuanced non-English and have it pick up on the intricacies, or have it respond in rich correct non-English, then it’s not the best model (Cohere recently released an aya model that I would recommend checking out if this is your use case).
If you want to be able to give basic commands and have the model reason about the logic behind your commands, gpt 4 is still the best, even in minority languages.
It's the best multilingual model out there and it's not even close.
Especially in terms of open models Mistral's are the most multilingual but outside a few handpicked ones the level of proficiency is just too poor for any real usage.
I don't disagree with you, but an open source model fine tuned for your use case, and embedded with your data is probably going to be way better at many companies uses cases than GPT4 is.
I'd say you should compare the models for your use case. Which is better depends on how much you're willing to pay, what kind of problems you need help with, speed, ease of use.
I personally didn't realize how fast other models would catch up to OpenAI.
There is a whole set of models now (and some like Meta are purposely trying to undermine OpenAI competitive advantage via open source models) and they are relatively interchangeable with nearly no lock-in.
OpenAI's main advantage is being first to market and having the strongest model (GPT 4), and maybe they can continue to run ahead faster than everyone else, but pure technical leadership is hard to maintain, especially with so many competitors entering.
I'm pretty bearish on GPT 5 being better than 4. With how neutered 4 has gotten over time, I'd be surprised if GPT 5 is able to perform better with all the same restrictions that GPT 4 has. GPT 4 is less and less willing to actually accomplish a task for you than it is to tell you how you can do it. It looks more and more like Markov chains every day.
For now. As others have said, there is no technological “moat” in this business that could prevent others from catching up.
Perhaps the best way for Open AI is to become THE established AI services company. AWS is still the leader in cloud computing space, and only has Azure competing, despite the fact that other big companies are also technologically capable of building similar products.
> AWS is still the leader in cloud computing space, and only has Azure competing, despite the fact that other big companies are also technologically capable of building similar products.
What happened to GCP? I personally switched away because of the bad experiences.. but is that happening in scale as well. I see it barely mentioned these days.
Their main advantage for now is their super clean API. Open source alternative are already on par with GPT-3.5 and 4 capabilities, they just don't have as good a package but that could change rather quickly too.
Is that true? I was running Llamas on my laptop a few days ago, and it was giving measurably worse results than ChatGPT. I think it was the uncensored 13B model, but if you got something that's on par with ChatGPT that I can run on my own hardware I'm pretty interested.
13B models probably cannot directly compare with ChatGPT 4 which maybe +1T parameters or a 5 way MoE of 200B each - or something like that. So you can not likely run a model competitive with ChatGPT locally in the near term.
> Open source alternative are already on par with GPT-3.5 and 4 capabilities
I'm not sure if this is true. With GPT-4, I can successfully ask questions in Japanese and receive responses in (mostly natural) Japanese. I have also found GPT-4 capable of understanding the semantics of prompts with Japanese and English phrases interleaved.
Out of curiosity, I tried doing the same with local models like Mistral 7B and I could never get the model to emit anything other than English. Maybe it's a difference in training data, but even then, GPT-4 has an allegedly small set of training data for non-European languages.
Sure, but it is somewhat disheartening to see GPT 4 still being the king by a clear margin after a full year, especially since it's been nerfed continuously for speed and cost effectiveness.
Not trying to be dismissive of Mistral, but I bet that's a large driving force behind the effort. Usually I'd prefer to focus on the technical aspects, but with the undeniable geopolitical impact of technology/AI, I think it necessitates a discussion.
You're going to have to give some specific reason why though.
Why this would have anything to do with antitrust is not at all obvious to me. Especially when Google has been inventing and acquiring its own generative AI technology that it is competing with.
The OAI stake/deal is already under regulatory review and generally EU is perceived as blocking most large tech mergers since the iRobot intervention.
I suspect we are going to soon see political backlash against regulation in the EU as it is becoming very clear that this is causal to their bad capital markets.
> I suspect we are going to soon see political backlash against regulation in the EU as it is becoming very clear that this is causal to their bad capital markets.
Who would have thought that human rights are bad for business..
Nothing. But you probably don't see one located in Europe, because they would need to allow strikes, there is good level of minimum wage protection in general and strong privacy laws. It is harder to stalk the toilet breaks for employees.
Not only does the official Mistral post have a whole section on the deal, it links out further to MS's official post on the deal also. It's all the same discussion
This is just another play to give out more Azure credits to anyone that can feasibly consume them. Azure credits show up as unearned revenue on their SEC filings where they state that they "expect to recognize approximately 45% of this revenue over the next 12 months and the remainder thereafter".
It's wild that you can give out gift cards that make your company's value go up so much more than the gift cards could ever cost you. It's almost like one of those financial schemes that end badly.
Just from experience in the cloud industry, Microsoft is really successful within Europe, potentially more so than in America. I think this partnership will be really successful.
It's not like Microsoft is working on "Windows AI Studio" [1], or released Orca, or Phi.
It's not like there's any talk of AI PCs with mandatory TOPs requirements for Windows 12.
Big bad Microsoft coming for your local AI, beware.
> Mistral Remove "Committing to open models" from their website
That was 5 hours ago.
Without having insider details it is hard to know why, but the coincidence of timing with the Microsoft deal is not lost on me. It could have even been a stipulation.
I have no explanation for why Microsoft has started aggressively innovating again (with the introduction of Satya) than my theory that US DoD realized the country's tool of dominance in the future will be predominantly with tech superiority instead of military power. Microsoft's new strategy of running everything on the cloud aligns with this, even if it may have been also motivated by the fact that most people now only own a battery-constrained mobile device and laptops getting smaller and thinner.
I don't see why Mistral would acquiesce. Like the other comment says, Microsoft has a lot of chips on the table for local AI. They didn't even mention DirectML, ONNX or Microsoft's other local AI frameworks - suffice to say Microsoft does care about on-device AI.
So... would Mistral deliberately sabotage their low-end models to appease Microsoft's cloud demand? I don't think so. Microsoft probably knows that letting Mistral fall behind would devalue their investment. It makes more sense to bolster the small models to increase demand for the larger ones, at least from where I'm standing.
If you're asking about Microsoft's APIs - I'd keep an eye on ONNX. It's the most ambitious, but also supports an insane amount of acceleration targets. It would be the proverbial "big guns" if vendors continued investing in more insular frameworks like Metal and CUDA.
quibble, but nvidia is the shovel maker. LLMs require a lot of vram but the underlying hardware requirements are actually fairly simple. There are already efforts in place to create inference specific ASICs.
That'll absolutely eat into Nvidia's profit margins.
Vscode is a bad example, almost all useful Plugins are closed source on purpose, the package they ship also includes modifications that aren't open either.
I don't quite understand the other comments.
The press release says Mistra Large is available first on Azure. Which means it will be available on other platforms as well ?
Also, MS's relationship with OpenAI is that they have access to OpenAI's model for use in their products as well as potentially the source code. Is there anything similar with Mistral ? I can't find any such wording anywhere.
It is the easiest to assume the worst for sure, but Microsoft is not the same as the Bill Gates era so Im gonna be a cautious optimist on this one. Lets hope it is to promote Azure, and they dont push the OpenAI route when it comes too openess. Which is closed and a big loss for the world and a disappointment
They're not the same, but they're still a for-profit company.
90's era Microsoft wasn't evil for the sake of being evil; they were evil because they felt that monopolistic practices were the easiest way to increase their share price. They have a responsibility to their shareholders to try and maximize their share price and so it's hardly unsurprising that they did the infamous Embrace Extend Extinguish, and until regulators stepped in, such practices worked pretty well.
Most companies don't get large enough to form any kind of real monopoly, so it's easy to get on a high-horse. It's also easy to act like it was just a product of "those people", but I fundamentally think that it's a natural consequence of a company that has achieved nearly-total market dominance.
I have very little faith that a multi-trillion-dollar company is going to prioritize what's best for the world. Fundamentally, I think that if they feel they can get away with it, they'll revert to monopolistic tendencies and try and increase share price.
I'm not just picking on Microsoft here either; replace them with basically any other near-monopoly in tech and my criticisms still hold.
> 90's era Microsoft wasn't evil for the sake of being evil
You are to forgiving. I don't care too much about the why behind the evil in the software business and this is probably why my rants fall under Godwin's law too often. I won't this time.
> They have a responsibility to their shareholders to try and maximize their share price
... but I was damn close.
I agree with your whole point about Microsoft. But I don't think it's the same company anymore, I like some of the recent stuff, and I trust their lack of monopoly on the web. For now.
If a hurricane kills a thousand people is it evil?
It's not really clear whether we're better off trying to treat corporations as moral actors and constantly being surprised when they aren't, or whether we should treat them as amoral quasi-natural processes no one has full control over and construct regulations around them the same way we do for earthquakes.
We should still treat corporations as moral actor (without being naive of course) because they need to be accountable for what they do. This is why you can sue a company. You can't sue an earthquake.
Some companies try to do the right things too. Bad corporate behavior should not be normalized this much.
I like the metaphor, but... a hurricane doesn't write values documents or employee handbooks. These things do inform actions in many orgs, and constrain opportunities to a degree.
I disagree with this larger idea (that others here are implying more indirectly) that companies are all the same, and like some force of nature. They are guided and there are better and worse ones.
Having said all that, I'm still an anti-capitalist to the extent that one can be one ;)
> I have very little faith that a multi-trillion-dollar company is going to prioritize what's best for the world
You don't have to have faith. Why would anyone EVER believe that? By definition, a company is just a profit making machine. People believing that making rich people richer will necessarily make the world a better place are living in a koolaid-boosted fantasy world
The ads I see on Windows desktop machines (and shenanigans with nonstandard Html extensions and browser defaults and etc...) tell me Microsoft is just as eager to leverage it's monopoly status as ever.
No doubt the company is cautious about some things now but even in these, it will push the boundaries.
Everyone who has ever bet against Microsoft trying to enclose or eliminate something it takes an interest in has been wrong. Why should this be any different?
Are you using Windows Phone running on Nokia to Bing! and decide, email with Hotmail, chat with Skype, updating your Silverlight blog with LiveWriter, navigating with Autoroute? ... nobody else is, the people who bet against all those things were right.
So wait, you're trying to interpret me saying "Microsoft tries to close up or eliminate anything it takes an interest in" to mean "Microsoft never loses"? Very different things.
You can't trust Microsoft to act in the interest of open-source projects, transparency in general, or the users of any of the things it buys. Not all of their attempts to harm ecosystems or products work, and this has very little to do with whether they win competitions with other tech giants. In fact, sometimes user distrust stemming from their long history of Embrace Extend Extinguish and other user-hostilities has played into their failures. But like most shambling behemoths of companies, their deep pockets allow them to stay the course through many failures
You said "Everyone who has ever bet against Microsoft [...] has been wrong", that's not my interpretation, that's your words. Unless you're going to argue that them spending $8Bn on Skype doesn't count as "taking an interest in" or their multiple legal attacks on Android weren't "trying to close up or eliminate" competition?
Yes, you can indeed make sentences mean different things by omitting entire important clauses from them. You have clearly misread what I said. Microsoft did indeed fuck up skype, and try to attack android, and this is in fact consistent with my claim that Microsoft will try to enclose or eliminate any software they take an interest in. My point being that you can't trust microsoft with stewardship over a tool that's free or open. I get that it's embarassing to misread something, but you're not only doubling down about your misunderstanding, but trying to have a completely irrelevant argument with me based on your misinterpretation
403 comments
[ 3.2 ms ] story [ 344 ms ] threadYou want both a backup for OpenAI as well as negotiating leverage if OpenAI gets too powerful and this achieves both.
Step 1: Get the industry leaders to be purchasable via Azure. Step 2: Slowly build your own clone and start stealing user share even though your offering is still worse.
> Nadella [in December 2022] abruptly cut off Lee midsentence, demanding to know how OpenAI had managed to surpass the capabilities of the AI project Microsoft’s 1,500-person research team had been working on for decades. “OpenAI built this with 250 people,” Nadella said, according to Lee, who is executive vice president and head of Microsoft Research. “Why do we have Microsoft Research at all?”
> At the same time, even as the company began weaving OpenAI into the fabric of Microsoft’s products, Nadella decided not to abort Microsoft’s own research efforts in AI. During the tense exchange at the December meeting between the Microsoft CEO and Lee, other executives spoke up to defend the work of Microsoft’s researchers, including Mikhail Parakhin, who oversees Microsoft’s Bing search and Edge browser groups, Lee said. After grilling Lee in the meeting, Nadella called him privately, thanking him for the work Microsoft Research had done to understand and implement OpenAI’s work in a way that passed muster for corporate customers. Nadella said he saw Lee’s group as a “secret weapon.”
While this is entirely speculation, it's easy to imagine that there are many levels of PR magic going on here, to share a quote that on the surface feels "leaked" and "explosive" but, among investors and clients who read beyond the (very good) paywall, actually shores up a narrative that Microsoft has a capability that significantly augments OpenAI's, and allows the existence of MSR to become headline news without even needing a product release.
The Mistral deal feels like yet another step in this direction. Microsoft is not afraid of seeming "messy" in the press as long as it can control the narrative around its value-add to customers in the context of its partnerships. By contrast, the rest of FAANG's more consumer-facing positioning makes it a lot harder for them to maneuver in a similar way.
The answer to that is till Google released the Attention is All You Need paper in 2017 there were no breakthroughs allowing models as we have now to be built, OpenAI being a small and nible team picked up on which direction the wind is blowing with LLMs and quickly brought a product to market whilst MS just did what corps do - move slowly (same for Google etc).
Altman is out there trying to raise ridiculous sums to get away from Azure, didn't he make the first move here?
Basically something that is more than just another bump in the scorecard for GPT 5 over GPT 4. Otherwise it is still just a horse race between relatively interchangeable GPT engines.
Until OpenAI releases GPT 5 and it blows everyone away, OpenAI's leverage is constantly decreasing as the gap between their best model and everyone else's best model decreases.
There doesn't seem to be moats right now in this industry except for pure model performance.
Maybe someone should as ChatGPT what OpenAI should do to maintain long-term leadership in this industry?
They might, in an upside-down world where the Shockley Semiconductor board tried to fire Shockley, and where the Traitorous Eight not only didn't bail out but took his side.
And when I was a kid, it seemed like all the teachers thought it would be a waste of time to learn MacOS because "Apple would be bankrupt soon". (Given how much all the app UIs changed, right decision for the wrong reason).
Those integrated AI solutions will usually be done via enterprise deals where brand name is not quite as important. It will be done by people who care about cost, reliability and ease of use.
Think of nginx's dominance in web servers even though it has no name recognition among the general population. Or Stripe's payment system.
https://techcrunch.com/2024/02/15/no-gpt-trademark-for-opena...
Hard disagree. OpenAI's function calling is something no other commercial model provides, not even Gemini and Mistral Large.
Compute?
At least in the short term, it seems like the biggest wallets are going to win by default.
Hire Ilya, get him to hire as many of the best folks he can.
Stop selling GPUs. Hoard them. Introduce some subtle bug into the drivers that dramatically increases their rate of burn out.
Figure out some reasonable way to give attribution to original content creators, approximately solve the content ID problem of the AI age. Cut the content creators into the rev share in proportion to their data importance to the model. Make the content creators incredibly pissed off that their work is being stolen by big AI companies unfairly and encourage to them to sue the other big AI firms. Their content share multiplier increases if they get injunctions against LLM firms.
Convince politicians that the AI firms have performed an intellectual heist of epic proportions, and that they must not be allowed to even generate synthetic training data from poisoned models. With the content creators united behind you, convince congress that poisoned models must be destroyed, that even using synthetic training data from poisoned models must be illegal. Make them start over from a clean room with no copyrighted data.
do runs of cards for themselves with higher core counts and clock speed that they dont release to others.
And when such models become popular[0], all the artists now have no job and no way to get compensation for being unable to work through no fault of their own.
I don't think that's really a winning condition. It might make you feel better about the world, but the end result is still all the artists being out of work.
[0] some models are already trained that way, although I assume you're using the word "copyrighted" in the conventional sense of "neither public domain nor an open license", as e.g. all my MIT licensed stuff is still copyrighted but it's fine to use.
There is still also money to be made in producing physical art or performances, even when AI can produce amazing digital works.
> There is still also money to be made in producing physical art or performances, even when AI can produce amazing digital works.
Perhaps, but it may be akin to the way there is still money to be made from horse drawn carriages in city centres, even when cars displaced them over a century ago — a rare treat for special occasions, to demonstrate wealth.
Is it not the same with every leap in technology? There were professions like street lamp lighters, alarm services etc that have become redundant now?
More specifically, I was responding to the idea that "compensating creators whose works are used to train the models" would actually solve anything; to use your examples, it would be as if the literal luddites were suggesting passing laws saying that "all textile machines that work like humans need to compensate the humans they displace, and also you need to make your new machines from scratch without talking to any textile workers to make sure you don't cheat", and my response would be analogous to saying "there's already machines which don't work like humans, so you're going to be out of work and have no compensation".
The Luddite movement preceded The Communist Manifesto by about 30 years. Everything's sped up since then, so I'd be surprised if we have to wait 30 years for a political shift which is to AI what Communism was to industrialisation. I'm just hoping we don't get someone analogous to Stalin or Pol Pot this time.
I've thought the same thing. NVIDIA getting into AI seriously is a vertical integration play and they often do that -- like NVIDIA trying to buy ARM.
Well, they didn't stop selling GPU when cryptomining was going strong. Instead they continued to sell them (with a hefty markup, tho).
It's like selling shovels and picks during a gold rush.
And then OpenAI tripped and fell over a magic money printing factory, and the complaints are now in the set ["it's just a stochastic parrot", "it's so good it's a professional threat to $category", "they've lobotomised it", "they don't have a moat", "they're too expensive"].
As the saying goes, "Prediction is very difficult, especially if it’s about the future!"
OpenAI has 700 people, Microsoft has 220,000 people.
OpenAI is strong but they're still dependent on MSFT.
Microsoft will want to avoid things regulators in the current regime will go after.
This seems like a step towards both and ultimately good for developers as it seems likely to bring costs down by increasing competition.
The con is that MS attracts more attention from regulators.
> [EEE] describe its strategy for entering product categories involving widely used standards, extending those standards with proprietary capabilities, and then using those differences in order to strongly disadvantage its competitors.
But yeah, not sure it is at play here.
Where are the "widely used standards"? Where are the "extending the standards with proprietary capabilities"? Where is the "strongly disadvantaging competitors"?
This way Microsoft is less dependent on a single deal and can diversify their offering based on use cases.
Diversifying their AI bets definitely makes total sense. If this wasn't their strategy originally, it almost certainly became so the moment the OpenAI board fired Sam Altman.
It's easy to make simplistic judgements from the outside, but with the limited information we have, it does seem like Satya Nadella came out of this OpenAI debacle looking pretty competent.
It's hard to reconcile the fact that the Microsoft that handled the unexpected OpenAI issue so well is the same Microsoft that seems intent on literally setting fire to their flagship product! (Windows)
>Microsoft will also take a minor stake in the 10-month-old Paris-based company, although the financial details have not been disclosed.
Google cache did: https://webcache.googleusercontent.com/search?q=cache:https:...
>@searchliaison
>is the cache link in the search results gone forever?
>Hey, catching up. Yes, it's been removed. I know, it's sad. I'm sad too. It's one of our oldest features. But it was meant for helping people access pages when way back, you often couldn't depend on a page loading. These days, things have greatly improved. So, it was decided to retire it.
>Personally, I hope that maybe we'll add links to @internetarchive from where we had the cache link before, within About This Result. It's such an amazing resource. For the information literacy goal of About The Result, I think it would also be a nice fit -- allowing people to easily see how a page changed over time. No promises. We have to talk to them, see how it all might go -- involves people well beyond me. But I think it would be nice all around.
>As a reminder, anyone with a Search Console account can use URL Inspector to see what our crawler saw looking at their own page: https://support.google.com/webmasters/answer/9012289
>You're going to see cache: go away in the near future, too. But wait, I hear you ask, what about noarchive? We'll still respect that; no need to mess with it. Plus, others beyond us use it.
From: https://twitter.com/searchliaison/status/1753156161509916873
For example, my work virtual machine on Azure can't ever use it.
If you want to be able to give basic commands and have the model reason about the logic behind your commands, gpt 4 is still the best, even in minority languages.
Especially in terms of open models Mistral's are the most multilingual but outside a few handpicked ones the level of proficiency is just too poor for any real usage.
There is a whole set of models now (and some like Meta are purposely trying to undermine OpenAI competitive advantage via open source models) and they are relatively interchangeable with nearly no lock-in.
OpenAI's main advantage is being first to market and having the strongest model (GPT 4), and maybe they can continue to run ahead faster than everyone else, but pure technical leadership is hard to maintain, especially with so many competitors entering.
They haven't though. Gemini is vaporware and other models are not as good as GPT-4.
https://mistral.ai/news/mistral-large/
Now once OpenAI launches GPT 5 well I am sure other models won't look so good, but right now these other models are approaching GPT 4 capabilities.
Perhaps the best way for Open AI is to become THE established AI services company. AWS is still the leader in cloud computing space, and only has Azure competing, despite the fact that other big companies are also technologically capable of building similar products.
What happened to GCP? I personally switched away because of the bad experiences.. but is that happening in scale as well. I see it barely mentioned these days.
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderb...
https://paperswithcode.com/sota/sentence-completion-on-hella...
https://paperswithcode.com/sota/common-sense-reasoning-on-wi...
https://paperswithcode.com/sota/common-sense-reasoning-on-ar...
https://paperswithcode.com/sota/common-sense-reasoning-on-co...
I'm not sure if this is true. With GPT-4, I can successfully ask questions in Japanese and receive responses in (mostly natural) Japanese. I have also found GPT-4 capable of understanding the semantics of prompts with Japanese and English phrases interleaved.
Out of curiosity, I tried doing the same with local models like Mistral 7B and I could never get the model to emit anything other than English. Maybe it's a difference in training data, but even then, GPT-4 has an allegedly small set of training data for non-European languages.
Why this would have anything to do with antitrust is not at all obvious to me. Especially when Google has been inventing and acquiring its own generative AI technology that it is competing with.
I suspect we are going to soon see political backlash against regulation in the EU as it is becoming very clear that this is causal to their bad capital markets.
Who would have thought that human rights are bad for business..
https://www.ft.com/content/db7c6cfc-8ab2-4ee8-a41d-ba20b28d4...
What if users want to user other AI services?
It's Internet Explorer and Media Player all over again.
Could be Apple wallet garden charging 30% of any profit their AI API generated.
Jokes aside I wonder why Apple doesn't invest in these prominent AI startups.
It's not lack of cash for sure.
Official MS post: https://techcommunity.microsoft.com/t5/ai-machine-learning-b...
Official Mistral post: https://mistral.ai/news/mistral-large/
More discussion: https://news.ycombinator.com/item?id=39511477
It's wild that you can give out gift cards that make your company's value go up so much more than the gift cards could ever cost you. It's almost like one of those financial schemes that end badly.
[1] https://github.com/microsoft/windows-ai-studio
> Mistral Remove "Committing to open models" from their website
That was 5 hours ago.
Without having insider details it is hard to know why, but the coincidence of timing with the Microsoft deal is not lost on me. It could have even been a stipulation.
So... would Mistral deliberately sabotage their low-end models to appease Microsoft's cloud demand? I don't think so. Microsoft probably knows that letting Mistral fall behind would devalue their investment. It makes more sense to bolster the small models to increase demand for the larger ones, at least from where I'm standing.
That'll absolutely eat into Nvidia's profit margins.
Excited to see how things go from here in the open-source space.
Also, MS's relationship with OpenAI is that they have access to OpenAI's model for use in their products as well as potentially the source code. Is there anything similar with Mistral ? I can't find any such wording anywhere.
90's era Microsoft wasn't evil for the sake of being evil; they were evil because they felt that monopolistic practices were the easiest way to increase their share price. They have a responsibility to their shareholders to try and maximize their share price and so it's hardly unsurprising that they did the infamous Embrace Extend Extinguish, and until regulators stepped in, such practices worked pretty well.
Most companies don't get large enough to form any kind of real monopoly, so it's easy to get on a high-horse. It's also easy to act like it was just a product of "those people", but I fundamentally think that it's a natural consequence of a company that has achieved nearly-total market dominance.
I have very little faith that a multi-trillion-dollar company is going to prioritize what's best for the world. Fundamentally, I think that if they feel they can get away with it, they'll revert to monopolistic tendencies and try and increase share price.
I'm not just picking on Microsoft here either; replace them with basically any other near-monopoly in tech and my criticisms still hold.
You are to forgiving. I don't care too much about the why behind the evil in the software business and this is probably why my rants fall under Godwin's law too often. I won't this time.
> They have a responsibility to their shareholders to try and maximize their share price
... but I was damn close.
I agree with your whole point about Microsoft. But I don't think it's the same company anymore, I like some of the recent stuff, and I trust their lack of monopoly on the web. For now.
I think the point is that companies aren't good or evil, they are amoral non-aligned super intelligences that maximise shareholder profits.
If I am intent on killing a million people, but can't, but a company kills a million people via inadvertent faulty products, which is evil?
Some companies try to do the right things too. Bad corporate behavior should not be normalized this much.
I disagree with this larger idea (that others here are implying more indirectly) that companies are all the same, and like some force of nature. They are guided and there are better and worse ones.
Having said all that, I'm still an anti-capitalist to the extent that one can be one ;)
You don't have to have faith. Why would anyone EVER believe that? By definition, a company is just a profit making machine. People believing that making rich people richer will necessarily make the world a better place are living in a koolaid-boosted fantasy world
No doubt the company is cautious about some things now but even in these, it will push the boundaries.
You can't trust Microsoft to act in the interest of open-source projects, transparency in general, or the users of any of the things it buys. Not all of their attempts to harm ecosystems or products work, and this has very little to do with whether they win competitions with other tech giants. In fact, sometimes user distrust stemming from their long history of Embrace Extend Extinguish and other user-hostilities has played into their failures. But like most shambling behemoths of companies, their deep pockets allow them to stay the course through many failures
https://slashdot.org/story/11/03/21/2014244/Microsoft-Contin...
https://mobile.slashdot.org/story/10/10/01/1936213/Microsoft...