Surprised to not see more comments on this, especially given the popularity of the Anthropic/Karpathy article. What a win for OpenAI - and what a loss for Google, just 2 years after paying $2.7bn to bring Noam back into the fold. Does not bode well for Gemini long-term... Or could be a signal for how deeply they are leaning into world models.
Trade secrets? Like how to invent a trillion dollar technology and then sit on it for years while others eat your lunch with it? Like how to consistenly release inferior quality models to others despite infinite compute and engineering talent and insane profitability in your legacy businesses?
Don't think it matters in the long run to be honest. The models have no moat, they are becoming a commodity.
Besides that, Google is in a pretty good position, they're not bleeding money on AI like Anthropic/OpenAI, and they own product verticals where they can integrate it. Plus they have a mature ads-model which is what might actually drive a bit of revenue for LLMs.
I feel like the models have no moat paradigm died when a single model expanded past the memory of single GPU slices. The moat is hosting the model. Even paying a server host to run a rack of GPUs has immense upstart cost, and then you're still struggling to compete on the add-ons of the things on top of the model (prompts, validation loops, etc). You can only throw so much money at a problem.
Noam Shazeer was one of the lead authors of the seminal paper "Attention Is All You Need", which introduced the transformer architecture. (From Wikipedia)
In this case, it's not a new thing ... back in 2005 (yes 21 years ago), people talked about the achievements of Noam Shazeer at Google (and Jeff Dean and Sanjay, etc)
I always appreciated Jeff having a level head ... which this article seems to confirm:
I guess this means Google is nowhere close, to even discern
a hint of an AGI? So when Demis Hassabis says AGI...could arrive in just 3 years he has learned the best from Larry Ellison?
Some context for people who haven’t followed the full loop: Shazeer was a long-time Google researcher, joined Google in 2000, and was one of the co-authors of “Attention Is All You Need.”
He left Google in 2021 to co-found Character.AI. In 2024, Google brought him and some Character.AI researchers back via a licensing/talent deal with Character.AI (reportedly around $2.7B). He was then made a Gemini co-lead.
[Edit: note that my comment was reparented, it was originally a response to someone claiming Noam was another "Scam Altman". I don't mind the reparenting or the killing of the original subthread, but I feel like this is necessary context to understand this.]
Noam is the real deal, he was pretty legendary within old-time ('00s) Google engineering. Paul Buchheit had a story about interviewing him with the "how to write a spellchecker" question and then him coming up with something better than the state-of-the-art, then basically delivering Google's spell corrector in his first 2-week Noogler project.
Question two: Why are OpenAI spending that money taking talent from Google, who can definitely outspend them for talent, and not Anthropic, who are leading the market and are at least somewhat financially constrained.
65 comments
[ 4.8 ms ] story [ 68.3 ms ] threadBesides that, Google is in a pretty good position, they're not bleeding money on AI like Anthropic/OpenAI, and they own product verticals where they can integrate it. Plus they have a mature ads-model which is what might actually drive a bit of revenue for LLMs.
I hope this is not accurate but I'm afraid it is: https://x.com/signulll/status/2067446889956430273
It's funny, but with the AI hires/moves it feels more like satire now.
I always appreciated Jeff having a level head ... which this article seems to confirm:
https://www.yahoo.com/news/articles/google-cracks-down-posts...
Seems like there are some insights here!
edit: it seems the post has been removed but comments are viewable.
1 liner summary:
To put it lightly, the dude was politically outspoken and held strong beliefs.
He left Google in 2021 to co-found Character.AI. In 2024, Google brought him and some Character.AI researchers back via a licensing/talent deal with Character.AI (reportedly around $2.7B). He was then made a Gemini co-lead.
Now he’s leaving Google again for OpenAI.
Exciting times!
https://www.youtube.com/watch?v=KIZt9YPAPZo&ra=m
As an outsider, I'd be really curious to understand why, given how well positioned they seem to be in the AI battle:
- huge, quasi unmatched data war chest
- huge, quasi unmatched, planet-scale infrastructure
- native AI chip design and production (TPU)
- the core ideas for what we now know as "AI" were invented there
- deepmind, enough said
- pretty much the deepest pocket of all the AI players with the possible exception of MSFT
- a massively large user base and reach to deploy AI to (Android, YT, Cloud, Search, Email, ...)
- supposedly one the best engineering culture of the valley
Why do the best people leave ?
Why do their AI product always come in 3rd place ?
Why can't they seem to take the lead, both in terms of product design or in term of raw LLM performance?
The only answer I can think of is:
- culture is completely broken
- management sucks something fierce
- company is so fat and rich no one is actually interested in winning anymore
Also, why didn't they nail him down contractually when they bought character.ai ... isn't that pretty standard with these type of superstar (re)hires?
Noam is the real deal, he was pretty legendary within old-time ('00s) Google engineering. Paul Buchheit had a story about interviewing him with the "how to write a spellchecker" question and then him coming up with something better than the state-of-the-art, then basically delivering Google's spell corrector in his first 2-week Noogler project.
https://www.youtube.com/watch?v=gilk-76W9rE&t=60
Question two: Why are OpenAI spending that money taking talent from Google, who can definitely outspend them for talent, and not Anthropic, who are leading the market and are at least somewhat financially constrained.