More like a tiny startup from a second tier Chinese city schools tech giants both in America and China. High-Flyer has only 160 employees and only some of those are working on Deepseek. They are beating companies 1000x bigger in budget and employees. All while embracing open models and publishing and having to deal with suspicion due to previous Chinese models cheating with eval data.
It's funny to read the SV cope. They are lying! They have thousands of H100s! Because a random SV CEO said so.
Deepseek's parent company was worth a miser $200m. And it looks like they were forced to pivot because CCP limited salaries in finance while getting no money from VCs.
This might be the biggest underdog upset in tech history.
"I’m not even exaggerating. Management is worried about justifying the massive cost of GenAI org. How would they face the leadership when every single ‘leader’ of GenAI org is making more than what it cost to train DeepSeek V3 entirely, and we have dozens of such ‘leaders.’ DeepSeek R1 made things even scarier. I can’t reveal confidential info but it’ll be soon public anyways.
It should have been an engineering focused small org but since a bunch of people wanted to join the impact grab and artificially inflate hiring in the org, everyone loses."
High-Flyer is pretty damn rich actually. Someone did some calculation and it turns out they are spending ~$200m (somewhere in that range) on 150 employee compensation alone.
“DeepSeek gave the model a set of math, code, and logic questions, and set two reward functions: one for the right answer, and one for the right format that utilized a thinking process.
Moreover, the technique was a simple one: instead of trying to evaluate step-by-step (process supervision), or doing a search of all possible answers (a la AlphaGo), DeepSeek encouraged the model to try several different answers at a time and then graded them according to the two reward functions.”
Sounds like traditional (versus test-based) teaching!
This blog has taken an odd turn. It seems like he is invested in meta stock given the timing of this post and its focus on the stock market reaction to deepseek. That’s fine and I find no fault with it except that it’s hard to read the past few posts and not filter it through someone very bullish on meta no matter how the facts change.
I don’t necessarily disagree, I think he has very thorough and detailed analyses of many topics and I’ve learned a ton reading his blog over the years.
Nevertheless he’s human and every human no matter how hard they try is prone to certain biases at various times.
So he’s not invested monetarily in meta stock. But he’s clearly spent time with the higher ups at meta and settled on a particularly bullish narrative as it pertains to their future. That in and of itself makes him vested in meta being the winner because he predicted it and wants to be proven right.
The reason I bring this up is as someone who has read this blog very consistently for a long time now this is the first time I’ve sensed a shift in the tone. And this post seemed to be fired off quickly about an hour before the market opened and it seemed like he was palpably nervous about tHe situation and trying to calm people’s nerves. That in and of itself feels like it’s squarely in the bullish camp and hoping for a particular outcome.
He talks about it on his SharpTech podcast that he has access to every major tech CEO except for Apple. He’s had interviews with the CEOs of Meta, Microsoft, Netflix, Nvidia and an SVP of Google.
He’s been bullish on all of the BigTech companies except Google and he’s had to admit that he was wrong.
He has been bullish about Meta every time that their stock has dropped and even after everyone thought they were doomed after Apple introduced Ad tracking transparency and when big companies started leaving - two separate times
It’s funny when someone calls the most financially successful, connected podcaster/writer “the guy” like he is some random blogger. Substack was actually inspired by his success trying to make the infrastructure he built - well paid to have built - to enable newsletters and podcasters for writers to be more widely available.
Next up, people on HN are going to ask who is this Gruber guy with opinions on how to monetize blogging through advertising and who is he to have opinions on Markdown?
21 comments
[ 2.7 ms ] story [ 52.8 ms ] threadChina schooling Silicon Valley at their own game. First a better scrolling dopamine app and now with a more efficient LLM.
Silicon Valley is a dinosaur at this point with only themselves to blame.
It's funny to read the SV cope. They are lying! They have thousands of H100s! Because a random SV CEO said so.
Deepseek's parent company was worth a miser $200m. And it looks like they were forced to pivot because CCP limited salaries in finance while getting no money from VCs.
This might be the biggest underdog upset in tech history.
"I’m not even exaggerating. Management is worried about justifying the massive cost of GenAI org. How would they face the leadership when every single ‘leader’ of GenAI org is making more than what it cost to train DeepSeek V3 entirely, and we have dozens of such ‘leaders.’ DeepSeek R1 made things even scarier. I can’t reveal confidential info but it’ll be soon public anyways.
It should have been an engineering focused small org but since a bunch of people wanted to join the impact grab and artificially inflate hiring in the org, everyone loses."
Pay bankers less for increased innovation. Talk about a win-win knock-on.
Hopefully, it shall motivate other small orgs/academic institutions to do research in LLM++.
Moreover, the technique was a simple one: instead of trying to evaluate step-by-step (process supervision), or doing a search of all possible answers (a la AlphaGo), DeepSeek encouraged the model to try several different answers at a time and then graded them according to the two reward functions.”
Sounds like traditional (versus test-based) teaching!
Do you disagree with his analysis?
Nevertheless he’s human and every human no matter how hard they try is prone to certain biases at various times.
So he’s not invested monetarily in meta stock. But he’s clearly spent time with the higher ups at meta and settled on a particularly bullish narrative as it pertains to their future. That in and of itself makes him vested in meta being the winner because he predicted it and wants to be proven right.
The reason I bring this up is as someone who has read this blog very consistently for a long time now this is the first time I’ve sensed a shift in the tone. And this post seemed to be fired off quickly about an hour before the market opened and it seemed like he was palpably nervous about tHe situation and trying to calm people’s nerves. That in and of itself feels like it’s squarely in the bullish camp and hoping for a particular outcome.
He’s been bullish on all of the BigTech companies except Google and he’s had to admit that he was wrong.
He has been bullish about Meta every time that their stock has dropped and even after everyone thought they were doomed after Apple introduced Ad tracking transparency and when big companies started leaving - two separate times
"Is it bad? Well, kind of. Actually, it depends. On the other hand, maybe."
https://blockbuster.thoughtleader.school/p/how-ben-thompson-...