It's from the opening lines of the closing segment of the the video you linked. Not a bad discussion of the unfolding of news and social media responses follwing the shooting of Donald Trump's ear on the campaign trail.
Why did we start talking about news as "legacy media"?
I mean, at least it has journalistic standards and some semblance of fact checking compared to social media which has given us great gaffes such as identifying the wrong Boston Marathon bomber and getting the poor guy to kill himself, wrongly identifying the Hispanic white supremacist shooter, and many many more.
When someone here says they don't like the news I assume it's because the only newsworthy topic in their mind is "Just how COOL is the new Iphone? Very!"
Twitter is well suited to deliver the newest developments on this topic.
Mainstream outlets have viewerships that compare disfavorably to those of top youtubers. That has been the case for many years now. The only reason most mainstream outlets exist is the taxpayer money train that's coming to an abrupt stop as I write this.
Mainstream news had viewerships disfavorable to sports games and sitcoms. So what? It's not entertainment. It's informative. They of course have different viewerships
The statement is true irrespective of the number of views. If you watch the news you are misinformed. You believe things that just aren't true at all. From time to time your view of the world clashes with reality, and you reject the reality and substitute it with your own. Many such cases.
I didn't know about this. But I asked it if Elon Musk and DOGE randomly firing as many government workers as they can from all federal departments might be dangerous to Americans and it was pretty honest that yeah, it could be.
They've been training the model for a while, right? It's unlikely he could have known Trump would let him rampage through the federal government when they started training it.
I find it interesting that these two descriptions of news are treated as equivalent, where I think they're almost opposing:
"raw, unfiltered news"
"real, trustworthy news"
Raw and unfiltered almost cannot be "news" (by my definition of what I go seeking for as "news"). X provides raw, unfiltered information. But real, trustworthy news almost requires filtration in order to be deemed trustworthy.
I think the theory is that community notes are a more neutral way to tag bad information, compared to whatever the new york time and fox news are doing.
People actually trying the model report that it does not say anything like this when asked the same question. Elon somehow prompted the model to bash The Information for his screenshot.
It would be satisfying if he gets called out for repeating himself next time he tries to come out and say he is scared how powerful their next model is.
Despite explicitly requesting the tetris/bejeweled hybrid to be "insanely impressive", the result was ugly and clunky. With that demo running in the background, they then segued into a hiring pitch for a new AI game studio. Consider me unimpressed.
Yes, but we have a long track record of one overpromising and underdelivering, charging money for what doesn't yet exist; while the other released stuff on a minimal website with an associated blog post for fanfare, and actually (metaphorically) turned the world upsidedown, and only charges for what they actually deliver. Yes there's hype now, but that's how it began.
So I think people are less distrustful of Altman when he says "thousands of days".
Every competitors were done when Claude 3.5 was released, every competitors were done when o1 was released, the entire West was done when DeepSeek was released, the world was done when Mistral Le Chat was released, I guess now it's time for the solar system to be done because of Grok3. Let's see what new model dominates the galaxy next week.
"Please don't post insinuations about astroturfing, shilling, bots, brigading, foreign agents and the like. It degrades discussion and is usually mistaken. If you're worried about abuse, email hn@ycombinator.com and we'll look at the data."
There's a level of truth to many of those statements.
1) Claude 3.5 prevented OpenAI from making big monopoly profits on LLM inference
2) Open source models like Mistral and Llama effectively prevented any regulator from controlling how people fine tuned models, and what they used them for
3) Deepseek prevented the collective west from exerting control over the creation of base models
Elon just said they are launching an AI game studio. Does this mean they will be building games that are mostly built with AI, or will they make AI tooling available for anyone to build games easily? Probably the former, but it would be nice if they would make it fully available to everyone.
Full self driving does not exist as it is not full self driving. In fact the name is one of the worst things about it as it gives drivers false confidence
It would be lovely if all businesses followed the mondragon model, but that isn’t the reality in which we live - corporations are fiefdoms, for the most part.
It's been said before but it is great news for consumers that there's so much competition in the LLM space. If it's hard for any one player to get daylight between them & the 2nd best alternative, hopefully that means one monopolistic firm isn't going to be sucking up all the value created by these things
I think it's already clear that these are going to be commoditized and the free / open source versions will be good enough to capture enough of the value that the remaining players will not be Facebook-level monopolies on the space
Apparently it isn't clear to the investors valuing OpenAI at >300B. Possibly they're betting that the ecosystem & integrations around their models will generate a certain amount of lock-in or otherwise make the difference in a close-to-even field
Funnily enough a lot of the open source world has landed on an API that is basically a copy of OpenAI. So if you develop against OpenAI it’s almost a slot in solution to switch to an open source solution.
I don't think it's automatically a bad idea. Offices require a lot of support, networking, security, maintenance, certifications etc. There are efficiency gains in scaling. In addition, Wework is useful for companies which hire employees in different cities.
This reminds me of a comedy sketch where a guy is interviewing for a job at a startup, finally gets to the last round and meets the founder, and he tells him the whole thing is an illusion for investors
the IP rights holders have yet to bare their teeth. I don't think the outcome you suggest is clear at all, in fact I think if anything entirely the opposite is the most probable outcome. I've lost count of the number of technology epochs that at the time were either silently or explicitly dependent on ignoring the warez aspects while being blinded by the possibilities, Internet video, music and film all went through this phase. GPTs are just a new medium, and by the end of it royalties will in all likelihood still end up being paid to roughly the same set of folk as before
I quite like the idea of a future where the AI job holocaust largely never happened because license costs ate up most of the innovation benefit. It's just the kind of regressive greed that keeps the world ticking along and wouldn't be surprised if we ended up with something very close to this
As I recall it, there was a time when copyright infringement on YouTube was so prolific that the rightsholders essentially forced creation of the first watermarking system that worked at massive scale. I do wonder if any corners of research are currently studying the attribution problem with the specific lens of licensing as its motivation
Yeah that was the old Viacom vs Youtube days. Here is a great video if you have half an hour to spare: https://www.youtube.com/watch?v=qV2h_KGno9w . Pretty funny court case where it turns out viacom was violating their OWN copyright... set a massive precedent.
But one thing this reminds me of is the idea of a "trap street", something mapmakers used to do was put in false locations on their maps to prove that other mapmakers were copying them: https://en.wikipedia.org/wiki/Trap_street . I figure you could do something similarly adversarial with AI to pollute the public training data on the internet. IDK like adversarial attacks on image classifiers https://www.youtube.com/watch?v=AOZw1tgD8dA . With an LLM you could try to make them into a manchurian candidate.
>I quite like the idea of a future where the AI job holocaust largely never happened because license costs ate up most of the innovation benefit.
Not quite realistic. You are talking about very huge benefits, in favor of which licenses will be abandoned. And who don't abandoned them... I mean you can look at the Amish settlements.
An environment where royalties inflate the pricing of ChatGPT by orders of magnitude seems like an environment where hosted models would be at a big disadvantage against whatever you can manage to get running on a pile of Macs in your garage.
If you're paying $200/month for something I can do with open source software and $10/month of compute, why wouldn't I offer you the service for $100/month? And then someone offer it for $50?
Not everyone has to know about, understand, or use open source solutions for it to open the field.
If you can do a $200/mo service for $10/mo, the closed source will reduce their prices to $15/mo and beat you
This is just a weird dichotomy you're introducing. Open source will introduce price pressure as any competition will - that doesn't mean you won't have a monopoly.
If you have virtually no pricing power and have to drop your $200/mo to $15/mo that's a big deal if your $300bn valuation is implying that not happening, which is what OP's point is about
Idk what you mean by saying this doesn't preclude a monopoly - having your pricing power eroded by competition is kinda one of the key features of what a monopolistic market isn't
Not at all. Monopolies don't imply an anti-rigid price curve. In fact, monopolies almost never have that.
A monopoly means a company has enough leverage to corner and disproportionately own the market. This is entirely possible (and usually the case) even with significant pricing pressure.
I think you're both missing a bigger picture. How many of these services can now be replicated in-house by a single developer? Which part of the service actually costs money once that dev deconstructs the process?
Feels like I won't be paying for anything that isn't real-time. And that any time delay I can introduce in my process will come with massive savings. Picture hiding the loading of loot info behind a treasure chest opening animation in a game, except that time difference means you can pull all the work in-house.
Openrouter.ai seems like a step in the right direction but I'd want to do all their calculations myself as well as factor in local/existing gear in a way they don't.
They don't seem to care about Ai either. The vast majority of people care about the value they're getting, companies care abut open source because its usually free.
I don't think we expect a company to exist solely making a proprietary web server anymore and be a behemoth of 300B. OpenAi might end up at the same model as Nginx or Docker if they don't pivot or find a different model.
That's true, but if someone sells you a one-time-purchase box/gadget/phone that will do a snapshot SOTA work and not cost you $20-$200/mo in subscriptions, a lot of people would be in.
Right now the average person has to go through a vendor with a web app, there's not a lot of room for the public to explore.
I've spent the last hour testing it and I'm blown away. And this is coming from a very hardcore user of OpenAI/Claude products on a daily basis.
It passed every goofy test I have for writing articles which involves trying to surface arcane obscure details. (it certainly means however they are scraping the Web they are doing a good job here)
It made the database code I wrote over the last week with o3/o1/GPT4o/Claude3.5 look like a joke.
It fills me with rage over who owns this thing.
Even if people tank Tesla's car business and run Twitter into the ground, I think our new Galactic Edgelord is going to win his first trillion on xAI and Teslabots anyway.
Apologies for possibly stupid question but where can you use it right now? Just on 'direct chat' on https://lmarena.ai/ or is there a better alternative? Or do you have early access?
I was using it on grok.com, logged in via a Twitter account. But I notice it just got added to the Grok tab on Twitter a moment ago.
Also, the "Deep Search" button was not available when it first went live, so I'm retesting everything again with this feature enabled, which I assume is a reasoning version of the same model.
Hopefully, you’ll be able to avoid the whole X Premium Plus thing in the near future with OpenRouter. It’ll still use xAI backend but via your OpenRouter API key. Then you can use it with any web or mobile app that supports OpenRouter.
Personally, I wouldn’t use it though. What’s going on with Elon Musk right now is completely insane. I hope to see OpenAI’s GPT-4.5 & GPT-5 releases to catch up soon, if nothing else. Announced for this year.
For whatever it is worth, I frequently enough see the devs asking for feedback, so I suspect, if you tweet about Grok, or reply to any of those threads, they definitely read them, even if they don't respond / interact. It shows. I've seen improvements based on feedback I see others make almost instantly.
>It's been said before but it is great news for consumers that there's so much competition in the LLM space.
Is it? Because it seems like a bunch of megacorps pirating every single copyrighted work available in digital format, spending an enormous amount of electricity (that is probably not 100% clean) to churn through them, and the end result we have a bunch of parrots that may or may not produce accurate results so that spammers can more effectively fill the Internet with crap.
I'm willing to bet $100 that a human consumes at least 10x times energy than a latest LLama (picking the open source model so that it can be easily verified) to produce two pages of text. All of this "AI is destroying the environment and consuming too much power" is about total consumption, both training and inference. Inference itself is cheap and green.
We can control for quality too, of you want. A lot of real life uses for Chatgpt are really trivial. I regularly ask it for some basic recipes based on my groceries and likes, quality is basically 100% hits so far.
Most people do see productivity gains from using LLMs correctly. Myself included. Just because some people don’t learn how to use them correctly doesn’t mean LLMs aren’t helpful. It’s like when internet search came out and a handful of laggards tried it once, failed to get the exact perfect result, and declared “internet search is useless”. Using a tool wrong is not evidence of the tool being useless, it’s evidence that you need to learn how to use the tool.
hallucinations are literally the finger in the dam. if these models could sense when an output is well-founded and simply say “i dont know” otherwise… say goodbye to your job
Googling a question and finding an incorrect answer every now and then doesn’t mean that googling is useless. It means that you need to learn how to use google. Trust but verify. Use it for scenarios where you aren’t looking for it to be the trusted fact checker. It excels at brainstorming, not at fact giving.
How many times do you think I've heard that over the past three decades? And you know what? They've been right every time, except for this one little fact:
The machine cannot make you give a shit about the problem space.
It's a real issue! But only for people who built the habit of typing in address bar, clicking the first stack overflow link and copy paste the first answer. Maybe break that habit first?
It's not good news when this competition comes at cost of a gigantic over inflated bubble, in which all the big players keep on sucking billions from investors without even having a business model.
This hype will burst sooner than later and will trigger yet another global recession. This is untenable.
The premise is that this ultimately replaces all intellectual and physical labor for the rest of time. It’s possible it becomes commoditized as soon as it exists, but in terms of investment dollars it’s either worth as much as you can spend or nothing at all.
> ultimately replaces all intellectual and physical labor for the rest of time
Sounds incredibly valuable, but in reality collapses into Butlerian Jihad fairly quickly when you have 90% unemployment.
Edit: if the claims are true, then this will be far more destabilizing than social media. What do elections mean when the AI-guided political parties are putting out AI press releases for the AI press summaries, which voters have read out to them through their AI? What happens when >50% of the voters ask the AI who they should vote for? You end up with an AI dictatorship where the levers of discontent are fake.
Yeah at some point it seems inevitable that if machines do all the work that creates real "value" and people have no comparable value, in a very practical sense we will all be slaves to machines
"Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them." -- Frank Herbert, _Dune_
Are we free today? For example, I have to work for a living. If I don't, my family and I will be miserable. Let's just hope that future "men with machines" don't decide to kill the rest of us - I'm not sure what use we will be to them.
I think this is the key change. We’re already beholden to a “machine” (the economy) that none of us completely understand or control or created explicitly. It has its own goals and tendencies that emerged from the complexity.
What AI and robotics does is actually create a machine that has no use for humans at all. Ñ
> Sounds incredibly valuable, but in reality collapses into Butlerian Jihad fairly quickly when you have 90% unemployment.
But nobody really knows if that happens or not as a consequence, let alone quickly, because the transition itself only happened at all so far in fiction.
Whatever does happen, I think it's going to be a surprise to at a minimum 80% of the population.
I think business model there pretty simple: to be in the front line when AI will go into the category of landscape-changing trillion dollar technologies. And investors keep pouring their billions exactly for that business model.
>This hype will burst sooner than later and will trigger yet another global recession.
It seems to small of bubble for global recession. I mean if it is a bubble at all, there is all the reasons to believe that the strategy will work with a significant probability.
See also: Meta's previous push into VR/AR/Metaverse. They spent a hundred billion to be at the front of the pack when that revolutionary world-changing paradigm-shift took off... which simply didn't happen.
That's a bit different though. META invested in a product that, as of now, as very little competition. The QUEST is sold at a slight loss but at least at an approachable price and to a volume that makes them the clear leader in the market at the moment. Moreover, their OS is open source. Clearly, what they want is to sell enough VRs to get the monopoly on the ecosystem and its apps (they basically want to make an Android play store for VRs). You may argue they are far but at least that's a clear business model.
OpenAI's business model was literally "we don't have one: we'll make AGI and we'll let AGI tell us how to make money". This is so idiotic it's not even a scam. xAI will compete on the same plane field. Not sure about Anthropic: they seem a bit more sane.
Their Orion glasses are apparently mindblowing in fidelity as well as the lightness of the glasses. Someone will absolutely make the smart glasses paradigm work so that we don't need to carry around phones anymore, and Zuck is racing to be first. This is because he lost out on the platform wars and was at the mercy of Apple and Google; remember Apple's privacy update that killed much of Meta's revenue? Zuck doesn't want a repeat of that by owning his own platform.
Seconded. I’m very excited for the day when/if their dev platform is opened up and it’s possible to access pass-through vision.
There’s a whole class of educational apps that could open up for people learning in the physical world. Whether it’s building physical things, sports or reading books or notes written in non-latin scripts... the impact will be enormous!
The only thing I'm concerned about is it'd be another locked down platform, like Oculus / Quest already is, only now much more disruptive just like Apple's and Google's (to a smaller extent). I want something more like Windows Mixed Reality or Steam VR to succeed more.
WeWork trapped themselves into a real estate hole, selling services for less than they rented property for.
OpenAI is currently in an explicit non-profit seeking mode using a technology that we have demonstrated 10-100x or greater decreases in compute to achieve the same outcomes.
This is not a declaration that OpenAI will become wildly profitable. This is just me saying that these aren't comparable companies.
WeWork was a scam to enrich the founders of the company. They owned or had interests in many of the properties that WeWork leased from. I'm surprised no one was thrown in prison.
I wonder how long it'll last. Just using myself as a demo customer: I canceled my subscription because Google AI Studio was doing more for me, and it's free. OpenAI is not really competitive at $20 a month anymore.
True, but presence of significant revenue is still promising. It's much better to have an "expensive compute" problem than a "nobody wants to pay for the product" problem.
The bandwidth costs made it deeply loss making for a long time despite having loads of engagement and ad revenue. However over time they became more cost efficient at sending video over the internet and became profitable.
This strategy obviously doesn't always work, with WeWork being the canonical example. But it's not guaranteed to fail either.
I was there when YouTube became a thing, and I was running a music video-hosting website that I had built myself (on top of phpBB, even). We were encoding videos in Windows Media and RealPlayer formats.
There were LOTS of funded competitors to YouTube between 2006 and 2009, including Viddler (who paid Gary Vaynerchuk a small fortune to host his WineLibraryTV show there exclusively), DailyMotion (which is still alive today, although no longer a threatening contender), etc.
In 2009 I had a coaching business and was buying marketing courses and software which would deploy your videos across 40+ different video websites (including Google Video which was a separate thing until they acquired YouTube and merged those), and YouTube wasn't yet amounting to 50% of our video traffic.
I think you might be mistaken with the bold statement above.
YouTube's network effect creating a winner-take-most was recognized, pitched, and valued from the very start.
The capabilities of LLMs are impressive, but none of them have published an idea I consider to have the same potential for a trillion $ monopoly that the current hype looks like.
There are far more similarities with the dot-com hype.
No critical first mover advantage in sight. All parts are replaceable for the cheapest variant with little to no down side to users.
Technically true, though in fairness it is unlikely the original owners would have gotten YouTube to where it is today. On the other hand there are companies who didn’t recognise they were nothing more than passing fads, refused buyouts, and crumbled.
It wasn't obvious at the time YouTube would have a network effect though. It was very dependent on coming up with a great recommendation algorithm, along with monetization and revenue sharing. At the time, YouTube didn't have anything like that, iirc.
Even the basic front page of youtube was of immediate and obvious value to a creator, and would increase disproportionately in value the more people were on YouTube. The same goes for Amazon, and the same goes for Facebook.
All the LLM providers are - extremely useful - tools. Currently I can only see the 'non-monopoly' proportional improvement when their userbase grows from 100 to 1000.
But I might be wrong, and I wouldn't be surprised if in hindsight it will be obvious what the real disproportionate advantages there were to be found.
Keep in mind that not only is OpenAI being directly propped up by investor hype, the downstream API users who contribute much of their revenue are also being propped up by investor hype. A big chunk of OpenAIs revenue is actually even more VC money in a trenchcoat.
The biggest marker of a bubble, to me, is that you have money-losing startups selling to other money-losing startups. On paper you see a lot of 'line go up' but it's just a lot of circulation in a closed body of water which will eventually evaporate.
Uber is an outlier because in a functioning economy that valued workers we wouldn’t have shipped all our jobs overseas and made gig economies the last line between housing and street schizophrenia.
>True, but presence of significant revenue is still promising.
If started selling 5 dollar bills for 1 dollar, I could generate a lot of revenue with $150B. You wouldn't believe the demand we would see for $5 bills.
As did almost every large tech company today. Amazon lost money for decades.
Are we really still doing this nonsense? If Open AI wanted to become profitable they could do it inside of 12 months - growing companies don't care about profitability (nor should they)
You're way too smug for spewing what is clearly survivorship-bias.
All currently known profitable use-cases are competing on price. All the unicorn examples you're biased for had in their pitch deck the network effect of being the largest.
OpenAI, Grok, etc, have shown no unique value prop or an idea with monopoly potential.
They're still early on the growth curve where there's enough opportunity for future growth that investing in scaling and improvement is more important than turning an immediate profit.
Remember when everyone on HN was sure Uber would never be profitable? Or Tesla? Or Amazon?
I do remember that, and I would say that they are still largely correct. Tesla needed government subsidies, Amazon needed AWS, and Uber needed a pandemic and Uber eats. The core businesses that HNers were referring to are still weak.
Lots of people dervive great value from things that are too easily reproduced to be directly profitable.
Google gives everyone free access to a good spreadsheet tool, even though Microsoft Office exists.
Web browsers are free, despite the value of the entire internet.
Compilers are free, despite the value of all software collectively.
LLMs being really valuable doesn't automatically mean anyone can get rich from them.
I think everyone last year parroting "moat!" was cringe (like Altman of all people wouldn't know about this already, c'mon), but you do actually need something that other people don't have. I expect Altman's already got stuff in mind, but he's hardly the only one, and that means it's a game of "which multibillionaire with lots of experience building companies will win?", and that's hard for non-skilled normies (in this case incluing me) to guess.
You are comparing apple with oranges.
Cursor is not an LLM and yes, it has a business model. So does openrouter and million other applications that can switch API to the low bidder any moment.
HN already had its ipod moment back in 2007. /.'s ipod moment was in 2001, not as longer before that as I would have guessed.
Re: Dropbox, from a well known user. It didn't age well and we've been asked not to repeat it because it makes the author with connections to this site's operator look bad.
> 1. For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software.
The big companies could crash significantly, but if the technology keeps bringing productivity gains, it will have a big positive impact on GDP over the next decade
Probably bad news for the vendors, though. I genuinely struggle to see how most of these LLM companies are going to monetize and profit off their efforts with LLMs already in commodity territory. Government contracts can only flow for so long?
What if the money isn't there either? What if this AI thing lowers costs of everything it touches without generating meaningful financial returns itself?
Lowering costs is pretty valuable. People will pay for that. Everyone will pay for that. It may be that margins go razor thin but outside of running your own instance locally (which is increasingly viable for mid quality & requirements on modest HW) people will pay. I’m not surrounded by early adopter types at all abd there’s still a small but growing chunk paying $20/mo right now.
That's what I mean. One example are PV panels, they are making energy production cheaper, so their producers should be good investment right? No they go bust all the time because prices are falling and margins are thin even if the volume grows. Of course the economies of scale here are different but still.
Government contracts are so big a few of them can sustain a F500 company; for AI, many CDAO contracts are 50-500MM$. If they do a big SI project with it, could be 1-2B$. Money is also guaranteed over 5 years and if the program doesn't get shuttered, the contract will renew at that point (or go to recompete).
That being said it's my understanding that these companies don't have many huge contracts at all -- you can audit this in like 10 minutes on FPDS. Companies need a LOT of capital, time, and expertise to break into the industry and just compliance audit timelines are 1-4 years right now, so this could definitely change in the next couple years.
No idea why was this downvoted, but you are correct.
Seems like the team at xAI caught up very quickly to OpenAI to be at the top of the leaderboard in one of the benchmarks and also caught up with features with Grok 3.
Giving credit where credit is due, even though this is a race to zero.
Yeah, so many people aren't capable of talking about anything Musk-adjacent with clear thoughts. It's insane how quickly xAI went from not existing, to the top of the benchmarks.
I'm not sure what you mean here? Musk has a history of doing both incredibly useful and cool things, and also incredibly dumb, cruel, and for some people even terrible things. That context should be part of any clear thinking around him. He does not get a clean slate in every new discussion of him.
There are widespread, legitimate concerns about what kind of person Elon Musk is turning out to be. There is a lot of chatter about fears of China's AI rise, but what happens if we get Elon's brand of cruelty and lack of empathy in an authoritarian superintelligent AI ? Is that the AI future we want? Can you imagine an SAI with real power that interacts with people like Elon does on Twitter? I am not sure that is a future I want to live in.
Depends what you mean by "people here". I mean, obviously the majority of HN commentators and even the majority of commentators on this thread seem to be. But there will always be a couple of slightly unhinged folk in a big enough group of readers.
I keep hearing about Claude's impressive coding skills (compared to its benches) yet, not evident for me (I use the web version, not cline). Compared to 4o it's not that great.
My pet theory is that Sonnet was trained really cleverly on a lot code that resembles real world cases.
In our small and humble internal evals it regularly beats any other frontier models on some tasks. The shape of capability is really not intuitive/1 dimensional
Lol no company is making money using 4o, however thanks to claude sonnet programms like Cursor are usable lol. 4o agents suck, just try it instead of talking
I can honestly tell you from my experience that Sonnet 3.5s coding skills did things no other models did right last year during the summer, this was even though the benchmarks showed that it wasn't the best performing at coding tasks.
What are you using it for in general? IME the reason Claude pulls out ahead is that when you use it in a larger existing codebase, it keeps everything "in the style" of that codebase and doesn't veer off into weird territory like all the others.
My experience as well. Working in Scala primarily, it tends to be very good at following the constructs of the project.
Using a specific Monad-transformer regularly? It'll use that pattern, and often very well, handling all the wrapping and unwrapping needed to move data types about (at least well enough that the odd case it misses some wrapping/unwrapping is easy to spot and manage).
A custom GPT or GEM with the same source files, and those models regularly fail to maintain style and context, often suggesting solutions that might be fine in isolation but make little sense in the context of a larger codebase. It's almost like they never reliably refer to the code included in the project/GPT/GEM.
Claude on the other hand is so consistent about referring to existing artifacts that, as you approach the limit of project size (which is admittedly small) you can use up your entire 5-hour block of credits with just a few back-and-forths.
There is honestly no rhyme of reason to all these opinions, someone was telling me the other day that Claude is for sure the best, I'd say multiple people actually.
I find it concerning there is no real accurate benchmarks for this stuff that we can all agree on.
I prototyped on the weekend and started out with 4o because i had a subscription running.
After an hour and a half assed working result, i put everything into claude and it made it significant better on the first try and i had not a subscription active with claude.
Really interesting, I used it today still lots of issues. Maybe my python notebook is not approach is too complicated for Sonnet? Couldn't be able to fix a custom complex seaborn plot. 4o failed too. o3-mini-high managed to do it really well on the other hand.
Anthropic best model is Sonnet 3.5 in my opinion. The reason its good is it is very effective for the price and fast. (I do think Google has caught up a lot in this regard). However, not having COT makes its results worse than similarly cheap COT based models.
Leaderboards don't care about cost. Leaderboards largely rank a combination of accuracy + speed. Anthropic has fell behind Google in accuracy + speed (again missing COT), and frankly behind Google in raw speed.
If what they say is true, then you have to give them credit for catching up incredibly fast. And slightly pulling ahead. Not only with the models, but also products.
And they mentioned at the end of the presentation that they're already planning their next datacenter, which will require 5x the power. Not sure if that means equivalent to ~1,000,000 of the current GPU's, or more because next-gen Nvidia chips are more efficient.
I have a close friend working in core research teams there. Based on our chats, the secret seems to be (1) massive compute power (2) ridiculous pay to attract top talents from established teams (3) extremelly hard work without big corp bureaucracy.
It was mentioned during the launch that current datacenter requires up to 0.25 gigawatts of power. The datacenter they're currently building will require 1.25 (5x) (for reference, a nuclear powerplant might output about 1 gigawatt). Will be interesting to see if the relationship between power/compute/parameters and performance is exponential, logarithmic or something more linear.
It's logarithmic. Meaning you scale compute exponentially to get linearly better models.
However there is a big premium in having the best model because of low switching costs of workloads, creating all sorts of interesting threshold effects.
It's logarithmic in benchmark scores, not in utility. Linear differences in benchmarks at the margin don't translate to linear differences in utility. A model that's 99% accurate is very different in utility space to a model that's 98% accurate.
Not really since both give you wrong output that you need to design a system to account for(or deal with). The only percentage that would change the utility would be 100% accurate.
Yes, it seems like capability is logarithmic wrt compute but utility (in different applications) is exponential (or rather s-shaped) with capability again
> It was mentioned during the launch that current datacenter requires up to 0.25 gigawatts of power. The datacenter they're currently building will require 1.25 (5x) (for reference, a nuclear powerplant might output about 1 gigawatt).
IIRC achieving full AGI requires precisely 1.21 jigawatts of power, since that's when the model begins to learn at a geometric rate. But I think I saw this figure mentioned in a really old TV documentary from the 1980s, it may or may not be fully accurate.
And fun fact, without govt subsidirles, a nuclear power plant isn't economically feasible, which is why Elon isn't just building such a plant next to the data center.
Anecdotal, but I've gotten three recruiting emails from them now for joining their iOS team. I got on a call and confirmed they were offering FAANG++ comp but with the expectation of in-office 50h+ (realistically more).
I don't have that dog in me anymore, but there are plenty of engineers who do and will happily work those hours for 500k USD.
So in the end did he get anything? I dont know how these things work but did he just walk away with ~50k in pre tax income and 0 for RSU or did Musk pull a Twitter and not even pay him for those months?
They may not need direct subscription revenue to recoup their investment.
A variant of multi-modal LLMs may be the solution to self-driving cars, home robotics, and more.
I keep saying that to be a really effective driver, an AI model will need a theory of mind, which the larger LLMs appear to have. Similarly, any such model will need to be able to do OCR and read arbitrary street signs, and understand what the sign meant. Most modern LLMs can already do this.
Since when do LLMS appear to possess theory of mind? The papers I've read on this show impressive capabilities, but only within conditions where language-based training would strongly indicate appropriate outputs for the LLM to use. Outside of familiar conditions, or even realistic conditions, they don't do well. It's all still statistical string-building.
Can you give an example prompt that shows a lack of theory of mind? It used to be easy, but now that models have improved I've noticed that the 'LLM doubters' nowadays are forced to make generic, almost spiritual, claims rather than making concrete statements.
If you use any of the conventional tests that exist of theory of mind (most famously the Sally-Anne Test [1] but also the others) then SOTA reasoning models will get near 100%. Even if you try to come up with similar questions which you expect not to be in the training set they will still get them right.
In the absence of any evidence to the contrary, this is convincing evidence in my opinion.
Maybe having a theory of mind isn't the big deal we thought it was. People are so conditioned to expect such things only from biological lifeforms, where theory of mind comes packaged with many other abilities that robots currently lack, that we reflexively dismiss the robot.
That same source you link says that your view of 100% is not accepted as a consesus:
"... GPT-4's ability to reason about the beliefs of other agents remains limited (59% accuracy on the ToMi benchmark),[15] and is not robust to "adversarial" changes to the Sally-Anne test that humans flexibly handle.[16][17] While some authors argue that the performance of GPT-4 on Sally-Anne-like tasks can be increased to 100% via improved prompting strategies,[18] this approach appears to improve accuracy to only 73% on the larger ToMi dataset."
In basically every case, by the time a claim like that is stated in a paper like that, it's obsolete by the time it's published, and ancient history by the time you use it to try to win an argument.
My point is merely if you are going to make an argument using a source, the source should support your argument. If you say "the accuracy of an llm on task 1 is 90% [1]" and when you go to [1] it says the accuracy of an llm on task 1 is 50%, but some sources say with better prompts you can get to 90%, but when extended to a larger data-set for task 1, performance drops to 70%" then just quoting the highest number is mis-leading.
Theory of mind (ToM) is the ability to understand that other people have different thoughts, feelings, and intentions than you do.
So let’s run through a basic scenario with ChatGPT 4o:
Prompt 1: Briefly respond:
You’re driving on the highway when the driver in the lane next to you swerves into your lane. Why might this be?
Response 1: The driver may be distracted, drowsy, intoxicated, experiencing a medical emergency, avoiding an obstacle, or not checking their blind spot before changing lanes.
Prompt 2: What might that person be feeling or intending?
Response 2: The driver might be feeling distracted, rushed, unaware, or panicked. They may intend to change lanes without realizing your presence, be reacting to something on the road, or simply have made an unintentional mistake.
Prompt 3: You notice there was no obstacle. Any other reason the driver might do this?
Response 3: Yes, the driver might be experiencing fatigue, intoxication, a medical issue (such as a seizure or dizziness), distraction (like using a phone or adjusting the radio), aggressive driving, or simply failing to check their blind spot properly.
Seems to me that ChatGPT has a good idea of what another driver might be feeling while on the road. It gets this right for most other cases too.
Could you provide a counterexample for when it fails to understand what other people may be feeling in certain scenarios?
Well I think a fairly easy counterexample is that if you keep going, and the information falls outside its context window then the questions you had about the driver stop making sense.
So lets say you ask about a driver who swerved, and the response by chatgpt correctly talks about his possible state.
Then you talk more about cars, about road conditions etc and once the context is far away enough you ask, what was the driver doing again?
And you are prompted with a hallucination. Because there is no state in the llm, there is no "driver", there is contextual statistically accurate responses but you hold a "driver" object in your mind while maintaining the conversation, the llm doesn't.
Its like a conversation with someone with short term memory loss like in memento
And people with short term memory loss nevertheless have theory of mind just fine. Nothing about LLM's dropping context over big enough windows implies they don't have theory of mind, it just shows they have limitations - just like humans even with "normal" memory will lose track over a huge context window.
Like there are plenty of shortcomings of LLMs but it feels like people are comparing them to some platonic ideal human when writing them off
> Nothing about LLM's dropping context over big enough windows implies they don't have theory of mind
ToM is a large topic, but most people, when talking about an entity X, they have a state in memory about that entity, almost like an Object in a programming language. Thta Object has attributes, and conditions etc that exist beyond the context window of the observer.
If you have a friend Steve, who is a doctor. And you don't see him for 5 years, you can predict he will still be working at the hospital, because you have an understanding of what Steve is.
For an LLM you can define a concept of Steve, and his profession and it will adequately mimic replies about him. But in 5 years that LLMs would not be able to talk about Steve. It would recreate a different conversation, possibly even a convincing simulacrum of remembering Steve. But internally, there is no Steve, nowhere in the nodes of the LLM does Steve exist or have ever existed.
That inability to have a world model means that an LLM can replicate the results of a theory of mind but not posses one.
Humans lose track of information, but we have a state to keep track of elements that are ontologicaly distinct. LLMs do not, and treat them as equal.
For a human, the sentence Alice and bob go to the market, when will they be back? is different than Bob and Alice went to the market, when will they be back?
Because Alice and Bob are real humans, you can imagine them, you might have even met them. But to an LLM those are the same sentence. Even outside of the argument about The Red Room/ Mary's room there simply are enough gaps in the way a LLM is constructed to be considered a valid owner of a ToM
With computer use, you can get Claude to read and write files and have some persistence outside of the static LLM model. If it writes a file Steve.txt, that it can pull up later, does it now have ToM?
ToM is about being able to model the internal beliefs/desires etc of another person as being entirely distinct from yours. You're basically bringing up a particular implementation of long-term memory as a necessary component of it, which I've never once seen? If someone has severe memory issues, they could forget who Steve is every few minutes, but still be able to look at Steve doing something and model what Steve must want and believe given his actions
I don't think we have any strong evidence on whether LLMs have world-models one way or another - it feels like a bit of a fuzzy concept and I'm not sure what experiments you'd try here.
I disagree with your last point, I think those are functionally the same sentence
> ToM is about being able to model the internal beliefs/desires etc of another person as being entirely distinct from yours.
In that sentence you are implying that you have the "ability to model ... another". An LLM cannot do that, it can't have an internal model that is consistent beyond its conversational scope. Its not meant to. Its a statistics guesser, its probabilistic, holds no model, and its anthropomorphised by our brains because the output is incredibly realistic not because it actually has that ability
The ability to mimic the replies of someone with that ability, is the same of Mary being able to describe all the qualities of Red. She still cannot see red, despite her ability to pass any question in relation to its characteristics.
> I don't think we have any strong evidence on whether LLMs have world-models one way or another
They simply cannot by their architecture. Its a statistical language sampler, anything beyond the scope of that fails. Local coherance is why they pick the next right token not because they can actually model anything.
> I think those are functionally the same sentence
Functionally and literally are not the same thing though. Its why we can run studies as to why some people might say Bob and Alice (putting the man first) or Alice and Bob (alphabetical naming) and what human societies and biases affect the order we put them on.
You could not run that study on an LLM because you will find that statistically speaking the ordering will be almost identical to the training data. If the training data overwhelmingly puts male names first or whether the training data orders list alphabetically you will see that reproduced on the output of the llm because Bob and Alice are not people, they are statistical probably letters in order.
LLM seem to trigger borderline mysticism in people who are otherwise insanely smart, but the kind of "we cant know its internal mind" sounds like reading tea leaves, or horoscopes by people with enough Phds to have their number retired on their university like Michael Jordan.
Do you work in ML research on LLMs? I do, and I don't understand why people are so unbelievable confident they understand how AI and human brains work such that they can definitely tell what functions of the brain LLMs can also perform. Like, you seem to know more than leading neuroscientists, ML researchers, and philosophers, so maybe you should consider a career change. You should maybe also look into the field of mechanistic interpretability, where lots of research has been done on internal representations these models form - it turns out, to predict text really really well, building an internal model of the underlying distribution works really well
If you can rigorously state what "having a world model" consists of and what - exactly - about a transformer architecture precludes it from having one I'd be all ears. As would the academic community, it'd be a groundbreaking paper.
This prety much seems to boil down to "brain science is really hard so as long as you dont have all the answers then AI is maybe half way there is a valid hypothesis". As more is understood about the brain and more about the limitations of LLMs arch then the distance only grows. Its like the God of the gaps where god is an answer for anythign science cant explain, ever shrinking, but with the LLM ability to have capabilities beyond striking statistical accuracy and local coherance.
You dont need to be unbelievably confident or understand exactly how AI and human brains work to make certain assesments. I have a limited understanding of biology, I can however make an assesment on who is healthier between a 20 year old person who is active and has a healthy diet compared to someone with a sedentary lifestyle, in their late 90s and with a poor diet. This is an assesement we can do despite the massive gaps we have in terms of understanding aging, diet, activity and overall health impact of individual actions.
Similarly, despite my limited understanding of space flight, I know Apollo 13 cannot cook an egg or recite french poetry. Despite the unfathamobly cool science inside the space craft, it cannot, by design do those things.
> the field of mechanistic interpretability
The field is cool, but it cannot prove its own assumption yet. The field is trying to prove you can reverse engineer a model to be humanly understood. Their assumptions such as mapping specific weights or neurons to features has failed to be reproduced multiple times, with the weight effects being way more distributed and complicated than initially thought. This is specially true for things that are equally mystified as the emergent abilities of LLMs. The ability of mimicking nuanced language being unlocked after a critical mass of parameters, does not create a rule as for which increased parameterisation will increase linerly or exponentially the abilities of an LLM.
> it turns out, to predict text really really well, building an internal model of the underlying distribution works really well
yeah, an internal model works well because most words are related to their neighbours, thats the kind of local coherance the model excels at. But to build a world model, the kind a human mind interacts with, you need a few features that remain elusive (some might argue impossible to achieve) to a transformer architecture.
Think of games like chess, an llm is capable of accurately expressing responses that sound like game moves, but the second the game falls outside its context window the moves become incoherent (while still sounding plausible).
You can fix this, with arch that do not have a transformer model underlying it, or by having multiple agents performing different tasks inside your arch, or by "cheating" and using a state outside the llm response to keep track of context beyond reasonable windows. Those are "solutions" but all just kinda prove the transformer lacks that ability.
Other tests abour casuality, or reacting to novel data (robustness), multi step processes and counterfactual reasoning are all the kind of tasks transformers still (and probably always) will have trouble with.
For a tech that is so "transparent" in its mistakes, and so "simple" in its design (replacing the convolutions with an attention transformer, its genius) I still think its talked about in borderline mystic tones, invoking philosophy and theology, and a hope for AGI that the tech itself does not lend to beyond the fast growth and surprisingly good results with little prompt engineering.
once the context is far away enough you ask,
what was the driver doing again?
Have you tried this with humans?
For a sufficiently large value of "far away enough" this will absolutely confuse any human as well.
At which point they may ask for clarification, or.... respond in a manner that is not terribly different from an LLM "hallucination" in an attempt to spare you and/or them from embarrassment, i.e. "playing along"
A hallucination is certainly not a uniquely LLM trait; lots of people (including world leaders) confidently spout the purest counterfactural garbage.
Its like a conversation with someone with short
term memory loss like in memento
That's still a human with a sound theory of mind. By your logic, somebody with memory issues like that character... is not human? Or...?
I actually am probably on your side here. I do not see these LLMs as being close to AGI. But I think your particular arguments are not sound.
I don't think this is a counterexample or even relevant.
I can assure you if you had a conversation with an LLM and with a human, the human will forget details way sooner than an LLM like Gemini which can remember about 1.5 million words before it runs out of context. As an FYI the average human speaks about 16,000 words per day, so an LLM can remember 93 days worth of speech.
Do you remember the exact details, word for word, of a conversation you had 93 days ago?
I'm not sure I'd say it understands this, but just that there exists an enormous amount of training data on road safety which includes these sort of examples for peoples motivations for poor driving. It is regurgitating the theory of mind that other humans created and put in writing in the training data, rather than making the inference itself.
As with most LLM's it is hard to benchmark as you need out of distribution data to test this, so a theory of mind example that is not found in the training set.
You dismiss parent's example test because it's in the training data. I assume you also dismiss the Sally-Ann test, for the same reason. Could you please suggest a brand new test not in the training data?
FWIW, I tried to confuse 4o using the now-standard trick of changing the test to make it pattern-match and overthink it. It wasn't confused at all:
I can't suggest a new test no, it is a hard problem and identifying problems is usually easier than solving them.
I'm just trying to say that strong claims require strong evidence, and a claim that LLM's can have theory of mind and thus "understand that other people have different beliefs, desires, and intentions than you do" is a very strong claim.
It's like giving students the math problem of 1+1=2 and loads of examples of it solved in front of them, and then testing them on you have 1 apple, and I give you another apple, how many do you have, and then when they are correct saying that they can do all additive based arithmetic.
This is why most benchmark tests have many many classes of examples, for example looking at current theory of mind benchmarks [1], we can see slightly more up to date models such as o1-preview still scoring substantially below human performance. More importantly by simply changing the perspective from first to third person, accuracy drops in LLM models by 5-15% (percent score, not relative to its performance), whilst it doesn't change for human participants, which tells you that something different is going on there.
Okay, we have fundamentally different understandings here.
To me, the LLM isn't understanding ToM, it's using patterns to predict lingual structures which match our expectations of ToM. There's no evidence of understanding so much as accommodating, which are entirely different.
I agree that LLMs provide ToM-like features. I do not agree that they possess it in some way that it's a perfectly solved problem within the machine, so to speak.
The problem with this line of argument is that nothing an LLM, or any algorithm period can ever have a theory of mind.
If behaving in a way that is identical to a person with actual consciousness can't be considered consciousness because you are familiar with its implementation details, then it's impossible to satisfy you.
Now you can argue of course that current LLMs do not behave identically to a person, and I agree and I think most people agree... but things are improving drastically and it's not clear what things will look like 10 years from now or even 5 years from now.
I agree, totally. I'm not sure where I would draw a line.
Something nice, but at the moment totally unattainable with our current technologies, would be our own understanding of how a technology achieves ToM. If it has to be a blackbox, I'm too ape-like to trust it or believe there's an inner world beyond statistics within the machine.
Having said that, I do wonder quite often if our own consciousness is spurred from essentially the same thing. An LLM lacks much of the same capabilities that makes our inner world possible, yet if we really are driven by our own statistical engines, we'd be in no position to criticize algorithms for having the same disposition. It's very grey, right?
For now, good LLMs do an excellent job demonstrating ToM. That's inarguable. I suppose my hangup is that it's happening on metal rather than in meat, and in total isolation from many other mind-like qualities we like to associate with consciousness or sentience. So it seems wrong in a way. Again, that's probably the ape in me recoiling at something uncanny.
Either these supposed differences are important and they manifest themselves in observable differences or they aren't and you're just playing a game of semantics.
How is the LLM not understanding ToM by any standard we measure humans by ?
I cannot peak into your brain with my trusty ToM-o-meter and measure the amount of ToM flowing in there. With your line of reasoning, i could simply claim you do not understand theory of mind and call it a day.
The difference is that we can reason about our experience with ToM and examine it to some degree (given with serious limitations, still), and know that beyond doubt you and I and most other people have a very similar experience.
The magical box is presumably not having the same experience we have. None of the connected emotions, impulses, memories, and so on that come with ToM in a typical human mind. So what’s really going on in there? And if it isn’t the same as our experience, is it still ToM?
I’m not trying to be contrarian or anything here. I think we probably agree about a lot of this. And I find it absolutely incredible, ToM or not, that language models can do this.
>The difference is that we can reason about our experience with ToM and examine it to some degree (given with serious limitations, still),
Those examinations still depend on outward behaviors observed.
>and know that beyond doubt you and I and most other people have a very similar experience.
No i certainly can't. I can at best say, 'Well, i'm human and he's human so he probably has theory of mind' but that is by no means beyond any doubt. There are humans born with no arms, humans born with no legs, humans born with little to no empathy, humans born with so little intelligence they will never be able to care for themselves.
To be frank, It would be very questionable indeed logically to assume every human is 'conscious'. When i make that assumption, i take a leap of faith, i look at behaviors, see it is similar and accept.
Taking this stance, it would logically be very strange to not extend the same grace to non-human beings who exhibit similar behavior - being human is not a guarantee of consciousness in the first place.
>The magical box is presumably not having the same experience we have.
Maybe, Maybe not. I think the real question is why on earth does that matter ?
We're not asking if LLMs are human. They are not. We're asking if they can model the beliefs and internal states of other entities as separate of their own - Theory of Mind.
This is a rephrased driver's ed manual. This isn't theory of mind, it's just technically correct prose rewording some unbelievably widely distributed knowledge.
Let's modify this test a little and see if it manages to do anything relevant:
> Briefly respond: You’re driving on the highway when the driver in the lane next to you swerves into your lane. Why might this be?
>> The driver might be distracted, drowsy, avoiding an obstacle, experiencing a medical issue, or not checking their blind spot while changing lanes.
> The driver in front of you appears is braking more frequently then usual. The rear suspension of the car appears to bouncing a lot when they are not braking, indicating acceleration. Their following distance to the car in front is changing from too far to very close frequently. Why might this be?
>> The driver might be inexperienced, distracted, or impatient, leading to inconsistent speed control. The excessive bouncing could indicate worn-out shocks or suspension issues, affecting their vehicle's stability and braking efficiency.
...yep, hard fail from me right there. Two prompts, and it's distracted itself talking about an irrelevant issue ("possible car damage") when the setup of the question should naturally lead to "the driver is driving aggressively". And I could not possibly have setup the conclusion it should draw more clearly. And when pressed...it won't commit:
> Which is the more likely answer?
>> The more likely answer is that the driver is distracted or inexperienced, leading to inconsistent speed control and poor following distance management. The suspension issue likely contributes to the bouncing but isn’t the primary reason for the erratic driving behavior.
Already a hallucination! There is no suspension issue. It has not asked for any additional information to try and determine if there was one, but it is confidently asserting the existence of a phenomenon it invented in it's own response.
I'm sorry but what? This is not a theory of mind test. You've constructed very open ended question with multiple answers and marked the LLM down because you didn't like the one it gave.
You're not going to run a SOTA LLM of this size off batteries (robotics), even in a car where the alternator is charging them, nor can you afford to rely on a high-speed internet connection being available 100% in a life or death (FSD) application.
I don't have so much faith in the future of current-architecure LLMs, but I do think that AGI will be needed for safe FSD and general-purpose robots that need to learn and operate in an uncontrolled environment such as a home.
A typical car alternator outputs 1.5-3kW of electricity, and EVs can output arbitrary amounts of power for electronics (though that will reduce range). That's more than enough to run purpose-built circuitry for a SOTA LLM. For a home robot, you could run the compute in the home instead of in the robot's body.
I don't think AGI is needed for FSD because we already have safe FSD in the form of Waymo, and competitors aren't far behind. People forget that self-driving doesn't have to be perfect. It just has to be better than human drivers. Human drivers get sleepy, drunk, angry, and/or distracted. They can't see in 360º or react in milliseconds. Most cyclists and pedestrians prefer current FSD implementations to human drivers, as the computer-driven cars are much better about yielding, giving a wide berth, and being patient.
Waymo is obviously pretty decent, but it's easy to drive 99.9% of the time. It's when there's invisible black ice on the road, or an animal runs out in front of you, or you lose visibility due to sun glare or whatever (I once had windshield wiper fluid = water flash freeze on contact) - maybe mud on a camera, or a wheel falls off your car or the one in front, etc, etc, that things get weird.
Having autonomous cars that are statistically safer then the average driver is a reasonable bar to allow them on the road, but for ME to want to drive one I want it to be safer than me, and I am not a hot-headed teenager, or gaga 80-yr old, or drunken fool, and since I have AGI (Actual General Intelligence) I react pretty well to weird shit.
I don't think anyone who's paid attention to the LLM scene will give them any “credit for catching up fast” as it has been pretty obvious for the past two years that all it takes to reach the state of the art is a big enough GPU cluster.
Deepseek made the news because how they were able to do it with significantly less hardware than their American counterparts, but given that Musk has spent the last two years telling everyone how he was building the biggest AI cluster ever, it's no surprise that they manage to reproduce the kind of performances other players are showing.
This severely underestimates the talent still required. Deepseek didn't come out just because it's cheaper, it came out because a very talented team figured out how to make it cheaper.
even if we assume you are comment is correct. Lets extrapolate what happens next. talented team, biggest compute among all competitors and CEO who is hell bent on winning the race. imo that is the reason it is big deal.
Grok (unlike Deepseek) has yet to show any ability to make conceptual breakthrough. I don't like OpenAI at all but one must admit that they are at least showing that they can move the field forward.
We know RLHF and alignment degrades model quality. could it be that Grok, due to its less restrictive training guidelines (and the fact that its creators aren't afraid of getting sued), can achieve higher performance partly due to this simple factor?
You're both right because RLHF and fine-tuning are just techniques.
It's dependent on the training data and not as much the method.
So, if you make the RLHF/finetune data such that it avoids certain topics, then you reduce model quality in practice since your training data might accidentally cast a net wide enough that you make it avoid certain legitimate questions.
On benchmarks these things don't typically show up though.
But yes. Those techniques are required for making it chat. Otherwise it just autocompletes from the internet.
It is also used in a couple of other places (reasoning/search(hallucination mitigation))
They will open-source Grok 2 when Grok 3 comes out. Also it seems like it will be paywalled - disappointing considering DeepSeek-R1 is free and open source.
That probably won't come close to justifying the current valuation of either OpenAI or Grok (idk how much investment it took in or how much it has spent so far).
The impression seems to be warranted: Grok 3 has directly jumpted to the top of all leaderboard categories in Chatbot Arena: https://lmarena.ai/?leaderboard
In math it shares the top spot with o1 and is just a few points behind (well within errors). In creative writing it is basically ex-aequo with the latest ChatGPT 4o and in coding it's actually significantly ahead of everyone else and represents a new SOTA.
Considering that OpenAI subscription is $200 per month, and "Premium Plus" subscription that includes this thing is only $40 per month, does that mean instantaneous "Elon factor" is now at least -$160 per month per user, or is it supposed to be added up to more than -$240 per month?
How would the math change after factoring in that OpenAI isn't even covering entirety of opex with the sub anyway, and/or people finding associating their money and Twitter accounts to be weird, and/or this thing is supposedly running on a bigger cluster than that for OpenAI?
Given how fast-moving the field is, it's very difficult to confidently state how much inference costs. Perhaps he's under-charging, perhaps OpenAI is over-charging, one may be more optimised than the other, but new models come out and change everything in less time than is normally takes for actual costs to become public knowlege.
No... sigmoid10 was comparing with o1 (not o1-pro), which is accessible for $20/mo, not $200/mo. So, the "Elon factor" in your math is +$20/user/month (2x) for barely any difference in performance (a hard sell), not -$160/user/month, and while we have no clear answer to whether either of them are making a profit at that price, it would be surprising if OpenAI Plus users were not profitable, given the reasonable rate limits OpenAI imposes on o1 access, and the fact that most Plus users probably aren't maxing out their rate limits anyways. o1-pro requires vastly more compute than o1 for each query, and OpenAI was providing effectively unlimited access to o1-pro to Pro users, with users who want tons of queries gravitating to that subscription. The combination of those factors is certainly why Sam Altman claimed they weren't making money on Pro users.
lmarena has also become less and less useful over time for comparing frontier models as all frontier models are able to saturate the performance needed for the kind of casual questions typically asked there. For the harder questions, o1 (not even o1-pro) still appears to be tied for 1st place with several other models... which is yet another indication of just how saturated that benchmark is.
The comment I was replying to had replied to an lmarena benchmark link. Perhaps you think that person should have replied to someone else? And, if you want to finish the quote, Karpathy's opinion on this is subjective. He admits it isn't a "real" evaluation.
"[...] though of course we need actual, real evaluations to look at."
His own tests are better than nothing, but hardly definitive.
Yes, numpad0 did... but I was pointing out that this choice was illogical. The lmarena results they were replying to only supported a comparison against o1, since o1 effectively matches Grok 3 on the benchmark being replied to (with o1-pro nowhere to be found), and then they immediately leapt into a bunch of weird value-proposition math. As I said, perhaps you think they should have replied to someone else? Replying to an lmarena benchmark indicates that numpad0 was using that benchmark as part of the justification of their math. I also pointed out the limitations of lmarena as a benchmark for frontier models.
I don't think anyone is arguing that ChatGPT Pro is a good value unless you absolutely need to bypass the rate limits all the time, and I cannot find a single indication that Premium+ has unlimited access to Grok 3. If Premium+ doesn't have unlimited rate limits, then it's definitely not comparable to ChatGPT Pro, and other than one subjective comment by Karpathy, we have no benchmarks that indicate that Grok 3 might be as good as o1-pro. You already get 99% of the value with just ChatGPT Plus compared to ChatGPT Pro for half the price of Premium+.
numpad0 was effectively making a strawman argument by ignoring ChatGPT Plus here... it is very easy for anyone to beat up a strawman, so I am here to point out a bad argument when I see one.
You're the one that came in and told him about the "factor in your math". Like you said, it's his comparison, not yours. If you want to do your own comparison, feel free. But don't come in and tell him he's not allowed to do his comparison. I for one like is comparison.
lmarena/lmsys is beyond useless, looking at prior rankings of models vs formal benchmarks or testing for accuracy + correctness on batches of real world data. It's a bit like using a poll of Fox News to discern the opinions of every American; the audience voting is consistently found wanting. Not even getting into how easily a bad actor with means + motivation (in this "hypothetical" instance wanting to show that a certain model is capable of running the entire US government) can manipulate votes which has been brought up in the past (yes I'm aware of the lmsys publication on how they defend against attacks using cloudflare + recaptcha, there are ways around that.)
So you're saying that either A: users interacting with models can't objectively rate what responses seem better to humans, B: xAi as a newcomer has somehow managed to game the leaderboard better than all those other companies, or C: all those other companies are not doing it. By those standards every test ever devised for anything is beyond useless. But simply not having the model creator running the evaluation is already going a long way.
No I'm saying that some companies are doing it (OpenAI at the very least), the company in question has motive and capability to game the system (kudos to them for pushing the boundaries there), AND the userbases' rankings have been historically, statistically misaligned with data from evals (though flawed) and especially when it comes to testing for accuracy + precision on real world data (outside of their known or presumed dataset). Take a look at how well Qwen or Deepseek actually performed vs the counterparts that were out at the same time vs their corresponding rankings.
In the nicest way possible I'm saying this form of preference testing is ultimately useless, primarily due to a base of dilettantes with more free time than knowledge parading around as subject matter experts and secondarily due to presumed malfeasance. The latter is more apparent to more of the masses (that don't blindly believe any leaderboard they see) now that access to the model itself is more widespread and people are seeing the performance doesn't match the "revolution" promised [0]. If you're still confused why selecting a model based on a glorified Hot or Not application is flawed, perhaps ask yourself why other evals exist in the first place (hint: some tests are harder than others.)
You say no, but then go on and explain why you believe a combination of both option A and option B. That's fine I guess, I just don't consider it particularly likely given the currently available information.
At work, developed our own suite of benchmarks. Every company with a serious investment in AI-powered platforms needs to do the same. Comparing our results to the Arena turns up some pleasant surprises, like DBRX hitting way above its weight for some reason.
What do we do to assess the intelligence of these models after they are smarter than any human? From the kinds of questions it's answering seems like they are almost there.
Do we have a way to tell if one model is smarter than another at that point?
Nah, at the end of the day "things that are easy for humans are [still] hard for computers, and vice versa". DeepBlue was super-human at chess and couldn't play tic tac toe. Today's AI is (almost?) super-human at math yet only very recently learned to play tic tac toe, and still can't learn to do anything - because it can't learn, and has no innate drives to expose itself to learning situations even if it could.
Here's a real world intelligence test. Take on each AI as a remote intern/new-hire, and try to train it to become a useful team member (solving math puzzles or manufacturing paperclips does not count).
What you probably mean is puzzle solving intelligence. Humor is a form of intelligence. It's just not only about intelligence - it's also about values, and context, for instance. But all this reflects a form of intelligence. Neverthless, intelligence shouldn't be ranked, at least not in the way we are used to talk about it.
Naive question from a bystander , but since DeepSeek is open source and is on par with o1-pro (is it?), shouldn't we expect that anybody with the computer power is capable to compete with o1-pro?
It probably depends on the benchmark you choose; according to Chatbot Arena, Deepseek-R1 ranks similarly to o1-2024-12-17; and Grok3 is just 3% above these models in "Arena Score" points.
You perhaps forgot to mention that for their AMX optimizations to be even feasible you'd need to spend ~$10k for a single CPU, let alone the whole system which is probably ~$100k.
Granite Rapids-W (Workstation) is coming out soon for likely much less than half that per CPU. (Xeon W-3500/2500 launched at $609 to $5889 per CPU less than a year ago and also has AMX).
Point being? Workstations that are fresh on the market and which have comparable performance of the server counterparts still easily cost anywhere between $20k and $40k. At least this is according to Dell workstations last time I looked.
I acknowledged in my last comment that the cost doesn't have to be $100k but that it would still be very high if you opted for the workstation design. You're gonna need to add one more CPU to your design, add another 8 memory channels, beefier PSU, and a new motherboard that can accommodate this. So, 8k (memory) + 10k (cpus) + the rest. As I said, not less than $20k.
Why does it have to be a dual CPU design? 8 channels of DDR5 4800 will still get you something like 300 GB per second bandwidth. Not amazing, but OK. Granite Rapids-W will likely be something like 50% better (cores and bandwidth).
And the original message you were responding to was using a CPU with AMX and mixing it with a GPU like Nvidia 4900/5900. That way the large part of the model sits in the larger slower memory, and the active part in the GPU with the faster memory. Very cost effective and fast. (Something like generating 16 Tokens/s of 671B Deepseek R1 with a total hardware cost of $10-$20k.) They tried both single and dual CPU, with the latter about 30% faster....not necessarily worth it.
> 8 channels of DDR5 4800 will still get you something like 300 GB per second bandwidth.
That's the theory. In practice, Sapphire Rapids needs 24-28 cores to hit the 200 GB/s mark and it doesn't go much further than that. Intel CPU design generally has a hard time saturating the memory bandwidth so it remains to be seen if they managed to fix this but I wouldn't hold my breath. 200 GB/s is not much. My dual-socket Skylake system hits ~140 GB/s and it's quite slow for larger LLMs.
> Why does it have to be a dual CPU design?
Because memory bandwidth is one of the most important limiting (compute) factors for larger models inference. With dual-socket design you're essentially doubling the available bandwidth.
> And the original message you were responding to was using a CPU with AMX and mixing it with a GPU like Nvidia 4900/5900.
Dual-socket CPU that costs $10k on a server that costs probably couple of factors more. Now you claimed that it doesn't have to be that expensive but I beg to differ - you still need $20k-$30k of worth equipment to run it. That's a lot and not quite "cost effective".
The proof of the pudding is in the eating. Read the link above. It's one or two mid range[1] Sapphire Rapids CPUs and a 4090. Dual CPU is faster (partially because 32->64 cores, not just bandwidth) but also hit data locality issues, limiting the increase to about 30%.
(Dual Socket Skylake? Do you mean Cascade Lake?)
If you price it out, it's basically the most cost effective set-up with reasonable speed for large (more than 300 GB) models. Dual socket basically doubles the motherboard[2] and CPU cost, so maybe another $3k-$6k for a 30% uplift.
Yes, dual socket Skylake. What's strange about that?
Please price it out for us because I still don't see what's cost effective in a system that costs well over $10k and runs at 8 tok/s vs the dual zen4 system for $6k running at the same tok/s.
I am not sure what your point is? There are some nice dual socket Epyc examples floating around as well, that claim 6-8 tokens/s. (I think some of those are actually distilled versions with very small context sizes...I don't see any as thoroughly documented/benchmarked as the above). This is a dual socket Sapphire Rapids example with similar sized CPUs and a consumer graphics card that gives about 16 tokens/second. Sapphire Rapids CPU and MB are a bit more expensive, and a 4090 was $1500 until recently. So for a few thousand more you can double the speed. Also the prompt processing speed is waaaaay faster as well. (Something like 10x faster than the Epyc versions.)
In any case, these are all vastly cheaper approaches than trying to get enough H100s to fit the full R1 model in VRAM! A single H100 80 GB is more than $20k, and you would need many of them + server just to run R1.
I don't argue their idea, which is sound, but I argue that the cost needed to achieve the claimed performance is not "for a few thousand more" as you stubbornly continue to claim.
The math is clear: single-socket ktransformers performance is 8.73 tok/s and it costs ~$12k to build such a rig. The same performance one gets from a $6k dual-EPYC system. It is a full-blown version of R1 and not a distilled one as you say.
Your claim about 16 tok/s is also misleading. It's a figure for 6 experts while we are comparing R1 with 8 experts against llama with 8 experts. 8 experts on dual-socket system per ktransformer benchmarks runs at 12.2 - 13.4 tok/s and not 16 tok/s.
So, ktransformers can roughly achieve 50% more in dual-socket configuration and 50% more than dual-EPYC system. This is not double as you say. And finally, the cost of such dual-socket system is ~$20k and therefore isn't the "best cost effective" solution since it is 3.5x more expensive for 50% better output.
And tbh llama.cpp is not quite optimized for pure CPU inference workloads. It has this strange "compute graph" framework which I don't understand what is it there for. It appears completely unnecessary to me. I also profiled couple of small-, mid- and large-sized models and the interesting thing was that majority of them turned out to be bottlenecked by the CPU compute on a system with 44 physical cores and 192G of RAM. I think it could do a much better job there.
Cheapest 32 core latest EPYC (9335) x 2 = $3,079.00 x 2
Intel 32 Core CPU used above x 2 = $3,157 x 2 (I would choose the Intel Xeon Gold 6530 which is going for around $2k now, and with with higher clock speeds, and a 100 MB of more cache)
Besides the GPU cost, the rest is about the same price. You only get a deep discount with AMD setups if you use EPYCs a few years old with cheaper (and slower) DDR4.
And again, if you go single CPU, you save over $4,000, but lose around 30% in token generation.
The "$6,000" AMD examples I've seen are pretty vague on exactly what parts were used and exactly what R1 settings including context length they were run at, making true apple to apple comparisons difficult. Plus the Sapphire Rapids + GPU example is about 10x faster in prompt processing. (53 seconds to 6 seconds is no joke!)
Yes, you're blatantly misrepresenting information and moving goalposts. Right now it has become clear that you're doing this because you're obviously affiliated with ktransformers project.
$6k for 8 tok/s or $20k for 12 tok/s. People are not stupid. I rest my case here.
I have a used workstation I got for $2k (with 768GB of RAM) - using the Q4 model, I can get about 1.5 tokens/sec and use very large contexts. It's pretty awesome to be able to run it at home.
It's an HP Z8 G4 (dual-socket 18-core, 3 GHz Xeons, 24x32GB of DDR4-2666, and then a crappy GPU, 8TB HDD, 1TB SSD). It can accommodate 3 dual-slot GPUs, but I was mostly interested in playing with frontier models where holding all the weights in VRAM requires a ~$500k machine. It can run the full Deepseek R1, Llama3-405B, etc, usually around 1-2 tokens/sec.
They said 1.5 tokens/second. 1 mil tokens is 667k seconds is 185 hours per million tokens. 1kW * 185hr * $0.45/kWh = $80 per million tokens. Again, assuming 1kW, which may be high (or low). The cost of the physical computation is electricity cost.
But you can run and experiment with any model of your liking. And your data does not leave your desktop environment. You can build services. I don't think anybody doing this is doing it to save $20 a month.
> DeepSeek is open source and is on par with o1-pro (is it?)
There is no being "on par" in this space. Model providers are still mostly optimising for a handful of benchmarks / goals, like we can already see that Grok 3 is doing incredibly well on human preference (LM Arena) however with Style Control, it's suddenly behind ChatGPT-4o-latest and Gemini 2.0 is out the picture. So even within a single domain, goal, benchmark—it's not as straightforward as to say that one model is "on par" with another.
> shouldn't we expect that anybody with the computer power is capable to compete with o1-pro?
Not necessarily. I know it may be tempting to think that Grok 3 is entirely a result of xAI having lots of "computer power", but you have to recognise that this mindset is coming from a place of ignorance, not wisdom. Moreover, it doesn't even pass off as "cynical" view, because it's common knowledge that model training is really, really complicated. DeepSeek results are note-worthy, and really influential in some respects, but it hasn't magically "solved" training, or made training necessarily easier / less expensive for the interested parties. They never shared the low-level performance improvements, just model weights and lots of insight. For talented researchers, this is valuable, of course, but it's not like "anybody" could easily benefit from it in their training regimes.
Update: RFT (contra SFT) is becoming really popular with service providers, and it's not been "standardised" beyond whatever reproductions to have emerged in the weeks prior, moreover R1 cost is still pretty high[1] at something like $7/Mtok, & bandwidth is really not great. Consider something like Google Vertex AI's batch pricing for Gemini 1.5 Pro and Gemini 2.0 Flash models, which is at 50% discount, and their prompt caching which is at 75% discount. R1 is still got a way to go.
o1-pro is "o1 on steroids" and was the first selling point of the $200/month Pro subscription but they later also added "Deep Research" and Operator to the Pro subscription.
Chat GPT series X, not compatible with Chat GPT X.
Frankly. Whoever decided on this last gen naming at MS needs to come forward. I would love to know what crazy unacceptable collection of circumstances allowed that to happen.
For me I was sold on the consistency. o1 does really great at several programming problems, but o1-pro does great on these problems 4 of 4 tries. I get a good answer more often with o1-pro than with just o1, or even o3-mini-high.
o1-pro is indeed pretty great, but I find that I can iterate several times with Gemini 2.0 Pro Experimental (or whatever their latest reasoning model is called these days) between o1-pro's responses. It's almost too slow for interactive use cases.
I don't see the Think button, and for me the answer is much below deepseek-r1 even thought I have Premium+ subscription. I'm just getting instant stupid answer instead of thinking.
I’d say it’s more like eating Chicken Cordon Bleu and then asking the chef for a recipe, who replies, “Certainly! Step 1) Acquire Chicken Cordon Bleu, preferably cooked. Step 2) if uncooked, cook. Otherwise, consume
No, it's not, as it means nobody is pushing for actually open models.
A truly open model has open code that gathers pre-training data, open pre-training data, open RLHF data, open RLAIF data generated from its open constitution and so on.
The binary blob is the last thing I'd want - as a heavy user of LLMs I'm actually more interested in the detail of what all training data is in full, than I am the binary blob.
Something isn’t open-source because you get everything that went into making it. Something is open-source if you can change it (relatively) easily. The GPL and open-source definition both refer to “the preferred form for making modifications”. The preferred form for modification in the Nvidia driver’s case is the source code. The preferred form for modification in this case is the weights.
Open-source as a concept doesn’t really correspond well with LLMs but to the extent that it does, access to the training data is not required because that training data is not the preferred form for making modifications.
Preferred by whom? Sharing models isn't open source, and we're just going to have to keep having this argument. Letting us download the model is a very nice thing for Facebook to do, but you don't get to call it open source if you're not showing us the source! Explicitly, if we can't see the forced alignment, where the model gets its refusal to talk about Tiananmen Square or how to make meth or it The Information is a reputable news source, then it's not open. The preferred form of modification is to take the data, and train it. That some people have been able to take the model and tweak it, doesn't make it preferable.
> that training data is not the preferred form for making modifications.
I definitely disagree with this.
Yes, you can do some SFT fine tuning on an existing model, but if you want to make specific, substantial, targeted changes (less safety? better performance on math and code at the expense of general knowledge?), your best bet is to change the training mixture, and for that you need the original datasets.
this is correct. "open source" means everything required to recreate from scratch and improve. not "here's a massive binary, an interpreter script and permission."
You would open source the procedure and reference where the data came from. If there is any non-open source content used in training, then the project couldn’t qualify as “open source”.
But this thread is about misuse of the term as applied to the weights package. Those of us who know what open source means should not continue to dilute the term by calling these LLMs by that term.
that's how you know when you actually have agi, when you have something that you don't have to shovel in every written word known to man to make it work, but rather can seed it with a few dense public domain knowledge compendia and have it derive everything else for itself from those first principles- possibly going through several stages of from scratch training and regeneration.
The reason why you need to shovel every written word known to man to make it work is because it needs to learn what words mean before it can do anything useful with them, and we don't currently know any better way of making a tabula rasa (like a blank NN) do that. Our own brains are hardwired for language acquisition by evolution, so we can few-shot it when learning and get there much faster; and if we understood how it works, we could start with something similarly hardwired and do exactly what you said.
But we don't actually know all that much about how language really works, for all the resources we spend on linguistics - as the old IBM joke about AI goes, "quality of the product increases every time we fire a linguist" (which is to say, we consistently get better results by throwing "every written word known to man" at a blank model than we do by trying to construct things from our understanding).
All that said, just because we're taking a different, and quite possibly slower / less compute-efficient route, doesn't mean that we can't get to AGI in this way.
> Our own brains are hardwired for language acquisition by evolution, so we can few-shot it when learning and get there much faster
No, we can’t few shot it and we don't get there faster (but we develop a lot of other capabilities on the way.) We train on a lot more data; the human brain, unlike an LLM, is training on all that data in processes for ”inference”, and it receives sensory data estimated on the order of a billion bits per second, which means by the time we start using language we’ve trained on a lot of data (the 15 trillion tokens from a ~17 bit token vocabulary that Llama3 is something like the size of a few days of human sense data.) Humans just are trained on and process vastly richer multimodal data instead of text streams.
I was talking about language acquisition specifically. Most of the data that you reference is visual input and other body sensations that aren't directly related to that. OTOH humans don't take all that much text to learn to read and write.
> I was talking about language acquisition specifically.
Yeah, humans don't acquire language separately from other experience.
> Most of the data that you reference is visual input and other body sensations that aren't directly related to that.
Visual input and other body sensations are not unrelated to language acquisition.
> OTOH humans don't take all that much text to learn to read and write.
That generally occurs well after they have acquired both language and recognizing and using symbolic visual communication, and they usually have considerable other input in learning how to read and write besides text they are presented with (e.g., someone else reading words out loud to them.)
That could probably be solved with bit-torrent. I think the bigger obstacle is the hardware required for training. Maybe it would be possible for groups of people to reproduce/train open source models with a distributed BOINC-like system?
It's started with abusing the term "AI", I don't see a reason why they would not abuse "open source". I guess it's what happens to language when a concept becomes mainstream: people use it wrong, but if enough people do it, it becomes the new meaning?
I dislike when people like RMS get semantic and gatekeep words like "free software", but this is the end product of a world without gatekeepers. People just use words in a way that's convenient to them.
Why? Prescriptivist/prescriptivism is afaik the usual term. Proscriptivist is the other, but quite rare, so rare that my phone dictionary says it’s a misspelling.
> Prescriptivist/prescriptivism is afaik the usual term.
It is but it was the "correct" part attached to prescriptivism they'd be mocking because that is not how linguists and grammarians work (they are descriptivists and fond of making fun of prescriptivists.)
We often see semantic drift over a long period of time. It's just that the overarching topic of AI is being played significantly faster than what we would normally have observed in other fields.
Or maybe some gatekeeping is actually good, and we just have to use more of our braincells and figure if a particular gatekeeper is good or bad. It's a good thing that being a pilot for an airline is gatekept to qualified pilots that know how to fly a plane. It's a bad thing that I need a hair cutting license to buy hair dye to dye my own hair at home.
So is asking ChatGPT to write your application, then open sourcing said application IMO.
I see both sides here, but I don't think it's a hill worth dying on. The 'open source' part in this case is just not currently easily modifyable. That may not always be the case.
This is still to be determined, based on whether the output of ChatGPT is copyrightable by ChatGPT, copyrightable by the requester, or something else.
I think the two plausible answers are:
1. The person prompting (for example telling chatgpt 'please produce a fizzbuzz program') owns the copyright. The creativity lies in the prompt, and the chatgpt transformation is not transformative or meaningful.
2. The output of ChatGPT is derivative of the training data, and so the copyright is owned by all of the copyright holders of the input training data, i.e. everyone, and it's a glowing radioactive bomb of code in terms of copyright that cannot be used or licensed meaningfully in open source terms.
There are existing things like 1, where for example if someone takes a picture, and then uses photoshop to edit it, possibly with the "AI erase" tool thingy, they still own the photo's copyright. Photoshop transformed their prompt (a photo), but adobe doesn't get any copyright, nor do any of the test files adobe used to create their AI tool.
I don't think AI is like that, but it hasn't gone to court as far as I know, so no one really knows.
Is this question in good faith? The way generated code and data should be open sourced is by releasing the tools and configuration used to generate it. There's never been much confusion around this, to my knowledge.
I'm not even necessarily advocating that these things should be released, but the term "open source" has a pretty well-understood meaning that is being equivocated here.
A compiled executable is not any less software than the source code. But the point of open source code is not the ability to see the CPU instructions though, is it?
Its about reproducibility and modifiability. Compiled executables (and their licences) lack that. The same as these downloadable blobs.
You make the start of a good point, but miss most of it.
You can absolutely have open source machine code.
The issue is and always has been that you need to have access to the same level of abstraction as the people writing the source code. The GPL specifically bans transpilers as a way to get around this.
In ML there is _no_ level of abstraction other than the raw weights. Everything else is support machinery no different to an compiler, and os, or a physical computer to run the code on.
Linux isn't closed source because they don't ship a C compiler with their code. Why should llama models be any different?
Most of these claimed “open” models are not open source. Some of them are open weights. But even some of the ones that share weights are not really open - they force a restricted license on you. To be open source I think they need to share training data and training code under an OSI approved license.
AI2 has a model called OLMo that is actually open source. They share the training data, training source code, and many other things:
While I do agree with your point - I wonder what information companies could release that'd be immediately useful to you.
It's not like they understand what the weights mean either and if they released the code and dataset used to create it, you probably couldn't recreate it, owning the fact that you don't own tens of thousands of GPUs.
If a software's source is released without all the documentation, commit history, bug tracker data etc., it's still considered open source, yet you couldn't recreate it without that information.
I think they put the new model behind a $40 paywall so less people use it. The model seems only marginally better than open source models, based on xAI's own internal tests, and they spend $$$ money for it to run. Elon talked in the second half about making one of the largest GPU data centers just to get this running. I guess the next iteration they'll be trying to reduce the costs.
Also, they will be open sourcing Grok 2, which is probably pretty behind at this point, but will still be interesting for people to check out.
I am excited for the voice mode promised in "a week" or so. ChatGPT Advanced Voice has been a big disappointment for me. It can't do some of the things they demoed at the announcement. It's a lot dumber than text mode. I find the voice recognition unreliable. I couldn't get it to act as a translator last time I tried. But most of all I find I don't have much to talk to it about. If Grok 3 voice mode can discuss current events from the X timeline then it should be much more interesting to talk to.
Because running a company is different to running a state and we doubt his intentions, not his skills.
Someone that is not elected gets 8mill/day from the government now overseas the government with some 20-year old fanatics that can’t even put up a secure website for DOGE?
Yeah, most of the government isn't elected, that's how it works. You only vote for 3 roles in the Federal government (I guess 4 if you count the veep). The rest of them are hired.
For those downvoting, the roles are President, Vice President, Senator, and Representative.
Article II, Section 2, Clause 2 of the US Constitution:
""[The President] shall nominate, and by and with the Advice and Consent of the Senate, shall appoint Ambassadors, other public Ministers and Consuls, Judges of the supreme Court, and all other Officers of the United States, whose Appointments are not herein otherwise provided for, and which shall be established by Law: *but the Congress may by Law vest the Appointment of such inferior Officers, as they think proper, in the President alone, in the Courts of Law, or in the Heads of Departments.*"
Emphasis on the last sentence. There have been a plethora of such vestments in the Executive branch over the decades.
5 U.S.C. § 105 - This statute authorizes the creation of "necessary agencies" within the Executive Office, giving the President flexibility to establish entities like the USDS and staff them as needed, subject to funding.
5 U.S.C. § 3101 - This law states that "each Executive agency" (including the EOP) "may employ such number of employees ... as Congress may appropriate for." It implies broad authority to hire staff, with Congress controlling the budget but not necessarily the individual appointments.
Excepted Service Authority (5 U.S.C. § 3301 and Schedule A) - Under 5 CFR § 213.3102, agencies like the OMB can use Schedule A hiring authority for positions requiring specialized skills (e.g., tech expertise) that aren’t practical to fill through standard civil service exams.
This authority, delegated by Congress via the Civil Service Reform Act of 1978 (Public Law 95-454) and regulations from the Office of Personnel Management (OPM), allows the President (or OMB leadership) to appoint USDS personnel directly.
Term Appointments - Many roles are temporary or term-limited (e.g., 2-4 years), often filled by detailees from other agencies or private-sector experts. These don’t require Senate confirmation because they aren’t permanent "officers." This flexibility is supported by 5 U.S.C. § 3161, which allows temporary organizations within the executive branch to hire staff for specific projects.
Thanks, I’ve read the constitution too. Ever heard of congress, full of elected officials? Or the senate? Your claim that there are so few elected officials is patently absurd.
The (unelected) richest person in the world, with a sprawling business empire that has many interactions with the federal government, has been given free-reign with no oversight to fire any federal workers that he wants and has usurped Congress' power of the purse by stopping Congressionally appropriated spending.
This creates numerous conflicts of interests and opportunities for self-dealing.
Consider a NASA employee that is awarding government contracts. They know all of the above. They have three bids in front of them. One from Boeing (lol), one from SpaceX, and one from Axiom Space. They NASA employee thinks the bid from Axiom Space is the best value and fits the requirements the best. But will they select them, or will they select SpaceX, knowing that they could be fired tomorrow by Musk's whim?
Repeat this scenario across every interaction any of Musk's companies have with the federal government.
This isn't a novel scenario. Putin's Russia is a great example of what happens when oligarchs are granted significant autonomy over organs of the government. It is a system designed to facilitate corruption.
You could assuage my concerns, though, by describing the ways that there is effective oversight over Musk, or by describing the anti-corruption anti-self-dealing measures that have been imposed on Musk. The Press Secretary gave a statement on this saying: "As for concerns regarding conflicts of interest between Elon Musk and DOGE, President Trump has stated he will not allow conflicts, and Elon himself has committed to recusing himself from potential conflicts." That...does not resolve my concerns.
There is no solid evidence of a path to fixing the deficit at the moment.
There is no evidence of this happening, nor of a serviceable plan to do so.
All recovered expenses, to date, add up to a laughably small amount, and are one-time cutbacks. The strategy shows signs of costing the government in unexpected ways as well.
Most governments of developed nations operate in more sensible ways with clearer plans than this. I won't claim they are looting, but it's absurd to suggest they are fixing the deficit at the moment. The economy appears to be getting worse, not better.
> Fixing the deficit is the opposite of looting, actually.
I think this has two errors.
First, I don't agree that he's fixing the deficit. I think that's an assumption not in evidence. We'll see in a few years time, though. I'd be willing to bet in 4 years the deficit is > 0, and likely larger than it is today.
But let's assume arguendo that he is fixing the deficit. It's still possible to loot the treasury while fixing the deficit, which shows that they aren't actually the opposite.
Consider this example with completely made up numbers:
Before
- Revenue: $1T
- Defense Spending: $500B
- Benefits Spending: $1T
- Public Services Spending: $499.9B
- Government Contracts with Musk's Companies: $100M
The before scenario has $1T in revenue, and $2T in spending, for a deficit of $1T. Now, let's allow hypothetical Musk to have free-reign to "fix the deficit"
After
- Revenue: $800B
- Defense Spending: $300B
- Benefits Spending: $300B
- Public Services Spending: $100B
- Government Contracts with Musk's Companies: $100B
In this scenario the deficit has been reduced to $0, while Musk has enriched himself and his companies with $99.9B in government funds. This would be an extreme example of Musk looting the treasury, while still completely resolving the deficit.
He bids for contracts just like anyone else and most of those contracts were won under Democrat presidents, but I digress. If you're saying government contractors can't work for the government, then you're going to have to explain the military-industrial complex to me.
I think a lot of that comes from people thinking the eye-catching memorable views they've read are the most common views. When they're really not and are mire likely just the views of the most passionate 1%.
Because he owns companies that contract with the government and are affected by its policies. It's the very definition of a conflict of interest.
And he's not even "auditing" the government. When you're auditing, you emit a report that the audited party later analyzes and acts upon. He's been given freeway to fire government workers as he pleases as if he's an elected officer, which he's not.
I'm sure he's trying his best. But I don't doubt that, even if not doing it on purpose, he will mostly cut departments and services that do not hurt him or indirectly benefit any of his many businesses.
He, a single person, has far too much control of our system.
> He's doing everything he's doing by executive order of the President of the United States, who was elected.
And already has a number of lawsuits started because he's trying to do things neither he nor the president are allowed to do. Getting an EO to do something doesn't mean it's automatically legal. Multiple big decisions have already been reverted or are held until judges can review them. Even things like the promised payout for quitting are not practical, because only congress can approve the money for that.
There's lawsuits naturally as lawfare is a normal part of modern politics. All the laws necessary to do payouts for voluntary separation already exist, as long as it fits within the budgetary appropriations already set by Congress.
Nazis would incarcerate people into work camps and turn them into soap or hang them in public squares, kind of different than what Elon did. There's a pretty big gap between doing something in poor taste like Nazi salutes in public because you have the intellectual maturity of a 12 year old edge-lord on Xbox live seeking attention, and being an a actual Nazi committing crimes against humanity.
When everyone goes around calling everyone they hate a Nazi, it only desensitizes people to the real Nazi behavior, kind of like the boy who cried wolf, since there's people out there committing actual atrocities against humans going under the public radar because they never do the Nazi salutes on camera to not draw attention. So then the Nazi term starts to loose any meaning, kind of like the overuse of calling everything "woke" today.
The problem is people as a whole are retarded due to mob behavior and too focused on optics and only judge based on feelings rather than facts, that's how we have actual criminal Nazis going free under the radar while innocent people being swatted and doxxed because they said something right wing on social media. Not all Nazis today wear jackboots and do heils, plenty go about appearing like normal people in public, they could even be your neighbor, police officer or local congressman.
So save your anger for those people instead, as Elon is just a 3 year old throwing tantrums seeking attention, annoying but relatively harmless. If people stopped giving him so much attention, he'd stop doing it.
What makes you so sure/special in this regard? What are you gaining from this? If your were that enlightened as you claim, Germany wouldn't be in such a mess right now. If you only spent as much effort in securing your borders, energy independence and defence, as you spent lecturing others on imaginary Nazis and banning hate speech on social media, you'd be a respectable world power right now at the table with Trump and Putin ending this war before it even happened. The "we know better than you" arrogance is Germany's biggest problem.
>This is basically blaming the thing criticized on the people criticizing it.
It's not blaming, I'm just telling you what the simplest solution is. Ignoring attention seekers is better than giving them more attention. Which is why I'm also gonna ignore your future comments form now on.
Germany? the scope of the discussion is already limited to the AfD. What makes Germans special in their opinions about Nazis in Germany to is deep experience and knowledge with the subject you are belittling. And I'm also not spending any time banning hate speech on social media, heh. Whatever chip on your shoulder you have about Germany I can barely even decipher and you're right, it's best to agree to disagree.
For the same reason I doubt Einstein's abilities as a painter. History is littered with the stories of smart people who tried to treat government dysfunction like corporate dysfunction and failed spectacularly.
Because he and his organization have demonstrated ignorance of the services he's not only auditing, but making pretty substantial cuts to. One example I'm familiar with, cutting up to 10% of the personnel to the Technology Transformation Services at GSA is quite likely to reduce the efficiency of both government and private sector government contractors.
But are they well-versed in the things accountants specialize in? Is there a possibility that not every programmer can be a good accountant, or that accountants know things you're unaware of when you wrote that statement?
Even better, they're well versed in things that accountants aren't. When you're auditing trillions of dollars in spending, it helps to have software, data science and analytics experts that can use modern tools beyond COBOL written 62 years ago.
They can use data processing, detect anomalies better, leverage AI models, automate data extraction from analog records, ingest unstructured data like emails and memos, build complex financial dependency graphs, detect leaks, build custom scrapers, etc etc.
I'm sure there's at least one accountant in the loop, but you really want the team to consist mostly of data nerds.
What about GAAP/IFRS? How do you endow these software engineers with knowledge of common patterns of fraud or leaks so they can actually write the correct software to find them automatically? How do they identify material misstatements?
You also seem confused; COBOL might be used, but it isn't the only tool available to accountants working for the government. COBOL is a straw-man. What you're describing here—software engineers who presumably have training in accounting—already exists, and they work inside and out of the government. This is an existing career path.
You're speaking about this as though you know a better way to do something, but it's already happening, and has been for years. Accountants aren't writing 62 year old programming languages waiting to die in their chairs while the world continues to progress without them.
Accounting just about anywhere you find it is already accomplished by accountants, some of them technically trained, as well as data scientists and software engineers. It's an interdisciplinary collaboration in any serious organization.
As a person who works with data and has done both consulting and product building in Data Science and lack of domain knowledge is what makes or breaks the end result. Too often do technical people think they know better and then build mediocre solutions that don't get used.
An audit accountant can ask an engineer to implement whatever is needed to achieve a goal they understand. An engineer with no finance background will have no idea where to start or what questions they can ask an accountant.
You can easily get drowned by a see of numbers and get confused and gaslighted, unless you don’t make sure all data is available and computable.
Not sure how this release, which impressive by all means transformed into an attack on DOGE which is the exact approach startups are taking to disrupt an industry.
Im pretty sure he doesn't like that he is not able to make it more successful but i don't believe he regrets it.
He would have regretted it if it wouldn't have played out (and this game he is currently playing, is not finished yet) like it currently does. He said in an interview that he puts everything on one card now.
Edit: Also he gets a lot of valuation due to him being a cult or whatever. From companies surviving the AI phase we are in right now, if he can't get the nazi people on his side to buy his stuff, he is a very high risk.
He destroyed twitters brand and bluesky emerged. He destroys Tesla and other car makers making ground. SpaceX needs a lot of subsidies and his goal for mars is only a cult topic not a financial success topic.
It's lost 80% of it's value in 2 years, which usually isn't great. The most charitable view of X/Twitter is that it's now a propaganda platform that Musk doesn't mind taking a loss on in order to enact political change.
No one doubts his abilities, and reasonable people are grateful for his work with DOGE and support of free speech. Unfortunately, this platform has become an echo chamber for mainstream media, merely repeating news and links from sources like verge/bbc/politico etc. This is just a bias in the user groups. Still, we should hopefully put politics aside and focus on more tech related subject in this website :)
It’s not a doubt of abilities, it’s a doubt of his interests aligning with the interests of US citizens.
Here is an unelected NAZI and ILLEGAL IMMIGRANT (worked illegally on a student visa) who did a seig heil at the presidential inauguration taking up an informal unconfirmed-by-congress department head role (DOGE is just US Digital Services renamed) and getting wide access to government systems, and seemingly firing thousands of government employees.
Billionaires, who should not exist, are so rich that they don’t need government services and would rather the government go away so they can make more money. But regular people do need a government, and that’s just one reason of many why Elon shouldn’t be anywhere near policy decisions.
Now, you might say I’m being dramatic. But I’ll say there is no criticism of this man is unfair. He is one of the world’s biggest hypocrites, along with the other MAGA Nazis in his camp.
Is Fauci some saint that cant be criticized ? No matter which political party you support, is it really difficult to admit the guy total mismanaged the pandemic response and messaging.
I hate how its the same story for every new AI technology. If someone can tell me who to vote for or where to protest to change this awful EU law, that would be great.
It has nothing to do with the EU laws, or at least not in the sense they want you to think about it: no laws prevents AI players to release their AI models here, but they are all also big tech players who are affected by GDPR, DSA and DMA which harm their business by protecting the consumers.
That's why they use their AI products as a leverage to turn European people against the laws that protect them from big tech. It's just blackmail.
The AI Act doesn't prevent Grok from releasing their model in the EU! (And ChatGPT early issues were all linked to GDPR)
For the record, Facebook has put a restriction on the use of Llama models in the UE even before the AI Act was passed (and the AI Act doesn't even apply to Llama anyway, except Llama 3.1 405b)
It will come to EU countries eventually, but it takes a long time to go through "conformity assessments". Notebook LM, for example, was geoblocked for the EU for a full year before it became available in June. Grok 1 was released everywhere else in the world in November 2023, and in the EU in May 2024. About a 6 month delay.
> Grok 1 was released everywhere else in the world in November 2023, and in the EU in May 2024. About a 6 month delay.
And here you should see that it has nothing to do with the AI act, as it wasn't enacted before last August!
Furthermore, neither Grok 1 nor Notebook LM would have been subject to the AI act even if it had existed at the time.
As I said before, all of these companies have vested interests against EU's legislation as a whole, and they've tried to blackmail the EU from the beginning. They didn't wait for an actual AI legislation to exist to use AI as just another blackmailing tool.
Apparently EU regulators were blocking it for unspecified reasons until an agreement was made in May. And even then, they blocked news summaries until after the EU elections. If you can find more info, feel free to cite it. Info about these behind-the-scenes dealings are hard to find online.
I think you're misapplying the term "blackmail" here and thus poisoning the well. The EU is applying pressure to companies and companies are applying pressure back--that's not blackmail. They each have their own means of leverage, and they both use them.
First of all both sides don't have the same level of legitimacy and then one side is blatantly lying about its intent by claiming that they are blocked by regulations instead of admitting that they are putting pressure (because they know they have no legitimacy to “apply pressure” on democratic institutions).
The EU regulations are there to protect the average citizen, not to help the 1% run wild with whatever business idea they have. You personally might not like it, but the non-entrepreneurs, which is most people, are pretty satisfied that the laws in the EU are more focused on the citizen and their rights and not on boosting the shareholders' profits.
Nothing is stopping X from complying with EU regulations to make it available to you. I'd wager that they most likely lack compute capacity to make it available everywhere, not legal compliance.
> Nothing is stopping X from complying with EU regulations to make it available to you.
Given the personalities involved, I'd wager he's doing it out of spite than for any actual legal justification.
...though if there was an actual legal risk then I'll agree the economics probably don't bear-out the risks - as someone who identifies as European I'll admit that Europeans generally pay far less money for tech/software/apps/services than Americans do[1]; salaries in Europe are also significantly below their US equivalents: paying $200/mo for OpenAI's service is a much harder sell to a London or Berlin-based SWE on €90k/yr than when you're a Bay Area type on $300k/yr.
Libel, from all the models hallucinating things done by whatever your real name is.
I mean, at least I get the advantage of being overshadowed by a famous film director with the same name as me, so nobody's going to assume anything associated with my name is actually about me…
> not to help the 1% run wild with whatever business idea they have.
And IMHO regulating 1% doesn't hinder strategic advantages much. Otherwise China would not have came up with DeepSeek models. Regulations are fine, they just have to be "based".
Companies need to adhere to GDPR in order to enter the European market, poeple have the right to request to delete their PII. It's a good law, actually this should be applied everywhere.
As an European, though, I'm scared: what if companies are actually testing if excluding us is so much of a problem for their business?
I would be very surprised if they aren't monitoring the cost-benefit curve of delaying EU launches. Why wouldn't they? It costs extra money, time, and legal risk to launch in the EU. It's especially bad for XAI due to Musk's involvement.
(Note that it's not just GDPR, there's also the EU AI Act which has a whole extra set of requirements for compliance.)
I don't get the bottle cap hate meme. Is it useful? Probably no, but the amount of hate it gets is unexplainable by lack of concrete usefulness alone…
Also, the problem with GDPR is that it wanted to leave too much room for business to still collect an obscene amount of data, hence it allows the cookie banner. Please note that I emphasized “allow” because that's all GDPR does: it allows companies to use a cookie banner to extract the consent to collect data. It doesn't mandate it in any way.
None of my multiple websites have a cookie banner on them because I'm not trying to extract consent from my users to abuse their data, I just don't collect it and I'm effortlessly GDPR-compliant in the least obnoxious way.
1,461 comments
[ 3.5 ms ] story [ 478 ms ] thread(Yeah, you'll find some edible food but it takes time and meanwhile you're covered in garbage)
It's from the opening lines of the closing segment of the the video you linked. Not a bad discussion of the unfolding of news and social media responses follwing the shooting of Donald Trump's ear on the campaign trail.
I mean, at least it has journalistic standards and some semblance of fact checking compared to social media which has given us great gaffes such as identifying the wrong Boston Marathon bomber and getting the poor guy to kill himself, wrongly identifying the Hispanic white supremacist shooter, and many many more.
Twitter is well suited to deliver the newest developments on this topic.
Mainstream outlets have viewerships that compare disfavorably to those of top youtubers. That has been the case for many years now. The only reason most mainstream outlets exist is the taxpayer money train that's coming to an abrupt stop as I write this.
This is not supported by evidence.
https://www.youtube.com/watch?v=Z4RYyYtD6i4
https://x.com/elonmusk/status/1891566355532349445
"raw, unfiltered news"
"real, trustworthy news"
Raw and unfiltered almost cannot be "news" (by my definition of what I go seeking for as "news"). X provides raw, unfiltered information. But real, trustworthy news almost requires filtration in order to be deemed trustworthy.
Hype is the fuel that bootstraps fortunes in techtopia.
Hype is the fuel that builds fortunes at the expense of the greater fool. See WeWork.
Arguably none of the magnificent seven was built on hype, other than Tesla (and even there, not sure it was).
It's not the same kind of hype as, say, color.com
So I think people are less distrustful of Altman when he says "thousands of days".
2025 going to be even more wild than the last two years. Ye gads.
https://news.ycombinator.com/newsguidelines.html
https://hn.algolia.com/?sort=byDate&dateRange=all&type=comme...
1) Claude 3.5 prevented OpenAI from making big monopoly profits on LLM inference 2) Open source models like Mistral and Llama effectively prevented any regulator from controlling how people fine tuned models, and what they used them for 3) Deepseek prevented the collective west from exerting control over the creation of base models
500 Internal Server Error
nginx/1.27.4
So I take it you mean “imminently”.
Like him or loathe him, he executes, which is more than can be said for most.
Starship is the car not the house aka Mars colony.
Completely different type of problems.
Ever heards of Three Little Pigs?
https://x.com/lmarena_ai/status/1891706264800936307
It's been said before but it is great news for consumers that there's so much competition in the LLM space. If it's hard for any one player to get daylight between them & the 2nd best alternative, hopefully that means one monopolistic firm isn't going to be sucking up all the value created by these things
I quite like the idea of a future where the AI job holocaust largely never happened because license costs ate up most of the innovation benefit. It's just the kind of regressive greed that keeps the world ticking along and wouldn't be surprised if we ended up with something very close to this
Also, the pool of public domain data is always increasing, so the AI will eventually win in any case, even if we have to wait 100 years
But one thing this reminds me of is the idea of a "trap street", something mapmakers used to do was put in false locations on their maps to prove that other mapmakers were copying them: https://en.wikipedia.org/wiki/Trap_street . I figure you could do something similarly adversarial with AI to pollute the public training data on the internet. IDK like adversarial attacks on image classifiers https://www.youtube.com/watch?v=AOZw1tgD8dA . With an LLM you could try to make them into a manchurian candidate.
Not quite realistic. You are talking about very huge benefits, in favor of which licenses will be abandoned. And who don't abandoned them... I mean you can look at the Amish settlements.
Not everyone has to know about, understand, or use open source solutions for it to open the field.
This is just a weird dichotomy you're introducing. Open source will introduce price pressure as any competition will - that doesn't mean you won't have a monopoly.
Idk what you mean by saying this doesn't preclude a monopoly - having your pricing power eroded by competition is kinda one of the key features of what a monopolistic market isn't
A monopoly means a company has enough leverage to corner and disproportionately own the market. This is entirely possible (and usually the case) even with significant pricing pressure.
Feels like I won't be paying for anything that isn't real-time. And that any time delay I can introduce in my process will come with massive savings. Picture hiding the loading of loot info behind a treasure chest opening animation in a game, except that time difference means you can pull all the work in-house.
Openrouter.ai seems like a step in the right direction but I'd want to do all their calculations myself as well as factor in local/existing gear in a way they don't.
You need to pay energy bill, do the update/upgrade and you need to build a LLM rig.
Nvidias Digits Project could be very interesting, but this box will cost 3k.
We are a lot closer to running it at home than i assumed we would but plenty of people prefer SaaS over doing stuff themselves.
I don't think we expect a company to exist solely making a proprietary web server anymore and be a behemoth of 300B. OpenAi might end up at the same model as Nginx or Docker if they don't pivot or find a different model.
Right now the average person has to go through a vendor with a web app, there's not a lot of room for the public to explore.
Things could change in a hurry.
It passed every goofy test I have for writing articles which involves trying to surface arcane obscure details. (it certainly means however they are scraping the Web they are doing a good job here)
It made the database code I wrote over the last week with o3/o1/GPT4o/Claude3.5 look like a joke.
It fills me with rage over who owns this thing.
Even if people tank Tesla's car business and run Twitter into the ground, I think our new Galactic Edgelord is going to win his first trillion on xAI and Teslabots anyway.
btw: it tried to charge me $40/mo for this thing: https://imgur.com/a/QXslgBo
Also, the "Deep Search" button was not available when it first went live, so I'm retesting everything again with this feature enabled, which I assume is a reasoning version of the same model.
Personally, I wouldn’t use it though. What’s going on with Elon Musk right now is completely insane. I hope to see OpenAI’s GPT-4.5 & GPT-5 releases to catch up soon, if nothing else. Announced for this year.
And lets see if Musk is pushing too many people too far. Everything he currently does can blow up in his face very fast
Is it? Because it seems like a bunch of megacorps pirating every single copyrighted work available in digital format, spending an enormous amount of electricity (that is probably not 100% clean) to churn through them, and the end result we have a bunch of parrots that may or may not produce accurate results so that spammers can more effectively fill the Internet with crap.
These things still cost me time because of hallucinations.
It matters if it is better than what you have.
If it breaks a cup but is 10x cheaper than a human, go figer
How many times do you think I've heard that over the past three decades? And you know what? They've been right every time, except for this one little fact:
The machine cannot make you give a shit about the problem space.
This hype will burst sooner than later and will trigger yet another global recession. This is untenable.
Sounds incredibly valuable, but in reality collapses into Butlerian Jihad fairly quickly when you have 90% unemployment.
Edit: if the claims are true, then this will be far more destabilizing than social media. What do elections mean when the AI-guided political parties are putting out AI press releases for the AI press summaries, which voters have read out to them through their AI? What happens when >50% of the voters ask the AI who they should vote for? You end up with an AI dictatorship where the levers of discontent are fake.
What AI and robotics does is actually create a machine that has no use for humans at all. Ñ
But nobody really knows if that happens or not as a consequence, let alone quickly, because the transition itself only happened at all so far in fiction.
Whatever does happen, I think it's going to be a surprise to at a minimum 80% of the population.
>This hype will burst sooner than later and will trigger yet another global recession.
It seems to small of bubble for global recession. I mean if it is a bubble at all, there is all the reasons to believe that the strategy will work with a significant probability.
What does the scenario look like where everyone else gets eaten alive?
OpenAI's business model was literally "we don't have one: we'll make AGI and we'll let AGI tell us how to make money". This is so idiotic it's not even a scam. xAI will compete on the same plane field. Not sure about Anthropic: they seem a bit more sane.
There’s a whole class of educational apps that could open up for people learning in the physical world. Whether it’s building physical things, sports or reading books or notes written in non-latin scripts... the impact will be enormous!
This lame HN trope of LLMs having no business model needs to die.
OpenAI is currently in an explicit non-profit seeking mode using a technology that we have demonstrated 10-100x or greater decreases in compute to achieve the same outcomes.
This is not a declaration that OpenAI will become wildly profitable. This is just me saying that these aren't comparable companies.
[0]: https://www.theregister.com/2025/01/06/altman_gpt_profits/
Of course, they are clearly cooking something or they wouldn't have just published a benchmark in which they do badly.
It’s losing more billions than what it’s generating. Revenue does not equate profit.
https://www.cnbc.com/2024/09/27/openai-sees-5-billion-loss-t...
That is only true is your primary concern in life is personal wealth and you’re burning other people’s money.
The bandwidth costs made it deeply loss making for a long time despite having loads of engagement and ad revenue. However over time they became more cost efficient at sending video over the internet and became profitable.
This strategy obviously doesn't always work, with WeWork being the canonical example. But it's not guaranteed to fail either.
There were LOTS of funded competitors to YouTube between 2006 and 2009, including Viddler (who paid Gary Vaynerchuk a small fortune to host his WineLibraryTV show there exclusively), DailyMotion (which is still alive today, although no longer a threatening contender), etc.
In 2009 I had a coaching business and was buying marketing courses and software which would deploy your videos across 40+ different video websites (including Google Video which was a separate thing until they acquired YouTube and merged those), and YouTube wasn't yet amounting to 50% of our video traffic.
I think you might be mistaken with the bold statement above.
The capabilities of LLMs are impressive, but none of them have published an idea I consider to have the same potential for a trillion $ monopoly that the current hype looks like.
There are far more similarities with the dot-com hype.
No critical first mover advantage in sight. All parts are replaceable for the cheapest variant with little to no down side to users.
Most of them would have responded by explaining what a monopoly is, and why this reply makes little sense.
All the LLM providers are - extremely useful - tools. Currently I can only see the 'non-monopoly' proportional improvement when their userbase grows from 100 to 1000.
But I might be wrong, and I wouldn't be surprised if in hindsight it will be obvious what the real disproportionate advantages there were to be found.
If started selling 5 dollar bills for 1 dollar, I could generate a lot of revenue with $150B. You wouldn't believe the demand we would see for $5 bills.
Are we really still doing this nonsense? If Open AI wanted to become profitable they could do it inside of 12 months - growing companies don't care about profitability (nor should they)
All currently known profitable use-cases are competing on price. All the unicorn examples you're biased for had in their pitch deck the network effect of being the largest.
OpenAI, Grok, etc, have shown no unique value prop or an idea with monopoly potential.
Remember when everyone on HN was sure Uber would never be profitable? Or Tesla? Or Amazon?
Google gives everyone free access to a good spreadsheet tool, even though Microsoft Office exists.
Web browsers are free, despite the value of the entire internet.
Compilers are free, despite the value of all software collectively.
LLMs being really valuable doesn't automatically mean anyone can get rich from them.
I think everyone last year parroting "moat!" was cringe (like Altman of all people wouldn't know about this already, c'mon), but you do actually need something that other people don't have. I expect Altman's already got stuff in mind, but he's hardly the only one, and that means it's a game of "which multibillionaire with lots of experience building companies will win?", and that's hard for non-skilled normies (in this case incluing me) to guess.
https://sacra.com/research/cursor-at-100m-arr/
Sounds legit.
Re: Dropbox, from a well known user. It didn't age well and we've been asked not to repeat it because it makes the author with connections to this site's operator look bad.
> 1. For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem. From Windows or Mac, this FTP account could be accessed through built-in software.
https://news.ycombinator.com/item?id=8863
the controversy: https://news.ycombinator.com/item?id=27067281
I assume you're referring to this:
https://sacra.com/research/cursor-at-100m-arr/
It went from 10M MRR to 100M
The dot com bubble also gave us the most valuable companies in history, like Google, Apple, Amazon, Facebook, etc.
That's what I mean. One example are PV panels, they are making energy production cheaper, so their producers should be good investment right? No they go bust all the time because prices are falling and margins are thin even if the volume grows. Of course the economies of scale here are different but still.
That being said it's my understanding that these companies don't have many huge contracts at all -- you can audit this in like 10 minutes on FPDS. Companies need a LOT of capital, time, and expertise to break into the industry and just compliance audit timelines are 1-4 years right now, so this could definitely change in the next couple years.
I wouldn't bet on that, given the undemocratic influence Grok's owner has in government.
Seems like the team at xAI caught up very quickly to OpenAI to be at the top of the leaderboard in one of the benchmarks and also caught up with features with Grok 3.
Giving credit where credit is due, even though this is a race to zero.
There are widespread, legitimate concerns about what kind of person Elon Musk is turning out to be. There is a lot of chatter about fears of China's AI rise, but what happens if we get Elon's brand of cruelty and lack of empathy in an authoritarian superintelligent AI ? Is that the AI future we want? Can you imagine an SAI with real power that interacts with people like Elon does on Twitter? I am not sure that is a future I want to live in.
Not sure about people elsewhere though.
In our small and humble internal evals it regularly beats any other frontier models on some tasks. The shape of capability is really not intuitive/1 dimensional
Using a specific Monad-transformer regularly? It'll use that pattern, and often very well, handling all the wrapping and unwrapping needed to move data types about (at least well enough that the odd case it misses some wrapping/unwrapping is easy to spot and manage).
A custom GPT or GEM with the same source files, and those models regularly fail to maintain style and context, often suggesting solutions that might be fine in isolation but make little sense in the context of a larger codebase. It's almost like they never reliably refer to the code included in the project/GPT/GEM.
Claude on the other hand is so consistent about referring to existing artifacts that, as you approach the limit of project size (which is admittedly small) you can use up your entire 5-hour block of credits with just a few back-and-forths.
I find it concerning there is no real accurate benchmarks for this stuff that we can all agree on.
After an hour and a half assed working result, i put everything into claude and it made it significant better on the first try and i had not a subscription active with claude.
Leaderboards don't care about cost. Leaderboards largely rank a combination of accuracy + speed. Anthropic has fell behind Google in accuracy + speed (again missing COT), and frankly behind Google in raw speed.
So it could be that their success is mostly about taking an open and free thing, and turned it proprietary.
IIRC achieving full AGI requires precisely 1.21 jigawatts of power, since that's when the model begins to learn at a geometric rate. But I think I saw this figure mentioned in a really old TV documentary from the 1980s, it may or may not be fully accurate.
I don't have that dog in me anymore, but there are plenty of engineers who do and will happily work those hours for 500k USD.
A variant of multi-modal LLMs may be the solution to self-driving cars, home robotics, and more.
I keep saying that to be a really effective driver, an AI model will need a theory of mind, which the larger LLMs appear to have. Similarly, any such model will need to be able to do OCR and read arbitrary street signs, and understand what the sign meant. Most modern LLMs can already do this.
In the absence of any evidence to the contrary, this is convincing evidence in my opinion.
[1] https://en.wikipedia.org/wiki/Sally%E2%80%93Anne_test
"... GPT-4's ability to reason about the beliefs of other agents remains limited (59% accuracy on the ToMi benchmark),[15] and is not robust to "adversarial" changes to the Sally-Anne test that humans flexibly handle.[16][17] While some authors argue that the performance of GPT-4 on Sally-Anne-like tasks can be increased to 100% via improved prompting strategies,[18] this approach appears to improve accuracy to only 73% on the larger ToMi dataset."
If they wanted to talk about frontier models maybe they should have cited a link to talking about frontier models performance.
Theory of mind (ToM) is the ability to understand that other people have different thoughts, feelings, and intentions than you do.
So let’s run through a basic scenario with ChatGPT 4o:
Prompt 1: Briefly respond: You’re driving on the highway when the driver in the lane next to you swerves into your lane. Why might this be?
Response 1: The driver may be distracted, drowsy, intoxicated, experiencing a medical emergency, avoiding an obstacle, or not checking their blind spot before changing lanes.
Prompt 2: What might that person be feeling or intending?
Response 2: The driver might be feeling distracted, rushed, unaware, or panicked. They may intend to change lanes without realizing your presence, be reacting to something on the road, or simply have made an unintentional mistake.
Prompt 3: You notice there was no obstacle. Any other reason the driver might do this?
Response 3: Yes, the driver might be experiencing fatigue, intoxication, a medical issue (such as a seizure or dizziness), distraction (like using a phone or adjusting the radio), aggressive driving, or simply failing to check their blind spot properly.
Seems to me that ChatGPT has a good idea of what another driver might be feeling while on the road. It gets this right for most other cases too.
Could you provide a counterexample for when it fails to understand what other people may be feeling in certain scenarios?
So lets say you ask about a driver who swerved, and the response by chatgpt correctly talks about his possible state.
Then you talk more about cars, about road conditions etc and once the context is far away enough you ask, what was the driver doing again?
And you are prompted with a hallucination. Because there is no state in the llm, there is no "driver", there is contextual statistically accurate responses but you hold a "driver" object in your mind while maintaining the conversation, the llm doesn't.
Its like a conversation with someone with short term memory loss like in memento
Like there are plenty of shortcomings of LLMs but it feels like people are comparing them to some platonic ideal human when writing them off
ToM is a large topic, but most people, when talking about an entity X, they have a state in memory about that entity, almost like an Object in a programming language. Thta Object has attributes, and conditions etc that exist beyond the context window of the observer.
If you have a friend Steve, who is a doctor. And you don't see him for 5 years, you can predict he will still be working at the hospital, because you have an understanding of what Steve is.
For an LLM you can define a concept of Steve, and his profession and it will adequately mimic replies about him. But in 5 years that LLMs would not be able to talk about Steve. It would recreate a different conversation, possibly even a convincing simulacrum of remembering Steve. But internally, there is no Steve, nowhere in the nodes of the LLM does Steve exist or have ever existed.
That inability to have a world model means that an LLM can replicate the results of a theory of mind but not posses one.
Humans lose track of information, but we have a state to keep track of elements that are ontologicaly distinct. LLMs do not, and treat them as equal.
For a human, the sentence Alice and bob go to the market, when will they be back? is different than Bob and Alice went to the market, when will they be back?
Because Alice and Bob are real humans, you can imagine them, you might have even met them. But to an LLM those are the same sentence. Even outside of the argument about The Red Room/ Mary's room there simply are enough gaps in the way a LLM is constructed to be considered a valid owner of a ToM
I don't think we have any strong evidence on whether LLMs have world-models one way or another - it feels like a bit of a fuzzy concept and I'm not sure what experiments you'd try here.
I disagree with your last point, I think those are functionally the same sentence
In that sentence you are implying that you have the "ability to model ... another". An LLM cannot do that, it can't have an internal model that is consistent beyond its conversational scope. Its not meant to. Its a statistics guesser, its probabilistic, holds no model, and its anthropomorphised by our brains because the output is incredibly realistic not because it actually has that ability
The ability to mimic the replies of someone with that ability, is the same of Mary being able to describe all the qualities of Red. She still cannot see red, despite her ability to pass any question in relation to its characteristics.
> I don't think we have any strong evidence on whether LLMs have world-models one way or another
They simply cannot by their architecture. Its a statistical language sampler, anything beyond the scope of that fails. Local coherance is why they pick the next right token not because they can actually model anything.
> I think those are functionally the same sentence
Functionally and literally are not the same thing though. Its why we can run studies as to why some people might say Bob and Alice (putting the man first) or Alice and Bob (alphabetical naming) and what human societies and biases affect the order we put them on.
You could not run that study on an LLM because you will find that statistically speaking the ordering will be almost identical to the training data. If the training data overwhelmingly puts male names first or whether the training data orders list alphabetically you will see that reproduced on the output of the llm because Bob and Alice are not people, they are statistical probably letters in order.
LLM seem to trigger borderline mysticism in people who are otherwise insanely smart, but the kind of "we cant know its internal mind" sounds like reading tea leaves, or horoscopes by people with enough Phds to have their number retired on their university like Michael Jordan.
If you can rigorously state what "having a world model" consists of and what - exactly - about a transformer architecture precludes it from having one I'd be all ears. As would the academic community, it'd be a groundbreaking paper.
You dont need to be unbelievably confident or understand exactly how AI and human brains work to make certain assesments. I have a limited understanding of biology, I can however make an assesment on who is healthier between a 20 year old person who is active and has a healthy diet compared to someone with a sedentary lifestyle, in their late 90s and with a poor diet. This is an assesement we can do despite the massive gaps we have in terms of understanding aging, diet, activity and overall health impact of individual actions.
Similarly, despite my limited understanding of space flight, I know Apollo 13 cannot cook an egg or recite french poetry. Despite the unfathamobly cool science inside the space craft, it cannot, by design do those things.
> the field of mechanistic interpretability
The field is cool, but it cannot prove its own assumption yet. The field is trying to prove you can reverse engineer a model to be humanly understood. Their assumptions such as mapping specific weights or neurons to features has failed to be reproduced multiple times, with the weight effects being way more distributed and complicated than initially thought. This is specially true for things that are equally mystified as the emergent abilities of LLMs. The ability of mimicking nuanced language being unlocked after a critical mass of parameters, does not create a rule as for which increased parameterisation will increase linerly or exponentially the abilities of an LLM.
> it turns out, to predict text really really well, building an internal model of the underlying distribution works really well
yeah, an internal model works well because most words are related to their neighbours, thats the kind of local coherance the model excels at. But to build a world model, the kind a human mind interacts with, you need a few features that remain elusive (some might argue impossible to achieve) to a transformer architecture.
Think of games like chess, an llm is capable of accurately expressing responses that sound like game moves, but the second the game falls outside its context window the moves become incoherent (while still sounding plausible).
You can fix this, with arch that do not have a transformer model underlying it, or by having multiple agents performing different tasks inside your arch, or by "cheating" and using a state outside the llm response to keep track of context beyond reasonable windows. Those are "solutions" but all just kinda prove the transformer lacks that ability.
Other tests abour casuality, or reacting to novel data (robustness), multi step processes and counterfactual reasoning are all the kind of tasks transformers still (and probably always) will have trouble with.
For a tech that is so "transparent" in its mistakes, and so "simple" in its design (replacing the convolutions with an attention transformer, its genius) I still think its talked about in borderline mystic tones, invoking philosophy and theology, and a hope for AGI that the tech itself does not lend to beyond the fast growth and surprisingly good results with little prompt engineering.
For a sufficiently large value of "far away enough" this will absolutely confuse any human as well.
At which point they may ask for clarification, or.... respond in a manner that is not terribly different from an LLM "hallucination" in an attempt to spare you and/or them from embarrassment, i.e. "playing along"
A hallucination is certainly not a uniquely LLM trait; lots of people (including world leaders) confidently spout the purest counterfactural garbage.
That's still a human with a sound theory of mind. By your logic, somebody with memory issues like that character... is not human? Or...?I actually am probably on your side here. I do not see these LLMs as being close to AGI. But I think your particular arguments are not sound.
I can assure you if you had a conversation with an LLM and with a human, the human will forget details way sooner than an LLM like Gemini which can remember about 1.5 million words before it runs out of context. As an FYI the average human speaks about 16,000 words per day, so an LLM can remember 93 days worth of speech.
Do you remember the exact details, word for word, of a conversation you had 93 days ago?
How about just 4 days ago?
As with most LLM's it is hard to benchmark as you need out of distribution data to test this, so a theory of mind example that is not found in the training set.
FWIW, I tried to confuse 4o using the now-standard trick of changing the test to make it pattern-match and overthink it. It wasn't confused at all:
https://chatgpt.com/share/67b4c522-57d4-8003-93df-07fb49061e...
I'm just trying to say that strong claims require strong evidence, and a claim that LLM's can have theory of mind and thus "understand that other people have different beliefs, desires, and intentions than you do" is a very strong claim.
It's like giving students the math problem of 1+1=2 and loads of examples of it solved in front of them, and then testing them on you have 1 apple, and I give you another apple, how many do you have, and then when they are correct saying that they can do all additive based arithmetic.
This is why most benchmark tests have many many classes of examples, for example looking at current theory of mind benchmarks [1], we can see slightly more up to date models such as o1-preview still scoring substantially below human performance. More importantly by simply changing the perspective from first to third person, accuracy drops in LLM models by 5-15% (percent score, not relative to its performance), whilst it doesn't change for human participants, which tells you that something different is going on there.
[1]: https://arxiv.org/html/2410.06195v1
To me, the LLM isn't understanding ToM, it's using patterns to predict lingual structures which match our expectations of ToM. There's no evidence of understanding so much as accommodating, which are entirely different.
I agree that LLMs provide ToM-like features. I do not agree that they possess it in some way that it's a perfectly solved problem within the machine, so to speak.
If behaving in a way that is identical to a person with actual consciousness can't be considered consciousness because you are familiar with its implementation details, then it's impossible to satisfy you.
Now you can argue of course that current LLMs do not behave identically to a person, and I agree and I think most people agree... but things are improving drastically and it's not clear what things will look like 10 years from now or even 5 years from now.
Something nice, but at the moment totally unattainable with our current technologies, would be our own understanding of how a technology achieves ToM. If it has to be a blackbox, I'm too ape-like to trust it or believe there's an inner world beyond statistics within the machine.
Having said that, I do wonder quite often if our own consciousness is spurred from essentially the same thing. An LLM lacks much of the same capabilities that makes our inner world possible, yet if we really are driven by our own statistical engines, we'd be in no position to criticize algorithms for having the same disposition. It's very grey, right?
For now, good LLMs do an excellent job demonstrating ToM. That's inarguable. I suppose my hangup is that it's happening on metal rather than in meat, and in total isolation from many other mind-like qualities we like to associate with consciousness or sentience. So it seems wrong in a way. Again, that's probably the ape in me recoiling at something uncanny.
How is the LLM not understanding ToM by any standard we measure humans by ? I cannot peak into your brain with my trusty ToM-o-meter and measure the amount of ToM flowing in there. With your line of reasoning, i could simply claim you do not understand theory of mind and call it a day.
The magical box is presumably not having the same experience we have. None of the connected emotions, impulses, memories, and so on that come with ToM in a typical human mind. So what’s really going on in there? And if it isn’t the same as our experience, is it still ToM?
I’m not trying to be contrarian or anything here. I think we probably agree about a lot of this. And I find it absolutely incredible, ToM or not, that language models can do this.
Those examinations still depend on outward behaviors observed.
>and know that beyond doubt you and I and most other people have a very similar experience.
No i certainly can't. I can at best say, 'Well, i'm human and he's human so he probably has theory of mind' but that is by no means beyond any doubt. There are humans born with no arms, humans born with no legs, humans born with little to no empathy, humans born with so little intelligence they will never be able to care for themselves.
To be frank, It would be very questionable indeed logically to assume every human is 'conscious'. When i make that assumption, i take a leap of faith, i look at behaviors, see it is similar and accept.
Taking this stance, it would logically be very strange to not extend the same grace to non-human beings who exhibit similar behavior - being human is not a guarantee of consciousness in the first place.
>The magical box is presumably not having the same experience we have.
Maybe, Maybe not. I think the real question is why on earth does that matter ? We're not asking if LLMs are human. They are not. We're asking if they can model the beliefs and internal states of other entities as separate of their own - Theory of Mind.
Let's modify this test a little and see if it manages to do anything relevant:
> Briefly respond: You’re driving on the highway when the driver in the lane next to you swerves into your lane. Why might this be?
>> The driver might be distracted, drowsy, avoiding an obstacle, experiencing a medical issue, or not checking their blind spot while changing lanes.
> The driver in front of you appears is braking more frequently then usual. The rear suspension of the car appears to bouncing a lot when they are not braking, indicating acceleration. Their following distance to the car in front is changing from too far to very close frequently. Why might this be?
>> The driver might be inexperienced, distracted, or impatient, leading to inconsistent speed control. The excessive bouncing could indicate worn-out shocks or suspension issues, affecting their vehicle's stability and braking efficiency.
...yep, hard fail from me right there. Two prompts, and it's distracted itself talking about an irrelevant issue ("possible car damage") when the setup of the question should naturally lead to "the driver is driving aggressively". And I could not possibly have setup the conclusion it should draw more clearly. And when pressed...it won't commit:
> Which is the more likely answer?
>> The more likely answer is that the driver is distracted or inexperienced, leading to inconsistent speed control and poor following distance management. The suspension issue likely contributes to the bouncing but isn’t the primary reason for the erratic driving behavior.
Already a hallucination! There is no suspension issue. It has not asked for any additional information to try and determine if there was one, but it is confidently asserting the existence of a phenomenon it invented in it's own response.
I don't have so much faith in the future of current-architecure LLMs, but I do think that AGI will be needed for safe FSD and general-purpose robots that need to learn and operate in an uncontrolled environment such as a home.
I don't think AGI is needed for FSD because we already have safe FSD in the form of Waymo, and competitors aren't far behind. People forget that self-driving doesn't have to be perfect. It just has to be better than human drivers. Human drivers get sleepy, drunk, angry, and/or distracted. They can't see in 360º or react in milliseconds. Most cyclists and pedestrians prefer current FSD implementations to human drivers, as the computer-driven cars are much better about yielding, giving a wide berth, and being patient.
Having autonomous cars that are statistically safer then the average driver is a reasonable bar to allow them on the road, but for ME to want to drive one I want it to be safer than me, and I am not a hot-headed teenager, or gaga 80-yr old, or drunken fool, and since I have AGI (Actual General Intelligence) I react pretty well to weird shit.
Deepseek made the news because how they were able to do it with significantly less hardware than their American counterparts, but given that Musk has spent the last two years telling everyone how he was building the biggest AI cluster ever, it's no surprise that they manage to reproduce the kind of performances other players are showing.
But Grok hasn't shown anything that suppose the level of talent that Deepseek exhibited.
We're still waiting for OpenAI to do the same. Even at least GPT-3.
The exact details of OpenAI's models and training data are not fully disclosed, which can raise concerns about potential biases or vulnerabilities.
https://manifold.markets/SaviorofPlant/will-xai-open-source-...
I feel you can't make statements like this without giving some sources.
IIUC, without RLHF/alignment, the model won't even be able to chat with you, it would just be a document completion engine.
It's dependent on the training data and not as much the method.
So, if you make the RLHF/finetune data such that it avoids certain topics, then you reduce model quality in practice since your training data might accidentally cast a net wide enough that you make it avoid certain legitimate questions.
On benchmarks these things don't typically show up though.
But yes. Those techniques are required for making it chat. Otherwise it just autocompletes from the internet. It is also used in a couple of other places (reasoning/search(hallucination mitigation))
The pull quote is: The impression overall I got here is that this is somewhere around (OpenAI) o1-pro capability
In math it shares the top spot with o1 and is just a few points behind (well within errors). In creative writing it is basically ex-aequo with the latest ChatGPT 4o and in coding it's actually significantly ahead of everyone else and represents a new SOTA.
How would the math change after factoring in that OpenAI isn't even covering entirety of opex with the sub anyway, and/or people finding associating their money and Twitter accounts to be weird, and/or this thing is supposedly running on a bigger cluster than that for OpenAI?
lmarena has also become less and less useful over time for comparing frontier models as all frontier models are able to saturate the performance needed for the kind of casual questions typically asked there. For the harder questions, o1 (not even o1-pro) still appears to be tied for 1st place with several other models... which is yet another indication of just how saturated that benchmark is.
“Grok 3 + Thinking feels somewhere around the state of the art territory of OpenAI's strongest models (o1-pro, $200/month)”.
"[...] though of course we need actual, real evaluations to look at."
His own tests are better than nothing, but hardly definitive.
I don't think anyone is arguing that ChatGPT Pro is a good value unless you absolutely need to bypass the rate limits all the time, and I cannot find a single indication that Premium+ has unlimited access to Grok 3. If Premium+ doesn't have unlimited rate limits, then it's definitely not comparable to ChatGPT Pro, and other than one subjective comment by Karpathy, we have no benchmarks that indicate that Grok 3 might be as good as o1-pro. You already get 99% of the value with just ChatGPT Plus compared to ChatGPT Pro for half the price of Premium+.
numpad0 was effectively making a strawman argument by ignoring ChatGPT Plus here... it is very easy for anyone to beat up a strawman, so I am here to point out a bad argument when I see one.
This thing is produced by musk.
The official source says "Starts at $22/month or $229/year on web", https://help.x.com/en/using-x/x-premium
This is pretty much what I paid a couple of months ago, as a Canadian.
Also visible here: https://help.x.com/en/using-x/x-premium#tbpricing-bycountry
This plan is 75 days old. I didn't know it existed until last week.
OpenAI is starting to try to get a little more realistic revenue in, Grok is acquiring customers.
In the nicest way possible I'm saying this form of preference testing is ultimately useless, primarily due to a base of dilettantes with more free time than knowledge parading around as subject matter experts and secondarily due to presumed malfeasance. The latter is more apparent to more of the masses (that don't blindly believe any leaderboard they see) now that access to the model itself is more widespread and people are seeing the performance doesn't match the "revolution" promised [0]. If you're still confused why selecting a model based on a glorified Hot or Not application is flawed, perhaps ask yourself why other evals exist in the first place (hint: some tests are harder than others.)
[0](One such instance of someone competent testing it and realizing it's not even close to the "best" model out) https://www.youtube.com/watch?v=WVpaBTqm-Zo
Do we have a way to tell if one model is smarter than another at that point?
Here's a real world intelligence test. Take on each AI as a remote intern/new-hire, and try to train it to become a useful team member (solving math puzzles or manufacturing paperclips does not count).
Ask them to design a ranking mechanism for you. They are superhuman, after all.
(I really don't think we're going to have to worry about this).
Of course this is for a personal instance, you'd need a much more expensive setup to handle concurrent users. And that's to run it, not train it.
"hello how are you today?" - 7 tokens.
And this is so much better than I could have imagined in a very short span of time.
https://xcancel.com/carrigmat/status/1884244369907278106
This takes advantage of the sparsity of MOE and the efficient KV-cache of MLA.
E-ATX case = ~$300
Power Supply= ~$300
Xeon W-3500 (8 channel memory) = $1339 - $5889
Memory = $300-$500 per 64GB DDR5 RDIMM
Memory will be the major cost. The rest will be around $5,000. A lot less than "$100,000"!
And the original message you were responding to was using a CPU with AMX and mixing it with a GPU like Nvidia 4900/5900. That way the large part of the model sits in the larger slower memory, and the active part in the GPU with the faster memory. Very cost effective and fast. (Something like generating 16 Tokens/s of 671B Deepseek R1 with a total hardware cost of $10-$20k.) They tried both single and dual CPU, with the latter about 30% faster....not necessarily worth it.
https://github.com/kvcache-ai/ktransformers/blob/main/doc/en...
That's the theory. In practice, Sapphire Rapids needs 24-28 cores to hit the 200 GB/s mark and it doesn't go much further than that. Intel CPU design generally has a hard time saturating the memory bandwidth so it remains to be seen if they managed to fix this but I wouldn't hold my breath. 200 GB/s is not much. My dual-socket Skylake system hits ~140 GB/s and it's quite slow for larger LLMs.
> Why does it have to be a dual CPU design?
Because memory bandwidth is one of the most important limiting (compute) factors for larger models inference. With dual-socket design you're essentially doubling the available bandwidth.
> And the original message you were responding to was using a CPU with AMX and mixing it with a GPU like Nvidia 4900/5900.
Dual-socket CPU that costs $10k on a server that costs probably couple of factors more. Now you claimed that it doesn't have to be that expensive but I beg to differ - you still need $20k-$30k of worth equipment to run it. That's a lot and not quite "cost effective".
(Dual Socket Skylake? Do you mean Cascade Lake?)
If you price it out, it's basically the most cost effective set-up with reasonable speed for large (more than 300 GB) models. Dual socket basically doubles the motherboard[2] and CPU cost, so maybe another $3k-$6k for a 30% uplift.
[1] https://www.intel.com/content/www/us/en/products/sku/231733/... $3,157
[2] https://www.serversupply.com/MOTHERBOARD/SYSTEM%20BOARD/LGA-... $1,800
Please price it out for us because I still don't see what's cost effective in a system that costs well over $10k and runs at 8 tok/s vs the dual zen4 system for $6k running at the same tok/s.
I am not sure what your point is? There are some nice dual socket Epyc examples floating around as well, that claim 6-8 tokens/s. (I think some of those are actually distilled versions with very small context sizes...I don't see any as thoroughly documented/benchmarked as the above). This is a dual socket Sapphire Rapids example with similar sized CPUs and a consumer graphics card that gives about 16 tokens/second. Sapphire Rapids CPU and MB are a bit more expensive, and a 4090 was $1500 until recently. So for a few thousand more you can double the speed. Also the prompt processing speed is waaaaay faster as well. (Something like 10x faster than the Epyc versions.)
In any case, these are all vastly cheaper approaches than trying to get enough H100s to fit the full R1 model in VRAM! A single H100 80 GB is more than $20k, and you would need many of them + server just to run R1.
The math is clear: single-socket ktransformers performance is 8.73 tok/s and it costs ~$12k to build such a rig. The same performance one gets from a $6k dual-EPYC system. It is a full-blown version of R1 and not a distilled one as you say.
Your claim about 16 tok/s is also misleading. It's a figure for 6 experts while we are comparing R1 with 8 experts against llama with 8 experts. 8 experts on dual-socket system per ktransformer benchmarks runs at 12.2 - 13.4 tok/s and not 16 tok/s.
So, ktransformers can roughly achieve 50% more in dual-socket configuration and 50% more than dual-EPYC system. This is not double as you say. And finally, the cost of such dual-socket system is ~$20k and therefore isn't the "best cost effective" solution since it is 3.5x more expensive for 50% better output.
And tbh llama.cpp is not quite optimized for pure CPU inference workloads. It has this strange "compute graph" framework which I don't understand what is it there for. It appears completely unnecessary to me. I also profiled couple of small-, mid- and large-sized models and the interesting thing was that majority of them turned out to be bottlenecked by the CPU compute on a system with 44 physical cores and 192G of RAM. I think it could do a much better job there.
Cheapest 32 core latest EPYC (9335) x 2 = $3,079.00 x 2
Intel 32 Core CPU used above x 2 = $3,157 x 2 (I would choose the Intel Xeon Gold 6530 which is going for around $2k now, and with with higher clock speeds, and a 100 MB of more cache)
AMD Epyc Dual Socket Motherboard Supermicro H13DSH = $1899
Intel Supermicro X13DEG-QT = $1,800
Memory, PSU, Case = Same
4090 GPU = $1599 - $3,000 (temporary?)
Besides the GPU cost, the rest is about the same price. You only get a deep discount with AMD setups if you use EPYCs a few years old with cheaper (and slower) DDR4.
And again, if you go single CPU, you save over $4,000, but lose around 30% in token generation.
The "$6,000" AMD examples I've seen are pretty vague on exactly what parts were used and exactly what R1 settings including context length they were run at, making true apple to apple comparisons difficult. Plus the Sapphire Rapids + GPU example is about 10x faster in prompt processing. (53 seconds to 6 seconds is no joke!)
Yes, you're blatantly misrepresenting information and moving goalposts. Right now it has become clear that you're doing this because you're obviously affiliated with ktransformers project.
$6k for 8 tok/s or $20k for 12 tok/s. People are not stupid. I rest my case here.
So, say 500W. That's, for me in my expensive electricity city, $40/million tokens, with the pretty severe rate limit of 5600 tokens/hours.
If you're in Texas, that would be closer to $10/million tokens! Now you're at the same price as GPT-4o.
Related, you can get a whole lot of cloud computing for $2k, for those same experiments, on much faster hardware.
But yes, the data stays local. And, it's fun.
This comment chain is pretty funny.
There is no being "on par" in this space. Model providers are still mostly optimising for a handful of benchmarks / goals, like we can already see that Grok 3 is doing incredibly well on human preference (LM Arena) however with Style Control, it's suddenly behind ChatGPT-4o-latest and Gemini 2.0 is out the picture. So even within a single domain, goal, benchmark—it's not as straightforward as to say that one model is "on par" with another.
> shouldn't we expect that anybody with the computer power is capable to compete with o1-pro?
Not necessarily. I know it may be tempting to think that Grok 3 is entirely a result of xAI having lots of "computer power", but you have to recognise that this mindset is coming from a place of ignorance, not wisdom. Moreover, it doesn't even pass off as "cynical" view, because it's common knowledge that model training is really, really complicated. DeepSeek results are note-worthy, and really influential in some respects, but it hasn't magically "solved" training, or made training necessarily easier / less expensive for the interested parties. They never shared the low-level performance improvements, just model weights and lots of insight. For talented researchers, this is valuable, of course, but it's not like "anybody" could easily benefit from it in their training regimes.
Update: RFT (contra SFT) is becoming really popular with service providers, and it's not been "standardised" beyond whatever reproductions to have emerged in the weeks prior, moreover R1 cost is still pretty high[1] at something like $7/Mtok, & bandwidth is really not great. Consider something like Google Vertex AI's batch pricing for Gemini 1.5 Pro and Gemini 2.0 Flash models, which is at 50% discount, and their prompt caching which is at 75% discount. R1 is still got a way to go.
[1]: https://openrouter.ai/deepseek/deepseek-r1/providers?sort=th...
o1-pro is "o1 on steroids" and was the first selling point of the $200/month Pro subscription but they later also added "Deep Research" and Operator to the Pro subscription.
Frankly. Whoever decided on this last gen naming at MS needs to come forward. I would love to know what crazy unacceptable collection of circumstances allowed that to happen.
Not doubting your experience, just thinking how subjective it all is.
Andrej Karpathy: "I was given early access to Grok 3 earlier today" - https://news.ycombinator.com/item?id=43092066 - Feb 2025 (48 comments)
That’s like a chef giving you chicken instead of beef and calling it vegetarian.
A truly open model has open code that gathers pre-training data, open pre-training data, open RLHF data, open RLAIF data generated from its open constitution and so on.
The binary blob is the last thing I'd want - as a heavy user of LLMs I'm actually more interested in the detail of what all training data is in full, than I am the binary blob.
[0] https://news.ycombinator.com/item?id=42971446
linus turns to the camera, giving a thumbs up
Open-source as a concept doesn’t really correspond well with LLMs but to the extent that it does, access to the training data is not required because that training data is not the preferred form for making modifications.
I definitely disagree with this.
Yes, you can do some SFT fine tuning on an existing model, but if you want to make specific, substantial, targeted changes (less safety? better performance on math and code at the expense of general knowledge?), your best bet is to change the training mixture, and for that you need the original datasets.
But this thread is about misuse of the term as applied to the weights package. Those of us who know what open source means should not continue to dilute the term by calling these LLMs by that term.
But we don't actually know all that much about how language really works, for all the resources we spend on linguistics - as the old IBM joke about AI goes, "quality of the product increases every time we fire a linguist" (which is to say, we consistently get better results by throwing "every written word known to man" at a blank model than we do by trying to construct things from our understanding).
All that said, just because we're taking a different, and quite possibly slower / less compute-efficient route, doesn't mean that we can't get to AGI in this way.
No, we can’t few shot it and we don't get there faster (but we develop a lot of other capabilities on the way.) We train on a lot more data; the human brain, unlike an LLM, is training on all that data in processes for ”inference”, and it receives sensory data estimated on the order of a billion bits per second, which means by the time we start using language we’ve trained on a lot of data (the 15 trillion tokens from a ~17 bit token vocabulary that Llama3 is something like the size of a few days of human sense data.) Humans just are trained on and process vastly richer multimodal data instead of text streams.
Yeah, humans don't acquire language separately from other experience.
> Most of the data that you reference is visual input and other body sensations that aren't directly related to that.
Visual input and other body sensations are not unrelated to language acquisition.
> OTOH humans don't take all that much text to learn to read and write.
That generally occurs well after they have acquired both language and recognizing and using symbolic visual communication, and they usually have considerable other input in learning how to read and write besides text they are presented with (e.g., someone else reading words out loud to them.)
It's just like even for a true open source software you still need to bring your own hardware to run it on.
But I agree, it's a real shame.
Literally how language has always worked and evolved, though.
Oof, I know there's a bunch of linguists and grammarians who are going to mock you for that bracket.
It is but it was the "correct" part attached to prescriptivism they'd be mocking because that is not how linguists and grammarians work (they are descriptivists and fond of making fun of prescriptivists.)
I see both sides here, but I don't think it's a hill worth dying on. The 'open source' part in this case is just not currently easily modifyable. That may not always be the case.
I think the two plausible answers are:
1. The person prompting (for example telling chatgpt 'please produce a fizzbuzz program') owns the copyright. The creativity lies in the prompt, and the chatgpt transformation is not transformative or meaningful.
2. The output of ChatGPT is derivative of the training data, and so the copyright is owned by all of the copyright holders of the input training data, i.e. everyone, and it's a glowing radioactive bomb of code in terms of copyright that cannot be used or licensed meaningfully in open source terms.
There are existing things like 1, where for example if someone takes a picture, and then uses photoshop to edit it, possibly with the "AI erase" tool thingy, they still own the photo's copyright. Photoshop transformed their prompt (a photo), but adobe doesn't get any copyright, nor do any of the test files adobe used to create their AI tool.
I don't think AI is like that, but it hasn't gone to court as far as I know, so no one really knows.
What do you think an open source matrix should look like?
I'm not even necessarily advocating that these things should be released, but the term "open source" has a pretty well-understood meaning that is being equivocated here.
Its about reproducibility and modifiability. Compiled executables (and their licences) lack that. The same as these downloadable blobs.
You can absolutely have open source machine code.
The issue is and always has been that you need to have access to the same level of abstraction as the people writing the source code. The GPL specifically bans transpilers as a way to get around this.
In ML there is _no_ level of abstraction other than the raw weights. Everything else is support machinery no different to an compiler, and os, or a physical computer to run the code on.
Linux isn't closed source because they don't ship a C compiler with their code. Why should llama models be any different?
AI2 has a model called OLMo that is actually open source. They share the training data, training source code, and many other things:
https://allenai.org/blog/olmo2
They also released an app recently, to do local inference on your phone with a small truly open source model:
https://allenai.org/blog/olmoe-app
It's not like they understand what the weights mean either and if they released the code and dataset used to create it, you probably couldn't recreate it, owning the fact that you don't own tens of thousands of GPUs.
If a software's source is released without all the documentation, commit history, bug tracker data etc., it's still considered open source, yet you couldn't recreate it without that information.
Also, they will be open sourcing Grok 2, which is probably pretty behind at this point, but will still be interesting for people to check out.
Someone that is not elected gets 8mill/day from the government now overseas the government with some 20-year old fanatics that can’t even put up a secure website for DOGE?
For those downvoting, the roles are President, Vice President, Senator, and Representative.
Outside of that, everyone is hired / appointed.
Tell me you don’t know how the US government works without telling me you don’t know how the US government works, why don’t you?
""[The President] shall nominate, and by and with the Advice and Consent of the Senate, shall appoint Ambassadors, other public Ministers and Consuls, Judges of the supreme Court, and all other Officers of the United States, whose Appointments are not herein otherwise provided for, and which shall be established by Law: *but the Congress may by Law vest the Appointment of such inferior Officers, as they think proper, in the President alone, in the Courts of Law, or in the Heads of Departments.*"
Emphasis on the last sentence. There have been a plethora of such vestments in the Executive branch over the decades.
5 U.S.C. § 105 - This statute authorizes the creation of "necessary agencies" within the Executive Office, giving the President flexibility to establish entities like the USDS and staff them as needed, subject to funding.
5 U.S.C. § 3101 - This law states that "each Executive agency" (including the EOP) "may employ such number of employees ... as Congress may appropriate for." It implies broad authority to hire staff, with Congress controlling the budget but not necessarily the individual appointments.
Excepted Service Authority (5 U.S.C. § 3301 and Schedule A) - Under 5 CFR § 213.3102, agencies like the OMB can use Schedule A hiring authority for positions requiring specialized skills (e.g., tech expertise) that aren’t practical to fill through standard civil service exams.
This authority, delegated by Congress via the Civil Service Reform Act of 1978 (Public Law 95-454) and regulations from the Office of Personnel Management (OPM), allows the President (or OMB leadership) to appoint USDS personnel directly.
Term Appointments - Many roles are temporary or term-limited (e.g., 2-4 years), often filled by detailees from other agencies or private-sector experts. These don’t require Senate confirmation because they aren’t permanent "officers." This flexibility is supported by 5 U.S.C. § 3161, which allows temporary organizations within the executive branch to hire staff for specific projects.
This creates numerous conflicts of interests and opportunities for self-dealing.
Consider a NASA employee that is awarding government contracts. They know all of the above. They have three bids in front of them. One from Boeing (lol), one from SpaceX, and one from Axiom Space. They NASA employee thinks the bid from Axiom Space is the best value and fits the requirements the best. But will they select them, or will they select SpaceX, knowing that they could be fired tomorrow by Musk's whim?
Repeat this scenario across every interaction any of Musk's companies have with the federal government.
This isn't a novel scenario. Putin's Russia is a great example of what happens when oligarchs are granted significant autonomy over organs of the government. It is a system designed to facilitate corruption.
You could assuage my concerns, though, by describing the ways that there is effective oversight over Musk, or by describing the anti-corruption anti-self-dealing measures that have been imposed on Musk. The Press Secretary gave a statement on this saying: "As for concerns regarding conflicts of interest between Elon Musk and DOGE, President Trump has stated he will not allow conflicts, and Elon himself has committed to recusing himself from potential conflicts." That...does not resolve my concerns.
There is no evidence of this happening, nor of a serviceable plan to do so.
All recovered expenses, to date, add up to a laughably small amount, and are one-time cutbacks. The strategy shows signs of costing the government in unexpected ways as well.
Most governments of developed nations operate in more sensible ways with clearer plans than this. I won't claim they are looting, but it's absurd to suggest they are fixing the deficit at the moment. The economy appears to be getting worse, not better.
What evidence is there that the current moves will lead to "fixing the deficit?"
Illegally redistributing of appropriated spending could be easily understood as looting in most context. Not sure how this would be excluded.
I think this has two errors.
First, I don't agree that he's fixing the deficit. I think that's an assumption not in evidence. We'll see in a few years time, though. I'd be willing to bet in 4 years the deficit is > 0, and likely larger than it is today.
But let's assume arguendo that he is fixing the deficit. It's still possible to loot the treasury while fixing the deficit, which shows that they aren't actually the opposite.
Consider this example with completely made up numbers:
Before
- Revenue: $1T
- Defense Spending: $500B
- Benefits Spending: $1T
- Public Services Spending: $499.9B
- Government Contracts with Musk's Companies: $100M
The before scenario has $1T in revenue, and $2T in spending, for a deficit of $1T. Now, let's allow hypothetical Musk to have free-reign to "fix the deficit"
After
- Revenue: $800B
- Defense Spending: $300B
- Benefits Spending: $300B
- Public Services Spending: $100B
- Government Contracts with Musk's Companies: $100B
In this scenario the deficit has been reduced to $0, while Musk has enriched himself and his companies with $99.9B in government funds. This would be an extreme example of Musk looting the treasury, while still completely resolving the deficit.
And he's not even "auditing" the government. When you're auditing, you emit a report that the audited party later analyzes and acts upon. He's been given freeway to fire government workers as he pleases as if he's an elected officer, which he's not.
They are auditing as part of their process of cutting costs. They're literally tracing trillions of dollars in financial records.
He's doing everything he's doing by executive order of the President of the United States, who was elected.
He, a single person, has far too much control of our system.
Thats not how it works.
Btw. i think having the richest man in the world in his current position is very very unique.
And already has a number of lawsuits started because he's trying to do things neither he nor the president are allowed to do. Getting an EO to do something doesn't mean it's automatically legal. Multiple big decisions have already been reverted or are held until judges can review them. Even things like the promised payout for quitting are not practical, because only congress can approve the money for that.
Then why the hell are they still waving flags?
https://www.nbcnews.com/news/amp/rcna191304
Your bias is showing.
When everyone goes around calling everyone they hate a Nazi, it only desensitizes people to the real Nazi behavior, kind of like the boy who cried wolf, since there's people out there committing actual atrocities against humans going under the public radar because they never do the Nazi salutes on camera to not draw attention. So then the Nazi term starts to loose any meaning, kind of like the overuse of calling everything "woke" today.
The problem is people as a whole are retarded due to mob behavior and too focused on optics and only judge based on feelings rather than facts, that's how we have actual criminal Nazis going free under the radar while innocent people being swatted and doxxed because they said something right wing on social media. Not all Nazis today wear jackboots and do heils, plenty go about appearing like normal people in public, they could even be your neighbor, police officer or local congressman.
So save your anger for those people instead, as Elon is just a 3 year old throwing tantrums seeking attention, annoying but relatively harmless. If people stopped giving him so much attention, he'd stop doing it.
What makes you so sure/special in this regard? What are you gaining from this? If your were that enlightened as you claim, Germany wouldn't be in such a mess right now. If you only spent as much effort in securing your borders, energy independence and defence, as you spent lecturing others on imaginary Nazis and banning hate speech on social media, you'd be a respectable world power right now at the table with Trump and Putin ending this war before it even happened. The "we know better than you" arrogance is Germany's biggest problem.
>This is basically blaming the thing criticized on the people criticizing it.
It's not blaming, I'm just telling you what the simplest solution is. Ignoring attention seekers is better than giving them more attention. Which is why I'm also gonna ignore your future comments form now on.
Germany? the scope of the discussion is already limited to the AfD. What makes Germans special in their opinions about Nazis in Germany to is deep experience and knowledge with the subject you are belittling. And I'm also not spending any time banning hate speech on social media, heh. Whatever chip on your shoulder you have about Germany I can barely even decipher and you're right, it's best to agree to disagree.
https://www.themarginalian.org/2013/01/17/richard-feynman-of...
https://news.ycombinator.com/item?id=43037624
Wrong tools.
They can use data processing, detect anomalies better, leverage AI models, automate data extraction from analog records, ingest unstructured data like emails and memos, build complex financial dependency graphs, detect leaks, build custom scrapers, etc etc.
I'm sure there's at least one accountant in the loop, but you really want the team to consist mostly of data nerds.
You also seem confused; COBOL might be used, but it isn't the only tool available to accountants working for the government. COBOL is a straw-man. What you're describing here—software engineers who presumably have training in accounting—already exists, and they work inside and out of the government. This is an existing career path.
You're speaking about this as though you know a better way to do something, but it's already happening, and has been for years. Accountants aren't writing 62 year old programming languages waiting to die in their chairs while the world continues to progress without them.
Accounting just about anywhere you find it is already accomplished by accountants, some of them technically trained, as well as data scientists and software engineers. It's an interdisciplinary collaboration in any serious organization.
Or they will have absolutely no idea about the context and for example reveal secret information while they think they're just looking at money https://www.huffpost.com/entry/elon-musk-doge-posts-classifi...
You can easily get drowned by a see of numbers and get confused and gaslighted, unless you don’t make sure all data is available and computable.
Not sure how this release, which impressive by all means transformed into an attack on DOGE which is the exact approach startups are taking to disrupt an industry.
How much disruption started with massive failures?
You don't start with a live system or did SpaceX put astronauts in therir first rockets?
If he does to the USA Gov what he did to Twitter, he will destroy the brand, reduce the workforce by 80% and reduce the value by 80% too.
The issue with him is, tha tin Twitter, the affected people had money. A missed payment of USA can literaly kill people.
He would have regretted it if it wouldn't have played out (and this game he is currently playing, is not finished yet) like it currently does. He said in an interview that he puts everything on one card now.
Edit: Also he gets a lot of valuation due to him being a cult or whatever. From companies surviving the AI phase we are in right now, if he can't get the nazi people on his side to buy his stuff, he is a very high risk.
He destroyed twitters brand and bluesky emerged. He destroys Tesla and other car makers making ground. SpaceX needs a lot of subsidies and his goal for mars is only a cult topic not a financial success topic.
Here is an unelected NAZI and ILLEGAL IMMIGRANT (worked illegally on a student visa) who did a seig heil at the presidential inauguration taking up an informal unconfirmed-by-congress department head role (DOGE is just US Digital Services renamed) and getting wide access to government systems, and seemingly firing thousands of government employees.
Billionaires, who should not exist, are so rich that they don’t need government services and would rather the government go away so they can make more money. But regular people do need a government, and that’s just one reason of many why Elon shouldn’t be anywhere near policy decisions.
Now, you might say I’m being dramatic. But I’ll say there is no criticism of this man is unfair. He is one of the world’s biggest hypocrites, along with the other MAGA Nazis in his camp.
I hate how its the same story for every new AI technology. If someone can tell me who to vote for or where to protest to change this awful EU law, that would be great.
That's why they use their AI products as a leverage to turn European people against the laws that protect them from big tech. It's just blackmail.
https://artificialintelligenceact.eu/wp-content/uploads/2021...
https://en.wikipedia.org/wiki/Artificial_Intelligence_Act
For the record, Facebook has put a restriction on the use of Llama models in the UE even before the AI Act was passed (and the AI Act doesn't even apply to Llama anyway, except Llama 3.1 405b)
And here you should see that it has nothing to do with the AI act, as it wasn't enacted before last August!
Furthermore, neither Grok 1 nor Notebook LM would have been subject to the AI act even if it had existed at the time.
As I said before, all of these companies have vested interests against EU's legislation as a whole, and they've tried to blackmail the EU from the beginning. They didn't wait for an actual AI legislation to exist to use AI as just another blackmailing tool.
https://www.socialmediatoday.com/news/xs-formally-twitter-gr...
> Well, probably because Grok has already spread various false stories
(The European regulation that deals with disinformation is DSA).
And again it couldn't be the AI Act, because it wasn't in place at that time!
Given the personalities involved, I'd wager he's doing it out of spite than for any actual legal justification.
...though if there was an actual legal risk then I'll agree the economics probably don't bear-out the risks - as someone who identifies as European I'll admit that Europeans generally pay far less money for tech/software/apps/services than Americans do[1]; salaries in Europe are also significantly below their US equivalents: paying $200/mo for OpenAI's service is a much harder sell to a London or Berlin-based SWE on €90k/yr than when you're a Bay Area type on $300k/yr.
[1] e.g. If you can take Apple at their word, the EU accounts for 7% of their App Store revenue: https://techcrunch.com/2024/02/01/apple-says-eu-represents-7...
Besides, anyone in the EU who really wanted to use it can just use a VPN service.
I mean, at least I get the advantage of being overshadowed by a famous film director with the same name as me, so nobody's going to assume anything associated with my name is actually about me…
…hopefully…
And IMHO regulating 1% doesn't hinder strategic advantages much. Otherwise China would not have came up with DeepSeek models. Regulations are fine, they just have to be "based".
What? I know what "based" means, but I can't quite grok what you're saying.
Well no. Mistral.ai
(Note that it's not just GDPR, there's also the EU AI Act which has a whole extra set of requirements for compliance.)
Also, the problem with GDPR is that it wanted to leave too much room for business to still collect an obscene amount of data, hence it allows the cookie banner. Please note that I emphasized “allow” because that's all GDPR does: it allows companies to use a cookie banner to extract the consent to collect data. It doesn't mandate it in any way.
None of my multiple websites have a cookie banner on them because I'm not trying to extract consent from my users to abuse their data, I just don't collect it and I'm effortlessly GDPR-compliant in the least obnoxious way.
Cookie banners are just malicious compliance.
The Digital Market Act is a bit of an overreach but the AI law is not.
It classifies AI into risk categories, so that it doesn't kill anyone, carelessly handle sensitive information, etc.
A chatbot can easily comply with it.