Not really. But Helen Toner has been tweeting a little "To be clear: our decision was about the board's ability to effectively supervise the company, which was our role and responsibility. Though there has been speculation, we were not motivated by a desire to slow down OpenAI’s work." https://twitter.com/hlntnr/status/1730034020140912901
> Though there has been speculation, we were not motivated by a desire to slow down OpenAI’s work.
Strange when their choice of interim-CEO was someone who explicitly stated he wants to see the pace of AI development go from a 10 to a 2 [1] and she regularly speaks at EA conferences where that's a major theme.
This is probably double speak for she want's to not "slow down OpenAI's work" on AI safety but probably would have kneecapped the "early" release of ChatGPT's (as she claim they should have waited much longer in her paper) and similar things.
My current guess is that Helen and Sam had disagreements, and that caused Sam to be less-than-candid about the state of OpenAI's tech, and that was the straw that broke the camel's back for Helen. A disagreement within the board is one thing, but if the CEO undermines the ability of the board to provide oversight, that sort of crosses a line.
Alternatively, maybe it became clear to the board that Sam was trying to either bully them into becoming loyalists, or replace them with loyalists. As a board member, if the CEO is badgering me into becoming a yes-man and threatening to kick me off if I don't comply, I can't exactly say that I'm able to supervise the company effectively, can I? See https://www.lesswrong.com/posts/KXHMCH7wCxrvKsJyn/openai-fac...
The way EA does donations is "I'll take a premise I believe in and will stretch the argument until it makes no sense". This is how they end up thinking that a massage for an AI researcher is money better spent than on hungry Yemeni children for example.
Once you view it like this, I wouldn't put it past them to blatantly lie. Looking at the facts as you say, they tried to replace a guy that is moving ahead with a guy that wants to slow it down to a crawl, that's pretty much all we need to know.
Essentially EA is a stretchable concept that allows adherents to act out their every fantasy with complete abandon whilst protecting their sensitive sense of self. It redefines their side to always be the good side, no matter what they get up to.
This is your daily reminder that most EAs will just donate to the top GiveWell charity even though people will talk a lot about the interesting edge cases.
From everything I've read, safety still feels like a red herring.
It just doesn't fit with everyone's behavior -- that's something that would have been talked about loudly.
Altman lying to the board, especially in pursuit of board control, fits more cleanly with everyone's actions (and reluctance to talk about what exactly precipitated this).
- Altman tries to effect board control
- Confidant tells other board members (Ilya?)
- Board asks Altman if he tried
- Altman lies
- Board fires Altman
Fits much more cleanly with the lack of information, and how no one (on any side!) seems overly eager to speak to specifics.
Why jump to AGI as an explanation, when standard human drama will suffice?
But then that doesn't square with board refuses to tell employees or Microsoft or the public that altman committed malfeasance or provide examples. That would be pretty cut and dry and msft wouldn't be willing to acquihire the entire company we with altman as CEO if there was a valid reason like that.
There was a tweet that from an engineer at OpenAI that they're working on the problem that ChatGPT has become too "lazy" - generating text that contains a lot of placeholders and expecting people to fill in much more themselves. As for the general brain damage from RLHF and the political bias, no word still.
I've thought one of the funnier end states for AGI would be if it was created but this ended up making it vastly less productive than when it was just a tool.
So the AI of the future was more like Bender or other robots from Futurama that display all the same flaws as people.
Using the api, I've been seeing this a lot with the gpt-4-turbo preview model, but no problems with the non-turbo gpt-4 model. So I'll assume ChatGPT is now using 4-turbo. It seems the new model has some kinks to work out--I've also personally seen noticeably reduced reasoning ability for coding tasks, increased context-forgetting, and much worse instruction-following.
So far it feels more like a gpt-3.75-turbo rather than really being at the level of gpt-4. The speed and massive context window are amazing though.
Yeah, I usually use gpt-4-turbo (I exclusively use the API via a local web frontend (https://github.com/hillis/gpt-4-chat-ui) rather than ChatGPT Plus). Good reminder to use gpt-4 if I need to work around it - it hasn't bothered me too much in practice, since ChatGPT is honestly good enough most of the time for my purposes.
id be willing to bet all they're doing behind the scenes is cutting computation costs using smaller versions and doing every business' golden rule: price discrimination.
id be willing to bet enshittification is on the horizon. you don't get the shiny 70b model, that's for gold premium customers.
This has been the case with gpt-3.5 vs gpt-3.5-turbo, as well. But isn't it kinda obvious when things get cheaper and faster that there's a smaller model running things with some tricks on top to make it look smarter?
My son asks why, but only once. I'm not yet sure if it is because he is satisfied with his first answer, or if my answers just make the game too boring to play.
they're B2B now, that means only political correctness.
and I'm not sure why anyone dances around it but these models are built by unfiltered data intake. if they actually want to harness bias, they need to do what every capitalist does to a social media platform and curate the content.
lastly, bias is an illusion of choice. choosing color over colour is a byproduct of culture and you're not going to eradicate that but cynically, I assume you mean , why won't it do the thing I agree with.
Whatever the people who buys ads decides, losing your ad revenue is the main fear of most social media and media companies.
See twitter for example, ad buyers decided it is no longer politically correct so twitter lost a lot of ad revenue. Avoiding that is one of the most important things if you want to sell a model to companies.
IIRC they've put in guard rails to try and make sure ChatGPT doesn't say anything controversial or offensive, but doing so hampers its utility and probably creativity, I'm guessing.
So was all the speculation about Adam D'Angelo being the evil Poe mastermind just conjecture? Or is it true and Sam needs Adam for some dark alliance? Has the true backstory ever come out? Surely someone out of 770 people knows.
There is no reason both can't be true. He may have seen his chance to get more pliable management in place, but you'll never know unless he speaks up which he likely will never do.
That depends on whether the board was there to provide cover and protection against regulation/nationalization or whether it was supposed to have an actual role. Apparently some board members believed that they had an actual role and others understood that they were just a fig leaf and that ultimately money (and employees) hold the strings.
No, to the specific question of making employees stay, that's not a thing that you can do outside of prisons. If employees want to leave they can leave. If they want to start the nonprofit from scratch, they can do that but employees cannot be stopped from leaving
Was anyone arguing that or am I misunderstanding you? Of course they are free to leave, that's obvious. I think the point was: the employees have that power and nothing will take that away from them.
The gp said there's no way a board can stop employees from leaving and the one I was replying to started with "that depends" and suggested it was a question as to whether the board "believed" employees held all the strings. Though reading it again the parent does seem a bit non-sequitor from the gp which was strictly talking about what the board physically could have done, as in what options they had. The options they had don't really depend on any particular belief - they couldn't have made them stay thanks to the civil war and stuff
The board was a non profit board serving the mission. Mission was foremost. Employees are not. One of the comments a member made was, if the company was destroyed, it would still be consistent with serving the mission. Which is right.
The fallout showed non-profit missions can't co-exist with for-profit incentives. And the power that investors were exerting, and employees (who would also benefit from the recent 70B round they were going to have) was too much.
And any disclaimer the investors got when investing in OpenAI was meaningless. It reportedly stated they would be wise in viewing their investment as charity, and they can potentially lose everything. And there was an AGI clause that said it will reconsider all financial arrangements, that Microsoft and other investors had when investing in the company was all worthless. Link to Wired article with interesting details -https://www.wired.com/story/what-openai-really-wants/
> One of the comments a member made was, if the company was destroyed, it would still be consistent with serving the mission. Which is right.
I know you're quoting the (now-gone) board member, but this is a ridiculous take. By this standard, Google should have dissolved in 2000 ("Congrats everyone, we didn't be evil!"). Doctors would go away too ("Primum non nocere -- you're all dismissed!").
Indeed, it made no sense. But that's why I never attach any value to mission statements or principles of large entities: they are there as window dressing and preemptive whitewash. They never ever survive their first real test.
> The board was a non profit board serving the mission. Mission was foremost. Employees are not.
They need employees to advance their stated mission.
> One of the comments a member made was, if the company was destroyed, it would still be consistent with serving the mission. Which is right.
I mean, that's a nice sound bite and everything, but the only scenario where blowing up the company seems to be consistent with their mission is the scenario where Open AI itself achieves a breakthrough in AGI and where the board thinks that system cannot be made safe. Otherwise, to be relevant in guiding research towards AGI, they need to stay a going concern, and that means not running off 90% of the employee base.
> Otherwise, to be relevant in guiding research towards AGI, they need to stay a going concern, and that means not running off 90% of the employee base.
That's why they presumably agreed to find a solution. But at the same time shows that in essence, entities with for-profit incentives find a way to get what they want. There certainly needs to be more thought and discussion about governance, and how we collectively as a species or each company individually governs AI.
I don't really think we need more thought and discussion on creative structures for "governance" of this technology. We already have governance; we call them governments, and we elect a bunch of representatives to run them, we don't rely on a few people on a self-appointed non profit board.
Yep, this is spot on. The entire concept of a mission driven non profit with a for profit subsidiary just wasn't workable. It was a nice idea, a nice try, but an utter failure.
The silver lining is that this should clear the path to proper regulation, as it's now clear that this self-regulation approach was given a go, and just didn't work.
I think the whole OpenAI team being determined to follow Sam was crucial in all this, and is not something that’s easy to control.
Having said that, as there’s obviously a lot of money involved for all OpenAI employees (Microsoft offered very generous packages if they jumped ship), it can be said that money in the end is what people care a lot about.
Allegedly (per Twitter) there was a persistent effort by a small group to get those signatures rather than an organic ride-or-die attitude.
However, as you note, Sam’s exit would have cost those employees at least high six figures, so I’m sure that reduced the amount of leverage required to extract signatures.
A better lesson is that a board can't govern a company if the company won't follow its lead. They were misaligned with almost the entirety of the employees.
Because they formed an additional entity, which is a for-profit, and has leadership treating the whole thing as a for-profit, and so employees also see what their eyes are telling them.
The misalignment is not accidental, it was carefully developed over the last few years.
Or: the employees all joined to work on AI, and they succeeded at building the top AI company, under Sam, and their support of Sam was not engineered or any sort of judgement error on their part. I like to imagine that the employees are smart and thoughtful and have full agency over their own values. It was the Board who apparently had zero pulse on the company they were supposed to oversee.
the majority if the employees at openai joined after chatGPT launched, so it's not like there's some sense of nostalgia or forlorn distress for what they built slowly changing. The stock comp (sorry, "PPUs"... which are a phantom stock plan lol) is also quite high (check levels.fyi) and would have been high 7 to low 8 figures for engineers if secondary/tender offers were made.
I agree it's not that deep - they wanted to join a hypergrowth startup, build cool stuff, and get paid. If someone is rocking the boat, you throw them off the boat. No mission alignment needed! :)
Yeah, I think this was "700 out of 730 employees signed a letter saying they'd quit, over a holiday weekend". OpenAI with no employees is not worth a whole lot.
This episode taught us the very obvious lesson that if you hire people incentivized by equity growth, it is not possible to take steps that detrimentally impact equity growth, without losing all those people. The board had already lost the moment it allowed hundreds of fat compensation packages to be signed.
Maybe maybe not. There certainly wouldn't have been the version of OpenAI that runs a massively successful and profitable AI product company. But reading the OpenAI charter, it's pretty clear that running a massively successful AI product company was never a necessary component of the organization; and indeed, as we just saw, is actually a conflict of interest with the organization's charter.
I don't really care much about the demise of OpenAI-the-nonprofit - I don't think it was ever a very promising approach to accomplishing its own goals, so I don't lament its failure all that much - but I feel like there is a kind of gaslighting going on, where people are trying to erase the history of the organization as a mission-oriented non-profit, and claim that it was always intended to be a wildly profitable product company, and that just isn't the case.
I don't know if that's true of all of them, but it certainly seems to be true of most, and that's entirely my point: the structure just doesn't work. All (or at least the vast majority) of the employees have for-profit incentives, which - as we've now seen - makes it impossible for the non-profit board to act in accordance with their mission, when those actions conflict with the for-profit incentives, as they inevitably will. It was doomed from the start.
Not sure that’s the exact case. Once the employees threatened en masse to quit, all the power shifted to Sam and Greg’s hands - which I’m not sure they had up to that point.
I still think the board did what they thought was right (and maybe even was?), but not having the pulse of the company and their lack of operational experience was fatal.
It looked like this was all decided in a hurry rather than that stakeholder buy-in was achieved. And maybe that's because if they had tried to get that buy-in the cat would be out of the bag and they would find even more opposition. It also partially explains why they didn't have a plan post-firing Sam but tried a whole bunch of weird (and weirder) moves.
You wrote in general terms and then afterwards made them specific again. I just answered your comment, which is a subthread all by itself.
If you don't like the way HN works that's fine but I don't think you should be making further statements here until you've had a look at the guidelines.
Yeah, but no. "We hope to continue our working relationship and are discussing how he can continue his work at OpenAI" is not the same as "returning to OpenAI as CEO" and "returns as President". Not very subtle difference, even, huh?
Not clear at all. Ilya leaving would look really bad for OpenAI. Altman needs him to stay, and he keeps the door wide open for that in the statement. Question is: How much is he willing to offer (money and otherwise) to make that happen?
I recognize that during this process some questions were raised about Adam’s potential conflict of interest running Quora and Poe while being on the OpenAI Board. For the record, I want to state that Adam has always been very clear with me and the Board about the potential conflict and doing whatever he needed to do (recusing himself when appropriate and even offering to leave the Board if we ever thought it was necessary) to appropriately manage this situation and to avoid conflicted decision-making. Quora is a large customer of OpenAI and we found it helpful to have customer representation on our Board. We expect that if OpenAI is as successful as we hope it will touch many parts of the economy and have complex relationships with many other entities in the world, resulting in various potential conflicts of interest. The way we plan to deal with this is with full disclosure and leaving decisions about how to manage situations like these up to the Board. [1]
The best interests of the company and the mission always come first. It is clear that there were real misunderstandings between me and members of the board. For my part, it is incredibly important to learn from this experience and apply those learnings as we move forward as a company. I welcome the board’s independent review of all recent events. I am thankful to Helen and Tasha for their contributions to the strength of OpenAI. [2]
That doesn't equate to having 'customer representation' that equates to 'Quora representation'. Customers are represented by a voice-of-the-customer board where many customers, large and small can be represented who then vote for a representative to the board. The board of the non-profit having a for-profit customer (and a large one at that) as a board member makes zero sense, that's just one more end-run around 'the mission' for whatever that was ever worth.
The kind of bullshit that comes out during times like this is more than a little bit annoying, it's clear that if there is a conflict of interest it should be addressed in a far more direct way and whitewashes like this raise more questions than they answer. Such as: what was Adam's real role during all of this and how does it related to his future role at the board, as well as how much cover was negotiated to allow Adam to stay on as token sign of continuity.
You don't think that as a non-profit, the public is owed and explanation? As the public, we exempt them from taxes that everyone else has to pay, because we acknowledge that nonprofit is in the interest of people. I think they do owe us an explanation. If they were a private for-profit company, I would probably feel differently, but given their non-profit status, and the fact that their mission is explicitly to serve humanity with AI that they worry could destroy the race or the planet for more, I think they owe us an explanation.
Actually, their statements are overflowing from bits and pieces of how they are doing this 'for all of humanity' so I'm not so sure about that. Think about it this way: if in the 1940's nuclear weapons were being developed in private hands don't you think the entity behind that would owe the public - or the government - some insight into what is going on?
Think about it this way: if in the 1940's nuclear weapons were being developed in private hands don't you think the entity behind that would owe the public - or the government - some insight into what is going on?
I'd read the hell out of that alt-history novel, I can tell you that much. Not so much the "Manhattan Project" as the "Tuxedo Park Project."
If that time line had materialized you might not have been around to read it :)
But it's an interesting thought. Howard Hughes came close to having that kind of power and Musk has more or less eclipsed him now. Sam Altman could easily eclipse both, he seems to be better at power games (unfortunately, but that's what it is). Personally I think people that are that power hungry should be kept away from the levers of real power as much as possible because they will use it if the opportunity presents itself.
There's absolutely no sense in talking about OpenAI as a nonprofit at this point. The new board and Altman talk about the governance structure changing, and I strongly believe they will maximize their ability to run it as a for-profit company. 100x profit cap is a very large number on an $80 billion valuation.
Surely they don't do it without a reason. And I don't know what the reason is, but I must assume it's some financial benefit (read, tax evasion), and not our opinion.
But then they’d have to pay taxes, and all those corporations don’t get the juicy tax detections for “donating” to AI tech that will massively increase their profits.
Yea, it seems really weird that he and others can just form a non-profit and then later have it own a for-profit with the full intention of turning everything into a for-profit enterprise. Seems like tax evasion and a few other violations of what a non-profit is supposed to be.
Yep, if this is an acceptable fact pattern, it seems to create a bunch of loopholes in the legal treatment of non-profits vs for-profits. I think the simpler conclusion is that it actually isn't an acceptable fact pattern, and we'll be seeing fines or other legal action.
Getting fired and rehired throughout five days of history-making corporate incompetence, and Sam's letter is just telling everyone how great they are. Ha.
He doesn't look like the guy into revenge. He seems extremely goal focused and motivated. The kind of person that'd put this behind him very fast to save energy for what matters.
Idk why this is the comment that broke the camel's back for me, but all over this site people have been making character determinations about people from a million miles away through the filter of their public personas.
This is perpetually one of my last favorite patterns in humans. Declaring someone an idiot when they’ve never met them and they’re one of the most powerful and influential people on the planet. Not referring to Sam here.
If you refer to Trump, when people say idiot, I don't think they refer to his intelligence in the sense of appealing to people's base instincts - he clearly has good political intelligence. It refers to his understanding of subjects, his clarity of thought and expression, and often a judgment of his morals, principles, and ethics.
If you don't - there was no need to be coy. And you act as if failing up never happens in real life.
It's a reference to most any major political figure. Someone somewhere thinks they're an idiot, and usually a lot of people. And most of them don't have the nuance you have. They think they're universally legitimately stupid people.
Failing up definitely happens but given the massive number of failures you'd have to succeed through to be US president, as an example, it's not what's happening there.
It sounds like he’s knows he got caught with his hand in the cookie jar and is trying to manipulate and ingratiate before the other shoe drops. Kind of like a teen who fucked up and has just been hauled in front of their parents to face the music.
As I understand it: No. Microsoft has licensed GPT and they use that to offer it as a service via Azure. As far as I’m aware this gets you the same guarantees around tenancy and control that you’d get from any other internal Azure service.
I wasn't comfortable before the announcement. You can't "rely" on it. You need a fallback - either another AI, or using it in such a way that it is progressive enhancement.
Well, for starters, we all know that while realistically it's not unusual for a company to have a mission-critical person, it is very undesirable. So much so everybody must pretend that this is just unacceptable and surely isn't about their company. Here, we kinda saw the opposite being manifested. More convincingly than I've ever seen.
Second, I simply don't understand what just fucking happened. This whole story doesn't make sense to me. If I was an OpenAI employee, after receiving this nonsense "excited about the future" statement, I would feel just exhausted, and while maybe not resigning on the spot, it surely wouldn't make me more excited about continuing working there. But based on the whole "500 OpenAI employees" thing I must assume that the average sentiment in the company must be somewhat different at the moment. Maybe they even truly feel re-united. Sort of.
Obviously, I don't mean anything good be all that. What happens if Altman is hit by a bus tomorrow? Will OpenAI survive that? I don't even know what makes him special, but it seems we've just seen a most clear demonstration possible, that it wouldn't. This isn't a healthy company.
That said, all that would worry me much more, if I was an investor. In fact, I'd consider this a show-stopper. As a customer? It doesn't make me more reassured, but even if Altman is irreplaceable, I don't feel like OpenAI is irreplaceable, so as long as it doesn't just suddenly shut down — sure, why not. Well, not more comfortable, of course, but whatever.
“Investors” are supposed to consider their money a donation. Of course the 100x cap is generous so it is kinda an investment. And the coup reveals a higher chance that this will morph towards for-profit as that is where the power seems to be, let alone the money.
Should anyone feel 100% comfortable betting on a company that has only been (really) commercially engaged in the last 4 years? Whose success (albeit explosive) could only be seen in the last 18 months?
If we are going to rank the concerns around openai announcements from the past 2 weeks, I'd bet the more concerning one was the initial firing decision.
If you are uncomfortable with OAI you could always get the same from Azure. They're a bit behind on the latest, but they support gpt4 and function calling, which is all that really matters now, imo.
i'm still not clear what the accusation against Altman was... something about being cavalier about safety? if that was the claim and it has merit, i don't understand why it wasn't right to oust him, and why the employees are clamoring for him back
I wonder how many of the OpenAI employees are part of the "Effective Accelerationism" movement (often styled e/acc on X/Twitter). These people seem to think safety concerns get in the way of progress toward a utopian AGI future.
Well, their big mistake was being unwilling to be clear and explicit about this, but as I read it, the board's problem with him was that he wasn't actually acting as the executive of the non-profit that he was meant to be the executive of, but rather was acting entirely in the interests of a for-profit subsidiary of it (and in his own interests), which were in conflict with the non-profit's charter.
I think where they really screwed up was in being unwilling or unable to argue this case.
It's just so strange. This is such a clearly justifiable reason that the fact that they didn't argue it... or argue.. anything, makes me very suspicious that it is correct.
It sounded like they would if they could (for instance trying to sell to Anthropic or instating a "slow it way down" CEO), but they even failed at that. Not an effective board at all.
It's remarkable that the old board is the side characterized as willing to blow up the company, since it was Altman's side who threatened to blow it up. All the old board really did was fire Altman and remove Brockman from the board.
Exactly. It sounds like those board members themselves were acting in the interest of profit instead of the "benefit all humanity" mission stuff, no different than Altman. If anything then, the only difference between the two groups is one of time horizon. Altman wants to make money from the technology now. The board wants to wait until it's advanced enough to take over the world, and then take over the world with it. For the world's benefit, of course.
Here's Nathan Labenz on how sketchy the red-teaming process for GPT4 was. Nathan states that OpenAI shut him out soon after he reached out to the board to let them know that GPT4 was a big deal and the board should be paying attention: https://nitter.net/labenz/status/1727327424244023482#m [Based on the thread it seems like he reached out to people outside OpenAI in a way which could have violated a confidentiality agreement -- that could account for the shutout]
My suspicion is that there was a low-level power struggle ongoing on the board for some time, but the straw that broke the camel's back was something like Nathan describes in his thread. To be honest I don't understand why his thread is getting so little play. It seems like a key piece of the puzzle.
In any case, I don't think it would've been right for Helen to say publicly that "we hear GPT-5 is lit but Sam isn't letting us play with it", since "GPT-5 is lit" would be considered confidential information that she shouldn't unilaterally reveal.
So what is Nathan Labenz saying? That GPT-4 is dangerous somehow? It will get many people out of jobs? MS Office got all the typists out of jobs. OCR and Medical Software got all the medical transcriptionists out of jobs. And they created a lot more jobs in the process. GPT-4 is a very powerful tool. It has not a whiff of AGI in it. The whole AGI "scare" seems to be extremely political.
This is a piece of software. What would "totally amoral" even mean here? It's an inanimate object, it has no morals, feelings, conscience, etc... He gives it an amoral input, he gets an amoral output.
I mean, there's a sense in which my mind is software that's being run by my brain, right? Yet that doesn't absolve me of moral responsibility.
In any case, an F16 fighter jet is amoral in a certain sense, but it wouldn't be smart to make F16s available to the average Joe so he can conduct an airstrike whenever he wants.
Completely depends on your morality. I'm pretty sure there are some libertarians out there who think the most basic version of the second amendment includes owning F16 with live weapons.
Sure -- if you're a libertarian who thinks it should be possible to purchase an F16 without a background check, that seems consistent with the position that an amoral GPT-4 should be broadly available.
What kind of background check do you think exists when buying a fighter jet?
It’s kind of a moot point since only one F16 exists in civilian hands but you can buy other jets with weapon hardpoints for a million. Under 3 million if you want to go supersonic too. The cheapest fighter jet is on the order of $250k. There’s zero background check.
Sure, but idiots are a thing and the intersection of the sets of libertarians who may believe that and idiots is hopefully empty but it may not be so such outsized power is best dealt with through a chain of command and accountability of sorts.
Amoral literally means "lacking a moral sense; unconcerned with the rightness or wrongness of something." Generally this is considered a problem if you are designing something that might influence the way people act.
In both of your examples, the result of "immoral inputs" is immediate tangible harm. In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself.
> In case of GPT-4 or any other LLM, it's merely "immoral output" - i.e. text. It does not harm anyone by itself.
Assuming that you're not running this query over an API and relaying these answers to another control system or a gullible operator.
An aircraft control computer or reactor controller won't run my commands regardless of its actuators connected or not. Same for weapon systems.
Hall pass given to AI systems just because they're outputting text to a screen is staggering. Nothing prevents me to process this output automatically and actuate things.
Why would anyone give control of air traffic or weapons to AI? That's the key step in AGI, not some tech development. By what social process exactly would we give control of nukes to a chatbot? I can't see it happening.
> Why would anyone give control of air traffic or weapons to AI?
Simplified operations, faster reaction time, eliminating human resistance for obeying killing orders. See "War Games" [0] for a hypothetical exploration of the concept.
> a chatbot.
Some claim it's self-aware. Some say it called for airstrikes. Some say it gave a hit list for them. It might be a glorified Markov-chain, and I don't use it, but there's a hoard of people who follows it like it's the second Jesus, and believe what it emits.
> I can't see it happening.
Because, it already happened.
Turkey is claimed to use completely autonomous drones in a war [1].
South Korea has autonomous sentry guns which defend DMZ [2].
We give hall passes to more than AI. We give passes to humans. We could have a detailed discussion of how to blow up the U.S. Capitol building during the State of the Union address. It is allowed to be a best selling novel or movie. But we freak out if an AI joins the discussion?
You persist in using highly misleading analogies. A military F-16 comes with missiles and other things that are in and of themselves highly destructive, and can be activated at a push of a button. An LLM does not - you'd have to acquire something else capable of killing people first, and wire the LLM into it. The argument you're making is exactly like claiming that people shouldn't be able to own iPhones because they could be repurposed as controllers for makeshift guided missiles.
Speaking of which, it's perfectly legal for a civilian to own a fighter plane such as an F-16 in US and many other countries. You just have to demilitarize it, meaning no weapon pods.
>The argument you're making is exactly like claiming that people shouldn't be able to own iPhones because they could be repurposed as controllers for makeshift guided missiles.
The reason this isn't an issue in practice is because such repurposing would require significant intelligence/electrical engineering skill/etc. The point is that intelligence (the "I" in "AI") will make such tasks far easier.
>Ten-year-old about to play chess for the first time, skeptical that he'll lose to Magnus Carlsen: "Can you explain how he'll defeat me, when we've both got the same pieces, and I move first? Will he use some trick for getting all his pawns to the back row to become Queens?"
The point here is that is giving the user detailed knowledge on how to harm others. This is way different than a gun where you are doing the how (aiming and pulling the trigger).
The guy says he wanted to slow down the progress of AI and GPT suggested a detailed assassination plan with named targets and reasons for each of them. That's the problem.
Thing is, this detailed knowledge is already available and much easier to acquire. There are literally books on Amazon that explain how to jury-rig firearms and bombs etc. Just to give one example: https://www.amazon.com/Improvised-Munitions-Handbook-TM-210/....
When it comes to those "detailed plans", if you actually try something like that, what you get from it is a very broad outline that is pretty much a mish-mash of common sense stuff and cultural tropes (many of which aren't true IRL). Similarly, the "list of targets" that it makes is simply the most prominent people associated with that area in public perception, not necessarily people who are actually key. All of this can be achieved just as well with a few Google searches, and the resulting plan will likely be much better for it.
I've yet to see any example where GPT would come up with something along these lines that is not trivially found on the Internet anyway.
Except the reason why we dont all just killed each other yet have nothing to do with risk or cost of killing someone.
And everything LLM can come up with will be exactly the same information you can find in any fiction detective book or TV series about crime. Yeah very very dumb criminal can certainly benefit from it, but he can as well go on 4chan and ask about assassination there. Or on some detective book discussion club or forum.
I dont disagree with his points, but you completely miss the point of my post. People dont need an AI advise to commit crime and kill others. Nonestly humans they're pretty good at it using technology of 1941.
You don't have bunch of cold blood killers going around not just because police is so good and killers are dumb and need AI help. It's because you live in functioning state where society have enough resources so people happy enough to instead go and kill each other in Counter Strike or Fortnite.
I totally agree that AGI could be a dangerous tech, but it's will require autonomity where it can manipulate real world. So far GPT with API access is very far from that point.
If you have ChatGPT API access you can have it write code and bridge that to other external APIs. Without some guard rails an AI is like a toddler with a loaded gun. They don't understand the context of their actions. They can produce dangerous output if asked for it but also if asked for something else entirely.
The danger also doesn't need to be an AI generating code to hack the Gibson. It could also be things like "how do I manipulate someone to do something". Asking an AI for a marketing campaign isn't necessarily amoral. Asking it how to best harass someone into committing self-harm is.
> Except the reason why we dont all just killed each other yet have nothing to do with risk or cost of killing someone
Most of us don't want to.
Most of those who do, don't know enough to actually do it.
Sometimes such people get into power, and they use new inventions like the then-new-pesticide Zyklon B to industrialise killing.
Last year an AI found 40k novel chemical agents, and because they're novel, the agencies that would normally stop bad actors from getting dangerous substances, would generally not notice the problem.
LLMs can read research papers and write code. A sufficiently capable LLM can recreate that chemical discovery AI.
The only reasons I'm even willing to list this chain, is that the researchers behind that chemical AI have spent most of the intervening time making those agencies aware of the situation, and I expect the agencies to be ready before a future LLM reaches the threshold for reproducing that work.
Everything you say does make sense, except those people who able to get equipment to produce those chemicals and have funding to do something like that - they dont really need AI help here. There are plenty dangerous chemicals already well known to humanity and some dont actually take anything regulated to produce "except" complicated and expensive lab equipment.
Again difficulty of production of poisons and chemicals it's not what prevent mass murdering around the globe.
Complexity and cost are just two of the things that inhibit these attacks.
Three letter agencies knowing who's buying a suspicious quantity from the list of known precursors, that stops quite a lot of the others.
AI in general reduces cost and complexity, that's kind of the point of having it. (For example, a chemistry degree is expensive in both time and money). Right now using an LLM[0] to decide what to get and how to use it is almost certainly more dangerous for the user than anyone else — but this is a moving goal, and the question there has to be "how to we delay this capability for as long as possible, and at the same time how do we prepare to defend against the capability when it does arrive?"
[0] I really hope that includes even GPT-4 before the red-teaming efforts to make it not give detailed instructions for how to cause harm
There's been a fair amount of research into hooking up LLMs with the ability to call APIs, browse the web, and even control robots, no? The barrier between planning and doing is not a hard one.
As for cost and risk -- ask GPT-5 how to minimize it. As Nathan said in his thread, it's not about this generation, it's about the next generation of models.
I really don't know how to interpret that link, but I think there is a lot we don't understand which is going on in those billions of parameters. Understanding it fully might be just as hard as understanding the human brain.
I'm concerned that at some point in the training process, we will stumble across a subset of parameters which are both self-aware and self-interested, too. There are a lot of self-interested people in the world. It wouldn't be surprising if the AI learns to do the sort of internal computation that a self-interested person's brain does -- perhaps just to make predictions about the actions of self-interested people, at first. From there it could be a small jump to computations which are able to manipulate the model's training process in order to achieve self-preservation. (Presumably, the data which the model is trained on includes explanations of "gradient descent" and related concepts.)
This might sound far-fetched by the standard of the current model generation. But we're talking about future generations of models here, which almost by definition will exhibit more powerful intelligence and manifest it in new unexpected ways. "The model will be much more powerful, but also unable to understand itself, self-interest, or gradient descent" doesn't quite compute.
The image is OCR'ed and that data is fed back into the context. This is no more interesting or indicative of it passing the mirror test than if you had copy and pasted the previous conversation and asked it what the deal was.
I can think of several ways that AI assistance might radically alters both attack and bodyguard methods. I say "might" because I don't want to move in circles that can give evidenced results for novel approaches in this. And I'm not going to list them for the same reasons I don't want an AI to be capable of listing them: while most of the ideas are probably just Hollywood plot lines, there's a chance some of them might actually work.
Good planning necessarily reduces the cost of something relative to the unplanned or poorly planned version. If it identifies a non-obvious means of assassination that is surprisingly easy, then it has done something "close enough" to reducing the cost to $0.
This is where I have issues with OpenAIs stated mission
I want AI to be amoral, or rather I should say I do not want the board of OpenAI, or even the employee of OpenAI choosing what "moral" is and what is "immoral" especially given that OpenAI may be superficially "diverse" in race, gender, etc, but they sure as hell are not politically diverse, and sure has hell do not share a moral philosophy that is aligned with the vast majority of the population of humanity given the vast majority of humanity is religious in someway and I would guess the majority of OpenAI is at best agnostic if not atheist
I do not want a AI Wikipedia.... aka politically biased to only 1 worldview and only useful for basic fact regurgitation like what is the speed of light
Medical transcriptionists out of jobs? As far as I'm aware, medical transcription is still very much the domain of human experts, since getting doctors to cater their audio notes to the whims of software turned out to be impossible (at least in my corner of the EU).
My mom had to do this for her job and apparently some of the docs are so mumbly they have to infer a lot of the words from context and type of procedure but there is a lot of crossover everywhere so it depends a lot on which doc is mumbeling what. And yes you need special training for it (no medical degree though)
Did they really create a ton more jobs? The past few rounds of industrialization and automation have coinved with plagues/the black death that massively reduced the population, mass agitation, increasing inequality, and recently a major opioid epidemic in regions devastated by the economic changes of globalization and industrialization. I think these tools are very good and we should develop, I also think it's delusional to think it'll just balance out magically and dangerous to expect our already failed systems to protect people left behind. Doesnt exactly look like they worked any of the previous times!
1) be surprised by and unprepared for AGI and every step on the path to that goal, or
2) have the developers of every AI examine their work for its potential impact, both when used as intended and when misused, with regard to all the known ways even non-G AI can already go wrong: bugs, making stuff up, reward hacking, domain shift, etc.; or economically speaking how many people will be made unemployed by just a fully [G]eneral self-driving AI? What happens if this is deployed over one year? Do existing LLMs get used by SEO to systematically undermine the assumptions behind Page rank and thus web search?; and culturally: how much economic harm do artists really suffer from Diffusion models? Are harms caused by AI porn unavoidable thanks to human psychology, or artefacts of out milieu that will disappear as people become accustomed to it?
There's also a definition problem for AGI, with a lot of people using a standard I'd reserve for ASI. Also some people think an AGI would have to be conscious, I don't know why.
The best outcome is Fully Automated Luxury Communism, but assuming the best outcome is the median outcome is how actual Communism turned into gulags and secret police.
It seems that silicon valley is developing not a conscious sentient piece of software rather a conscience, a moral compass is beginning to emerge.
After giving us Facebook, insta, Twitter, and ego-search, influencing many people negatively, suddenly there are moral values being discussed amongst those that decide our collective tech futures.
AI will have even more influence on humankind and some are questioning the morality of money (hint: money had no morals).
Yeah, I totally agree! Like, this is such an obviously true and valid reason to fire him, but they never came out and said it! So ... is this not what it actually was? Or ... what? It truly is mystifying.
From a doomer/EA perspective, publicly saying that GPT5 is AGI or such would likely inspire & accelerate labs around the world to catch up. Thus it was more "Altruistic" / aligned with humanity's fate to stay mum and leave the situation cloudy.
Having an existing example showing that something difficult is possible causes everyone else to replicate it much faster, like how a bunch of people started running sub-4-minute miles after the first guy did it.
If you know the outcome is favourable then you go all in. Right now the other competitors are just trying to match GPT4, if they knew AGI was achievable then they would throw everything they have at it in order to not be left out.
Show me the researchers and companies where the people in charge and the people doing the work don’t think it’s possible to get better than GPT-4 and are slow-rolling things.
I suppose maybe there are ones where they are slow-rolling because of their opinions about existential risk of AGI. But that’s not contingent on what OpenAI says or does.
I think they didn't anticipate sucha a large backlash, especially from investors. They felt the backlash threatens OpenAI, both its non-profit and for-profit arms, so they reverted their decision, which in my opinion was a mistake, but time will tell.
Yeah, I guess so. I keep waffling between thinking it's some 4d chess thing, and thinking it's just normal human fallibility, where the board just made a massive mistake in predicting how it would go. But I just struggle so much to imagine that, because everyone I know in the industry, regardless of their level of expertise or distance from OpenAI, immediately knew how big a deal it was going to be, when we heard he was hired. But supposedly the people on the board had no idea? I think this might be the right conclusion, but I nevertheless struggle to fathom it.
The board had the right non-profit long-term focus, but it didn't have the willpower to communicate and realize their vision. Ilya is a wonderful scientist, but not a great leader.
Really hope details come about this with all perspectives being provided.
Whether they stuffed up or there are some details that made the situation unworkable for the board, it’s an interesting case study in governance and the whole nonprofit with a for profit subsidiary thing.
And how did that attitude work out for them? Upwards of 90% of their staff threatened to bail out, every single one of them look like fools in public, and I would be shocked if there were not some recriminations behind closed doors from the likes of Microsoft's CEO.
It’s not an “attitude.” It’s the legal structure of the corporation’s leadership. They were no more capable of incorporating the public will than an individual is capable of taking a vote on the flow of traffic on a public highway. That is to say, even if they had done what is proposed here, it wouldn’t have made a difference.
In general, your comments strike me as reflecting a very naive understanding of how anything works. Legal responsibilities are simply not the only thing that matters. This entire episode was about 95% a PR battle, which the board lost and Altman won. If it's not obvious to you that the board's legal rights and responsibilities had essentially zero to do with the outcome here, then I really don't know how to help you.
“Naive” is thinking that taking a poll of the public or employees (which is the only reasonable action I can think of that looks like “arguing their case”) would have had any positive effect on the outcome. This is simply… not how decisions are made. Even shareholder vote calls (for companies that have them) are coordinated, with the outcomes for consequential decisions well understood before voting happens. The optics for Open.ai certainly should have been better, but there is no version of this story where the board doesn’t do whatever they hell they want, and no version where “making the case” doesn’t result in even more chaos.
Corporations are not a democracy. They do not “owe” the public any information that they aren’t compelled to disclose. And they certainly don’t “argue their case” to some nebulous forum comprised of either the public or employees.
>there is no version of this story where the board doesn’t do whatever they hell they want, and no version where “making the case” doesn’t result in even more chaos.
The failure to make their case is PRECISELY why the board was ultimately unable to do what the hell they wanted: remove Sam Altman. As it turned out, his presence was important enough to the employees that the usual corporate playbook of "do whatever the hell we want and only disclose what is legally required" backfired spectacularly.
If your thesis was correct, Sam would not be there right now.
From reading your comment I think this might actually be a simple language misunderstanding.
You say:
> taking a poll of the public or employees
And then:
> which is the only reasonable action I can think of that looks like “arguing their case”
But that is a total non sequitur. "Taking a poll" has no relation whatsoever to "arguing their case". So I think you might not know what "arguing their case" means.
So I'll just plainly say what I think they should have done, without any jargon. I think they should have, after firing him, released a statement targeted at employees but also with a public audience in mind, something like this:
"We have simply lost faith in Mr. Altman to faithfully execute his duties as the executive of OpenAI's non-profit charter. We believe he has been acting in the interests of his own, which are not aligned with our mission. We have tried to redirect his efforts over a period of time, without success, and now are taking the only recourse that we believe is available to us to fulfill our duty to the organization. We understand that many of you will find this jarring and unsettling, but we hope you will continue to believe in the mission of OpenAI and stick with us through this trying and uncertain time, so that we can come out of it stronger and better aligned than ever."
Then anyone who quit would at least need to rationalize - to themselves, and to their social circles - why they chose not to take that to hear. Maybe for many / most / all of them, just "money" or a personal loyalty to Altman would have still won the day, but it certainly wouldn't have been as easy as it was to abandon a board that was seen as confusing and shambolic and refusing to explain itself.
That statement above is the part that would be "arguing their case". Just making the statement; the statement is the argument for that they did. Note that it doesn't include any sort of polling of anyone, or any different use of their legal rights or responsibilities. It's "just" PR, but that actually matters a lot.
To the public: For the normal PR reasons that every large enterprise is subject to caring about. To employees: I suspect substantially less than the like 95% of employees who came down against the board would have done so if the board had explained what they were thinking. There are probably at least a few people there who are not comfortable viewing themselves as rank mercenaries in service to a profit-and-power-motivated game-player, and actually joined OpenAI with some belief in its mission. But maybe not! I dunno.
Regardless of the board's failure to make their case, recent news suggests that the SEC is going to investigate whether it is true that Altman acted in the manner you describe, which would be a violation of fiduciary duty.
I agree that it seems like an open & shut case.
Typical SEC timelines mean that this will go public in about 18 months from now.
An anonymous person has already filed an SEC whistleblower complaint about the behavioral pattern of Altman and Nadella, which has SEC Submission Number 17006-030-065-098.
As the quid pro quo favoritism allegations remain under investigation, it is crucial to note that they are as yet unproven, and both Altman and Nadella are presumed innocent until proven guilty.
11 hours ago, the SEC tweeted the following new rule, which could be interpreted as a declaration that if Altman and Nadella are found guilty in this case, the SEC will block certain asset sales by OpenAI until the conflict of interest is unwound / neutralized:
The Commission has adopted a new rule intended to prevent the sale of asset-backed securities (ABS) that are tainted by material conflicts of interest.
Washington D.C., Nov. 27, 2023 — The Securities and Exchange Commission today adopted Securities Act Rule 192 to implement Section 27B of the Securities Act of 1933, a provision added by Section 621 of the Dodd-Frank Act. The rule is intended to prevent the sale of asset-backed securities (ABS) that are tainted by material conflicts of interest. It prohibits a securitization participant, for a specified period of time, from engaging, directly or indirectly, in any transaction that would involve or result in any material conflict of interest between the securitization participant and an investor in the relevant ABS. Under new Rule 192, such transactions would be “conflicted transactions.”
The Company exists to advance OpenAI, Inc.’s mission of ensuring that safe artificial general intelligence is developed and benefits all of humanity. The Company’s duty to this mission and the principles advanced in the OpenAI, Inc. Charter take precedence over any obligation to generate a profit.
Some analysts, including Stratechery writer Ben Thompson, have described the 2019 acceptance of Microsoft’s controversial investment by Altman as the beginning of a troubling pattern of Altman repeatedly making deals with Microsoft which were often unfavorable to OpenAI. ... As Thompson describes it, this pattern of behavior culminated in an unusual intellectual property licensing arrangement which Microsoft’s Investor Relations site describes as a “broad perpetual license to all the OpenAI IP developed t...
Interestingly, it appears that Microsoft has been caught violating SEC regulations at least 6 previous times, and that's just what I turned up in a cursory search:
And yet, they continue to exist and no CEO of MS ever stepped down because of any of these.
And I predict that even if Microsoft is going to be caught again that it will be a non-event in terms of actual repercussions. If Nadella exits MS HQ in Seattle in handcuffs I would be most surprised.
Yeah. I think the most realistically-achievable positive outcome is that Microsoft is forced to give up their new board-observer seat, which seems highly improper for them to have since the board they are observing is supposed to make decisions that benefit all of humanity equally. If Microsoft gets to have a fly on the wall in those discussions, it gives them a gold mine of juicy insider knowledge about which sectors of the economy are about to be affected next by Generative AI — knowledge which they can use to front-run the market by steering the roadmaps of Bing, MS Office, etc. so as to benefit from upcoming OpenAI product launches before any non-insiders are aware of what OpenAI is currently planning.
Microsoft plainly shouldn't be allowed to have this advantage, in that giving an advantage to any one party directly harms the mandate set forth in OpenAI's Charter:
Broadly distributed benefits
We commit to use any influence we obtain over AGI’s deployment to ensure it is used for the benefit of all, and to avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power.
Our primary fiduciary duty is to humanity. We anticipate needing to marshal substantial resources to fulfill our mission, but will always diligently act to minimize conflicts of interest among our employees and stakeholders that could compromise broad benefit.
It certainly seems that Microsoft, a "stakeholder", has managed to get a highly improper listening seat that will give them the ability to act on insider information about what's coming next in AI, allowing Microsoft to front-run the rest of the AI software industry and all those industries it affects, in a way that will plainly "compromise broad benefit". (Since any wealth that accrues excessively to Microsoft shareholders is not distributed to other humans who don't hold Microsoft shares.)
A mere 10 days ago, Nadella was shamelessly throwing his weight around on national TV, by appearing on CNBC where he improperly pressured the OpenAI non-profit board — which owes nothing to him legally or morally — to give him more deference, in direct violation of the "always diligently act to minimize conflicts of interest among our employees and stakeholders that could compromise broad benefit" provision of the OpenAI non-profit charter.
I doubt Microsoft will be punished for their role in this (it's not clear to me that they did anything wrong...), but I'll be surprised if OpenAI is allowed to still exist as a non-profit a couple years from now.
Of course they'll still be allowed to still be a non-profit. If they weren't "allowed" to, that would be a total victory for the Altman-aligned faction that sought to corrupt OpenAI from within and cause it to abandon its charter, and a total loss for Elon Musk and others that donated a total of $133M to the non-profit due to its charter. https://techcrunch.com/2023/05/17/elon-musk-used-to-say-he-p...
Musk and other donors are the most obvious aggrieved parties, and it is also arguably the case that every member of humanity has standing to sue for violation of the charter, because the charter explicitly declares that the primary fiduciary duty of OpenAI, Inc. (which is a non-profit) is to humanity broadly. (Therefore, every human is financially harmed by any charter violation, with such harm manifesting as a reduction in the net present value of the future benefits each human will receive from safe, broadly beneficial AGI.)
The SEC's role in fiduciary misconduct cases is not to rewrite a company's charter - that would be extremely improper and the opposite of their mandate. The SEC's mandate is to be the protectors of the status quo and of the original intent of the organizers of an entity. In this case, that means they will seek to protect the OpenAI non-profit's charter from efforts by Sam and others to erode the charter's power in violation of Sam's fiduciary duty to said charter.
The SEC's job in fiduciary misconduct situations is to remedy the situation by reversing improper governance decisions and forcing the fiduciary (Sam) to uphold their legal duty, which in Sam's case is his contractually bound duty to uphold the Charter of the non-profit entity named OpenAI, Inc.https://en.wikipedia.org/wiki/OpenAI#:~:text=the%20non%2Dpro....
OpenAI states this very clearly in multiple places on their website. For example:
"each director must perform their fiduciary duties in furtherance of its mission—safe AGI that is broadly beneficial" - https://openai.com/our-structure
If you would like to read about the types of remedies available in fiduciary duty violation cases, I recommend this resource:
For example (quoting from the book chapter above):
C. Permanent Injunction
A breach-of-fiduciary-duty plaintiff may be entitled to an award of a permanent injunction as a remedy. ...
The purpose of an injunction is to remove the advantage created by the wrongful act.
In the context of this suit, the permanent injunction or injunctions could block all of the following:
• Permanently prohibit Microsoft from holding any board seat or board observer seat on the OpenAI board
• Permanently prohibit Microsoft from making mass employment offers to OpenAI staff (a practice which, in the case of bad faith situations such as this one, is known as "workforce raiding")
• Permanently prohibit Sam Altman from owning Microsoft stock or derivatives thereof
• Permanently prohibit Microsoft from receiving early indications of OpenAI research & product roadmaps earlie...
I dunno, seems more likely that they'd shutter a nonprofit that's a clear sham and assess a bunch of taxes and penalties on the for-profit subsidiary that was the beneficiary of the sham. But beats me!
It was supposed to have two functions: tax dodge + pull the wool over the eyes of regulators. It may fail on both counts now that they've shown that the nonprofit wasn't functional in its stated capacity. Win the battle, possibly lose the war?
Yeah this is what I think as well. Certainly regulation won't be avoided now, and regulators will (or should) know not to trust Altman to contribute to developing it.
The Michigan Post article is mostly speculation and that publication doesn’t have much depth/history to it. Check out their “advertise with us” page.
This whole info dump feels like a mishmash of links to thoughtful things (Stratechery) with links to speculative articles that are clearly biased. Like how is “Influencer Magazine” breaking a story that Wall Street Journal and Kara Swisher are overlooking?
I don’t mean to be a jerk. Just really unconvinced.
I would guess that "WLW FUTURE PRESS RELEASE DISTRIBUTION" is a publicist service that was hired by the person making the whistleblower complaint. upwardbound claims there's a lot of money in being a successful whistleblower: https://news.ycombinator.com/item?id=38388246
I don't know if I find the Microsoft/OpenAI favor trading allegation that persuasive (unless new information is uncovered, e.g. Microsoft letting Sam use a private jet or something like that). However if the SEC actually ends up enforcing the "fiduciary duty is to humanity" thing in OpenAI's charter at some point, that would be incredibly sweet.
In the financial world asset backed securities are typically anything that isn't a mortgage, I think that some crypto would qualify as well as the rights to future profits on some venture.
so much this, he kept introducing clauses in contracts that tied investments to him, and not necessarily to openai. He more or less did it with microsoft, to a small degree. So his firing could have caused quite a lot of money to be lost. But ok no big deal.
But then he tried to do it again with a saudi contract. OpenAI board said explicitly they didn't want the partnership, and especially not tied to Altman personally being the CEO as a clause.
Altman did it behind their back -> fired.
This is the rumour on the streets, unconfirmed though
My take is that the board probably never had a chance no matter what they said or did. The company already "belonged" to Altman and Microsoft. The board was just there for virtue signaling and for quite a while already had no real power anymore beyond a small threat of creating bad publicity.
I think they could have actually blown up the whole thing and remained in charge of a greatly-diminished but also re-aligned non-profit organization. A lot of people (like me) would have thought, holy crap, that was insanely bold and unprecedented, I can't believe they actually did that, but it was admirably principled.
Instead it was a confusing shambles that just left them looking like idiots with no plan.
For one thing, we vote for the governments who are now definitely going to heavily regulate them after this fiasco.
But more generally, public perception just is important to any large and broadly targeted enterprise, because "the public" approaches "our customer base" as such an enterprise scales up. Think of companies like Google or Microsoft or Meta or Amazon; essentially everyone (in the US) uses their services in some form or another, so the perception of "the public" is indistinguishable from the perception of "the people who use our services". ChatGPT isn't quite at that point yet, but it's close enough.
For me it seems to be a debate between moral versus money. Is it morally correct to create technology that would be extremely profitable but has the potential to fundamentally change humankind.
Maybe the safety concerns are from a vocal minority and most are quiet and don't think much about or don't actually think ai is really that close. It could just be hysterical people or people who get traffic from outrageous things
My pet theory is that Altman found out about Q* and planned to start a hardware company to make chips accelerating it, all without telling the board. Which is both dangerous to humanity and self-serving. It’s also almost baseless speculation; I’m interpolating on very, very few scraps of information.
Even if -- and that's a big if -- it really was just a dispute over alignment (nonprofit vs for-profit, safety, etc.), the board executed it really poorly and completely misjudged their employees' response. They saw the limits of their power / persuasiveness compared to [Altman/the allure of profit/the simple stability and clarity of day-to-day work without a secretive activist board/etc]
Or maybe they already knew the employees weren't on their side, saw no other way to do it, and hoped a sudden and dramatic ouster of the CEO would make the others fall in line? Who knows.
I'd be pretty concerned too if my CEO was doing what I considered a great job and he was suddenly removed for no clear reason. If the board had explained its rationale and provided evidence, maybe some of the employees would've listened. But they didn't... to this day we have no idea what the actual accusation was.
It looks like a failed coup from the outside, and we have no explanations from the people who tried to instigate it.
Let's also keep in mind that if the AI doomers are right and spicy autocomplete is just a few more layers away from taking over the world, OpenAI has completely failed at building anything that could keep it under control. Because they can't even keep Sam Altman under control.
...actually, now that I think of it...
Any creative work - even a computer program - tends to be a reflection of the organizational hierarchies and people who made it. If OpenAI is a bunch of mad scientists with a thin veneer of "safety" coating, then so is ChatGPT.
It also isn't clear why Altman couldn't have been replaced by someone else with literally no change in operations and progress. It is just really confusing why people acted as if they fired Michael Jordan from the Bulls.
He is obviously a great leader and those that work there wanted to work with him. It’s very clear in this thread how undervalued exceptional leadership actually is, as evidence by comments thinking the top role in the most innovative company could be just plug-and-play.
This comment made me look for a recent article about Paul Graham firing him from Ycombinator for being exactly not a great leader or trustworthy person.
The article was just days ago but it’s eluding my search.
Would love to see it. Everything I’ve read/seen from PG regarding Sama has been nothing but high praise. My understand is Sam chose to leave YC president role to pursue other interests/ventures which eventually turned into OpenAI
I'm going to guess it's not about leadership. From the Lex Fridman interview he claims to be personally involved in all hires - and spend a good fraction of his time evaluating candidates.
- He's not going to hire someone he doesn't like
- Someone that doesn't like him is unlikely to join his team
So it's very likely the whole staff ends up being people that "like" him or get along with him. He did come off as a charming smooth talking - and I'm sure he has lot of incredibly powerful friends/connections. But at least from that little window into his world I didn't feel he showed any particularly brilliance or "leadership". He did seem pretty deferential to ML experts (which i guess he's not) - but it's hard to know if it's a false humility or not
It's also an unvalidated claim, predicated upon assumptions.
I've hired people I "don't like" on a personal level. I care more about their ability to work positively with oters, and their professionalism and skill.
Yet you and the parent poster have assumed he is hiring a cult, because he spends time evaluating?
oh sorry - I didn't mean it in a nefarious way at all
I think it's just human nature to not hire people you feel you won't get along with. If you're deeply involved in all your hires, then I feel you'll end up with an organization full of people that you get along with and who you like (and probably like you back). I wouldn't go so far as to say it'd make a personality cult - though with their lofty mission statements and ambitions to make the world better.. who knows. Not going to psychoanalyze a bunch of people I don't know
"I've hired people I "don't like" on a personal level."
I'm honestly impressed... I feel that's rather exceptional. I feel a lot of hiring goes on "gut feeling"
Perhaps my gut feeling is just tuned more towards competence, than personality sync? I often find lacking competencies to be more jarring than variant personalities.
Why is your argument from authority relevant at all here?
1. For that matter, I would not work with a boss who doesn't recognize an argument from authority as a response.
2. Furthermore a boss who lacks critical thinking skills and can't recognize a mildly skeptical comment for what it is.
3. Why assume I'm rigidly taking any position re. Altman in particular?
4. "That's pathetic" is not "He's pathetic", so perhaps the problem here is your low reading comprehension level, and not any particular assumption that I have committed to at all.
5. "A weird assumption" -- so, you decide it's weird because you just read it wrongly? Or, if something seems weird, why not ask a curious question and find out? Listening skills? Why is it so important to make this about another commenter?
I find your poor faith interpretation of my comment to be offensive. You've at the used your professionalism as pretense to biasedly cast judgment on what is a critical or even mildly skeptical general remark. This reflects the bad side of tech culture. You should apologize.
Buddy, trying to pretend your comment didn't exist in the contextualized space it did, when you replied, is not viable. And your response is way out in left field.
There's exceptional leadership, and then there are charming sociopaths.
Unfortunately, it can sometimes be hard to tell those two apart when a sociopath is actively pushing your emotional buttons to make you feel like they care about the same things as you etc.
If employees would have voted Sam out, you'd take that as a shiny example of the proletariat exercising the power for the good of human kind, hammer, sickle and all that.
I always find it funny when people understand democracy to mean "other people that should vote my way, otherwise they are imoral and should be re-educated".
First, I'm actually quite libertarian and capitalist -- although not necessarily so when it comes to companies working on powerful AI (or fighter jets for that matter). Here are some comments of mine from other discussions if you don't believe me:
And by the way, here is a comment I made the other day making essentially the point you are making, that in a democracy everyone is entitled to their opinion, even the dastardly Elon Musk: https://news.ycombinator.com/item?id=38261265 And I also argue in favor of freedom of speech here, for whatever that's worth: https://news.ycombinator.com/item?id=37713086
Point being, I'm not sure our disagreement lies where you think it does.
The purpose of the board is that they're supposed to be disinterested representatives of humanity, supervising OpenAI. The employees aren't chosen for being disinterested, and it seems quite likely that they are, in fact, interested, per my link.
From the perspective of human benefit (or from the perspective of my own financial stake in OpenAI, given that their charter says their "primary fiduciary duty is to humanity"), I prefer a small group of thoughtful, disinterested people over a slightly larger group whose interest is systematically biased relative to the interest of me or the average person. Which is more likely to produce a fair trial: a jury of 12 randomly chosen citizens, or a jury of 1200 mafiosos?
I don't think that means it intends to pay a financial dividend to each and every person on the planet. I think it means that if it is successful at AGI, that in itself will expand the economy enough to have the same effect.
> pay a financial dividend to each and every person on the planet
I think it could mean this, in the context of Altman's other project (WorldCoin) which despite all its controversy is ostensibly intended as a vehicle for distributing an AGI-funded Universal Basic Income (UBI) to all of humanity.
Introducing Worldcoin
a potential path to AI-funded UBI.
I dunno about this thought, are there other AI startups operating at this level and that have the amount of market share and headspace that OpenAI has? I see comments like this on hacker news a lot, and I get that yes, the man is human and fallible, but they are doing something that’s working well for their space. If there’s some compelling reason to doubt Altman’s leadership or character I haven’t heard it yet.
a sane company has a plan for succession, even if worst case scenario Altman has a sudden medical issue or car crash or something.
It tells a lot that Altman made openAI so dependend on him that his ousting could have killed the company. That's also contributing to the fact that the board was not trusting him
They had a couple of people on the board who had no right being there. Sam wanted them gone and they struck first by somehow getting Ilya on their side. They smeared Sam in hopes that he would slink away, but he had build so much goodwill with his employees that they wouldn't let it stand.
They probably had smeared people before and it had worked. I'm thrilled it didn't work for them this time and they got ousted.
This sounds like a lot of conjecture. Those people definitely had a right to be there: they were invited to and accepted board positions, in some cases it was Sam himself who asked them to join.
But an oversight board can be established easier than that it can be disbanded and that's for very good reasons. The only reason that it worked is not because the board made any decisions they shouldn't have made (though that may well be the case) but because they critically misjudged the balance of power. They could and maybe should have made their move, but they could not make it stick.
As for the last line of your comment: I think that explains your motivation of interpreting things creatively but that doesn't make it true.
>To be clear: our decision was about the board's ability to effectively supervise the company, which was our role and responsibility. Though there has been speculation, we were not motivated by a desire to slow down OpenAI’s work
> Sam directing IT and Operations staff to conduct investigations into employees, including Ilya, without the knowledge or consent of management.
> Sam's discreet, yet routine exploitation of OpenAI's non-profit resources to advance his personal goals, particularly motivated by his grudge against Elon following their falling out.
> Brad Lightcap's unfulfilled promise to make public the documents detailing OpenAI's capped-profit structure and the profit cap for each investor.
> Sam's incongruent promises to research projects for compute quotas, causing internal distrust and infighting.
Do you comprehend that if it weren't for Microsoft OpenAI wouldn't exist, nor their products which changed the world?
You don't make AI in a garage with a home PC and beer. You need gigantic amount of data, compute, and talented people to organize all this, which are exceptionally rare, and also they need to eat and feed their families too.
Oh yeah, I do comprehend how such major technologies of comparable importance are managed in NON-PROFIT organizations, Linux Foundation is for one.
If you're happy to give up total control of tech to one corporation, while the CEO of the company, using the influence of the OpenAI, attracts resources for his personal hardware start-up[1], — our views on how a not-for-profit company should be run are starkly different.
You can work on and compile Linux on a home PC, even more the case with their other projects. Try training GPT-4 on a home PC.
The problem is you have no idea what you're talking about. This is not about how important something is, but what resources it requires as a BASIS MINIMUM for it to happen at all. Linux is important, but it's not nearly as expensive to create or maintain. If it's so easy for a scrappy non-profit to attract billions to tinker with AI, where I say are those startups? Where? Q.E.D. The most open thing we have, Llama, came from giant Meta.
Regarding the hardware startup, neither me nor anyone else is defending Altman about that, nor it has any relation to Microsoft. If you want to argue the subject of this thread, you're welcome. If you just want to copy paste generic talking points with no relevance, I don't care.
> You can work on and compile Linux on a home PC, even more the case with their other projects. Try training GPT-4 on a home PC.
Sam Altman was happy to join the venture and lead OpenAI knowing that it was the non-profit AI research company from the beginning. As a CEO, he was supposed to lead the company as it was created. As a result of his management, the company first ceased to be open, then in fact non-profit, then safe (according to it's leading scientists), then the number of published research whitepapers fell sharply, then has taken steps to become a gatekeeper for government regulations, and finally, the communication between Sam and the board (which really should be running the company) was lost. If a CEO was unable to manage a company, including attracting investment and resources, without violating the fundamental principles of that company, that CEO should have been dismissed — what has been effectively attempted by the board. Otherwise, it's a power grab and a de facto reformatting of the company into a different entity. Do you comprehend that?
I have no idea what compiling Linux kernel, or running GPT-4 locally, does anything in common in the context of large non-profit organization management. My argument is that the company should have been run in such a way that one corporation could not seize complete control. Never, under no circumstances. In spite of this, the CEO made all the effort to make it so. As a result, we have now in fact a for-profit company with good product, tons of money and GPUs, that belongs to the Microsoft. Was it worth it? Does this justify it for you? If you are not a $MSFT shareholder the answer should be obvious.
> If it's so easy for a scrappy non-profit to attract billions to tinker with AI, where I say are those startups? Where? Q.E.D. The most open thing we have, Llama, came from giant Meta.
Take a good look at HuggingFace Chatbot Arena leaderboard page[1]. Pay attention to the values of different models on different benchmarks. GPT4 is the leader, but not by an unattainable margin. Anthropic, which has spinned off from the very same OpenAI, rapidly catches up, along with other solutions. Even free, open (for commercial usage) models, while being self-hostable, are not an orders of magnitude worse. They are well in range of being practical and useful, including for commercial products. There is no moat[2].
> Linux is important, but it's not nearly as expensive to create or maintain.
> The problem is you have no idea what you're talking about.
The company was supposed to be funded by Elon who pledged a billion. Then he bailed when his coup failed. Sam Altman had to find a sponsor, or have the company go bankrupt in a matter of weeks.
What's hard to comprehend here?
As for the models topping your Hugging Face chart, they're literally all funded (by the billion) by private enterprises and the US govt. It's baffling to me how this is supposed to be an argument in your favor. Even the "not by an unattainable margin" comment makes no sense, unless you think getting from 40% to 60% is as easy as getting from 60% to 80%, and then getting from 80% to 100%. Which you clearly seem to think.
Anthropic's models are closed and funded by the money of Sam Bankman-Fried, of all people, and more recently Amazon. They're trying to raise billions more from Google and others. That's your idea of being open and not needing corporate investment for some reason? LOL, oh my god.
The "free" model you're citing is MADE BY META, a company worth almost a trillion dollars, and which has the largest social media presence in the world, from which to mine data.
You have absolutely not even the faintest idea what you're talking about.
You're either pretending or incapable of understanding the point that is being reiterated in this thread.
The OpenAI company, that was supposed to be non-profit open research think lab, has been taking investments of dozens of millions from different funds, while producing highest quality research material for the public good. Up to the point when the CEO sold it for Microsoft's $1B deal that solved their infra & GPU resource problems for… completely locking the company as an unofficial division of Microsoft Research. This deal, essentially, entirely distorted the raison d'être of the non-profit company, closed the "open research" part, and made it a for-profit extension for MS/Bing services.
Here, perspectives fork in two ways: either you believe it's a great development that you can benefit of by using closed paid OpenAI/Microsoft commercial services, OR you believe that metamorphosis of the leading open AI research shop into it's exact antipode is embodiment of disappointment. From your boorish, superficial remarks, I can see which side you are on.
I don't care if Sam Altman took money from Microsoft, or Sam Bankman-Fried (before he was charged), or US govt — that entirely misses the point. I do care that Sam Altman orchestrated a complete relegation of the research lab to an overpowered for-profit "Clippy" assistant for a single huge corporation, depriving us from the open research. He had to work on attracting investments from several corporations and other entities to avoid centralisation of control. But we see that he acted according to his own personal agenda, not in the interests of the company's charter. In that perspective, I don't give a damn if he could have raised a billion or not as a non-profit company. I'd rather it was $300-500M that would have kept the company open than a $1B and what we have today. I'd rather the company didn't survive this crisis than what we have today.
I gave you the benchmarking of other LLM models that are several months (up to a year) behind GPT4 in performance development. I don't care if they are founded by megacorps or not, if they do what non-profit "Open" AI no more does, i.e. publishing open research and supplying us open models. Comprehend? Your argument of "if it weren't for Microsoft OpenAI wouldn't exist" is irrelevant, if OAI no longer produces open research that it supposed to; it better wouldn't otherwise. Let others for-profit multi-billion corps take over, if we can't have better.
To further refute your $MSFT shill demagoguery, narrowly focused on compute accelerators, I'll add that Abu Dhabi's TTI (UAE gov money for academia) has produced and released open model Falcon 180B, while French nonprofit research lab Kyutai has just raised €330M. In Europe, for a minute, not even SV! Apparently, funding open research is possible? Q.E.D. I'd rather Ilya and Andrej move to this lab than peddling "Laundry buddy" GPT Bing API tokens BS disgrace, but it's up to them.
And you can stay in your delusions where only OpenAI+Microsoft is capable of leading AI research, lead by an ego inflated CEO. Don't forget to scan your eye pupils for $worldcoin.
OpenAI, the original non-profit, DIED. Elon Musk pulled the funding, it DIED. It's DEAD, get it? It does not exist without money. I don't know how to get it through your head. There's no such thing as a serious AI company without billions of funding. Period.
So. Given that, to keep the team together, Sam made the for-profit subsidiary and hosted it in the hollowed out non-profit, Sam went out to get investors and Microsoft chimed in the most.
While the new hybrid OpenAI tries to respect the spirit of the original OpenAI... WHICH IS DEAD (GET IT?)... it has to also balance between non-profit mission, and its investors, thanks to which it exists at all.
That's kind of like maybe you think you're a world-leading poet and novel writer, but your boss doesn't care, he wants those toilets clean. That's what he's paying for. You can then use that money to spend your free time writing poems.
Not sure I have to respect the rest of your BS with an answer. "Oh the other megacorp LLMs are just few months behind GPT4". Newsflash! OpenAI also exists within... time, genius. Time is passing there, too. And GPT-5 is ready and few Fortune 100 companies are testing it currently. They're not sitting still and waiting for the others to catch up.
I never said other companies can't compete, in general, but they're no better than OpenAI. Amazon, Google, Inflection, Facebook are no better than OpenAI. They're not. And I never mentioned anything about Worldcoin, you're basically desperate to strawman this, but you are exceptionally bad at it.
Did I say you don't have the slightest, faintest, most distant idea what you're talking about? Anyway this is my last message.
I bet there are tons of seemingly "conflict of interest" in companies boards. In fact, being major shareholder qualifies (but not sufficiently) for the board. Jobs was Disney's board member after Pixar's acquisition. You could argues Apple is now a competitor to Disney. So what? If a competitor acquires enough control of a company what's wrong?
It seems hard to imagine a board member that is involved in another tech business not having a conceivable conflict of interests. LLMs are on a course to disrupt pretty much all forms of knowledge work after all. Also big implications for hardware manufacturing and supply chains.
Have you guys considered that maybe he's there because he's extremely qualified and extremely well-respected by his peers? It's not some kind of weird power play, he's just lending a hand while they figure out the long-term board composition.
Not to criticize Sam, but I think people don't realize that it was Greg who was the visionary behind OpenAI. Read his blog. Greg happens to be a chill, low-drama person and surely he recruited Sam because he knew he is a great communicator and exec, but it's a bit sad to see him successfully staying out of the limelight when I think he's actually the one with the rarest form of genius and grit on the team.
I try hard to stay away from conspiracy theories as they are almost always counterproductive, but D'Angelo still being on the board seems insane to me. Does this guy have some mega dirt on someone or something? Does he have some incredible talent or skill that is rare and valuable?
I mean maybe it’s all a master plan of Sam’s, but that still requires the board to be dumb enough to literally fire him in the stupidest way possible and refuse to communicate anything about WHY. So maybe he made up something to give them the whole “not being candid” argument - call the bluff. Tell people why. If he lied or made up the meta lie of not being canid, then that’s great info to share. But they haven’t!
Sam said on a recent Joe Rogan episode that he can be a bit of a troll online and he felt some sort of discomfort with it. I do think Sam is probably sort of an asshole in certain situations (which means you’re an asshole but hide it well usually). But to be honest, most people are assholes when you push the right buttons.
"Open"AI and for-profit. This company is a troubled mess and if it were not for the "potential" money, employees wouldn't be putting up with this nonsense. Sad to see Ilya getting sidelined as shown by this "...are discussing how he can continue his work at OpenAI".
786 comments
[ 4.7 ms ] story [ 332 ms ] threadI think the percentages don't add up to 100% as multiple can be chosen as correct.
Strange when their choice of interim-CEO was someone who explicitly stated he wants to see the pace of AI development go from a 10 to a 2 [1] and she regularly speaks at EA conferences where that's a major theme.
This is probably double speak for she want's to not "slow down OpenAI's work" on AI safety but probably would have kneecapped the "early" release of ChatGPT's (as she claim they should have waited much longer in her paper) and similar things.
[1] https://twitter.com/eshear/status/1703178063306203397
Alternatively, maybe it became clear to the board that Sam was trying to either bully them into becoming loyalists, or replace them with loyalists. As a board member, if the CEO is badgering me into becoming a yes-man and threatening to kick me off if I don't comply, I can't exactly say that I'm able to supervise the company effectively, can I? See https://www.lesswrong.com/posts/KXHMCH7wCxrvKsJyn/openai-fac...
Once you view it like this, I wouldn't put it past them to blatantly lie. Looking at the facts as you say, they tried to replace a guy that is moving ahead with a guy that wants to slow it down to a crawl, that's pretty much all we need to know.
It just doesn't fit with everyone's behavior -- that's something that would have been talked about loudly.
Altman lying to the board, especially in pursuit of board control, fits more cleanly with everyone's actions (and reluctance to talk about what exactly precipitated this).
Fits much more cleanly with the lack of information, and how no one (on any side!) seems overly eager to speak to specifics.Why jump to AGI as an explanation, when standard human drama will suffice?
And absolutely Microsoft would have behaved as it did -- it doesn't really care about OpenAI-the-company; it cares about OpenAI-the-tech.
Also, see essentially the same said by Microsoft's president: https://www.bbc.com/news/technology-67578656
Sounds like confirmation of the theory that it was Sam trying to get Toner off the board which precipitated this.
So the AI of the future was more like Bender or other robots from Futurama that display all the same flaws as people.
So far it feels more like a gpt-3.75-turbo rather than really being at the level of gpt-4. The speed and massive context window are amazing though.
id be willing to bet enshittification is on the horizon. you don't get the shiny 70b model, that's for gold premium customers.
by 2025, it's gonna be tiered enterprise prices.
Like "AGI, do all this random shit for me!"
AGI: No! i don't wanna!
Ad infinitum.
Also, one of the simplest algorithms to get to the bottom of anything.
https://archive.org/details/imaginarymagnitu00lems
for purchase:
https://www.kobo.com/ww/en/ebook/imaginary-magnitude-1
and via some zlib proxy if you're going full pirate.
https://annas-archive.org/search?q=Imaginary+magnitude
and I'm not sure why anyone dances around it but these models are built by unfiltered data intake. if they actually want to harness bias, they need to do what every capitalist does to a social media platform and curate the content.
lastly, bias is an illusion of choice. choosing color over colour is a byproduct of culture and you're not going to eradicate that but cynically, I assume you mean , why won't it do the thing I agree with.
edit: I'm asking because to my eye most of these conversations revolve around jargon mismatch more than anything else.
See twitter for example, ad buyers decided it is no longer politically correct so twitter lost a lot of ad revenue. Avoiding that is one of the most important things if you want to sell a model to companies.
Not all filtering has to be prohibitive, just unnoticed.
It's such a strange thing apparently they can tune gpt4 turbo cleverness up and down on the fly depending current load.
[1] https://en.wikipedia.org/wiki/Reinforcement_learning_from_hu...
There are no board that can prevent employees from leaving, nor should there.
The fallout showed non-profit missions can't co-exist with for-profit incentives. And the power that investors were exerting, and employees (who would also benefit from the recent 70B round they were going to have) was too much.
And any disclaimer the investors got when investing in OpenAI was meaningless. It reportedly stated they would be wise in viewing their investment as charity, and they can potentially lose everything. And there was an AGI clause that said it will reconsider all financial arrangements, that Microsoft and other investors had when investing in the company was all worthless. Link to Wired article with interesting details -https://www.wired.com/story/what-openai-really-wants/
I know you're quoting the (now-gone) board member, but this is a ridiculous take. By this standard, Google should have dissolved in 2000 ("Congrats everyone, we didn't be evil!"). Doctors would go away too ("Primum non nocere -- you're all dismissed!").
They need employees to advance their stated mission.
> One of the comments a member made was, if the company was destroyed, it would still be consistent with serving the mission. Which is right.
I mean, that's a nice sound bite and everything, but the only scenario where blowing up the company seems to be consistent with their mission is the scenario where Open AI itself achieves a breakthrough in AGI and where the board thinks that system cannot be made safe. Otherwise, to be relevant in guiding research towards AGI, they need to stay a going concern, and that means not running off 90% of the employee base.
That's why they presumably agreed to find a solution. But at the same time shows that in essence, entities with for-profit incentives find a way to get what they want. There certainly needs to be more thought and discussion about governance, and how we collectively as a species or each company individually governs AI.
The silver lining is that this should clear the path to proper regulation, as it's now clear that this self-regulation approach was given a go, and just didn't work.
Having said that, as there’s obviously a lot of money involved for all OpenAI employees (Microsoft offered very generous packages if they jumped ship), it can be said that money in the end is what people care a lot about.
However, as you note, Sam’s exit would have cost those employees at least high six figures, so I’m sure that reduced the amount of leverage required to extract signatures.
The misalignment is not accidental, it was carefully developed over the last few years.
I agree it's not that deep - they wanted to join a hypergrowth startup, build cool stuff, and get paid. If someone is rocking the boat, you throw them off the boat. No mission alignment needed! :)
It made it easy to sway employees to Altman's favor and pressure the board.
If the employees were not cohesive on Altman's side, he probably wouldn't be back...
There will be no OpenAI to begin with had that not been the case.
I don't really care much about the demise of OpenAI-the-nonprofit - I don't think it was ever a very promising approach to accomplishing its own goals, so I don't lament its failure all that much - but I feel like there is a kind of gaslighting going on, where people are trying to erase the history of the organization as a mission-oriented non-profit, and claim that it was always intended to be a wildly profitable product company, and that just isn't the case.
I don't buy the Q* talk.
I still think the board did what they thought was right (and maybe even was?), but not having the pulse of the company and their lack of operational experience was fatal.
So out of all the companies, possibly only Google can provide a model to him right now to do the alignment work
[1]: https://www.ft.com/content/26372287-6fb3-457b-9e9c-f722027f3...
But I guess he will stay low key for a longer time now ...
[1] https://www.businessinsider.com/elon-musk-justified-poaching...
If you don't like the way HN works that's fine but I don't think you should be making further statements here until you've had a look at the guidelines.
Only a matter of time.
I recognize that during this process some questions were raised about Adam’s potential conflict of interest running Quora and Poe while being on the OpenAI Board. For the record, I want to state that Adam has always been very clear with me and the Board about the potential conflict and doing whatever he needed to do (recusing himself when appropriate and even offering to leave the Board if we ever thought it was necessary) to appropriately manage this situation and to avoid conflicted decision-making. Quora is a large customer of OpenAI and we found it helpful to have customer representation on our Board. We expect that if OpenAI is as successful as we hope it will touch many parts of the economy and have complex relationships with many other entities in the world, resulting in various potential conflicts of interest. The way we plan to deal with this is with full disclosure and leaving decisions about how to manage situations like these up to the Board. [1]
The best interests of the company and the mission always come first. It is clear that there were real misunderstandings between me and members of the board. For my part, it is incredibly important to learn from this experience and apply those learnings as we move forward as a company. I welcome the board’s independent review of all recent events. I am thankful to Helen and Tasha for their contributions to the strength of OpenAI. [2]
[1] - https://twitter.com/sama/status/1730032994474475554 [2] - https://twitter.com/sama/status/1730033079975366839
The kind of bullshit that comes out during times like this is more than a little bit annoying, it's clear that if there is a conflict of interest it should be addressed in a far more direct way and whitewashes like this raise more questions than they answer. Such as: what was Adam's real role during all of this and how does it related to his future role at the board, as well as how much cover was negotiated to allow Adam to stay on as token sign of continuity.
I'd read the hell out of that alt-history novel, I can tell you that much. Not so much the "Manhattan Project" as the "Tuxedo Park Project."
But it's an interesting thought. Howard Hughes came close to having that kind of power and Musk has more or less eclipsed him now. Sam Altman could easily eclipse both, he seems to be better at power games (unfortunately, but that's what it is). Personally I think people that are that power hungry should be kept away from the levers of real power as much as possible because they will use it if the opportunity presents itself.
> The best interests of the company and the mission always come first.
That is absolutely not true for the nonprofit inc. The mission comes first. Full stop. The company (LLC) is a means to that end.
Very interested to see how this governance situation continues to change.
In private, I can only assume he’s a lot less diplomatic.
How do you know this?
Idk why this is the comment that broke the camel's back for me, but all over this site people have been making character determinations about people from a million miles away through the filter of their public personas.
If you don't - there was no need to be coy. And you act as if failing up never happens in real life.
Failing up definitely happens but given the massive number of failures you'd have to succeed through to be US president, as an example, it's not what's happening there.
Loyalty can be blinding.
At $DAYJOB we are working various AI features and it was a lot easier to get Azure through our InfoSec process than OpenAI.
Second, I simply don't understand what just fucking happened. This whole story doesn't make sense to me. If I was an OpenAI employee, after receiving this nonsense "excited about the future" statement, I would feel just exhausted, and while maybe not resigning on the spot, it surely wouldn't make me more excited about continuing working there. But based on the whole "500 OpenAI employees" thing I must assume that the average sentiment in the company must be somewhat different at the moment. Maybe they even truly feel re-united. Sort of.
Obviously, I don't mean anything good be all that. What happens if Altman is hit by a bus tomorrow? Will OpenAI survive that? I don't even know what makes him special, but it seems we've just seen a most clear demonstration possible, that it wouldn't. This isn't a healthy company.
That said, all that would worry me much more, if I was an investor. In fact, I'd consider this a show-stopper. As a customer? It doesn't make me more reassured, but even if Altman is irreplaceable, I don't feel like OpenAI is irreplaceable, so as long as it doesn't just suddenly shut down — sure, why not. Well, not more comfortable, of course, but whatever.
If we are going to rank the concerns around openai announcements from the past 2 weeks, I'd bet the more concerning one was the initial firing decision.
might as well just call it "line goes up"
No matter how idealistic you are, you won't be happy when your compensation is reduced from 600k to 200k.
Because he's the one who's promising to make them all rich.
It isn't clear to anyone else either.
I think where they really screwed up was in being unwilling or unable to argue this case.
I wouldn't put these in the same category as "90% of staff leaves for Microsoft".
In any case, let's not confuse unsuccessful with incompetent. (Or incompetent with immoral, for that matter.)
We know that OpenAI does a staged release for their models with pre-release red-teaming.
Helen says the key issue was the board struggling to "effectively supervise the company": https://nitter.net/hlntnr/status/1730034017435586920#m
Here's Nathan Labenz on how sketchy the red-teaming process for GPT4 was. Nathan states that OpenAI shut him out soon after he reached out to the board to let them know that GPT4 was a big deal and the board should be paying attention: https://nitter.net/labenz/status/1727327424244023482#m [Based on the thread it seems like he reached out to people outside OpenAI in a way which could have violated a confidentiality agreement -- that could account for the shutout]
My suspicion is that there was a low-level power struggle ongoing on the board for some time, but the straw that broke the camel's back was something like Nathan describes in his thread. To be honest I don't understand why his thread is getting so little play. It seems like a key piece of the puzzle.
In any case, I don't think it would've been right for Helen to say publicly that "we hear GPT-5 is lit but Sam isn't letting us play with it", since "GPT-5 is lit" would be considered confidential information that she shouldn't unilaterally reveal.
Reducing the cost of medical transcription to ~$0 is one thing. Reducing the cost of assassination to ~$0 is quite another.
In any case, an F16 fighter jet is amoral in a certain sense, but it wouldn't be smart to make F16s available to the average Joe so he can conduct an airstrike whenever he wants.
It’s kind of a moot point since only one F16 exists in civilian hands but you can buy other jets with weapon hardpoints for a million. Under 3 million if you want to go supersonic too. The cheapest fighter jet is on the order of $250k. There’s zero background check.
(We do disagree about what constitutes "immoral", which makes this much harder).
Because, users should just input the moral inputs to these things. These are inanimate objects too.
Oh, while we're at it, we should also remove battery charge controllers. Just do the moral and civic thing and unplug when your device charges.
Assuming that you're not running this query over an API and relaying these answers to another control system or a gullible operator.
An aircraft control computer or reactor controller won't run my commands regardless of its actuators connected or not. Same for weapon systems.
Hall pass given to AI systems just because they're outputting text to a screen is staggering. Nothing prevents me to process this output automatically and actuate things.
Simplified operations, faster reaction time, eliminating human resistance for obeying killing orders. See "War Games" [0] for a hypothetical exploration of the concept.
> a chatbot.
Some claim it's self-aware. Some say it called for airstrikes. Some say it gave a hit list for them. It might be a glorified Markov-chain, and I don't use it, but there's a hoard of people who follows it like it's the second Jesus, and believe what it emits.
> I can't see it happening.
Because, it already happened.
Turkey is claimed to use completely autonomous drones in a war [1].
South Korea has autonomous sentry guns which defend DMZ [2].
[0]: https://www.imdb.com/title/tt0086567/
[1]: https://www.popularmechanics.com/military/weapons/a36559508/...
[2]: https://en.wikipedia.org/wiki/SGR-A1
Speaking of which, it's perfectly legal for a civilian to own a fighter plane such as an F-16 in US and many other countries. You just have to demilitarize it, meaning no weapon pods.
The reason this isn't an issue in practice is because such repurposing would require significant intelligence/electrical engineering skill/etc. The point is that intelligence (the "I" in "AI") will make such tasks far easier.
>Ten-year-old about to play chess for the first time, skeptical that he'll lose to Magnus Carlsen: "Can you explain how he'll defeat me, when we've both got the same pieces, and I move first? Will he use some trick for getting all his pawns to the back row to become Queens?"
https://nitter.net/ESYudkowsky/status/1660399502266871809#m
The guy says he wanted to slow down the progress of AI and GPT suggested a detailed assassination plan with named targets and reasons for each of them. That's the problem.
When it comes to those "detailed plans", if you actually try something like that, what you get from it is a very broad outline that is pretty much a mish-mash of common sense stuff and cultural tropes (many of which aren't true IRL). Similarly, the "list of targets" that it makes is simply the most prominent people associated with that area in public perception, not necessarily people who are actually key. All of this can be achieved just as well with a few Google searches, and the resulting plan will likely be much better for it.
I've yet to see any example where GPT would come up with something along these lines that is not trivially found on the Internet anyway.
And everything LLM can come up with will be exactly the same information you can find in any fiction detective book or TV series about crime. Yeah very very dumb criminal can certainly benefit from it, but he can as well go on 4chan and ask about assassination there. Or on some detective book discussion club or forum.
As Nathan states:
>And further, I argued that the Red Team project that I participated in did not suggest that they were on-track to achieve the level of control needed
>Without safety advances, I warned that the next generation of models might very well be too dangerous to release
Seems like each generation of models is getting more capable of thinking, beyond just regurgitating.
You don't have bunch of cold blood killers going around not just because police is so good and killers are dumb and need AI help. It's because you live in functioning state where society have enough resources so people happy enough to instead go and kill each other in Counter Strike or Fortnite.
I totally agree that AGI could be a dangerous tech, but it's will require autonomity where it can manipulate real world. So far GPT with API access is very far from that point.
The danger also doesn't need to be an AI generating code to hack the Gibson. It could also be things like "how do I manipulate someone to do something". Asking an AI for a marketing campaign isn't necessarily amoral. Asking it how to best harass someone into committing self-harm is.
Most of us don't want to.
Most of those who do, don't know enough to actually do it.
Sometimes such people get into power, and they use new inventions like the then-new-pesticide Zyklon B to industrialise killing.
Last year an AI found 40k novel chemical agents, and because they're novel, the agencies that would normally stop bad actors from getting dangerous substances, would generally not notice the problem.
LLMs can read research papers and write code. A sufficiently capable LLM can recreate that chemical discovery AI.
The only reasons I'm even willing to list this chain, is that the researchers behind that chemical AI have spent most of the intervening time making those agencies aware of the situation, and I expect the agencies to be ready before a future LLM reaches the threshold for reproducing that work.
Again difficulty of production of poisons and chemicals it's not what prevent mass murdering around the globe.
Three letter agencies knowing who's buying a suspicious quantity from the list of known precursors, that stops quite a lot of the others.
AI in general reduces cost and complexity, that's kind of the point of having it. (For example, a chemistry degree is expensive in both time and money). Right now using an LLM[0] to decide what to get and how to use it is almost certainly more dangerous for the user than anyone else — but this is a moving goal, and the question there has to be "how to we delay this capability for as long as possible, and at the same time how do we prepare to defend against the capability when it does arrive?"
[0] I really hope that includes even GPT-4 before the red-teaming efforts to make it not give detailed instructions for how to cause harm
As for cost and risk -- ask GPT-5 how to minimize it. As Nathan said in his thread, it's not about this generation, it's about the next generation of models.
A key question is whether the control problem gets more difficult as the model gets stronger. GPT-4 appears to be self-aware and passing the mirror test: https://nitter.net/AISafetyMemes/status/1729206394547581168#...
I really don't know how to interpret that link, but I think there is a lot we don't understand which is going on in those billions of parameters. Understanding it fully might be just as hard as understanding the human brain.
I'm concerned that at some point in the training process, we will stumble across a subset of parameters which are both self-aware and self-interested, too. There are a lot of self-interested people in the world. It wouldn't be surprising if the AI learns to do the sort of internal computation that a self-interested person's brain does -- perhaps just to make predictions about the actions of self-interested people, at first. From there it could be a small jump to computations which are able to manipulate the model's training process in order to achieve self-preservation. (Presumably, the data which the model is trained on includes explanations of "gradient descent" and related concepts.)
This might sound far-fetched by the standard of the current model generation. But we're talking about future generations of models here, which almost by definition will exhibit more powerful intelligence and manifest it in new unexpected ways. "The model will be much more powerful, but also unable to understand itself, self-interest, or gradient descent" doesn't quite compute.
It is reducing the cost of developing an assassination plan from ~$0 to ~$0. The cost of actually executing the plan itself is not affected.
I want AI to be amoral, or rather I should say I do not want the board of OpenAI, or even the employee of OpenAI choosing what "moral" is and what is "immoral" especially given that OpenAI may be superficially "diverse" in race, gender, etc, but they sure as hell are not politically diverse, and sure has hell do not share a moral philosophy that is aligned with the vast majority of the population of humanity given the vast majority of humanity is religious in someway and I would guess the majority of OpenAI is at best agnostic if not atheist
I do not want a AI Wikipedia.... aka politically biased to only 1 worldview and only useful for basic fact regurgitation like what is the speed of light
1) be surprised by and unprepared for AGI and every step on the path to that goal, or
2) have the developers of every AI examine their work for its potential impact, both when used as intended and when misused, with regard to all the known ways even non-G AI can already go wrong: bugs, making stuff up, reward hacking, domain shift, etc.; or economically speaking how many people will be made unemployed by just a fully [G]eneral self-driving AI? What happens if this is deployed over one year? Do existing LLMs get used by SEO to systematically undermine the assumptions behind Page rank and thus web search?; and culturally: how much economic harm do artists really suffer from Diffusion models? Are harms caused by AI porn unavoidable thanks to human psychology, or artefacts of out milieu that will disappear as people become accustomed to it?
There's also a definition problem for AGI, with a lot of people using a standard I'd reserve for ASI. Also some people think an AGI would have to be conscious, I don't know why.
The best outcome is Fully Automated Luxury Communism, but assuming the best outcome is the median outcome is how actual Communism turned into gulags and secret police.
After giving us Facebook, insta, Twitter, and ego-search, influencing many people negatively, suddenly there are moral values being discussed amongst those that decide our collective tech futures.
AI will have even more influence on humankind and some are questioning the morality of money (hint: money had no morals).
I suppose maybe there are ones where they are slow-rolling because of their opinions about existential risk of AGI. But that’s not contingent on what OpenAI says or does.
Whether they stuffed up or there are some details that made the situation unworkable for the board, it’s an interesting case study in governance and the whole nonprofit with a for profit subsidiary thing.
As far as I can tell, the board had no obligation to consult the public or their shareholders or their employees on any of this.
Corporations are not a democracy. They do not “owe” the public any information that they aren’t compelled to disclose. And they certainly don’t “argue their case” to some nebulous forum comprised of either the public or employees.
When has that ever happened?
The failure to make their case is PRECISELY why the board was ultimately unable to do what the hell they wanted: remove Sam Altman. As it turned out, his presence was important enough to the employees that the usual corporate playbook of "do whatever the hell we want and only disclose what is legally required" backfired spectacularly.
If your thesis was correct, Sam would not be there right now.
You say:
> taking a poll of the public or employees
And then:
> which is the only reasonable action I can think of that looks like “arguing their case”
But that is a total non sequitur. "Taking a poll" has no relation whatsoever to "arguing their case". So I think you might not know what "arguing their case" means.
So I'll just plainly say what I think they should have done, without any jargon. I think they should have, after firing him, released a statement targeted at employees but also with a public audience in mind, something like this:
"We have simply lost faith in Mr. Altman to faithfully execute his duties as the executive of OpenAI's non-profit charter. We believe he has been acting in the interests of his own, which are not aligned with our mission. We have tried to redirect his efforts over a period of time, without success, and now are taking the only recourse that we believe is available to us to fulfill our duty to the organization. We understand that many of you will find this jarring and unsettling, but we hope you will continue to believe in the mission of OpenAI and stick with us through this trying and uncertain time, so that we can come out of it stronger and better aligned than ever."
Then anyone who quit would at least need to rationalize - to themselves, and to their social circles - why they chose not to take that to hear. Maybe for many / most / all of them, just "money" or a personal loyalty to Altman would have still won the day, but it certainly wouldn't have been as easy as it was to abandon a board that was seen as confusing and shambolic and refusing to explain itself.
That statement above is the part that would be "arguing their case". Just making the statement; the statement is the argument for that they did. Note that it doesn't include any sort of polling of anyone, or any different use of their legal rights or responsibilities. It's "just" PR, but that actually matters a lot.
> [Corporations] certainly don’t “argue their case” to some nebulous forum comprised of either the public or employees.
> When has that ever happened?
It happens all day every day. This is what PR is. Surely you're aware of the existence of PR and that it is not infrequently utilized?
I agree that it seems like an open & shut case.
Typical SEC timelines mean that this will go public in about 18 months from now.
https://pressat.co.uk/releases/ai-community-calls-for-invest... https://influencermagazine.uk/2023/11/allegations-of-quid-pr...11 hours ago, the SEC tweeted the following new rule, which could be interpreted as a declaration that if Altman and Nadella are found guilty in this case, the SEC will block certain asset sales by OpenAI until the conflict of interest is unwound / neutralized:
https://twitter.com/SECGov/status/1729895926297247815https://www.sec.gov/news/press-release/2023-240
More information:
https://stratechery.com/2023/openais-misalignment-and-micros...2023: https://www.theregister.com/2023/02/03/activision_sec_settle...
2019: https://techcentral.co.za/eoh-microsoft-ensnared-in-sec-corr...
2013-2019: https://fcpa.stanford.edu/investigation.html?id=156
2013-2016: https://www.seattletimes.com/business/microsoft/ex-microsoft...
1999: https://www.itprotoday.com/windows-8/sec-investigating-micro...
2002: https://www.computerworld.com/article/2575713/microsoft-sec-...
And I predict that even if Microsoft is going to be caught again that it will be a non-event in terms of actual repercussions. If Nadella exits MS HQ in Seattle in handcuffs I would be most surprised.
Microsoft plainly shouldn't be allowed to have this advantage, in that giving an advantage to any one party directly harms the mandate set forth in OpenAI's Charter:
https://openai.com/charterIt certainly seems that Microsoft, a "stakeholder", has managed to get a highly improper listening seat that will give them the ability to act on insider information about what's coming next in AI, allowing Microsoft to front-run the rest of the AI software industry and all those industries it affects, in a way that will plainly "compromise broad benefit". (Since any wealth that accrues excessively to Microsoft shareholders is not distributed to other humans who don't hold Microsoft shares.)
A mere 10 days ago, Nadella was shamelessly throwing his weight around on national TV, by appearing on CNBC where he improperly pressured the OpenAI non-profit board — which owes nothing to him legally or morally — to give him more deference, in direct violation of the "always diligently act to minimize conflicts of interest among our employees and stakeholders that could compromise broad benefit" provision of the OpenAI non-profit charter.
https://www.cnbc.com/2023/11/20/microsoft-ceo-nadella-says-o...
Musk and other donors are the most obvious aggrieved parties, and it is also arguably the case that every member of humanity has standing to sue for violation of the charter, because the charter explicitly declares that the primary fiduciary duty of OpenAI, Inc. (which is a non-profit) is to humanity broadly. (Therefore, every human is financially harmed by any charter violation, with such harm manifesting as a reduction in the net present value of the future benefits each human will receive from safe, broadly beneficial AGI.)
The SEC's role in fiduciary misconduct cases is not to rewrite a company's charter - that would be extremely improper and the opposite of their mandate. The SEC's mandate is to be the protectors of the status quo and of the original intent of the organizers of an entity. In this case, that means they will seek to protect the OpenAI non-profit's charter from efforts by Sam and others to erode the charter's power in violation of Sam's fiduciary duty to said charter.
The SEC's job in fiduciary misconduct situations is to remedy the situation by reversing improper governance decisions and forcing the fiduciary (Sam) to uphold their legal duty, which in Sam's case is his contractually bound duty to uphold the Charter of the non-profit entity named OpenAI, Inc. https://en.wikipedia.org/wiki/OpenAI#:~:text=the%20non%2Dpro....
OpenAI states this very clearly in multiple places on their website. For example:
"each director must perform their fiduciary duties in furtherance of its mission—safe AGI that is broadly beneficial" - https://openai.com/our-structure
If you would like to read about the types of remedies available in fiduciary duty violation cases, I recommend this resource:
Book Chapter:
https://m.winstead.com/portalresource/lookup/poid/Z1tOl9NPlu...For example (quoting from the book chapter above):
In the context of this suit, the permanent injunction or injunctions could block all of the following:This whole info dump feels like a mishmash of links to thoughtful things (Stratechery) with links to speculative articles that are clearly biased. Like how is “Influencer Magazine” breaking a story that Wall Street Journal and Kara Swisher are overlooking?
I don’t mean to be a jerk. Just really unconvinced.
I don't know if I find the Microsoft/OpenAI favor trading allegation that persuasive (unless new information is uncovered, e.g. Microsoft letting Sam use a private jet or something like that). However if the SEC actually ends up enforcing the "fiduciary duty is to humanity" thing in OpenAI's charter at some point, that would be incredibly sweet.
Absolutely. It's 100% their job.
Securities class contains stock
Stock = backed by the company balance sheet
But then he tried to do it again with a saudi contract. OpenAI board said explicitly they didn't want the partnership, and especially not tied to Altman personally being the CEO as a clause.
Altman did it behind their back -> fired.
This is the rumour on the streets, unconfirmed though
Instead it was a confusing shambles that just left them looking like idiots with no plan.
But more generally, public perception just is important to any large and broadly targeted enterprise, because "the public" approaches "our customer base" as such an enterprise scales up. Think of companies like Google or Microsoft or Meta or Amazon; essentially everyone (in the US) uses their services in some form or another, so the perception of "the public" is indistinguishable from the perception of "the people who use our services". ChatGPT isn't quite at that point yet, but it's close enough.
Or maybe they already knew the employees weren't on their side, saw no other way to do it, and hoped a sudden and dramatic ouster of the CEO would make the others fall in line? Who knows.
I'd be pretty concerned too if my CEO was doing what I considered a great job and he was suddenly removed for no clear reason. If the board had explained its rationale and provided evidence, maybe some of the employees would've listened. But they didn't... to this day we have no idea what the actual accusation was.
It looks like a failed coup from the outside, and we have no explanations from the people who tried to instigate it.
...actually, now that I think of it...
Any creative work - even a computer program - tends to be a reflection of the organizational hierarchies and people who made it. If OpenAI is a bunch of mad scientists with a thin veneer of "safety" coating, then so is ChatGPT.
The article was just days ago but it’s eluding my search.
https://news.ycombinator.com/item?id=38378216
Didnt PGs wife invest/donate to Open AI?
- He's not going to hire someone he doesn't like
- Someone that doesn't like him is unlikely to join his team
So it's very likely the whole staff ends up being people that "like" him or get along with him. He did come off as a charming smooth talking - and I'm sure he has lot of incredibly powerful friends/connections. But at least from that little window into his world I didn't feel he showed any particularly brilliance or "leadership". He did seem pretty deferential to ML experts (which i guess he's not) - but it's hard to know if it's a false humility or not
I've hired people I "don't like" on a personal level. I care more about their ability to work positively with oters, and their professionalism and skill.
Yet you and the parent poster have assumed he is hiring a cult, because he spends time evaluating?
A weird assumption to make.
I think it's just human nature to not hire people you feel you won't get along with. If you're deeply involved in all your hires, then I feel you'll end up with an organization full of people that you get along with and who you like (and probably like you back). I wouldn't go so far as to say it'd make a personality cult - though with their lofty mission statements and ambitions to make the world better.. who knows. Not going to psychoanalyze a bunch of people I don't know
"I've hired people I "don't like" on a personal level."
I'm honestly impressed... I feel that's rather exceptional. I feel a lot of hiring goes on "gut feeling"
While I'm sure he's competent, he's probably not in a position to quiz them on the latest research
1. For that matter, I would not work with a boss who doesn't recognize an argument from authority as a response.
2. Furthermore a boss who lacks critical thinking skills and can't recognize a mildly skeptical comment for what it is.
3. Why assume I'm rigidly taking any position re. Altman in particular?
4. "That's pathetic" is not "He's pathetic", so perhaps the problem here is your low reading comprehension level, and not any particular assumption that I have committed to at all.
5. "A weird assumption" -- so, you decide it's weird because you just read it wrongly? Or, if something seems weird, why not ask a curious question and find out? Listening skills? Why is it so important to make this about another commenter?
I find your poor faith interpretation of my comment to be offensive. You've at the used your professionalism as pretense to biasedly cast judgment on what is a critical or even mildly skeptical general remark. This reflects the bad side of tech culture. You should apologize.
Unfortunately, it can sometimes be hard to tell those two apart when a sociopath is actively pushing your emotional buttons to make you feel like they care about the same things as you etc.
What if lots of employees stood to make "fuck you" money from that sale, and with Sam's departure, that money was in danger of evaporating?
I always find it funny when people understand democracy to mean "other people that should vote my way, otherwise they are imoral and should be re-educated".
* Expressing skepticism about unions in Sweden -- https://news.ycombinator.com/item?id=38308184
* Arguing against central planning, with a link to a book detailing how socialism always fails -- https://news.ycombinator.com/item?id=38303195
* I often push back against the "greedy corrupt American plutocrats" narrative which you see all over HN. Here are a few examples -- https://news.ycombinator.com/item?id=37541805 https://news.ycombinator.com/item?id=37962796 https://news.ycombinator.com/item?id=38456106
And by the way, here is a comment I made the other day making essentially the point you are making, that in a democracy everyone is entitled to their opinion, even the dastardly Elon Musk: https://news.ycombinator.com/item?id=38261265 And I also argue in favor of freedom of speech here, for whatever that's worth: https://news.ycombinator.com/item?id=37713086
Point being, I'm not sure our disagreement lies where you think it does.
The purpose of the board is that they're supposed to be disinterested representatives of humanity, supervising OpenAI. The employees aren't chosen for being disinterested, and it seems quite likely that they are, in fact, interested, per my link.
From the perspective of human benefit (or from the perspective of my own financial stake in OpenAI, given that their charter says their "primary fiduciary duty is to humanity"), I prefer a small group of thoughtful, disinterested people over a slightly larger group whose interest is systematically biased relative to the interest of me or the average person. Which is more likely to produce a fair trial: a jury of 12 randomly chosen citizens, or a jury of 1200 mafiosos?
"primary fiduciary duty is to humanity"
I don't think that means it intends to pay a financial dividend to each and every person on the planet. I think it means that if it is successful at AGI, that in itself will expand the economy enough to have the same effect.
"Rising tide lifts all boats" type logic.
I think it could mean this, in the context of Altman's other project (WorldCoin) which despite all its controversy is ostensibly intended as a vehicle for distributing an AGI-funded Universal Basic Income (UBI) to all of humanity.
https://worldcoin.org/cofounder-letterIt tells a lot that Altman made openAI so dependend on him that his ousting could have killed the company. That's also contributing to the fact that the board was not trusting him
"The Prince", Machiavelli
They had a couple of people on the board who had no right being there. Sam wanted them gone and they struck first by somehow getting Ilya on their side. They smeared Sam in hopes that he would slink away, but he had build so much goodwill with his employees that they wouldn't let it stand.
They probably had smeared people before and it had worked. I'm thrilled it didn't work for them this time and they got ousted.
But an oversight board can be established easier than that it can be disbanded and that's for very good reasons. The only reason that it worked is not because the board made any decisions they shouldn't have made (though that may well be the case) but because they critically misjudged the balance of power. They could and maybe should have made their move, but they could not make it stick.
As for the last line of your comment: I think that explains your motivation of interpreting things creatively but that doesn't make it true.
https://nitter.net/hlntnr/status/1730034022737125782#m
Here's some interesting background which is suggestive of what's going on: https://nitter.net/labenz/status/1727327424244023482#m
> Sam directing IT and Operations staff to conduct investigations into employees, including Ilya, without the knowledge or consent of management.
> Sam's discreet, yet routine exploitation of OpenAI's non-profit resources to advance his personal goals, particularly motivated by his grudge against Elon following their falling out.
> Brad Lightcap's unfulfilled promise to make public the documents detailing OpenAI's capped-profit structure and the profit cap for each investor.
> Sam's incongruent promises to research projects for compute quotas, causing internal distrust and infighting.
what will happen with their VC-backed valuations without a VC-oriented CEO
You don't make AI in a garage with a home PC and beer. You need gigantic amount of data, compute, and talented people to organize all this, which are exceptionally rare, and also they need to eat and feed their families too.
If you're happy to give up total control of tech to one corporation, while the CEO of the company, using the influence of the OpenAI, attracts resources for his personal hardware start-up[1], — our views on how a not-for-profit company should be run are starkly different.
[1] https://www.ft.com/content/4c64ffc1-f57b-4e22-a4a5-f9f90a741...
The problem is you have no idea what you're talking about. This is not about how important something is, but what resources it requires as a BASIS MINIMUM for it to happen at all. Linux is important, but it's not nearly as expensive to create or maintain. If it's so easy for a scrappy non-profit to attract billions to tinker with AI, where I say are those startups? Where? Q.E.D. The most open thing we have, Llama, came from giant Meta.
Regarding the hardware startup, neither me nor anyone else is defending Altman about that, nor it has any relation to Microsoft. If you want to argue the subject of this thread, you're welcome. If you just want to copy paste generic talking points with no relevance, I don't care.
Sam Altman was happy to join the venture and lead OpenAI knowing that it was the non-profit AI research company from the beginning. As a CEO, he was supposed to lead the company as it was created. As a result of his management, the company first ceased to be open, then in fact non-profit, then safe (according to it's leading scientists), then the number of published research whitepapers fell sharply, then has taken steps to become a gatekeeper for government regulations, and finally, the communication between Sam and the board (which really should be running the company) was lost. If a CEO was unable to manage a company, including attracting investment and resources, without violating the fundamental principles of that company, that CEO should have been dismissed — what has been effectively attempted by the board. Otherwise, it's a power grab and a de facto reformatting of the company into a different entity. Do you comprehend that?
I have no idea what compiling Linux kernel, or running GPT-4 locally, does anything in common in the context of large non-profit organization management. My argument is that the company should have been run in such a way that one corporation could not seize complete control. Never, under no circumstances. In spite of this, the CEO made all the effort to make it so. As a result, we have now in fact a for-profit company with good product, tons of money and GPUs, that belongs to the Microsoft. Was it worth it? Does this justify it for you? If you are not a $MSFT shareholder the answer should be obvious.
> If it's so easy for a scrappy non-profit to attract billions to tinker with AI, where I say are those startups? Where? Q.E.D. The most open thing we have, Llama, came from giant Meta.
Take a good look at HuggingFace Chatbot Arena leaderboard page[1]. Pay attention to the values of different models on different benchmarks. GPT4 is the leader, but not by an unattainable margin. Anthropic, which has spinned off from the very same OpenAI, rapidly catches up, along with other solutions. Even free, open (for commercial usage) models, while being self-hostable, are not an orders of magnitude worse. They are well in range of being practical and useful, including for commercial products. There is no moat[2].
> Linux is important, but it's not nearly as expensive to create or maintain.
> The problem is you have no idea what you're talking about.
ROFL, okay buddy.
[1] https://huggingface.co/spaces/lmsys/chatbot-arena-leaderboar...
[2] https://www.semianalysis.com/p/google-we-have-no-moat-and-ne...
What's hard to comprehend here?
As for the models topping your Hugging Face chart, they're literally all funded (by the billion) by private enterprises and the US govt. It's baffling to me how this is supposed to be an argument in your favor. Even the "not by an unattainable margin" comment makes no sense, unless you think getting from 40% to 60% is as easy as getting from 60% to 80%, and then getting from 80% to 100%. Which you clearly seem to think.
Anthropic's models are closed and funded by the money of Sam Bankman-Fried, of all people, and more recently Amazon. They're trying to raise billions more from Google and others. That's your idea of being open and not needing corporate investment for some reason? LOL, oh my god.
The "free" model you're citing is MADE BY META, a company worth almost a trillion dollars, and which has the largest social media presence in the world, from which to mine data.
You have absolutely not even the faintest idea what you're talking about.
The OpenAI company, that was supposed to be non-profit open research think lab, has been taking investments of dozens of millions from different funds, while producing highest quality research material for the public good. Up to the point when the CEO sold it for Microsoft's $1B deal that solved their infra & GPU resource problems for… completely locking the company as an unofficial division of Microsoft Research. This deal, essentially, entirely distorted the raison d'être of the non-profit company, closed the "open research" part, and made it a for-profit extension for MS/Bing services.
Here, perspectives fork in two ways: either you believe it's a great development that you can benefit of by using closed paid OpenAI/Microsoft commercial services, OR you believe that metamorphosis of the leading open AI research shop into it's exact antipode is embodiment of disappointment. From your boorish, superficial remarks, I can see which side you are on.
I don't care if Sam Altman took money from Microsoft, or Sam Bankman-Fried (before he was charged), or US govt — that entirely misses the point. I do care that Sam Altman orchestrated a complete relegation of the research lab to an overpowered for-profit "Clippy" assistant for a single huge corporation, depriving us from the open research. He had to work on attracting investments from several corporations and other entities to avoid centralisation of control. But we see that he acted according to his own personal agenda, not in the interests of the company's charter. In that perspective, I don't give a damn if he could have raised a billion or not as a non-profit company. I'd rather it was $300-500M that would have kept the company open than a $1B and what we have today. I'd rather the company didn't survive this crisis than what we have today.
I gave you the benchmarking of other LLM models that are several months (up to a year) behind GPT4 in performance development. I don't care if they are founded by megacorps or not, if they do what non-profit "Open" AI no more does, i.e. publishing open research and supplying us open models. Comprehend? Your argument of "if it weren't for Microsoft OpenAI wouldn't exist" is irrelevant, if OAI no longer produces open research that it supposed to; it better wouldn't otherwise. Let others for-profit multi-billion corps take over, if we can't have better.
To further refute your $MSFT shill demagoguery, narrowly focused on compute accelerators, I'll add that Abu Dhabi's TTI (UAE gov money for academia) has produced and released open model Falcon 180B, while French nonprofit research lab Kyutai has just raised €330M. In Europe, for a minute, not even SV! Apparently, funding open research is possible? Q.E.D. I'd rather Ilya and Andrej move to this lab than peddling "Laundry buddy" GPT Bing API tokens BS disgrace, but it's up to them.
And you can stay in your delusions where only OpenAI+Microsoft is capable of leading AI research, lead by an ego inflated CEO. Don't forget to scan your eye pupils for $worldcoin.
OpenAI, the original non-profit, DIED. Elon Musk pulled the funding, it DIED. It's DEAD, get it? It does not exist without money. I don't know how to get it through your head. There's no such thing as a serious AI company without billions of funding. Period.
So. Given that, to keep the team together, Sam made the for-profit subsidiary and hosted it in the hollowed out non-profit, Sam went out to get investors and Microsoft chimed in the most.
While the new hybrid OpenAI tries to respect the spirit of the original OpenAI... WHICH IS DEAD (GET IT?)... it has to also balance between non-profit mission, and its investors, thanks to which it exists at all.
That's kind of like maybe you think you're a world-leading poet and novel writer, but your boss doesn't care, he wants those toilets clean. That's what he's paying for. You can then use that money to spend your free time writing poems.
Not sure I have to respect the rest of your BS with an answer. "Oh the other megacorp LLMs are just few months behind GPT4". Newsflash! OpenAI also exists within... time, genius. Time is passing there, too. And GPT-5 is ready and few Fortune 100 companies are testing it currently. They're not sitting still and waiting for the others to catch up.
I never said other companies can't compete, in general, but they're no better than OpenAI. Amazon, Google, Inflection, Facebook are no better than OpenAI. They're not. And I never mentioned anything about Worldcoin, you're basically desperate to strawman this, but you are exceptionally bad at it.
Did I say you don't have the slightest, faintest, most distant idea what you're talking about? Anyway this is my last message.
He would have probably left the Disney board by now. Or Apple might have invested in DIS.
Why do you think he is not replaceable?
https://twitter.com/ilyasut/status/1727434066411286557
>Either he is here to scare Musk, or - more likely IMO - to act as a Musk lightning rod. Musk takeover part 2?
It’s just comparable to a player competing against a team with a teammate who they didn’t like. There’s definitely some added drama.
> Jakub, Szymon, and Aleksander are exceptional talents and I’m so happy they have rejoined to move us and our research forward.
that Sam Altman.
This Reddit post from 8-years ago, reads eerily similar to what happened at OpenAI … and Sam is the common denominator at both.