Surely these couldn’t be the exact incentives that the founders (from a distance) predicted would exert themselves on the project as its economic value increased, right?
Is everyone now believing that AGI is within reach? This scrambling to have a non profit based structure is odd to me. They clearly want to be a for profit company, is this the threat of Elon talking?
If AGI were in reach, why would something so human as money matter to these people? The choice to transition to a more pocket-lining structure is surely a vote of no-confidence in reaching AGI anytime soon.
The only proof is in benchmarks and carefully selected demos. What we have is enough AI to do some interesting things, and that's good enough for now. AGI is a fuzzy goal that keeps the AI companies working at an incredible pace.
Open AI has build tools internally that scale not quite infinitely but close enough and they seem to have reached above human performance on all tasks - at the cost of being more expensive than hiring a few thousand humans to do it.
I did work around this last year and there was no limit to how smart you could get a swarm of agents using different base models at the bottom end. This at the time was a completely open question. It's still the case that no one has build an interactive system that _really_ scales - even the startups and off the record conversations I've had with people in these companies say that they are still using python across a single data center.
AGI is now no longer a dream but a question of if we want to:
1). Start building nuclear power plants like it's 1950 and keep going like it's Fallout.
2). Wait and hope that Moore's law keeps applying to GPUs until the cost of something like o3 drops to something affordable, in both dollar terms and watts.
We don't have AGI until there's code you can plug into a robot and then trust it to watch your kids for you. (This isn't an arbitrary bar, childcare is a huge percentage of labor hours.)
> Start building nuclear power plants like it's 1950 and keep going like it's Fallout
Nuclear has a (much) higher levelized cost of energy than solar and wind (even if you include a few hours of battery storage) in many or most parts of the world.
Nuclear has been stagnant for ~two decades. The world has about the same installed nuclear capacity in 2024 as it had in 2004. Not in percent (i.e. “market share”) but in absolute numbers.
If you want energy generation cheap and fast, invest in renewables.
And yet when data enters need power all day every day nuclear is the only solution. Even Bill Gates stop selling solar when it wasn't for the poors who probably don't need hot water every day anyway.
Most people do not buy from specific sources of production. They buy “from the grid”, the constituents of which are a dynamic mix of production sources (solar, wind, hydro, nuclear and fossile where I live).
Wind is strong at night when solar produces nothing. Same in the winter months.
As I said: if the power consumer is grid connected, this does not matter. Example: I have power in the socket even at night time :)
As long as you have uninterrupted power (i.e. as long as connected to the grid), the important metric is mean cost of energy, not capacity factor of the production plant.
For a nuclear sub or a space ship, which is not grid connected, capacity factor is very important. But data centers are usually grid connected.
> when data enters need power all day every day nuclear is the only solution
Do you think data centers running at night are running exclusively on nuclear-generated power :)?
We already have lots of data centers that need power all day every day. Most are just grid connected. It works.
The more I look the more I think it's ever so more out of reach and if there's a chance at it, OpenAI doesn't seem to be the one that will deliver it.
To extrapolate, (of LLMs and GenAI) the more I see use of and how it's used the more it shows severe upwards limits, even though the introduction of those tools has been phenomenal.
On business side, OpenAI lost key personnel and seemingly the plot as well.
I think we've all been drinking a bit too much on the hype of it all. It'll al;l settle down into wonderful set of (new) tools, but not on AGI. Few more (AI) winters down the road, maybe..
Every letter of "AGI" means different things to different people, and the thing as a whole sometimes means things not found in any of the letters.
We had what I, personally, would count as a "general-purpose AI" already with the original release of ChatGPT… but that made me realise that "generality" is a continuum not a boolean, as it definitely became more general-purpose with multiple modalities, sound and vision not just text, being added. And it's still not "done" yet: while it's more general across academic fields than any human, there's still plenty that most humans can do easily that these models can't — and not just counting letters, until recently they also couldn't (control a hand to) tie shoelaces*.
There's also the question of "what even is intelligence?", where for some questions it just matters what the capabilities are, and for other questions it matters how well it can learn from limited examples: where you have lots of examples, ChatGPT-type models can be economically transformative**; where you don't, the same models *really suck*.
(I've also seen loads of arguments about how much "artificial" counts, but this is more about if the origin of the training data makes them fundamentally unethical for copyright reasons).
** the original OpenAI definition of AGI: "by which we mean highly autonomous systems that outperform humans at most economically valuable work" — found on https://openai.com/charter/ at time of writing
I believe e.g. Ilya Sutskever believed AGI is in reach at the founding, and was in earnest about the reasons for the nonprofit. AFAICT the founders who still think that way all left.
It's not that the remainder want nonprofit ownership, it's that they can't legally just jettison it, they need a story how altering the deal is good actually.
"Why we've decided to activate our stock maximizing AIs despite it buying nothing but paperclip manufacturing companies because reaching AGI is in humanitys best interest no matter the costs"
Maybe I’m missing it in this article or elsewhere on the website, but how exactly is OpenAI’s vision of making AGI going to “benefit humanity as as a whole”?
I’m not asking to be snarky or imply any hidden meaning…I just don’t see how they plan on getting from A to B.
From this recent press release the answer seems to be: make ChatGPT really good and offer it for free to people to use. Which is a reasonable answer, I suppose, but not exactly one that matches the highfalutin language being used around AGI.
> Investors want to back us but, at this scale of capital, need conventional equity and less structural bespokeness
Well you see the AGI can only benefit humanity if it is funded via traditional equity... Investors WANT to give their money but the horrendous lack of ownership is defeating their goodwill
The seasoned advice from experts is that AGI could very easily END humanity as a whole. But they need to pay their mortgage right now, and uhh... if we don't do it our competitors will.
We're basically betting our species' future on these guys failing, because for a short period of time there's a massive amount of shareholder value to be made.
Isn't the idea that AGI could replace a bunch of labor allowing us to help more poor and increase net positive vibes or just have more leisure time?
Obviously the way our economy and society are structured that's not what will happen, but I don't think that has much to do with tools and their tendency to increase our efficiency and output.
Put another way, there are powerful benefits from AGI that we will squander because our system sucks. That is not a critique against AGI, that is a critique of our system and will continue to show up. It's already a huge point of conversation in our collective dialogue.
That is the idea being marketed by the companies, yes.
But AI does not come with glands, with brainstem instincts, with a limited physical strength, with childhood morality tales, with the ability to feel pain or empathy, with the desire for respect and public social approval, with the feeling of emotional connection with friends and family, with the feeling of helplessness and desire for freedom & variety, with the feeling of boredom and dislike of menial work, with the idealism of various religious and ethical mores.
It does not come packaged with any of the moral values or psychological quirks that constrain our behavior and allow us to work together. It is in fact, very difficult to teach these things to a machine - far more difficult than less subjective ideas like "Reason a way to do the thing that does the other thing". We do not have a mathematical proof of Correct Human Ethics, and we have reason to believe that this is a Difficult Problem.
It's arguably much easier to build a fully general AI that serves as what's termed a "paperclipper", which eats the world in pursuit of its task because we didn't formulate a reason for it not to do so and express it in the correct ten gigabyte matrix of floating point values, than to build a fully general AI which pursues what we actually implicitly want it to do for us in a way that the median eight year old would be able to follow.
The people studying AI alignment specifically were pessimistic fifteen years ago, and today the dominant stance is "Dying with dignity", give up the struggle to warn everybody and just try to enjoy what time you have.
Once OpenAI becomes fabulously rich, the world will surely be transformed, and then the benefits of all this concentration of power will simply trickle down.
The path between A and B has enough tangled branches that I'm reminded of childhood maze puzzles where you have to find which entrance even gets to the right goal and not the bad outcomes.
The most positive take is: they want to build a general-purpose AI, to allow fully automated luxury for all; to built with care to ensure it can only be used for positive human flourishing and cannot (easily or at all) be used for nefarious purposes by someone who wants to sow chaos or take over the world; and to do so in public so that the rest of us can prepare for it rather than wake up one day to a world that is alien to us.
Given the mental image I have here is of a maze, you may well guess that I don't expect this to go smoothly — I think the origin in Silicon Valley and startup culture means OpenAI, quite naturally, has a bias towards optimism and to the idea that economic growth and tech is a good thing by default. I think all of this is only really tempered by the memetic popularity of Eliezer Yudkowsky, and the extent to which his fears are taken seriously, and his fears are focussed more on existential threat of an optimising agent that does the optimising faster than we do, not on any of the transitional dynamics going from the current economy to whatever a "humans need not apply" economy looks like.
Covid does not hate you, nor does it love you, it simply follows an optimisation algorithm — that of genetic evolution — for the maximum reproductive success, and does so without regard to the damage it causes your body while it consumes you for parts.
Covid is pretty stupid, it's just a virus.
And yet: I've heard the mutation rate is 3.8 × 10e−6 / nucleotide / cycle, and at about 30,000 base pairs and 10e9 to 1e11 virons in an infected person, so that's ~1e8-1e10 mutations per reproductive cycle in an infected person, and that the replication cycle duration is about 10 hours. Such mutations are both how it got to harm us in the first place, why vaccination isn't once-and-done, and this logic also applies to all the other diseases in the world (including bacterial ones, which is why people are worried about bacterial resistance).
As an existence proof, Covid shows how an agent going off and doing its own thing, if it does it well enough, doesn't even need to be smart to kill a percentage point or so of the human species *by accident*.
The hope is that AI will be smart enough that we can tell it: humans (and the things we value) are not an allowed source of parts. The danger happens well before it's that smart… and that even when it is that smart, we may well not be smart enough to describe all the things we value, accurately, and without bugs/loopholes in our descriptions.
I understand it how it applies to a virus, because a virus has a mechanism for action. In fact a virus is all mechanism for action; its behaviour is emergent from just being the surviving behaviour, rather than any grand strategizing.
But for AI, they are all brains (or that's the claim) but no brawn. How do they act?
Some of those viruses shut down hospitals, some damage uranium enrichment facilities, some mine or steal bitcoin, some steal normal money.
Polymorphic viruses have been around for decades, so it's not inconceivable that there's already some almost entirely evolved (simulated evolution is a kind of AI training mechanism) virus on the internet already, even before LLMs.
Right now, I'd expect an LLM to be fairly mediocre at writing a virus. But it doesn't need to be an LLM that does the writing, just as it doesn't need to be an LLM that is the payload once the infection has happened.
I’m pretty sure the “for profit cap” for Microsoft is something like a trillion dollars in return. 100x return cap at $10 billion invested. It basically prevents Microsoft from becoming a world super power with a military and nuclear weapons but not much else, especially considering they will reinvest a lot of their money for even more returns over time
Their charter defines AGI as "highly autonomous systems that outperform humans at most economically valuable work", and their intent is to ""directly build safe and beneficial AGI" or "[aid] others to achieve this".
They don't address benefits beyond the wide-scale automation of economically-valuable work, and as those benefits require significant revisions to social structures it's probably appropriate that they keep their mouth shut on the subject.
I'm woefully uneducated but I think it's a red herring. It does not matter what their vision of AGI is if they are just going to be peddling LLMs as a service to customers.
Does anybody else here also think openai lost it? This year was all about drama and no real breakthrough while competitors caught up without the drama.
Whatever other zaniness is going on with Musk/Sam/etc, I can't escape the feeling that if I had donated a lot of money to a non-profit, and then a few years later that non-profit said "SURPRISE, WE'RE NOW FOR PROFIT AND MAKING INVESTORS RICH but you're not an investor, you're a donor, so thank-you-and-goodbye"... ya, I'd feel miffed too.
If we're a for-profit company with investors and returns etc... then those initial donations seem far closer to seed capital than a not-for-profit gift. Of course hindsight is 20/20, and I can believe that this wasn't always some devious plan but rather the natural evolution of the company... but still seems inequitable.
As much as Elon's various antics might deserve criticism (especially post-election) he seems to be in the right here? Or am I missing something?
I believe they offered Elon shares in return for the initial donation, and he turned them down because he didn't want a few billion worth of OpenAI, he wanted total executive control.
But we're all kind of arguing over which demon is poking the other the hardest with their pitchfork, here.
“Our plan is to transform our existing for-profit into a Delaware Public Benefit Corporation (PBC) with ordinary shares of stock…The non-profit’s significant interest in the existing for-profit would take the form of shares in the PBC at a fair valuation determined by independent financial advisors.”
The details here matter and is bs. What should take place is this, OpenAI creates a new for profit entity (PBC, or whatever structure). That company sets an auction for 5-10% of the shares. This yields the valuation. The new company acquires the old company with equity, using the last valuation. So say $30b is raised for 10%, means $300B.
So the $160B becomes like 53% and then 63% with the 10% offer. So the non-profit keeps 37% plus whatever it owns of the current for profit entity.
Auction means the price is fair and arms-length, not trust me bro advisors that rug pull valuations.
I believe on this path, Elon Musk has a strong claim to get a significant portion of the equity owned by the non-profit, given his sizable investment when a contemporaneous valuation of the company would have been small.
> Elon Musk strong claim to get a significant portion of the equity owned by the non-profit, given his sizable investment when a contemporaneous valuation of the company would have been small
Sorry, what's the path to this? Musk's 'investment' was in the form of a donation, which means he legally has no more claim to the value of the nonprofit than anyone else.
So there's really no legal recourse if a 501c3 markets itself as a research institute / OSS developer, collects $130MM, and then uses those donations to seed a venture-backed company with closed IP in which the donors get no equity? One that even competes with some of the donors?
There is recourse, in that a 501c3 is limited in what it can do with it's assets: it must use them to advance its charitable purpose. In this case the OpenAI board will attempt to make the case that this approach, with partial ownership of a public benefit company, is what's best for their mission.
If donors got equity in the PBC or the owned PBC avoided competing with donor owned companies this would not be consistent with the non-profit's mission, and would not be compliant with 501c3 restrictions.
Right, I pointed out the competition to show that the donors are suffering actual damages from the restructuring. I don't think any of the critics here seriously expect the PBC model to fulfill the nonprofit's stated mission in any case.
This is not just an issue for the change that OpenAI is contemplating right now, but also the capped for-profit change that happened years ago. If that's found to be improper, I'm curious if that entitles the donors to any kind of compensation.
> A historic precedent for this is when Blue Cross' at the time nonprofit health insurers converted into for-profit enterprises. California Blue Cross converted into what's now Anthem. They tried to put a small amount of money into a nonprofit purpose. The California Attorney General intervened and they ultimately paid out about $3 billion into ongoing significant health charitable foundations in California. That's a good model for what might happen here.
So my guess is there's no compensation for any of the donors, but OpenAI may in the end be forced to give some money to an open, nonprofit AI research lab (do these exist?). IANAL so that's a low-confidence guess.
Still, that makes me so queasy. I would never donate to a YC-backed nonprofit if this is how it can go, and I say that as a YC alum.
The argument was because they’ve converted the nonprofit to a for profit, which enriches the employees and investors and doesn’t serve the nonprofit or its mission, the nonprofit was only so in disguise and should be viewed as having been a for profit all along. So the donation should be viewed as an investment.
> A highly autonomous system that outperforms humans at most economically valuable work.
That is not the real definition of AGI. The real "definition" can mean anything at this point and in their leaked report from the Information [0] they have defined it as "returning $100 billion or so in profits"
In other words raise more money until they reach AGI. This non-profit conversion to for-profit is looking like a complete scam from the original mission [1] when they started out:
> Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return. Since our research is free from financial obligations, we can better focus on a positive human impact.
"AGI" at this point is a meaningless term abused to fleece investors to raise billions for the displacement of jobs either way which that will "benefit humanity" with no replacement or alternative for those lost jobs.
Yeah IMO they should be required to dissolve the company and re-form it as for-profit. And pay any back-taxes their non-profit status exempted them from in the meantime.
> Investors want to back us but, at this scale of capital, need conventional equity and less structural bespokeness.
This seems exactly backwards. At this scale you can establish as much "bespokeness" as you want. The investors want in and will sign pretty much anything.
It reminds me of the old joke from Paul Getty:
If you owe the bank $100 that's your problem. If you owe the bank $100 million, that's the bank's problem.
Can they quit this "non profit" larp already? Is this some recruiting tactic to attract idealistic engineers or a plan to evade taxes, or both? Sam Altman was offering crypto-alms to third world people in exchange for scanning their eyeballs. There is no altruism here.
Our current structure does not allow the Board to directly consider the interests of those who would finance the mission and does not enable the non-profit to easily do more than control the for-profit.
I kind of thought that was the point of the current structure.
They referring to the reconstructed board that solely exists to rubber stamp every Altman decision as officially great for humanity, so what do they care what the board's considerations are? They'll go along with literally anything.
>I kind of thought that was the point of the current structure.
Yes, it is but if we only stopped the analysis right there, we could take pleasure in the fact that Sam Altman checkmated himself in his own blog post. "Dude, the non-profit is _supposed_ to control the profit company because that's how you formed the companies in the first place! Duh!!!"
To go beyond that analysis, we have to at least entertain (not "agree" but just _entertain_ for analysis) ... what Sam is saying:
- the original non-profit and profit structure was a mistake that was based on what they thought they knew at the time. (They thought they could be a "research" firm.)
- having a non-profit control the profit becomes a moot point if the for-profit company becomes irrelevant in the marketplace.
Here is a key paragraph:
>The hundreds of billions of dollars that major companies are now investing into AI development show what it will really take for OpenAI to continue pursuing the mission. We once again need to raise more capital than we’d imagined. Investors want to back us but, at this scale of capital, need conventional equity and less structural bespokeness.
In other words, let's suppose the old Friendster social network was structured as "non-profit-board-controlling-a-for-profit-Friendster" like OpenAI. The ideals of the "non-profit being in control" is a moot point when a competitor like Facebook makes non-profit-Friendster irrelevant.
Or put another way, pick any hard problem today (self-driving, energy discovery, etc) that requires billions of investment and hypothetically create a new company to compete in that space. Would creating that company as a non-profit-controlling-the-profit-company confer any market advantage or would it be a handicap? It looks like it's a handicap.
OpenAI is finding it hard to compete for investors' billions in the face of Tesla's billions spent on 50000 GPU supercluster, Google billions spent on Gemini, Anthropic billions spent on Claude, Alibaba's billions, etc. OpenAI doesn't have an unassailable lead with ChatGPT.
The issue is Sam Altman & OpenAI look bad because he and the investors want to use the existing "OpenAI" brand name in a restructured simpler for-profit company. But to outsiders, it looks like a scam or bait-and-switch. Maybe they could have done an alternative creative procedure such as spin up a totally new for-profit company called ClosedAI to take OpenAI's employees and then pay a perpetual ip license to OpenAI. That way, CloseAI is free of encumbrances from OpenAI's messy structure. But then again, Elon Musk would still probably file another lawsuit calling those license transfers as "bad faith business dealings".
I could see an argument for your example that putting the non profit in control of a social media company could help the long term financial success of the platform because you’re not chasing short term revenue that annoys users (more intrusive ads, hacking engagement with a stream of shallow click bait, etc). So it’d be a question of whether you could survive long enough to outlast competitors getting a big short term boost.
I’m not sure what the equivalent would be for llm products.
the original non-profit and profit structure was a mistake that was based on what they thought they knew at the time. (They thought they could be a "research" firm.)
There’s no slavery here. If Sam decided it was a mistake to dedicate his time to a non-profit, he’s perfectly free to quit and start an entirely new organization that comports with his current vision. That would be the honorable thing to do.
>If Sam decided it was a mistake to dedicate his time to a non-profit, he’s perfectly free to quit [...]
To be clear, I don't think Sam can do anything and come out looking "good" from a public relations standpoint.
That said, Sam probably thinks he's justified because he was one of the original founders and co-chair of OpenAI -- so he feels he should have a say in pivoting it to something else. He said he got all the other donors on board ... except for Elon.
That leaves us with the messy situation today... Elon is the one filing the lawsuit and Sam is writing a PR blog post that's received as corporate doublespeak.
he was one of the original founders and co-chair of OpenAI -- so he feels he should have a say in pivoting it to something else
No. Non-profit is a deal between those founders and society. That he was an original founder is irrelevant. I don’t care about Elon, it’s the pivoting that’s inherently dishonorable.
> Non-profit is a deal between those founders and society.
Yes I get that but did OpenAI ever take any public donations from society? I don't think they did. It thought it was only funded by wealthy private donors.
>, it’s the pivoting that’s inherently dishonorable.
Would creating a new company (i.e. ClosedAI) that recruits OpenAI's employees and buys the intellectual property such that it leaves a "shell" of OpenAI be acceptable?
That's basically the roundabout thing Sam is trying to do now with a re-incorporated for-profit PBC that's not beholden to the 2015 non-profit organization ... except he's also trying to keep the strong branding of the existing "OpenAI" name instead of "ClosedAI".
The existing laws allow for non-profit 501c3 organizations to "convert" (scare quotes) to for-profit status by re-incorporating to a (new) for-profit company. That seems to be Sam's legal roadmap.
EDIT REPLY: They received benefits by dint of their status. If there were no such benefits
The main benefit is tax exemption but OpenAI never had profits to be taxed. Also to clarify, there's already a for-profit OpenAI Global LLC. That's the subsidiary company Microsoft invested in. It has the convoluted "capped profit" structure. Sam says he can't attract enough investors to that for-profit entity. Therefore, he wants to create another for-profit OpenAI company that doesn't have the convoluted ("bespoke" as he put it) self-imposed rules to be more attractive to new investors.
The 2 entities of non-profit and for-profit is like Mozilla Foundation + Mozilla Corporation.
Yes I get that but did OpenAI ever take any public donations from society? I don't think they did. It thought it was only funded by wealthy private donors.
They received benefits by dint of their status. If there were no such benefits they wouldn’t have incorporated that way.
In any case, would creating a new company (i.e. ClosedAI) that recruits OpenAI's employees and buys the intellectual property such that it leaves a "shell" of OpenAI be acceptable?
There’s no problem with recruiting employees. The intellectual property purchase is problematic. If it’s for sale, it should be for sale to anyone and no one connected to a bidder should be involved in evaluating offers.
The existing laws allow for non-profit 501c3 organizations to "convert" (scare quotes) to for-profit status by re-incorporating to a (new) for-profit company. That seems to be Sam's legal roadmap.
All I care about, (and I guess I don't care than much because I'm not a US citizen) is, has this move allowed them, or their contributors to pay less tax. That is their only obligation to the public.
Was the non profit a way to just avoid tax until the time came to start making money?
> But to outsiders, it looks like a scam or bait-and-switch.
It doesn't look like one - it is one.
> We’re excited to welcome the following new donors to OpenAI: Jed McCaleb (opens in a new window), Gabe Newell (opens in a new window), Michael Seibel (opens in a new window), Jaan Tallinn (opens in a new window), and Ashton Eaton (opens in a new window) and Brianne Theisen-Eaton (opens in a new window). Reid Hoffman (opens in a new window) is significantly increasing his contribution. Pieter Abbeel (opens in a new window) (having completed his sabbatical with us), Julia Galef (opens in a new window), and Maran Nelson (opens in a new window) are becoming advisors to OpenAI.
>It doesn't look like one - it is one. > We’re excited to welcome the following new donors to OpenAI: ...
The story is that OpenAI worked out some equity conversion of the for-profit co for the donors to the non-profit. Elon Musk was the notable holdout. Elon was offered some unknown percentage for his ~$40 million donation but he refused.
Seems like the donors are ok with OpenAI pivot to for-profit ... except for Elon. So no bait-n-switch as seen from "insiders" perspective.
If you have information that contradicts that, please add to the thread.
These people "donated to a non-profit". They did not "invest in a for-profit".
If the Red Cross suddenly turns into a for-profit and then says "well we'll give our donators of the past few years equity in our new company", this does not make it any less of a scam.
> Seems like the donors are ok with OpenAI pivot to for-profit
If you have information that shows this, feel free to add it. "Not suing" is not the same as that. Very few people sue even when they feel they're scammed.
>These people "donated to a non-profit". They did not "invest in a for-profit".
Sure, nobody wants to be tricked into donating to a charity and then have their money disappear into a for-profit company.
Based on the interviews I saw from some donors (Reid Hoffman, etc), there's more nuance to it than that. The donors also wanted an effective non-profit entity. The TLDR is that they donated to a non-profit under 2015 assumptions of AGI research costs that turned out to be wrong and massively underestimated.
- 2015... the non-profit OpenAI in its original idea of "charity research organization" was flawed from the beginning because they realized they couldn't attract A.I. talent at the same level as Google/Facebook/etc as those competitors offered higher salaries and lucrative stock options. Then, they realized the initial ~$100 million in donations was also not enough to pay for very expensive hardware like GPUs and datacenters. It's hard for researchers to make discoveries in AGI if there's no cutting edge hardware for them to work on. A non-profit tech charity getting billions in donations was not realistic. These money problems compound and lead to...
- 2019... create the for-profit OpenAI Global LLC as a vehicle for the Microsoft investment of $1 billion and also create stock incentives for recruiting employees. This helps solve the talent acquisition and pay-for-expensive-hardware problems. This for-profit entity is capped. (https://openai.com/our-structure/)
(Side note: other non-profit entities with for-profit subsidiaries to supplement funding include Goodwill, Girl Scouts of America, Salvation Army, Mozilla, etc.)
We can't know all the back room dealings but it seemed like the donors were on board with the 2019 for-profit entity. The donors understood the original non-profit was not viable to do AGI work because they underestimated the costs. The publicly revealed emails that as early as 2017, Elon was also pushing to switch OpenAI to be a for-profit company.[1] But the issue was Elon wanted to run it and Sam disagreed. Elon left OpenAI and now he has competing AI businesses with xAI Grok and Tesla AI which makes his lawsuit have some conflicts of interest. I don't which side to believe but that's the soap opera drama.
Now in 2024, the 2019 for-profit OpenAI Global LLC has shown structural flaws because the next set of investors with billions don't want to put money into that LLC. Instead, the next investors need a public incorporated company with ability to IPO as the vehicle. That's where Sam wants to create another for-profit OpenAI Inc without the cap. We should be skeptical but he argues that a successful OpenAI will funnel more money back to the non-profit OpenAI than if it were a standalone non-profit that didn't take billions in investment.
It's not just about those whose money is at stake. The whole point of having non-profits as an option for corporations in the first place is to encourage things that broadly benefit society. Effectively turning a non-profit into a for-profit company is directly counter to that, and it means that society as a whole was defrauded when non-profit status was originally claimed with its associated perks.
> In other words, let's suppose the old Friendster social network was structured as "non-profit-board-controlling-a-for-profit-Friendster" like OpenAI. The ideals of the "non-profit being in control" is a moot point when a competitor like Facebook makes non-profit-Friendster irrelevant.
This feels like it's missing a really really important point, which is that in this analogy, the mission of the non-profit would be something like, "Make social media available to all of humanity without advertising or negative externalities", and the for-profit plans to do advertising to compete with Facebook.
The for-profit's only plan for making money goes directly against the goals of the nonprofit. That's the problem. Who cares if it's competitive if the point of the competition is to destroy the things the non-profit stands for?
“We’re going public because we want more money. We need more money for more computer time but that’s not all. ChatGPT has been so influential that we deserve a bigger share of the rewards than we have gotten.”
"We're beginning to see that this path isn't going where we thought it would so we're going to extract as much value as we can before it crashes into mundaneness."
Strong principled stance until the valuations got big (helped in no small measure by the principled stance)…and then backtracked it when everyone saw the riches there for the taking with a little let’s call it reframing
Everyone has a price, is this meant to be shocking? I mean, I’m disappointed… but I’d have been far more surprised if they’d stood fast with the philanthropic mission once world-changing money was on the table.
No, not everyone has a price. Obviously anecdotal but I have met some truly passionate people in real life who wouldn't compramise their values. Humanity has not lost just yet.
(I would say the same about some people who I haven't personally met, but it would be speculation)
people who do have a price, tend to cluster around situations where that can come into play; people who do not have a price, tend to cluster around situations where that does not come into play (?)
I’ve thought about this, a lot. My price is way higher than it once was, but still, if someone came along and dropped $10bn on my desk, I’d hear them out. There’s things I’d say no to, regardless of price, but otherwise things are probably negotiable.
It might sound fun to 'have' $10bn but consider losing your family, knowing that every person you meet is after your money (because they are), not having anyone give you responsible feedback, spending large amounts of time dealing with lawyers and accountants and finance bros, and basically never being 'normal' again. Winning a huge amount of money in a lottery carries a huge chance of ruining your life.
There's a limit to the amount of money I'd want (or could even use), and it's well below $10b. If someone came around with $10m and an honest deal and someone else offered $10b to buy the morality in my left toe, I'd take the $10m without question.
> Everyone has a price, is this meant to be shocking?
Left a very-well paying job over conscience reasons. TC was ~3x higher than I could get elsewhere without immigrating, probably higher than anywhere relative to CoL. I wasn't even doing defense stuff, crypto scams or anything clearly questionable like that, just clients were mostly in fossil-fuel adjacent sectors. Come from a lower-class background and haven't built up sizable assets at all, will likely need to work until retirement age.
AMA.
If anyone similar reads this, would love to get in touch as I'm sure we'll have a lot in common. In case someone here knows me, hi!
There is a huge difference between being “principled” between 3x and 1x when you are going from 200K to 600K and when you are going from $50K to $150K.
Once you have “enough”, your personal marginal utility for money changes. Would you go from what you are making now to being an at home nurse taking care of special needs kids for $16/hour?
> There is a huge difference between being “principled” between 3x and 1x when you are going from 200K to 600K and when you are going from $50K to $150K.
If it'd be anything remotely like the former, I would have built up sizable assets (which I didn't), especially as I mentioned relative to CoL :)
Did you give up so much potential income that you couldn’t meet your short term and long term wants and needs?
What did you give up in your lifestyle that you personally valued to choose a lower paying job?
I’m 50, (step)kids are grown, we downsized from the big house in the burbs to a condo in a state tax free state, etc and while I could make “FAANG” total compensation (been there done that), I much prefer a more laid back job, remote work, freedom to travel,etc. I also have always hated large companies.
I would have made different choices a decade ago if the opportunities had arisen.
I’m well aware that my income puts me at the top quintile of household income (while still lower than a mid level developer at any of the BigTech companies).
Isn’t it crazy how everything that could maybe dig us out of this hole is being held back because a few extremely rich people have a small chance of losing a tiny bit of money.
There had never been one. Not with Sam Altman. It was a play put on to get $100+ million in donations. This was always the goal, from day 0. This is trivially obvious considering the person Sam Altman.
Anyone who thought Sam Altman wasn’t in it for the money from the start, was being naive. In the extreme. Not only Mr Altman, but most of the people giving the larger donations were hoping for a hit as well. Why is that so incredible to people? How else would you get that kind of money to fund a ludicrously speculative research based endeavor? You don’t even know if it’s possible before the research. What else could they have done?
Agreed, hence why he scammed people into believing he was in it for the "good of humanity" - 1. Get donors 2. Get bright minds in the space (e.g. Ilya) with such ideals to work for him.
This is specifically why I caution people against trusting OpenAI's "we won't train on your data" checkbox. They are specifically financially incentivized to do so, and have a demonstrated history of saying the nice, comforting thing and then doing the thing that benefits them instead.
20 years of social media has turns you into a petty teenager. This stuff reeks of Sam Altman having uncontested power and being able to write whatever dumb shit he wants on openai.com. But it's not just OpenAI, Microsoft is also stunningly childish, and I think professionalism in corporate communications has basically collapsed across the board.
The important bit (which seems unclear from this article), is the exact relationship between the for-profit and the not for profit?
Before, profits were capped, with remainder going to the non-profit to distribute benefits equally across the world in event of agi / massive economic progress from agi. Which was nice, as at least on paper, a plan for an “exit to humanity”.
This reads to me like the new structure might offer uncapped returns to investors, with a small fraction reserved to benefit the wider public via this nonprofit. So dropping the “exit to humanity”, which seemed like a big part of OpenAI’s original vision.
Early on they did some good research on this too, thinking about the investor model, and its benefits for raising money and having accountability etc in todays world, vs what the right structure could be post SI, and taking that conversation pretty seriously. So it’s sad to see OpenAI seemingly drifting away from some of that body of work.
The world is moving to build out a new infrastructure of energy, land use, chips, datacenters, data, AI models, and AI systems for the 21st century economy.
In other news, one of OpenAI's top talents, and first author on the GPT-1 paper, Alec Radford, left a few days ago to pursue independent research.
In additional other news, Microsoft and OpenAI have now reportedly agreed on a joint definition of relationship-ending AGI as "whatever makes $100B". Not kidding.
> In additional other news, Microsoft and OpenAI have now reportedly agreed on a joint definition of relationship-ending AGI as "whatever makes $100B". Not kidding.
Here's the broadly sarcastic reaction on this very site at the time of the announcement, I'm particularly noticing all the people who absolutely did not believe that the 100x cap on return on investments was a meaningful limit: https://news.ycombinator.com/item?id=19359928
As I understand it, Microsoft invested about a billion in 2019 and 13-14 billion more recently, so if the 100x applied to the first, the 100 billion limit would hit around now, while the latter would be a ~1.3 trillion USD cap assuming the rules hadn't been changed for the next round.
I don't think the new $100B=AGI thing is about investment return, but rather about reassuring sugar-daddy Microsoft, and future investors. The old OpenAI-Microsoft agreement apparently gave OpenAI the ludicrous ability to self-define themselves as having reached AGI arbitrarily, with "AGI reached" being the point beyond which Microsoft had no further rights to OpenAI IP.
With skyrocketing training/development costs, and OpenAI still unprofitable, they are still totally dependent on Microsoft, and Microsoft rightfully want to protect their own interests as they continue to expand their AI datacenters. Future investors want the Microsoft relationship to be good since OpenAI are dependent on it.
Sure, but many of these people who have left still appear interested in developing AGI (not just enjoying their f u money), but apparently think they have better or same chance of doing so independently, or somewhere else ...
> Our current structure does not allow the Board to directly consider the interests of those who would finance the mission
Why should it? You can't serve two masters. They claim to serve the master of human-aligned AI. Why would they want to add another goal that's impossible to align with their primary one?
I want to put my opinion somewhere and as a reply to this seems like a good option.
I am anti-capitalism and anti-profit and anti-Microsoft, but I can take a step back and recognize the truth to what’s being said here. They give the context that hundreds of billions are being spent on AI right now, and if OpenAI wants to remain competitive, they need significantly more money than they originally anticipated. They’re recognizing they can’t get this money unless there is a promise of returns for those who give it. There any not many people equipped to fork over $50 billion with zero return and I assume of the very few people who can, none expressed interest in doing so.
They need money to stay competitive. They felt confident they’d be surpassed without it. This was their only avenue for getting that money. And while it would have been more true to the original mission to simply let other people surpass them, if that’s what happens, my guess is that Sam is doing some mental gymnastics as to how now letting that happen and at least letting some part of OpenAI’s success be nonprofit is better than the competition who would have 0% be charitable
It seems to me like they started out aiming for a smaller role, pursuing AGI through the advancement of algorithms and technologies. After their 15min of fame where they released half-baked technology (in the true Stanford Way), they seem to be set on monopolizing AGI.
They only need the billions to compete with the other players.
The mission was never to stay competitive. The mission was to develop AGI. They need to be competitive if they want to make billions of dollars for themselves and their shareholders; they don't need to be competitive to develop AGI.
> my guess is that Sam is doing some mental gymnastics as to how now letting that happen and at least letting some part of OpenAI’s success be nonprofit is better than the competition who would have 0% be charitable
"I don't know about you people, but I don't want to live in a world where someone else makes the world a better place better than we do." - Gavin Belson
“ OpenAI is a non-profit artificial intelligence research company. Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return. Since our research is free from financial obligations, we can better focus on a positive human impact.
We believe AI should be an extension of individual human wills and, in the spirit of liberty, as broadly and evenly distributed as possible. The outcome of this venture is uncertain and the work is difficult, but we believe the goal and the structure are right.”
…
“ We’re hoping to grow OpenAI into such an institution. As a non-profit, our aim is to build value for everyone rather than shareholders. Researchers will be strongly encouraged to publish their work, whether as papers, blog posts, or code, and our patents (if any) will be shared with the world. We’ll freely collaborate with others across many institutions and expect to work with companies to research and deploy new technologies.”
It’s interesting that their original mission statement is basically their own evaluation that by having to consider financial goals they are detracting from their ability to have positive impacts on humanity. They’ve plainly said there is a trade off.
338 comments
[ 3.2 ms ] story [ 276 ms ] threadI did work around this last year and there was no limit to how smart you could get a swarm of agents using different base models at the bottom end. This at the time was a completely open question. It's still the case that no one has build an interactive system that _really_ scales - even the startups and off the record conversations I've had with people in these companies say that they are still using python across a single data center.
AGI is now no longer a dream but a question of if we want to:
1). Start building nuclear power plants like it's 1950 and keep going like it's Fallout.
2). Wait and hope that Moore's law keeps applying to GPUs until the cost of something like o3 drops to something affordable, in both dollar terms and watts.
Nuclear has a (much) higher levelized cost of energy than solar and wind (even if you include a few hours of battery storage) in many or most parts of the world.
Nuclear has been stagnant for ~two decades. The world has about the same installed nuclear capacity in 2024 as it had in 2004. Not in percent (i.e. “market share”) but in absolute numbers.
If you want energy generation cheap and fast, invest in renewables.
I don’t think blackouts are very common for grid connected data centers :)?
Wind is strong at night when solar produces nothing. Same in the winter months.
As I said: if the power consumer is grid connected, this does not matter. Example: I have power in the socket even at night time :)
As long as you have uninterrupted power (i.e. as long as connected to the grid), the important metric is mean cost of energy, not capacity factor of the production plant.
For a nuclear sub or a space ship, which is not grid connected, capacity factor is very important. But data centers are usually grid connected.
> when data enters need power all day every day nuclear is the only solution
Do you think data centers running at night are running exclusively on nuclear-generated power :)?
We already have lots of data centers that need power all day every day. Most are just grid connected. It works.
Especially if it takes so much compute to do any one task.
Sounds legit.
To extrapolate, (of LLMs and GenAI) the more I see use of and how it's used the more it shows severe upwards limits, even though the introduction of those tools has been phenomenal.
On business side, OpenAI lost key personnel and seemingly the plot as well.
I think we've all been drinking a bit too much on the hype of it all. It'll al;l settle down into wonderful set of (new) tools, but not on AGI. Few more (AI) winters down the road, maybe..
We had what I, personally, would count as a "general-purpose AI" already with the original release of ChatGPT… but that made me realise that "generality" is a continuum not a boolean, as it definitely became more general-purpose with multiple modalities, sound and vision not just text, being added. And it's still not "done" yet: while it's more general across academic fields than any human, there's still plenty that most humans can do easily that these models can't — and not just counting letters, until recently they also couldn't (control a hand to) tie shoelaces*.
There's also the question of "what even is intelligence?", where for some questions it just matters what the capabilities are, and for other questions it matters how well it can learn from limited examples: where you have lots of examples, ChatGPT-type models can be economically transformative**; where you don't, the same models *really suck*.
(I've also seen loads of arguments about how much "artificial" counts, but this is more about if the origin of the training data makes them fundamentally unethical for copyright reasons).
* 2024, September 12, uses both transformer and diffusion models: https://deepmind.google/discover/blog/advances-in-robot-dext...
** the original OpenAI definition of AGI: "by which we mean highly autonomous systems that outperform humans at most economically valuable work" — found on https://openai.com/charter/ at time of writing
It's not that the remainder want nonprofit ownership, it's that they can't legally just jettison it, they need a story how altering the deal is good actually.
I’m not asking to be snarky or imply any hidden meaning…I just don’t see how they plan on getting from A to B.
From this recent press release the answer seems to be: make ChatGPT really good and offer it for free to people to use. Which is a reasonable answer, I suppose, but not exactly one that matches the highfalutin language being used around AGI.
Well you see the AGI can only benefit humanity if it is funded via traditional equity... Investors WANT to give their money but the horrendous lack of ownership is defeating their goodwill
Considering their definition of AGI is “makes a lot of money”, it’s not going to—and was never designed to—benefit anyone else.
https://techcrunch.com/2024/12/26/microsoft-and-openai-have-...
https://www.youtube.com/watch?v=gjQUCpeJG1Y
What else could we have expected from someone who made yet another cryptocurrency scam?
https://www.technologyreview.com/2022/04/06/1048981/worldcoi...
https://www.buzzfeednews.com/article/richardnieva/worldcoin-...
Sam Altman doesn’t give a rat’s ass about improving humanity, he cares about personal profit.
https://www.newyorker.com/cartoon/a16995
We're basically betting our species' future on these guys failing, because for a short period of time there's a massive amount of shareholder value to be made.
Obviously the way our economy and society are structured that's not what will happen, but I don't think that has much to do with tools and their tendency to increase our efficiency and output.
Put another way, there are powerful benefits from AGI that we will squander because our system sucks. That is not a critique against AGI, that is a critique of our system and will continue to show up. It's already a huge point of conversation in our collective dialogue.
But AI does not come with glands, with brainstem instincts, with a limited physical strength, with childhood morality tales, with the ability to feel pain or empathy, with the desire for respect and public social approval, with the feeling of emotional connection with friends and family, with the feeling of helplessness and desire for freedom & variety, with the feeling of boredom and dislike of menial work, with the idealism of various religious and ethical mores.
It does not come packaged with any of the moral values or psychological quirks that constrain our behavior and allow us to work together. It is in fact, very difficult to teach these things to a machine - far more difficult than less subjective ideas like "Reason a way to do the thing that does the other thing". We do not have a mathematical proof of Correct Human Ethics, and we have reason to believe that this is a Difficult Problem.
It's arguably much easier to build a fully general AI that serves as what's termed a "paperclipper", which eats the world in pursuit of its task because we didn't formulate a reason for it not to do so and express it in the correct ten gigabyte matrix of floating point values, than to build a fully general AI which pursues what we actually implicitly want it to do for us in a way that the median eight year old would be able to follow.
The people studying AI alignment specifically were pessimistic fifteen years ago, and today the dominant stance is "Dying with dignity", give up the struggle to warn everybody and just try to enjoy what time you have.
The most positive take is: they want to build a general-purpose AI, to allow fully automated luxury for all; to built with care to ensure it can only be used for positive human flourishing and cannot (easily or at all) be used for nefarious purposes by someone who wants to sow chaos or take over the world; and to do so in public so that the rest of us can prepare for it rather than wake up one day to a world that is alien to us.
Given the mental image I have here is of a maze, you may well guess that I don't expect this to go smoothly — I think the origin in Silicon Valley and startup culture means OpenAI, quite naturally, has a bias towards optimism and to the idea that economic growth and tech is a good thing by default. I think all of this is only really tempered by the memetic popularity of Eliezer Yudkowsky, and the extent to which his fears are taken seriously, and his fears are focussed more on existential threat of an optimising agent that does the optimising faster than we do, not on any of the transitional dynamics going from the current economy to whatever a "humans need not apply" economy looks like.
I still don't understand this. What does it mean in practice?
Covid does not hate you, nor does it love you, it simply follows an optimisation algorithm — that of genetic evolution — for the maximum reproductive success, and does so without regard to the damage it causes your body while it consumes you for parts.
Covid is pretty stupid, it's just a virus.
And yet: I've heard the mutation rate is 3.8 × 10e−6 / nucleotide / cycle, and at about 30,000 base pairs and 10e9 to 1e11 virons in an infected person, so that's ~1e8-1e10 mutations per reproductive cycle in an infected person, and that the replication cycle duration is about 10 hours. Such mutations are both how it got to harm us in the first place, why vaccination isn't once-and-done, and this logic also applies to all the other diseases in the world (including bacterial ones, which is why people are worried about bacterial resistance).
As an existence proof, Covid shows how an agent going off and doing its own thing, if it does it well enough, doesn't even need to be smart to kill a percentage point or so of the human species *by accident*.
The hope is that AI will be smart enough that we can tell it: humans (and the things we value) are not an allowed source of parts. The danger happens well before it's that smart… and that even when it is that smart, we may well not be smart enough to describe all the things we value, accurately, and without bugs/loopholes in our descriptions.
But for AI, they are all brains (or that's the claim) but no brawn. How do they act?
Some of those viruses shut down hospitals, some damage uranium enrichment facilities, some mine or steal bitcoin, some steal normal money.
Polymorphic viruses have been around for decades, so it's not inconceivable that there's already some almost entirely evolved (simulated evolution is a kind of AI training mechanism) virus on the internet already, even before LLMs.
Right now, I'd expect an LLM to be fairly mediocre at writing a virus. But it doesn't need to be an LLM that does the writing, just as it doesn't need to be an LLM that is the payload once the infection has happened.
They don't address benefits beyond the wide-scale automation of economically-valuable work, and as those benefits require significant revisions to social structures it's probably appropriate that they keep their mouth shut on the subject.
Demonizing someone who helped you is an awful thing to do. If he gave 1/3 of the initial funding, he helped a lot.
Saying "less than" is peculiar phrasing for such a substantial amount, but maybe some people believe Elon initially funded about all of it
Sam Altman is one of the most well connected people in Silicon Valley.
And investors like Reid Hoffman aren't having their lives being dictated by Musk.
However, Mr. Musk is continually called out by OpenAI in the public and OpenAI has quite the megaphone.
From what I see, this is mainly due to Musk complaining loudly in public.
And unlike the caving instructor that Musk libelled, OpenAI has the means to fight back as an equal.
That said, I don't see anything in this post that I'd describe as Musk "being called out".
Unfortunately, it’s been in a quite negative tone.
If all of this is really for humanity — then humanity needs to shape up, get along and do this together.
He didnt just want to help, he wanted to control the company by being the CEO.
If we're a for-profit company with investors and returns etc... then those initial donations seem far closer to seed capital than a not-for-profit gift. Of course hindsight is 20/20, and I can believe that this wasn't always some devious plan but rather the natural evolution of the company... but still seems inequitable.
As much as Elon's various antics might deserve criticism (especially post-election) he seems to be in the right here? Or am I missing something?
But we're all kind of arguing over which demon is poking the other the hardest with their pitchfork, here.
The details here matter and is bs. What should take place is this, OpenAI creates a new for profit entity (PBC, or whatever structure). That company sets an auction for 5-10% of the shares. This yields the valuation. The new company acquires the old company with equity, using the last valuation. So say $30b is raised for 10%, means $300B.
So the $160B becomes like 53% and then 63% with the 10% offer. So the non-profit keeps 37% plus whatever it owns of the current for profit entity.
Auction means the price is fair and arms-length, not trust me bro advisors that rug pull valuations.
I believe on this path, Elon Musk has a strong claim to get a significant portion of the equity owned by the non-profit, given his sizable investment when a contemporaneous valuation of the company would have been small.
Sorry, what's the path to this? Musk's 'investment' was in the form of a donation, which means he legally has no more claim to the value of the nonprofit than anyone else.
If donors got equity in the PBC or the owned PBC avoided competing with donor owned companies this would not be consistent with the non-profit's mission, and would not be compliant with 501c3 restrictions.
This is not just an issue for the change that OpenAI is contemplating right now, but also the capped for-profit change that happened years ago. If that's found to be improper, I'm curious if that entitles the donors to any kind of compensation.
ETA: looking into this, I found the following precedent (https://www.techpolicy.press/questioning-openais-nonprofit-s...).
> A historic precedent for this is when Blue Cross' at the time nonprofit health insurers converted into for-profit enterprises. California Blue Cross converted into what's now Anthem. They tried to put a small amount of money into a nonprofit purpose. The California Attorney General intervened and they ultimately paid out about $3 billion into ongoing significant health charitable foundations in California. That's a good model for what might happen here.
So my guess is there's no compensation for any of the donors, but OpenAI may in the end be forced to give some money to an open, nonprofit AI research lab (do these exist?). IANAL so that's a low-confidence guess.
Still, that makes me so queasy. I would never donate to a YC-backed nonprofit if this is how it can go, and I say that as a YC alum.
That is not the real definition of AGI. The real "definition" can mean anything at this point and in their leaked report from the Information [0] they have defined it as "returning $100 billion or so in profits"
In other words raise more money until they reach AGI. This non-profit conversion to for-profit is looking like a complete scam from the original mission [1] when they started out:
> Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return. Since our research is free from financial obligations, we can better focus on a positive human impact.
"AGI" at this point is a meaningless term abused to fleece investors to raise billions for the displacement of jobs either way which that will "benefit humanity" with no replacement or alternative for those lost jobs.
This is a total scam.
[0] https://www.theverge.com/2024/12/26/24329618/openai-microsof...
[1] https://openai.com/index/introducing-openai/
and do so without any human involvement
This seems exactly backwards. At this scale you can establish as much "bespokeness" as you want. The investors want in and will sign pretty much anything.
It reminds me of the old joke from Paul Getty:
If you owe the bank $100 that's your problem. If you owe the bank $100 million, that's the bank's problem.
I kind of thought that was the point of the current structure.
Yes, it is but if we only stopped the analysis right there, we could take pleasure in the fact that Sam Altman checkmated himself in his own blog post. "Dude, the non-profit is _supposed_ to control the profit company because that's how you formed the companies in the first place! Duh!!!"
To go beyond that analysis, we have to at least entertain (not "agree" but just _entertain_ for analysis) ... what Sam is saying:
- the original non-profit and profit structure was a mistake that was based on what they thought they knew at the time. (They thought they could be a "research" firm.)
- having a non-profit control the profit becomes a moot point if the for-profit company becomes irrelevant in the marketplace.
Here is a key paragraph:
>The hundreds of billions of dollars that major companies are now investing into AI development show what it will really take for OpenAI to continue pursuing the mission. We once again need to raise more capital than we’d imagined. Investors want to back us but, at this scale of capital, need conventional equity and less structural bespokeness.
In other words, let's suppose the old Friendster social network was structured as "non-profit-board-controlling-a-for-profit-Friendster" like OpenAI. The ideals of the "non-profit being in control" is a moot point when a competitor like Facebook makes non-profit-Friendster irrelevant.
Or put another way, pick any hard problem today (self-driving, energy discovery, etc) that requires billions of investment and hypothetically create a new company to compete in that space. Would creating that company as a non-profit-controlling-the-profit-company confer any market advantage or would it be a handicap? It looks like it's a handicap.
OpenAI is finding it hard to compete for investors' billions in the face of Tesla's billions spent on 50000 GPU supercluster, Google billions spent on Gemini, Anthropic billions spent on Claude, Alibaba's billions, etc. OpenAI doesn't have an unassailable lead with ChatGPT.
The issue is Sam Altman & OpenAI look bad because he and the investors want to use the existing "OpenAI" brand name in a restructured simpler for-profit company. But to outsiders, it looks like a scam or bait-and-switch. Maybe they could have done an alternative creative procedure such as spin up a totally new for-profit company called ClosedAI to take OpenAI's employees and then pay a perpetual ip license to OpenAI. That way, CloseAI is free of encumbrances from OpenAI's messy structure. But then again, Elon Musk would still probably file another lawsuit calling those license transfers as "bad faith business dealings".
I’m not sure what the equivalent would be for llm products.
There’s no slavery here. If Sam decided it was a mistake to dedicate his time to a non-profit, he’s perfectly free to quit and start an entirely new organization that comports with his current vision. That would be the honorable thing to do.
To be clear, I don't think Sam can do anything and come out looking "good" from a public relations standpoint.
That said, Sam probably thinks he's justified because he was one of the original founders and co-chair of OpenAI -- so he feels he should have a say in pivoting it to something else. He said he got all the other donors on board ... except for Elon.
That leaves us with the messy situation today... Elon is the one filing the lawsuit and Sam is writing a PR blog post that's received as corporate doublespeak.
No. Non-profit is a deal between those founders and society. That he was an original founder is irrelevant. I don’t care about Elon, it’s the pivoting that’s inherently dishonorable.
Yes I get that but did OpenAI ever take any public donations from society? I don't think they did. It thought it was only funded by wealthy private donors.
>, it’s the pivoting that’s inherently dishonorable.
Would creating a new company (i.e. ClosedAI) that recruits OpenAI's employees and buys the intellectual property such that it leaves a "shell" of OpenAI be acceptable?
That's basically the roundabout thing Sam is trying to do now with a re-incorporated for-profit PBC that's not beholden to the 2015 non-profit organization ... except he's also trying to keep the strong branding of the existing "OpenAI" name instead of "ClosedAI".
The existing laws allow for non-profit 501c3 organizations to "convert" (scare quotes) to for-profit status by re-incorporating to a (new) for-profit company. That seems to be Sam's legal roadmap.
EDIT REPLY: They received benefits by dint of their status. If there were no such benefits
The main benefit is tax exemption but OpenAI never had profits to be taxed. Also to clarify, there's already a for-profit OpenAI Global LLC. That's the subsidiary company Microsoft invested in. It has the convoluted "capped profit" structure. Sam says he can't attract enough investors to that for-profit entity. Therefore, he wants to create another for-profit OpenAI company that doesn't have the convoluted ("bespoke" as he put it) self-imposed rules to be more attractive to new investors.
The 2 entities of non-profit and for-profit is like Mozilla Foundation + Mozilla Corporation.
[] https://www.google.com/search?q=conversion+of+501c3+to+for-p...
They received benefits by dint of their status. If there were no such benefits they wouldn’t have incorporated that way.
In any case, would creating a new company (i.e. ClosedAI) that recruits OpenAI's employees and buys the intellectual property such that it leaves a "shell" of OpenAI be acceptable?
There’s no problem with recruiting employees. The intellectual property purchase is problematic. If it’s for sale, it should be for sale to anyone and no one connected to a bidder should be involved in evaluating offers.
The existing laws allow for non-profit 501c3 organizations to "convert" (scare quotes) to for-profit status by re-incorporating to a (new) for-profit company. That seems to be Sam's legal roadmap.
Legal and honorable are not synonyms.
Was the non profit a way to just avoid tax until the time came to start making money?
You'd want to do it the other way around
It's also notable that this is in fact what the now vast majority of other OpenAI founders chose to do.
It doesn't look like one - it is one.
> We’re excited to welcome the following new donors to OpenAI: Jed McCaleb (opens in a new window), Gabe Newell (opens in a new window), Michael Seibel (opens in a new window), Jaan Tallinn (opens in a new window), and Ashton Eaton (opens in a new window) and Brianne Theisen-Eaton (opens in a new window). Reid Hoffman (opens in a new window) is significantly increasing his contribution. Pieter Abbeel (opens in a new window) (having completed his sabbatical with us), Julia Galef (opens in a new window), and Maran Nelson (opens in a new window) are becoming advisors to OpenAI.
[1] https://openai.com/index/openai-supporters/
The story is that OpenAI worked out some equity conversion of the for-profit co for the donors to the non-profit. Elon Musk was the notable holdout. Elon was offered some unknown percentage for his ~$40 million donation but he refused.
Seems like the donors are ok with OpenAI pivot to for-profit ... except for Elon. So no bait-n-switch as seen from "insiders" perspective.
If you have information that contradicts that, please add to the thread.
These people "donated to a non-profit". They did not "invest in a for-profit".
If the Red Cross suddenly turns into a for-profit and then says "well we'll give our donators of the past few years equity in our new company", this does not make it any less of a scam.
> Seems like the donors are ok with OpenAI pivot to for-profit
If you have information that shows this, feel free to add it. "Not suing" is not the same as that. Very few people sue even when they feel they're scammed.
Sure, nobody wants to be tricked into donating to a charity and then have their money disappear into a for-profit company.
Based on the interviews I saw from some donors (Reid Hoffman, etc), there's more nuance to it than that. The donors also wanted an effective non-profit entity. The TLDR is that they donated to a non-profit under 2015 assumptions of AGI research costs that turned out to be wrong and massively underestimated.
- 2015... the non-profit OpenAI in its original idea of "charity research organization" was flawed from the beginning because they realized they couldn't attract A.I. talent at the same level as Google/Facebook/etc as those competitors offered higher salaries and lucrative stock options. Then, they realized the initial ~$100 million in donations was also not enough to pay for very expensive hardware like GPUs and datacenters. It's hard for researchers to make discoveries in AGI if there's no cutting edge hardware for them to work on. A non-profit tech charity getting billions in donations was not realistic. These money problems compound and lead to...
- 2019... create the for-profit OpenAI Global LLC as a vehicle for the Microsoft investment of $1 billion and also create stock incentives for recruiting employees. This helps solve the talent acquisition and pay-for-expensive-hardware problems. This for-profit entity is capped. (https://openai.com/our-structure/)
(Side note: other non-profit entities with for-profit subsidiaries to supplement funding include Goodwill, Girl Scouts of America, Salvation Army, Mozilla, etc.)
We can't know all the back room dealings but it seemed like the donors were on board with the 2019 for-profit entity. The donors understood the original non-profit was not viable to do AGI work because they underestimated the costs. The publicly revealed emails that as early as 2017, Elon was also pushing to switch OpenAI to be a for-profit company.[1] But the issue was Elon wanted to run it and Sam disagreed. Elon left OpenAI and now he has competing AI businesses with xAI Grok and Tesla AI which makes his lawsuit have some conflicts of interest. I don't which side to believe but that's the soap opera drama.
Now in 2024, the 2019 for-profit OpenAI Global LLC has shown structural flaws because the next set of investors with billions don't want to put money into that LLC. Instead, the next investors need a public incorporated company with ability to IPO as the vehicle. That's where Sam wants to create another for-profit OpenAI Inc without the cap. We should be skeptical but he argues that a successful OpenAI will funnel more money back to the non-profit OpenAI than if it were a standalone non-profit that didn't take billions in investment.
[1] https://www.google.com/search?q=elon+musk+emails+revealed+op...
This feels like it's missing a really really important point, which is that in this analogy, the mission of the non-profit would be something like, "Make social media available to all of humanity without advertising or negative externalities", and the for-profit plans to do advertising to compete with Facebook.
The for-profit's only plan for making money goes directly against the goals of the nonprofit. That's the problem. Who cares if it's competitive if the point of the competition is to destroy the things the non-profit stands for?
If you cash them out at the last round price, I bet they wouldn’t be happy.
They want to convert at a trust me bro conversion rate. “Independent financial advisors” lol.
Who buys this stuff?
“We’re going public because we want more money. We need more money for more computer time but that’s not all. ChatGPT has been so influential that we deserve a bigger share of the rewards than we have gotten.”
"We're beginning to see that this path isn't going where we thought it would so we're going to extract as much value as we can before it crashes into mundaneness."
Strong principled stance until the valuations got big (helped in no small measure by the principled stance)…and then backtracked it when everyone saw the riches there for the taking with a little let’s call it reframing
(I would say the same about some people who I haven't personally met, but it would be speculation)
“Somebody please think of the investors they only have 500 years of generational wealth”
Speak for yourself.
There's a limit to the amount of money I'd want (or could even use), and it's well below $10b. If someone came around with $10m and an honest deal and someone else offered $10b to buy the morality in my left toe, I'd take the $10m without question.
You can only buy the International Space Station twice.
I do think part of the price (paid) should be getting bluntly called out for it.
Left a very-well paying job over conscience reasons. TC was ~3x higher than I could get elsewhere without immigrating, probably higher than anywhere relative to CoL. I wasn't even doing defense stuff, crypto scams or anything clearly questionable like that, just clients were mostly in fossil-fuel adjacent sectors. Come from a lower-class background and haven't built up sizable assets at all, will likely need to work until retirement age.
AMA.
If anyone similar reads this, would love to get in touch as I'm sure we'll have a lot in common. In case someone here knows me, hi!
Once you have “enough”, your personal marginal utility for money changes. Would you go from what you are making now to being an at home nurse taking care of special needs kids for $16/hour?
If it'd be anything remotely like the former, I would have built up sizable assets (which I didn't), especially as I mentioned relative to CoL :)
What did you give up in your lifestyle that you personally valued to choose a lower paying job?
I’m 50, (step)kids are grown, we downsized from the big house in the burbs to a condo in a state tax free state, etc and while I could make “FAANG” total compensation (been there done that), I much prefer a more laid back job, remote work, freedom to travel,etc. I also have always hated large companies.
I would have made different choices a decade ago if the opportunities had arisen.
I’m well aware that my income puts me at the top quintile of household income (while still lower than a mid level developer at any of the BigTech companies).
https://dqydj.com/household-income-percentile-calculator/
Be as vague as you are comfortable with. But where are you with respect to the median income locally?
Decades earlier retirement. Bringing with it the exact freedom you're talking about.
There had never been one. Not with Sam Altman. It was a play put on to get $100+ million in donations. This was always the goal, from day 0. This is trivially obvious considering the person Sam Altman.
Anyone who thought Sam Altman wasn’t in it for the money from the start, was being naive. In the extreme. Not only Mr Altman, but most of the people giving the larger donations were hoping for a hit as well. Why is that so incredible to people? How else would you get that kind of money to fund a ludicrously speculative research based endeavor? You don’t even know if it’s possible before the research. What else could they have done?
The amount of trashy pique in OpenAI PR against Elon specifically is hilarious.
I'm no fan of the way either has behaved, but jesus, can't even skip an opportunity to slight him in an unrelated announcement?
Before, profits were capped, with remainder going to the non-profit to distribute benefits equally across the world in event of agi / massive economic progress from agi. Which was nice, as at least on paper, a plan for an “exit to humanity”.
This reads to me like the new structure might offer uncapped returns to investors, with a small fraction reserved to benefit the wider public via this nonprofit. So dropping the “exit to humanity”, which seemed like a big part of OpenAI’s original vision.
Early on they did some good research on this too, thinking about the investor model, and its benefits for raising money and having accountability etc in todays world, vs what the right structure could be post SI, and taking that conversation pretty seriously. So it’s sad to see OpenAI seemingly drifting away from some of that body of work.
Emphasis mine.
Land use? Land use?
I do not welcome our new AI landlords, ffs.
What a bunch of pompous twits.
In other news, one of OpenAI's top talents, and first author on the GPT-1 paper, Alec Radford, left a few days ago to pursue independent research.
In additional other news, Microsoft and OpenAI have now reportedly agreed on a joint definition of relationship-ending AGI as "whatever makes $100B". Not kidding.
OpenAI did that all by themselves before most people had heard of them. The 100x thing was 2019: https://openai.com/index/openai-lp/
Here's the broadly sarcastic reaction on this very site at the time of the announcement, I'm particularly noticing all the people who absolutely did not believe that the 100x cap on return on investments was a meaningful limit: https://news.ycombinator.com/item?id=19359928
As I understand it, Microsoft invested about a billion in 2019 and 13-14 billion more recently, so if the 100x applied to the first, the 100 billion limit would hit around now, while the latter would be a ~1.3 trillion USD cap assuming the rules hadn't been changed for the next round.
With skyrocketing training/development costs, and OpenAI still unprofitable, they are still totally dependent on Microsoft, and Microsoft rightfully want to protect their own interests as they continue to expand their AI datacenters. Future investors want the Microsoft relationship to be good since OpenAI are dependent on it.
So they haven’t even scratched the surface of that given they are widely unprofitable.
Why should it? You can't serve two masters. They claim to serve the master of human-aligned AI. Why would they want to add another goal that's impossible to align with their primary one?
I am anti-capitalism and anti-profit and anti-Microsoft, but I can take a step back and recognize the truth to what’s being said here. They give the context that hundreds of billions are being spent on AI right now, and if OpenAI wants to remain competitive, they need significantly more money than they originally anticipated. They’re recognizing they can’t get this money unless there is a promise of returns for those who give it. There any not many people equipped to fork over $50 billion with zero return and I assume of the very few people who can, none expressed interest in doing so.
They need money to stay competitive. They felt confident they’d be surpassed without it. This was their only avenue for getting that money. And while it would have been more true to the original mission to simply let other people surpass them, if that’s what happens, my guess is that Sam is doing some mental gymnastics as to how now letting that happen and at least letting some part of OpenAI’s success be nonprofit is better than the competition who would have 0% be charitable
They only need the billions to compete with the other players.
The mission was never to stay competitive. The mission was to develop AGI. They need to be competitive if they want to make billions of dollars for themselves and their shareholders; they don't need to be competitive to develop AGI.
"I don't know about you people, but I don't want to live in a world where someone else makes the world a better place better than we do." - Gavin Belson
We believe AI should be an extension of individual human wills and, in the spirit of liberty, as broadly and evenly distributed as possible. The outcome of this venture is uncertain and the work is difficult, but we believe the goal and the structure are right.”
…
“ We’re hoping to grow OpenAI into such an institution. As a non-profit, our aim is to build value for everyone rather than shareholders. Researchers will be strongly encouraged to publish their work, whether as papers, blog posts, or code, and our patents (if any) will be shared with the world. We’ll freely collaborate with others across many institutions and expect to work with companies to research and deploy new technologies.”
https://openai.com/index/introducing-openai/