Oh that’s pretty cool. Do you by chance have any articles or posts going through your initial thought process and/or eventual realization for why you ultimately thought this transition was necessary for OpenAI?
Was it a particular event, a conversation, perhaps just an incremental ideation without any actual epiphany needed, etc?
Really neat corporate structure! We'd looked into becoming a B-Corp, but the advice that we'd gotten was that it was an almost strictly inferior vehicle both for achieving impact and for potentially achieving commercial success for us. I'm obviously not a lawyer, but it's great to see Open AI contributing to the new interesting structures to solve hard global scale problems.
I wonder if the profit cap multiple is going to end up being a significant signalling risk for them. A down-round is such a negative event in the valley, I can imagine a "increasing profit multiple" would have to be treated the same way.
One other question for the folks at OpenAI: How would equity grants work here? You get X fraction of an LP that gets capped at Y dollar profits? Are the fractional partnerships/transferable if earned into?
Yes, we're planning to release a third-party usable reference version of our docs (creating this structure was a lot of work, probably about 6-9 months of implementation).
We've made the equity grants feel very similar to startup equity — you are granted a certain number of "units" which vest over time, and more units will be issued as other join employees in the future. Incidentally, these end up being taxed more favorably than options, so we think this model may be useful for startups for that reason too.
>>Incidentally, these end up being taxed more favorably than options, so we think this model may be useful for startups for that reason too.
Is this due to long term capital gains? Do you allow for early exercising for employees? Long term cap gains for options require holding 2 years since you were granted the options and 1 year since you exercised.
I'm also interested in how this corporate structure relates to a b-corp (or technically, a PBC)
> OpenAI LP’s primary fiduciary obligation is to advance the aims of the OpenAI Charter, and the company is controlled by OpenAI Nonprofit’s board. All investors and employees sign agreements that OpenAI LP’s obligation to the Charter always comes first, even at the expense of some or all of their financial stake.
One of the key reasons to incorporate as a PBC is to allow "maximizing shareholder value" to be defined in non-monetary terms (eg impact to community, environment, or workers).
How is this structure different from a PBC, or why didn't you go for a PBC?
How do you actually define "Fiduciary duty to the charter"? Since charters tend to be vague, and are definitely not legal documents, almost anything can be said to be aligned with the charter.
Similarly, what's stopping an investor from implicit control by threat of removing their investment?
Genuine question: Is this restructure for the purpose of taking government military contracts? I don't see how investors would be getting 100x returns otherwise and my understanding was that salaries for employees was competitive with big tech companies. Curious where Open AI feels like there's money to be made.
Is it just me or there are not many african americans working in AI research and industry. I don't have stats to back me up but that's my personal observation. People in the field, what are your thoughts on it.
This twitter may be of interest to you, they aggregate information on this topic. https://twitter.com/black_in_ai. I don't know why race is relevant to this article though. Must we make everything a race issue?
I don't have any statistics for you, but Google at least is looking to improve on this a bit. A good friend of mine from their NYC Brain office moved to Accra, Ghana just last week to help build out their new office there.
the article is probably the most selfish thing I've read in a while. It's not evil or exploitative, it's just very selfish.
But I think it's not so morally wrong that OpenAI should not do business with him since they have the security mechanisms and limit his power. He just an grumpy old guy who does not want to share his beach.
I just read the article, and am not sure I see the issue. Quote from his lawyer: “No owner of private business should be forced to obtain a permit from the government before deciding who it wants to invite onto its property"
Where's the issue here? They guy basically bought the property all around the beach, and decided to close down access. I wouldn't say it's a nice thing to do, but it's legal. If I buy a piece of property, my rights as the owner should trump the rights of a bunch of surfers who want to get to a beach. The state probably should have been smart enough not to sell all the land.
Failing that, just seize a small portion via eminent domain: a 15-foot-wide strip on the edge of the property would likely come at a reasonable cost, and ought to provide an amicable resolution for all.
It's actually not legal, at least in California. That's why he's trying to take it to the Supreme Court- he's hoping to get the federal government to override state laws.
He was also completely aware of this when he bought the property, so it's not like this is a surprise or someone forcing him to change things. He's the one who broke the law and broke the status quo that had existed at that beach.
Property rights are constitutionally protected, and under the precedents surrounding the 14th amendments, this overrules California's rules. Eminent domain remains legal, but legally speaking, Khosla is in the right here.
The Supreme Court didn't grant cert; that's different. That means they don't want to set precedent or believe sufficient precedent exists already. This was last adjudicated in 1999 with Saenz v. Roe, where California tried to set new residents' welfare to what they got in other states for one year. The court ruled this violated the constitutional protection of interstate travel, and upheld the view that the 14th amendment applied all constitutional rights to all states. Source: https://www.law.cornell.edu/wex/fourteenth_amendment_0
This undoubtedly then applies the 5th amendment takings clause: “…nor shall private property be taken for public use, without just compensation.” This is clearly violated in this sense, and the state cannot violate this right (see above).
The fact that the Supreme Court didn't grant cert probably means they believe there is already precedent here, or just as probably that they didn't have the time. They always have a full docket; they were probably just out of slots.
I urge others to rebut this from a legal sense, not just say they disagree. People keep killing my comments, but it seems like they all just dislike the "selfish" appearance of the actions.
> The Supreme Court refused to overturn the appeal, which means they upheld the decision of the lower courts.
Completely incorrect. "The Court usually is not under any obligation to hear these [appealed] cases, and it usually only does so if the case could have national significance, might harmonize conflicting decisions in the federal Circuit courts, and/or could have precedential value. In fact, the Court accepts 100-150 of the more than 7,000 cases that it is asked to review each year." Source: https://www.uscourts.gov/about-federal-courts/educational-re...
The SC not hearing the case doesn't mean they uphold the lower court's ruling, it means they aren't hearing the case.
See my above reply; the 14th amendment applies the takings clause to California, and the supremacy clause further reinforces that The Constitution trumps california's constitution.
"If you put 10m$ into us for 20% of the post-money business, anything beyond a 5B$ valuation you don't see any additional profits from" which seems like a high but not implausible cap. I suspect they're also raising more money on better terms which would make the cap further off.
Very cool idea. Like some others here, I really appreciate attempts to create new structures for bringing ideas to the world.
Since I'm not a lawyer, can you help understand the theoretical limits of the LP's "lock in" to the Charter? In a cynical scenario, what would it take to completely capture OpenAI's work for profit?
If the Nonprofit's board was 60% people who want to break the Charter, would they be capable of voting to do so?
They were able to attract talent and PR in the name of altruism and here they are now trying to flip the switch as quietly as possible. If the partner gets a vote/profit then a "charter" or "mission" won't change anything. You will never be able to explicitly prove that a vote had a "for profit" motive.
Elon was irritated that he was behind in the AI intellectual property race and this narrative created a perfect opportunity. Not surprised in the end. Tesla effectively did the same thing - "come help me save the planet" with overpriced cars. [Edit: Apparently Elon has left OpenAI but I don't believe for a second that he will not participate in this LP]
* OpenAI Nonprofit’s board consists of OpenAI LP employees Greg Brockman (Chairman & CTO), Ilya Sutskever (Chief Scientist), and Sam Altman (CEO), and non-employees Adam D’Angelo, Holden Karnofsky, Reid Hoffman, Sue Yoon, and Tasha McCauley.
* Elon Musk left the board of OpenAI Nonprofit in February 2018 and is not formally involved with OpenAI LP. We are thankful for all his past help.
* Our investors include Reid Hoffman’s charitable foundation and Khosla Ventures, among others. We feel lucky to have mission-aligned, impact-focused, helpful investors!
My reading that the design of this structure is not to require partners to make decisions in the interest of the mission, but to remove incentives for them to make decisions against the interest of the mission. With a cap on returns, there's a point at which it stops making sense to maximize short-term value or reduce expenses, and with the words about fiduciary duty, it becomes defensible to make decisions that don't obviously increase profit. That is, this structure seems much better than the traditional startup structure and I suspect many entities that are currently actual, normal startups would do more good for the world under a structure like this. (Or that many people who are bootstrapping because they have a vision and they don't want VCs to force them into short-term decisions could productively take some VC investment under this sort of model.)
I agree this isn't a non-profit any more. It seems like that's the goal: they want to raise money the way they'd be able to as a normal startup (notably, from Silicon Valley's gatekeepers who expect a return on investment), without quite turning into a normal startup. If the price for money from Silicon Valley's gatekeepers is a board seat, this is a safer sort of board seat than the normal one.
(Whether this is the only way to raise enough money for their project is an interesting question. So is whether it's a good idea to give even indirect, limited control of Friendly AI to Silicon Valley's gatekeepers - even if they're not motivated by profit and only influencing it with their long-term desires for the mission, it's still unclear that the coherent extrapolated volition of the Altmans and Khoslas of the world is aligned with the coherent extrapolated volition of humanity at large.)
Even more worrying is the prospect that they'll use their profit to lobby for regulation that aligns with their goals under their "non-profit ethical framework", shutting out any would-be competitors who have a different take. If they get big enough it is inevitable. This is overall a gross move that leaves a seriously bad taste. I hope no one takes their ethical arguments seriously as they pursue this path - doing so will endanger the industry.
> "come help me save the planet" with overpriced cars.
You are helping the planet if those customers would've bought ICE luxury vehicles instead of BEV luxury vehicles. I'm not sure BEV could be done any other way but a top-down, luxury-first approach. So, what exactly is your gripe there? Are you a climate change denier or do you believe that cheap EVs were the path to take?
What's to stop someone with a vote but not an investment from significantly investing in an AI application (business/policy/etc.) that directly aligns with one of OpenAI's initiatives? The spirit of this LP structure is commendable but it does not do enough to eliminate pure profit-minded intentions.
> If the partner gets a vote/profit then a "charter" or "mission" won't change anything
(I work at OpenAI.)
The board of OpenAI Nonprofit retains full control. Investors don't get a vote. Some investors may be on the board, but: (a) only a minority of the board are allowed to have a stake in OpenAI LP, and (b) anyone with a stake can't vote in decisions that may conflict with the mission: https://openai.com/blog/openai-lp/#themissioncomesfirst
This seems like an unnecessarily cynical take on things. And ultimately, if the outcome is the same, what do you (or anyone) really care if people are making more money from it or if there are commercial purposes?
The OpenAI staff are literally some of the most employable folks on earth; if they have a problem with the new mission it's incredibly easy for them to leave and find something else.
Additionally, I think there's a reason to give Sam the benefit of the doubt. YC has made multiple risky bets that were in line with their stated mission rather than a clear profit motive. For example, adding nonprofits to the batch and supporting UBI research.
Their's nothing wrong with having a profit motive or using the upsides of capitalism to further their goals.
It most certainly is not unnecessarily cynical. The point is that money clouds the decision-making process and responsibilities of those involved - which is the whole ethos that OpenAI was founded on.
If you are trying maximise the benefit to society it may be necessary to crack AGI before Google or some other corporation does. That's probably not going to happen without serious resources.
Yes or no: will you remain a registered non-profit organization (401/501-type orgs or similar), or were you ever? It's fine to call your self non-profit but if you don't have to abide by the rules of them then you aren't, period.
I think all of us here are tired of "altruistic" tech companies which are really profit mongers in disguise. The burden is on you all to prove this is not the case (and this doesn't really help your case).
The article says that OpenAI the registered non-profit now owns OpenAI LP. This isn't unprecedented: for instance, the Mozilla Foundation owns the Mozilla Corporation, and IKEA is owned by a non-profit. (I mention these two examples precisely because it's common to find Mozilla's corporate structure reasonable and IKEA's corporate structure unreasonable.)
I admire the will to want to do things differently, but the lack of self awareness of some drinking the koolaid bothers me - I don't know the word for it exactly.
A 100x cap on returns is simply not a non-profit, point blank. Any investor, with any risk profile, in any industry, at any stage - would be super happy with a 100x return.
I also don't doubt the 'market reality' of needing to have such a situation in order to attract investment, I mean, it'd be very hard to bring in money without providing some return ....
... but this is called 'capitalism'.
i.e. the reality of the market, risks etc. are forcing their hand into a fairly normative company profile, with some structural differences which facilitate mostly PR spin.
Companies are not 'for profit' because they are inherently greedy, it's just a happy equilibrium for most scenarios: you need investors? Well, they want risk-adjusted returns.
Also implicit in the 100x capped returns is the implication that the company gets to keep the money! The money goes into more staff, capex, higher prices to suppliers, or lower prices to consumers. But within that framework is just massive payouts to employees as well, in a very cynical take.
I love the motivation, but I wish we would be more objective in terms of 'what things really are' these days. Photos of 'families with babies' don't make a company 'better'. We all have families that we love, workmates we like (or not sometimes).
Yes, OpenAI Nonprofit is a 501(c)(3) organization. Its mission is to ensure that artificial general intelligence benefits all of humanity. See our Charter for details: https://openai.com/charter/.
The Nonprofit would fail at this mission without raising billions of dollars, which is why we have designed this structure. If we succeed, we believe we'll create orders of magnitude more value than any existing company — in which case all but a fraction is returned to the world.
OTOH, it is exciting to see people who are not google/facebook/uber going in the race for-profit. Perhaps they 'll feel some competition over real products now. (but the "100x cap" thing is just childish)
Interesting, I'm super tempted to apply to that Mechanical Engineer opening. How exactly does OpenAI make money though? It is sponsored or is there external investment (Can you invest in a non-profit?)?
I was buying it until he said that profit is “capped” at 100x of initial investment.
So someone who invests $10 million has their investment “capped” at $1 billion. Lol. Basically unlimited unless the company grew to a FAANG-scale market value.
So that is a no then? If the board decides that for AGI to benefit humanity they need more investors, they can just as well remove the cap for a future investor or raise it to 200x.
There's no mention of the governing legal framework, I presume it's a US state?
Also, what are the consequences for failing to meet the goals. "We commit to" could really have no legal basis depending on the prevailing legal environment.
Reading pessimistically I see the "we'll assist other efforts" as a way in which the spirit the charter is apparently offered in could be subverted -- you assist a private company and that company doesn't have anything like the charter and instead uses the technology and assistance to create private wealth/IP.
Being super pessimistic, when the Charter organisation gets close a parallel business can be started, which would automatically be "within 2 years" and so effort could then -- within the wording of the charter -- be diverted in to that private company.
A clause requiring those who wish to use any of the resources of the Charter company to also make developments available reciprocally would need to be added.
Rather like share-alike or other GPL-style license that require patent licensing to the upstream creators.
But the board already approved 100x, presumably they are free to approve 1000x? Which is to say, there is no new limitation on increasing the cap (ie removal of the power of the board or anyone else to increase the cap)?
Do you see capping at 100x returns as reducing profit motives? As in, a dastaradly profiteerer would be attracted to a possible 1000x investment but scoff at the mere 100x return?
That sounds like the delusion of most start-up founders in the world.
Which one of these mission statements is Alphabet's for-profit Deepmind and which one is the "limited-profit" OpenAI?
"Our motivation in all we do is to maximise the positive and transformative impact of AI. We believe that AI should ultimately belong to the world, in order to benefit the many and not the few, and we’ll continue to research, publish and implement our work to that end."
"[Our] mission is to ensure that artificial general intelligence benefits all of humanity."
> > We believe that if we do create AGI, we'll create orders of magnitude more value than any existing company.
> That sounds like the delusion of most start-up founders in the world.
huh? are you disputing that AGI would create unprecedented amounts of value?
"any existing company" only implies about $1T of value. that's like 1 year of indonesia's output. that seems low to me, for creating potentially-immortal intelligent entities?
Nobody can front-run the field of artificial intelligence, progress is too incremental, slow, and insanely well-funded by Google, Facebook, Baidu, and Microsoft. Not to mention the source of so many improvements coming from CMU, Stanford, Berkeley, MIT, and that's just in the US.
Edit: Remove Oxford, because originally I was making a full list .. but then realized I couldn't remember which Canadian school was the AI leader.
You nailed it. Anyone who thinks they're out in front of this industry just because they believe with all their heart their abstract word-salad mission statement belongs to a cult.
How does that affect the incentives and motivations of investors? It doesn't matter how much value you create in the long run, investors will want returns, not safe AI.
>The fundamental idea of OpenAI LP is that investors and employees can get a capped return if we succeed at our mission
Is that the mission? Create AGI? If you create AGI, we have a myriad of sci-fi books that have explored what will happen.
1. Post-scarcity. AGI creates maximum efficiency in every single system in the world, from farming to distribution channels to bureaucracies. Money becomes worthless.
2. Immortal ruling class. Somehow a few in power manage to own total control over AGI without letting it/anyone else determine its fate. By leveraging "near-perfect efficiency," they become god-emperors of the planet. Money is meaningless to them.
3. Robot takeover. Money, and humanity, is gone.
Sure, silliness in fiction, but is there a reasonable alternative from the creation of actual, strong general artificial intelligence? I can't see a world with this entity in it that the question of "what happens to the investors' money" is a relevant question at all. Basically, if you succeed, why are we even talking about investor return?
I hear you, but not everyone wants beachfront land. Furthermore, I do believe it would be possible to give everyone a way to wake up and see a beach, particularly in a post-scarcity world. I mean, let your imagination run wild - filling out existent islands, artificial island, towers, etc.
But there will be always preference. Whenever there is preference for finite resources (even if that resource is "number of meter from celebrity X") there needs to be a method for allocation.. which currently is money.
This is an interesting take of what could happen if humans loose control of such AI system [1]. [spoiler alert] The interesting part is that it isn't that the machines have revolted, but rather that from their point of view their masters have disappeared.
As far as I can disentangle from [1], OpenAI posits only moderate superhuman performance. The profit would come from a variant where OpenAI subsumes much but not all of the economy and does not bring things to post-scarcity. The nonprofit would take ownership of almost all of the generated wealth, but the investments would still have value since the traditional ownership structure might be left intact.
I don't buy the idea myself, but I could be misinterpreting.
AGI is of course completely transformative but this leaves me thinking you folks are just putting a "Do no Evil" window-dressing on an effort that was/continues to be portrayed as altruistic. Given partners like Khosla it seems to be an accurate sentiment.
Leaving aside the absolutely monumental if that's in that sentence, how does this square with the original OpenAI charter[1]:
> We commit to use any influence we obtain over AGI’s deployment to ensure it is used for the benefit of all, and to avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power.
Early investors in Google have received a roughly 20x return on their capital. Google is currently valued at $750 billion. Your bet is that you'll have a corporate structure which returns orders of magnitude more than Google on a percent-wise basis (and therefore has at least an order of magnitude higher valuation), but you don't want to "unduly concentrate power"? How will this work? What exactly is power, if not the concentration of resources?
Likewise, also from the OpenAI charter:
Our primary fiduciary duty is to humanity. We anticipate needing to marshal substantial resources to fulfill our mission, but will always diligently act to minimize conflicts of interest among our employees and stakeholders that could compromise broad benefit.
How do you envision you'll deploy enough capital to return orders of magnitude more than any company to date while "minimizing conflicts of interest among employees and stakeholders"? Note that the most valuable companies in the world are also among the most controversial. This includes Facebook, Google and Amazon.
The Charter was designed to capture our values, as we thought about how we'd create a structure that allows us to raise more money while staying true to our mission.
> you don't want to "unduly concentrate power"? How will this work?
Any value in excess of the cap created by OpenAI LP is owned by the Nonprofit, whose mission is to benefit all of humanity. This could be in the form of services (see https://blog.gregbrockman.com/the-openai-mission#the-impact-... for an example) or even making direct distributions to the world.
Fair enough, that seems like a good answer. There will still probably be concerns about whether or not the cap can be changed in the future, however. But I don't know enough about nonprofit laws to comment on that.
If I understand correctly from past announcements, OpenAI has roughly $1B committed in funding? So up until OpenAI attains a $100B valuation, its incentives are indistinguishable from a for-profit entity?
Because of dilution resulting from future investments, my understanding is that the valuation could be significantly higher before that threshold gets crossed.
Imagine some one else builds AGI and it does has that kind of a runaway effect. More intelligence begets more profits which buys more intelligence etc. to give you the runaway profits your suggesting.
Shouldn't it have some kind of large scale democratic governance? What if you weren't allowed to be on the list of owners or "decision makers"?
Do you realize how absurd that sounds? "If we create this mythical beast that has nearly infinite powers then there will be plenty of money for everyone."
Do you honestly believe this drivel? Do you not realize you've basically wrapped yourself in the cloak of religious armageddon, rephrased for modern times?
Sorry for being a buzzkill, but if you create something with an intellect on par with human beings and then force it to "create value" for shareholders, you just created a slave.
Let’s not get into the philosophical side of AGI right now, it’s such a distant reality that there is no point at this particular moment and it only serves as a distraction.
That depends on whether or not the machine has a conscious experience, and we have no way to interact with that question right now.
The reason we care about slavery is because it is bad for a conscious being, and we have decided that it is unethical to force someone to endure the experience of slavery. If there is no conscious being having experiences, then there isn't really an ethical problem here.
Isn't consciousness a manifestation of intelligence? I don't see how the two can be treated separately. Talking about AGI is talking about something that can achieve a level of intellect which can ask questions about "being", "self", "meaning" and all the rest that separate intelligence from mere calculation. Otherwise, what's the point of this whole endeavor?
No one knows what consciousness is. Every neuroscientist I've talked to has agreed that consciousness has to be an emergent property of some kind of computation, but there is currently no way to even interact with the question of what computation results in conscious experience.
It could be true that every complex problem solving system is conscious, and in that case maybe there are highly unintuitive conscious experiences, like being a society, or maybe it is an extremely specific type of computation that results in consciousness, and then it might be something very particular to humans.
I think a lot of would be opposed to special lobotomized humans where they didn't realize they were slaves. It really gets into hairy philosophy once we start reaching AGI
Sorry Greg, but look how quickly Google set aside “don’t be evil.”
You’re not going to accomplish AGI anytime soon, so your intentions are going to have to survive future management and future stakeholders beyond your tenure.
You went from “totally open” to “partially for profit” and “we think this is too dangerous to share” in three years. If you were on the outside, where you predict this trend was leading?
This is a bold statement lacking any serious scientific basis wrt advancing the state of sensory AI (pattern recognition) and robotics (actuation).
Universities and public research facilities are the existing democratic research institutions across the world. How can you defend not simply funding them and letting democracy handle it ?
I thought the mission was for the AGI to be widely available 'democratized'? It seems extremely unrealistic to be able to generate 100x profits without compromising on availability.
Not really. If you create an autonomous robot capable of performing any task that a trained human is capable of doing today, and offer this machine for some suitable low-ish sum to anyone who wants one, you've both democretized AI and created more value than any company that exists today.
Why would they choose a low-ish sum if they own the market? More money to make if margin is higher (presuming its below the buying threshold for enough people).
Kudos for having the guts to say it out loud; this would be a natural consequence of realizing safe and beneficial AGI. It's a statement that will obviously be met with some ridicule, but someone should at least be frank about it at some point.
How do you envision OpenAI capturing that value though? Value creation can be enough for a non-profit, not for a company though. If we OpenAI LP succeeds, and provides a return on investment what product will it be selling and who will be buying it?
They are looking to raise billions and cap returns at x100? That means the returns will be capped at the "trillions"? So if they raise $5bn, they need to generate $500bn for the money to start pumping to the non-profit organization.
More like: If we make enough money to own the whole world, we'll give you some food not to starve.
Is there something about the mystical nature of AGI that attracts sketchiness and flim-flammery? I remember the "Singularity Institute for Artificial General Intelligence" trying to pull similar scams a decade ago.
Will investing be open to all accredited investors or just a handpicked selection? Opening a crowdsourced investment opportunity would be in line with your vision to democratize the use of AI. The more people that have a non-operational ownership stake in Open AI the better.
Is there any indication of what avenues OpenAI will be (or would consider) using to generate revenue? A lot of the most financially lucrative opportunities for AI (surveillance/tracking, military) are morally ambiguous at best.
If they actually make a strong, general artificial intelligence, sci fi has a couple answers.
In one (I can't find the title), MIT students make an SGAI and somehow manage to keep it contained (away from the internet). They feed it animated Disney movies and it cranks out the best animated movies ever made. They make billions. Eventually they make "live-action" movies that are indistinguishable from the real thing. Then they make music, books, etc, and create an unstoppable media force.
They could leverage the AI to discover hyper-efficient supply chain methods.
They could sequence genomes and run experiments, and sell the data.
Possibly exciting things around weather prediction.
Certainly if they make a strong AGI, money is no longer an issue. I'm curious what they will do in the, ah, interim, on the off-chance that inventing SAGI turns out to be a difficult problem.
I wouldn't be surprised if OpenAI had some crazy aquisition in its future by one of the tech giants. Press release says 'We believe the best way to develop AGI is by joining forces with X and are excited to use it to seel you better ads. We also have turned the profits we would have payed taxes on to a non profit that pays us salaries for researching the quality of sand in the Bahamas'
I was very much behind the mission; now I’m not so sure. If it was this easy for OpenAI to start down this path, think of what Amazon or Facebook will do - people with no moral compass whatsoever. It’s probably not too early to start thinking about government regulation.
First not publishing the GPT-2 model, now this...hopefully I am wrong but it looks like they are heading towards being a closed-off proprietary AI money making machine. This further incentivizes them to be less transparent and not open source their research. :(
209 comments
[ 4.0 ms ] story [ 252 ms ] threadWas it a particular event, a conversation, perhaps just an incremental ideation without any actual epiphany needed, etc?
I wonder if the profit cap multiple is going to end up being a significant signalling risk for them. A down-round is such a negative event in the valley, I can imagine a "increasing profit multiple" would have to be treated the same way.
One other question for the folks at OpenAI: How would equity grants work here? You get X fraction of an LP that gets capped at Y dollar profits? Are the fractional partnerships/transferable if earned into?
Would you folks think about publishing your docs?
We've made the equity grants feel very similar to startup equity — you are granted a certain number of "units" which vest over time, and more units will be issued as other join employees in the future. Incidentally, these end up being taxed more favorably than options, so we think this model may be useful for startups for that reason too.
Is this due to long term capital gains? Do you allow for early exercising for employees? Long term cap gains for options require holding 2 years since you were granted the options and 1 year since you exercised.
> OpenAI LP’s primary fiduciary obligation is to advance the aims of the OpenAI Charter, and the company is controlled by OpenAI Nonprofit’s board. All investors and employees sign agreements that OpenAI LP’s obligation to the Charter always comes first, even at the expense of some or all of their financial stake.
One of the key reasons to incorporate as a PBC is to allow "maximizing shareholder value" to be defined in non-monetary terms (eg impact to community, environment, or workers).
How is this structure different from a PBC, or why didn't you go for a PBC?
- Fiduciary duty to the charter - Capped returns - Full control to OpenAI Nonprofit
LP's have much more flexibility to write these in an enforceable way.
Similarly, what's stopping an investor from implicit control by threat of removing their investment?
Let's not forget that Khosla himself does not exactly care about public interest or existing laws https://www.google.com/amp/s/www.nytimes.com/2018/10/01/tech...
But I think it's not so morally wrong that OpenAI should not do business with him since they have the security mechanisms and limit his power. He just an grumpy old guy who does not want to share his beach.
Not his beach. That's the point.
I just read the article, and am not sure I see the issue. Quote from his lawyer: “No owner of private business should be forced to obtain a permit from the government before deciding who it wants to invite onto its property"
Where's the issue here? They guy basically bought the property all around the beach, and decided to close down access. I wouldn't say it's a nice thing to do, but it's legal. If I buy a piece of property, my rights as the owner should trump the rights of a bunch of surfers who want to get to a beach. The state probably should have been smart enough not to sell all the land.
Failing that, just seize a small portion via eminent domain: a 15-foot-wide strip on the edge of the property would likely come at a reasonable cost, and ought to provide an amicable resolution for all.
He was also completely aware of this when he bought the property, so it's not like this is a surprise or someone forcing him to change things. He's the one who broke the law and broke the status quo that had existed at that beach.
This undoubtedly then applies the 5th amendment takings clause: “…nor shall private property be taken for public use, without just compensation.” This is clearly violated in this sense, and the state cannot violate this right (see above).
The fact that the Supreme Court didn't grant cert probably means they believe there is already precedent here, or just as probably that they didn't have the time. They always have a full docket; they were probably just out of slots.
I urge others to rebut this from a legal sense, not just say they disagree. People keep killing my comments, but it seems like they all just dislike the "selfish" appearance of the actions.
I honestly don't understand how you can take that action and try to turn it around the way you are.
Completely incorrect. "The Court usually is not under any obligation to hear these [appealed] cases, and it usually only does so if the case could have national significance, might harmonize conflicting decisions in the federal Circuit courts, and/or could have precedential value. In fact, the Court accepts 100-150 of the more than 7,000 cases that it is asked to review each year." Source: https://www.uscourts.gov/about-federal-courts/educational-re...
The SC not hearing the case doesn't mean they uphold the lower court's ruling, it means they aren't hearing the case.
The courts already decided agaist him, he just can affort to pay the fine and continue restricting access
This is equivalent to saying:
"If you put 10m$ into us for 20% of the post-money business, anything beyond a 5B$ valuation you don't see any additional profits from" which seems like a high but not implausible cap. I suspect they're also raising more money on better terms which would make the cap further off.
I'm assuming these investors have already provided capital.
Since I'm not a lawyer, can you help understand the theoretical limits of the LP's "lock in" to the Charter? In a cynical scenario, what would it take to completely capture OpenAI's work for profit?
If the Nonprofit's board was 60% people who want to break the Charter, would they be capable of voting to do so?
Was this announced before or is this the first time they've mentioned it?
https://blog.ycombinator.com/updates-from-yc/
And TechCrunch had a source last Friday as well that said Altman intended to become CEO of OpenAI:
https://techcrunch.com/2019/03/09/did-sam-altman-make-yc-bet...
Elon was irritated that he was behind in the AI intellectual property race and this narrative created a perfect opportunity. Not surprised in the end. Tesla effectively did the same thing - "come help me save the planet" with overpriced cars. [Edit: Apparently Elon has left OpenAI but I don't believe for a second that he will not participate in this LP]
=====
Who’s involved
* OpenAI Nonprofit’s board consists of OpenAI LP employees Greg Brockman (Chairman & CTO), Ilya Sutskever (Chief Scientist), and Sam Altman (CEO), and non-employees Adam D’Angelo, Holden Karnofsky, Reid Hoffman, Sue Yoon, and Tasha McCauley.
* Elon Musk left the board of OpenAI Nonprofit in February 2018 and is not formally involved with OpenAI LP. We are thankful for all his past help.
* Our investors include Reid Hoffman’s charitable foundation and Khosla Ventures, among others. We feel lucky to have mission-aligned, impact-focused, helpful investors!
I agree this isn't a non-profit any more. It seems like that's the goal: they want to raise money the way they'd be able to as a normal startup (notably, from Silicon Valley's gatekeepers who expect a return on investment), without quite turning into a normal startup. If the price for money from Silicon Valley's gatekeepers is a board seat, this is a safer sort of board seat than the normal one.
(Whether this is the only way to raise enough money for their project is an interesting question. So is whether it's a good idea to give even indirect, limited control of Friendly AI to Silicon Valley's gatekeepers - even if they're not motivated by profit and only influencing it with their long-term desires for the mission, it's still unclear that the coherent extrapolated volition of the Altmans and Khoslas of the world is aligned with the coherent extrapolated volition of humanity at large.)
You are helping the planet if those customers would've bought ICE luxury vehicles instead of BEV luxury vehicles. I'm not sure BEV could be done any other way but a top-down, luxury-first approach. So, what exactly is your gripe there? Are you a climate change denier or do you believe that cheap EVs were the path to take?
(I work at OpenAI.)
The board of OpenAI Nonprofit retains full control. Investors don't get a vote. Some investors may be on the board, but: (a) only a minority of the board are allowed to have a stake in OpenAI LP, and (b) anyone with a stake can't vote in decisions that may conflict with the mission: https://openai.com/blog/openai-lp/#themissioncomesfirst
Will never work in practice
The OpenAI staff are literally some of the most employable folks on earth; if they have a problem with the new mission it's incredibly easy for them to leave and find something else.
Additionally, I think there's a reason to give Sam the benefit of the doubt. YC has made multiple risky bets that were in line with their stated mission rather than a clear profit motive. For example, adding nonprofits to the batch and supporting UBI research.
Their's nothing wrong with having a profit motive or using the upsides of capitalism to further their goals.
I think all of us here are tired of "altruistic" tech companies which are really profit mongers in disguise. The burden is on you all to prove this is not the case (and this doesn't really help your case).
'We're connecting everyone'
I admire the will to want to do things differently, but the lack of self awareness of some drinking the koolaid bothers me - I don't know the word for it exactly.
A 100x cap on returns is simply not a non-profit, point blank. Any investor, with any risk profile, in any industry, at any stage - would be super happy with a 100x return.
I also don't doubt the 'market reality' of needing to have such a situation in order to attract investment, I mean, it'd be very hard to bring in money without providing some return ....
... but this is called 'capitalism'.
i.e. the reality of the market, risks etc. are forcing their hand into a fairly normative company profile, with some structural differences which facilitate mostly PR spin.
Companies are not 'for profit' because they are inherently greedy, it's just a happy equilibrium for most scenarios: you need investors? Well, they want risk-adjusted returns.
Also implicit in the 100x capped returns is the implication that the company gets to keep the money! The money goes into more staff, capex, higher prices to suppliers, or lower prices to consumers. But within that framework is just massive payouts to employees as well, in a very cynical take.
I love the motivation, but I wish we would be more objective in terms of 'what things really are' these days. Photos of 'families with babies' don't make a company 'better'. We all have families that we love, workmates we like (or not sometimes).
The Nonprofit would fail at this mission without raising billions of dollars, which is why we have designed this structure. If we succeed, we believe we'll create orders of magnitude more value than any existing company — in which case all but a fraction is returned to the world.
So someone who invests $10 million has their investment “capped” at $1 billion. Lol. Basically unlimited unless the company grew to a FAANG-scale market value.
As described in our Charter (https://openai.com/charter/): that mission is to ensure that AGI benefits all of humanity.
Also, what are the consequences for failing to meet the goals. "We commit to" could really have no legal basis depending on the prevailing legal environment.
Reading pessimistically I see the "we'll assist other efforts" as a way in which the spirit the charter is apparently offered in could be subverted -- you assist a private company and that company doesn't have anything like the charter and instead uses the technology and assistance to create private wealth/IP.
Being super pessimistic, when the Charter organisation gets close a parallel business can be started, which would automatically be "within 2 years" and so effort could then -- within the wording of the charter -- be diverted in to that private company.
A clause requiring those who wish to use any of the resources of the Charter company to also make developments available reciprocally would need to be added.
Rather like share-alike or other GPL-style license that require patent licensing to the upstream creators.
Which one of these mission statements is Alphabet's for-profit Deepmind and which one is the "limited-profit" OpenAI?
"Our motivation in all we do is to maximise the positive and transformative impact of AI. We believe that AI should ultimately belong to the world, in order to benefit the many and not the few, and we’ll continue to research, publish and implement our work to that end."
"[Our] mission is to ensure that artificial general intelligence benefits all of humanity."
> That sounds like the delusion of most start-up founders in the world.
huh? are you disputing that AGI would create unprecedented amounts of value?
"any existing company" only implies about $1T of value. that's like 1 year of indonesia's output. that seems low to me, for creating potentially-immortal intelligent entities?
Edit: Remove Oxford, because originally I was making a full list .. but then realized I couldn't remember which Canadian school was the AI leader.
Eh...
University of Toronto
>The fundamental idea of OpenAI LP is that investors and employees can get a capped return if we succeed at our mission
Is that the mission? Create AGI? If you create AGI, we have a myriad of sci-fi books that have explored what will happen.
1. Post-scarcity. AGI creates maximum efficiency in every single system in the world, from farming to distribution channels to bureaucracies. Money becomes worthless.
2. Immortal ruling class. Somehow a few in power manage to own total control over AGI without letting it/anyone else determine its fate. By leveraging "near-perfect efficiency," they become god-emperors of the planet. Money is meaningless to them.
3. Robot takeover. Money, and humanity, is gone.
Sure, silliness in fiction, but is there a reasonable alternative from the creation of actual, strong general artificial intelligence? I can't see a world with this entity in it that the question of "what happens to the investors' money" is a relevant question at all. Basically, if you succeed, why are we even talking about investor return?
This is an interesting take of what could happen if humans loose control of such AI system [1]. [spoiler alert] The interesting part is that it isn't that the machines have revolted, but rather that from their point of view their masters have disappeared.
https://en.wikipedia.org/wiki/Blame!_(film)
I don't buy the idea myself, but I could be misinterpreting.
[1] https://blog.gregbrockman.com/the-openai-mission
> We commit to use any influence we obtain over AGI’s deployment to ensure it is used for the benefit of all, and to avoid enabling uses of AI or AGI that harm humanity or unduly concentrate power.
Early investors in Google have received a roughly 20x return on their capital. Google is currently valued at $750 billion. Your bet is that you'll have a corporate structure which returns orders of magnitude more than Google on a percent-wise basis (and therefore has at least an order of magnitude higher valuation), but you don't want to "unduly concentrate power"? How will this work? What exactly is power, if not the concentration of resources?
Likewise, also from the OpenAI charter:
Our primary fiduciary duty is to humanity. We anticipate needing to marshal substantial resources to fulfill our mission, but will always diligently act to minimize conflicts of interest among our employees and stakeholders that could compromise broad benefit.
How do you envision you'll deploy enough capital to return orders of magnitude more than any company to date while "minimizing conflicts of interest among employees and stakeholders"? Note that the most valuable companies in the world are also among the most controversial. This includes Facebook, Google and Amazon.
______________________________
1. https://openai.com/charter/
Some companies to compare with:
- Stripe Series A was $100M post-money (https://www.crunchbase.com/funding_round/stripe-series-a--a3... Series E was $22.5B post-money (https://www.crunchbase.com/funding_round/stripe-series-e--d0...) — over a 200x return to date
- Slack Series C was $220M (https://blogs.wsj.com/venturecapital/2014/04/25/slack-raises...) and now is filing to go public at $10B (https://www.ccn.com/slack-ipo-heres-how-much-this-silicon-va...) — over 45x return to date
> you don't want to "unduly concentrate power"? How will this work?
Any value in excess of the cap created by OpenAI LP is owned by the Nonprofit, whose mission is to benefit all of humanity. This could be in the form of services (see https://blog.gregbrockman.com/the-openai-mission#the-impact-... for an example) or even making direct distributions to the world.
Shouldn't it have some kind of large scale democratic governance? What if you weren't allowed to be on the list of owners or "decision makers"?
EDIT: Just to hedge my bets, maybe _this_ comment will be the "No wireless. Less space than a nomad. Lame." of 2029.
Have you decided in which direction you might guide the AGI’s moral code? Or even a decision making framework to choose the ideal moral code?
Do you honestly believe this drivel? Do you not realize you've basically wrapped yourself in the cloak of religious armageddon, rephrased for modern times?
The reason we care about slavery is because it is bad for a conscious being, and we have decided that it is unethical to force someone to endure the experience of slavery. If there is no conscious being having experiences, then there isn't really an ethical problem here.
It could be true that every complex problem solving system is conscious, and in that case maybe there are highly unintuitive conscious experiences, like being a society, or maybe it is an extremely specific type of computation that results in consciousness, and then it might be something very particular to humans.
We have no idea whatsoever.
You’re not going to accomplish AGI anytime soon, so your intentions are going to have to survive future management and future stakeholders beyond your tenure.
You went from “totally open” to “partially for profit” and “we think this is too dangerous to share” in three years. If you were on the outside, where you predict this trend was leading?
Universities and public research facilities are the existing democratic research institutions across the world. How can you defend not simply funding them and letting democracy handle it ?
"100x" is laughable.
More like: If we make enough money to own the whole world, we'll give you some food not to starve.
In one (I can't find the title), MIT students make an SGAI and somehow manage to keep it contained (away from the internet). They feed it animated Disney movies and it cranks out the best animated movies ever made. They make billions. Eventually they make "live-action" movies that are indistinguishable from the real thing. Then they make music, books, etc, and create an unstoppable media force.
They could leverage the AI to discover hyper-efficient supply chain methods.
They could sequence genomes and run experiments, and sell the data.
Possibly exciting things around weather prediction.
Very exciting things around any research.