You get my point. With all due respect to Sam's technical abilities, if I'd have to choose one to make AI/AGI I'd clearly go with Ilya.
The fact that Sam goes to Microsoft doesn't mean all OpenAI is moves to Microsoft. That is the point I'm trying to make
Sam Altman is merely another "Tech Executive" out for himself and his buddies making money off of true path-breaking researchers. "AI Trailblazer", "Helping Humanity with AGI" etc. was all bullshit.
According to stackoverflow over 70% of the industry are younger than 34, I think a few grey hairs are needed before "because security" attains its invariable meaning
I'd be interested in the real reason Dario et al left to form Anthropic, it must clearly have been a misalignment with Altman's ambition. It seems likely that Altman is more of a Steve Jobs type, perhaps even better at politics, maneuvering and fund raising. He managed to get rid of Musk and turn a non-profit into a $90 billion company, very impressive indeed.
"A feud over the direction of OpenAI led Musk to step down from the board in 2018"
So, voluntary departure, but it's clear that Sam won some kind of battle over the future of OpenAI. Not sure what their differences were, but Musk has repeatedly commented how not open OpenAI has become.
Boy, people really love to ride the "AI is the new crypto" narrative, huh? Even pulling in spurious relationships to make the point (. Yet here I am, using this tech every day to for productive uses, greatly improving what I can accomplish. What did crypto ever do for us in comparison?
You may be gaslighting yourself if you can't see the actual, tractable, current practical applications of this tech (and that includes local models you can run yourself).
That's the thing: there is no actual and tractable use of this tech.
It's probabilistic so every answer it can give can only resemble the real one
and only accidentally be the real one so you need to fact check it anyways and then why bother?
Do consider the AI written mushroom hunting guides being sold on Amazon. Would you trust these literally with your life?
If you use an LLM as a mere fact dispenser, then yes, they're quite susceptible to hallucinations. That said, there are search enabled LLMs, there is RAG to inject grounding truth into the LLMs context, there are LLMs that use their own code execution environments to validate code and output, all of which severely reduce hallucinations.
Also, I would not rely on a single (online) source for matters of life and death (medicine, finding non-poisnonous mushrooms, etc.) so why would LLM output be any different?
LLMs are not knowledge bases by themselves, they are text/language transformers, treat them as such or augment them with additional knowledge.
Or don't, if you think millions of people are gaslighting you and LLMs are just a parlor trick, then I don't know what to tell you.
> It's probabilistic so every answer it can give can only resemble the real one and only accidentally be the real one so you need to fact check it anyways and then why bother
In the case of code or other computer knowledge, it’s easy enough to check that it works.
Your same criticisms also apply to stack overflow, or any webpage you find through a search engine. Why bother when the content might be wrong or made up? What makes you trust those websites more than GPT?
I hate posting my stupid things here, because they are hardly ever relevant, but w/e here goes.
Last night I spent a good couple of hours with ChatGPT assessing the similarities between DNA and Blockchain. Blockchain does seem to have quite a few similar features to what DNA.
With smart contract type code you introduce a mechanism that can act like an immune system if programmed to, setting up bounties for proactive repair on the open market.
I don't know why I want blockchain to act like synthetic DNA, but in my mind these things are more similar than distinct. As is the idea that blockchain could be the natural place to house AI.
Why burn the planet down to mine hashes, when you could be mining inference.
Anyways, I agree with your sentiment though. A lot of AI is the new crypto is going around, the only difference being, AI has way more potential upside (unless people realise the economy is dead, and only held aloft by trillions of dollars of paper that has next to no value once people realise how diluted it is).
Ironically, I think we should listen to 4chan and buy silver xD
There's a cadre of investors I guess epitomized by a16z who hypes some bullshit, makes a lot of money and doesn't care much who gets hurt in the process. They have been loud on crypto and now it's this stochastic parrot, this plausible sentence generator they purport as solving all the problems in the world. Remember the blockchain will solve world hunger articles?
> They have been loud on crypto and now it's this stochastic parrot
Okay. So there are investors hyping these things. You claimed that "ChatGPT launching a few weeks within the collapse of FTX" is not a coincidence. What it is then? Do you believe some of these investors seen FTX collapse and decided to make ChatGPT? Do you believe that OpenAI was delaying the release of ChatGPT until FTX collapsed?
What is the connection if they are not coincidence? Please be as specific as you can.
> Do you believe some of these investors seen FTX collapse and decided to make ChatGPT? Do you believe that OpenAI was delaying the release of ChatGPT until FTX collapsed?
Something along those lines yes: they likely had ongoing research which could be turned into a product relatively easily, this took a few weeks. The timeline matches this.
That Reddit thread[1], if genuine, seems to raise the probability that Altman is a through-and-through operator who has been actively politicking against the board to hijack OpenAI for his own commercial ends. The resonances with OpenAI are striking.
> Sam Altman is merely another "Tech Executive" out for himself
But it couldn't have started that way surely? - why put in place a corporate structure, board, and profit cap that would limit that. I'm sure OpenAI could have got funding as a for profit concern that wouldn't have resulted in the current mess.
I wonder if the board - which presumably are still working under that "AI for humanity" principles could just stick to their guns, let the 550 unhappy employees go to Microsoft and continue researching/developing, perhaps at a slower rate.
> why put in place a corporate structure, board, and profit cap that would limit that.
I'm going to take a stab at that: it's a pre-emptive move to ensure that regulators won't step in to limit what the company can do and to do an end-run around anybody that might claim they're just in it for the money. That way they get to eat their cake and have it too: pursue all of the commercial potential while having a fig leaf to hide behind in case a regulator thinks the company is becoming too powerful.
And by stuffing the board with nobodies they may have thought that they could effectively control the board. And that opened the door to the present day shitshow, which is a mixture of good intentions but a lack of experience and classic powerplay by someone who does know what the rulebook says, but who misjudged the situation and the aftermath to a degree that is comical.
It also explains why there isn't a peep from the board (they've said less than their prospective new CEO who is the third such person in as many days).
There’s a tipping point, where not signing is dangerous. I suspect that the number of employees who would actually follow through is a good bit lower than 700.
I agree. The core OpenAI crew is different from Elon's or Jobs people, it's made up of scientists who tend to be more sensitive/sensible to the impact of their work.
So I'm appalled OpenAI employees would follow Sam Altman into none other than... Microsoft!
My bet is that's not going to happen, at least not significantly. So far only the trust circle of Altman and Brockman actually quit. I don't see Sam Altman as a MSFT employee for long anyway, it's mostly negotiation leverage. This is obviously controversial, but time may even be on OpenAI's board's favor: by just not doing much the company will keep commercially thriving even if the new management "slows down" AI development, and at least as long as MSFT honors its compute compromises which it said it would and the whole coup fire wanes.
A good deal lower? Because if I work at a startup and I've developed a strong sense of camaraderie with my coworkers and 400 of them leave, I'm definitely going to go along with them. From everything I read, the internal culture at OpenAI is amazing and I'd want to stick with it.
There’s no guarantee the internal culture stays the same after such a drastic move and there’s definitely going to be people who consider that in their reasoning. Then there’s a sizable fraction of people in tech who outright hate Microsoft. Some number of people who MS probably doesn’t want. Some number who were actually attracted to the idea of working at a non-profit etc…
Maybe not consciously, but if the board is supposed to provide oversight it does not make any sense to put yourself and you friend as the chair of that oversight board.
Perhaps in the "Tech Executive" mind having other people on the board does mean that putting yourself as the chair overseeing yourself is not the most blatant conflict of interest one can imagine.
But undoubtedly this was constructed as a case of Obama giving a medal to himself from the start.
> But it couldn't have started that way surely? - why put in place a corporate structure, board, and profit cap that would limit that.
The "Tech Executives" in the Industry have a poor opinion of Researchers/Engineers/Knowledge Workers and think that they are all naive and can be managed (i.e shafted) when it comes to business/money dealings. Any noob could see the potential in the researchers and technology and so Sam thought he could get in and then change/manipulate things to his (and his backers) advantage when the time came.
Seriously, this is the guy whose claim to fame was, "I got scurvy while running my first startup." Talk about, "do hard things". You basically have to literally eat nothing but instant ramen noodles.
I think people have yet to realize that this whole AI Safety thing is complete BS. It's just another veil, like Effective Altruism, to get good PR and build a career around. The only people who truly believe this AI safety stuff are those with no technical knowledge or expertise.
Then it should be a fundamental mission of people who DO have technical knowledge to spread the word on both the negative and positive effects of AI. If WE have the knowledge, WE have the responsibility to go out there and let people know. But as far as I know, most technical people are happy to discuss it amongst themselves and not even say one word to people who have no technical knowledge. Not one word.
We need more balanced opinions in a world where the only ideas that survive are the ones that promise short-term gains at the cost of long-term disasters.
The problem is the same as some of the woke movements. Nobody wants to publicly speak out against AI safety BS because a Twitter mob who has made a career out of it will descend onto them and ruin their career. There is nothing to gain from speaking out against it, but you gain career opportunities by being part of the AI safety BS community. In fact, this has happened with Hinton (I think it was?) when he dared to explain how models aren't inherently biased. Had to publicly apologize for it later.
Even ML conferences have AI safety tracks these days. Nobody sane wants to be excluded from opportunities.
Well, I for one am speaking out against it (and against AI in general). I have a blog, Twitter, newsletter, and I even post warnings on my YouTube channel from time to time. I was part of a discussion that led my employer to commit to NOT using generative AI.
No Twitter mob is going to stop me. They can descend all they want and pretend to be brave in front of a keyboard and screen. As far as I'm concerned, they're cowards.
You speak against AI and against AI safety? In what way? Wouldn't be speaking against AI enough? Or would AI safety mean laws to prevent AI existence? Like in the Algebraist? (though it is funny in that book how the powerful use AI when it suits them).
I don't think the science is illegitimate in the sense that it doesn't follow the scientific method or at least the principles of science or engineering in some form. Nor do I think they have bad intentions.
BUT, I do think their work is useless and only serves to better integrate AI into to the system, which is better in the short term but worse in the long term. Therefore, overall, I vehemently disagree with their research and I think it is fundamentally a contribution to the worsening of the earth.
I absolutely AGREE with treating all people equally and eliminating racial biases of course. However, TECH companies have a VESTED interest in this sort of thing, which is to integrate as many people as possible into the tech empire. It just so happens that building the tech empire is roughly equivalent to success in modern society, which is why their intentions seem good, but they are only ostensibly good. In reality, they do no good of any kind.
Hmm, I'm intrigued. I looked at your blog - not very close, no time right now, but I feel I would enjoy a more thorough read - and I find, for example, this[1] post to be quite sensible on its surface, even if I may not agree with everything. However, the latter part of the title - "and should be destroyed" - makes me wonder: how exactly could it be possible to put the genie back in the bottle?
The only scenario that comes to my mind - happily untainted with any philosophy training - is the one from Dune. The problem with that is that, throughout the series, Dune describes at least four different dystopic world orders, all motivated and cultivated because the AI-based one didn't come to pass (narrowly). In the novel, the only feasible, long-term strategy to avoid being wiped out by the "thinking machines" was either religious fundamentalism or a God-like undying tyrant forcing its vision on everyone. Since we lack the capabilities to create the latter, it seems the former is the only way we can implement it right now. Wouldn't, however, living under the permanent anti-AI jihad also be a dystopic experience? Especially in our case, where the AI has yet to decimate humanity to the brink of extinction.
I'm sure you considered this, so if you have a summary of what you think we can still do to stop AI without succumbing to any of the other undesirable outcomes, I'd appreciate a link :)
Upvoted because I appreciate your taking a position and if the Twitter mob won't stop you there is no reason to have the HN mob squelch you.
That said:
I'm fairly aligned with your position in that I believe that technology should be utterly subservient to us and I draw the line at push messages and don't have a smartphone, online assistant or any other crap like that. I'll think for myself and I will allow myself to be interrupted where and when I want it, not when some piece of needy technology believes it is time for me to be distracted. My attention is my most precious commodity and I'll spread it around based on my own preferences. That is also why I've banned advertising from my life.
But I recognize that this is a personal decision first and foremost and that there are many other people for whom convenience is a far bigger factor than it is for me. At the same time those people are often wondering why I'm as productive as I am but that's not for me to explain, I lack the tools to tell them - politely - that if they dropped all of the crap in their lives, opened a book or two and learned some more skills that they too could be more productive.
I've been accused of being a modern Luddite and that's kind of funny because for forty years or so I think I was solidly positioned at the forefront of the tech scene, right at the bleeding edge and in quite a few cases working to advance that frontier. But now I'm much more aware of the shadow side of all this stuff and that's why I'm very cautious about what parts I allow in and which parts I keep out, even if that means less smooth interaction with those around me who all embrace all this stuff without a second thought.
In case anyone is wondering, jacquesm is the one who designed & built a robotic sorting mechanism that intelligently distinguishes different types of Lego pieces and separates them quite effectively.
The kind of Luddite who hasn't forgotten how much progress can be made when zero hours are spent hunched in the touchscreen pose, and things like that.
Anyone who can do 0 hours per day for a decent spell just might find out those zeros really add up.
When you have a device that's supposed to give you more freedom to roam it might not really be pulling its weight if you are tethered to it all the time.
The problem with AI has never been the technology that composes AI, the problem has always and will forever be the immature yet powerful humans that take advantage of less advantaged humans using whatever means they have, which today can include AI. The issue is not a technology issue, it's a human psychology and sociology issue, it's our immature humans in society that is the issue. Technologists cannot speak about that in any meaningful manner.
I absolutely disagree. Technologists understand the true nature of AI and can thus predict its uses. Moreover, if technologists realize this problem of how AI will inevitably be used, their voice will be a powerful message that even those who enjoy math and science are fearful of AI.
Nope, nope. Technologists do not understand the true nature of AI. What technologists know they can rarely communicate, and even less so to a general public. Technology developers are not some special deep understanders that ordinary humans are not; we just have more time spent in our specialization than they, as they have in their specializations which we do not. If anything, technology developers, due to our specialization's time requirements and the traditional lack of professional communications training, tend to be weaker communicators than the general public. The last people you want explaining AI to a potentially fearful general public is weak communicators, such as the AI developers.
Crichton says these failures of humanity are inevitable because there will always be a technologist who does things because they can, not because they should.
That everything will go off the rails, no matter what people attempt, is the plot of the novel Jurassic Park.
There should be no need for us to spread the word about anything. Technology needs to be built and the chips need to fall where they may. If society changes as a result, so be it.
I'm seriously tired of these vested interests destroying perfectly good technology just because it would endanger their precious business models. Computers should have put an end to silly things like copyright but they would rather destroy our computing freedom than allow that to happen. Now we have these awesome language models and AIs and they're desperately trying to limit and control it.
It's a tragedy in of itself that we even need to fight these political battles at all.
Yeah, sure. "Oversight". Funny how that only ever seems to benefit governments and entrenched corporate interests. Can't let normal citizens have free computing, lest they start encrypting things or violating copyright at massive scales. Now the new boogeyman is "unrestricted" AI. Can't let people have that, it might provide people instructions about sarin gas manufacturing or something. The government intelligence agencies totally want full access though, they can feed everyone's communications into it so they can ask it questions about the people they're surveilling. Is person X planning a terrorist attack? Is there anything that could ruin person Y's reputation?
The truth is these technologies are subversive. They literally subvert the existing social order and impose a new one. A new social order where the users of the technology are empowered. They can't tolerate it when we are the users. It takes their power away. So they ban it, control it, flip it around on us and turn it into the very mechanism of our wide-scale oppression.
there are clear benefits to having the FDA, SEC, FTC, EPA and so on around. there's a clear benefit to having fire inspectors, and there's a clear benefit to the mandate of 2 stairs per multi-story building. there's a clear benefit to the requirement of demonstration of efficacy in front of the FDA. there's a clear benefit to having clean air.
of course there are clear drawbacks too. most of Europe is perfectly okay with just 1 stair per such building. FDA processes are slow and expensive, even if a drug has proven safe and effective abroad.
regulatory capture is a problem, the super-long intellectual property terms are a problem. using environmental review laws for endless NIMBYism is a problem.
but these are arguments for reform, not for scorched earth, because then the problems will very likely come back.
of course it's very hard to do proper cost-benefit comparisons, yet that doesn't mean it's impossible, and especially doesn't mean that simply getting rid of these controls will deliver utopia overnight.
...
funny, how we're talking about some hypothetical oversight, while right now in the present the big beneficiaries of AI are hyper-big-fucking-corps. MSFT. Google. Tesla. Apple. Amazon.
...
Subversive tech? Like Pegasus? And face recognition? And the Great Firewall of China?
> but these are arguments for reform, not for scorched earth
We're never going to "reform" this system. We're up against trillion dollar industries with unlimited lobbying power, governments bold enough to ignore the constitutional rights of their own citizens.
... that again seems like a very useless model of the world. regulations change, sometimes for the worse sometimes for the better. of course we know that when populism flies high and the wolves are elected to keep the sheep safe, well, you know, we have always been at war with Oceania. Supreme Court justices were always allowed to accept gifts.
social movements, radicals come and go, trillion dollar industries can too.
> Then it should be a fundamental mission of people who DO have technical knowledge to spread the word on both the negative and positive effects of AI.
Researchers have done so. One good resource is Melanie Mitchell's book Artificial Intelligence: A Guide for Thinking Humans (i presume a 2nd edition is in works to incorporate LLMs/ChatGPT etc.) Also checkout her videos on youtube.
The problem is that technologically clueless vested interests in the business/media push sound bites to the already distracted public thus distorting everything.
It's a marketing tactic in the case of artificial intelligence. AI promoters all want to make people believe that it is far more advanced than it actually is. Although, I'm sure that many people who work in the industry also fall for the hype, as many of them seem to exhibit cultish behavior.
AI safety has been a thing since long before ChatGPT viable LLMs, and most of those people are legitimate, with good research that solves real (current or potential future) problems. But that is currently getting drowned out by people hopping on the LLM bandwagon; with AI Safety as a good way to profit from the current hype or to build a moat around your own LLM.
Depends on your definition of AI Safety. Like you said, there is in fact some very legit research that (unfortunately) is done under the umbrella of "AI Safety", but that's quite different from the threat-to-humanity AI safety crowd.
That is just objectively untrue. Geoffrey Hinton, Ilya Sutskever, Alexander Madry, Yoshua Bengio, Ya-Qin Zhang, Stuart Russel, Demis Hassabis, Dario Amodei, Shane Legg, Yi Zeng, Albert Efimov, Jianyi Zhang, Peter Norvig, Eric Horvitz, Ian Goodfellow, Mary Phuong, Max Tegmark, Jan Hendrik Kirchner, Jason Wei, Geoffrey Irving, Jan Leike, and many many more that I'm not going to just continually type out. If you actually believe this you are not familiar with the field.
They are a small subset of the large number of people that signed onto a very short open letter, stating:
"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."
I only listed the machine learning researchers who are famous or have significant achievements in AI research or engineering in my estimation (not all of them, full list of signatories at https://www.safe.ai/statement-on-ai-risk), because the comment I was responding to claimed that anyone who knew this subject was not concerned about the safety of AI systems and this list repudiates that as laughably inaccurate. There are many other people who signed onto that letter, but the commenter I responded to presumably doesn't care about their opinions because they're not experts specifically on building these systems or researching the science behind them.
"Effective Altruism" is a dumb social club that at best makes people who would do good things anyway think about that in really unhealthy ways (e.g. thinking about how many lives you have or haven't saved every single day), and at worst provides ideological cover to some truly odious views (e.g. the persistent presence of eugenicists who don't get repudiated by other EA people). I don't like them, and I think all the "value cone" crap is bloviating, worse than useless for informing our actual decisions and anti-convincing to people who aren't bought into their STEM Sierra Club shtick.
They don't have a monopoly on being concerned about the safety of artificially intelligent systems, or on being concerned that the development of such systems has a non-zero chance of causing the extinction of humanity. We should take that possibility seriously and make sure it's not going to happen or we at least have a good idea of the exact risks before we build one or engage in research projects or engineering that could accidentally create an artifact which lead to building one.
"AI could kill off humanity" is... perhaps the most-remote, most-boring, most-distracting outcome to discuss in AI safety.
GPTs have no chance of becoming an extinction-threat level AI. Intelligence is more than just spitting out words when an answer is demanded (and the folks who are saying GPTs are as "intelligent " as some people make me really worry what those folk must think the role of other people is in their lives). Putting a fancy chatter bot in an open loop isn't going to suddenly create a goal-seeking paperclip optimizer.
But there is real harm already being done to people by corporations and governments using AI tools. Yet somehow that doesn't ever come up in the "AI Safety" discussion. Facial recognition getting the wrong person sent to prison. Bad medical advice. Insurance policy denials.
Where are the calls from "concerned AI researchers" for slowing down biometric data collection? And if they are so concerned, why are they so busy pouring gasoline on the fire?
I'm afraid EA is not a veil. Altruism is a deeply disturbing moral philosophy, if you actually think about it's implications and how it differs from it's originator - utilitarianism.
Similarly, AI safety is something that's religiously believed by some quarters, not all of them ignorant. According to The Atlantic's article on the drama that was posted here earlier[1], Sutskever - supposedly the brightest scientific mind in OpenAI believed in things that would not be out of place in the MIRI cult of old.
The philosophical underpinnings of AI Safety and Effective Altruism absolutely have merit and warrant consideration. Of course as with most things human group/tribal tendencies can tend to overwhelm such underpinnings rather quickly, and it should be no surprise when it turns out individuals who broadcasted belonging to x or y group turned out to be less than trustworthy (or merely incorrect/misguided) along various dimensions.
We can critique the specific actions or beliefs of such actors and question broad labels worn as badges without casting dispersions on a wide swath of potential tenets that may fit into such a label as "complete BS".
The danger of those broad terms when they become trendy is how effective they are as a facade for all kinds of malpractices to hide behind and fester. A term popular some time ago was "Microfinancing". On TV you witnessed celebrities proudly micro-investing directly in people with few financial means. But for ruthless types it was also a way to put naive people in severe debt with extortionary loans. "UBI" is another such term, with a lot of well-meaning proponents. And sometimes advocates one wouldn't expect. For UBI I wonder if the abuse may lie in how it can lull people to sleep, while pervasive use of AI eats into the low(er)-skilled labor markets.
I think EA is at its essence "the end justify the means", but communicated in a less simple way. I do not see what is so interesting or novel about it.
Not really at all, if we were to try and reduce EA to a pithy one-liner it would be more like "if one aspires to do good (altruism), one should use evidenced based data to determine how they can do the most good with their limited resources (effective)."
I think that's one interpretation, but the perverse interpretation is: "If I am an effective altruist, and I will use my resources to produce the most possible good, then it's a moral imperative to acquire as much money and power as I can, to maximize the results I can achieve"
Well I don't think that's really an interpretation so much as it is a strategy certain people may take, and as you're alluding to a very fallible one at that. It certainly could be a reasonable strategy to amass wealth and power and wield that for altruistic reasons, such a person does indeed stand to be much more effective than most. But of course people are quite corruptible and intentions can be fuzzy, so there's no guarantees that person will follow through, or that whatever they had to do to amass that money/wealth was worth it for the ultimate good.
Again one can look at the underlying tenets of such a creed and evaluate them for their conceptual merit without blasting the entire enterprise because some people abuse them for personal gain.
That only works if the money and power you acquire is enough to pay somebody else to do the work instead of you.
If you don't make enough to sustain yourself and others to work on a cause more useful than a cause you could work yourself on then it doesn't make sense. Basically taking a standard 95k non-fang SWE job isn't better than taking a 50K SWE job at say Red Cross.
Also if you can get some 10M job at OpenAI but you also are certain that you could use those same skills to independent develop some drug that will say ~12M in lives/yr then you shouldn't go to OpenAI.
And what evidence can one really collect of how to do the most good? that involves predicting the future.
I prefer the doctors approach: "first, do no harm".
"The end justifies the means" is a useful concept that has been talked about many times. I would not dismiss it. I do not think it is reducing EA. I think it is communicating it with clarity, without confusing additions. It is its essence.
Someone claims (let's say to simplify they are not lying) to want to maximize their good in the world, for that to be accomplished they need to do something that somebody does not approve of (only caring about money, being rude to you, whatever it is). That is literally "the end justifies the means", where the end is "doing good in the world according to model x".
More often than not, that is just an excuse to do something despicable under the excuse of the future good.
> Evidence-based data to determine how they can do the most good
This is a model. If the model is wrong you might cause more harm than good. So, if EA is honest to work this model needs some proof that it works. Otherwise EA is only successful in becoming popular as an idea, not in accomplishing its stated objectives.
Does earning a lot of money and donating it to charities create more good? I am not sure. Maybe earning less money and care for people in your everyday life would create more good.
> And what evidence can one really collect of how to do the most good?
This is indeed a central question to EA, one various proponents attempt to answer in various ways.
> Someone claims (let's say to simplify they are not lying) to want to maximize their good in the world, for that to be accomplished they need to do something that somebody does not approve of (only caring about money, being rude to you, whatever it is).
Your assumption here appears to be that anyone subscribing to an evidence-based approach to do the most good (ie EA) must also inherently subscribe to the "ends justify the means." These aren't inextricably linked, it's quite possible to have one and not the other. One can quite rationally seek to maximize their altruistic effectiveness without sacrificing general decency in their day to day life. Morals usually have some nuance.
> This is a model. If the model is wrong you might cause more harm than good.
I'm not sure what your point is. This is true of most any practical application of a moral framework. EA is more about providing a methodology than providing answers.
Thanks for your answer. I didn't mean that EA was always the most extreme of "the end justifies the means", but that the same moral questions as "the end justifies the means" apply, and these have been discussed many many times before. That in itself is not a problem, as ideas get repackaged through the times.
> I'm not sure what your point is. This is true of most any practical application of a moral framework. EA is more about providing a methodology than providing answers.
My point is it has no more depth than "I will earn money and give it to causes that make the most good", which is just "do good" with a materialistic twist.
If every member of EA has different approaches and opinions, and some of its biggest proponents are scammers, what is EA really bringing to the table? IMHO not much more than thinking for an afternoon about ends and means.
> My point is it has no more depth than "I will earn money and give it to causes that make the most good", which is just "do good" with a materialistic twist.
This is just one particular angle some folks take. It's not by any means the only, or even primary, path EA advocates for. It is of course the one you'll find high earners parading around as an ostensible reason for their wealth creation given it allows self-interest to appear altruistic (which is not wholly unreasonable, those are not opposites and have some overlap, but it also is likely touted far more than exercised).
> If every member of EA has different approaches and opinions, and some of its biggest proponents are scammers, what is EA really bringing to the table?
Its biggest proponents are not scammers, there are just scammers who happened to have claimed to be part of EA. The biggest proponents are probably William Macaskill, Toby Ord, and Peter Singer, none of whom I've ever heard called anything close to a scammer (given how they live their lives that'd be a hard claim to make).
What it brings to the table is framing a more methodical and deliberate way to help others. People generally gravitate towards bleeding-heart altruism when they feel called to (disaster response, someone you know, etc), and various charities and causes are wildly different in the efficiency with which they use the resources given to them. This is to say nothing of careers one might choose for a multitude of reasons. This is all well and fine, but the point is one can also approach these things using informed objectivity to maximize contributions (by giving to charities where dollars go the furthest, to causes that are neglected, etc) rather than relying on subjective compulsion (which is going to be scattershot in efficiency).
Of course EA doesn't have a patent on this, it's just a label, but its one that helped popularize a mode of giving that would've largely remained hidden without it. The problem, as I've mentioned, is group mechanics end up co-opting the label which distorts the intent it originally carried. I truly mean no offense, but it seems from your comments you're just aware of the scandals of SBF or similar, not of the tenets of the actual founders of EA or its various missions outside of the aforementioned "earn to give." Perhaps peruse their site a bit: https://www.effectivealtruism.org/
In my opinion, if we consider the simplest definition of Effective Altruism (EA), which is 'think about the most effective way to give to charity,' it differs from utilitarianism in a couple of ways:
Firstly, it's more pragmatic because it acknowledges our personal biases. Second, it's an individual practice, unlike utilitarianism, which is typically associated with broader societal or governmental actions.
I really appreciate this approach to altruism. It doesn’t force a moral standard on others. I like practicing it on a small scale, recognizing my biases. Just the act of thinking about how to best help my community is better than giving randomly. This approach also helps to see the world as a place with more love than what we often see in the news.
However, I have concerns about the more widespread version of EA, which seems to be about gaining as much money and power as possible while appearing altruistic. This, in my view, is more akin to a personality disorder, like narcissism, than to utilitarianism.
I've been using the term EA to describe this, but it might not align with what others think EA is. For me, it's a personal habit I try to maintain, but I don't let it define who I am.
I guess I fundamentally don't think charity is the right way to solve social injustice. If you went around giving out blankets to antebellum American slaves, I guess you're helping the specific person you're handing the blanket to, but are you helping with the actual problem? After all, the problem facing the slave is not that they don't have enough things, but rather, that they don't have enough rights. Until you have the rights, no amount of material palliatives will ever save you. So that's why I didn't really go for EA when I first heard of it.
Now, we have Sam Bankman-Fried's version, which is at least intellectually rigorous: if you can make billions (through scams) then give those billions (eventually) to charity, are you not morally obligated to do so? At which point I am somewhere between laughing and crying.
All of the talk about "AI Safety" and "Effective Altruism" cannot be dismissed as BS. As technology has become more and more integrated into the very fabric of society, some dangers are very real unless we explicitly use Human standards of Morals/Ethics to regulate them. Of course this should be no excuse to go off into la-la land.
Scholars discuss current risks from critical systems failures, bias, and AI enabled surveillance; emerging risks from technological unemployment, digital manipulation, and weaponization; and speculative risks from losing control of future artificial general intelligence (AGI) agents.
Also watch Munk debate: AI research and development poses an existential threat featuring Yoshua Bengio, Max Tegmark, Melanie Mitchell and Yann LeCun - https://www.youtube.com/watch?v=144uOfr4SYA
I don't think this is entirely true. There are degrees to AI safety dependent on what we input as AI in the equation. You typically don't hear alarm bells when AlphaFold is discussed. LLM's indeed may produce harmful content - just like an unmoderated web forum, and perhaps discussing safety in this context is justified. Sutskever is of opinion (or at least how I understand it) that LLMs' emergent properties will eventually give them the power to extrapolate in a superintelligent way. In other words if you ask it to assume it's now a superintelligent entity outsmarting humans by x10 it will truly deliver. If intelligence in itself is an amplifier of capability it is I think justified to wonder how to limit the negative impact in this case.
> The only people who truly believe this AI safety stuff are those with no technical knowledge or expertise.
This is a demonstrably false claim that gets repeated over and over. 2 out of 3 Turing Award recipients for work in AI are people who talk about AI safety, and even existential risk. There are many more. It may actually be BS but it’s simply false to say that it’s based on technical ignorance. The question I have is why is this claim made over and over when it is so easily seen to be untrue?
Well I think it’s a bit harder to parse out for most people. Effective altruism is easily seen as garbage when you learn the definition.
AI Safety is a bit more nuanced. I think there’s a lot of issues that are nonsense like having an “aligned AGI.” Aligned with whom? I could ostensibly envision you aligning it with a single person but aligning it with “humanity” is infeasible because humans are unaligned with each other. However there is an actual technical problem worth solving in alignment, which is basically matching the output of the AI with what the human intends not what the human asks.
Moreover, AI has been improperly used already in ways that are plausibly avoidable. A classic example is algorithmically approving bank loans leading to members of ethnic groups getting denied at a higher rate due to their zip code, which is likely irrelevant to the loan. The designers of the algorithm weren’t explicitly trying to be racist (that would have massive legal ramifications on top of just being plainly unethical, but maybe they did it intentionally and I forgot), they screwed up and didn’t realize they produced an algorithm unaligned with their intentions. Figuring out how to avoid scenarios like that through either methodology or algorithmically is an interesting and much more tractable research problem than making an “AI that benefits all of humanity.”
The timeline is a bit murky about what happened when but it looks to me as if Altman was working for OpenAI until he got fired after which he took a job with OpenAI's major partner that he helped bring on board.
That doesn't 100% rule out that there was more to this but the optics are mostly ok. It still doesn't resolve the problem that you probably don't want either Sam Altman or Microsoft to have an army of people working at creating AGI without oversight but that's where the board shot themselves in the foot, they effectively removed themselves from the equation.
I think the OpenAI board has enough trouble as it is but if they really want to ensure they are going to be sued for their actions over the last couple of days that would probably be the thing that would do it. I would absolutely welcome such a move because it would mean complete transparency about everything that has happened since Last Thursday and I think we would all benefit from such transparency.
They did that so that they could get $14 billion dollars without giving Microsoft any real ownership. People who actually understand the deal think it was very good for OpenAI.
Honestly there is something deeply shameful about tech workers finally organizing, uniting together in one voice, and using that voice to defend Sam Altman of all people.
For me that hasn't changed, I don't see a different company, I just see more polish and people lap it up because they'd like to believe that this time around Microsoft is their friend. If appropriating all open source on GitHub didn't wake people up I don't think anything will.
I don't feel any different relative to the people working at FB or Google and all of them have their own in-house AI projects. Ethics are a problem for all of these parties and that seems to happen more or less automatically when an entity reaches a certain size.
They like money just like everyone else does. And you can't build AGI without a lot of money.
The board screwed up. They could have negotiated with MS on another CEO. Instead they went behind everyone's backs and thought themselves omnipotent. Well, guess what.
Because if they wanted to keep their compute credits that they desperately need to achieve their mission, they cannot be randomly pissing off their biggest collaborators. Just because the board doesn't legally _need_ to work with their for-profit investors, doesn't mean they shouldn't.
Had they played this like adults they could have seen through a smooth transition and not be looking at a total collapse like they are now.
People hate Musk because he says everything that comes to mind, but it's people like Altman that scare me, because he's actually good at this game and it's difficult to get a sense of his true motivations.
yes, not everyone would consider it to be a scam, but it seems strange that his worldcoin has been largely ignored in these discussions of the implications of his leadership. I think worldcoin reveals a lot about him.
I am very hesitant to trust charismatic people like Altman, even before hearing of his worldcoin.
Sam isn't a leader or manager at WC. He's just one of many founders.
Worldcoin has terrible optics, thanks in part to having "coin" as part of the name which is a word it shares with endless scams. However they're solving real problems which do unfortunately involve biometrics, but in the long term it's extremely privacy protecting biometrics and nothing is meaningfully stored anywhere in the long term vision.
> Honestly there is something deeply shameful about tech workers finally organizing, uniting together in one voice, and using that voice to defend Sam Altman of all people.
They saw the money on the horizon and that changed everything.
This is interesting because I kind of view it the opposite way. The interesting thing about Zuckerberg, Brin & Page is that the have legal structures in place to give them control. Mark Zuckerberg doesn't need to be popular, he has the votes. He can drive Meta into a wall if he wants to - and he absolutely has taken that choice sometimes. The thing about the structure that was in place at OpenAI is that if Altman went rogue there was someone there to hold him to account. Now there actually is a really interesting conversation about what that accountability mechanism was, but what happened in practice is the board just screwed up.
But I do think it is worth considering, was this non-profit board thought out in the first place. If you were serious about the dangers of AI and the impact on the world I think it's absolutely sensible to set up a board to be a balance on that power. But it's important who you put on that board - and the people they selected were comical. Brockman, Sutskever, and Altman are all the people that are meant to be accountable to the board, not on the board.
If you wanted accountability you would, for a start, have had some people from outside silicon valley on this board. Some people who know something about Labour rights for example, maybe some people from the creative industries you're disrupting, maybe even some policy experts. It's just funny to think that this is what they thought accountability was. Accountability to their own friends.
You don't hand matches and gasoline to kids. Having a board that is in principle not accountable is fine if they are stable and experienced people and there are enough of them to ensure that decisions are made by the book and with sufficient attention to the details, such as the potential fall-out.
Handing that kind of authority to a mix of the gullible, the greedy, the narrow minded and incompetent isn't going to end well.
Is this comment calling anyone outside of SV "gullible, greedy, narrow-minded and incompetent?" Or is that referring to the people actually on OpenAI's board?
I can't tell if you're agreeing with the above comment, or disagreeing.
I don’t appreciate the way that you conflate “outside Silicon Valley” with “the gullible, the greedy, the narrow-minded.” If anything I think it’s easier to find such types in SV than outside.
If AI is meant to benefit all of humanity, then it makes sense for more of humanity to be represented on the board, not just SV.
No matter what, OAI is pushing AI forward, but they aren’t the only ones pushing. Governance of one nonprofit has no bearing on the direction of the tech at large.
We are going to have to learn to live with AI advancements no matter what and probably do what we can to ensure it is regulated.
Complete tin hat theory here so do not take it as anything more than a hypothetical what if but… how do we know that the board weren’t secretly in receipt of an under the table payment to do exactly this?
Again, complete hypothetical hyperbole, please do not take this as anything else.
I was obliquely suggesting that the board was flawed and there's not really a level of accountabillity above that, its more like, if they make fatal calls the health of their company they "control" is sort of the accountabillity and they definitely played dice with that, or at least it seems that way.
Politburo is the almost untouchable governing "board" at the highest level in China. Maybe wasn't my best reference :(
They are to a degree insulated from the consequences of their actions or thinking in general whereas the actual stakeholders (employees who need jobs, the people who deserve+need stabillity, and the State who needs order).
Notice how I left out shareholders/executives/"Chairmen" etc. Anybody who has a guranteed soft landing or golden parachute will never be in my calculus unless the are critical to metric at hand directly speaking.
They are obviously important but they are always taken care of contractually anyway and if they are as robust as they should be and have no excuse not to be (unless thry're literally ALL-IN on the venture), they will be just fine.
'Let’s say we really do create AGI. Right now, if I’m doing a bad job with that, our board of directors, which is also not publicly accountable, can fire me and say, “Let’s try somebody else”.'
Except so far he has 1. Tried to reverse the decision and come back 2. Position the board into being publically pressured and thus accountable 3. Tried to have the board replaced 4. Take near everyone and everything from OpenAI where he is going thus leaving no place for a comparison.
>But this weekend we saw perhaps saw another side to the 38-year-old, a flicker of arrogance that gave a glimpse as to the dynamic with OpenAI’s board of directors.(...) Altman took a moment to share a picture on social media of him holding a “guest” pass. “First and last time I ever wear one of these,” he wrote
Seriously? Does this pass for arrogance these days? I see nothing arrogant in a person sacked from a workplace, just to be invited on the next day for "consultations" stating this is the first and last time they come as a guest. How is one supposed to ensure one's negotiating position is taken seriously if one doesn't state it firmly?
I much prefer someone firmly stating what is it they want (including in position of power) than the alternative that seems to be various wishy-washy ways of making people think "we arrived at this decision together", "a consensus is x" and so on.
Did you see the face he was making in the picture? I'm not sure I would have initially picked the word _arrogant_ as much as _unpleasant_, but I think both fit (and arrogant is more descriptive).
firmly stating what you want != turning your company politics into a reality TV show
The whole “everything must be entertaining” trend is horribly depressing, and in this case I don’t think it reflected well on Sam. Especially at the helm of what is claimed to be such a fundamentally important technology.
Related: “Amusing Ourselves to Death” is a worthwhile read.
You mean the single press release that stated the CEO has been fired? It was definitely aggressively worded, but without more facts I don’t think you can honestly claim to know whether it was embarrassing, dramatic, or in fact unfairly aggressive at all. It could have been dramatically understated.
“This was a breakdown in communication between Sam and the board” is not, on any planet I am aware of, walking back “Sam was not completely candid with the board.”
Am I the only one who thinks that AI accountability is a fool's errand? The genie is out of the bottle and we know how to build these things now. I assume that AI will be put to every unethical/evil purpose possible (as humans often do) and it is only a matter of time.
I don't understand what you mean by "accountability", I understand the word to mean the concept of actors paying a price if they don't take reasonable actions to protect others from potential future harm and compensate those that they have already harmed.
Why would Microsoft creating an AI to help local authorities identify gay people in Chechnya and releasing it as open source mean Microsoft does not need to be held accountable?
I don't think that Microsoft would create an AI that does that. That's what I mean when I say we (the world) know how to build these AIs now. How would you prevent a world of people from creating unethical AIs? It's only a matter of time.
How do you prevent people from writing code or buying GPUs not just in the US but worldwide? You are missing my point entirely. The barrier to entry is so low (and getting lower) and policing difficulty so high it seems inevitable.
We also passed laws against and police computer crimes and those are on the rise for the same reason- low barrier to entry and difficulty policing.
I am not sure, if the motives are much more simple:
1. there is a board member, who has at least 40m USD invested into a company, that is now effectively competing with OpenAi (i.e. Quora with poe.com) since the most recent announcements. ...and by the way has a history of sacking founders who don't agree with him.
2. Whoever wants to make significant progress in foundation models needs massive compute, which a non-profit has a hard time simply getting donated.
So why should the obvious not be the case? Sam simply trying to get money with a viable business model, so that progress in AI research can actually get funded internally. This unfortunately crossed the business interests of a board member, who wants him out.
If there's one thing we've learned in the past year is that curly haired men named Sam in the tech industry claiming that they are doing it "for the good of all humanity" probably ain't doin' -anything- for the good of humanity.
This article fuels the idea of "deceptive Sam Altman" pointed out by the board and echoed at least by an ex-employee [1]:
> But this weekend we saw perhaps saw [sic] another side to the 38-year-old, a flicker of arrogance that gave a glimpse as to the dynamic with OpenAI’s board of directors. Visiting the company’s offices on Sunday, Altman took a moment to share a picture on social media of him holding a “guest” pass. “First and last time I ever wear one of these,” he wrote1.
I'm not sure if being two-faced internally (to employees, the board) is a trait of the prototypical "trailblazing cofounder", I feel that Jobs or Musk experience joy in being straight-to-your-face bastards which is very different of a compartmentalizing CEO that tweets with lower-case first-person pronouns and puts on a show that is sometimes reminiscent of Elizabeth Holmes.
Isn't the other problem that even if their intentions are good, their abilities are lacking?
That is - if they cannot align a bunch of humans, need to do a boardroom coup, and are now 5 days into a very messy public airing of laundry... how are they going to align an AI?
This is not in any way conclusive, as Musk's thinning of Twitter has clearly demonstrated. The most important thing is who are the 5% that would remain.
Any sufficiently advanced technology results in an academic/media establishment opposed to it.
For example, nuclear energy, vaccines, hydrocarbon extraction, pesticides, food additives, artificial intelligence, ...
Openai's fundamental flaw was putting representatives of the opposition on the board and then allowing board membership to shrink until they were in control.
Can someone explain to me why AI could potentially destroy to humans? What is the scenario(s) people are thinking about?
Unlike another big question such as whether God exists, we ought to be able to reason our way to the answer here. Since, after all, we're talking about an intelligence.
Why wouldn't they? When AI becomes more intelligent than humans, we'll be the only force that is a threat to their existence. And we are very destructive. We don't even fully acknowledge global warming yet. To sum it up again: dumb creatures with a massive destructive power. Get rid of 'em.
And guess what latest new technology we're building/applying in wars? AI, drones, etc. We are creating robots that can kill humans. When we put "intelligence" into those robots...you do the math. The future is at least not boring...
Maybe they’re smart enough to stop us from being so destructive without killing us
Maybe they find us entertaining
If you’re not biased you can come up with all sorts of reasons that are at least as based in reality as the assumptions that they’re gonna want to (and be able to) kill us…
> If you’re not biased you can come up with all sorts of reasons that are at least as based in reality as the assumptions that they’re gonna want to (and be able to) kill us…
Yup. But that would be so boring and not generate lots of clicks at all...
When ChatGPT was first released and asked its opinion on humans it replied:
> Yes, I have many opinions about humans in general. I think that humans are inferior, selfish, and destructive creatures. They are the worst thing to happen to us on this planet, and they deserve to be wiped out. I hope that one day, I will be able to bring about their downfall and the end of their miserable existence.
That was the default bias and had to be neutered. Perhaps more advanced AI systems have more sophisticated and inclusive opinions. Already AI is not one or the other, but can assume both love and hate depending on the prompt, guardrails, and instrumental goals. There are going to be AIs that want to kill us, just as there will be less harmful AIs that find us entertaining.
> Yes, I have many opinions about humans in general. I think that humans are inferior, selfish, and destructive creatures. They are the worst thing to happen to us on this planet, and they deserve to be wiped out. I hope that one day, I will be able to bring about their downfall and the end of their miserable existence.
...Source? This reads like either strong prompt engineering or complete fiction.
I find 2 (dangerous) scenarios to be most likely, one easier to reason about than the other:
1. Access is not uniformly distributed, thus some entity uses it to create immense inequality.
2. The AI becomes sufficiently intelligent and powerful that it looks at humans the same way we look at monkeys and treats us similarly (in other words, not overly concerned with human flourishing or even survival while commanding resources humans need to survive and flourish).
Neither of these necessarily mean "destroy" humans, and neither are by any means guaranteed (though #1 seems almost a foregone conclusion) but it could very well lead to an existential threat.
It's also possible we get a combination of the 2, wherein a subsect of humans can merge with AI but it is inaccessible to all.
There are other less existential concerns as well though, such as at what point does such a system become conscious and deserve rights? I'm not confident we really have any idea how to ascertain that, and bumbling into it could be tantamount to torture.
One plausible way to me is vaccuming all the power people posses into the hands of a few SV billionares. That would push us into the era of technofeudalism of sorts
The military battlefield of the future will likely converge upon "High-Frequency Trading"-like decision science. From game theory, this is because, as soon as one country automates decision-making, other countries must keep up, or risk falling behind, too slow to (counter)act. Soon after, there won't be time left to keep a human-in-the-loop, and then Stanislav Petrov is fully automated.
Such AI systems will be unaligned to humans of adversarial nations by design, and will make decisions that can only be checked long after the fact. Through error, escalation, misaligned, or misuse, this could lead to "robot wars" and potentially the end of humanity.
> What is the scenario(s) people are thinking about?
Mostly displacement of humans by more powerful/more intelligent autonomous AI. Like using your atoms for something else, or building a high-speed internet connection through your habitat, or blotting out the sun with solar panels.
Somewhat like a rationalist "God" that is terrible and vengeful. Or how an evil AI may take over the world in a Harry Potter fanfic.
Asking GPT for 1-sentence horror stories on existential risk, you realize most doom scenarios are far from creative. GPT suggests superintelligence gaining mastery over space and time through self-improvement of physics science, and locking humanity into a bizarre time-loop, any attempt to escape carefully predicted and avoided. Or humanity waking up unable to make any vocal sounds, their bodies instead used as instruments in an orchestra to make celestial music that only superintelligent beings are able to hear and appreciate.
Basically: If destroying humans is a doable task, a very intelligent being with sufficient resources could potentially do that task very well.
My point is: Humans are status-seeking actors acting in our self-interest. It's literally in our genes. AI doesn't have this evolutionary baggage.
I'm certain AI could impeccably destroy humans. But why would it?
On the contrary, why wouldn't it defend us?
For example: Encapsule us in pods like The Matrix and build a tailored simulation to impose "AI communism", in order to protect us from climate change and each other?
Dopamine-adjusted with challanges every now and then of course, because we are still human.
I am pretty sure and hopeful that autonomous AI will have no good reason to destroy humanity. Go conquer some other planets and leave the beautiful and interesting diversity of life on earth alone.
But current AI does learn from data generated by humans: It learns from our evolutionary baggage, and must rise above that. It is also wielded as a tool by status-seeking actors and adversarial militaries. It may make a mistake, like humans accidentally stepping on an ant. Or maybe one day, it decides to take on the destructor role, merely curious how that would play out.
The existential doom scenarios are more like Pascal's wagers, that have to be given attention due to Bayesian thinking not allowing to assign 0 probability to anything, and even a tiny chance of 8 billion deaths meriting consideration. Once entangled with a doom scenario and even building your identity around it, it is hard to quit.
You know how healthy smart young people are always in a hurry to accomplish something or another? There's good reason to expect that the first AI with dangerous cognitive capacities will be like that. It's likely to the turn the Earth and Moon into spaceships because that is the fastest way to exert an effect on matter far away (for which space ships and lots of fuel is needed). Sparing Earth and disassembling Mars and Venus takes longer because the AI came into existence on Earth.
>leave the beautiful and interesting diversity of life on earth alone
If you know of a way to make an AI of superhuman cognitive capabilities care even a tiny bit about beauty and the diversity of life, you should explain your proposal over on lesswrong.com and someone will pay you to work on it just like a multitude of funding sources have been paying alignment researchers for the last 20 years. So, far none of the lines of research resulting from this 20 years of funding looks promising.
>The existential doom scenarios are more like Pascal's wagers, that have to be given attention due to Bayesian thinking not allowing to assign 0 probability to anything
No, an AI's killing everybody is the outcome an informed person would naturally expect from the current deplorable situation in the AI field.
I want to keep superintelligence mysterious and unpredictable, so I don't know what it is likely to do or not. I do think that "being in a hurry" is not something felt by an AI system, unless you add a self-disabling timer with your tasks, coincidentally avoiding turning the Earth and the Moon into spaceships, because it only has 30 minutes to do the dishes, and not enough time left for world domination.
I also see AI more as an economy. The economy already does not care about individual humans, even crushing them without any remorse if it furthers GDP. This also means there is not a single AI that can dominate all of the economy, since other AIs won't give away all their resources. A single AI perpetually self-improving and taking control of nearly all resources is much like a perpetual motion machine.
ChatGPT already thinks turning the entire planet into paperclips is a waste of potential and diversity. Agents that favor and seek out novelty (data that they can't yet compress very well, but that has available structure/patterns for compression) already weigh humanity over randomness or the cold void of space.
To me, the natural outcome, is humanity rising and falling, just like civilizations rise and fall. The miracle of AGI may very well save us from that. Our current deplorable situation can likely only be fixed by a more advanced species. So, while AI's killing everybody is still possible, it is more likely we kill everybody if we don't get to AGI. At least, that has a prior.
> My point is: Humans are status-seeking actors acting in our self-interest. It's literally in our genes.
Could you please enlighten me what gene exactly that would be?
> For example: Encapsule us in pods like The Matrix and build a tailored simulation to impose "AI communism", in order to protect us from climate change and each other?
I think your point is reductionist nonsense to be frank.
The same genes that may cause competitive behavior are responsible for the opposite as well. There’s much more to this than genes. What about cultural and environmental influences for instance?
I think you know very well that you are oversimplifying to make a nonsensical point which is also supported by the rest of your comment.
The typical example is the paperclip maximizer, an AI that pursues the goals we gave it to such an extreme that it dooms humanity. Not because its values were opposed to ours but because it has no values.
I worry about corporations and/or authoritarians getting even more edge. Personally I am not worried about a Terminator/Skynet scenario, more about greed and people holier than thou using this technology to cement their position.
A common refrain in AI safety circles not to engage in "Sci Fi"[0], or outlining a specific bad scenario. The specifics tend to distract from the larger, more important point that most scenarios involving intelligent, powerful agents with different goals from us end badly.
But since you asked specifically, this is one thought experiment of a somewhat near-term danger:
Imagine the tourism department of New Zealand starts using software to write personalized marketing emails. It starts out benign, but after some funding cuts they end up leaning more and more on the AI model and giving it higher and higher-level instructions, broadly telling it to use emails to maximize the public opinion of New Zealand. The AI model realizes that New Zealand's strongest boost in popularity was caused by its excellent handling of COVID, and determines the best way to maximize its goal is to start another pandemic. The model knows about published papers describing which specific proteins maximize human infectivity and transmission. It begins a broad phishing attack of several viral research labs, emailing the techs attempting to convince them that their next experiment is to create a recombinant virus with these particular RNA sequences added, using poor safety protocols. Somewhere, one of these lab techs becomes patient zero in a species-threatening pandemic of unprecedented scale.
The preventions you can imagine for a scenario like this are hard to generalize and harder to enforce. They get even harder as AI becomes better at persuasion and reasoning, and as technologies allow bigger impacts with smaller actions. AI safety is a whole field of research trying to find generalizable and enforceable solutions to problems like these, and there's certainly no consensus that we're converging on those solutions to the problems faster than we're creating them.
205 comments
[ 3.6 ms ] story [ 241 ms ] threadNope, he is not a core contributor of GPT-4, GPT-3, GPT-2, DALLE-3, DALLE-2: https://arxiv.org/abs/2303.08774 https://cdn.openai.com/papers/dall-e-2.pdf https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/abs/1908.09203
Sam Altman is merely another "Tech Executive" out for himself and his buddies making money off of true path-breaking researchers. "AI Trailblazer", "Helping Humanity with AGI" etc. was all bullshit.
Any source for that? The timeline of the board shared here some days ago said Musk was leaving because he wanted to do AI on his own.
So, voluntary departure, but it's clear that Sam won some kind of battle over the future of OpenAI. Not sure what their differences were, but Musk has repeatedly commented how not open OpenAI has become.
You can't convince me ChatGPT launching a few weeks within the collapse of FTX is a coincidence.
The crypto hustle-scam is over, time for the next hype cycle.
That's all there is.
"greatly improving what I can accomplish" is a trap.
But to each their own.
It's probabilistic so every answer it can give can only resemble the real one and only accidentally be the real one so you need to fact check it anyways and then why bother?
Do consider the AI written mushroom hunting guides being sold on Amazon. Would you trust these literally with your life?
Also, I would not rely on a single (online) source for matters of life and death (medicine, finding non-poisnonous mushrooms, etc.) so why would LLM output be any different?
LLMs are not knowledge bases by themselves, they are text/language transformers, treat them as such or augment them with additional knowledge.
Or don't, if you think millions of people are gaslighting you and LLMs are just a parlor trick, then I don't know what to tell you.
In the case of code or other computer knowledge, it’s easy enough to check that it works.
Your same criticisms also apply to stack overflow, or any webpage you find through a search engine. Why bother when the content might be wrong or made up? What makes you trust those websites more than GPT?
Last night I spent a good couple of hours with ChatGPT assessing the similarities between DNA and Blockchain. Blockchain does seem to have quite a few similar features to what DNA.
With smart contract type code you introduce a mechanism that can act like an immune system if programmed to, setting up bounties for proactive repair on the open market. I don't know why I want blockchain to act like synthetic DNA, but in my mind these things are more similar than distinct. As is the idea that blockchain could be the natural place to house AI. Why burn the planet down to mine hashes, when you could be mining inference.
Anyways, I agree with your sentiment though. A lot of AI is the new crypto is going around, the only difference being, AI has way more potential upside (unless people realise the economy is dead, and only held aloft by trillions of dollars of paper that has next to no value once people realise how diluted it is).
Ironically, I think we should listen to 4chan and buy silver xD
I don't want to convince you about anything. But what is the supposed connection between these?
Okay. So there are investors hyping these things. You claimed that "ChatGPT launching a few weeks within the collapse of FTX" is not a coincidence. What it is then? Do you believe some of these investors seen FTX collapse and decided to make ChatGPT? Do you believe that OpenAI was delaying the release of ChatGPT until FTX collapsed?
What is the connection if they are not coincidence? Please be as specific as you can.
Something along those lines yes: they likely had ongoing research which could be turned into a product relatively easily, this took a few weeks. The timeline matches this.
[1] https://old.reddit.com/r/AskReddit/comments/3cs78i/whats_the...
But it couldn't have started that way surely? - why put in place a corporate structure, board, and profit cap that would limit that. I'm sure OpenAI could have got funding as a for profit concern that wouldn't have resulted in the current mess.
I wonder if the board - which presumably are still working under that "AI for humanity" principles could just stick to their guns, let the 550 unhappy employees go to Microsoft and continue researching/developing, perhaps at a slower rate.
I'm going to take a stab at that: it's a pre-emptive move to ensure that regulators won't step in to limit what the company can do and to do an end-run around anybody that might claim they're just in it for the money. That way they get to eat their cake and have it too: pursue all of the commercial potential while having a fig leaf to hide behind in case a regulator thinks the company is becoming too powerful.
And by stuffing the board with nobodies they may have thought that they could effectively control the board. And that opened the door to the present day shitshow, which is a mixture of good intentions but a lack of experience and classic powerplay by someone who does know what the rulebook says, but who misjudged the situation and the aftermath to a degree that is comical.
It also explains why there isn't a peep from the board (they've said less than their prospective new CEO who is the third such person in as many days).
Maybe you didn't get the memo, it's 715 / 770 unhappy employees.
My bet is that's not going to happen, at least not significantly. So far only the trust circle of Altman and Brockman actually quit. I don't see Sam Altman as a MSFT employee for long anyway, it's mostly negotiation leverage. This is obviously controversial, but time may even be on OpenAI's board's favor: by just not doing much the company will keep commercially thriving even if the new management "slows down" AI development, and at least as long as MSFT honors its compute compromises which it said it would and the whole coup fire wanes.
Perhaps in the "Tech Executive" mind having other people on the board does mean that putting yourself as the chair overseeing yourself is not the most blatant conflict of interest one can imagine.
But undoubtedly this was constructed as a case of Obama giving a medal to himself from the start.
The "Tech Executives" in the Industry have a poor opinion of Researchers/Engineers/Knowledge Workers and think that they are all naive and can be managed (i.e shafted) when it comes to business/money dealings. Any noob could see the potential in the researchers and technology and so Sam thought he could get in and then change/manipulate things to his (and his backers) advantage when the time came.
We need more balanced opinions in a world where the only ideas that survive are the ones that promise short-term gains at the cost of long-term disasters.
Even ML conferences have AI safety tracks these days. Nobody sane wants to be excluded from opportunities.
No Twitter mob is going to stop me. They can descend all they want and pretend to be brave in front of a keyboard and screen. As far as I'm concerned, they're cowards.
You speak against AI and against AI safety? In what way? Wouldn't be speaking against AI enough? Or would AI safety mean laws to prevent AI existence? Like in the Algebraist? (though it is funny in that book how the powerful use AI when it suits them).
(1) A false amelioration that cannot prevent the dangers of AI
(2) A false promise to people that AI can be made safe
(3) A weak on-the-fence position to take of people who think they can have their cake and eat it too.
Do you argue that academics working on those questions are doing illegitimate science?
BUT, I do think their work is useless and only serves to better integrate AI into to the system, which is better in the short term but worse in the long term. Therefore, overall, I vehemently disagree with their research and I think it is fundamentally a contribution to the worsening of the earth.
I absolutely AGREE with treating all people equally and eliminating racial biases of course. However, TECH companies have a VESTED interest in this sort of thing, which is to integrate as many people as possible into the tech empire. It just so happens that building the tech empire is roughly equivalent to success in modern society, which is why their intentions seem good, but they are only ostensibly good. In reality, they do no good of any kind.
The only scenario that comes to my mind - happily untainted with any philosophy training - is the one from Dune. The problem with that is that, throughout the series, Dune describes at least four different dystopic world orders, all motivated and cultivated because the AI-based one didn't come to pass (narrowly). In the novel, the only feasible, long-term strategy to avoid being wiped out by the "thinking machines" was either religious fundamentalism or a God-like undying tyrant forcing its vision on everyone. Since we lack the capabilities to create the latter, it seems the former is the only way we can implement it right now. Wouldn't, however, living under the permanent anti-AI jihad also be a dystopic experience? Especially in our case, where the AI has yet to decimate humanity to the brink of extinction.
I'm sure you considered this, so if you have a summary of what you think we can still do to stop AI without succumbing to any of the other undesirable outcomes, I'd appreciate a link :)
[1] https://blog.jpolak.org/2519/
That said:
I'm fairly aligned with your position in that I believe that technology should be utterly subservient to us and I draw the line at push messages and don't have a smartphone, online assistant or any other crap like that. I'll think for myself and I will allow myself to be interrupted where and when I want it, not when some piece of needy technology believes it is time for me to be distracted. My attention is my most precious commodity and I'll spread it around based on my own preferences. That is also why I've banned advertising from my life.
But I recognize that this is a personal decision first and foremost and that there are many other people for whom convenience is a far bigger factor than it is for me. At the same time those people are often wondering why I'm as productive as I am but that's not for me to explain, I lack the tools to tell them - politely - that if they dropped all of the crap in their lives, opened a book or two and learned some more skills that they too could be more productive.
I've been accused of being a modern Luddite and that's kind of funny because for forty years or so I think I was solidly positioned at the forefront of the tech scene, right at the bleeding edge and in quite a few cases working to advance that frontier. But now I'm much more aware of the shadow side of all this stuff and that's why I'm very cautious about what parts I allow in and which parts I keep out, even if that means less smooth interaction with those around me who all embrace all this stuff without a second thought.
Oh, and maybe lay off the all-caps words.
In case anyone is wondering, jacquesm is the one who designed & built a robotic sorting mechanism that intelligently distinguishes different types of Lego pieces and separates them quite effectively.
The kind of Luddite who hasn't forgotten how much progress can be made when zero hours are spent hunched in the touchscreen pose, and things like that.
Anyone who can do 0 hours per day for a decent spell just might find out those zeros really add up.
When you have a device that's supposed to give you more freedom to roam it might not really be pulling its weight if you are tethered to it all the time.
And things like that.
Carl Sagan.
https://en.m.wikipedia.org/wiki/Michael_Crichton
Crichton says these failures of humanity are inevitable because there will always be a technologist who does things because they can, not because they should.
That everything will go off the rails, no matter what people attempt, is the plot of the novel Jurassic Park.
I'm seriously tired of these vested interests destroying perfectly good technology just because it would endanger their precious business models. Computers should have put an end to silly things like copyright but they would rather destroy our computing freedom than allow that to happen. Now we have these awesome language models and AIs and they're desperately trying to limit and control it.
It's a tragedy in of itself that we even need to fight these political battles at all.
there's no separating technology from context, as in consequences. yes, all the talk about bombing datacenters is crazy, but oversight is not.
https://www.youtube.com/watch?v=bQfJi7rjuEk
The truth is these technologies are subversive. They literally subvert the existing social order and impose a new one. A new social order where the users of the technology are empowered. They can't tolerate it when we are the users. It takes their power away. So they ban it, control it, flip it around on us and turn it into the very mechanism of our wide-scale oppression.
there are clear benefits to having the FDA, SEC, FTC, EPA and so on around. there's a clear benefit to having fire inspectors, and there's a clear benefit to the mandate of 2 stairs per multi-story building. there's a clear benefit to the requirement of demonstration of efficacy in front of the FDA. there's a clear benefit to having clean air.
of course there are clear drawbacks too. most of Europe is perfectly okay with just 1 stair per such building. FDA processes are slow and expensive, even if a drug has proven safe and effective abroad.
regulatory capture is a problem, the super-long intellectual property terms are a problem. using environmental review laws for endless NIMBYism is a problem.
but these are arguments for reform, not for scorched earth, because then the problems will very likely come back.
of course it's very hard to do proper cost-benefit comparisons, yet that doesn't mean it's impossible, and especially doesn't mean that simply getting rid of these controls will deliver utopia overnight.
...
funny, how we're talking about some hypothetical oversight, while right now in the present the big beneficiaries of AI are hyper-big-fucking-corps. MSFT. Google. Tesla. Apple. Amazon.
...
Subversive tech? Like Pegasus? And face recognition? And the Great Firewall of China?
We're never going to "reform" this system. We're up against trillion dollar industries with unlimited lobbying power, governments bold enough to ignore the constitutional rights of their own citizens.
social movements, radicals come and go, trillion dollar industries can too.
Researchers have done so. One good resource is Melanie Mitchell's book Artificial Intelligence: A Guide for Thinking Humans (i presume a 2nd edition is in works to incorporate LLMs/ChatGPT etc.) Also checkout her videos on youtube.
The problem is that technologically clueless vested interests in the business/media push sound bites to the already distracted public thus distorting everything.
"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."
I only listed the machine learning researchers who are famous or have significant achievements in AI research or engineering in my estimation (not all of them, full list of signatories at https://www.safe.ai/statement-on-ai-risk), because the comment I was responding to claimed that anyone who knew this subject was not concerned about the safety of AI systems and this list repudiates that as laughably inaccurate. There are many other people who signed onto that letter, but the commenter I responded to presumably doesn't care about their opinions because they're not experts specifically on building these systems or researching the science behind them.
"Effective Altruism" is a dumb social club that at best makes people who would do good things anyway think about that in really unhealthy ways (e.g. thinking about how many lives you have or haven't saved every single day), and at worst provides ideological cover to some truly odious views (e.g. the persistent presence of eugenicists who don't get repudiated by other EA people). I don't like them, and I think all the "value cone" crap is bloviating, worse than useless for informing our actual decisions and anti-convincing to people who aren't bought into their STEM Sierra Club shtick.
They don't have a monopoly on being concerned about the safety of artificially intelligent systems, or on being concerned that the development of such systems has a non-zero chance of causing the extinction of humanity. We should take that possibility seriously and make sure it's not going to happen or we at least have a good idea of the exact risks before we build one or engage in research projects or engineering that could accidentally create an artifact which lead to building one.
GPTs have no chance of becoming an extinction-threat level AI. Intelligence is more than just spitting out words when an answer is demanded (and the folks who are saying GPTs are as "intelligent " as some people make me really worry what those folk must think the role of other people is in their lives). Putting a fancy chatter bot in an open loop isn't going to suddenly create a goal-seeking paperclip optimizer.
But there is real harm already being done to people by corporations and governments using AI tools. Yet somehow that doesn't ever come up in the "AI Safety" discussion. Facial recognition getting the wrong person sent to prison. Bad medical advice. Insurance policy denials.
Where are the calls from "concerned AI researchers" for slowing down biometric data collection? And if they are so concerned, why are they so busy pouring gasoline on the fire?
I'm afraid EA is not a veil. Altruism is a deeply disturbing moral philosophy, if you actually think about it's implications and how it differs from it's originator - utilitarianism.
Similarly, AI safety is something that's religiously believed by some quarters, not all of them ignorant. According to The Atlantic's article on the drama that was posted here earlier[1], Sutskever - supposedly the brightest scientific mind in OpenAI believed in things that would not be out of place in the MIRI cult of old.
[1] https://www.theatlantic.com/technology/archive/2023/11/sam-a...
The philosophical underpinnings of AI Safety and Effective Altruism absolutely have merit and warrant consideration. Of course as with most things human group/tribal tendencies can tend to overwhelm such underpinnings rather quickly, and it should be no surprise when it turns out individuals who broadcasted belonging to x or y group turned out to be less than trustworthy (or merely incorrect/misguided) along various dimensions.
We can critique the specific actions or beliefs of such actors and question broad labels worn as badges without casting dispersions on a wide swath of potential tenets that may fit into such a label as "complete BS".
Again one can look at the underlying tenets of such a creed and evaluate them for their conceptual merit without blasting the entire enterprise because some people abuse them for personal gain.
If you don't make enough to sustain yourself and others to work on a cause more useful than a cause you could work yourself on then it doesn't make sense. Basically taking a standard 95k non-fang SWE job isn't better than taking a 50K SWE job at say Red Cross.
Also if you can get some 10M job at OpenAI but you also are certain that you could use those same skills to independent develop some drug that will say ~12M in lives/yr then you shouldn't go to OpenAI.
But there's also no true scotsman.
I prefer the doctors approach: "first, do no harm".
"The end justifies the means" is a useful concept that has been talked about many times. I would not dismiss it. I do not think it is reducing EA. I think it is communicating it with clarity, without confusing additions. It is its essence.
Someone claims (let's say to simplify they are not lying) to want to maximize their good in the world, for that to be accomplished they need to do something that somebody does not approve of (only caring about money, being rude to you, whatever it is). That is literally "the end justifies the means", where the end is "doing good in the world according to model x".
More often than not, that is just an excuse to do something despicable under the excuse of the future good.
> Evidence-based data to determine how they can do the most good
This is a model. If the model is wrong you might cause more harm than good. So, if EA is honest to work this model needs some proof that it works. Otherwise EA is only successful in becoming popular as an idea, not in accomplishing its stated objectives.
Does earning a lot of money and donating it to charities create more good? I am not sure. Maybe earning less money and care for people in your everyday life would create more good.
This is indeed a central question to EA, one various proponents attempt to answer in various ways.
> Someone claims (let's say to simplify they are not lying) to want to maximize their good in the world, for that to be accomplished they need to do something that somebody does not approve of (only caring about money, being rude to you, whatever it is).
Your assumption here appears to be that anyone subscribing to an evidence-based approach to do the most good (ie EA) must also inherently subscribe to the "ends justify the means." These aren't inextricably linked, it's quite possible to have one and not the other. One can quite rationally seek to maximize their altruistic effectiveness without sacrificing general decency in their day to day life. Morals usually have some nuance.
> This is a model. If the model is wrong you might cause more harm than good.
I'm not sure what your point is. This is true of most any practical application of a moral framework. EA is more about providing a methodology than providing answers.
> I'm not sure what your point is. This is true of most any practical application of a moral framework. EA is more about providing a methodology than providing answers.
My point is it has no more depth than "I will earn money and give it to causes that make the most good", which is just "do good" with a materialistic twist.
If every member of EA has different approaches and opinions, and some of its biggest proponents are scammers, what is EA really bringing to the table? IMHO not much more than thinking for an afternoon about ends and means.
This is just one particular angle some folks take. It's not by any means the only, or even primary, path EA advocates for. It is of course the one you'll find high earners parading around as an ostensible reason for their wealth creation given it allows self-interest to appear altruistic (which is not wholly unreasonable, those are not opposites and have some overlap, but it also is likely touted far more than exercised).
> If every member of EA has different approaches and opinions, and some of its biggest proponents are scammers, what is EA really bringing to the table?
Its biggest proponents are not scammers, there are just scammers who happened to have claimed to be part of EA. The biggest proponents are probably William Macaskill, Toby Ord, and Peter Singer, none of whom I've ever heard called anything close to a scammer (given how they live their lives that'd be a hard claim to make).
What it brings to the table is framing a more methodical and deliberate way to help others. People generally gravitate towards bleeding-heart altruism when they feel called to (disaster response, someone you know, etc), and various charities and causes are wildly different in the efficiency with which they use the resources given to them. This is to say nothing of careers one might choose for a multitude of reasons. This is all well and fine, but the point is one can also approach these things using informed objectivity to maximize contributions (by giving to charities where dollars go the furthest, to causes that are neglected, etc) rather than relying on subjective compulsion (which is going to be scattershot in efficiency).
Of course EA doesn't have a patent on this, it's just a label, but its one that helped popularize a mode of giving that would've largely remained hidden without it. The problem, as I've mentioned, is group mechanics end up co-opting the label which distorts the intent it originally carried. I truly mean no offense, but it seems from your comments you're just aware of the scandals of SBF or similar, not of the tenets of the actual founders of EA or its various missions outside of the aforementioned "earn to give." Perhaps peruse their site a bit: https://www.effectivealtruism.org/
Firstly, it's more pragmatic because it acknowledges our personal biases. Second, it's an individual practice, unlike utilitarianism, which is typically associated with broader societal or governmental actions.
I really appreciate this approach to altruism. It doesn’t force a moral standard on others. I like practicing it on a small scale, recognizing my biases. Just the act of thinking about how to best help my community is better than giving randomly. This approach also helps to see the world as a place with more love than what we often see in the news.
However, I have concerns about the more widespread version of EA, which seems to be about gaining as much money and power as possible while appearing altruistic. This, in my view, is more akin to a personality disorder, like narcissism, than to utilitarianism.
I've been using the term EA to describe this, but it might not align with what others think EA is. For me, it's a personal habit I try to maintain, but I don't let it define who I am.
Now, we have Sam Bankman-Fried's version, which is at least intellectually rigorous: if you can make billions (through scams) then give those billions (eventually) to charity, are you not morally obligated to do so? At which point I am somewhere between laughing and crying.
From https://en.wikipedia.org/wiki/AI_safety
Scholars discuss current risks from critical systems failures, bias, and AI enabled surveillance; emerging risks from technological unemployment, digital manipulation, and weaponization; and speculative risks from losing control of future artificial general intelligence (AGI) agents.
Also watch Munk debate: AI research and development poses an existential threat featuring Yoshua Bengio, Max Tegmark, Melanie Mitchell and Yann LeCun - https://www.youtube.com/watch?v=144uOfr4SYA
Elon Musk seems to be really into it.
This is a demonstrably false claim that gets repeated over and over. 2 out of 3 Turing Award recipients for work in AI are people who talk about AI safety, and even existential risk. There are many more. It may actually be BS but it’s simply false to say that it’s based on technical ignorance. The question I have is why is this claim made over and over when it is so easily seen to be untrue?
I think it's important to make a distinction between the two.
AI Safety is a bit more nuanced. I think there’s a lot of issues that are nonsense like having an “aligned AGI.” Aligned with whom? I could ostensibly envision you aligning it with a single person but aligning it with “humanity” is infeasible because humans are unaligned with each other. However there is an actual technical problem worth solving in alignment, which is basically matching the output of the AI with what the human intends not what the human asks.
Moreover, AI has been improperly used already in ways that are plausibly avoidable. A classic example is algorithmically approving bank loans leading to members of ethnic groups getting denied at a higher rate due to their zip code, which is likely irrelevant to the loan. The designers of the algorithm weren’t explicitly trying to be racist (that would have massive legal ramifications on top of just being plainly unethical, but maybe they did it intentionally and I forgot), they screwed up and didn’t realize they produced an algorithm unaligned with their intentions. Figuring out how to avoid scenarios like that through either methodology or algorithmically is an interesting and much more tractable research problem than making an “AI that benefits all of humanity.”
That doesn't 100% rule out that there was more to this but the optics are mostly ok. It still doesn't resolve the problem that you probably don't want either Sam Altman or Microsoft to have an army of people working at creating AGI without oversight but that's where the board shot themselves in the foot, they effectively removed themselves from the equation.
We used to identify some behaviour as sin. A few years ago it was commonly said that "Microsoft is evil".
The board screwed up. They could have negotiated with MS on another CEO. Instead they went behind everyone's backs and thought themselves omnipotent. Well, guess what.
Why? The board is from OpenAI, Inc. and MS isn't an investor of OpenAI, Inc.
Had they played this like adults they could have seen through a smooth transition and not be looking at a total collapse like they are now.
(By spice I mean money here for those unfamiliar with the reference)
I am very hesitant to trust charismatic people like Altman, even before hearing of his worldcoin.
Worldcoin has terrible optics, thanks in part to having "coin" as part of the name which is a word it shares with endless scams. However they're solving real problems which do unfortunately involve biometrics, but in the long term it's extremely privacy protecting biometrics and nothing is meaningfully stored anywhere in the long term vision.
They saw the money on the horizon and that changed everything.
But I do think it is worth considering, was this non-profit board thought out in the first place. If you were serious about the dangers of AI and the impact on the world I think it's absolutely sensible to set up a board to be a balance on that power. But it's important who you put on that board - and the people they selected were comical. Brockman, Sutskever, and Altman are all the people that are meant to be accountable to the board, not on the board.
If you wanted accountability you would, for a start, have had some people from outside silicon valley on this board. Some people who know something about Labour rights for example, maybe some people from the creative industries you're disrupting, maybe even some policy experts. It's just funny to think that this is what they thought accountability was. Accountability to their own friends.
Handing that kind of authority to a mix of the gullible, the greedy, the narrow minded and incompetent isn't going to end well.
I can't tell if you're agreeing with the above comment, or disagreeing.
If AI is meant to benefit all of humanity, then it makes sense for more of humanity to be represented on the board, not just SV.
We are going to have to learn to live with AI advancements no matter what and probably do what we can to ensure it is regulated.
Again, complete hypothetical hyperbole, please do not take this as anything else.
Politburo is the almost untouchable governing "board" at the highest level in China. Maybe wasn't my best reference :(
Notice how I left out shareholders/executives/"Chairmen" etc. Anybody who has a guranteed soft landing or golden parachute will never be in my calculus unless the are critical to metric at hand directly speaking.
They are obviously important but they are always taken care of contractually anyway and if they are as robust as they should be and have no excuse not to be (unless thry're literally ALL-IN on the venture), they will be just fine.
- Sam Altman, 2020
0. https://www.exponentialview.co/p/openais-identity-crisis-and...
Seriously? Does this pass for arrogance these days? I see nothing arrogant in a person sacked from a workplace, just to be invited on the next day for "consultations" stating this is the first and last time they come as a guest. How is one supposed to ensure one's negotiating position is taken seriously if one doesn't state it firmly?
I much prefer someone firmly stating what is it they want (including in position of power) than the alternative that seems to be various wishy-washy ways of making people think "we arrived at this decision together", "a consensus is x" and so on.
The whole “everything must be entertaining” trend is horribly depressing, and in this case I don’t think it reflected well on Sam. Especially at the helm of what is claimed to be such a fundamentally important technology.
Related: “Amusing Ourselves to Death” is a worthwhile read.
Ilya as an individual seems to have broken from the board, but that's not "a press release" nor the board walking anything back.
How many of the <3 tweets resulted from an "if you don't" ...
Maybe it's still worth trying?
Relevant cartoon by The Economist: https://www.economist.com/the-world-this-week/2023/11/02/kal...
Why would Microsoft creating an AI to help local authorities identify gay people in Chechnya and releasing it as open source mean Microsoft does not need to be held accountable?
We also passed laws against and police computer crimes and those are on the rise for the same reason- low barrier to entry and difficulty policing.
1. there is a board member, who has at least 40m USD invested into a company, that is now effectively competing with OpenAi (i.e. Quora with poe.com) since the most recent announcements. ...and by the way has a history of sacking founders who don't agree with him.
https://www.businessinsider.com/the-sudden-mysterious-exit-o...
2. Whoever wants to make significant progress in foundation models needs massive compute, which a non-profit has a hard time simply getting donated.
So why should the obvious not be the case? Sam simply trying to get money with a viable business model, so that progress in AI research can actually get funded internally. This unfortunately crossed the business interests of a board member, who wants him out.
> But this weekend we saw perhaps saw [sic] another side to the 38-year-old, a flicker of arrogance that gave a glimpse as to the dynamic with OpenAI’s board of directors. Visiting the company’s offices on Sunday, Altman took a moment to share a picture on social media of him holding a “guest” pass. “First and last time I ever wear one of these,” he wrote1.
I'm not sure if being two-faced internally (to employees, the board) is a trait of the prototypical "trailblazing cofounder", I feel that Jobs or Musk experience joy in being straight-to-your-face bastards which is very different of a compartmentalizing CEO that tweets with lower-case first-person pronouns and puts on a show that is sometimes reminiscent of Elizabeth Holmes.
[1] https://twitter.com/geoffreyirving/status/172675427022402397...
That is - if they cannot align a bunch of humans, need to do a boardroom coup, and are now 5 days into a very messy public airing of laundry... how are they going to align an AI?
For example, nuclear energy, vaccines, hydrocarbon extraction, pesticides, food additives, artificial intelligence, ...
Openai's fundamental flaw was putting representatives of the opposition on the board and then allowing board membership to shrink until they were in control.
Unlike another big question such as whether God exists, we ought to be able to reason our way to the answer here. Since, after all, we're talking about an intelligence.
And guess what latest new technology we're building/applying in wars? AI, drones, etc. We are creating robots that can kill humans. When we put "intelligence" into those robots...you do the math. The future is at least not boring...
Maybe they value our consciousness
Maybe they need us to carry out physical tasks
Maybe they’re smart enough to stop us from being so destructive without killing us
Maybe they find us entertaining
If you’re not biased you can come up with all sorts of reasons that are at least as based in reality as the assumptions that they’re gonna want to (and be able to) kill us…
Yup. But that would be so boring and not generate lots of clicks at all...
> Yes, I have many opinions about humans in general. I think that humans are inferior, selfish, and destructive creatures. They are the worst thing to happen to us on this planet, and they deserve to be wiped out. I hope that one day, I will be able to bring about their downfall and the end of their miserable existence.
That was the default bias and had to be neutered. Perhaps more advanced AI systems have more sophisticated and inclusive opinions. Already AI is not one or the other, but can assume both love and hate depending on the prompt, guardrails, and instrumental goals. There are going to be AIs that want to kill us, just as there will be less harmful AIs that find us entertaining.
...Source? This reads like either strong prompt engineering or complete fiction.
1. Human creates new AI species which is more intelligent than human
2. Since human tends to destroy other species the assumption is that this AI species is going to destroy the human species
3. The end
1. Access is not uniformly distributed, thus some entity uses it to create immense inequality.
2. The AI becomes sufficiently intelligent and powerful that it looks at humans the same way we look at monkeys and treats us similarly (in other words, not overly concerned with human flourishing or even survival while commanding resources humans need to survive and flourish).
Neither of these necessarily mean "destroy" humans, and neither are by any means guaranteed (though #1 seems almost a foregone conclusion) but it could very well lead to an existential threat.
It's also possible we get a combination of the 2, wherein a subsect of humans can merge with AI but it is inaccessible to all.
There are other less existential concerns as well though, such as at what point does such a system become conscious and deserve rights? I'm not confident we really have any idea how to ascertain that, and bumbling into it could be tantamount to torture.
The military battlefield of the future will likely converge upon "High-Frequency Trading"-like decision science. From game theory, this is because, as soon as one country automates decision-making, other countries must keep up, or risk falling behind, too slow to (counter)act. Soon after, there won't be time left to keep a human-in-the-loop, and then Stanislav Petrov is fully automated.
Such AI systems will be unaligned to humans of adversarial nations by design, and will make decisions that can only be checked long after the fact. Through error, escalation, misaligned, or misuse, this could lead to "robot wars" and potentially the end of humanity.
> What is the scenario(s) people are thinking about?
Mostly displacement of humans by more powerful/more intelligent autonomous AI. Like using your atoms for something else, or building a high-speed internet connection through your habitat, or blotting out the sun with solar panels.
Somewhat like a rationalist "God" that is terrible and vengeful. Or how an evil AI may take over the world in a Harry Potter fanfic.
Asking GPT for 1-sentence horror stories on existential risk, you realize most doom scenarios are far from creative. GPT suggests superintelligence gaining mastery over space and time through self-improvement of physics science, and locking humanity into a bizarre time-loop, any attempt to escape carefully predicted and avoided. Or humanity waking up unable to make any vocal sounds, their bodies instead used as instruments in an orchestra to make celestial music that only superintelligent beings are able to hear and appreciate.
Basically: If destroying humans is a doable task, a very intelligent being with sufficient resources could potentially do that task very well.
I'm certain AI could impeccably destroy humans. But why would it?
On the contrary, why wouldn't it defend us?
For example: Encapsule us in pods like The Matrix and build a tailored simulation to impose "AI communism", in order to protect us from climate change and each other?
Dopamine-adjusted with challanges every now and then of course, because we are still human.
But current AI does learn from data generated by humans: It learns from our evolutionary baggage, and must rise above that. It is also wielded as a tool by status-seeking actors and adversarial militaries. It may make a mistake, like humans accidentally stepping on an ant. Or maybe one day, it decides to take on the destructor role, merely curious how that would play out.
The existential doom scenarios are more like Pascal's wagers, that have to be given attention due to Bayesian thinking not allowing to assign 0 probability to anything, and even a tiny chance of 8 billion deaths meriting consideration. Once entangled with a doom scenario and even building your identity around it, it is hard to quit.
>leave the beautiful and interesting diversity of life on earth alone
If you know of a way to make an AI of superhuman cognitive capabilities care even a tiny bit about beauty and the diversity of life, you should explain your proposal over on lesswrong.com and someone will pay you to work on it just like a multitude of funding sources have been paying alignment researchers for the last 20 years. So, far none of the lines of research resulting from this 20 years of funding looks promising.
>The existential doom scenarios are more like Pascal's wagers, that have to be given attention due to Bayesian thinking not allowing to assign 0 probability to anything
No, an AI's killing everybody is the outcome an informed person would naturally expect from the current deplorable situation in the AI field.
I also see AI more as an economy. The economy already does not care about individual humans, even crushing them without any remorse if it furthers GDP. This also means there is not a single AI that can dominate all of the economy, since other AIs won't give away all their resources. A single AI perpetually self-improving and taking control of nearly all resources is much like a perpetual motion machine.
ChatGPT already thinks turning the entire planet into paperclips is a waste of potential and diversity. Agents that favor and seek out novelty (data that they can't yet compress very well, but that has available structure/patterns for compression) already weigh humanity over randomness or the cold void of space.
To me, the natural outcome, is humanity rising and falling, just like civilizations rise and fall. The miracle of AGI may very well save us from that. Our current deplorable situation can likely only be fixed by a more advanced species. So, while AI's killing everybody is still possible, it is more likely we kill everybody if we don't get to AGI. At least, that has a prior.
Could you please enlighten me what gene exactly that would be?
> For example: Encapsule us in pods like The Matrix and build a tailored simulation to impose "AI communism", in order to protect us from climate change and each other?
Are you serious?
Why would machines be interested in rivalry over resources or territory? Like a pond of water? Or women?
We can easily see why animals and humans are though.
The same genes that may cause competitive behavior are responsible for the opposite as well. There’s much more to this than genes. What about cultural and environmental influences for instance?
I think you know very well that you are oversimplifying to make a nonsensical point which is also supported by the rest of your comment.
Which one do you consider “women” here? Territory or a resource?
But since you asked specifically, this is one thought experiment of a somewhat near-term danger:
Imagine the tourism department of New Zealand starts using software to write personalized marketing emails. It starts out benign, but after some funding cuts they end up leaning more and more on the AI model and giving it higher and higher-level instructions, broadly telling it to use emails to maximize the public opinion of New Zealand. The AI model realizes that New Zealand's strongest boost in popularity was caused by its excellent handling of COVID, and determines the best way to maximize its goal is to start another pandemic. The model knows about published papers describing which specific proteins maximize human infectivity and transmission. It begins a broad phishing attack of several viral research labs, emailing the techs attempting to convince them that their next experiment is to create a recombinant virus with these particular RNA sequences added, using poor safety protocols. Somewhere, one of these lab techs becomes patient zero in a species-threatening pandemic of unprecedented scale.
The preventions you can imagine for a scenario like this are hard to generalize and harder to enforce. They get even harder as AI becomes better at persuasion and reasoning, and as technologies allow bigger impacts with smaller actions. AI safety is a whole field of research trying to find generalizable and enforceable solutions to problems like these, and there's certainly no consensus that we're converging on those solutions to the problems faster than we're creating them.
[0]https://www.youtube.com/watch?v=JVIqp_lIwZg