It's an age old question. Do you not build the thing that will destroy the world while hoping nobody else does as well, or do you build it first so that you can "control" it?
Yep they made the exact same claim about GPT-2. Back then language models talked like markov chains. I guess OpenAI's "fake it till you make it" strategy was so successful that they can't stop even after they've already made it
Are you so deeply set in this mindset/idea that OpenAI is dishonest that you can't imagine that they might just be doing what they say they are doing?
Would you be willing to say literally anything I ask you to to anyone I specify, including telling your family that you have been kidnapped and need money?
Sometimes, when someone tells you they are concerned about something, that something is concerning. We should evaluate the actual concern. I am concerned you do not realize this.
I don’t think NavinF shows any lack of concern for the issue as such. What’s being talked about here is whether OpenAI’s being forthright concerning their motives for keeping the tech closed.
I think we can all agree it is valid to be cautious with some of this tech, but the question is whether OpenAI is being cautious or simply using caution as PR. And given their track record, the latter seems, if not likely, then possible.
Their action of removing the ban on military use of their technology has made me disbelieve their claims.
Couple that with deceptive naming practices ('Open' AI) and how much they've compromised their independence with so much Microsoft ownership.. why _would_ you take their claims at face value?
Regarding the kidnapping scenario, my family isn't stupid enough to send money to anyone that sounds like me. Impersonating the gov't and sending letters would be a much better strategy since it's much harder to verify an organization's identity. Btw voice cloning is fairly mainstream and I've already used free voice cloning apps to humorous effect. Do you think people can't clone your voice today thanks to OpenAI holding back?
moderately convinced openai is a huge scam by now.
I think they've hit a plateau or something. Compare the recent Sam Altman interview by Lex Fridman with the one only a few months prior. The dude's in limbo. You can almost hear the legal team breathing down his neck not to make unsubstantiated, deceptive claims... They got high on their own supply and now have to cut their product with ground glass to make the same dose hit harder. There is some uncomfortable AI sobriety incoming.
I usually would've commented something along the lines of how "open" their AI is, but to be fair, this is a pretty understandable move. Sadly, it's only a matter of time before someone else will offer this feature to the public. So "open"AI is only slowing down the process.
I don't see it as sad. Has the world fallen apart now that GPT-4 and DALL-E exist? Hardly. It's not going to collapse when we have easy voice cloning either.
And despite some commenters who lack imagination saying "I can only think of bad uses for this" there are tons of good potential uses. Video editing, game voiceovers, music, phone menus, audio books, etc.
That was my impression too. Maybe OpenAI’s model is impressively better than anything out there right now, or they’re just embarrassed cause it’s so much worse :P
Potentially unpopular opinion- this may have been done in part with the objective of enforcing regulations or setting up AI laws. Given OpenAI’s position that comes across as regulatory capture.
Instead of phrasing it as stating an unpopular opinion, try asking a question with an unpopular answer instead, that way someone else takes the downvotes on your behalf.
It is an Easter miracle, whom amongst us has not prayed for 500+ words on some software that may or may not exist but certainly has zero impact on anything in the world?
It's already happening. There have been AI ads targeting vulnerable, elderly populations for months, promising free Social Security cash in the likeness of our current POTUS:
Ads are 100% blocked on my devices, so I wouldn't have been aware if not for a recent trip to the doctor's office where separate ones played on two old men's phones at full volume.
I had a job about a month or two ago on Upwork which was supposedly going to be about a character.ai clone or something similar. After discussing it for weeks, on the last day they decided it wasn't about personas in general and absolutely had to be impersonation of Trump and Biden.
I told them it was a very questionable idea.
But I needed the money to be able to make rent. So I took the job hoping that they were still interested in other types of characters and would be reasonable.
I had a basic demo with Trump's voice up in about a week. I showed them the first demo and had a phone call suggesting that this might actually be illegal pretty soon.
They were sure there was no problem and said they had spoken to lawyers.
The next day, Eleven Labs sent out an email explaining that some voices were getting banned. The attempt to clone Biden's was blocked by Eleven Labs.
They had structured the contract as a single fixed fee, although what we had discussed was weekly payments. At the last minute without discussion they had thrown in a talking head requirement.
So I received the first 451 HTTP status code I have ever seen when requesting that D-ID do a lip sync animation of Trump's head.
I ended up using a different video model from replicate that was not a great lip sync but very engaging if you squint. I did not do Biden. Did about three weeks of work and collected around $630 after Upwork took their fee.
I'm proud to say I did make my rent payment though since I had another project at the same time which was a low hourly but still helped.
Given how good the newly released available-weight VoiceCraft[1] project seem to be, I would say the cat is already out of the bag.
People will soon have to learn that they should not trust voiced content. In regards to scam prevention, the key learning will to validate the security of the channel and never trust a voice alone.
If someone you know calls you from an established channel (their usual account / phone etc), it's probably safe. But any unauthenticated channel communication (unknown account etc) should be treated as suspicious.
build up for a mole leaking the code or a cyberattack originating in eastern europe or russia or a combination of both, thus including corruption and creating the need for stronger oversight of people working on AI by some clandestine agency and of course extreme regulation of open source AI and people capable of writing that kind of code, which would include extreme regulation of access to the training data and so on ... the interest groups in power censored the training data, took openai under their wing, and the fun continues. sorry, couldn't resist.
50 comments
[ 1.6 ms ] story [ 114 ms ] threadGain of knowledge, and function.
It's an age old question. Do you not build the thing that will destroy the world while hoping nobody else does as well, or do you build it first so that you can "control" it?
Yes it's a meme. But it's the summary of OpenAIs public demeanor
Would you be willing to say literally anything I ask you to to anyone I specify, including telling your family that you have been kidnapped and need money?
Sometimes, when someone tells you they are concerned about something, that something is concerning. We should evaluate the actual concern. I am concerned you do not realize this.
I think we can all agree it is valid to be cautious with some of this tech, but the question is whether OpenAI is being cautious or simply using caution as PR. And given their track record, the latter seems, if not likely, then possible.
Couple that with deceptive naming practices ('Open' AI) and how much they've compromised their independence with so much Microsoft ownership.. why _would_ you take their claims at face value?
Regarding the kidnapping scenario, my family isn't stupid enough to send money to anyone that sounds like me. Impersonating the gov't and sending letters would be a much better strategy since it's much harder to verify an organization's identity. Btw voice cloning is fairly mainstream and I've already used free voice cloning apps to humorous effect. Do you think people can't clone your voice today thanks to OpenAI holding back?
Maybe they just wait for somebody else to take the brunt of the reputation damage.
I think they've hit a plateau or something. Compare the recent Sam Altman interview by Lex Fridman with the one only a few months prior. The dude's in limbo. You can almost hear the legal team breathing down his neck not to make unsubstantiated, deceptive claims... They got high on their own supply and now have to cut their product with ground glass to make the same dose hit harder. There is some uncomfortable AI sobriety incoming.
And despite some commenters who lack imagination saying "I can only think of bad uses for this" there are tons of good potential uses. Video editing, game voiceovers, music, phone menus, audio books, etc.
Wiki article on the topic: https://en.wikipedia.org/wiki/Regulatory_capture
https://www.economist.com/films/2024/01/12/elections-around-...
https://www.newsweek.com/tik-tok-scam-social-security-biden-...
Ads are 100% blocked on my devices, so I wouldn't have been aware if not for a recent trip to the doctor's office where separate ones played on two old men's phones at full volume.
I told them it was a very questionable idea. But I needed the money to be able to make rent. So I took the job hoping that they were still interested in other types of characters and would be reasonable.
I had a basic demo with Trump's voice up in about a week. I showed them the first demo and had a phone call suggesting that this might actually be illegal pretty soon.
They were sure there was no problem and said they had spoken to lawyers.
The next day, Eleven Labs sent out an email explaining that some voices were getting banned. The attempt to clone Biden's was blocked by Eleven Labs.
They had structured the contract as a single fixed fee, although what we had discussed was weekly payments. At the last minute without discussion they had thrown in a talking head requirement.
So I received the first 451 HTTP status code I have ever seen when requesting that D-ID do a lip sync animation of Trump's head.
I ended up using a different video model from replicate that was not a great lip sync but very engaging if you squint. I did not do Biden. Did about three weeks of work and collected around $630 after Upwork took their fee.
I'm proud to say I did make my rent payment though since I had another project at the same time which was a low hourly but still helped.
https://slate.com/technology/2019/02/openai-gpt2-text-genera...
People will soon have to learn that they should not trust voiced content. In regards to scam prevention, the key learning will to validate the security of the channel and never trust a voice alone.
If someone you know calls you from an established channel (their usual account / phone etc), it's probably safe. But any unauthenticated channel communication (unknown account etc) should be treated as suspicious.
[1]: https://github.com/jasonppy/VoiceCraft
Some more discussion on official post: https://news.ycombinator.com/item?id=39866493