I remember reading a story on reddit where children for their mother's Mother's day created deep fake of their passed grandparents, hoping it would make their mother happy...yet their mothers reaction was more creeped out over happiness.
I know plenty of people that keep voicemails and even voicemail greetings from dead loves ones. I can see how this could be an extension of that for a user behavior.
Its happenstance that they chose to market it this way, it's just a voice filter that can be used for anything. But they are going for "empathy" and for people that don't already have an Alexa yet.
There's a big difference between keeping a memento that's really from those dead relatives, and resurrecting an aspect of those dead relatives through technology.
I'm sure it's technologically very impressive, but why do they choose to market it as a form of necromancy? There are so many less-inappropriate takes they could have chosen, like defeating voice print security or something.
Movies have been doing this for a while, and people just seem to have gotten used to it after the initial "yuck" reaction. Peter Cushing, Coke Commercials, etc.
If a total war breaks out, we'll all be getting phone calls from our "relatives" tell us to do whatever $CURRENT_POLICY says.
If not, in a few years advertisements will be narrated by someone who's voice eerily sounds like that of your mother or father. I too love Empathy®. It produces great value for the shareholders.
Did anyone reading this think back to the TV series Caprica [1], where Zoe Graystone [2] is digitally resurrected and her digital mimic speaks with her living uncle?
I used to not focus on the AI ethics concerns, mainly because I think the ethics of subsequent AIs will have to match the ethics of whatever true AGI is created first in order to survive (that first AI will have so much of an evolutionary advantage), and I thought the first one would be created by OpenAI or some such with decent ethics. Now I think I was naïve, and wonder if the worry over AI ethics being more important than I thought, if something like the Cylons or Skynet is more probable than I thought.
Okay, so they're showing off the ability for a system to analyze a voice sample and mimic the voice it 'heard' with a reasonable level of fidelity. Intellectually interesting, has some potentially good use cases (IVR call trees, more distinct virtual assistant 'personalities', voice interfaces that seem 'familiar' to dementia patients and the like...)
But they choose to present it in the context of perhaps the most ghoulish use case possible? And not only a use case that's entangled with taboos surrounding death and respect for the dead, but one that's been openly lampooned in popular media franchises like Black Mirror? I'm failing to comprehend why this was the chosen approach. Perhaps because it's controversial enough to distract from the implications for more actively malevolent uses of deepfakery?
> The company demoed the feature at its annual MARS conference, showing a video in which a child asks Alexa to read a bedtime story in the voice of his dead grandmother.
> “As you saw in this experience, instead of Alexa’s voice reading the book, it’s the kid’s grandma’s voice,” said Rohit Prasad, Amazon’s head scientist for Alexa AI. Prasad introduced the clip by saying that adding “human attributes” to AI systems was increasingly important “in these times of the ongoing pandemic, when so many of us have lost someone we love.”
> “While AI can’t eliminate that pain of loss, it can definitely make their memories last,” said Prasad. You can watch the demo itself below:
Sometimes I wonder who is approving these presentations. Like who saw the brief for this and said "yep, that sounds like a good idea!". I fully expected this article to be clickbait and/or be misrepresenting the facts but no, instead it's coming awfully close to Black Mirror's "Be Right Back" episode.
My grandma actually recorded herself reading books to me as a child and I cherried those cassette tapes and listened to them over and over again. To the point that in early elementary school I "read" a book for "show and tell" that surprised my teachers (reading at that level, for my age). Turns out I had just memorized it (a story I've seen repeated online more than once). Those recordings are special, this feels gross. Alexa is not making their memories last, it's a lie, it's fake, it's not real, and it borders on delusional.
Bruh, this is not cool for the hustle when getting hard-earned money. Clever people could use Alexa to steal money from banks when mimicking voices. Better get those bank workers trained to detect mimicked voices from Alexa.
16 comments
[ 3.3 ms ] story [ 44.0 ms ] threadI remember reading a story on reddit where children for their mother's Mother's day created deep fake of their passed grandparents, hoping it would make their mother happy...yet their mothers reaction was more creeped out over happiness.
Wow. Words fail me.
Its happenstance that they chose to market it this way, it's just a voice filter that can be used for anything. But they are going for "empathy" and for people that don't already have an Alexa yet.
I'm sure it's technologically very impressive, but why do they choose to market it as a form of necromancy? There are so many less-inappropriate takes they could have chosen, like defeating voice print security or something.
If not, in a few years advertisements will be narrated by someone who's voice eerily sounds like that of your mother or father. I too love Empathy®. It produces great value for the shareholders.
I used to not focus on the AI ethics concerns, mainly because I think the ethics of subsequent AIs will have to match the ethics of whatever true AGI is created first in order to survive (that first AI will have so much of an evolutionary advantage), and I thought the first one would be created by OpenAI or some such with decent ethics. Now I think I was naïve, and wonder if the worry over AI ethics being more important than I thought, if something like the Cylons or Skynet is more probable than I thought.
[1] https://en.wikipedia.org/wiki/Caprica
[2] https://en.battlestarwikiclone.org/wiki/Zoe_Graystone
But they choose to present it in the context of perhaps the most ghoulish use case possible? And not only a use case that's entangled with taboos surrounding death and respect for the dead, but one that's been openly lampooned in popular media franchises like Black Mirror? I'm failing to comprehend why this was the chosen approach. Perhaps because it's controversial enough to distract from the implications for more actively malevolent uses of deepfakery?
> “As you saw in this experience, instead of Alexa’s voice reading the book, it’s the kid’s grandma’s voice,” said Rohit Prasad, Amazon’s head scientist for Alexa AI. Prasad introduced the clip by saying that adding “human attributes” to AI systems was increasingly important “in these times of the ongoing pandemic, when so many of us have lost someone we love.”
> “While AI can’t eliminate that pain of loss, it can definitely make their memories last,” said Prasad. You can watch the demo itself below:
Sometimes I wonder who is approving these presentations. Like who saw the brief for this and said "yep, that sounds like a good idea!". I fully expected this article to be clickbait and/or be misrepresenting the facts but no, instead it's coming awfully close to Black Mirror's "Be Right Back" episode.
My grandma actually recorded herself reading books to me as a child and I cherried those cassette tapes and listened to them over and over again. To the point that in early elementary school I "read" a book for "show and tell" that surprised my teachers (reading at that level, for my age). Turns out I had just memorized it (a story I've seen repeated online more than once). Those recordings are special, this feels gross. Alexa is not making their memories last, it's a lie, it's fake, it's not real, and it borders on delusional.
Read off this script in my neighbor's voice so we can convince this bank clerk on the phone to move their funds to another account.