Author here. Wrote this after watching Lapsus$ post the Mercor archive on their leak site earlier this month. The thing that struck me is the combination: voice samples paired with ID document scans. Most breaches leak one or the other. This one ships a deepfake-ready kit. Tried to keep the writeup practical: what an attacker can actually do with this combo (banking voiceprint bypass, Arup-style video calls, insurance fraud), and a 5-step checklist for the contractors who were in the dump.
Happy to discuss the forensic detection side. AudioSeal
watermarks, AASIST anti-spoofing, and how the detection landscape changes
once voice biometrics start leaking at scale.
Interesting - thanks for the rabbit hole today. ;)
Mercer hasn't released many public statements over the incident. Social media posts aren't necessarily public; but I did find this breach notification sample filed with CA - https://oag.ca.gov/ecrime/databreach/reports/sb24-621099 . I guess we'll see if our legislators finally take data privacy seriously.
HSBC offered voice verification years ago and I just laughed and said nope.
I don’t even use biometrics on apple devices, I use a 6 digit pin.
It was always a stupid idea.
The thing about been willing to trade convenience for security is you get called paranoid and then when the other shoe does drop and you are still doing that you still get called paranoid for the current thing you are not doing that “everyone does”.
Assuming Apple is truthful on this matter (so far it seems so), Apple devices store a mathematical representation of the data, not the data itself (i.e. not a picture of your finger) and keep it only on device on a special hardware section designed for extra security. When apps ask for authentication, they can never inspect the data, they can only ask “does this match?”.
Even if you were somehow able to exfiltrate the data and find some way to transform it for something nefarious, you’d still need to first attack and bypass a specific hardware feature of the target’s device.
So sure, not having any representation of the data anywhere is technically more secure (maybe, as typing your code could be intercepted by a shoulder surfer or a camera), but biometrics on Apple devices are fundamentally not the same as having your raw data available on a random server somewhere.
> Self-audit your public audio footprint. Search YouTube, podcast directories, and old Zoom recording
This is suggestion #1 on your list of remediation steps for victims, but you didn't provide any information on how anyone would actually do that. How exactly would I search the internet for copies of my voice?
Please don't tell me the solution is giving an embedding of my voice to another third party.
Great question. There's no "reverse voice search" yet the way there is for images — that's genuinely a tool the world needs. In the meantime, the most useful thing is searching your name across YouTube and podcast platforms to map out what's already public. And for Mercor contractors specifically, the California AG
breach notice gives you a solid legal basis to request full deletion. Worth doing today.
One more data point for why sueing companies should lead to CEO getting prison time as well. And ideally invent some kind a of equivalent of pruson for non human persons like organisations.
Because right now the incentive to do what's right are so low. Taking a risk with other's people lives is becomming the norm for companies.
You could have seen this coming a mile away. So far I have gotten away with never uploading my ID and/or interacting with one of those companies (though one idiot working for some VC thought it was ok to sign a document on my behalf by uploading my signature!!, never mind a bit of fraud) but it is getting harder and harder. Banks and in some cases even governments forcing you to send data to these operators is a very bad idea. But hey, who ever got hurt by some security theater?
I've had to open a bank account for a company here a few years ago and that was right on the bubble of this happening and they still had an option to come by in person with the proper documentation, which I did, now it is all outsourced.
These companies are the fattest targets and they're run by incompetents. You should assume that anything you give them will eventually be part of some hack.
This kind of event is the best argument against needless data hoarding. But it would help if the law better provided for some kind of consequences for negligence.
Man that’s pretty shitty that Mercor tricked 40k contractors, and then did a poor job of securing their data. There should be stronger consequences for stuff like this.
I've been doing similar things on a different platform because as a uni student the pay is kinda nice, but I limit myself to task without voice/video and just input from mouse/keyboard to do reinforcement learning/data tagging. No way I'm trusting these companies or the companies they contract the work with
I wonder how many of the current text-to-speech ML models have large parts of leaked or "stolen" data in their training data? Almost none of the TTS releases seem to talk about exactly where they get their training data from, for some reason. I also wonder if we'll see an explosion in SOTA TTS in ~6 months from now.
>Set up a verbal codeword with family and finance contacts. Pick a phrase that has never been spoken on a recording and never typed in chat. Brief the people who handle money on your behalf. If a call ever asks for a transfer, the codeword is mandatory.
good luck with this. most finance people deal with hundreds to thousands of clients. they obviously cant remember everyones code word. commonly used finance systems arent setup to securely store these codewords. they dont have processes or policies in place to implement or adhere to any sort of codeword verification.
>Rotate where voiceprints are still in use. [...] Do that now, ideally from a new recording in a different acoustic environment than the leaked sample.
would this even have an effect? i have never heard of "rotating" a voice print. isnt the whole point of a voice print that you cant really change it? if simply switching your environment completely changes your voice print, that would make voice prints utterly useless to begin with.
> If you were a Mercor contractor and you believe your voice may already be in circulation, ORAVYS will analyze the first three suspect samples free of charge.
Awesome, if you're a victim of an AI company having your voice, you can help yourself by sending another AI company your voice!
> Audio is never used to train commercial models without explicit consent
I'm sure Mercor has explicit consent as well, legal teams are reasonably good at legally covering their asses with license terms.
Hi oefrha. I totally understand your point of view. The only thing is: we give you an opportunity to recognize how unique your voice actually is. We only analyze the audio file to show you how much uniqueness a voice carries. Audio is never used for training without explicit consent and is purged on a defined retention schedule. That is very important to us and also to let people know what their voice has to offer them for their growth and self awareness. Big difference wiTh what Mercor did.
So, they should all just rotate their voices ... right?
I jest but the majority of the "normal" people I know are happy to hand over biometrics because _it's easier_. We need to start branding biometrics as "forever passwords" or something to help people understand just what they're handing over when they validate access to their checking account or enter Disney World or whatever else.
The biometric pairing is what makes this particularly bad. A leaked password is recoverable. A leaked voiceprint combined with ID scans is permanent, you can not rotate your voice.
The deeper problem is that most of these companies collected this data because they could, not because they needed it for the core service. 'Datensparsamkeit' is the right frame: the voice samples were a liability sitting on a server waiting for exactly this.
they literally handed over their voice, their face, and their government id to train ai models for peanuts - and now lapsus is sitting on 4tb of 'you' that you can never change like a password
Mercor is the most scummy company out there, run by a bunch of sleazeball 20 somethings who are getting a lot of press as the youngest billionaires in the making.
Isn’t this going to immediately become daily news?
Half the time I call a company they say “we are recording your voice for security / authentication purposes”.
The companies that do that have all the information on me that they require for me to set up an account, so their data breaches will be just like this one, but 1000x larger.
Can we just fast forward through the part where this works for ID theft, past the firefox age verification plugin that uses these datasets, and even through the part where people in the plugin dataset are digital outcasts (this voice has been used too many times. Want to try another?)
At the end of this dark predictable tunnel, maybe there will be a ban on biometrics for important stuff, a repeal of the age verification laws, and actual privacy legislation with teeth.
I'm pretty sure Google and Apple already have some decent examples of a LOT of people's voices in concert with other data collation. Google Voice IIRC was bought for audio sampling voicemail in the first place. Not sure if Apple has done similar, but would be more surprised if they didn't... Let alone the voice search options for both.
61 comments
[ 5.1 ms ] story [ 73.3 ms ] threadMercer hasn't released many public statements over the incident. Social media posts aren't necessarily public; but I did find this breach notification sample filed with CA - https://oag.ca.gov/ecrime/databreach/reports/sb24-621099 . I guess we'll see if our legislators finally take data privacy seriously.
I don’t even use biometrics on apple devices, I use a 6 digit pin.
It was always a stupid idea.
The thing about been willing to trade convenience for security is you get called paranoid and then when the other shoe does drop and you are still doing that you still get called paranoid for the current thing you are not doing that “everyone does”.
Assuming Apple is truthful on this matter (so far it seems so), Apple devices store a mathematical representation of the data, not the data itself (i.e. not a picture of your finger) and keep it only on device on a special hardware section designed for extra security. When apps ask for authentication, they can never inspect the data, they can only ask “does this match?”.
Even if you were somehow able to exfiltrate the data and find some way to transform it for something nefarious, you’d still need to first attack and bypass a specific hardware feature of the target’s device.
So sure, not having any representation of the data anywhere is technically more secure (maybe, as typing your code could be intercepted by a shoulder surfer or a camera), but biometrics on Apple devices are fundamentally not the same as having your raw data available on a random server somewhere.
This is suggestion #1 on your list of remediation steps for victims, but you didn't provide any information on how anyone would actually do that. How exactly would I search the internet for copies of my voice?
Please don't tell me the solution is giving an embedding of my voice to another third party.
Because right now the incentive to do what's right are so low. Taking a risk with other's people lives is becomming the norm for companies.
Germans (because of course) have a word for this: "Datensparsamkeit". Being frugal with your data.
I've had to open a bank account for a company here a few years ago and that was right on the bubble of this happening and they still had an option to come by in person with the proper documentation, which I did, now it is all outsourced.
These companies are the fattest targets and they're run by incompetents. You should assume that anything you give them will eventually be part of some hack.
:)
good luck with this. most finance people deal with hundreds to thousands of clients. they obviously cant remember everyones code word. commonly used finance systems arent setup to securely store these codewords. they dont have processes or policies in place to implement or adhere to any sort of codeword verification.
>Rotate where voiceprints are still in use. [...] Do that now, ideally from a new recording in a different acoustic environment than the leaked sample.
would this even have an effect? i have never heard of "rotating" a voice print. isnt the whole point of a voice print that you cant really change it? if simply switching your environment completely changes your voice print, that would make voice prints utterly useless to begin with.
Awesome, if you're a victim of an AI company having your voice, you can help yourself by sending another AI company your voice!
> Audio is never used to train commercial models without explicit consent
I'm sure Mercor has explicit consent as well, legal teams are reasonably good at legally covering their asses with license terms.
I jest but the majority of the "normal" people I know are happy to hand over biometrics because _it's easier_. We need to start branding biometrics as "forever passwords" or something to help people understand just what they're handing over when they validate access to their checking account or enter Disney World or whatever else.
The deeper problem is that most of these companies collected this data because they could, not because they needed it for the core service. 'Datensparsamkeit' is the right frame: the voice samples were a liability sitting on a server waiting for exactly this.
Can't wait for them to crash and burn.
Half the time I call a company they say “we are recording your voice for security / authentication purposes”.
The companies that do that have all the information on me that they require for me to set up an account, so their data breaches will be just like this one, but 1000x larger.
Can we just fast forward through the part where this works for ID theft, past the firefox age verification plugin that uses these datasets, and even through the part where people in the plugin dataset are digital outcasts (this voice has been used too many times. Want to try another?)
At the end of this dark predictable tunnel, maybe there will be a ban on biometrics for important stuff, a repeal of the age verification laws, and actual privacy legislation with teeth.