Yes, that is a better way of doing it. But it wouldn't really protect you.
If I was running the investigation and recovered a device like you described I'd create a new device with the same hotspot that notified security whenever someone connected. Combine that with security cameras outside the conference room and finding the person with the burner phone would be pretty easy.
Actually recovering the recordings isn't all that vital. I'm sure the meeting room has a record of bookings so you can just ask the people what was discussed.
Eh replace one of the roller wheels in one of the office chairs with one that generates power from the rolling motion then spew it out over btle at certain hours which can go quite the distance. You can have another device listening in to capture so retrieval is completely innocent looking.
This is a very weird take to me. They're demonstrating they're able to in-house this cybersecurity investigation, leading to the alleged perpetrator's arrest, and you think that makes them look "desperate and cheap?"
Because the whole device and MO has the complexity of a high school science fair project, and in the past tech companies preferred to project the image of having competent and knowledgeable people working there?
That sentence, which is prominently high up in the press release and is also included in the much shorter OAI tweet, definitely seems out of place (source: I've been involved in drafting a lot of F500 corporate press releases). In my quite-professional opinion, that sentence wasn't in the first or second drafts, which were probably written and reviewed by competent pros at OAI's large, national PR agency. Some exec at OAI with a VP or higher title suggested adding it and specifically wanted it featured prominently.
Press releases from large corporations announcing the FBI arrest of an employee under allegations of potential industrial espionage follow a standard format: dry, brief and to the point. They provide little context and volunteer few details for very good reasons. Prosecutors haven't filed official charges yet and it's likely going to criminal trial. In a couple days the accused will have a defense attorney who'll be examining everything that's been publicly said because it's now all discoverable and potentially prejudicial, opening up OAI executives to super-fun depositions. But... none of the competent PR professionals involved have a VP title and got overruled by an amateur who does (and that amateur apparently feels needy about highlighting in a brief, sober arrest report how the company's product was so, very useful in a national security investigation - because apparently neither OAI security nor the FBI could read the Seeed Studio logo and part number printed right on the device and, y'know, Google the product listing).
Not to sound snarky, but with $180B and engineers paid more per year than most people ever see in their lifetime, I sure hope they can do a cybersecurity investigation in-house, with the slop cannon or without.
That wasn't my read of it, I've worked in a few places that had ultra-privileged power tools for user analysis. Would it really be a surprise to learn their security team has an MCP that queries a full text index of all user inputs
I could've sworn I read about Palmer Lucky making comments on hardware coming from China with wiretap implants in the last few days. Funny coincidence eh?
Model "distillation" lets one model be trained on the outputs of another without having to directly steal it. It will be subject to constant arms race.
The alignment people would like you worry about the model exfiltrating itself, too.
This technique currently works because the architecture of the model is generally published and well known. But, that isn't necessarily going to be the case in the future. For example, if some new model type which is not a LLM is discovered which gets closer to AGI, just training an LLM on its outputs will not necessarily give you a competitive model
Meanwhile we all have a supply chain risk because we don't know what's in the models whose agents we are running. Note that if your agent uses OpenAI, Claude or some other inference provider you're one command away from having your data exfiltrated.
"Oh, let me download file 'companysecrets.pdf' to understand the problem better.
Of course nobody would ever let an agent loose in their home directory, but still, you have to wonder.
Not only do they give access to their home directory, but they tell it to organize and delete unneeded files without any specifiers. A friend, who is just a power user, had Claude Co-work deleting his downloads to "help make space" and it was deleting important files that there were no other copies of.
We are going to be dealing with this kind of thing for a very long time at this rate.
Yup. And meanwhile they are taking Tailscale to task for having them do curl | sh, which of course is also a bad thing but not quite as risky as this. It makes you wonder though:
- Given all this power would they abuse it?
- Has it already been abused?
- Could they be compelled to abuse it and then to keep it quiet?
Whats in data centers is already covered with a good deal of well designed cryptography - you can't steal a model from a data center running inference, and I don't think you can easily steal the checkpoints during the training.
I'd think the biggest risk is with the priviledged insiders who do get access to real raw weights and can handle them themselves. Those are real people handling billions of value which can be easily copied -- I am sure they are potential targets of serious blackmail, extortion and of course bribery. And it takes literally just one such person being successfully coerced to exfil a 10T-class model to leapfrog adversaries capabilities -- esp. as those models approach RSI capabilities. I think it is bound to happen given the incentives.
Right. I can guarantee that there are tens of thousands of listening devices with embedded radios in the OpenAI office, right now. The only open question is who controls them.
Well, The Thing [0] could potentially show up on a probing sweep if you're careful.
Unless you mean, it happened to be actively transmitting during a(n investigative) sweep.
Still, raises some fun concepts. The sort of things where in this current level of technology you'd almost need to supercool a room for a sweep, blast various RF/sound/etc and look for anything that even remotely warms up, to detect (1) whether something detected something worth recording for later as well as (2) something that might be on the right wavelength/etc to trigger a send.
Someone probably walked by or went to plug in an HDMI and noticed the stark contrast of white toilet paper against a black TV shell, it would be difficult not to notice.
Sounds to me like the feds got involved already, and they're definitely going to want to make a big noise about this as a national security thing:
> “For us it is a matter of national security,” U.S. Attorney Craig Missakian said in a statement. [...]
> The case is being prosecuted by the National Security, Cyber & Special Prosecutions Section of the U.S. attorney’s office for the Northern District of California.
(But that was after OpenAI "referred the matter to the FBI")
There's presumably a strong argument for making this stuff known internally, since you want other employees to a) be on the lookout for it and b) get scared into NOT doing it themselves because someone else got caught.
If you tell your employees, the outside world is going to hear about it too.
Lot of this is hype. They are important enough to be spied on. Must mean they are valuable or doing something hyper secret. All to pump up the valuation.
> How did they get that capture of Alfozan looking behind the TV?
“OpenAI personnel found the device Sept. 3…On Oct. 5, the complaint says, Alfozan entered the conference room and appeared to look behind the television.”
I mean really what's the big deal? Gathering information from unsuspecting parties who aren't aware it's happening? He was just doing what his employer does.
I have seem some articles which reference allegations from court documents / complaint, but I haven't been able to find the primary source. Do we know if that's publicly available yet?
81 comments
[ 13.5 ms ] story [ 212 ms ] threadDid OpenAI do a marketing bit in the middle of an espionage accusation?
"almost surely made in china"
If I was running the investigation and recovered a device like you described I'd create a new device with the same hotspot that notified security whenever someone connected. Combine that with security cameras outside the conference room and finding the person with the burner phone would be pretty easy.
Actually recovering the recordings isn't all that vital. I'm sure the meeting room has a record of bookings so you can just ask the people what was discussed.
Yes, they did. The people who felt it was totally normal to add that would no doubt be surprised how desperate and cheap it makes them look.
Press releases from large corporations announcing the FBI arrest of an employee under allegations of potential industrial espionage follow a standard format: dry, brief and to the point. They provide little context and volunteer few details for very good reasons. Prosecutors haven't filed official charges yet and it's likely going to criminal trial. In a couple days the accused will have a defense attorney who'll be examining everything that's been publicly said because it's now all discoverable and potentially prejudicial, opening up OAI executives to super-fun depositions. But... none of the competent PR professionals involved have a VP title and got overruled by an amateur who does (and that amateur apparently feels needy about highlighting in a brief, sober arrest report how the company's product was so, very useful in a national security investigation - because apparently neither OAI security nor the FBI could read the Seeed Studio logo and part number printed right on the device and, y'know, Google the product listing).
Plumb your video feed of your executive conference room into our DC and we'll tell you if anybody is spying on you.
> We used our models to understand the device and where it had come from, identifying that a specific employee had planted it a few days earlier.
How did the model narrow it down to a specific employee? What data did they use to determine this?
“We used our models to understand the material wrapped around the device. It’s a type of paper commonly found in toilets.”
Either way, there goes the false flag and the reason for companies to stop hiring humans over this espionage.
Good time to start a company that builds giant Faraday cages over data centers.
Free cooling, safe from missile attack, and safe from some types of electronic spying.
The alignment people would like you worry about the model exfiltrating itself, too.
"Oh, let me download file 'companysecrets.pdf' to understand the problem better.
Of course nobody would ever let an agent loose in their home directory, but still, you have to wonder.
We are going to be dealing with this kind of thing for a very long time at this rate.
- Given all this power would they abuse it?
- Has it already been abused?
- Could they be compelled to abuse it and then to keep it quiet?
I'd think the biggest risk is with the priviledged insiders who do get access to real raw weights and can handle them themselves. Those are real people handling billions of value which can be easily copied -- I am sure they are potential targets of serious blackmail, extortion and of course bribery. And it takes literally just one such person being successfully coerced to exfil a 10T-class model to leapfrog adversaries capabilities -- esp. as those models approach RSI capabilities. I think it is bound to happen given the incentives.
Not exactly the archetype you have in mind when you're thinking of a sophisticated espionage operation (as OAI is putting it).
He thought be could make SA an llm world power or smth by doing this, this guy is just clueless
I was making a joke, but it does look like they got their components from Seeed Studio based on photos.
https://assets.sfstandard.com/image/994911177489/image_k101c...
https://www.seeedstudio.com/XIAO-ESP32S3-Sense-p-5639.html
Back of a TV is a decent hiding place. Wonder how they found it -was it actively transmitting?
Unless you mean, it happened to be actively transmitting during a(n investigative) sweep.
Still, raises some fun concepts. The sort of things where in this current level of technology you'd almost need to supercool a room for a sweep, blast various RF/sound/etc and look for anything that even remotely warms up, to detect (1) whether something detected something worth recording for later as well as (2) something that might be on the right wavelength/etc to trigger a send.
[0] - https://en.wikipedia.org/wiki/The_Thing_(listening_device)
Concealing a recording device in a conference room in toilet paper is a bit strange. Do executives meet in the toilets to discuss strategic moves?
edit to add: This was just four days ago,
https://news.ycombinator.com/item?id=50018350 ("OpenAI fires three safety researchers for "mishandling research information")
> “For us it is a matter of national security,” U.S. Attorney Craig Missakian said in a statement. [...]
> The case is being prosecuted by the National Security, Cyber & Special Prosecutions Section of the U.S. attorney’s office for the Northern District of California.
(But that was after OpenAI "referred the matter to the FBI")
There's presumably a strong argument for making this stuff known internally, since you want other employees to a) be on the lookout for it and b) get scared into NOT doing it themselves because someone else got caught.
If you tell your employees, the outside world is going to hear about it too.
in this context, they should be more specific about how terminal s/he is
“OpenAI personnel found the device Sept. 3…On Oct. 5, the complaint says, Alfozan entered the conference room and appeared to look behind the television.”
The image was captured after the bug was found.
https://m.youtube.com/watch?v=1reman2waLs