> However, this ongoing incident has been tagged as an advisory, a flag commonly used to describe service issues typically involving limited scope or impact.
How is having Copilot breach trust and privacy an “advisory”? Am I missing something?
Advisory doesn't have the same meaning in security research as it does in the english language.
Unfortunately "Advisory" is a report written about a security incident, like an official statement about the bug, it's impact, and how to fix it -- which differs from the english meaning... it's not meant to mean to "advise" people or to "take something" under "advisory" (which, is a very soft statement typically).
The LLM that wrote this nearly content-free story doesn't know what it's talking about.
The basic distinction in the infosec industry is that advisories are what you publish to tell customers that you had a bug in your product that might have exposed them or their data to attacks and you want them to take some specific action (e.g., upgrade a package, review logs); while an incident report is what you publish when you know that the damage happened, it involved your infrastructure, and you want to share some details about happened and how you're going to prevent it from happening again.
Because the latter invites a lot more public attention and regulatory scrutiny, a company like Microsoft will go out of their way to stick to advisories whenever possible (or just keep incidents under wraps). It might have happened at some points in their history, but off the top of my head, I don't recall Microsoft ever publishing a first-party security incident report.
If you inflate severity, people simply ignore incident warnings.
What's the actual action needed here by a security team? None. You can hate it or not care but the end of the day there's no remediation or imminent harm, just a potential issue with DLP policies. Don't make it look like a 0-day that they actually have to deal with.
Reads to me like it is not accessing other users mailboxes, its just accessing the current user's mailbox (like its meant to) but its supposed to ignore current user's emails that have a 'confidential' flag and that bit had a bug
There are two issues I see here (besides the obvious “Why do we even let this happen in the first place?”):
1. What happened to all the data Copilot trained on that was confidential? How is that data separated and deleted from the model’s training? How can we be sure it’s gone?
2. This issue was found; unfortunately without a much better security posture from Microsoft, we have no way of knowing what issues are currently lurking that are as bad as —- if not worse than —- what happened here.
There’s a serious fundamental flaw in the thinking and misguided incentives that led to “sprinkle AI everywhere”, and instead of taking a step back and rethinking that approach, we’re going to get pieced together fixes and still be left with the foundational problem that everyone’s data is just one prompt injection away from being taken; whether it’s labeled as “secure” or not.
None of this should surprise anyone by now. You are being lied to, continually.
You guys need to read the actual manifestos these AI leaders have written. And if not them, then read the propagandist stories they have others write like The Overstory by Richard Powers which is an arrogant pile of trash that culminates in the moral:
humans are horrible and obsolete and all should die and leave the earth for our new AI child
Which is of course, horseshit. They just want most people to die off, not all. And certainly not themselves.
They don't care about your confidential information, or anything else about you.
calling it a bug is generous. the whole point of these tools is to read everything you have access to. the 'bug' is that it worked exactly as designed but on the wrong emails
Microsoft somehow sees a future where LLMs have access to everything in your screen. In that dystopia, adding "confidential" tags or prompt instructions to ignore some types of content is never going to be enough. If you don't want LLMs to exfiltrate content then they cannot have access to it, period.
Oh, poor desperate Microsoft. No amount of bug fixing is going to fix Microsoft. Now that they've embarked on the LLM journey they're not going to know what's going to hit them next.
All these government contractors are forced to pay astronomical cloud bills to get "GCC-High" because it passes the right security-theater checklist, and then it totally ignores the DLP settings anyway!
34 comments
[ 3.2 ms ] story [ 53.1 ms ] threadHow is having Copilot breach trust and privacy an “advisory”? Am I missing something?
Unfortunately "Advisory" is a report written about a security incident, like an official statement about the bug, it's impact, and how to fix it -- which differs from the english meaning... it's not meant to mean to "advise" people or to "take something" under "advisory" (which, is a very soft statement typically).
An advisory gives notice and/or warns about something, and may give recommendations on possible actions (but doesn’t have to).
The basic distinction in the infosec industry is that advisories are what you publish to tell customers that you had a bug in your product that might have exposed them or their data to attacks and you want them to take some specific action (e.g., upgrade a package, review logs); while an incident report is what you publish when you know that the damage happened, it involved your infrastructure, and you want to share some details about happened and how you're going to prevent it from happening again.
Because the latter invites a lot more public attention and regulatory scrutiny, a company like Microsoft will go out of their way to stick to advisories whenever possible (or just keep incidents under wraps). It might have happened at some points in their history, but off the top of my head, I don't recall Microsoft ever publishing a first-party security incident report.
What's the actual action needed here by a security team? None. You can hate it or not care but the end of the day there's no remediation or imminent harm, just a potential issue with DLP policies. Don't make it look like a 0-day that they actually have to deal with.
1. What happened to all the data Copilot trained on that was confidential? How is that data separated and deleted from the model’s training? How can we be sure it’s gone?
2. This issue was found; unfortunately without a much better security posture from Microsoft, we have no way of knowing what issues are currently lurking that are as bad as —- if not worse than —- what happened here.
There’s a serious fundamental flaw in the thinking and misguided incentives that led to “sprinkle AI everywhere”, and instead of taking a step back and rethinking that approach, we’re going to get pieced together fixes and still be left with the foundational problem that everyone’s data is just one prompt injection away from being taken; whether it’s labeled as “secure” or not.
You guys need to read the actual manifestos these AI leaders have written. And if not them, then read the propagandist stories they have others write like The Overstory by Richard Powers which is an arrogant pile of trash that culminates in the moral:
humans are horrible and obsolete and all should die and leave the earth for our new AI child
Which is of course, horseshit. They just want most people to die off, not all. And certainly not themselves.
They don't care about your confidential information, or anything else about you.
Trusted operating system Mandatory Access Control where art thou?
I assume that whatever that is processed by AI service are generally retained for product improvements (training).