8 million active users after almost 2 years from making the $30pupm license available. Less than 2% of Microsoft 365 paying customers choosing to pay extra for Copilot. It's not difficult to see why Microsoft themselves have stopped reporting on AI revenue, as well as not disclosing any official numbers on M365 Copilot sales. Luckily, a source leaked these figures for Ed Zitron to report in his newsletter.
Yes I regularly use it for things like “I remember communicating about subject X in 2023, please find all references to that topic since that date”, and it will search and summarize all Teams, Outlook and OneDrive datapoints that I have access to in the organization and neatly summarizing it and linking directly to said datapoints.
I also use it to play around with data and roughly graph it to understand trends without having to roll out a Jupyter notebook.
Yes. Worthwhile in terms of 'worth using it', not 'worth 30$ a month' though.
What I like:
- Meeting catch-up (I don't use meeting minutes much)
- Finding stuff in my messy uncategorised mailbox
- Finding stuff in sharepoint
Things that were a heavy disappointment:
- Everything in excel
- Rewriting text (it's always too noticeable that it's AI, it should be able to learn and match my style)
- Powerpoint presentation generation. It could not modify for a long time and even now it's just not useful
Tbh I only use it a few times per month where it's actually useful. For other things I've tried but it needs so much explaining to get it to do exactly what I want that it's more efficient to just do it myself.
I still think there's a potential niche for 365 Copilot in its boringness.
At work (not a tech company), there's an ongoing, slow, 365 rollout. The people who participate in the rollout are not technical in any way, but they all love it because they're not regular ChatGPT/Claude users, either. In a way, the restricted feature-set of Copilot compared to ChatGPT helps them, they're overwhelmed enough by Copilot.
IT loves 365 because it's so risk-averse. No big jumps, no surprises, clearly defined data and risk policies.
I think if they drill down into the boring, slow, predictable, they will capture the market of risk-averse non-tech companies, not people.
I've tried building agents using 365 for our internal documents and they're OK for basic stuff (what's in what document where - max 20 documents only!!), but langchain/RAG/whatever are a million times more powerful.
> IT loves 365 because it's so risk-averse. No big jumps, no surprises,
Umm no, it's the opposite. It's super high-risk right now for us. Microsoft is constantly shifting stuff around leading us to have to constantly change our processes, documentation etc. Often with zero heads-up and often defaults to on. Some incidents:
- They suddenly started a "free promotion" with Sharepoint agents. We don't want to offer that to our users but it just popped in one day and the admin setting defaulted to on so people were already using it before we turned it off. This was a big deal for us.
- Constant rebranding of their product names leading to confusion among users and zero-value documentation and process rewrites for us. Also constantly fiddling with the URLs is sooo annoying.
- Constant changes in features leading to impact to our DPIA. For example copilot chat didn't have history at first. So no data was kept (they also promise they don't store any for training). Suddenly they added that one day, so we had to redo our entire DPIA because it now does suddenly store personal info which it didn't before so a whole lot of overhead comes into scope (data lifecycle, privacy regulations, security, data loss prevention etc). This is exhausting and there is no way to delay these features until we have approved them. Also, it caused our DPIA team to be highly critical after this incident. Because of course: If they did this before, what guarantees that they won't change something worse next month?
- Limited granularity of access controls - a lot is very high-scope on/off style controls. Meaning that if we want to block something we often block unintended features as well.
A lot of these things are definitely 'big jumps' and 'surprises'.
What about the secondary effects of AI? I mean, now large-scale data gathering on your company simply by recording glasses + having AI transcribe the whole thing is possible. Hell, with modern cell phones you could do it live.
And the first product that lets a cell phone control keyboard and mouse sending camera to ChatGPT, having ChatGPT do all the work is not far off either.
Presumably both of these would violate your ... policies something awful. I should say will violate all your policies, because we both know this is going to happen.
You have no way of preventing both of these from occurring with IT policy.
Unsurprising. I tried it probably a year ago. I asked it what meetings were in my calendar for the day and it couldn’t even tell me that. To add insult to injury, they wanted an annual commitment with up front payment at the time.
Microsoft's moat and cash cow is Active Directory.
If corps ever stop needing that, MS will be in trouble. Until then, few worries. They can play at following fads (not that I think AI is a fad) and not worry about execution.
I perceive MS as just being in the "get people using it, accustomed to it, used to it, habituated to using it throughout their day" phase for the default free Win10/11 tier.
That little copilot icon is sitting there in the task tray, clickable and usable. I think people will, to whatever degree makes sense for them, get used to making some use of the basic free tier. Monetization / restrictions requiring some degree of limitations to upsell will eventually come for the currently half-billion or so Win11 users.
I get the point that few are currently paying for it, but there isn't all that much pressure to do so right now. That will come, later.
> I get the point that few are currently paying for it, but there isn't all that much pressure to do so right now. That will come, late
Yes that is inevitable. But it's exactly why I won't use it. It feels like a scam, the typical drug dealer model. Get them hooked on free stuff, and then milk their addiction.
I almost only use local LLMs now. They're not as good, no. But at least they are mine and nobody will raise the price suddenly. Some pro AI models I just use with API, for the rare instance where local just doesn't cut it. It costs me a few bucks per month that way, not 20 or 30.
It’s garbage. My company has it, presumably by default. You ask it a simple question like “find the company holidays document” and it just can’t do it. It’s somehow not hooked up to the basic sharepoint document storage for the company.
Then you probably don't have the paid version. The free version ("Copilot 365 *chat*") does not have the ability to search in any company information even in M365.
If you see a work and web tab at the top, click work. Then it should be able to. If you don't see this, your company is not paying and it just won't work.
I used M365 Copilot at work, and it's amazing at summarizing discussions in Teams meetings, and terrible at accessing cells in a spreadsheet from which it's called. I'm guessing that will improve with time and a few releases.
I am just a typical Office user in a small company environment. Copilot buttons started popping in various Office applications. When I asked Copilot in Outlook to summarise mails from one of my contacts it happily searched the Internet and said it found nothing. AAA experience.
The most pathetic aspect of Microsoft’s Copilot rollout is not Copilot’s poor performance & integration with SharePoint, but that Microsoft actually renamed Microsoft Office to Microsoft 365 Copilot on Windows.
You’re a company whose main product has a brand recognition that ranks right up there with Coca-Cola and Apple, and you rename it after a new product that is a pile of hot garbage. Absolutely mind-boggling.
It’s an annual commitment by default. Admins might enable it if it was monthly. However, you can’t risk one license locking you into an M365 contract. Most admins can’t and shouldn’t make that decision. Failure on Microsoft’s part for the annual requirement. Otherwise, most would toggle the license on and give it a try.
Yeah what we did for a while was just have a bunch of licenses and rotate them through the org, getting people's feedback after a month or so.
Of course all the people asking for these were already very interested in AI so they are not a very representative group. We did take that into account.
1.8% is terrible adoption, especially in light of MSFT hijacking the right-CTRL key of every Windows licensed PC to rename it "CoPilot" key. I'm thrilled that Copilot is sucking wind because their over-aggressive pushiness about CoPilot in Windows and Office has been annoying and wasted my time having to figure out how to disable it but the REAL reason I despise CoPilot is that while stealing a key on my laptop's keyboard, MSFT literally broke the key permanently.
They changed how that key works at a low level so it cannot be cleanly remapped back to right-CTRL. This is because, unlike the CTRL, ALT, Shift and Windows keys, the now-CoPilot key no longer behaves like a modifier key. Now when you press the CoPilot key down it generates both key down and key up events - even when you keep it pressed down. You can work around this somewhat with clever key remapping in tools like AutoHotKey but it is literally impossible to fully restore that key back so it will behave like a true modifier key such as right-CTRL in all contexts. There are a limited number of true modifier keys built into a laptop. Stealing one of them to upsell a monetized service is shitty but intentionally preventing anyone from being able to restore it goes beyond shitty to just maliciously evil.
I agree. Making deals with laptop manufacturers to put their trademark on keyboards is an unhealthy-for-society abuse of their market power and should be treated as such in every jurisdiction that they trade in.
However, I've had no issue remapping it to the Context Menu key on the Chinese Lenovo Xiao Xin laptop I bought recently. It shows up in the PowerToys keyboard remapper tool as an F23 key. (Yes, there are more than 12 F key codes! I believe there is 24 in Windows.)
Other OSes will not even be able to say things like "Press your Copilot key to open FEDORA SEARCH" or whatever without being cease'n'desisted to high heaven.
It's being shoved down our throats at work, which is really annoying. Beyond being able to search for documents, there is little utility beyond what a local <1B param LLM gives me.
Even the "productivity" suggestions in teams are ridiculous - one of them is "roast me by my calendar"...wtf is that? How is that related to productivity at all?
Github Copilot is a pain too. Somehow they are capable of building potentially world changing LLM's, but can't build a working subscription billing system? I'm completely locked out of my Copilot access, that I paid for, with no response from support. Whole experience has somewhat soured me on relying on Microsoft for any critical part of my workflow if the response when things go wrong is complete silence.
Put the LLM on answering (or at least triaging) your support tickets faster.
> Somehow they are capable of building potentially world changing LLM's
Copilot doesn't run on Microsoft's own LLMs, they're all third party. ChatGPT in office copilot, in github you can choose various other ones too.
Microsoft is really just a reseller there.
Personally I think Claude Code (with Opus in particular) is far better. Unfortunately they are very aware of this and thus it is eyewateringly expensive.
That's strange, my experience with GitHub and GH Copilot have been great, unlike with other Microsoft products that have their billing integrated in the Microsoft account or MS365, or Entra or whatever they call the "company accounts" today.
Plus for 10$ a month (~8.50 EUR last month) I get more-than-enough-for-me access to Gemini, Claude and OpenAI models from my code editor. I wouldn't pay double for Claude Code or Cursor, so I feel like it's a good enough deal.
Since when has Microsoft had a hit product before version 3? It's simply not in their corporate DNA.
Besides, I just recently saw a review of the newest Copilot for Excel by a serious spreadsheet jockey, and he was truly blown away. Basically, it knows everything Excel can do, how to do it, and it now truly understands numbers as numbers and when to use formulas, etc. etc. Basically, it went from smart automation to full on AI.
If Microsoft doesn't fall for short term thinking, the product will eventually succeed.
54 comments
[ 3.1 ms ] story [ 67.3 ms ] threadCopilot is often wordy, wrong, and seems like it can't do a lot of the same things I can accomplish without logging in on ChatGPT
something, something, clown world
I also use it to play around with data and roughly graph it to understand trends without having to roll out a Jupyter notebook.
I find it quite useful.
What I like:
- Meeting catch-up (I don't use meeting minutes much)
- Finding stuff in my messy uncategorised mailbox
- Finding stuff in sharepoint
Things that were a heavy disappointment:
- Everything in excel
- Rewriting text (it's always too noticeable that it's AI, it should be able to learn and match my style)
- Powerpoint presentation generation. It could not modify for a long time and even now it's just not useful
Tbh I only use it a few times per month where it's actually useful. For other things I've tried but it needs so much explaining to get it to do exactly what I want that it's more efficient to just do it myself.
At work (not a tech company), there's an ongoing, slow, 365 rollout. The people who participate in the rollout are not technical in any way, but they all love it because they're not regular ChatGPT/Claude users, either. In a way, the restricted feature-set of Copilot compared to ChatGPT helps them, they're overwhelmed enough by Copilot. IT loves 365 because it's so risk-averse. No big jumps, no surprises, clearly defined data and risk policies.
I think if they drill down into the boring, slow, predictable, they will capture the market of risk-averse non-tech companies, not people.
I've tried building agents using 365 for our internal documents and they're OK for basic stuff (what's in what document where - max 20 documents only!!), but langchain/RAG/whatever are a million times more powerful.
Umm no, it's the opposite. It's super high-risk right now for us. Microsoft is constantly shifting stuff around leading us to have to constantly change our processes, documentation etc. Often with zero heads-up and often defaults to on. Some incidents:
- They suddenly started a "free promotion" with Sharepoint agents. We don't want to offer that to our users but it just popped in one day and the admin setting defaulted to on so people were already using it before we turned it off. This was a big deal for us.
- Constant rebranding of their product names leading to confusion among users and zero-value documentation and process rewrites for us. Also constantly fiddling with the URLs is sooo annoying.
- Constant changes in features leading to impact to our DPIA. For example copilot chat didn't have history at first. So no data was kept (they also promise they don't store any for training). Suddenly they added that one day, so we had to redo our entire DPIA because it now does suddenly store personal info which it didn't before so a whole lot of overhead comes into scope (data lifecycle, privacy regulations, security, data loss prevention etc). This is exhausting and there is no way to delay these features until we have approved them. Also, it caused our DPIA team to be highly critical after this incident. Because of course: If they did this before, what guarantees that they won't change something worse next month?
- Limited granularity of access controls - a lot is very high-scope on/off style controls. Meaning that if we want to block something we often block unintended features as well.
A lot of these things are definitely 'big jumps' and 'surprises'.
And the first product that lets a cell phone control keyboard and mouse sending camera to ChatGPT, having ChatGPT do all the work is not far off either.
Presumably both of these would violate your ... policies something awful. I should say will violate all your policies, because we both know this is going to happen.
You have no way of preventing both of these from occurring with IT policy.
If corps ever stop needing that, MS will be in trouble. Until then, few worries. They can play at following fads (not that I think AI is a fad) and not worry about execution.
That little copilot icon is sitting there in the task tray, clickable and usable. I think people will, to whatever degree makes sense for them, get used to making some use of the basic free tier. Monetization / restrictions requiring some degree of limitations to upsell will eventually come for the currently half-billion or so Win11 users.
I get the point that few are currently paying for it, but there isn't all that much pressure to do so right now. That will come, later.
Yes that is inevitable. But it's exactly why I won't use it. It feels like a scam, the typical drug dealer model. Get them hooked on free stuff, and then milk their addiction.
I almost only use local LLMs now. They're not as good, no. But at least they are mine and nobody will raise the price suddenly. Some pro AI models I just use with API, for the rare instance where local just doesn't cut it. It costs me a few bucks per month that way, not 20 or 30.
Traditional search works fine.
If you see a work and web tab at the top, click work. Then it should be able to. If you don't see this, your company is not paying and it just won't work.
You’re a company whose main product has a brand recognition that ranks right up there with Coca-Cola and Apple, and you rename it after a new product that is a pile of hot garbage. Absolutely mind-boggling.
Of course all the people asking for these were already very interested in AI so they are not a very representative group. We did take that into account.
They changed how that key works at a low level so it cannot be cleanly remapped back to right-CTRL. This is because, unlike the CTRL, ALT, Shift and Windows keys, the now-CoPilot key no longer behaves like a modifier key. Now when you press the CoPilot key down it generates both key down and key up events - even when you keep it pressed down. You can work around this somewhat with clever key remapping in tools like AutoHotKey but it is literally impossible to fully restore that key back so it will behave like a true modifier key such as right-CTRL in all contexts. There are a limited number of true modifier keys built into a laptop. Stealing one of them to upsell a monetized service is shitty but intentionally preventing anyone from being able to restore it goes beyond shitty to just maliciously evil.
However, I've had no issue remapping it to the Context Menu key on the Chinese Lenovo Xiao Xin laptop I bought recently. It shows up in the PowerToys keyboard remapper tool as an F23 key. (Yes, there are more than 12 F key codes! I believe there is 24 in Windows.)
Other OSes will not even be able to say things like "Press your Copilot key to open FEDORA SEARCH" or whatever without being cease'n'desisted to high heaven.
Terrible.
Even the "productivity" suggestions in teams are ridiculous - one of them is "roast me by my calendar"...wtf is that? How is that related to productivity at all?
Put the LLM on answering (or at least triaging) your support tickets faster.
Copilot doesn't run on Microsoft's own LLMs, they're all third party. ChatGPT in office copilot, in github you can choose various other ones too.
Microsoft is really just a reseller there.
Personally I think Claude Code (with Opus in particular) is far better. Unfortunately they are very aware of this and thus it is eyewateringly expensive.
Plus for 10$ a month (~8.50 EUR last month) I get more-than-enough-for-me access to Gemini, Claude and OpenAI models from my code editor. I wouldn't pay double for Claude Code or Cursor, so I feel like it's a good enough deal.
Besides, I just recently saw a review of the newest Copilot for Excel by a serious spreadsheet jockey, and he was truly blown away. Basically, it knows everything Excel can do, how to do it, and it now truly understands numbers as numbers and when to use formulas, etc. etc. Basically, it went from smart automation to full on AI.
If Microsoft doesn't fall for short term thinking, the product will eventually succeed.