At some point, a publicly-listed company will go bankrupt due to some catastrophic AI-induced fuck-up. This is a massive reputational risk for AI platforms, because ego-defensive behaviour guarantees that the people involved will make as much noise as they can about how it's all the AI's fault.
Brenda has been getting slower over the years -as we all have-, but soon the boss will learn that it was a small price to pay for knowing well how to keep such house of cards from collapsing.
I'm actually not that worried about this, because again I would classify this as a problem that already exists. There are already idiots in senior management who pass off bullshit and screw things up. There are natural mechanisms to cope with this, primarily in business reputation - if you're one of those idiots who does this people very quickly start just discounting what you're saying, they might not know how you're wrong, but they learn very quickly to discount what you're saying because they know you can't be trusted to self-check.
I'm not saying that this can't happen and it's not bad. Take a look at nudge theory - the UK government created an entire department and spent enormous amounts of time and money on what they thought was a free lunch - that they could just "nudge" people into doing the things they wanted. So rather than actually solving difficult problems the uk government embarked on decades of pseudo-intellectual self agrandizement. The entire basis of that decades long debacle was based on bullshit data and fake studies. We didn't need AI to fuck it up, we managed it perfectly well by ourselves.
This reminds me of a friend whose company ran a daily perl script that committed every financial transaction of the day to a database. Without the script, the company could literally make no money irrespectively of sales because this database was one piece in a complex system for payment processor interoperability.
The script ran in a machine located at the corner of a cubicle and only one employee had the admin password. Nobody but a handful of people knew of the machine's existence, certainly not anyone in middle management and above. The script could only be updated by an admin.
Copilot may be good, but sure as hell doesn't know that admin password.
Co-pilot and AI has been shoved at the Microsoft Stack in my org for months. Most of the features were disabled or hopelessly bad. It’s cheaper for Microsoft to push this junk and claim they’re doing something, it’s going to improve their stock far more than not doing it, even though it’s basically useless currently.
Another issue is that my org disallows AI transcription bots. It’s a legit security risk if you have some random process recording confidential info because the person was too busy to attend the meeting and take notes themselves. Or possibly they just shirk off the meetings and have AI sit in.
“There are two Brendas - their job is to make spreadsheets in the Finance department. Well, not quite - they add the months and categories to empty spreadsheets, then they ask the other departments to fill in their sales numbers every month so it can be presented to management.
“The two Brendas don’t seem to talk, otherwise they would realize that they’re both asking everyone for the same information, twice. And they’re so focused on their little spreadsheet worlds that neither sees enough of the bigger picture to say, ‘Wait… couldn’t we just automate this so we don’t need to do this song and dance every month? Then we wouldn’t need two people in different parts of the company compiling the same data manually.’
“But that’s not what Brenda was hired for. She’s a spreadsheet person, not a process fixer. She just makes the spreadsheets.”
We need fewer Brendas, and more people who can automate away the need for them.
Excel is the “beast that drives the ENTIRE economy” and he’s worried about Brenda from the finance department losing her job because then her boss will get bad financial reports
I suppose the person that wrote that have not ideia Excel is just an app builder where you embed data together with code.
You know that we have excel because computers didn’t understand column names in databases and so data extraction needed to be made by humans. Humans then design those little apps in excel to massage the data.
Well, now an agent can read the boss saying gimme the sales from last month and the agent don’t need excel for that, because it can query the database itself, massage the data itself using python and present the data itself with html or PNGs.
So, we are in the process of automating Brenda AND excel away.
Also, finance departments are a very small part of excel users. Just think everywhere were people need small programs, excel is there.
This is transparent nonsense. People are very very happy to introduce errors into excel spreadsheets without any help from AI.
Financial statements are correct because of auditors who check the numbers.
If you have a good audit process then errors get detected even if AI helped introduce them. If you aren't doing a good audit then I suspect nobody cares whether your financial statement is correct (anyone who did would insist on an audit).
"the sweat from Brenda's brow is what allows us to do capitalism."
The CEO has been itching to fire this person and nuke her department forever. She hasn't gotten the hint with the low pay or long hours, but now Copilot creates exactly the opening the CEO has been looking for.
I find the contrast between two narratives around technology use so fascinating:
1. We advocate automation because people like Brenda are error-prone and machines are perfect.
2. We disavow AI because people like Brenda are perfect and the machine is error-prone.
These aren't contradictions because we only advocate for automation in limited contexts: when the task is understandable, the execution is reliable, the process is observable, and the endeavour tedious. The complexity of the task isn't a factor - it's complex to generate correct machine code, but we trust compilers to do it all the time.
In a nutshell, we seem to be fine with automation if we can have a mental model of what it does and how it does it in a way that saves humans effort.
So, then - why don't people embrace AI with thinking mode as an acceptable form of automation? Can't the C-suite in this case follow its thought process and step in when it messes up?
I think people still find AI repugnant in that case. There's still a sense of "I don't know why you did this and it scares me", despite the debuggability, and it comes from the autonomy without guardrails. People want to be able to stop bad things before they happen, but with AI you often only seem to do so after the fact.
Narrow AI, AI with guardrails, AI with multiple safety redundancies - these don't elicit the same reaction. They seem to be valid, acceptable forms of automation. Perhaps that's what the ecosystem will eventually tend to, hopefully.
> 2. We disavow AI because people like Brenda are perfect and the machine is error-prone.
Some of these Brenda types are actually, really perfect. Unless they are sick, they never make mistakes. Sure they are a small minority, but they do exist.
I disavow AI because while neither it or Brenda are perfect, Brenda consistently follows the instructions and can have an actual conversation with me to take feedback into account going forward. An AI will happily participate in those conversations, but whether it actually improves based on that feedback is not at all consistent. It's also a lot easier to find other humans who are meaningfully different than Brenda if I prefer to hire someone with a different working style, whereas right now the issues I describe above with BrendaBot are going to be essentially the same with any other AI I try to use instead.
That mirrors my experience as well. LLMs get instantly confused in real world scenarios in Excel and confidently hallucinate millions in errors
If you look at the demos for these it’s always something that is clean and abundantly available in training data. Like an income statement. Or a textbook example DCF. Or my personal fav „here is some data show me insights“. Real world excel use looks nothing like that.
I’m getting some utility out of them for some corporate tasks but zilch in excel space.
Many fears of “AI mucking it up” could be mitigated with an ability to connect a workbook to a git repository. Not for data, but for VBA, cell formulas, and cell metadata. When you can encapsulate the changes a contributor (in this case co-pilot) makes into a commit, you can more easily understand what changes it/they made.
Even if "teaching the rest of the workforce how to use Git" wasn't a massive obstacle, many (most?) corporate workflows involve editing files that weren't designed to be human-readable. There are a couple of approaches out there for comparing diffs to excel spreadsheets specifically, but it's not exactly pleasant.
Considering that Microsoft perfected change control in the legal industry (ms word’s track changes), something can be done to add a degree of change control to spreadsheets.
Using ai does not absolve you from the responsibility of doing it correctly. If you use ai, then you better have the skills to have done the job yourself, and so have the ability to check the AI did things correctly.
You can save time still, but perhaps not as much as you think, because you need to check the ai's work thoroughly.
Coding agents are useful and good and real products because when they screw up, things stop working almost always before they can do damage. Coding agents are flawed in ways that existing tools are good at catching, never mind the more obvious build and runtime errors.
Letting AI write your emails and create your P&L and cash flow projections doesn't have to run the gauntlet of tools that were created to stop flawed humans from creating bad code.
55 comments
[ 4.8 ms ] story [ 65.0 ms ] threadI'm not saying that this can't happen and it's not bad. Take a look at nudge theory - the UK government created an entire department and spent enormous amounts of time and money on what they thought was a free lunch - that they could just "nudge" people into doing the things they wanted. So rather than actually solving difficult problems the uk government embarked on decades of pseudo-intellectual self agrandizement. The entire basis of that decades long debacle was based on bullshit data and fake studies. We didn't need AI to fuck it up, we managed it perfectly well by ourselves.
(I pulled the quote by using yt-dlp to grab the MP4 and then running that through MacWhisper to generate a transcript.)
The script ran in a machine located at the corner of a cubicle and only one employee had the admin password. Nobody but a handful of people knew of the machine's existence, certainly not anyone in middle management and above. The script could only be updated by an admin.
Copilot may be good, but sure as hell doesn't know that admin password.
Another issue is that my org disallows AI transcription bots. It’s a legit security risk if you have some random process recording confidential info because the person was too busy to attend the meeting and take notes themselves. Or possibly they just shirk off the meetings and have AI sit in.
“There are two Brendas - their job is to make spreadsheets in the Finance department. Well, not quite - they add the months and categories to empty spreadsheets, then they ask the other departments to fill in their sales numbers every month so it can be presented to management.
“The two Brendas don’t seem to talk, otherwise they would realize that they’re both asking everyone for the same information, twice. And they’re so focused on their little spreadsheet worlds that neither sees enough of the bigger picture to say, ‘Wait… couldn’t we just automate this so we don’t need to do this song and dance every month? Then we wouldn’t need two people in different parts of the company compiling the same data manually.’
“But that’s not what Brenda was hired for. She’s a spreadsheet person, not a process fixer. She just makes the spreadsheets.”
We need fewer Brendas, and more people who can automate away the need for them.
I suppose the person that wrote that have not ideia Excel is just an app builder where you embed data together with code.
You know that we have excel because computers didn’t understand column names in databases and so data extraction needed to be made by humans. Humans then design those little apps in excel to massage the data.
Well, now an agent can read the boss saying gimme the sales from last month and the agent don’t need excel for that, because it can query the database itself, massage the data itself using python and present the data itself with html or PNGs.
So, we are in the process of automating Brenda AND excel away.
Also, finance departments are a very small part of excel users. Just think everywhere were people need small programs, excel is there.
Financial statements are correct because of auditors who check the numbers.
If you have a good audit process then errors get detected even if AI helped introduce them. If you aren't doing a good audit then I suspect nobody cares whether your financial statement is correct (anyone who did would insist on an audit).
The CEO has been itching to fire this person and nuke her department forever. She hasn't gotten the hint with the low pay or long hours, but now Copilot creates exactly the opening the CEO has been looking for.
1. We advocate automation because people like Brenda are error-prone and machines are perfect.
2. We disavow AI because people like Brenda are perfect and the machine is error-prone.
These aren't contradictions because we only advocate for automation in limited contexts: when the task is understandable, the execution is reliable, the process is observable, and the endeavour tedious. The complexity of the task isn't a factor - it's complex to generate correct machine code, but we trust compilers to do it all the time.
In a nutshell, we seem to be fine with automation if we can have a mental model of what it does and how it does it in a way that saves humans effort.
So, then - why don't people embrace AI with thinking mode as an acceptable form of automation? Can't the C-suite in this case follow its thought process and step in when it messes up?
I think people still find AI repugnant in that case. There's still a sense of "I don't know why you did this and it scares me", despite the debuggability, and it comes from the autonomy without guardrails. People want to be able to stop bad things before they happen, but with AI you often only seem to do so after the fact.
Narrow AI, AI with guardrails, AI with multiple safety redundancies - these don't elicit the same reaction. They seem to be valid, acceptable forms of automation. Perhaps that's what the ecosystem will eventually tend to, hopefully.
If we think of AI as a machine, it is stochastic, not deterministic. It breaks our cause-and-effect understanding of machines.
If we think of AI as a agency, it does not have goals or beliefs. It mimics goals, following a mathematical path to an optimal answer.
So, we are confused, frustrated, angry: "What the heck is this thing?!"
Some of these Brenda types are actually, really perfect. Unless they are sick, they never make mistakes. Sure they are a small minority, but they do exist.
If you look at the demos for these it’s always something that is clean and abundantly available in training data. Like an income statement. Or a textbook example DCF. Or my personal fav „here is some data show me insights“. Real world excel use looks nothing like that.
I’m getting some utility out of them for some corporate tasks but zilch in excel space.
When tools break, people stop using them before they sink the ship down. If AI is that terrible at spreadsheet, people will just revert to Brenda.
And it's not like spreadsheets have no errors right now.
You can save time still, but perhaps not as much as you think, because you need to check the ai's work thoroughly.
Brenda"
I don't know about that. There could be lots of interesting ways Brenda can (be convinced to) hallucinate.
Coding agents are useful and good and real products because when they screw up, things stop working almost always before they can do damage. Coding agents are flawed in ways that existing tools are good at catching, never mind the more obvious build and runtime errors.
Letting AI write your emails and create your P&L and cash flow projections doesn't have to run the gauntlet of tools that were created to stop flawed humans from creating bad code.
AI may be able to spit out ann excel sheet or formula - But if it can’t be verified, so what ?
And here’s my analogy to think about the debugging of an excel sheet - you can debug most corporate excel sheets with a calculator.
But when AI is spitting out excel sheets - when the program is making smaller programs - what is the calculator in this analogy ?
Are we going to be using excel sheets to debug the output of AI?
I think this is the inherent limiter to the uptake of AI.
There’s only so much intellectual / experiential / training depth present.
And now we’re going to be training even fewer people.
At the end of the day I /customers need something to work.
But failing that - I will settle for someone to blame.
Brenda handles a lot of blame. Is OpenAI going to step into that gap ?