> Proposing the motion at this week's meeting, Dr Marks said: "I'm no expert on HR software, but I do know that manually copying and pasting from seven different Excel sheets, all laid out in different ways, is bound to go wrong one day. So who set up the system? Who set up the QA processes and the staff training?"
This sounds pretty standard to me. Spreadsheet stacks are actually effective - and often robust if the Copy Paste technician has a little bit of nous. I don't think it makes sense to get the individual involved in trouble, but it isn't intellectually honest to pretend this is that strange or unreasonable. Offices are full of people who make things work in inefficient ways. Such is life. Improve the system if it fails, but there isn't any call to act surprised that the process is awkward.
If done by a civil servant (government employee), that would be grade AA (Admin Assistant) or AO (Admin Officer). An AO in London gets £24k/year, which is around £11.50/h, or $14/h
Is prompt engineer a derogatory term for people who spend their work day at a shell prompt? If so, I can own up to being a prompt engineer. Or maybe it’s an engineer who does their work promptly. In that case, I’m not sure I qualify.
“Typical process latency: Someone copies and pastes data from a thing into another thing”[1]. Jokes aside, though, I don’t think it’s fair to criticise people for using the only user-accessible interapplication communication channel the industry has universally agreed to make work and not use as lock-in leverage.
The spreadsheets being in different formats and lacking additional sheets that’d automatically reformat the data the way the next step needs is something I’d criticise, but I undertstand how that might be nonobvious if you aren’t lazy enough to think about automating everything you possibly can.
>> often robust if the Copy Paste technician has a little bit of nous
This is faulty logic. Human error isn't an exceptional thing, it's extremely common and expected. There's an entire field of study dedicated to it: https://en.wikipedia.org/wiki/Human_reliability
In many scenarios the right thing to do is just accept the error risk - as humans, we often don't need perfect execution but i don't think this is one of those.
The person quoted in the article gets it right: "We need senior management that know what they're doing, who realize that this is not, and never was, the way to run an HR system"
Excel stacks can be effective, but they need to be setup correctly and to have effective operational control. Royal College have a right to surprise - they are a client of ANRO and had to winkle the information out of them. Given the way things turned out, I think Dr Marks comments are also appropriate.
Spreadsheet stacks are actually effective - and often robust if the Copy Paste technician has a little bit of nous
This is true. Sounds like they have cobbled together a system using Excel because thats why you have to do when processes change all the time.
The alternative is to find a 'proper' HR system to handle all the weird custom processes which is a massive three year project that costs tens of millions and then probably fails to deliver.
IT people complain about people cobbling things together in Excel but in the short and medium term thats usually the only way for people to get shit done
But yes management dropped the ball and should have had some checks and balances. No-one realised that every candidate from Wales got rejected? Thats a clanger.
>IT people complain about people cobbling things together in Excel but in the short and medium term thats usually the only way for people to get shit done
This is true, and why I think it's a shame we never got a real replacement for Microsoft Access (not necessarily the functionality of Access but its intended purpose).
If only there was a simple system for producing a database-driven set of screens in a webapp. It's the one thing which seems to have got worse since the eras of dBASE and FoxPro.
Having worked with "spreadsheet stacks" for the last two decades I'd argue that although they can be effective in most cases they are error prone which is the point being made in your quote. And they become more error prone the more complex the "system" is and that's bad news because it's oh so easy to add more complexity whenever you want.
I agree the practices described don't sound strange - they sound normal, but so does the resulting errors that occurred - also par for the course.
>Spreadsheet stacks are actually effective - and often robust if the Copy Paste technician has a little bit of nous.
I broadly agree with this, but there are ways to make the process effective and ways to make it incredibly error prone, like not having checks to ensure you paste into the correct columns. Obviously there are no such checks here.
Nobody cares about inefficient or brittle process that produce reasonable results, but when you mess up the careers of a number of people, someone's going to take a look at the process.
"ANRO decided to honor the 10 job offers it had made by mistake and used Oriel to tell the candidates. Unfortunately, a system error in Oriel meant it then erroneously sent that communication to an additional 16 candidates. ANRO decided to honor these 16 additional offers too, and find the candidates posts."
They're still people with valid medical licenses - the pool of candidates wouldn't have people who couldn't legally practice medicine, so I'd assume the qualification criteria they were being assessed on is unrelated to their ability to not accidentally kill me on the OR table.
I know what you mean. But reading again the recruitment process was for "third-level specialist training position (ST3)" which I think is the 3rd year of a 5 year training. So everyone was in training, its just that the relative ranking got messed up. Its not like random people could apply. So I guess thats why they decided to honor their (frankly terrible) job offer fuck ups.
While this may be true, (but someone has to be the lowest scoring one - it could've been a group where everyone is great) keep in mind it was a list of candidates who are aware of their lowest scores, getting a training position in a place aware of their lowest scores, with 5 more years ahead of them. It sucks that it happened, but I think they'll be under extreme scrutiny/pressure now.
Not necessarily -- perhaps they'd found 26 suitable candidates, picked ten of them to fill their ten open positions, but accidentally sent offers to all 26.
Assume that ANRO is a "normal" government bureaucracy - any people inclined to double-checking, questioning things, or caring much about the real-world outcomes get systematically weeded out.
People think that Excel is the most used "programming language" in the world. Which may or may not be true.
But the one truth is this: the world runs on broken Excel spreadsheets.
People have no idea. The number of decisions taken higher up the chain based on wrong numbers are beyond belief.
If you've ever been tasked to rewrite a semi-advanced Excel spreadsheet to a dedicated (web)app, you know.
I'm a coder and I do use spreadsheets (not Excel though). Spreadsheets in the hands of programmers are okay.
But thinking that somehow because Joel Spolsky can use advanced Excel features, regular people can too, then you're being completely delusional.
You can't have it both ways: either it's the most used programming language in the world and it has to be used by programmers, or it's a recipe for incorrectness.
If it’s a programming language, then its users are programmers. Just like with javascript or c, some of those programmers are more skilled than others.
> People have no idea. The number of decisions taken higher up the chain based on wrong numbers are beyond belief.
Yup. That's a lesson in how robust organizational systems are - the errors mostly get compensated for somewhere downstream, so visibly bad outcomes are rare.
> If you've ever been tasked to rewrite a semi-advanced Excel spreadsheet to a dedicated (web)app, you know.
Yeah, I know. I've worked in a place that tried and failed to beat an Excel plugin with a web app.
The thing is, in my experience, you'll have a team spend two years doing the rewrite - with a lot of that going into designing and then building complex web frontend using modern tech stack - and in the end, you'll get a dedicated application that has less functionality and worse ergonomics than the spreadsheet you tried to replace.
The timeframes and costs involved with getting a Proper Software Development Team to build you a Proper App For That are absurd, and are only getting worse with everyone wanting to do bespoke web SPAs using modern practices and tool stacks. Even if one could afford that, the outcome is unlikely to be better for actual users. And so Excel stays, covering for all the processes that are custom enough that there is no ready-made popular Enterprise SaaS on the market that management can order people to use instead.
> The timeframes and costs involved with getting a Proper Software Development Team to build you a Proper App For That are absurd, and are only getting worse with everyone wanting to do bespoke web SPAs using modern practices and tool stacks. Even if one could afford that, the outcome is unlikely to be better for actual users. And so Excel stays, covering for all the processes that are custom enough that there is no ready-made popular Enterprise SaaS on the market that management can order people to use instead.
I've had similar thoughts recently, though without the direct personal experience of trying to convert from a spreadsheet to an app.
Makes me wonder: perhaps what we really need isn't a shiny new framework like React or a design pattern like async/await, but rather "VB for the web"[0] — drag and drop a text field here, a number field there, perhaps a checkbox or some radio buttons, then add some "done" and "cancel" buttons… it shouldn't be as hard as it has become.
Heck, give Figma or whatever a way to be event-driven like VB was, it's not like we should need engineers to hand-craft code that replicates the look of a design document.
[0] Xojo and VB.net both exist, so the question is really: why are they not dominant? Is it as simple as "BASIC-family languages are fundamentally bad"? But the language itself obviously isn't a necessary constraint; and likewise Xcode has Interface Builder (though less reliable in SwiftUI than it was in UIKit, still present, yet why isn't it better?)
You have devs in here shrugging and saying "hey it works, no big deal" when it comes to a business process. But if it comes to svelte vs react vs whatever it's a butlerian jihad where everyone's accusing eachother of not even knowing how to program.
sometimes I wonder if most of us do the job to play with silly tools rather than actually make shit.
There's a very "bikeshedding" property at work where the fewer "real" consequences come from a decision, the more heated the discussion gets because it's purely opinionated.
(there are also a number of HN commentators who are reliably noisily wrong about things, but not in such a blatant way that they get obliterated in downvotes, and one of them started that top level thread)
> sometimes I wonder if most of us do the job to play with silly tools rather than actually make shit.
Yes? I thought this is common knowledge.
Here's a universal tool whose potential is limited only by one's imagination, and yet most of the jobs in our profession are at best boring - the high-paying ones usually involve making actively harmful things that shouldn't exist in the first place (see e.g. whole adtech for starters), and most of the rest are so mired in office politics or business constraints that the outcomes are guaranteed to be shit. A lot of strangeness in the dev culture can be viewed as coping mechanisms.
Meanwhile, after doing this work long enough, you end up realizing that disturbingly many SaaS tools offer much less utility and worse ergonomics than an Excel spreadsheet.
Often I can do what I need in a few minutes in excel where à programmer would take hours and not give me what I actually asked for. And in excel I can modify and update my work. Where a programme solution is static and needs tons more hours for every change needed.
Often we see people claiming that "This new thing is the spreadsheet killer, like ai at the moment. People have been saying this forever but spreadsheets are still here. And there's a good reason for it.
There are only two things I've seen that ever came close to actual "spreadsheet killers" - MS Access and computational notebooks (Jupyter, Observable webapp). And it so happens that both are very similar in principle to Excel spreadsheets. They're optimized for certain paradigm (reactive cells, RDBMS CRUD, reactive cells accepting code and outputting rich content), but otherwise don't constrain you - instead, they invite you to build your thing incrementally, figuring your problem out as you build the solution, with full visibility into intermediate states.
The only way AI has a chance of killing spreadsheets is if it replaces a Real Programmers, by becoming a tool that lets anyone interactively build arbitrary apps, for free (on the margin), with near zero turnaround.
> Often I can do what I need in a few minutes in excel where à programmer would take hours and not give me what I actually asked for
That's likely not down to the tool, more likely a complex requirement you understand but they don't. (And are you sure you're getting the right answer? IIRC an analysis showed 10% of spreadsheets have errors - devs do care about good results, and that does take extra time)
> Where a programme solution is static and needs tons more hours for every change needed.
Hmm. Sounds like there's something wrong here, with your guys or some aspect of the business (and sure, excel is not a bad tool, just to be clear).
Economics. The scarcity of devs always leads to rationing and its related sources of friction. From the "requestor" standpoint, friction could dominate the cost of getting things done.
What I think more people should appreciate is that in any business process, computer or otherwise, there's a speed/generality/correctness/accountability tradeoff.
AI sits at the "speed" corner. Excel, a fully dynamic interactive programming language with one type "cell", gives slightly better correctness and accountability. Coalescing process into program continues to move you to the other end. Once you start employing typechecking and later formal verification, you get more correctness and accountability - BUT at a cost of development time, AND generality. The correctness is achieved precisely by being tailored to the specifics and rejecting the general.
This can apply to entirely physical processes. Some "aerospace" grade parts are very similar to normal parts .. but with a huge overhead of accountability. Because getting the right outcome matters.
This is huge. Programmers and programming are not the problem. the process is. For a wide variety of uses, the cost of programming erases the value added.
> Here's a universal tool whose potential is limited only by one's imagination
And it crashing, and it being limited in capacity, and not being as flexible as SQL (in areas SQL is good at). It's not a universal tool, good as it is.
I work to earn money, I work to feel useful, I work for the challenge, etc. I do not really work to deliver adtech or LoB features. That's just a (un?)fortunate byproduct of the process.
We should create an internal tool that relies on a framework which needs quarterly code changing updates for a yearly(?) administrative process with less than 24 candidates?
IMO the arguments people make in frontend are similar to the infinite scaling requirements in back end that require 10X overpriced machine rental.
Whether or not the current site could ever be appropriate for all 10 billion users, a developer wants the universal skills tuned to the top end.
A university HR employee OTOH probably realizes they won't code themselves if they end up at the largest employer on Earth.
Excel is superb for automating/number crunching short lived business processes or those that are one offs or change rapidly. Especially if you don't have technical resources available to do it any other way.
The biggest issue they had was not understanding that the data from one area (Wales) was in a different format from the others - an integer that represented rank (lowest being best) rather than a test score - that would probably still not been noticed if they'd used something else.
I'm sure Byzantine HR processes are constantly being bungled, but you almost never hear about it. These processes only exist because every hiring decision needs a paper trail to show compliance with a myriad of employment and discrimination law. Next time you wonder why orgs seem to have a high concentration of talent when small, and then get diluted with deadweight as they grow, know that this is one of the reasons why.
They had 24 applicants and they ran the process seemingly fully on excel and technocratic HR and email. If these decisions were only involving human subject matter experts this would not have happened. Imagine you need someone, define criteria, get two dozens CVs and then HR tells you nope: they are all no-go. Most would say WTF or go ballistic. But if HR is in the driving seat alone and makes final decisions alone without human contact - anything goes.
Thankfully it will be all getting better when we delegate that to A.I..
Reminds me of the excel-bungling during the covid pandemic.
>A million-row limit on Microsoft’s Excel spreadsheet software may have led to Public Health England misplacing nearly 16,000 Covid test results, it is understood. The data error, which led to 15,841 positive tests being left off the official daily figures, means than 50,000 potentially infectious people may have been missed by contact tracers and not told to self-isolate.
67 comments
[ 3.2 ms ] story [ 122 ms ] threadThis sounds pretty standard to me. Spreadsheet stacks are actually effective - and often robust if the Copy Paste technician has a little bit of nous. I don't think it makes sense to get the individual involved in trouble, but it isn't intellectually honest to pretend this is that strange or unreasonable. Offices are full of people who make things work in inefficient ways. Such is life. Improve the system if it fails, but there isn't any call to act surprised that the process is awkward.
I expect there is a lot of politics at play here.
https://www.ons.gov.uk/aboutus/transparencyandgovernance/fre...
The spreadsheets being in different formats and lacking additional sheets that’d automatically reformat the data the way the next step needs is something I’d criticise, but I undertstand how that might be nonobvious if you aren’t lazy enough to think about automating everything you possibly can.
[1] https://xkcd.com/2565/
This is faulty logic. Human error isn't an exceptional thing, it's extremely common and expected. There's an entire field of study dedicated to it: https://en.wikipedia.org/wiki/Human_reliability
In many scenarios the right thing to do is just accept the error risk - as humans, we often don't need perfect execution but i don't think this is one of those.
The person quoted in the article gets it right: "We need senior management that know what they're doing, who realize that this is not, and never was, the way to run an HR system"
This is true. Sounds like they have cobbled together a system using Excel because thats why you have to do when processes change all the time.
The alternative is to find a 'proper' HR system to handle all the weird custom processes which is a massive three year project that costs tens of millions and then probably fails to deliver.
IT people complain about people cobbling things together in Excel but in the short and medium term thats usually the only way for people to get shit done
But yes management dropped the ball and should have had some checks and balances. No-one realised that every candidate from Wales got rejected? Thats a clanger.
This is true, and why I think it's a shame we never got a real replacement for Microsoft Access (not necessarily the functionality of Access but its intended purpose).
I agree the practices described don't sound strange - they sound normal, but so does the resulting errors that occurred - also par for the course.
I broadly agree with this, but there are ways to make the process effective and ways to make it incredibly error prone, like not having checks to ensure you paste into the correct columns. Obviously there are no such checks here.
https://www.theregister.com/2023/10/12/excel_anesthetist_rec...
"ANRO decided to honor the 10 job offers it had made by mistake and used Oriel to tell the candidates. Unfortunately, a system error in Oriel meant it then erroneously sent that communication to an additional 16 candidates. ANRO decided to honor these 16 additional offers too, and find the candidates posts."
People get rejected because they are not the best fit, not because they are unfit.
They're still people with valid medical licenses - the pool of candidates wouldn't have people who couldn't legally practice medicine, so I'd assume the qualification criteria they were being assessed on is unrelated to their ability to not accidentally kill me on the OR table.
Hopefully not unqualified, just not the "best" available
While this may be true, (but someone has to be the lowest scoring one - it could've been a group where everyone is great) keep in mind it was a list of candidates who are aware of their lowest scores, getting a training position in a place aware of their lowest scores, with 5 more years ahead of them. It sucks that it happened, but I think they'll be under extreme scrutiny/pressure now.
They also blam Oriel which is another system
But the one truth is this: the world runs on broken Excel spreadsheets.
People have no idea. The number of decisions taken higher up the chain based on wrong numbers are beyond belief.
If you've ever been tasked to rewrite a semi-advanced Excel spreadsheet to a dedicated (web)app, you know.
I'm a coder and I do use spreadsheets (not Excel though). Spreadsheets in the hands of programmers are okay.
But thinking that somehow because Joel Spolsky can use advanced Excel features, regular people can too, then you're being completely delusional.
You can't have it both ways: either it's the most used programming language in the world and it has to be used by programmers, or it's a recipe for incorrectness.
They are usually looking at numbers to support decisions, not much for actual guidance/directions.
Yup. That's a lesson in how robust organizational systems are - the errors mostly get compensated for somewhere downstream, so visibly bad outcomes are rare.
> If you've ever been tasked to rewrite a semi-advanced Excel spreadsheet to a dedicated (web)app, you know.
Yeah, I know. I've worked in a place that tried and failed to beat an Excel plugin with a web app.
The thing is, in my experience, you'll have a team spend two years doing the rewrite - with a lot of that going into designing and then building complex web frontend using modern tech stack - and in the end, you'll get a dedicated application that has less functionality and worse ergonomics than the spreadsheet you tried to replace.
The timeframes and costs involved with getting a Proper Software Development Team to build you a Proper App For That are absurd, and are only getting worse with everyone wanting to do bespoke web SPAs using modern practices and tool stacks. Even if one could afford that, the outcome is unlikely to be better for actual users. And so Excel stays, covering for all the processes that are custom enough that there is no ready-made popular Enterprise SaaS on the market that management can order people to use instead.
I've had similar thoughts recently, though without the direct personal experience of trying to convert from a spreadsheet to an app.
Makes me wonder: perhaps what we really need isn't a shiny new framework like React or a design pattern like async/await, but rather "VB for the web"[0] — drag and drop a text field here, a number field there, perhaps a checkbox or some radio buttons, then add some "done" and "cancel" buttons… it shouldn't be as hard as it has become.
Heck, give Figma or whatever a way to be event-driven like VB was, it's not like we should need engineers to hand-craft code that replicates the look of a design document.
[0] Xojo and VB.net both exist, so the question is really: why are they not dominant? Is it as simple as "BASIC-family languages are fundamentally bad"? But the language itself obviously isn't a necessary constraint; and likewise Xcode has Interface Builder (though less reliable in SwiftUI than it was in UIKit, still present, yet why isn't it better?)
You have devs in here shrugging and saying "hey it works, no big deal" when it comes to a business process. But if it comes to svelte vs react vs whatever it's a butlerian jihad where everyone's accusing eachother of not even knowing how to program.
sometimes I wonder if most of us do the job to play with silly tools rather than actually make shit.
(there are also a number of HN commentators who are reliably noisily wrong about things, but not in such a blatant way that they get obliterated in downvotes, and one of them started that top level thread)
Yes? I thought this is common knowledge.
Here's a universal tool whose potential is limited only by one's imagination, and yet most of the jobs in our profession are at best boring - the high-paying ones usually involve making actively harmful things that shouldn't exist in the first place (see e.g. whole adtech for starters), and most of the rest are so mired in office politics or business constraints that the outcomes are guaranteed to be shit. A lot of strangeness in the dev culture can be viewed as coping mechanisms.
Meanwhile, after doing this work long enough, you end up realizing that disturbingly many SaaS tools offer much less utility and worse ergonomics than an Excel spreadsheet.
Often we see people claiming that "This new thing is the spreadsheet killer, like ai at the moment. People have been saying this forever but spreadsheets are still here. And there's a good reason for it.
The only way AI has a chance of killing spreadsheets is if it replaces a Real Programmers, by becoming a tool that lets anyone interactively build arbitrary apps, for free (on the margin), with near zero turnaround.
That's likely not down to the tool, more likely a complex requirement you understand but they don't. (And are you sure you're getting the right answer? IIRC an analysis showed 10% of spreadsheets have errors - devs do care about good results, and that does take extra time)
> Where a programme solution is static and needs tons more hours for every change needed.
Hmm. Sounds like there's something wrong here, with your guys or some aspect of the business (and sure, excel is not a bad tool, just to be clear).
AI sits at the "speed" corner. Excel, a fully dynamic interactive programming language with one type "cell", gives slightly better correctness and accountability. Coalescing process into program continues to move you to the other end. Once you start employing typechecking and later formal verification, you get more correctness and accountability - BUT at a cost of development time, AND generality. The correctness is achieved precisely by being tailored to the specifics and rejecting the general.
This can apply to entirely physical processes. Some "aerospace" grade parts are very similar to normal parts .. but with a huge overhead of accountability. Because getting the right outcome matters.
> Here's a universal tool whose potential is limited only by one's imagination
And it crashing, and it being limited in capacity, and not being as flexible as SQL (in areas SQL is good at). It's not a universal tool, good as it is.
> No. Most of us work to deliver
I mean, yes, of course - but "playing with silly tools" is one of many methods of delivering without losing your mind in the process.
IMO the arguments people make in frontend are similar to the infinite scaling requirements in back end that require 10X overpriced machine rental.
Whether or not the current site could ever be appropriate for all 10 billion users, a developer wants the universal skills tuned to the top end.
A university HR employee OTOH probably realizes they won't code themselves if they end up at the largest employer on Earth.
Major differences let you say "well that's just what THOSE people do". It's not a threat to your sense of self.
Minor differences challenge your own sense of self, and thus must be destroyed.
The biggest issue they had was not understanding that the data from one area (Wales) was in a different format from the others - an integer that represented rank (lowest being best) rather than a test score - that would probably still not been noticed if they'd used something else.
Thankfully it will be all getting better when we delegate that to A.I..
We'll... There's your problem. It doesn't matter what tool you use if the definition of the system is broken.
>A million-row limit on Microsoft’s Excel spreadsheet software may have led to Public Health England misplacing nearly 16,000 Covid test results, it is understood. The data error, which led to 15,841 positive tests being left off the official daily figures, means than 50,000 potentially infectious people may have been missed by contact tracers and not told to self-isolate.
https://www.theguardian.com/politics/2020/oct/05/how-excel-m...