This problem is not confined to the government. I've worked in more than a dozen different enterprises, big and small. I think it wouldn't be an overstatement to say that the majority of computer systems I've come across are pretty useless.
EDIT: Maybe enterprises should embrace systems like coda.io as a reasonable compromise between rigid enterprise systems and spreadsheets.
Since your statement is based on personal experience I cannot comment, I would say, though, that the majority of enterprise systems I've used are far from useless. Most of them have been: bloated, slow, outdated and complicated but that's a far cry from being useless.
I just watched my company embark on a 9 figure multi-year automation journey. For some reason, the powers that be thought it would be easier for our folks to figure out an RPA tool than write code — never mind with RPA you have to navigate 3 abstraction layers (the OS, browser DOM and the application itself) instead of just the application. The complexity RPA introduces for web apps kills any benefit it has. I can’t teach an accountant how the DOM works and why some fields have to be selected by an INPUT tag and others by a SPAN tag — they’re numbers geeks. They just don’t think visually.
What kills me here is that we use Google for e-mail and documents — and App Script is disabled. Sheets with App Script is one of those “unreasonably effective” tools because it basically allows business analysts to build and test business logic behind even complex web applications (Sheets has some extra data management functions that make it far more powerful than Excel). Or really anything else they want, but with a relatively easy path to converting that process step to a scalable API if that becomes necessary. Heck, I’ve been known to build working wireframes in Sheets — most business applications are glorified spreadsheets anyway.
Sure, this is reminiscent of the Access databases that proliferated across companies in the 90s — with the notable difference that Sheets doesn’t have the locking issues Access does and is performant outside a local LAN. Anyone who is going to be able to manage the complexity of an RPA tool is going to be able to figure out App Script. Sure, the quality of code they write will be crap but it’ll be a lot better than the output of an RPA SpaghettiWare tool. Versioning / inheritance are a lot easier to manage. And there are a wealth of existing libraries and connectors.
TLDR: code > spaghettiware; always. Any employee who doesn’t have the patience to learn to code will not have the patience to build and maintain reliable RPA automations. If you want to enable “citizen-led automation” (fka “shadow IT”) don’t buy expensive RPA tools — spend that money teaching your folks to code in a tool they’re already familiar with.
There are review and approval steps that my agency requires, but aren’t part of the workflow in the system, so I have to remember who needs to see what, whether they’ve responded, etc.
The question is whether the workflow the developers are being told to implement is what the workers actually need to do to get the work done. Also, will there be an ongoing mechanism to revise the software as the work requirements change? In most cases, the answer to both questions is no.
As a government bureaucrat, I'm constantly amazed at the number of systems bought and paid for that essentially recreate functionality that is already built into the enterprise sharepoint.
We can't find money for an in-house sharepoint programmer but we have seven figures to build something in salesforce from scratch.
You're exactly right. Officials like to award contracts for new systems--it makes them look important, it gets the systems integrators to kiss up to them, and the failures won't become apparent until they've moved to their next job.
I'm not in the public sector, but I've seen this a lot in the enterprise world.
That is the problem with how organizations prioritize capex over opex in their budgeting. Most teams I've worked for can request a bigger capex budget for a project because it is "one time". Even though they ask for a bigger budget every year.
But asking for more annual opex is a harder sell on the bean counters, because the increase of operating expense of a department is seen as a permanent thing. It only goes up, never down, so lets just not give them money for it.
I worked two maybe three years doing just this. Hired into a non-IT department, tasked with others to script/automate the in-house ERP (Enterprise Resource Planning) system. We were able to increase the productivity of the department, tasks taking two weeks turning to an hour or less. However, honestly it was a house of cards because the automation we created could fail when the IT department changed the ERP system significantly.
> I’ve talked to IT about matching the system to what we actually need to do, but they say the development contract ended years ago
From the dev perspective this is the problem. There are few workflows that can be described to the developer, who then develops it and throws it over a wall, and it's successful. All of the successful project's I've been part of have been iterative.
Best is when I'm an actual user of a system I'm developing, eating my own dog food as I go, being the user who is feeling the pain. That was the case for much of my career.
Next best is having to handle the user tickets for the system you wrote, which is what I frequently do now. Every ticket is an opportunity to patch and fix and refactor to remove the need for the next such ticket.
I don't think I've ever written a non-trivial program, however thoroughly planned and specified, that was fit to purpose before it was used, and heavily.
Development contracts should be written with this in mind. They either need to extend well into the "post development" phase, or include staffing or training as needed to continue development far past release if not for the life of the software.
Yes, but by the time the system is implemented, the official whose pet project it was had moved on. The new official wants to channel resources to his own pet project, not making the old one work.
Or, alternatively: When the system is originally contracted, the bean counters refuse to permit any budget for any development beyond the initial creation.
Yesterday, there was a discussion (https://news.ycombinator.com/item?id=20174418) about most OSS being so underfinanced that the authors must be doing it because they are either very kind or very crazy.
I was reminded of the David Graeber's Bullshit Jobs, where he describes the testimony of a programmer, who is working on OSS in free time so that he could then cover it with duct tape during the day to make a living. That part of the essay just seemed very apt.
Here, another parts of that essay seem also apt, those about how useless jobs get created in a bureaucracy.
Is there something fundamental to these things, that we don't know how to deal with?
Is it worse is better at play? That people are so damn good at adaptive innovation, that they will rather adapt than to address the root problem?
At IBM we used Lotus Notes as our shadow IT. Lotus Notes databases are just really ambitious excel spreadsheets and most of ours were written by self empowered excel hackers. It was kind of beautiful but as you might expect, a pain in the ass to maintain.
I think the root of the problem is in the title: _Bureaucrat_.
Bureaucrats have no incentive, profit motive, or real tests of competence. In private industry they must perform now or find another job. If a Bureaucrat doesn't perform there are remediation plans that take years to execute. Even then there's no guarantee that the same standard will exist at the end as did the beginning. The Bureaucrat is just a warm body filling a seat.
Source: nearly ten years working in the government. I finally wised up. Working in the government nearly turned me into an anarchist. If this offends anyone I'm sorry, and I know a lot of this resentment is from living that jaded life. But I do encourage you to look elsewhere. There are greener pastures.
My conception of what the government does that is worthwhile, in the abstract, is that it lowers transaction costs, enabling free markets to function a la Ronald Coase's theorem. This is what the private sector is always fighting, what requires regulation. It's like entropy. This is ultimately what makes a modern civilization different from a stone age one.
So measuring productivity in the usual manner and comparing it with the private sector is a non sequitur to me. The benefits aren't booked as revenue or profit.
Fair enough - if it's done well and with competence. My experience was nearly the exact opposite. In terms of 80-20 it was really on 20% of the people doing _any_ of the work.
25 comments
[ 2.3 ms ] story [ 63.5 ms ] threadEDIT: Maybe enterprises should embrace systems like coda.io as a reasonable compromise between rigid enterprise systems and spreadsheets.
What kills me here is that we use Google for e-mail and documents — and App Script is disabled. Sheets with App Script is one of those “unreasonably effective” tools because it basically allows business analysts to build and test business logic behind even complex web applications (Sheets has some extra data management functions that make it far more powerful than Excel). Or really anything else they want, but with a relatively easy path to converting that process step to a scalable API if that becomes necessary. Heck, I’ve been known to build working wireframes in Sheets — most business applications are glorified spreadsheets anyway.
Sure, this is reminiscent of the Access databases that proliferated across companies in the 90s — with the notable difference that Sheets doesn’t have the locking issues Access does and is performant outside a local LAN. Anyone who is going to be able to manage the complexity of an RPA tool is going to be able to figure out App Script. Sure, the quality of code they write will be crap but it’ll be a lot better than the output of an RPA SpaghettiWare tool. Versioning / inheritance are a lot easier to manage. And there are a wealth of existing libraries and connectors.
TLDR: code > spaghettiware; always. Any employee who doesn’t have the patience to learn to code will not have the patience to build and maintain reliable RPA automations. If you want to enable “citizen-led automation” (fka “shadow IT”) don’t buy expensive RPA tools — spend that money teaching your folks to code in a tool they’re already familiar with.
The VA has a solution for you:
https://github.com/department-of-veterans-affairs/caseflow
We can't find money for an in-house sharepoint programmer but we have seven figures to build something in salesforce from scratch.
That is the problem with how organizations prioritize capex over opex in their budgeting. Most teams I've worked for can request a bigger capex budget for a project because it is "one time". Even though they ask for a bigger budget every year.
But asking for more annual opex is a harder sell on the bean counters, because the increase of operating expense of a department is seen as a permanent thing. It only goes up, never down, so lets just not give them money for it.
And now I have a name for about 50% of what I do each day.
From the dev perspective this is the problem. There are few workflows that can be described to the developer, who then develops it and throws it over a wall, and it's successful. All of the successful project's I've been part of have been iterative.
Best is when I'm an actual user of a system I'm developing, eating my own dog food as I go, being the user who is feeling the pain. That was the case for much of my career.
Next best is having to handle the user tickets for the system you wrote, which is what I frequently do now. Every ticket is an opportunity to patch and fix and refactor to remove the need for the next such ticket.
I don't think I've ever written a non-trivial program, however thoroughly planned and specified, that was fit to purpose before it was used, and heavily.
Development contracts should be written with this in mind. They either need to extend well into the "post development" phase, or include staffing or training as needed to continue development far past release if not for the life of the software.
I was reminded of the David Graeber's Bullshit Jobs, where he describes the testimony of a programmer, who is working on OSS in free time so that he could then cover it with duct tape during the day to make a living. That part of the essay just seemed very apt.
Here, another parts of that essay seem also apt, those about how useless jobs get created in a bureaucracy.
Is there something fundamental to these things, that we don't know how to deal with?
Is it worse is better at play? That people are so damn good at adaptive innovation, that they will rather adapt than to address the root problem?
Bureaucrats have no incentive, profit motive, or real tests of competence. In private industry they must perform now or find another job. If a Bureaucrat doesn't perform there are remediation plans that take years to execute. Even then there's no guarantee that the same standard will exist at the end as did the beginning. The Bureaucrat is just a warm body filling a seat.
Source: nearly ten years working in the government. I finally wised up. Working in the government nearly turned me into an anarchist. If this offends anyone I'm sorry, and I know a lot of this resentment is from living that jaded life. But I do encourage you to look elsewhere. There are greener pastures.
So measuring productivity in the usual manner and comparing it with the private sector is a non sequitur to me. The benefits aren't booked as revenue or profit.