I've seen this with my own two eyes. When asked why they removed the unit test they said after they made some rather extensive code changes the tests were failing so they deleted them. Nor was there any trace of irony in their response. When asked didn't they realize that was the whole point of having unit tests they replied that they knew, but they had deadlines to meet and there wasn't enough time. Since controls had been put in place to not ship with failing unit tests but not to check for test coverage they took the easy way out and deleted the tests. I informed management what they had done and thankfully they were relieved of their contract and their consulting company was informed what they had done. I don't know what happened to them after that.
Thing is, once the PM was informed what they had done we did a couple of sprints where we restored the unit tests (which we still had in our history) and got them passing. They had no idea the consultants were doing such shoddy work in order to meet deadlines. The upside of all this is to this day that PM now actively checks the Jenkins dashboard to see the current state of tests and that no tests have been deleted - and that's for all his projects not just that one particular projects. Now that the story is part of our IT lore the new consultants get the message loud and clear we're not afraid to send them packing.
In the end this turned out to be a good story for us - but wow.
Sometimes it's a tough sell. I work for a consulting company that I think would rate as very high quality. We try to bake it into contracts and sales pitches that we work a bit slower because we don't cut corners. This isn't always appreciated by a client who's rushing to market. If you think about the old project management triangle, it's scope, budget quality and you can only have 2 out of 3. If you have a contract with fixed scope and budget, it's quality that goes. Getting a client to sign a contract with flexible scope is not easy unless you have a ton of credibility.
My take on this is that automated tests is a developer tool and doesn't matter to them. As much as they shouldn't care which text editor we use, they shouldn't need to know that we write tests. In fact I tend to avoid as much as possible commits that only impact tests in case they would look at the SCM history and be tempted to blame tests for a delayed delivery (no "fixed the test suite" commit messages). Also I never explicitly count automated testing in my estimates, it's counted implicitly as part of normal development.
The article and this subthread talks about "unit tests", but i think it's really functional or acceptance tests that would have been valuable in these cases. I think it's entirely reasonable for product owners to have some exposure to those.
Hopefully not, because it is completely wrong. They are a way to check that what you test is not broken, which is usually not even remotely the same as "nothing is broken". I have seen so many pieces of code that didn't work in the real world but had many unit tests, good test coverage and were therefore "high quality" - yeah, sure.
I always use double accounting from finances to explain tests. You don't do logic changes in one places, but in a second to double check it actually works out.
The principle is simple. Basically your entire accounting log is always a balanced equation: sum(assets) = sum(liabilities) + sum(owner's equity). Every transaction gets recorded on both sides of this equation, so you can check for (certain kinds of) mistakes by adding up everything on both sides and seeing if it matches. This is a vast oversimplification, and possibly slightly wrong, but hopefully illuminates the connection with unit testing. :)
Thanks, I'm a consultant and will use this analogy in the future. I apologize to all the in-house developers, we truly try to sell in quality but it's extremely hard to convince technologically impaired managers this, especially when the budget is fixed.
The American notion of "manager" is somebody who has ultimate power over the work. The American notion of "employee" is basically "minion"; they are supposed to do whatever they're told. You see it over and over on HN, and even in this thread: developers believe they are supposed to do whatever the boss/client says, even if they know that will lead to disaster.
With power goes responsibility, so if managers want to boss people around, they must understand unit tests and all the other sustainability-oriented development practices.
The alternative, which I would prefer, is that developers start acting like professionals. E.g., like doctors or structural engineers. Those people have codes of ethics and see themselves as having a professional duty. When people ask them for something that's dangerously bad, they say no.
But until that happens, managers really should learn how to manage the stuff they're claiming authority over.
Well of course. If unit tests aren't in the requirements, and it sounds like they are not, why do them? You have a dealine, people want you out of there asap, you don't start doing extra work. That is going to piss off everyone.
Most companies don't value tests, and it sounds like these guys didn't either.
Regarding the manager's response, are unit tests any use against someone who's trying to circumvent them? If a test passes a couple of values into a function and checks the result, wouldn't the "attacker"/programmer just hardcode those expected results into the function for those input values? Perhaps because that kind of code would look obviously suspicious?
My experience is that if you can cheat the tests in this way then they are either too narrow in scope, or badly written tests. This is something that TDD tries to expose when being adhered to strictly, that is, do the simplest thing to make this test pass - if you `assertTrue(isEmpty())` then I can simply `return true` to make you improve your tests.
> The upside of all this is to this day that PM now actively checks the Jenkins dashboard to see the current state of tests and that no tests have been deleted
I just hope that story doesn't continue in the future with something like:
@Test
public void testFoo(){
/* Original testFoo() code here. */
assertTrue(true); // Unnecessary bloat.
}
Several times in the past when I've needed to complain about external consultants cutting corners, I've found the consultants have got in first and told my own management about their unique red-tape cutting powers, and how things like continuous integration (yes, the first thing they did was to switch off the CI server so no-one knew what was happening) were a time-consuming luxury.
Ah, the sadness of seeing your code again after somebody else has had it for a couple years. I feel the author's pain.
I had a company license the source code from one of my old projects to use as the basis for a new site they were building. I built out the bits they needed on a consulting basis and got them launched. Then they started thinking about how expensive I was, and how much they would save if they moved the development overseas.
Fair enough. My philosophy on such matters is that it's best to let a client learn this lesson through painful experience. All the best, guys.
They made it about a year before the site stopped working completely. Reading this article reminded me of the first time I opened the source that came back. You just wouldn't believe that you could do that much damage in such a short time.
I quoted them double my old bill rate and got them back up and running, with all the new functionality they had been wanting, and re-launched. They brought in a new guy to run things, and he started making comments about expenses, and how surely there must be a way to do this cheaper. He even found a local kid who knew some C# and would be willing to help out for a lot less than I was billing.
No problem. All the best.
I took a quick look at the codebase when they got back in touch a few months later and politely declined to come back on board.
Someone tell that to the CEO of the company I worked at last. There were a bunch of undocumented Qlikview reports I was landed with when the CEO decided to fire the IT team who looked after them. We are talking about 35 reports, each in a ridiculous state where you would need to literally modify some date variables and comment out some fields in a load statement, and ensure that the old data was loaded out of a QVD file, new data appended to the end and then the data reloaded into that same QVD file after clearing the file. In short, it worked but an utter disaster and virtually impossible to maintain.
Luckily the CEO kept the lady who sort of knew the procedure for several months, but due to fire fighting dozens of different issues I never really got the chance to fully find a better solution. Luckily, I worked out enough to understand the general principle when this lady left.
Fast forward to my old manager burning out after 4 weeks and me firefighting for several months while the lady the CEO kept around maintained the reports. A new manager, let's call him DW for short, comes in claiming expertise in Qlikview. He works on his reports whilst I work on other important issues.
After 4 weeks a client services colleague asks when her Qliiview dashboards will be fixed. This is news to me, the old maintainer has apparently explained the process in detail to my new manager, and although I've offered DW my help to go through the process he waves me off telling me other work is more important.
DW is in lengthy meetings that day, so I take a deep breath and open said horribly put together but working Qlikview report. Sure enough, data is wrong and virtually no graphs even display. I look at the load script, which sadly relies as part of the procedure an Excel spreadsheet lookup in the mapped drive, which we have standardised to S:. I notice it is now pointing to a mythical Z:. Then I look for the load scripts that work out pending work... gone. Sales data load script statements - gone. You name it, it has been changed. There is no documentation anywhere - not even to explain what Z: points to.
I take a deep breath and wait for my new manager to return. I'm not looking forward to asking about this - he's already stated in front of my other colleagues that "Actiobs speak louder than words" when I explain what I'm working on, and when he asks for a technical explanation about a system issue I know about he tends to cut me off half way through the explanation.
Turns out, he has been "tweaking" and "automating" the reports on live production. In the process he hasn't documented anything and the Z: maps to a Windows share on his laptop. In the process he has literally broken every single Qlikview dashboard through his changes, and he has neglected to even make backups.
I have been working the last two weekends and evenings from home (often till 11pm and later) to fix another more specialised report for a multimillion dollar contract - mainly because day to day operational issues don't allow me any time to fix this complex report - and I'm pretty much at burnout stage. I tell said manager I can't do any more weekends with no break. DW has other ideas and on my third weekend in a row I arrive at 10:30am at the office, rebuild the first report (luckily the basic data schema for each report is similar) which I intend to use as a basis for the 15 other broken reports I've been told are critical to get working, tangle the black box that is the wield semi-pivoted row based data schema of the custom corporate app we use to gather the data, rebook up the dozens of graphs, tabular forms and other odd graphs driven by strange Qkiview expressions and leave at 4:30ammthe Subday morning. Exhausted, I fall asleep at 5:39am, then wake up at 10:45, serve into work, attempt to get the rest of the reports done, but instead get chewed out by my manager and a "project manager", who schedules in a meeting with himself and the CEO the heat day. Rather...
> But without any apparent understanding of why or how print magazines do it
I have honestly never had any idea why print magazines do that. Maybe I've never seen a print magazine do it "right", but I think it seems just as pointless and annoying in print or online.
Well, the problem is I think they are not for you. They are for lazy people like me who don't always read the whole articles. It is a sort of a few sprinkled tl;dr who just want to drive by.
A common way of consuming magazines is to flip through them looking for something interesting, and a good pull quote will intrigue you and make you want to read the surrounding article.
That's called a pull quote, which is pretty common in publishing, but it's usually not done inline right after the text being quoted. That's definitely distracting.
I've been on the other side of this as a consultant. I once was consulting for a place where I asked if I could run the existing tests to confirm I hadn't broken anything when adding in some new functionality. The owner of the project yelled (actually, really yelled) at me to never run the tests, ever. Apparently they deleted production data.
Why on Earth would they not just disable/delete the tests? Especially if they have outside contractors coming in who might not know what a catastrophe of a code base they're working on?
I don't agree with the implicit premise that if the cheaper consultants had been required to maintain the tests and provide coverage for what they wrote, the code quality would have stayed high. In my experience, it would have turned into just as much of a mess, except would have taken three times as much work to maintain all the tests in the process.
Maybe the tests would have had some benefits, but would they have enough to justify all the extra time to write/maintain them? Certainly not.
In my experience, good programmers write good code, tests or not, and bad programmers write bad code, tests or not. I'm not against testing altogether, it makes sense in some contexts (like very widely used libraries). It's also useful for beginners who aren't sure "where to start" with a project. But even in cases where people swear that their tests are invaluable because it's tricky code that breaks often, that's a sign to me that that code needs to be made less fragile, not more well tested. TDD is a cargo cult.
I couldn't agree more. Almost every other form of testing, automated or manual, is more effective at catching bugs than unit testing in my experience. I rarely find unit tests catch any actual bugs (two I can remember in two years in two major codebases) and they are actually a liability rather than an asset because of that in many cases due to the time it takes to create and maintain them.
Often, tests are so far removed from their actual run context with mocks and stubs that they're only testing the functionality of the mock/stub framework itself. OOP code is hardly ever unit test friendly by definition and a lot of bugs are at integration points and due to app state that won't show up in unit tests. There are few places, in other words, where they are useful outside known algorithms with expected outputs in OOP. For one bug prevented per application per year, it's easy to see why spending weeks a year on preventing said bug might not be worth it. I'd rather use them sparingly and test in other ways that are more effective at catching bugs.
What kind of code are you writing? Sure, a Spring service that just integrates a bunch of other services is hard to test due to mocks galore, but any dense logic blocks should be moved to standalone, testable classes.
For my math libraries, Fibonacci compression algos, etc, I find the hundreds of unit tests invaluable. Add to that, in tools like SBT, "sbt ~testOnly com.MyTest" will rerun tests on every change, giving me feedback in under a second, rather than the whole dev/test cycle of save/restart_server/click_on_UI. Write unit tests for the tricky bits that will benefit from them.
Yes, unit tests are ideal for stuff like math libraries. A math function has a specification which will stay fixed for centuries, which you can easily test with a unit test and in the mean while you can change the implementation to optimize performance.
But this is not what coding looks like for the average IT job, which is more about routing data flows between a database and a web service endpoint or building web gui's.
That code can also be unit tested and it can certainly have some benefit to do that while writing new code, because as other have said, it promotes a good coding style with small blocks of testable code, can serve as documentation etc. But for catching regressions, unit tests have much less value there.
These unit tests will typically test only a few lines of business logic where database and web service calls have been mocked away. And chances are that the code will only need to be changed again when some requirement has changed. Which means that your unit test needs to be changed as well, or even completely rewritten. If you're not careful, maintaining your unit test suite can take you more time than updating the business logic.
Which doesn't mean that you shouldn't write any tests at all, but for catching regressions, integration and system tests will have much more value.
These are known algorithms you're talking about, the one place where unit tests actually consistently make sense. I'm not writing a math library, however. I'm writing APIs and web GUIs. We have an immense amount of business logic and we do unit test a lot of it, but as I mentioned above, those unit tests hardly ever catch any bugs and they take a lot of time to maintain. Every time there's a change in the logic, the tests have to be updated. 999/1000 times, when a unit test fails after a chance it's because it needs to be updated, not because there is a bug. Keeping up with those updates is simply not worth the time to catch one bug per year. Neither is writing new unit tests in these areas. Writing them for well-defined algorithms with a limited (or at least definable) set of known inputs and outputs, as I mentioned in my original reply, is still useful. Other types of testing that our developers could do to make sure the code is high quality are much more useful of course and therefore preferred because they actually help make the codebase better rather than creating the illusion of doing so as unit tests often do.
I don't use tests, and I find that when I need to make a change to some code that has, say, four or five different paths through the logic, a lot of times one of the paths will break. Testing all of the paths manually each time there is a change is incredibly tedious and time-consuming. I feel like if I had unit tests I would be able to just see instantly if anything was broken. But, I never can seem to find the time to write them...
Gosh I love made-up numbers, don't you? I've had unit tests just these past few weeks catch bugs within our Redux logic flows, from Action Creators. We have to ship this app next week. Guess what would've happened if I hadn't written those tests?
Now, the thing is I actually agree with you; as does nearly everyone who's replied to you here: Unit tests are not a panacea, and they really shine when as a part of a full test-suite that covers interaction, functionality, integration and more. Of course, this is why any decent testing framework gives you those abilities as well... You're disagreeing with an argument that no-one has put forward, from what I can see.
This is also true in my experience when writing unit tests that are forced as part of the process. I've seen situations where unit tests were prioritized over glaring critical bugs. Sure in an ideal world you would do both but life is a trade-off and there are some people who when faced with a failing project cling to conventional wisdom fanatically instead facing their fear and thinking clearly, because at the end of the day its easier writing unit tests instead of fixing the critical bugs that you haven't been able to solve for the past year.
Sometimes you need to throw the baby out with the bathwater in order to realize that the baby was never in the bathtub, that it had left the bathtub a long time ago.
I agree most of the tests one writes before integration only capture the most trivial stuff that seldom breaks. Now I write tests after a bug or some weird condition arise as not to fall in that trap twice. Capturing bugs as test before solving them also ensure the fix is proper and doesn't regress other stuff. The real issue is to make sure if anything happens, tests can replicate that
So I agree with the sentiment (tests are not a panacea, author says the same), but the first sentence is wrong.
A lot of ink gets spilled about TDD on the internet... Hoping not to rehash all that stuff, I think I can put forward the point simply:
Tests are one of the best (only?) ways to prevent breaking changes to a codebase. If you want to ensure some functionality, a regression test (whether that's unit, integration, or end to end) is the way to do it. Of course, you can't stop someone from coming along and changing the test and invalidating it, but if written correctly a test will offer a 100% guarantee that the codebase fulfills certain requirements.
It doesn't matter how much of a mess is made, what matters is that the system remains running at the end of the day (it just gets harder and harder to change, and runs worse and worse).
TDD may be a cargo cult, but don't throw the baby out with the bath water -- tests are a useful tool in a software developer's repertoire. This fact should become more and more apparent for any developer who has written lots of software. A good developer, in anticipation of the fact that they will eventually write a bug/commit some regression in functionality, will guard themselves.
Tests _promote_ good coding behavior. Yes, people can circumvent them. Yes, people can write terrible tests. Yes, people could theoretically write great code without tests (but most won't).
But if you're doing TDD with the intention of doing it right, you'll find code smells early, you'll tend to write things in a more change-tolerant way the first time, and you'll get some extra confidence in your code.
There's a reason switching to TDD from not using tests takes time and effort (general estimates say 6 months before you reach previous productivity levels) - this is the time in which you learning to code better. After which, your productivity tends to exceed your previous values.
I went through this a couple of years ago when TDD became something that moved from "it sounds cool but I don't have time for that" to something I could really do. It was hard to learn, but in the end I found that instead of writing code to pass tests, I wrote code that I liked, and the tests promoted those skills/behaviors.
Much of the pain people have from writing tests (in my limited experience) stems from:
* They are trying to write their tests after writing the code, and while their code may be "okay", it's not as free of side effects and external state as it could be == hard to test.
* Poor general education as to HOW to write good tests. I recall a few weeks where I tried to reconcile advice from person who said "don't have tests for simple existence" and another that said "write tests as you code". Eventually I figured out that the red/green/refactor cycle means you CAN and SHOULD delete redundant tests, but there's no reason not to write them as you start.
* Tests can be hard for exploratory code. Again, this experience. Once you find the proper balance, tests are not only NOT a cost, but a BENEFIT.
"After which, your productivity tends to exceed your previous values."
Really, this is the only bit that I disagree with. Tests do promote good code behavior, and they can be useful for certain cases (another poster mentioned math libraries, which are a good case for them.) And they are good for learning how to code and learning to identify "code smells". But once a programmer has learned those skills, maybe they can stop wasting time writing tests.
In my experience, the real productivity gains from unit testing are less than the productivity drag from writing and maintaining the tests. Not to mention the upfront cost of bringing a messy codebase under test. Tests also often require contorting your codebase in weird ways to make it testable. Sometimes this is an improvement and makes things more modular, sometimes it's just adding layers and indirections for no real reason other than "testability".
If your codebase is breaking all the time, you've got problems. But maybe those problems are sloppy programmers, lack of code review and badly designed codebase. Unit testing is one way to address those problems, but discipline, experience, code reviews, firing bad coders, mentoring and not being sloppy are other ways that are often overlooked.
Every team and organization varies, and if unit testing works for you, great. I just feel like it's becoming dogma, rather than simply another tool or strategy.
In my (personal, anecdotal) experience, you might be able to get away with rely on skills encouraged by tests without tests much of the time, but when a test fails that I didn't expect because I made a small change, and reveals I had a false assumption, I'm saved hours of going the wrong way, because without the test I'd likely have not noticed the unexpected bug until after I added a lot more code. When it was discovered, I'd have no idea which section of code caused it, and because I clearly hadn't anticipated the problem, it won't be my first guess either.
Similarly, when I need to make a quick last minute change, the tests give me much more confidence that there are no unintended impacts. I don't know how to rate that confidence, but it's definitely worth something.
Lastly, tests serve as the ultimate API documentation for me. I have a terrible memory, so when I return to code I've not touched in months, I'm usually a bit lost. The tests tell me exactly what I want to know: what the code does, how to call it, and specific requirements that are on it. So I'm more able to switch around my code with confidence and less spin up/task switching time. (And I'll admit, this is not a highly transferable benefit of tests - Of my coworkers, I can only think of 1 out of 5 in the last two years where I gained the same benefit from their tests. I pin this on the poor education regarding the real details of writing tests, but it might just be that the tests as are personal as some comments: Where I write exactly what future me will need, future someone else will have different needs)
All of that said, we completely agree that blind dogma without true understanding is just meaningless rote. To tie it back to the OP, he mentioned that he didn't shoot for 100% test coverage, but instead for where he found value in having tests. I'm likely far closer to that 100% coverage than you are, but if we're both making our choices based on experience and consideration, we're doing a lot better than some others.
Of course, then when the OP was asked back for one particular thing, he didn't go and setup the unit tests again, either. He did the same exact thing as the consultants.
I don't think this should be surprising to anyone.
Outside developers don't have any real ownership of their work product. Even if they know better in principle (and many do), they don't have any incentive to practice good stewardship. And bargain-basement outside developers aren't magical productivity pixies; the reason they are able to charge lower prices because they don't need to charge you for all the things they aren't taking the time to do.
Moral of the story: Great developers are profit centers, bad developers are cost centers.
Unfortunately, many execs without development experience neglect to see it this way.
They see "expensive cost center" that can easily replaced with cheaper labor. Cool thing is, as others (like the author) have said, they tend to get burned and end up re-hiring you on even better-than-original terms.
I have this same underlying feeling every place I work. Maybe it's because I'm not in the Valley, but in the last several jobs I've had, I was the only person advocating for things like unit tests, release pipelines, or even VCS(!). And each time I get a new job, team members and bosses from the old one inevitably call with similar self-inflicted crisis like this.
I don't think it's because you're not in the valley. I think it's because only a few percent of developers in the world care enough to become leads and make this stuff happen across the team.
That's normal. Any non-essential complexity gets dropped. Accidental complexity accumulates.
In one project next developer ate my build script. Why bother with compiling less to css if you can just edit css and forget the less files and build script. Same went for r.js bundler.
In other project I participated angular got dropped because people couldn't be bothered with it.
The bulk of the first comment of the article was as good as the article IMO. Quoting here:
>> But maybe the more disorganized the code, the more “less expensive” consultant work there is to do (charge)?
> This is implying maybe they intentionally disorganize the code so they'll get more billable hours?
> I seriously seriously doubt it.
> Rather, it's as simple as: If you want to add features or make changes paying the cheapest possible amount, you simply aren't paying for well-organized code.
> It's not just less experienced programmers, it's less experienced programmers trying to get you the feature in the least possible hours. They aren't doing artificial things to bill you more hours, it's in fact quite the reverse -- they really are trying to get the feature done in as few hours as possible, to bill you as few hours as possible.
> And when you do that, you simply don't have time to organize the code well, or keep the tests working. You're shoving it out the door as quick as you can. Of course it takes more time to keep the code well-organized than to just hack it until the feature works as quickly as possible. Of course it takes more time to maintain the tests -- over the short term, and all you ever have is the short term when you're paying and getting paid little by little feature by feature.
> They have other customers, they don't need to bill you artificial hours. They really are billing you as few hours as they possibly can -- because that's how they get customers, being the cheapest. That's exactly what the client asked for, it to be done as cheaply as possible.
> You get what you pay for.
I have only recently started working for a consultant that bills hourly. It's an eye opener. We are a small shop of experienced developers, we work for people that do want us to produce quality, and we do, we do good work, and we do write tests and maintain our tests. But it is a still a constant struggle inside my head between doing it as right as I really want to, and not charging the client more hours than the feature seems 'worth'.
> If you as a client prioritizes price even higher over quality (and you have no way to judge internal quality anyway, not being a coder, so why wouldn't you? It looks nice and works, what else can you judge?), if you're hiring less expensive developers to save money... they might be skilled devs in fact, but if you are trying to get it done as cheaply as possible, you are not going to get well-organized code. And you as a client don't care, what do you care about well-organized code? Until it reaches a breaking point where your technical debt is so high you can't get any more features at all, and you realize, oops... and probably still don't understand what you did wrong or could have done differently.
> Good software is expensive. Too expensive. More expensive than most people who need good software can afford. Which is why we have so much shitty software.
I'm going to be a contrarian here and say that if unit tests, in most cases, made economic sense they'd be a given in almost all software development.
But they're not. Do we assume business owners are dumb? Do we assume managers are dumb? That's a popular position for technical people. "Oh, you just don't understand how valuable those unit tests are!"
But the cost of maintaining unit test vs. the cost of fixing bugs later might favor just fixing the bugs later. Maybe it doesn't sometimes - but one is an upfront cost you know about, and one is a nebulous cost that you might or might not have to pay later.
Given that, I know what most people paying the bills are going to choose.
Business owners are not dumb, they're ignorant ... like the rest of us.
Software engineering is an immature field ; we still have doubts about what practises are beneficial.
It comes as no surprise that our managers might not always know what will benefit their software in the long run.
The whole notion of technical debt is just starting to make its way into their heads (and into ours).
How many times was your manager astonished by the time estimate you just gave her while she was "only asking you to add a checkbox"?
It's a question of incentive alignment. Managers are not dumb, their incentives are aligned to short-term success rather than long term. Consider the difference in scenarios between producing ok code very quickly and good code more slowly. The manager will immediately feel the negative consequences of taking the second decision, if their incentives are badly designed. Meanwhile they may never feel the negative consequences of the first decision. When bad code breaks down, how likely is the manager of three years prior the one to take the blame or heat for it, let alone face real consequences? Often they won't.
Some of the reasons why I delete unit tests from time to time:
- Often the quality of unit test code is a lot worse than the production code.
- Some times people test too much in a single test (ie. more than one unit) making it very hard to change the design of the application.
- Some times a unit test misunderstands how the application is supposed to work or the requirements change.
- Some times a unit test tests something too little or too trivial.
- Some other test is already covering the case.
I personally don't weight 100% test coverage as critical. Often other aspects of QA in application development are more important (i.e. integration test or documentation).
I had a different experience with tests. 18 years ago at one company I laid out a test plan. Some of it was automated, the majority of it was manual. It applied specifically to a large custom built application.
2 years ago, I was at an entirely different company. They handed me their test plan documentation. Lo and behold, it was my document, now laden with another consulting company's logo. It was missing a few tests, but there was nothing new.
I was angry for a solid 4 hours. Then I had to laugh about it. I would be flattered that my 18 year old work was still being used, but it was quite simply dated. If a big company was being cheap and getting things built by offshore resources who had no better way to accomplish it than by stealing my work, then they got what they deserved.
maybe the more disorganized the code, the more “less expensive” consultant work there is to do (charge)?
I know of one consulting shop that had a dependable stream of income from a major European bank, because that bank had a project that didn't use source control, so they got to be the periodic hero and get paid for it.
I, very recently, began writing a suite of tests to go along-side a complex mobile app. During a stand-up meeting I (somewhat) proudly announced that for a particularly onerous piece of business logic, I had 100% coverage with tests that run in less than a second.
The project manager in the room was worried about the time it was going to take. All the developers in the room piped up to say "oh, no, this actually saves time"
The next day, I found myself in a private meeting room with the project manager and my line manager.
Both were trying to tell me that unit tests are not part of the scope of the project, not were they approved and, that I was to stop writing them.
Manual QA was the way to go at the end of the project.
Yesterday, I was in the office from 10am to 2am on a national holiday addressing issues arising from QA.
It was unproductive to spend the day wondering how many of those defects I'd have been able to catch months ago if I'd been writing tests all along.
When you're the only one on your team who respects the tests, the world feels like a very bleak place.
It does suck how much of the time you wind up in the situation that, to make any progress, you have to work in secret behind some smokescreen of other less useful work, to keep the micromanaging and the hemming and hawing from consuming any improvements.
It's not uncommon that orders of magnitude more time wasted in debate and FUD'ing round on things than it takes to actually do them.
Cheap labor != consultant. In fact, real consultants tend to be expensive. A real consultant is an expert in a field, not an asterisk or wildcard for anything.
Man, this makes me sad. Not about the atrophied codebase, but about the inevitably extreme difficulty of keeping one properly maintained.
Expertise in software maintenance isn't common and it isn't cheap. Every greenfield dev is going to eventually see his project handed off to people who won't understand or respect his conventions and methods. That mostly destroys it, but business requirements can put it on chemo and a food tube and keep it alive against its will for a long, long time, and it's not a pretty sight.
It's practically impossible to. I've been maintaining an ancient Rails codebase for the last two and a half years. I'm perhaps the fourth or fifth such guy to do so since it was built. The project was never updated past a certain point release and it's more work than it would be economical to do to do that now. I'm experienced enough now to take it on, but both me and the company have better things to do with our time and resources.
It is so ridiculously easy to make a bad decision that seems like a good decision that instead ultimately dooms the project. Software is like this weird life-form from another part of the galaxy whose motives are so alien to us that we can't even begin to come to grips with it. We know it's not actively plotting against us, well, pretty sure, but more than that? All you can do is shrug your shoulders and hope it doesn't consider you a threat today.
FWIW I've been eating my own unit tests these days. I find they slow me down and distract me from what I'm doing.
You've got the "large scope" unit tests that test functions that take lots of dependencies, via mocking. These often end up being so mocked, that you're testing the mocks more than the function itself. You feel smart when you write it, but a week afterwards you can't even remember what it's testing.
You've got the "small scope" tests that test small pure logic functions. Great, but these hardly ever change. It doesn't seem worth the overhead to maintain a test for unchanging code.
There's a "fuzzy middle" that could be reasonably worth unit testing, but these are the functions that are most likely to get completely ripped up during refactoring and new feature dev.
To me, a sound architecture (no global variables, a minimum of mutation[1], etc) and a sound type system means things generally work the first time anyway, without the need to maintain a big honking test suite and everything required to run it.
I'm finding it's much quicker to get things out the door without this overhead, the architecture ends up being simpler (far simpler in some cases) because I don't have to put test hooks in for everything, I can refactor without the worry of how many unit tests will break, and I've experienced no downturn in quality.
Of course there are cases (public core libraries, large distributed dev teams, really complex pure logic) where unit tests are absolutely necessary, but for most of my projects these days I'm finding them to be redundant.
[1] In fact, one could argue that the greatest benefit from the TDD phase was that TDD is harder with globals and mutation, so it reduced incidences thereof.
105 comments
[ 3.2 ms ] story [ 181 ms ] threadThing is, once the PM was informed what they had done we did a couple of sprints where we restored the unit tests (which we still had in our history) and got them passing. They had no idea the consultants were doing such shoddy work in order to meet deadlines. The upside of all this is to this day that PM now actively checks the Jenkins dashboard to see the current state of tests and that no tests have been deleted - and that's for all his projects not just that one particular projects. Now that the story is part of our IT lore the new consultants get the message loud and clear we're not afraid to send them packing.
In the end this turned out to be a good story for us - but wow.
It's not laziness or apathy. It's just that we're only paid to do a task.
That's contractor-talk.
If you're a consultant, consult. Educate your client, make sure they understand the consequences of what they're not paying you for.
Reminds me of the kid going "I want to play, I want to play".. sorry George you suck at basketball, go shoot some free throws.
"You won't believe one startup's new growth hack, their teammates hate them!"
They shouldn't have to because they'll listen to us and allow us to write and maintain tests as we develop, right?
The American notion of "manager" is somebody who has ultimate power over the work. The American notion of "employee" is basically "minion"; they are supposed to do whatever they're told. You see it over and over on HN, and even in this thread: developers believe they are supposed to do whatever the boss/client says, even if they know that will lead to disaster.
With power goes responsibility, so if managers want to boss people around, they must understand unit tests and all the other sustainability-oriented development practices.
The alternative, which I would prefer, is that developers start acting like professionals. E.g., like doctors or structural engineers. Those people have codes of ethics and see themselves as having a professional duty. When people ask them for something that's dangerously bad, they say no.
But until that happens, managers really should learn how to manage the stuff they're claiming authority over.
Both consultant sales dudes and customers are responsible for putting shitty SOWs together.
Most companies don't value tests, and it sounds like these guys didn't either.
I just hope that story doesn't continue in the future with something like:
Several times in the past when I've needed to complain about external consultants cutting corners, I've found the consultants have got in first and told my own management about their unique red-tape cutting powers, and how things like continuous integration (yes, the first thing they did was to switch off the CI server so no-one knew what was happening) were a time-consuming luxury.
I had a company license the source code from one of my old projects to use as the basis for a new site they were building. I built out the bits they needed on a consulting basis and got them launched. Then they started thinking about how expensive I was, and how much they would save if they moved the development overseas.
Fair enough. My philosophy on such matters is that it's best to let a client learn this lesson through painful experience. All the best, guys.
They made it about a year before the site stopped working completely. Reading this article reminded me of the first time I opened the source that came back. You just wouldn't believe that you could do that much damage in such a short time.
I quoted them double my old bill rate and got them back up and running, with all the new functionality they had been wanting, and re-launched. They brought in a new guy to run things, and he started making comments about expenses, and how surely there must be a way to do this cheaper. He even found a local kid who knew some C# and would be willing to help out for a lot less than I was billing.
No problem. All the best.
I took a quick look at the codebase when they got back in touch a few months later and politely declined to come back on board.
Why not make code acceptance contractually dependent on a third party code review?
Luckily the CEO kept the lady who sort of knew the procedure for several months, but due to fire fighting dozens of different issues I never really got the chance to fully find a better solution. Luckily, I worked out enough to understand the general principle when this lady left.
Fast forward to my old manager burning out after 4 weeks and me firefighting for several months while the lady the CEO kept around maintained the reports. A new manager, let's call him DW for short, comes in claiming expertise in Qlikview. He works on his reports whilst I work on other important issues.
After 4 weeks a client services colleague asks when her Qliiview dashboards will be fixed. This is news to me, the old maintainer has apparently explained the process in detail to my new manager, and although I've offered DW my help to go through the process he waves me off telling me other work is more important.
DW is in lengthy meetings that day, so I take a deep breath and open said horribly put together but working Qlikview report. Sure enough, data is wrong and virtually no graphs even display. I look at the load script, which sadly relies as part of the procedure an Excel spreadsheet lookup in the mapped drive, which we have standardised to S:. I notice it is now pointing to a mythical Z:. Then I look for the load scripts that work out pending work... gone. Sales data load script statements - gone. You name it, it has been changed. There is no documentation anywhere - not even to explain what Z: points to.
I take a deep breath and wait for my new manager to return. I'm not looking forward to asking about this - he's already stated in front of my other colleagues that "Actiobs speak louder than words" when I explain what I'm working on, and when he asks for a technical explanation about a system issue I know about he tends to cut me off half way through the explanation.
Turns out, he has been "tweaking" and "automating" the reports on live production. In the process he hasn't documented anything and the Z: maps to a Windows share on his laptop. In the process he has literally broken every single Qlikview dashboard through his changes, and he has neglected to even make backups.
I have been working the last two weekends and evenings from home (often till 11pm and later) to fix another more specialised report for a multimillion dollar contract - mainly because day to day operational issues don't allow me any time to fix this complex report - and I'm pretty much at burnout stage. I tell said manager I can't do any more weekends with no break. DW has other ideas and on my third weekend in a row I arrive at 10:30am at the office, rebuild the first report (luckily the basic data schema for each report is similar) which I intend to use as a basis for the 15 other broken reports I've been told are critical to get working, tangle the black box that is the wield semi-pivoted row based data schema of the custom corporate app we use to gather the data, rebook up the dozens of graphs, tabular forms and other odd graphs driven by strange Qkiview expressions and leave at 4:30ammthe Subday morning. Exhausted, I fall asleep at 5:39am, then wake up at 10:45, serve into work, attempt to get the rest of the reports done, but instead get chewed out by my manager and a "project manager", who schedules in a meeting with himself and the CEO the heat day. Rather...
I have honestly never had any idea why print magazines do that. Maybe I've never seen a print magazine do it "right", but I think it seems just as pointless and annoying in print or online.
The Verge does it as well and I think it is OK.
The presentation isn't particularly nice in this case, I suppose.
I didn't spend much time consulting for them.
The yelling problem, I'd be tempted to fix for free...
So, run the tests?
I would love to see the downhill-rolling trainwreck that eventually produced this sort of inferno.
Maybe the tests would have had some benefits, but would they have enough to justify all the extra time to write/maintain them? Certainly not.
In my experience, good programmers write good code, tests or not, and bad programmers write bad code, tests or not. I'm not against testing altogether, it makes sense in some contexts (like very widely used libraries). It's also useful for beginners who aren't sure "where to start" with a project. But even in cases where people swear that their tests are invaluable because it's tricky code that breaks often, that's a sign to me that that code needs to be made less fragile, not more well tested. TDD is a cargo cult.
Often, tests are so far removed from their actual run context with mocks and stubs that they're only testing the functionality of the mock/stub framework itself. OOP code is hardly ever unit test friendly by definition and a lot of bugs are at integration points and due to app state that won't show up in unit tests. There are few places, in other words, where they are useful outside known algorithms with expected outputs in OOP. For one bug prevented per application per year, it's easy to see why spending weeks a year on preventing said bug might not be worth it. I'd rather use them sparingly and test in other ways that are more effective at catching bugs.
What kind of code are you writing? Sure, a Spring service that just integrates a bunch of other services is hard to test due to mocks galore, but any dense logic blocks should be moved to standalone, testable classes.
For my math libraries, Fibonacci compression algos, etc, I find the hundreds of unit tests invaluable. Add to that, in tools like SBT, "sbt ~testOnly com.MyTest" will rerun tests on every change, giving me feedback in under a second, rather than the whole dev/test cycle of save/restart_server/click_on_UI. Write unit tests for the tricky bits that will benefit from them.
But this is not what coding looks like for the average IT job, which is more about routing data flows between a database and a web service endpoint or building web gui's. That code can also be unit tested and it can certainly have some benefit to do that while writing new code, because as other have said, it promotes a good coding style with small blocks of testable code, can serve as documentation etc. But for catching regressions, unit tests have much less value there. These unit tests will typically test only a few lines of business logic where database and web service calls have been mocked away. And chances are that the code will only need to be changed again when some requirement has changed. Which means that your unit test needs to be changed as well, or even completely rewritten. If you're not careful, maintaining your unit test suite can take you more time than updating the business logic. Which doesn't mean that you shouldn't write any tests at all, but for catching regressions, integration and system tests will have much more value.
If you don't have unit tests for your stored procedures, how do you know any changes to them actually work? Run it on production data?
Gosh I love made-up numbers, don't you? I've had unit tests just these past few weeks catch bugs within our Redux logic flows, from Action Creators. We have to ship this app next week. Guess what would've happened if I hadn't written those tests?
Now, the thing is I actually agree with you; as does nearly everyone who's replied to you here: Unit tests are not a panacea, and they really shine when as a part of a full test-suite that covers interaction, functionality, integration and more. Of course, this is why any decent testing framework gives you those abilities as well... You're disagreeing with an argument that no-one has put forward, from what I can see.
Sometimes you need to throw the baby out with the bathwater in order to realize that the baby was never in the bathtub, that it had left the bathtub a long time ago.
I would suggest TDD or not. Vast swathes of unit tests that are unrelated to the API/contract or not
But yes, that's the essence.
A lot of ink gets spilled about TDD on the internet... Hoping not to rehash all that stuff, I think I can put forward the point simply:
Tests are one of the best (only?) ways to prevent breaking changes to a codebase. If you want to ensure some functionality, a regression test (whether that's unit, integration, or end to end) is the way to do it. Of course, you can't stop someone from coming along and changing the test and invalidating it, but if written correctly a test will offer a 100% guarantee that the codebase fulfills certain requirements.
It doesn't matter how much of a mess is made, what matters is that the system remains running at the end of the day (it just gets harder and harder to change, and runs worse and worse).
TDD may be a cargo cult, but don't throw the baby out with the bath water -- tests are a useful tool in a software developer's repertoire. This fact should become more and more apparent for any developer who has written lots of software. A good developer, in anticipation of the fact that they will eventually write a bug/commit some regression in functionality, will guard themselves.
Tests _promote_ good coding behavior. Yes, people can circumvent them. Yes, people can write terrible tests. Yes, people could theoretically write great code without tests (but most won't).
But if you're doing TDD with the intention of doing it right, you'll find code smells early, you'll tend to write things in a more change-tolerant way the first time, and you'll get some extra confidence in your code.
There's a reason switching to TDD from not using tests takes time and effort (general estimates say 6 months before you reach previous productivity levels) - this is the time in which you learning to code better. After which, your productivity tends to exceed your previous values.
I went through this a couple of years ago when TDD became something that moved from "it sounds cool but I don't have time for that" to something I could really do. It was hard to learn, but in the end I found that instead of writing code to pass tests, I wrote code that I liked, and the tests promoted those skills/behaviors.
Much of the pain people have from writing tests (in my limited experience) stems from:
* They are trying to write their tests after writing the code, and while their code may be "okay", it's not as free of side effects and external state as it could be == hard to test.
* Poor general education as to HOW to write good tests. I recall a few weeks where I tried to reconcile advice from person who said "don't have tests for simple existence" and another that said "write tests as you code". Eventually I figured out that the red/green/refactor cycle means you CAN and SHOULD delete redundant tests, but there's no reason not to write them as you start.
* Tests can be hard for exploratory code. Again, this experience. Once you find the proper balance, tests are not only NOT a cost, but a BENEFIT.
Really, this is the only bit that I disagree with. Tests do promote good code behavior, and they can be useful for certain cases (another poster mentioned math libraries, which are a good case for them.) And they are good for learning how to code and learning to identify "code smells". But once a programmer has learned those skills, maybe they can stop wasting time writing tests.
In my experience, the real productivity gains from unit testing are less than the productivity drag from writing and maintaining the tests. Not to mention the upfront cost of bringing a messy codebase under test. Tests also often require contorting your codebase in weird ways to make it testable. Sometimes this is an improvement and makes things more modular, sometimes it's just adding layers and indirections for no real reason other than "testability".
If your codebase is breaking all the time, you've got problems. But maybe those problems are sloppy programmers, lack of code review and badly designed codebase. Unit testing is one way to address those problems, but discipline, experience, code reviews, firing bad coders, mentoring and not being sloppy are other ways that are often overlooked.
Every team and organization varies, and if unit testing works for you, great. I just feel like it's becoming dogma, rather than simply another tool or strategy.
In my (personal, anecdotal) experience, you might be able to get away with rely on skills encouraged by tests without tests much of the time, but when a test fails that I didn't expect because I made a small change, and reveals I had a false assumption, I'm saved hours of going the wrong way, because without the test I'd likely have not noticed the unexpected bug until after I added a lot more code. When it was discovered, I'd have no idea which section of code caused it, and because I clearly hadn't anticipated the problem, it won't be my first guess either.
Similarly, when I need to make a quick last minute change, the tests give me much more confidence that there are no unintended impacts. I don't know how to rate that confidence, but it's definitely worth something.
Lastly, tests serve as the ultimate API documentation for me. I have a terrible memory, so when I return to code I've not touched in months, I'm usually a bit lost. The tests tell me exactly what I want to know: what the code does, how to call it, and specific requirements that are on it. So I'm more able to switch around my code with confidence and less spin up/task switching time. (And I'll admit, this is not a highly transferable benefit of tests - Of my coworkers, I can only think of 1 out of 5 in the last two years where I gained the same benefit from their tests. I pin this on the poor education regarding the real details of writing tests, but it might just be that the tests as are personal as some comments: Where I write exactly what future me will need, future someone else will have different needs)
All of that said, we completely agree that blind dogma without true understanding is just meaningless rote. To tie it back to the OP, he mentioned that he didn't shoot for 100% test coverage, but instead for where he found value in having tests. I'm likely far closer to that 100% coverage than you are, but if we're both making our choices based on experience and consideration, we're doing a lot better than some others.
Of course they don't bother with unit tests when asked to change thing A that breaks a bunch of them.
Outside developers don't have any real ownership of their work product. Even if they know better in principle (and many do), they don't have any incentive to practice good stewardship. And bargain-basement outside developers aren't magical productivity pixies; the reason they are able to charge lower prices because they don't need to charge you for all the things they aren't taking the time to do.
Unfortunately, many execs without development experience neglect to see it this way.
They see "expensive cost center" that can easily replaced with cheaper labor. Cool thing is, as others (like the author) have said, they tend to get burned and end up re-hiring you on even better-than-original terms.
Kudos to the author for doing good work!
In one project next developer ate my build script. Why bother with compiling less to css if you can just edit css and forget the less files and build script. Same went for r.js bundler.
In other project I participated angular got dropped because people couldn't be bothered with it.
>> But maybe the more disorganized the code, the more “less expensive” consultant work there is to do (charge)?
> This is implying maybe they intentionally disorganize the code so they'll get more billable hours?
> I seriously seriously doubt it.
> Rather, it's as simple as: If you want to add features or make changes paying the cheapest possible amount, you simply aren't paying for well-organized code.
> It's not just less experienced programmers, it's less experienced programmers trying to get you the feature in the least possible hours. They aren't doing artificial things to bill you more hours, it's in fact quite the reverse -- they really are trying to get the feature done in as few hours as possible, to bill you as few hours as possible.
> And when you do that, you simply don't have time to organize the code well, or keep the tests working. You're shoving it out the door as quick as you can. Of course it takes more time to keep the code well-organized than to just hack it until the feature works as quickly as possible. Of course it takes more time to maintain the tests -- over the short term, and all you ever have is the short term when you're paying and getting paid little by little feature by feature.
> They have other customers, they don't need to bill you artificial hours. They really are billing you as few hours as they possibly can -- because that's how they get customers, being the cheapest. That's exactly what the client asked for, it to be done as cheaply as possible.
> You get what you pay for.
I have only recently started working for a consultant that bills hourly. It's an eye opener. We are a small shop of experienced developers, we work for people that do want us to produce quality, and we do, we do good work, and we do write tests and maintain our tests. But it is a still a constant struggle inside my head between doing it as right as I really want to, and not charging the client more hours than the feature seems 'worth'.
> If you as a client prioritizes price even higher over quality (and you have no way to judge internal quality anyway, not being a coder, so why wouldn't you? It looks nice and works, what else can you judge?), if you're hiring less expensive developers to save money... they might be skilled devs in fact, but if you are trying to get it done as cheaply as possible, you are not going to get well-organized code. And you as a client don't care, what do you care about well-organized code? Until it reaches a breaking point where your technical debt is so high you can't get any more features at all, and you realize, oops... and probably still don't understand what you did wrong or could have done differently.
> Good software is expensive. Too expensive. More expensive than most people who need good software can afford. Which is why we have so much shitty software.
But they're not. Do we assume business owners are dumb? Do we assume managers are dumb? That's a popular position for technical people. "Oh, you just don't understand how valuable those unit tests are!"
But the cost of maintaining unit test vs. the cost of fixing bugs later might favor just fixing the bugs later. Maybe it doesn't sometimes - but one is an upfront cost you know about, and one is a nebulous cost that you might or might not have to pay later.
Given that, I know what most people paying the bills are going to choose.
Software engineering is an immature field ; we still have doubts about what practises are beneficial. It comes as no surprise that our managers might not always know what will benefit their software in the long run. The whole notion of technical debt is just starting to make its way into their heads (and into ours).
How many times was your manager astonished by the time estimate you just gave her while she was "only asking you to add a checkbox"?
- Often the quality of unit test code is a lot worse than the production code.
- Some times people test too much in a single test (ie. more than one unit) making it very hard to change the design of the application.
- Some times a unit test misunderstands how the application is supposed to work or the requirements change.
- Some times a unit test tests something too little or too trivial.
- Some other test is already covering the case.
I personally don't weight 100% test coverage as critical. Often other aspects of QA in application development are more important (i.e. integration test or documentation).
2 years ago, I was at an entirely different company. They handed me their test plan documentation. Lo and behold, it was my document, now laden with another consulting company's logo. It was missing a few tests, but there was nothing new.
I was angry for a solid 4 hours. Then I had to laugh about it. I would be flattered that my 18 year old work was still being used, but it was quite simply dated. If a big company was being cheap and getting things built by offshore resources who had no better way to accomplish it than by stealing my work, then they got what they deserved.
I know of one consulting shop that had a dependable stream of income from a major European bank, because that bank had a project that didn't use source control, so they got to be the periodic hero and get paid for it.
The project manager in the room was worried about the time it was going to take. All the developers in the room piped up to say "oh, no, this actually saves time"
The next day, I found myself in a private meeting room with the project manager and my line manager.
Both were trying to tell me that unit tests are not part of the scope of the project, not were they approved and, that I was to stop writing them.
Manual QA was the way to go at the end of the project.
Yesterday, I was in the office from 10am to 2am on a national holiday addressing issues arising from QA.
It was unproductive to spend the day wondering how many of those defects I'd have been able to catch months ago if I'd been writing tests all along.
When you're the only one on your team who respects the tests, the world feels like a very bleak place.
It's not uncommon that orders of magnitude more time wasted in debate and FUD'ing round on things than it takes to actually do them.
You'll possibly seen as a genius, but you certainly won't be working on public holidays.
Expertise in software maintenance isn't common and it isn't cheap. Every greenfield dev is going to eventually see his project handed off to people who won't understand or respect his conventions and methods. That mostly destroys it, but business requirements can put it on chemo and a food tube and keep it alive against its will for a long, long time, and it's not a pretty sight.
It's practically impossible to. I've been maintaining an ancient Rails codebase for the last two and a half years. I'm perhaps the fourth or fifth such guy to do so since it was built. The project was never updated past a certain point release and it's more work than it would be economical to do to do that now. I'm experienced enough now to take it on, but both me and the company have better things to do with our time and resources.
It is so ridiculously easy to make a bad decision that seems like a good decision that instead ultimately dooms the project. Software is like this weird life-form from another part of the galaxy whose motives are so alien to us that we can't even begin to come to grips with it. We know it's not actively plotting against us, well, pretty sure, but more than that? All you can do is shrug your shoulders and hope it doesn't consider you a threat today.
You've got the "large scope" unit tests that test functions that take lots of dependencies, via mocking. These often end up being so mocked, that you're testing the mocks more than the function itself. You feel smart when you write it, but a week afterwards you can't even remember what it's testing.
You've got the "small scope" tests that test small pure logic functions. Great, but these hardly ever change. It doesn't seem worth the overhead to maintain a test for unchanging code.
There's a "fuzzy middle" that could be reasonably worth unit testing, but these are the functions that are most likely to get completely ripped up during refactoring and new feature dev.
To me, a sound architecture (no global variables, a minimum of mutation[1], etc) and a sound type system means things generally work the first time anyway, without the need to maintain a big honking test suite and everything required to run it.
I'm finding it's much quicker to get things out the door without this overhead, the architecture ends up being simpler (far simpler in some cases) because I don't have to put test hooks in for everything, I can refactor without the worry of how many unit tests will break, and I've experienced no downturn in quality.
Of course there are cases (public core libraries, large distributed dev teams, really complex pure logic) where unit tests are absolutely necessary, but for most of my projects these days I'm finding them to be redundant.
[1] In fact, one could argue that the greatest benefit from the TDD phase was that TDD is harder with globals and mutation, so it reduced incidences thereof.