Ask HN: How common is it to work on projects with no testing?
I have been working in an investment bank for the past months(my first job), and I was surprised to find out out that my team doesn't write any tests at all.
There is also no code reviews, you are responsible for your own code. There seems to exist a "hustle" mentality, where you need to ship at all costs, the engineers frequently work from 9am-9pm and then some weekends. Is this common practice in the industry?
112 comments
[ 4.4 ms ] story [ 58.3 ms ] threadI read a book written by a brand marketer a couple years ago. He dissected about two dozen big companies that failed and, guess what, every one was due to poor brand marketing.
If the early engineers were experienced guys, a basic test suite and deployment pipeline would have been the first priority at the onset. It makes releasing frequent commits a breeze. Or else, they learn from subsequent hires to organize code, write tests and add extra processes. If they fail to hire such talent, they learn the hard way, frequent downtimes leading to user frustration and failing to retain them. It's crucial they realize this early on though, otherwise, this could lead to things blowing up pretty quickly leading to the death of the product.
Although I'd say, it sounds unusual for an investment bank. Doesn't sound like a company anyone would stick around for long. Sounds like inexperienced engineering leadership. If you're early in your career, I'd suggest looking for better companies.
Good unit tests mock out all external dependencies. But, that sometimes means there's little or nothing left to test. Some methods are essentially glorified database calls.
I've worked in places where "write tests" is so heavily cargo culted that those kinds of tests get written. The cost is whatever time it takes to write the test, plus the amount of time it takes to execute the test, every time it's run. You also have to maintain that "test" code, and all these things have costs, which could be avoided by simply thinking about whether to write the test in the first place.
Yes, quite common. I've been a software engineer in the Investment bank industry for about 15 years, worked at various hedge funds, brokerages and asset management firms in Boston, NY and recently in San Francisco.
Just 1 out of 6 firms I worked at had any kind of test driven development. In my experience, the lack of testing is not from a "hustle" mentality. It has more to do with the fact that the Fintech space in general, and investment banking in particular always lags behind in technology and processes. In companies that I tried to incorporate some sort of unit tests and TDD, the push back always was "We don't have time in the project timeline for writing tests. There is a QA team that will do that." Either the benefits of tests in software development process hasn't been "sold" to the IT Managers, or they are completely unaware of it, coming from a traditional waterfall methodology.
Another thing that's been the driven of this workflow is that traders and portfolio manager routine so some sort of un-scalable automation for their workflow. I once worked with a Bonds trader who had an enormous spreadsheet with over 80 tabs, links to bloomberg realtime pricing, a bunch of macros and such. The work that my team used to get was to take that spreadsheet and turn it into a database driven application that wouldn't "Freeze" or "choke" periodically as it used to in Excel.
In such situations, which is also very common and you'll likely run into it, your manager will ask you to work with said trader or portfolio manager and do the conversion. In these type of projects also I wasn't allocated any time to write tests. As soon as a portion of the application is ready, the traders or PMs would themselves run tests against a QA instance. In order to keep my sanity, I would write tests that wouldn't be in source control and wouldn't tell my manager about it.
You have to start with integration tests, then go down the stack if you have time and resources.
Source?
Please. This is just plain wrong. Integration tests also belong to TDD.
Unit tests are very focused, individual, component level tests. Suppose, I want to test my mortgage calculator logic, without bothering about caching, data storage, network IO etc. Then, unit tests are the way to go. But, if I want to test the functionality, just like an end user then Integration Tests are the way to go.
You might also use Integration tests to get your stories approved.
One might say Integration tests belong to BDD. I'll say that is just a lower level categorization. At the top, it's just TDD.
I don't know the OP's world, but I can vouch for starting with high level tests being a good strategy in some cases.
If you need to ship some functionality in particular, you need a way to check whether what you've written is that functionality.
If you're only releasing once, then maybe a manual/interactive session with the artifact is good enough. But if you're making and releasing frequent changes, then automating the process saves time.
What they are really saying is, "Currently if a bug makes it to production I can shift blame onto QA for missing it during testing, but if we start writing tests now I have to completely own it."
Let’s be clear, most of the software empires of the present day were built on this style. It is possible to ship software like this!
But if you do neither testing nor code review, you almost certainly have a team that thinks downtime is for losers and heroes are coding 24/7. I believe they get addicted to the twin dopamine hits of cranking out code and saving the company from disasters that their code causes.
Yes, and so are burnout, abusive interpersonal behavior and the physical / mental problems that inevitably go along with such working environments.
But there's no reason you need to put up with it any longer than absolutely necessary. Do yourself a favor and find another job as soon as you feel you've had enough.
I for one believe unit tests are not the be-all, end-all that most of the industry has swallowed hook, line & sinker.
Some open source projects I've seen don't have too many tests. IIRC the source for uBlock Origin doesn't have too many tests in it. But it's small enough that a motivated maintainer can keep the quality high.
For what it's worth, I worked at a Fintech company that built trading systems for the FX marketplace. They had automated tests out the wazoo and a super thorough and on-the-ball QA team. YMMV.
I'm curious, for the projects you worked on in the Fintech space, how much money could be lost due to software glitches? At this firm I worked for they were responsible for the gain/loss if a bug caused a trade to go into an error state. So it could easily cost ten to hundreds of thousands of dollars in a few minutes during market volatility. A friend of mine had a bug slip into prod and it cost the company $90k.
I'm fine with not writing tests so long as I'm not held responsible when something unexpected happens. Unfortunately for you, they are making you responsible for your own code and have no form of code review.
If you can, you ought to write tests anyway as a part of your work, even if it's just a handful of end-to-end tests to ensure you don't cause anything catastrophic. And if they ask you for an explanation, just tell them that having those tests in place helps you get more work done and not waste time.
There's a chance they're going to be irrational and opt to run their own software into the ground long-term. If they tell you not to write tests, I'd suggest continue working there until you've held your position for a year and, in the meantime, keep looking out for better opportunities. The reason I say that is because if you adopt their cowboy-coding culture long enough, it will sink in with you and it will be harder for you to adopt best practices at your next employer.
If you do want to stay there, there's a vast happy medium between "no tests at all" and "test driven development". Check out the "git effort" command for a list of the most-changed files in the codebase. (If it's not installed, try looking for a "git-extras" package for your operating system.)
The files changed most frequently are often also the ones most in need of refactoring and tests. If nothing else, you can cobble together a little test suite of your own and be the guy that finds bugs faster and introduces fewer new bugs.
However we've really little unit tests.
There is balance between what tests are useful and which are not. Testing for sake of testing can be counterproductive. However in your case, it seems there is no reason to skip some form of testing or code reviews. Some companies have just bad practices.
Are we actually doing things too complicated and too good for the business case?
But the OP's description of his job doesn't seem to fall into any of those cases. Quite the opposite, I'd guess.
https://riskfirst.org/Evaluating-Risk#discounting-the-future...
Although it’s sometimes reasonable to not write tests, this kind of thing usually happens when everyone is being threatened with the sack on a daily basis, rather than lacking engineering skills
When you get a better job then think about tests. Your health is first.
That was a solid product!
https://en.m.wikipedia.org/wiki/Cowboy_coding
Granted, there were theoretically some bits that could have been tested using automation, but those weren't the bits that ever caused any trouble.
If you’re building a lot of small, one-off tools for internal use, it may well be the case that some limited manual QA or UAT is sufficient to ensure that your work is good enough.
If you’re working on larger, more complex projects that are frequently updated, the shorter feedback loop that some amount of automated tests provide will probably save time and money by catching problems earlier, avoiding regressions, and reducing the need for repetitive, time-intensive manual testing.
But in any case, your testing needs will always be highly specific to the actual nature and needs of the project.
I view it differently.
We get paid to solve business problems, most often -- but not always -- with software.
Software that's broken in preventable ways doesn't solve business problems, it just creates new ones.
Outcomes, not output.
I think you may be missing OPs point. Software "that's broken in preventable ways" very frequently does solve business problems. And solves them well. That's why the industry works the way it does w.r.t. testing.
6 months to deliver 2 applications that each work 80% often provides more business value than 6 months to provide one application that works 100%. More functionality almost always is more valuable to a business - including warts and all.
I agree with this:
> 6 months to deliver 2 applications that each work 80% often provides more business value than 6 months to provide one application that works 100%.
Though this is a question of product management prioritisation, not engineering practice. "It's faster to not write tests" depends entirely on a definition of completion that ignores the cost of repairing defects and coping with faults and failures in production.
If a piece of software literally will not launch period, then it is broken. If it can do pretty much anything more than that, then it "works to some degree", and just has some number of scenarios that are broken. And my point is that if it has 80% of scenarios that work and 20% of scenarios that are completely functionally broken (ie do not work at all for the customer) that still provides business value. That's the point I'm making. It's not ideal, but in the real world half broken software is immensely valuable.
> Software that's broken in preventable ways doesn't solve business problems, it just creates new ones.
So software that is broken in preventable ways, absolutely does solve business problems. And just about the entire software industry is proof of this.
If your software is designed to support, augment, or automate some sort of internal need for non-IT teams, there's a significant chance that there's already an inefficient, error-prone process in place as-is. Because even without your software, they still have to get their job done. And they're used to having a "good enough" mindset because they're used to making due with whatever tools/capabilities they have, regardless of fit.
As long as your software addresses their needs[1] and isn't any more error prone than whatever they're currently doing, they'll be happy. That said, whenever possible you do want to code your failure conditions or edge cases to fail hard and visibly or expose visible checks for the user. If it covers 80% of their needs, but they have to fall back to their old process for the other 20% of their needs, business users are generally happy with that. But if it pretends to work 100% of the time, but subtly and silently causes problems 20% of the time (and they don't know what 20%), then they'll be distrustful of it and resist (or resent) adopting it.
For externally-exposed software, I'd be a bit more wary of not having any tests at all. But for internal tools, I've seen far more systems without tests than with them.
I've heard this double-standard before, and it sounds plausible at first glance, but is quite demonstrably nonsense.
1) Automated tests are software
2) Engineers aren't paid to produce software (nobody would pay me to auto-generate a billion "hello world" programs per second). Engineers are paid to solve problems and/or create value. Tests solve problems (finding bugs, demonstrate usage, clarify thinking, document intent, etc.).
3) Software is a liability, not an asset. We should produce as little as needed, and delete it if possible.
4) Engineers aren't paid to perform manual testing (running commands over and over, clicking around web forms to see what happens, etc.)
a) users do crazy things. b) eating your own dogfood is worth it.
That doesn't mean engineers are paid to go through a list of regressions and edge-cases, look for broken links, double-check calculations with pen-and-paper, etc. All of that can be done across an entire site automatically in seconds; there's no excuse not to.
> a) users do crazy things
Again, thinking we can check such things better than a computer is misguided. Test frameworks like QuickCheck can stress-test a system with far more strange inputs/transitions than I could hope to think of.
Which is assumed to be free of defects because people are suckers for circular reasoning.
We can increase our confidence by trying a few examples, but only a little, since the scope for failure is so huge.
Hammering a system with thousands of edge-cases, interleavings, previous regressions, etc. can give us a bit more confidence; but takes a huge amount of time for a person to do manually. It's pretty easy to automate though, and there are a load of libraries and frameworks that make it even easier.
If we want even more confidence in a system, we can verify it against a machine-checkable specification. That's still too hard to automate in most cases.
In any particular situation we need to find an appropriate balance between the cost of increasing our confidence in the system (i.e. how much effort we're willing to spend on prove ourselves wrong) versus the cost of failure.
Automated tests (especially property checks) are so low-cost and high-yield that I find them always worth doing, unless the consequence of failure is near-zero.
1. "Automated tests are software" 2. "all software tends to have bugs"
You're presumably using a whole heap of software to post your comments, all of which will tend to have bugs; yet you still find it useful.
Boolean logic rarely captures the nuances of real-world situations, and the "rounding errors" accumulate at each step of such arguments, which can lead to some silly (but logically consistent) outcomes. I think you've fallen victim to this here.
In a similar way, it's a false dichotomy to think of software as being either buggy xor perfect; of testing (or any other QA) as proving brokenness xor correctness; or to consider bugginess as implying uselessness.
Notice that I framed things in terms of confidence (in a Bayesian sense) and costs; not absolutes.
Seems most of my days are made of this lately.
Tests are a great way to take away tedious, boring, uninteresting essential tasks that can and will wear down a team.
In my experience (20+ years of writing software professionally) I've never built an automated test. I've never used an automated test.
My preference is to write code and push it straight to production without even debugging it. tested afterwards, certainly, but if you're in such severe doubt that you're going to ruin the system, you need some serious code review.
My strong belief is that you cannot test your code into goodness, that is something that only a person can evaluate. By making myself better at coding, by not relying upon something else to make my code good for me, I am way more efficient than I could be otherwise. Given, this may limit the types of projects that I engage in, and certainly changes the project flow. However, when I look at my productivity as compared to others, I don't find any problems in my method.
Only human eyeballs can find some problems, despite how many cases you throw at something with automated testing.
But it's pretty well understood that tests are not a substitute for code reviews and vice versa. If that's your impression of TDD advocacy, I'm glad I could update it.
That has to be quite an outlier. How do you find teams and managers that accept working like that? What's the failure rate of bugs found in production, and how much value at risk are we talking here?
I've written plenty of systems where automated testing was infeasible or useless .. but we always did manual testing before shipping.
(At the other end I've done IC design where if a mistake isn't found it's another £25k at least plus staff time to do a re-spin, so obviously we had automated tests with near 100% coverage)