It's the Swiss cheese paradox: "cheese has holes; bigger cheese have more holes; therefore, the more cheese, less cheese".
> However, I still haven’t found any studies which show what this relationship is like. Does the number of bugs grow linearly with code size? Sub-linearly? Super-linearly? My gut feeling still says “sub-linear”.
We know from observing reality that even buggy software is better than no software – a software so buggy it adds negative value is a rare exception. So I guess it has to be "sub-linear", otherwise software wouldn't have economies of scale.
The contradiction though is: how do we explain the apparent stability of large but mature codebases? Thinking something like Emacs for example here, where some commits date back to 197X. I guess at some point it becomes an inverted U shape even if codebase grows, if it grows at a conservative pace.
It's not so much "sub-linear" as "capable of being bypassed".
The CEAC State Department website is the most common one for me... the file upload only works in incognito and "once" per session. So people have figured out that you can log out, log back in and keep uploading your immigration paperwork.
Number of bugs can grow super-linearly while their impact might be decreasing preserving the overall usability increases. Basically as your software grows there's more corner cases that will get tested very rarely if ever.
Number of possible states grows almost exponentially with code size, so it would be very weird if the number of bugs grew sub-linearly.
> The contradiction though is: how do we explain the apparent stability of large but mature codebases?
The problem with these sort of analyses, in my mind, is that they seem to neglect the iceberg of code you build on top of, which works perfectly fine and is virtually bug-free. For instance, imagine that I'm building a website. If size is a predictor of quality, then you'd expect me to add bugs as time goes on, and that's probably true. But my site is stacked on top of Chrome, an abstraction with hundreds (?) of millions of lines of code that virtually never give me an issue. This is stacked on top of the typescript compiler, the C++ compiler, the operating system, etc, and those all run in the millions (TS) to tens or hundreds of millions (OS) of lines of code! So while I don't doubt that codebases get buggier over time, this seems to neglect the fact that I use truly immense amounts of code every day that don't seem to run into an issue. Perhaps the bugs grow around the edges rather than in the center, or something like that?
Chromium has 63,438 open bugs in its bug tracker so the theory remains true, but maybe the broader point is that strong abstractions have enabled us to reach great levels of complexity. Testing, staffing, and programming best practices probably go a long way in making software like Chrome appear to work flawlessly from the outside.
An analogy would be the craftsman-factory. A craftsman knows and performs every step of the process, and the end product depends on the skill of the craftsman. While a factory each worker is limited to a specific job and the combined effort of factory workers allows products well beyond the reach of any craftsman.
> Chromium has 63,438 open bugs in its bug tracker so the theory remains true
How many of those bugs affect you on a day-to-day basis? For me, the answer is 0. In fact, I can only think of one time ever when I was affected by a browser bug as a developer. I think that most of the reason Chrome has so many bugs is probably just because a staggering amount of people use Chrome, meaning they can inspect every nook and cranny.
For a somewhat contrived example, imagine I write a program to add two numbers together. I write a single line of code: `var foo = bar + baz;` in JavaScript. If I ship it to no one, I can claim it has no bugs. But if I ship it to a million people, then I might start getting bug reports from people running it under extremely precise circumstances and running into issues. For instance, maybe I'd get a bug where two numbers sum to be negative because the user ran into integer overflow when bar and baz are very close to INT_MAX. I might also get reports of lack of precision when bar is extremely large and baz is not (e.g. sometimes foo + 1 === foo in JS if a is huge). I suspect most bugs in Chrome are of this variety, or, more likely, they are even more obscure.
It seems to me that it's not a strict 1:1 correlation between size and quality - there's also a factor where more popular software has a lot more people looking for bugs.
The majority of the bugs I see are failed tests, null dereferences, unreachable code reached, very slow code, UI bugs, crashes, etc.
Come to think of it, I've had 2 chrome bugs on my phone in the last week or so. Some weird UI bug with tabs, and a more severe one where it just kept crashing once in like 10 seconds. It magically stopped after a while.
Things like chrome, or Linux, are so complex, and interact with so many different variables that it's completely understandable that it can have tens of thousands of bugs, and yet most people won't ever notice one.
> how do we explain the apparent stability of large but mature codebases
I would hypothesize three potential factors are at play, especially for software like Emacs.
The first is that some bugs becomes, themselves, "features" over time. It might be wrong from the standpoint of what was intended or desired when the code was written, or what is documented, but because that is how it works, that's how it is now expected to work, and so it's no longer a bug.
Another factor is that some bugs are "shielded" by the surrounding code, effectively two pieces of code offsetting each other's bugginess. Unless and until you either replace one or the other, or need to interact with one or the other in isolation, the bug isn't apparent. Kind of like how going to the chiropractor for one problem can "unlock" other, latent, problems.
The last one, somewhat related to the first two, is that some well factored software systems do a very good job of isolating their most stable parts, leaving their less stable, more commonly changed or iterated parts in separate modules or components. Because the change rate and focus of effort is on the "outer" parts of the system, less time and energy is spent on the "inner" part. Very often, developers of the "outer" parts will accommodate the bugs of the "inner" part, because changing the inner part requires both greater knowledge of the overall system, as well as the potential for a larger "blast radius" (and therefore assumed responsibility) for ones change. So rather than debugging the "inner" part, the "outer" part grows lots of accommodations for the inner part. It's not until that inner part can no longer be accommodated, or it becomes a security vulnerability, that the community developing the system will see the value in "fixing" the "core".
> We know from observing reality that even buggy software is better than no software – a software so buggy it adds negative value is a rare exception.
Whom observing which reality?
I have made no scientific study - I am not stating any sort of strong opinion, just an uneasy feeling - that about 90% of the software and uses of computers subtract from humane well being and welfare, not contribute to it.
Looking at you Javascript frameworks.
Looking at you databases and tracking.
Looking at you spyware and advertising bots.
Looking at you killer robots (yes they exist - mostly thinking of the horrific USA military drone assassination programme)
So many examples to choose from. I should not complain, I make my living from computers and the uses I facilitate are no shining examples of improving people's lives
i think you are mixing two concepts s imperfect software like
databases and JavaScript frameworks, the others aren't imperfect they are evil, they produce what they want to produce, yes is negative for society, but not incorrect.
Agree that a lot of it has to do with rate of growth. Compare emacs or postgres codebases with some of the horror shows that are the codebases at VC-hyper-growth startups.
Taken literally - "why on Earth", as in, in nature - it becomes a matter of timescale. Something we seem to be only fully realizing now - ecosystems and climate are in constant flux; they may look stable on a timescale of a human lifetime, or the lifetime of a society, but at the scale of several centuries or more, they're chaotic.
> Mature systems tend to have lower Evolution. Thus, there are more bugs being fixed than created.
Ideally. But in practice, they must add features to sell more copies. And even if they have no evolution, OS's evolve that make the software worse over time (e.g. "modern" UIs, hardware vs software rendering, etc.).
We see this in many products that were once best-of-class, e.g. Paint Shop Pro, ACID Pro, etc.
> otherwise software wouldn't have economies of scale
Does it? I tend to think it doesn't (in terms of code size; obviously it does if we're scaling by number of users)
Even if bugs scale sublinearly in practice- my gut tells me it's because development as a whole slows down even more than the bugs accelerate, partially making up for it
I.e. suppose your codebase size grows 5x, and the bugs per change grows 10x, but then the org's rate of change also slows down by 5x. Now your bugs per time has only grown by 2x.
eMacs has the Church of eMacs backing it, so it might not be a good example of a normal codebase. I personally feel a compulsion to dedicate time to giving back to the eMacs ecosystem in a way that I can’t extend to any other codebase, and there are others that I likely depend on to a much greater extent.
Jokes aside and that being said, I agree that there is something helpful about slow, organic (in your words, conservative) growth. It’s neither necessary nor sufficient for stabilization, but it certainly seems to correlate. GNU in general seems to be a good example.
If you fix commonly occurring bugs, you're left with rare bugs - and there may be tons of them in mature codebase. For legacy systems with dedicated operators, it may make more financial sense to teach users a workaround to a bug, than to fix it.
> a software so buggy it adds negative value is a rare exception.
Survivorship bias will explain all of that. You'll rarely encounter negative software because it gets discarded. It doesn't mean it's never created, it just dies before getting wide visibility.
> otherwise software wouldn't have economies of scale.
The fact that software can be developed once and sold indefinitely means the marginal cost is near zero. That is an unbelievably huge economy of scale that would dwarf many other superlinear problems.
> how do we explain the apparent stability of large but mature codebases?
Not all software changes are additions. If assume that codebase size is positively correlated with number of bugs (i.e. more code = more bugs) and that most changes that do not grow the codebase are likely to fix at least one bug, then mature codebases should be expected to improve in quality to the degree that number of changes outpaces total codebase size.
I had to kill one of those projects. It was so buggy it was causing issues in "unrelated" areas. I find out Thanksgiving week if I killed it early enough for the next release.
> The fact that software can be developed once and sold indefinitely means the marginal cost is near zero. That is an unbelievably huge economy of scale that would dwarf many other superlinear problems.
I'd propose a slight revision: "software can be developed once and used indefinitely." I would bet that most code nowdays is written inside organizations and never directly "sold" over and over again to a customer. But even that tends to result in those organizations wanting more and more, because even a buggy one tends to save people time compared to doing the process manually, and then there's another process they want to automate next, etc...
I'd propose replacing "indefinitely" with "until the systems it interacts with require changing it". It takes surprisingly short time for a mobile app that is written to current standards to turn into something that doesn't run on latest updated OS version. It's a good point that selling software indefinitely isn't possible. There are only a limited amount of people who want the software and are willing to pay for it.
It's still effectively "sold" in economic terms. You have one codebase and it generates revenue per customer (where "customer" may either be an end user, or an advertiser for ad-supported software).
Software size is positively correlated with bugs, but software age is negatively correlated? If your codebase doesn’t grow much in size but grows older, the quality should become better.
> We know from observing reality that even buggy software is better than no software – a software so buggy it adds negative value is a rare exception. So I guess it has to be "sub-linear", otherwise software wouldn't have economies of scale.
Your logic is flawed here. First, software has economies of scale because it's easy to produce the second copy of a program once you have the first one. It has nothing to do with lines of code.
Second, buggy software being better than no software would still be true even if bugs were super-linear in the number of lines of code.
I'm pretty sure that bugs aren't sub-linear. As you pile on more and more lines of code, the number of bugs per thousand lines goes down? The code gets better as you create more and more of it? Experientially, no.
I think it's super-linear, for a couple of reasons. First, the bigger the program, the bigger the team. The bigger the team, the more below-average coders on it. Second, the bigger the program, the more room for weird interactions between obscure conditions.
No, it's definitely not sub-linear. Bugs per line of code do not decrease as you have more lines. I suspect it's
> A bigger team doesn't create buggy software because it contains below average coders.
But it must, by the very meaning of "below average coders" - it's implied that we're rating hypothetical coders on software development skills, which include the ability to make correct decisions (both at code and architecture level).
> It creates buggy software because the team members have to coordinate with more people.
Those are not independent variables. The bigger the team, the bigger coordination problems, the more bugs. But also, ceteris paribus, the worse the coder, the more problems there are coordinating with them, thus the more bugs. And also, ceteris paribus, the worse coders are on average in the team relative to global average, the more of them are needed for a project, which makes the team bigger and coordination more difficult. Etc.
> We know from observing reality that even buggy software is better than no software – a software so buggy it adds negative value is a rare exception. So I guess it has to be "sub-linear", otherwise software wouldn't have economies of scale.
I'm not sure about this, our e-health system had millions spent on it and eventually still was discarded, after "sort of" working for months and slowing down all of the doctors and pharmacy workers who had to deal with it, as opposed to just using paper slips for prescription medicine, or even individual hospital data management systems: https://www-lsm-lv.translate.goog/raksts/zinas/zinu-analize/...
In addition to instability, even in my country, Latvia, there have also been large enterprise systems that have had serious performance issues, like the government e-system platform latvija.lv went down when the COVID vaccination signups became available, which was problematic, given how many other systems depend on it as a login provider. We've also occasionally have had the national tax system go down in the dates when people can start submitting their tax reports.
Also, there have been cases of problematic deployments bringing down entire orgs, Roblox had their core business be halted for a while a year ago because of problems with their Nomad cluster, which resulted in a 3 day long outage: https://roblox.fandom.com/wiki/2021_Roblox_outage
I've personally seen systems that have gotten so complex that they're a liability - if there is no full test coverage (as close to 100% as you can get) then you're usually under the risk of wrong calculations or behavior in the system, which may or may not be possible to roll back, depending on the nature of the system. Sometimes it might not even be easy to detect, but the implications can be pretty great, especially if you're dealing with finances. Or having people depend on the systems and not having a business continuity plan were they to go down, due to how integrated things are in our current society.
There is plenty of software out there and everything around that software (other code that determines how it will run, how it will scale, how it will be secured) that has plenty of potential for creating lots of negative value, all the way up to making organizations evaporate, or causing lots of suffering for numerous people. One doesn't even have to consider something like t...
If the codebase/project became "feature complete". Once a project becomes feature complete, then finally bugs are able to be squashed without reintroducing new ones. The code becomes better because the deaign begins to accurately model the program since there is no feature creep.
The problem with large programs is when you _continue_ to add on to them.
Non-open source very rarely gets to the feature complete stage.
Meta observation: At least on Mobile Safari, this page breaks the back button. I had to spam press the back button like 10x in order to come back to HN. I absolutely despise websites that do this, and immediately feel a bias against whatever the site is selling or trying to convince me to believe.
RE: the article itself
I’ve been on projects where we eventually shipped the most poorly written code (variables with ambiguous naming, methods that are several hundreds of lines, classes with 5-10k lines, little or no test coverage). This code was baked into a consumer device, and we shipped over 16 million units over almost the span of a decade.
I’ve also been on projects where the code was practically a work of art. By the time we were “done”, the market had changed from under our feet. What we were doing wasn’t so interesting or novel anymore. We never shipped.
It doesn’t surprise me that more code == more bugs (or an increasing probability), but it’s good to keep in mind that value is what we care to produce. Everything else is a cost, and spending too much on any particular cost can take the whole enterprise down the ravine, regardless of good intentions.
I think use/time of the code needs to be factored in as the more the code is used, the more bugs are exposed and fixed, such that old actively managed code can be far less buggy (if all goes well and the architecture is good). Less code that does the same job is always better.
There are also two dimensions to code size, one being source code size and another is executable size/number of instructions executed. This path of thinking brings up the connection between optimization and correctness.
In particular if adding source lines describe the program more specifically and accurately then that should enable compiler to produce more compact/efficient code and I'd imagine also contribute to correctness.
haldar's summary is completely incorrect, as you would expect from someone who says 'tl;dr'
the paper finds that faults (and wmc (methods per class), cbo (coupling between object classes), response for a class (rfc), number of methods added (nma), etc.), correlate to class size (non-comment source lines of code), not program size
it only looked at one program so it could not measure any effects due to program size
moreover it wasn't measuring code quality in the sense of 'defects per kloc', it was measuring defects (whether a bug had been detected in a class in the field or not)
stripping away the acronyms, what they found was that classes that contained more code more often had at least one bug, and also had more methods, but that having more methods without having more code didn't make classes significantly more likely to have a bug
and similarly for the other complexity metrics like how many different methods a class calls and implements (rfc)
this is unsurprising, since things like the number of lines of code in a class and the number of different methods in a class are just alternative metrics of class size, and everyone knows that in general more code means more bugs
that's why we measure code quality in defects per kloc and not total defects. the paper didn't even try to measure code quality in that sense
that doesn't mean the paper is bad. if the paper's authors are correct that many other papers have failed to control for size in their defect metrics, they have identified a serious shortcoming in the existing research literature; haldar merely totally failed to understand them
so the paper haldar tried to summarize doesn't measure either program size or code quality
(some of their references did look at program size tho)
why are none of the other comments pointing this out
are they all just commenting without having read the paper
'tl;dr' is an abbreviation for 'too long; didn't read'
people who don't read long things sometimes know a lot about things like how to weld, how to comfort the grieving, and what makes them happy, but they are always profoundly ignorant when it comes to book learning
that's what happened in this case, where haldar's summary claims that this paper shows that a variable they didn't attempt to measure is the best predictor of another variable they didn't attempt to measure
evidently he just looked at the diagrams and guessed what the words meant
>people who don't read long things sometimes [...] but they are always profoundly ignorant when it comes to book learning
You seem to have fundamentally misunderstood how this acronym is used.
When provided by an author, it doesn't mean that the author didn't read...
It's literally just a substitute for "Summary:". An indication to the reader that if they find the article too long to read, they can read the immediately following subsection to get a summary of the article.
Do you also think that people who say "Summary:" are profoundly ignorant? Or are you just against acronyms?
probably you haven't noticed this, but using 'tl;dr' instead of 'summary' or 'abstract' is a group identity marker by which the author shows solidarity with their (presumably profoundly ignorant) readers. it's the same sort of gesture as describing the paper's authors as 'eggheads' or 'boffins'
and evidently in this case the blog post author really didn't read the paper they were 'summarizing'
i find your contributions so far to the discussion of this paper profoundly disappointing
>i find your contributions so far to the discussion of this paper profoundly disappointing
I'm not sure why you are so hostile, and why you think everyone except yourself is profoundly ignorant, but it's clear that you have some sort of world view where you're superior than anyone else you interact with. I doubt anything I say would convince you otherwise.
tl;dr: I found this conversation disappointing, too.
i frequently interact with people i admire and who know more than i do (and sometimes they are the same people), and, as i said above, even people who are profoundly ignorant about book learning are often wise about many other things
rather than being hostile, i have taken the time to explain in more detail the things you asked about in my previous comments because they were unclear to you
i am surprised that you have stooped to launching personal attacks on me in this comment, since so far i have criticized only your low-quality comments (and the original blog post) rather than any presumed personal attributes of yours
>i am surprised that you have stooped to launching personal attacks on me in this comment, since so far i have criticized only your low-quality comments (and the original blog post) rather than any presumed personal attributes of yours
You've called me "profoundly ignorant" several times in this chain. How you can say this with a straight face is baffling.
i haven't called you profoundly ignorant even once. please do not lie about what i have said; there is no point in doing so in any case because anyone reading this thread can look half a page up or click the 'context' link and see that you are lying
i said that people who don't read long things are profoundly ignorant, at least within the sphere of book learning, though often not in other ways. that's not a personal attack on them, it's just an obviously true statement if the phrase 'book learning' refers to anything at all
you haven't said even once that you don't read long things, so there was no reason for me to even suspect that what i said applies to you, even if it were a personal attack (and, well, some people are sensitive about being ignorant because they're used to being criticized for it)
From now on I will strive to only write "abstract" instead of "TL;DR" so I can avoid showing solidarity with the presumed profoundly ignorant. It's a matter of principle that I must appear to be the most intelligent person in any room I enter, virtual or physical.
maybe you could show solidarity with them in a way that doesn't reinforce a dichotomy between them and people who read books
for example, by emphasizing the praiseworthy attributes you have in common rather than the misfortunes that have befallen you and the self-destructive choices which, in cases like this one, perpetuate those misfortunes
i mean it might be useful to read a book from time to time, or at least a 25-page article, and that's less likely if you turn it into an identity threat
we're all born profoundly ignorant but we don't have to stay that way
if you're the most intelligent person in the room maybe you're in the wrong room. this can easily be reversed to provide a plan of action if being the most intelligent person in the room is what you value most. beware, though, some people are smarter than they look
Some of us know how to weld, how to comfort the grieving, and have good book learning. Still trying to find the balance between happiness, satisfaction, and pleasure.
agreed, i'm not trying to say it's a zero-sum sort of thing, just that using 'tl;dr' is not an indication of profound ignorance in all possible ways, just the forms of ignorance that result from not reading long things
it turns out they thought they were being criticized so they got defensive, explaining their otherwise shockingly aggressive and even mendacious comment thread here: https://news.ycombinator.com/item?id=33567361
not sure that's a good excuse for their total failure to contribute anything substantive though
Ah but I was mainly wondering why that page would touch the history at all. If it was doing something complex I could understand there being some compatibility issues, but I don't see any reason this page has any business meddling with history.
> Does the number of bugs grow linearly with code size? Sub-linearly? Super-linearly? My gut feeling still says “sub-linear”.
My loose reasoning says it has to be linear (like interest generated on bank accounts). Otherwise you could simply merge two large codebases to squash bugs (in the case of sublinear bug growth) or divide a large codebase (in the case of superlinear bug growth).
ALternatively, in a sublinear-bug-growth codebase, you could just keep adding and adding to it until eventually you'd add no more bugs.
You can't just divide a code base that's entangled. If you disentangle it, you'll remove some bugs in the process. My feeling is that it's super-linear for this reason.
Completely agree. And my favourite add-on is if it’s broken apart such that one half of it ends up being a library, adding a second consumer to that library will tend to further reduce bugs, even more so if the second consumer is written by a different team. (Eventually, at least, because the second consumer is going to find bugs so the start)
I don't quite follow this logic. If one manipulates a codebase in the ways you describe (merging with another, or dividing a larger one), the result doesn't even have to be a working program; how can we compare the result with the original?
The real answer is probably that it might depends on how tightly coupled the code is, how much test coverage there is (assuming that tests are included in code size), the quality of the authors who wrote the code (this might be circular), maybe even on the technology used, or what you consider distinct bugs, etc.
My intuition leans towards super-linear, but only slightly so -- large codebases tend to have many authors (many of whom may have since stopped working on the project) and be old.
I think it would make sense to imagine the codebase as a graph of some sorts, and the number of bugs as a function of both the number of vertices and edges.
My gut feeling is it grows linearly with any actions over codebase that are not accompanied by both testing and real usage. Well, until tech debt comes into play.
But there are many ways to make it non-linear, e.g. one can build a nice high-level structure which becomes irrelevant when goals shift. Or one can sign up for a paradigm which requires better developers and deeper understanding.
Bugs grow linearly when writing them, not after they are written.
> divide a large codebase (in the case of superlinear bug growth).
I do almost exactly that when approaching a problematic system: refactor aspects of it as pure components or separate services in order to isolate behaviours from each other.
Even just what you describe - breaking a large codebase into two smaller (working) ones - exposes an interface that can reveal failed intentions. It's just usually an incredible bitch to do, unless you initially refactor the parts to minimize internal state and the types of interactions the two parts have (ie - make them more 'pure').
Bugs definitely grow super-linearly, at least exponentially. They scale with the number of interactions between lines, which grows exponentially. (That, in turn, scales with the number of unforeseen-behavior-we-don't-like, which is generally what is meant by a "bug".)
To be sure, well-written code decouples the different components so that distant pairs of lines interact as little as possible, and thus it's more difficult for unforeseen constellations of factors to arise. Well-written code also minimizes the possible values that are used and have to behave correctly. But all else equal, it will be exponential.
Your intuition is correct, that you can significantly reduce bugs by splitting an N-size code-base into two N/2-size code bases. But there's only so much you can split it down while achieving the desired functionality. Plus, there's overhead to constantly switching between codebases.
I agree with the premise of this article. You want a program to have as few lines of code as are required to satisfy the needs of stakeholders (especially customers and other operators of your software).
The size of the solution must not exceed the size of the problem it solves. You need to focus on the essence of the problem and make compromises based on what is most important. Keeping the lines of code count low is a worthy target.
There is a balance to be reached but devs today are far too permissive when it comes to complexity creep. It should be setting off alarm bells much sooner.
Code size is also a good (but not great) predictor of development velocity. If you need to write fewer lines of code to do something (assuming standardized formatting so no code golf), it's probably the case that you can write that code faster, and ship features faster. And of course, some languages simply enable to work on such a high level of abstraction that you write much less code (e.g. http://www.paulgraham.com/avg.html).
>I've been writing software almost non-stop for close to 25 years, and I still don't understand most of the code written by others.
I think you nailed it. In my experience, the increase in complexity isn't necessarily inherent to the amount of code so much as the amount of abstraction used as code size increases.
I've worked on some large codebases that only used simple, well-understood and/or well-documented abstractions which didn't feel as nearly complex as other codebases with abstractions that were more complicated than they were worth.
Yes. All modern systems run a huge amount of code if we count the underlying microcode, operating system, runtime libraries, 3rd party dependencies and the application itself.
The thing is not the total size but the boundaries and the understanding of each layer and how manageable each layer is for the humans modifying and mantaining it ... which is, of course, the total cognitive complexity.
Citation needed. As far as I know the best available evidence (which is still pretty weak) says number of bugs is correlated only with number of lines of code, regardless of whether they're simple or complex lines.
It's not so easy. In my experience, most bugs that aren't extremely trivial are not obvious locally. A function, whether simple or complex, will look totally fine, but it's in fact buggy, because it violates the implied expectation of the caller, or makes the caller violate implied expectations of their caller, etc. Decoding a complex line of code isn't the hard part here - narrowing down the mismatched expectations and identifying where to apply the fix is.
Splitting code into functions isolates blocks of their context of what situations they are called in and what parameters are passed in. In my experience, Transcribing code into notes, where struct fields are inlined as indented blocks into their surrounding types, and function calls' interiors are inlined as indented blocks into their call sites, restores this context, places lines on-screen in the order they execute, and enables casual reasoning. (You still won't know data invariants without asking the original programmers, but you can guess and check at this point.) http://number-none.com/blow/john_carmack_on_inlined_code.htm... advocates for a style where code is inlined in the actual source code whenever possible.
I'm heavily in favor of inlining over tiny little functions, and the transcribing you describe is something that should, IMO, be done automatically by your IDE, on the fly, as a different rendering mode. It's unfortunate that there's no such feature for any of the languages I know.
I don't think that's a valid way of framing things, beacuse it's not like the amount of review time available is proportional to the number of lines of code.
I think this is a bit of a small-scale conway's law-style effect. You shield your ability to reason about the code from the complexity of the code of others by abstracting around it, thus increasing the total complexity with an ad-hoc abstraction layer.
Essentially for every layer of indirection, a reset of complexity happens at the expense of unknown unknown's creeping into the simulacrum the layer is modeled as. At least, that's how I see it. :)
I admit that it took me having to look at "clever" short code that I wrote years ago to really take this lesson home. Now I get that code has to not only be correct and concise, but also it needs to document to the reader it's function. That often means being more verbose than it could be, and being broken into obvious discrete steps.
Certainly the best code is simple and easy to read, and yet still concise. But even "clever" code can be better than truly bad code.
I'd wager that those of us that frequent HN probably see less of the sort of truly bad code though since it is usually the code written at lower paying jobs and fields, but which possibly makes up more LOC than all the "ok" code.
On the flip side, there's code that will never be easy to read. Many algorithms and data structures simply don't break down to anything that is easy to comprehend.
Like even something as freshman as mergesort falls in this category where you're sort of squinting and wincing trying to figure out how p, q and w relates to i, j, i0 and j0.
Sure you can say let the PhD students write that code and put it in a library and we'll just worry about `sort(array, start, end)`, but that's just sweeping this reality under the rug.
Truth is if you're not willing to venture into this territory of non-trivial algorithms once in a while, you're seriously hobbling what you're able to do.
The trick is knowing when it's the right thing to do. It's the closest thing we've got to dark magic. Dangerous and unpredictable, certainly, but also immensely powerful.
Entire industries are built around someone one day sitting down and developing a piece of non-trivial algorithm. The big data boom of a few years ago, block chains, machine learning, ...
Myself, I lean towards compact code, but the team pressure is towards verbose code, and - hopefully - the compromise ends up being at the right level[0].
The tricky thing is, "clean code"[1] ends up hitting against the limit of human working memory. Past some point of verboseness, that "clean but long code" gets very hard to work with, because the meaning is spread too thin, it doesn't fit in short-term memory anymore. Conversely, "super compact code" may be hard to decipher, but you know the whole thing is contained in the handful of lines you're looking at. As a crude analogy, it fits in L1 cache, whereas dealing with "clean code" means chasing pointers.
--
[0] - FWIW, I rarely have people complaining during code review, and more often I receive compliments for readable (and documented!) code, so I feel tentatively confident I'm hitting the right balance. I think the trick is to not be defensive, but instead to pay attention when someone signals the code is too difficult to understand, and adjust accordingly.
[1] - As a somewhat generic term; the advice from the book "Clean Code"... isn't very good.
The trick is to make clean layers with distinct responsibilities. And magically you have no to keep all details in a memory. Unfortunately there is no good formal method to make clean layers. Currently we practice it via bruteforce through experience.
Layering alone isn't enough, for (at least) two reasons:
- Usually the work crosses several layers in at least one place - e.g. implementing or modifying a feature will require you to touch implementation and interface layers. If those layers are spread too thin, they won't fit in the head.
- Too much "clean code" will make a single layer no longer fit in working memory even at an "overview" level. This is the most common issue in my experience; a problem created by adherents to the idea of "lots of short, simple functions". Even most trivial feature makes you jump between half a dozen functions, and IDEs don't help there the way they should (i.e. by allowing you to view - and perhaps edit - code with select function calls inlined in place).
> Usually the work crosses several layers in at least one place
You are doing it wrong then. It's basically a violation of SRP. Your changes should touch only one layer. If it's not, you have fragile abstractions.
I agree there are a number of cases when you introduce a new thing you should thread implementation through all layers. But not for maintenance and bugs (common case).
> Too much "clean code" will make a single layer no longer fit in working memory even at an "overview" level.
But why you need to keep it in memory? Do you keep all OS APIs? Or standard library? They are other layers you use and have no issues with broad interface. Your application layers is the same thing.
I've found this to be absolutely true throughout my career.
The only adjustment I'd make is that size should be measured after some normalization pass (eliminate comments, whitespace, make variable names the same length, etc)
The older/experienced in the industry I get the more I don't believe software engineering research.
If it's in a uni- expect for the researchers to be biased towards the practices that are advocated in their academic text books on the subject.
If it's an organization: Expect for it to be biased towards what they want for that language (ms for supporting c# for new features or moving people away from old features)
---
I saw this in the anti-TDD/unit testing/Testing papers. They're making claims based on a flatten view of development and aren't capturing what happens on the local machine where those are intended. Additoinally they don't consider the externalities about how you approach the code base.
197 comments
[ 5.4 ms ] story [ 241 ms ] thread> However, I still haven’t found any studies which show what this relationship is like. Does the number of bugs grow linearly with code size? Sub-linearly? Super-linearly? My gut feeling still says “sub-linear”.
We know from observing reality that even buggy software is better than no software – a software so buggy it adds negative value is a rare exception. So I guess it has to be "sub-linear", otherwise software wouldn't have economies of scale.
The contradiction though is: how do we explain the apparent stability of large but mature codebases? Thinking something like Emacs for example here, where some commits date back to 197X. I guess at some point it becomes an inverted U shape even if codebase grows, if it grows at a conservative pace.
The CEAC State Department website is the most common one for me... the file upload only works in incognito and "once" per session. So people have figured out that you can log out, log back in and keep uploading your immigration paperwork.
Number of possible states grows almost exponentially with code size, so it would be very weird if the number of bugs grew sub-linearly.
The problem with these sort of analyses, in my mind, is that they seem to neglect the iceberg of code you build on top of, which works perfectly fine and is virtually bug-free. For instance, imagine that I'm building a website. If size is a predictor of quality, then you'd expect me to add bugs as time goes on, and that's probably true. But my site is stacked on top of Chrome, an abstraction with hundreds (?) of millions of lines of code that virtually never give me an issue. This is stacked on top of the typescript compiler, the C++ compiler, the operating system, etc, and those all run in the millions (TS) to tens or hundreds of millions (OS) of lines of code! So while I don't doubt that codebases get buggier over time, this seems to neglect the fact that I use truly immense amounts of code every day that don't seem to run into an issue. Perhaps the bugs grow around the edges rather than in the center, or something like that?
Chrome, popular compilers, popular operating systems skew towards the highest end of quality and developmental rigor.
An analogy would be the craftsman-factory. A craftsman knows and performs every step of the process, and the end product depends on the skill of the craftsman. While a factory each worker is limited to a specific job and the combined effort of factory workers allows products well beyond the reach of any craftsman.
How many of those bugs affect you on a day-to-day basis? For me, the answer is 0. In fact, I can only think of one time ever when I was affected by a browser bug as a developer. I think that most of the reason Chrome has so many bugs is probably just because a staggering amount of people use Chrome, meaning they can inspect every nook and cranny.
For a somewhat contrived example, imagine I write a program to add two numbers together. I write a single line of code: `var foo = bar + baz;` in JavaScript. If I ship it to no one, I can claim it has no bugs. But if I ship it to a million people, then I might start getting bug reports from people running it under extremely precise circumstances and running into issues. For instance, maybe I'd get a bug where two numbers sum to be negative because the user ran into integer overflow when bar and baz are very close to INT_MAX. I might also get reports of lack of precision when bar is extremely large and baz is not (e.g. sometimes foo + 1 === foo in JS if a is huge). I suspect most bugs in Chrome are of this variety, or, more likely, they are even more obscure.
It seems to me that it's not a strict 1:1 correlation between size and quality - there's also a factor where more popular software has a lot more people looking for bugs.
Superfluous use of the strict equality check there. An ordinary comparison (i.e. using `==` aka "double equals") is sufficient.
The majority of the bugs I see are failed tests, null dereferences, unreachable code reached, very slow code, UI bugs, crashes, etc.
Come to think of it, I've had 2 chrome bugs on my phone in the last week or so. Some weird UI bug with tabs, and a more severe one where it just kept crashing once in like 10 seconds. It magically stopped after a while.
I would hypothesize three potential factors are at play, especially for software like Emacs.
The first is that some bugs becomes, themselves, "features" over time. It might be wrong from the standpoint of what was intended or desired when the code was written, or what is documented, but because that is how it works, that's how it is now expected to work, and so it's no longer a bug.
Another factor is that some bugs are "shielded" by the surrounding code, effectively two pieces of code offsetting each other's bugginess. Unless and until you either replace one or the other, or need to interact with one or the other in isolation, the bug isn't apparent. Kind of like how going to the chiropractor for one problem can "unlock" other, latent, problems.
The last one, somewhat related to the first two, is that some well factored software systems do a very good job of isolating their most stable parts, leaving their less stable, more commonly changed or iterated parts in separate modules or components. Because the change rate and focus of effort is on the "outer" parts of the system, less time and energy is spent on the "inner" part. Very often, developers of the "outer" parts will accommodate the bugs of the "inner" part, because changing the inner part requires both greater knowledge of the overall system, as well as the potential for a larger "blast radius" (and therefore assumed responsibility) for ones change. So rather than debugging the "inner" part, the "outer" part grows lots of accommodations for the inner part. It's not until that inner part can no longer be accommodated, or it becomes a security vulnerability, that the community developing the system will see the value in "fixing" the "core".
Anyhow, just a set of hypotheses.
Whom observing which reality?
I have made no scientific study - I am not stating any sort of strong opinion, just an uneasy feeling - that about 90% of the software and uses of computers subtract from humane well being and welfare, not contribute to it.
Looking at you Javascript frameworks.
Looking at you databases and tracking.
Looking at you spyware and advertising bots.
Looking at you killer robots (yes they exist - mostly thinking of the horrific USA military drone assassination programme)
So many examples to choose from. I should not complain, I make my living from computers and the uses I facilitate are no shining examples of improving people's lives
Evolution: modifications_size / time / codebase_size
Mature systems tend to have lower Evolution. Thus, there are more bugs being fixed than created. Over time, this leads to stability.
On the other hand, one bug fix creates three more bugs, on average. So, I don't know why on Earth stable systems are even possible...
Sounds really interesting, where is this stat from?
They aren't, but it just looks like they exist. The more complex something is, the less likely it is to notice a bug in the complex dependencies.
Ideally. But in practice, they must add features to sell more copies. And even if they have no evolution, OS's evolve that make the software worse over time (e.g. "modern" UIs, hardware vs software rendering, etc.).
We see this in many products that were once best-of-class, e.g. Paint Shop Pro, ACID Pro, etc.
Does it? I tend to think it doesn't (in terms of code size; obviously it does if we're scaling by number of users)
Even if bugs scale sublinearly in practice- my gut tells me it's because development as a whole slows down even more than the bugs accelerate, partially making up for it
I.e. suppose your codebase size grows 5x, and the bugs per change grows 10x, but then the org's rate of change also slows down by 5x. Now your bugs per time has only grown by 2x.
Jokes aside and that being said, I agree that there is something helpful about slow, organic (in your words, conservative) growth. It’s neither necessary nor sufficient for stabilization, but it certainly seems to correlate. GNU in general seems to be a good example.
Survivorship bias will explain all of that. You'll rarely encounter negative software because it gets discarded. It doesn't mean it's never created, it just dies before getting wide visibility.
> otherwise software wouldn't have economies of scale.
The fact that software can be developed once and sold indefinitely means the marginal cost is near zero. That is an unbelievably huge economy of scale that would dwarf many other superlinear problems.
> how do we explain the apparent stability of large but mature codebases?
Not all software changes are additions. If assume that codebase size is positively correlated with number of bugs (i.e. more code = more bugs) and that most changes that do not grow the codebase are likely to fix at least one bug, then mature codebases should be expected to improve in quality to the degree that number of changes outpaces total codebase size.
I'd propose a slight revision: "software can be developed once and used indefinitely." I would bet that most code nowdays is written inside organizations and never directly "sold" over and over again to a customer. But even that tends to result in those organizations wanting more and more, because even a buggy one tends to save people time compared to doing the process manually, and then there's another process they want to automate next, etc...
Having any ongoing development means you get an equilibrium level of defects, though.
Your logic is flawed here. First, software has economies of scale because it's easy to produce the second copy of a program once you have the first one. It has nothing to do with lines of code.
Second, buggy software being better than no software would still be true even if bugs were super-linear in the number of lines of code.
I'm pretty sure that bugs aren't sub-linear. As you pile on more and more lines of code, the number of bugs per thousand lines goes down? The code gets better as you create more and more of it? Experientially, no.
I think it's super-linear, for a couple of reasons. First, the bigger the program, the bigger the team. The bigger the team, the more below-average coders on it. Second, the bigger the program, the more room for weird interactions between obscure conditions.
No, it's definitely not sub-linear. Bugs per line of code do not decrease as you have more lines. I suspect it's
It creates buggy software because the team members have to coordinate with more people.
But it must, by the very meaning of "below average coders" - it's implied that we're rating hypothetical coders on software development skills, which include the ability to make correct decisions (both at code and architecture level).
> It creates buggy software because the team members have to coordinate with more people.
Those are not independent variables. The bigger the team, the bigger coordination problems, the more bugs. But also, ceteris paribus, the worse the coder, the more problems there are coordinating with them, thus the more bugs. And also, ceteris paribus, the worse coders are on average in the team relative to global average, the more of them are needed for a project, which makes the team bigger and coordination more difficult. Etc.
I'm not sure about this, our e-health system had millions spent on it and eventually still was discarded, after "sort of" working for months and slowing down all of the doctors and pharmacy workers who had to deal with it, as opposed to just using paper slips for prescription medicine, or even individual hospital data management systems: https://www-lsm-lv.translate.goog/raksts/zinas/zinu-analize/...
In addition to instability, even in my country, Latvia, there have also been large enterprise systems that have had serious performance issues, like the government e-system platform latvija.lv went down when the COVID vaccination signups became available, which was problematic, given how many other systems depend on it as a login provider. We've also occasionally have had the national tax system go down in the dates when people can start submitting their tax reports.
In regards to security, you occasionally see all sorts of problems even in large enterprise codebases, a recent case was the Krete education system source code (not in my country, just a relevant example), where for some reason code like this was running in production: https://www.reddit.com/r/ProgrammerHumor/comments/yqp2o6/our... here's an article about it https://hungary.postsen.com/trends/86603/The-CRETE-study-sys... (though it got hacked in a phishing attack)
Also, there have been cases of problematic deployments bringing down entire orgs, Roblox had their core business be halted for a while a year ago because of problems with their Nomad cluster, which resulted in a 3 day long outage: https://roblox.fandom.com/wiki/2021_Roblox_outage
The Knight Capital Group also lost more than 400 million USD in less than an hour and went bankrupt because of a bad deployment: https://dougseven.com/2014/04/17/knightmare-a-devops-caution...
I've personally seen systems that have gotten so complex that they're a liability - if there is no full test coverage (as close to 100% as you can get) then you're usually under the risk of wrong calculations or behavior in the system, which may or may not be possible to roll back, depending on the nature of the system. Sometimes it might not even be easy to detect, but the implications can be pretty great, especially if you're dealing with finances. Or having people depend on the systems and not having a business continuity plan were they to go down, due to how integrated things are in our current society.
There is plenty of software out there and everything around that software (other code that determines how it will run, how it will scale, how it will be secured) that has plenty of potential for creating lots of negative value, all the way up to making organizations evaporate, or causing lots of suffering for numerous people. One doesn't even have to consider something like t...
If the codebase/project became "feature complete". Once a project becomes feature complete, then finally bugs are able to be squashed without reintroducing new ones. The code becomes better because the deaign begins to accurately model the program since there is no feature creep.
The problem with large programs is when you _continue_ to add on to them. Non-open source very rarely gets to the feature complete stage.
https://dl.acm.org/doi/10.1109/32.935855 (15 pages without appendices)
RE: the article itself
I’ve been on projects where we eventually shipped the most poorly written code (variables with ambiguous naming, methods that are several hundreds of lines, classes with 5-10k lines, little or no test coverage). This code was baked into a consumer device, and we shipped over 16 million units over almost the span of a decade.
I’ve also been on projects where the code was practically a work of art. By the time we were “done”, the market had changed from under our feet. What we were doing wasn’t so interesting or novel anymore. We never shipped.
It doesn’t surprise me that more code == more bugs (or an increasing probability), but it’s good to keep in mind that value is what we care to produce. Everything else is a cost, and spending too much on any particular cost can take the whole enterprise down the ravine, regardless of good intentions.
The UI is somewhat hidden, but a long press on the back button will open a pop up menu, which makes it easy to jump back two pages.
Other than that, it is totally a bug in the website. I wonder how many lines of code it has??
It's WordPress, so right about all of them.
In particular if adding source lines describe the program more specifically and accurately then that should enable compiler to produce more compact/efficient code and I'd imagine also contribute to correctness.
the paper finds that faults (and wmc (methods per class), cbo (coupling between object classes), response for a class (rfc), number of methods added (nma), etc.), correlate to class size (non-comment source lines of code), not program size
it only looked at one program so it could not measure any effects due to program size
moreover it wasn't measuring code quality in the sense of 'defects per kloc', it was measuring defects (whether a bug had been detected in a class in the field or not)
stripping away the acronyms, what they found was that classes that contained more code more often had at least one bug, and also had more methods, but that having more methods without having more code didn't make classes significantly more likely to have a bug
and similarly for the other complexity metrics like how many different methods a class calls and implements (rfc)
this is unsurprising, since things like the number of lines of code in a class and the number of different methods in a class are just alternative metrics of class size, and everyone knows that in general more code means more bugs
that's why we measure code quality in defects per kloc and not total defects. the paper didn't even try to measure code quality in that sense
that doesn't mean the paper is bad. if the paper's authors are correct that many other papers have failed to control for size in their defect metrics, they have identified a serious shortcoming in the existing research literature; haldar merely totally failed to understand them
so the paper haldar tried to summarize doesn't measure either program size or code quality
(some of their references did look at program size tho)
why are none of the other comments pointing this out
are they all just commenting without having read the paper
now i am sad
...What? This is hilarious.
How is using an acronym to provide a brief summary of something indicative of correctness?
people who don't read long things sometimes know a lot about things like how to weld, how to comfort the grieving, and what makes them happy, but they are always profoundly ignorant when it comes to book learning
that's what happened in this case, where haldar's summary claims that this paper shows that a variable they didn't attempt to measure is the best predictor of another variable they didn't attempt to measure
evidently he just looked at the diagrams and guessed what the words meant
You seem to have fundamentally misunderstood how this acronym is used.
When provided by an author, it doesn't mean that the author didn't read... It's literally just a substitute for "Summary:". An indication to the reader that if they find the article too long to read, they can read the immediately following subsection to get a summary of the article.
Do you also think that people who say "Summary:" are profoundly ignorant? Or are you just against acronyms?
and evidently in this case the blog post author really didn't read the paper they were 'summarizing'
i find your contributions so far to the discussion of this paper profoundly disappointing
>their (presumably profoundly ignorant)
>i find your contributions so far to the discussion of this paper profoundly disappointing
I'm not sure why you are so hostile, and why you think everyone except yourself is profoundly ignorant, but it's clear that you have some sort of world view where you're superior than anyone else you interact with. I doubt anything I say would convince you otherwise.
tl;dr: I found this conversation disappointing, too.
rather than being hostile, i have taken the time to explain in more detail the things you asked about in my previous comments because they were unclear to you
i am surprised that you have stooped to launching personal attacks on me in this comment, since so far i have criticized only your low-quality comments (and the original blog post) rather than any presumed personal attributes of yours
try to do better please
You've called me "profoundly ignorant" several times in this chain. How you can say this with a straight face is baffling.
i said that people who don't read long things are profoundly ignorant, at least within the sphere of book learning, though often not in other ways. that's not a personal attack on them, it's just an obviously true statement if the phrase 'book learning' refers to anything at all
you haven't said even once that you don't read long things, so there was no reason for me to even suspect that what i said applies to you, even if it were a personal attack (and, well, some people are sensitive about being ignorant because they're used to being criticized for it)
for example, by emphasizing the praiseworthy attributes you have in common rather than the misfortunes that have befallen you and the self-destructive choices which, in cases like this one, perpetuate those misfortunes
i mean it might be useful to read a book from time to time, or at least a 25-page article, and that's less likely if you turn it into an identity threat
we're all born profoundly ignorant but we don't have to stay that way
if you're the most intelligent person in the room maybe you're in the wrong room. this can easily be reversed to provide a plan of action if being the most intelligent person in the room is what you value most. beware, though, some people are smarter than they look
I’m struggling to understand how this is the point that’s most worth debating here. What do you propose we’ll learn?
not sure that's a good excuse for their total failure to contribute anything substantive though
The caveat is code bases with targets on coverage. The ratio becomes less meaningful then.
My loose reasoning says it has to be linear (like interest generated on bank accounts). Otherwise you could simply merge two large codebases to squash bugs (in the case of sublinear bug growth) or divide a large codebase (in the case of superlinear bug growth).
ALternatively, in a sublinear-bug-growth codebase, you could just keep adding and adding to it until eventually you'd add no more bugs.
The real answer is probably that it might depends on how tightly coupled the code is, how much test coverage there is (assuming that tests are included in code size), the quality of the authors who wrote the code (this might be circular), maybe even on the technology used, or what you consider distinct bugs, etc.
My intuition leans towards super-linear, but only slightly so -- large codebases tend to have many authors (many of whom may have since stopped working on the project) and be old.
But there are many ways to make it non-linear, e.g. one can build a nice high-level structure which becomes irrelevant when goals shift. Or one can sign up for a paradigm which requires better developers and deeper understanding.
> divide a large codebase (in the case of superlinear bug growth).
I do almost exactly that when approaching a problematic system: refactor aspects of it as pure components or separate services in order to isolate behaviours from each other.
Even just what you describe - breaking a large codebase into two smaller (working) ones - exposes an interface that can reveal failed intentions. It's just usually an incredible bitch to do, unless you initially refactor the parts to minimize internal state and the types of interactions the two parts have (ie - make them more 'pure').
Decidedly super linear, IME.
To be sure, well-written code decouples the different components so that distant pairs of lines interact as little as possible, and thus it's more difficult for unforeseen constellations of factors to arise. Well-written code also minimizes the possible values that are used and have to behave correctly. But all else equal, it will be exponential.
Your intuition is correct, that you can significantly reduce bugs by splitting an N-size code-base into two N/2-size code bases. But there's only so much you can split it down while achieving the desired functionality. Plus, there's overhead to constantly switching between codebases.
Depends on whay you're doing; are you implementing features, refactoring, or fixing bugs?
Generally, I'm guessing that the number of bugs will be linear, but the number of bugs in common code paths will be logarithmic.
Or perhaps that the likelyhood of a bug in a given piece of code decays exponentially with the age of that code.
There's also the issue of developer familiarity with the code base, and where in the code base the changes are occurring.
Make it work Make it right Make it fast Make it small
(Classically attributed to Kent Beck, but he left off the make it small which I added)
The size of the solution must not exceed the size of the problem it solves. You need to focus on the essence of the problem and make compromises based on what is most important. Keeping the lines of code count low is a worthy target.
There is a balance to be reached but devs today are far too permissive when it comes to complexity creep. It should be setting off alarm bells much sooner.
Of course, everything being equal, more code leads to more complexity.
In this context, complexity is related to human capacity to collectively read/understand/maintain a code base.
I've been writing software almost non-stop for close to 25 years, and I still don't understand most of the code written by others.
I especially struggle to read anything with templates, generics, lambda...
But I can read straight C code, code written by Fabrice Bellard or John Carmack in the early days of id software.
I think you nailed it. In my experience, the increase in complexity isn't necessarily inherent to the amount of code so much as the amount of abstraction used as code size increases.
I've worked on some large codebases that only used simple, well-understood and/or well-documented abstractions which didn't feel as nearly complex as other codebases with abstractions that were more complicated than they were worth.
The thing is not the total size but the boundaries and the understanding of each layer and how manageable each layer is for the humans modifying and mantaining it ... which is, of course, the total cognitive complexity.
X lines of simple code containing a bug vs X lines of complex code also containing a bug.
How long does it take for average joe coder to find it in both cases? Is the experiment really needed?
https://blog.eutopian.io/winning-systems-security-practition...
Size is the best predictor of code quality - https://news.ycombinator.com/item?id=3037293 - Sept 2011 (123 comments)
Some people write clean but long code. Others write super compact code that nobody else can decipher (except the compiler I guess).
I'd wager that those of us that frequent HN probably see less of the sort of truly bad code though since it is usually the code written at lower paying jobs and fields, but which possibly makes up more LOC than all the "ok" code.
Like even something as freshman as mergesort falls in this category where you're sort of squinting and wincing trying to figure out how p, q and w relates to i, j, i0 and j0.
Sure you can say let the PhD students write that code and put it in a library and we'll just worry about `sort(array, start, end)`, but that's just sweeping this reality under the rug.
Truth is if you're not willing to venture into this territory of non-trivial algorithms once in a while, you're seriously hobbling what you're able to do.
The trick is knowing when it's the right thing to do. It's the closest thing we've got to dark magic. Dangerous and unpredictable, certainly, but also immensely powerful.
Entire industries are built around someone one day sitting down and developing a piece of non-trivial algorithm. The big data boom of a few years ago, block chains, machine learning, ...
That is not clean code. period.
The tricky thing is, "clean code"[1] ends up hitting against the limit of human working memory. Past some point of verboseness, that "clean but long code" gets very hard to work with, because the meaning is spread too thin, it doesn't fit in short-term memory anymore. Conversely, "super compact code" may be hard to decipher, but you know the whole thing is contained in the handful of lines you're looking at. As a crude analogy, it fits in L1 cache, whereas dealing with "clean code" means chasing pointers.
--
[0] - FWIW, I rarely have people complaining during code review, and more often I receive compliments for readable (and documented!) code, so I feel tentatively confident I'm hitting the right balance. I think the trick is to not be defensive, but instead to pay attention when someone signals the code is too difficult to understand, and adjust accordingly.
[1] - As a somewhat generic term; the advice from the book "Clean Code"... isn't very good.
- Usually the work crosses several layers in at least one place - e.g. implementing or modifying a feature will require you to touch implementation and interface layers. If those layers are spread too thin, they won't fit in the head.
- Too much "clean code" will make a single layer no longer fit in working memory even at an "overview" level. This is the most common issue in my experience; a problem created by adherents to the idea of "lots of short, simple functions". Even most trivial feature makes you jump between half a dozen functions, and IDEs don't help there the way they should (i.e. by allowing you to view - and perhaps edit - code with select function calls inlined in place).
You are doing it wrong then. It's basically a violation of SRP. Your changes should touch only one layer. If it's not, you have fragile abstractions.
I agree there are a number of cases when you introduce a new thing you should thread implementation through all layers. But not for maintenance and bugs (common case).
> Too much "clean code" will make a single layer no longer fit in working memory even at an "overview" level.
But why you need to keep it in memory? Do you keep all OS APIs? Or standard library? They are other layers you use and have no issues with broad interface. Your application layers is the same thing.
http://www.smashcompany.com/business/a-program-of-zero-bytes...
The only adjustment I'd make is that size should be measured after some normalization pass (eliminate comments, whitespace, make variable names the same length, etc)
If it's in a uni- expect for the researchers to be biased towards the practices that are advocated in their academic text books on the subject.
If it's an organization: Expect for it to be biased towards what they want for that language (ms for supporting c# for new features or moving people away from old features)
---
I saw this in the anti-TDD/unit testing/Testing papers. They're making claims based on a flatten view of development and aren't capturing what happens on the local machine where those are intended. Additoinally they don't consider the externalities about how you approach the code base.