I'm a programmer that hates to comment his code so I don't do it. Most programmers feel the same way. Basically the reason we don't do it is because we don't have to. I suspect the only way it will happen is if our code needs it to properly compile.
Why don't you have to comment? The first question should be, do you mean you aren't forced to write comments or that your code simply does not need the help of comments?
Strangely, I really like to comment my code, although it frustrates me that even the best IDEs have zero support for keeping comments up to date with the code.
Changing, say, a function should prompt the IDE to somehow "tag" the associated comment so that you remember to check its accuracy, at the bare minimum. Other nice-to-haves would be having references to code, e.g., if you talk about variable x in your comment and then rename it, the comment reference should be updated.
You can feel the disdain for code commenting in the industry by looking at the anemic tool support for comments.
Agreed. One of my projects is about updating a specific code editor to manage comments as a "meta-data" of the code itself. Otherwise, maintaining comments becomes a nightmare in the maintenance regime.
? I comment my code all of the time even more my own projects. I want to know what I was thinking at the time I wrote it. Unless a condition is explicit, I explain what it is doing.
If you think you don't have to comment your code, I don't think your understand the purpose of comments.
If you've been a developer for any length of time, I'm sure you will have encountered a situation where you've read some code, thought to yourself "wow, that's terrible, I'm going to change that". Then, several hours later, realised why it was done the way it was and actually, you can't really improve it after all. That is why we comment; to save the next person who enters the cave (and, if you're the kind of dev who spends several hours trying to understand something and doesn't retrospectively comment what you've found, then I'm afraid we can never be friends).
And please, don't assume the next person has the same knowledge of the system that you do.
Assuming that is the case and let's also assume there are no related test cases. I will just see where the method is called and try to build test cases around that. Then I can start to deconstruct the method and rebuild it without even understanding it fully. Backed on tests.
Situations like that should be few and far between. I write an occasional comment if I'm doing something confusing or unintuitive but the hope is you're not writing a lot of confusing code.
But commenting is mostly for yourself! For example, it is very helpful to you comment only your variables (including function arguments).
Declare each variable and argument on separate lines, and add a one-liner comment next to each. If you are afraid that you will fall into comment verbosity hell, you can set up a rule to force yourself to use only single-line comments.
I have to comment my code so future-me will understand it. I'm often grateful to 2-weeks-ago-me for them. But this awesome rant isn't actually about comments, it's about academic writing in general.
I've always thought that academic writing is designed to impress rather than educate. I haven't read much academic code, but it seems the trait has carried over.
I really can't stand code that is designed to make the programmer look smart. I stopped messing about with CodeWars for this reason. All this dick-measuring about how few instructions you can use to implement a problem eyeroll. It misses the important thing: when some poor sap comes back to maintain the code in 2 years' time, can they?
Believe me, very few grad students are writing code to "look smart".
First of all, coding isn't exactly a high class activity in most of academia. Any implementation that's not an uncommented mathematica notebook or matlab mess is in the top 1% of academic code.
Second of all, no one reads the code. So while there are incentives to intelli-obfuscate papers to trick naive reviewers, those incentives don't exist with code. (Oh, and it doesn't really work out well in papers either most of the time.)
Don't forget that most academics cannot code correctly. Yes, even CS academics. In the AI field (not ML, but "real", symbolic AI), top-level open source academic software is, 90% of the time, pure shit. Sure, it does work, provided conditions don't change too much (for example, undefined behavior that will break if you compile with a newer version of gcc), but I hope you won't have to maintain it.
Academics are not rewarded for good software engineering. The incentives are not properly aligned. And more-over, the that's not necessarily a bad thing! Research is research, not engineering.
Stop expecting production quality code out research groups. The purpose of research is to explore ideas, not implement some epsilon of your product.
If you want a perfect exposition in addition to a high quality implementation, there are ways to get it. Pay the consulting fees. Or pay the tuition. Or hire the students, Or if you can't afford that, wait for the thesis/book.
> Is pseudo-code a fucking premium?
In a paper? Well... yes. Categorically, emphatically, yes. If you want to read an exposition where space does not come at an extreme premium, you'll have to wait for the thesis/journal article.
Space comes at a premium for stupid reasons (artificial page limits) but also for good reasons (expositions cost time, which costs money and grad student time, and fuck if I'm doing a 10 year Ph.D. at $7/hr because you're irritated at some python code.
> Can you at least try to give some intuition before showering the reader with equations?
Either 1) no because you're not the intended audience, in which wait for the thesis/TR/dissemination blog post where I'll work every gory detail; or 2) no because the equations are hard for all of us to understand and no one has solved the exposition problem yet.
> How the fuck do you dare to release a paper without source code?
So all the products you work on all totally open source, eh? Either fully commit to Stallmanism while living off less than minimum hourly wage or get off your high horse.
> Why the fuck do you never ever add comments to you code When naming things, are you charged by the character? Do you get a bonus for acronyms?
Do you realize that OpenAI having needed to release a "baseline" TRPO implementation is a fucking disgrace to your profession?
Jesus christ, who decided to name a tensor concatenation function cat?
Because they're being paid 30k a year at best for 80 hour weeks under intense pressure to implement new ideas and the prop firms and Googles are knocking with 200k comp packages and honest to god 5 day work weeks. Time is (SERIOUS) money when you're slogging through grad school. Again, walk a mile.
And again, really, the point is to test ideas. For every publicly release line of code coming out a research group there's 5x or 10x unreleased code implementing dead ends.
If you want all this, stop bitching at over worked and absurdly under-paid grad students, and start calling your congressman to support higher NSF budgets so research groups can hire more permanent research support staff.
I agree with you but I think this highlights a point. If academic articles are out of reach of people who work in the industries they're researching, doesn't something need to change? Shouldn't incentives be better aligned. Should this type of complaining be celebrated?
> Shouldn't incentives be better aligned. Should this type of complaining be celebrated?
IMO it's not as clear cut as some people think. Honestly, is the cost worth the cost to tax payers if the super high quality implementation is going to be out-of-date in 3 months?
It depends on the research project and how fast the field is moving.
At base, I wonder if the current system isn't reasonable. If research can't even stay fresh long enough to make it from conference paper to journal paper, maybe we shouldn't be throwing hundreds of thousands of engineering dollars at quality implementations.
I think you're building a straw man of "hundreds of thousands of engineering dollars" here
All we're talking about is better quality code. This isn't a huge cost. It's usually just a matter of a small change in mindset while coding, and a few minutes refactoring and commenting once the code is working. Those minutes are repaid (and more!) the very next time the author has to revisit that code again. Good quality code saves money, it doesn't cost it.
You can leave error conditions unmanaged, leave the UI as a complete mess, don't worry about internationalisation, none of the things that add serious cost to a commercial project. This is not that.
If research coders aren't aware of code quality currently, maybe it's time that they got taught it? Writing good code will help researchers who write code, not hinder them.
> I think you're building a straw man of "hundreds of thousands of engineering dollars" here
Been there, done that, no. Refactoring a crappy implementation into a high quality, documented code base takes an enormous amount of time. And literally no one in academia cares about it. Industry doesn't care about it either because there's a presumption that any software created while in academia is inferior quality.
If you advise students to do this type of work with the expectation of any type of personal reward, you're a liar. They're basically doing software engineering that academia won't reward them for, and their year of software engineering won't count as job experience if they go into industry. And they aren't being paid software engineering salaries. It's just exploitative.
> All we're talking about is better quality code. This isn't a huge cost
This flies in the face of what the market bears in tech salaries. But no matter; I'll just never pay my software devs more than 30k, and also only give them maybe 20% of their time for actual software development... I'm sure I'll get high quality code! /s
> Those minutes are repaid (and more!) the very next time the author has to revisit that code again.
Which for 99% of research code will literally never happen.
> If research coders aren't aware of code quality currently
Tons of grad students leave academia after their phd, take very high paying jobs in industry, and become valued members of software engineering teams.
Research jobs are not software jobs. The goals and incentives are different. Sometimes it makes sense to create a maintainable research software system, but usually it makes no sense because the code is throw-away for testing an idea. That's OK. No one owes you an epsilon of your product.
You might be conflating software graduate students with research developers. I have translated a lot of research code into production code; whether it be sourced in Matlab, incomprehensible C, confused C++, or convoluted Numpy.
Okay - so grad students have very little incentive to write readable code... fine, understandable. They are usually lucky if they can write an algorithm which isn't finely tuned to a very narrow dataset. In that sense, most Ph.D papers are outright fabrications lying about capabilities written by authors seeking paychecks. If people were able to investigate the underlying source code, that would be fairly apparent.
That said, why do so many professional research developers also suck as writing comprehensible code? They have fat paychecks, what's the problem now? There absolutely exist research developers who write very high quality software. They may still use very terse nomenclature (due to high cognitive load), but their code can be logically comprehended by those who do not understand the field. There are a lot of reasons (I think) for this; no incentive to improve, poor understanding of their own discipline, no acknowledgment of the problem, expectation that such work is for "engineers".
In reality - a lot of research developers are simply incompetent, but they find a way to regularly convince others they are not.. often through obfuscation. They're stringing together implementations from various publications (which themselves might be total fabricated lies) with relatively little comprehension.
That's okay, a lot of software engineers are incompetent too.
> You might be conflating software graduate students with research developers
I'm not conflating. The vast majority of research software is written by grad students.
> why do so many professional research developers also suck as writing comprehensible code?
Because incentives aren't aligned. And it's not even clear they should be aligned.
> There absolutely exist research developers who write very high quality software... In reality - a lot of research developers are simply incompetent
There absolutely exist software developers who write very high quality software... In reality - a lot of software developers are simply incompetent.
> If people were able to investigate the underlying source code, that would be fairly apparent... Own it
1. A huge amount of research software is open-sourced and does work as advertised.
2. You don't get it both ways. Either no open source demands, or no expectation that grad students should "own their work" -- and then making demands that they literally don't own their work.
> When naming things, are you charged by the character? Do you get a bonus for acronyms?
Funny you're saying this, because it took me a while to understand what you meant by "DL" (which is required to understand what code you're complaining about).
Making code readable is only part of the reason for comments. The other part is why the code is there and the history of why certain decisions were made. Without this information, changes to the code can have ongoing subtle and not so subtle ramifications.
That isn't a compramise, it's the goal. Your comments should be about the "why", and not the "how". We already know how, that's what code is - instructions about how something works.
What we don't know is why it was done in the first place, or why it was done a particular way, or why I shouldn't change that line that looks wrong but is really correct.
I hate to disagree but that's fairly clean code. It's not doing anything tricky or exceedingly clever, so the code speaks for itself. They're even polite enough to separate their implementation from their interface, which is rare for projects in dynamic languages. It reads like idiomatic, OO, pep8'd python. Python is basically executable pseudocode as it is, provided the method names and parameters convey the semantics of the code.
The code speaks for itself, so there is no way to confirm it works as intended. Because the intention is undocumented.
The code tells what the program does. The comments should state what it is supposed to be doing and why it should do it. When they don't match you have found a bug.
As a general rule I comment code that I don't find self-explanatory, and that I can work up a meaningful comment for.
The first condition is of course a problem that one has to deal with if the person writing code finds things self-explanatory that to you are highly complicated.
The second condition is of course a problem for those people who are not able to explain things well in human language.
edited: for clarity (prior actually my really clever example of people who are not able to explain things well in human language)
This is less a problem with the code being well commented and more a problem with it being difficult to understand. Comments can help clarify a difficult to understand block of code, but they don't make that code more readable, which doesn't really solve anything when you go to maintain it.
The fact that the code is unreadable is because it's written by a ML expert. I'd say it's actually higher quality than you'd usually get from academia, but nowhere near what you'd see outside of that. Academic code serves a very different purpose than production code. It's meant to prove a point and usually not used after. There simply isn't incentive to write well tested maintainable code.
We run an Incubator for university students, helping them bridge the gap between "academic" and "useable" code, so I have to deal with this topic all the time.
Why is code coming out of research labs/universities so bad?
1. DON'T WANT TO WRITE CLEAR CODE
Different incentives between academic research (publications count, citation count, fear of being scooped if research too clear...) and industry (code maintainability, clarity, robustness, handling corner cases, performance...).
2. DON'T SEE WHY CLEAR CODE MATTERS
Academic projects are one-offs, not grounded in a wider context. Even if the researcher would like to build something long-term useful and robust, they don't have the requisite domain knowledge to go that deep. This external knowledge, the subject matter expertise, is hard to come by.
In lieu of that, researchers start solving artificial problems on artificial datasets. The proper project cycle is broken, there's no feedback from other people using the fruits of your labour.
3. DON'T KNOW HOW TO WRITE CLEAR CODE
Lack of programming experience. Choosing the right abstraction boundaries and expressing them clearly and succinctly in code is HARD. Naming things is hard, structuring things is hard -- clarity in your head first, the code follows naturally.
But it's a skill like any other. When we put students to work on real ML projects, they are shocked. Many have never seen a properly designed piece of code in their life, don't know any tools, how to share or collaborate (git, SSH...). It's not like they'd sprinkle some extra comments here and there and be done -- the whole code structure and how they attain clarity of thought is different.
The GOOD NEWS is, it doesn't cost any more time to write good code than bad code.
So once they learn, everyone wins. That's the main purpose of our Incubator programme. It's not like researchers write crap code on purpose, no need to assume bad faith. Seeing the fruits of your labour useful and used by others is immensely rewarding!
45 comments
[ 2.6 ms ] story [ 35.5 ms ] threadChanging, say, a function should prompt the IDE to somehow "tag" the associated comment so that you remember to check its accuracy, at the bare minimum. Other nice-to-haves would be having references to code, e.g., if you talk about variable x in your comment and then rename it, the comment reference should be updated.
You can feel the disdain for code commenting in the industry by looking at the anemic tool support for comments.
If you've been a developer for any length of time, I'm sure you will have encountered a situation where you've read some code, thought to yourself "wow, that's terrible, I'm going to change that". Then, several hours later, realised why it was done the way it was and actually, you can't really improve it after all. That is why we comment; to save the next person who enters the cave (and, if you're the kind of dev who spends several hours trying to understand something and doesn't retrospectively comment what you've found, then I'm afraid we can never be friends).
And please, don't assume the next person has the same knowledge of the system that you do.
Declare each variable and argument on separate lines, and add a one-liner comment next to each. If you are afraid that you will fall into comment verbosity hell, you can set up a rule to force yourself to use only single-line comments.
I've always thought that academic writing is designed to impress rather than educate. I haven't read much academic code, but it seems the trait has carried over.
I really can't stand code that is designed to make the programmer look smart. I stopped messing about with CodeWars for this reason. All this dick-measuring about how few instructions you can use to implement a problem eyeroll. It misses the important thing: when some poor sap comes back to maintain the code in 2 years' time, can they?
First of all, coding isn't exactly a high class activity in most of academia. Any implementation that's not an uncommented mathematica notebook or matlab mess is in the top 1% of academic code.
Second of all, no one reads the code. So while there are incentives to intelli-obfuscate papers to trick naive reviewers, those incentives don't exist with code. (Oh, and it doesn't really work out well in papers either most of the time.)
Plus the code is still fresh in your mind so you haven't forgotten why did it that way, and discarded trade-offs.
Academics are not rewarded for good software engineering. The incentives are not properly aligned. And more-over, the that's not necessarily a bad thing! Research is research, not engineering.
Stop expecting production quality code out research groups. The purpose of research is to explore ideas, not implement some epsilon of your product.
If you want a perfect exposition in addition to a high quality implementation, there are ways to get it. Pay the consulting fees. Or pay the tuition. Or hire the students, Or if you can't afford that, wait for the thesis/book.
> Is pseudo-code a fucking premium?
In a paper? Well... yes. Categorically, emphatically, yes. If you want to read an exposition where space does not come at an extreme premium, you'll have to wait for the thesis/journal article.
Space comes at a premium for stupid reasons (artificial page limits) but also for good reasons (expositions cost time, which costs money and grad student time, and fuck if I'm doing a 10 year Ph.D. at $7/hr because you're irritated at some python code.
> Can you at least try to give some intuition before showering the reader with equations?
Either 1) no because you're not the intended audience, in which wait for the thesis/TR/dissemination blog post where I'll work every gory detail; or 2) no because the equations are hard for all of us to understand and no one has solved the exposition problem yet.
> How the fuck do you dare to release a paper without source code?
So all the products you work on all totally open source, eh? Either fully commit to Stallmanism while living off less than minimum hourly wage or get off your high horse.
> Why the fuck do you never ever add comments to you code When naming things, are you charged by the character? Do you get a bonus for acronyms? Do you realize that OpenAI having needed to release a "baseline" TRPO implementation is a fucking disgrace to your profession? Jesus christ, who decided to name a tensor concatenation function cat?
Because they're being paid 30k a year at best for 80 hour weeks under intense pressure to implement new ideas and the prop firms and Googles are knocking with 200k comp packages and honest to god 5 day work weeks. Time is (SERIOUS) money when you're slogging through grad school. Again, walk a mile.
And again, really, the point is to test ideas. For every publicly release line of code coming out a research group there's 5x or 10x unreleased code implementing dead ends.
If you want all this, stop bitching at over worked and absurdly under-paid grad students, and start calling your congressman to support higher NSF budgets so research groups can hire more permanent research support staff.
> Shouldn't incentives be better aligned. Should this type of complaining be celebrated?
IMO it's not as clear cut as some people think. Honestly, is the cost worth the cost to tax payers if the super high quality implementation is going to be out-of-date in 3 months?
It depends on the research project and how fast the field is moving.
At base, I wonder if the current system isn't reasonable. If research can't even stay fresh long enough to make it from conference paper to journal paper, maybe we shouldn't be throwing hundreds of thousands of engineering dollars at quality implementations.
All we're talking about is better quality code. This isn't a huge cost. It's usually just a matter of a small change in mindset while coding, and a few minutes refactoring and commenting once the code is working. Those minutes are repaid (and more!) the very next time the author has to revisit that code again. Good quality code saves money, it doesn't cost it.
You can leave error conditions unmanaged, leave the UI as a complete mess, don't worry about internationalisation, none of the things that add serious cost to a commercial project. This is not that.
If research coders aren't aware of code quality currently, maybe it's time that they got taught it? Writing good code will help researchers who write code, not hinder them.
Been there, done that, no. Refactoring a crappy implementation into a high quality, documented code base takes an enormous amount of time. And literally no one in academia cares about it. Industry doesn't care about it either because there's a presumption that any software created while in academia is inferior quality.
If you advise students to do this type of work with the expectation of any type of personal reward, you're a liar. They're basically doing software engineering that academia won't reward them for, and their year of software engineering won't count as job experience if they go into industry. And they aren't being paid software engineering salaries. It's just exploitative.
> All we're talking about is better quality code. This isn't a huge cost
This flies in the face of what the market bears in tech salaries. But no matter; I'll just never pay my software devs more than 30k, and also only give them maybe 20% of their time for actual software development... I'm sure I'll get high quality code! /s
> Those minutes are repaid (and more!) the very next time the author has to revisit that code again.
Which for 99% of research code will literally never happen.
> If research coders aren't aware of code quality currently
Tons of grad students leave academia after their phd, take very high paying jobs in industry, and become valued members of software engineering teams.
Research jobs are not software jobs. The goals and incentives are different. Sometimes it makes sense to create a maintainable research software system, but usually it makes no sense because the code is throw-away for testing an idea. That's OK. No one owes you an epsilon of your product.
I'm talking about Person A just not writing bad code in the first place.
Okay - so grad students have very little incentive to write readable code... fine, understandable. They are usually lucky if they can write an algorithm which isn't finely tuned to a very narrow dataset. In that sense, most Ph.D papers are outright fabrications lying about capabilities written by authors seeking paychecks. If people were able to investigate the underlying source code, that would be fairly apparent.
That said, why do so many professional research developers also suck as writing comprehensible code? They have fat paychecks, what's the problem now? There absolutely exist research developers who write very high quality software. They may still use very terse nomenclature (due to high cognitive load), but their code can be logically comprehended by those who do not understand the field. There are a lot of reasons (I think) for this; no incentive to improve, poor understanding of their own discipline, no acknowledgment of the problem, expectation that such work is for "engineers".
In reality - a lot of research developers are simply incompetent, but they find a way to regularly convince others they are not.. often through obfuscation. They're stringing together implementations from various publications (which themselves might be total fabricated lies) with relatively little comprehension.
That's okay, a lot of software engineers are incompetent too.
Own it.
I'm not conflating. The vast majority of research software is written by grad students.
> why do so many professional research developers also suck as writing comprehensible code?
Because incentives aren't aligned. And it's not even clear they should be aligned.
> There absolutely exist research developers who write very high quality software... In reality - a lot of research developers are simply incompetent
There absolutely exist software developers who write very high quality software... In reality - a lot of software developers are simply incompetent.
> If people were able to investigate the underlying source code, that would be fairly apparent... Own it
1. A huge amount of research software is open-sourced and does work as advertised.
2. You don't get it both ways. Either no open source demands, or no expectation that grad students should "own their work" -- and then making demands that they literally don't own their work.
Of course if it doesn't matter if the code runs correctly, by all means don't comment it.
See "Comments are Lies" http://www.codingblocks.net/podcast/clean-code-comments-are-...
That being said, if I’m doing some obscure calculations or something ambiguous, I’ll usually comment on it (for future me to understand the context).
Funny you're saying this, because it took me a while to understand what you meant by "DL" (which is required to understand what code you're complaining about).
The best compromise is to make your code readable and only comment about the "why", not the "how".
What we don't know is why it was done in the first place, or why it was done a particular way, or why I shouldn't change that line that looks wrong but is really correct.
https://github.com/facebookresearch/end-to-end-negotiator/bl...
The code tells what the program does. The comments should state what it is supposed to be doing and why it should do it. When they don't match you have found a bug.
I forget who that quote is attributed to.
Error: Not enough emotional investment to fix a single line, but enough to write several paragraphs of rant.
Nobody gets a free lunch, not even those trying to get a free lunch from free academic software.
The first condition is of course a problem that one has to deal with if the person writing code finds things self-explanatory that to you are highly complicated.
The second condition is of course a problem for those people who are not able to explain things well in human language.
edited: for clarity (prior actually my really clever example of people who are not able to explain things well in human language)
The fact that the code is unreadable is because it's written by a ML expert. I'd say it's actually higher quality than you'd usually get from academia, but nowhere near what you'd see outside of that. Academic code serves a very different purpose than production code. It's meant to prove a point and usually not used after. There simply isn't incentive to write well tested maintainable code.
Why is code coming out of research labs/universities so bad?
1. DON'T WANT TO WRITE CLEAR CODE
Different incentives between academic research (publications count, citation count, fear of being scooped if research too clear...) and industry (code maintainability, clarity, robustness, handling corner cases, performance...).
2. DON'T SEE WHY CLEAR CODE MATTERS
Academic projects are one-offs, not grounded in a wider context. Even if the researcher would like to build something long-term useful and robust, they don't have the requisite domain knowledge to go that deep. This external knowledge, the subject matter expertise, is hard to come by.
In lieu of that, researchers start solving artificial problems on artificial datasets. The proper project cycle is broken, there's no feedback from other people using the fruits of your labour.
3. DON'T KNOW HOW TO WRITE CLEAR CODE
Lack of programming experience. Choosing the right abstraction boundaries and expressing them clearly and succinctly in code is HARD. Naming things is hard, structuring things is hard -- clarity in your head first, the code follows naturally.
But it's a skill like any other. When we put students to work on real ML projects, they are shocked. Many have never seen a properly designed piece of code in their life, don't know any tools, how to share or collaborate (git, SSH...). It's not like they'd sprinkle some extra comments here and there and be done -- the whole code structure and how they attain clarity of thought is different.
The GOOD NEWS is, it doesn't cost any more time to write good code than bad code.
So once they learn, everyone wins. That's the main purpose of our Incubator programme. It's not like researchers write crap code on purpose, no need to assume bad faith. Seeing the fruits of your labour useful and used by others is immensely rewarding!