If you abstract things but don't give meaningful enough names, you loose everything (because your callers are still forced to lookup the definitions of your abstractions, and now this definition can be hard to search for).
"By the time I’m done—adding more recovery scenarios, better logging, augmenting errors with more context, handle more edge cases—most of my boilerplate gets a lot more interesting. I did not realize that good boilerplate could be fertile ground for iteration."
This point makes sense, however I think I like to have my cake and eat it too. Java 8's stream constructions can be relatively terse, but if you need to switch back to a for loop, IntelliJ has nifty right-click menu items for doing so. I think the argument made by TFA is at least related to John Carmack's take on inlining vs. abstraction: http://number-none.com/blow/john_carmack_on_inlined_code.htm...
I followed Carmack's suggestion and never looked back. I write simpler code on average now, and I spend less time having to re-read it too because I'm changing the same lines less often. What I lose in terseness in the short-term, I regain in being able to spot key abstractions that are more crucial to long-term program health. It plays well with debugging since there are fewer places where state can act at a distance.
It's very unfashionable advice in some circles, though. It sounds just enough like "go back to writing unstructured copy-paste spaghetti" to trigger some people's kneejerk. But the ground rules of "don't take subroutines, don't jump backwards" actually keep it from getting too far out of control, because they align you towards code that only uses enough flexibility for the task at hand, hence there's a lower likelihood of missing something. Very good practice for greenfield code where you might reflexively add a generalization that you don't use. I can still get better at it.
Edit: And one of the most important things it taught me, is that I can use scope blocks and a comment instead of a function, and it'll be sufficient.
I assumed the title was about boilerplate in terms of some skeleton or framework you download. Turns out this is different from what youre talking about (verbosity). Its interesting that even in programming jargon we have two uses for the same term. They are not totally separate though. The repetitive, DRY violations eventually get abstracted into a framework, at which point the boilerplate becomes ok. And if its still too verbose, than another boiler may arose to encompass it.
Well, when you have several projects using same framework/skeleton, and the latter is downloaded and copied over and over again in every project - it became verbose on highest level and can be called boilerplate.
Yes. By using higher level constructs, it is much easier to see what is going on in the code. If there are too many lines, it is much more difficult to understand a piece of code as a whole.
Those examples are just clear indications that languages sometimes have really terrible default behaviors. Namely, Perl should have been strict by default (with a "no strict" if this didn’t perform well enough), and C++ should have at least added #import.
Interesting take. I think everyone can find something in this with which to agree, so I don't think it's a particularly polarizing way of thinking, but I haven't seen terminology like this used to describe it before. I fall into the same psychological trap in C++ that the author falls into with Python: the "if this is over here instead it's more [elegant/right/concise/impressive/less boilerplatey]". Unfortunately, I find it too tempting to value beauty and elegance (and often masking real weight and complexity) over the in-your-face 'rawness' of simple loops, branches and so on. In Go this is harder to do in general, so you only try for so long before you learn not to fight it. This comment really spoke to me:
> Maybe this is subjective but I posit that the code’s shape looks more like what it is.
Indentation isn't merely stylistic or syntactic, but it tells us useful things about code at these lower levels. A more complex code shape should convey more complex logic, and we should want that complexity apparent to us...at least sometimes. It looks great to have a nice, terse one-liner, but visually flattening possible control flow paths feels really nasty.
I iterate code like he does, then when it gets to where it's doing what I want it to do, I refactor it to make it more readable even if it reduces iterability. After the initial phase of writing the code, going back and forth, etc, the code is going to be read many more times than edited, if in the future it needs to be edited again, the future editor can refactor the nice looking code into whatever shape, and the important thing was that future editor understood my code in the first place, and I hope that future editor, when done editing, will refactor the code into being easily read for the next person, too.
It's always bad.
The second line makes you understand with a glance what it does because it is declarative and explains what is doing instead of how.
The go code is awful (personal opinion, although I hope it is widely accepted), much more verbose and it opens you up to a whole series of errors that are just impossible in the python version.
Boilerplate code by definition is just duplication and it is always bad because opens you up to bugs in a specific part of the boilerplate while you can't have any bug at all in any boilerplate if you don't have any of it.
Honestly I can't really see how anyone can prefer code that is more verbose and bug-prone to code that is more concise, clear and with less bugs.
This argument can be abstracted to subjective thinking vs. objective. In subjective thinking, every situation is unique and even the best objectively-validated boilerplate can increase risk by lowering flexibility, control, and responsibility for & deep knowledge of the situation.
Objective thinkers tend to go looking for existing, established boilerplate or frameworks because of the immediate leverage that is provided. They may think of too much subjective thought or originality as ridiculous NIH or wheel-reinvention. Their struggle is in the weighty task of grokking and maintaining third-party resources.
Scott Adams has kind of unwittingly uncovered this dichotomy with his own inventive, subjective thinking. He considers more objective thinkers as "word thinkers" or some name like that, because they go looking up the established names of things rather than understanding the things themselves.
Also Chris Langan similarly trashed Jeopardy contestants in a news interview, because they aren't subjective thinkers like he is; their minds are just filled with objective facts about various things. Well, true for him; his bias dictated the way he developed his own unique universal model--he prefers and has a gift for subjective thought.
While I don't disagree with the distinction, I don't agree that it applies here.
The thing with code is that you can take those existing patterns and make them a first-class concept in your program. You're still in "objectively thinking" - still reusing existing code - just in a more compact and safe way.
Except, subjective thinking is what leads to objective facts. If not for thinkers, there would be nothing to share.
And words are amazing. All these complex and nuanced concepts we're talking about can be expressed and shared almost instantly because of the words we share. If we had to invent words as we communicated, or worse yet, had no words, there would be no discourse because the sun would go down before we got anywhere.
It is true that words are set in their definitions and intuitions, much in the way that scientific models are and common sense is. These can be considered objective facts.
But challenging these facts and inventing new words is just an important part of the process. Some have more success with it than others, but not looking up what is already known is just wasting your time. Same with not using words that already mean what you mean.
Subjective thought leads to objective facts with fact checking. Objective facts are then better shared, and better learned, because learning is far easier than arriving at them yourself. Learning just takes communication, which is proportional to the complexity of the information. Inventing or discovering new facts requires the entire lengthy and laborious process which is will inevitably include experiments and fact checking and evidence validation if we are to be scientific. If the problem can be solved with pure thought, chances are it was a philosophical one, although philosophy too highly relies on existing language.
Generalisation is always[tm] a bad idea. Taken to illogical extremes, at the other end of the spectrum we have the monstrocities of old Perl. After all, with Perl it was often said that you could fit most solutions to a oneliner.
In fact, I would go as far as to claim that using Python for n-tuple iterators inside branched comprehensions are getting awfully close to the worst aspects of what Perl scripts were commonly criticised for.
You can code golf in any language and it will probably result in less comprehensible code. If there's one thing I think I agree with the original author on, it's that terseness is not an end unto itself. But I think the overall point that terseness can sometimes be bad so therefore we shouldn't work at a higher level of abstraction is really at odds with everything we've learned about programming in the last century.
Perl golf is certainly an extreme example, but more commonplace is the convention-over-configuration in Ruby on Rails. Mostly this is useful in avoiding boilerplate and increasing "programmer happiness", but (in my opinion) the C-over-C is slightly overcooked. I believe a small amount of boilerplate declaration goes a long way towards assisting with debugging and production correctness. Two examples from Rails to illustrate:
1. RESTful routes leading to implied controller actions that render a view without any controller declaration at all. I've seen more than one codebase with routes that could be invoked by users despite being never linked in production and having no test coverage. It's basically undocumented code due to a total absence of declaration, and a case of needing to know the implied framework behaviour to understand a program.
2. I have an issue with ActiveRecord's schema behaviour. By avoiding attribute declaration in models and just defining ORM methods by asking the database for column names, it only takes one poorly handled migration or a DBA oops to lead to some very strange production behaviour. Situation is only partially offset via the (new) attributes API and/or a schema cache dump during CI but one can never fully do away with the hazard. Philosophically speaking, sometimes Rails thinks it is dictating the database schema, sometimes Rails accepts what the database tells it, and out-of-the-box rails has no model boilerplate that bridges the gap to guarantee a specific DB schema.
In both cases I add my own safeguards for production environments, but that's a matter of programmer discipline rather than mandatory declaration.
Convention most of the time doesn't reduce boilerplate at all while it increases exponentially the headaches for anyone reading the code.
A typical example is using the awful Prism way to display a view in a WPF application.
For some reason completely alien to me the people at Microsoft practices, or whatever it was called at the time, explicitly suggested to use the IOC container directly (or with a kind of service locator) to create a view that automatically will try to load a view model with the same name if it existed, without throwing any error if it wasn't found.
This is the worst possible suggestion ever.
It is much easier and much more readable to just get your view model as a parameter in your main viewmodel, the IoC container will create it for you, you assign it to a public property and the view defined in a template is automatically shown in a content presenter.
This is extremely easy, much easier to do than to explain, there is no whatsoever connection to the container, if the viewmodel is not found you receive a runtime error, the view model has no relation at all with the view, it can be called whatever you like and the view has an explicit connection with the viewmodel.
So in this case the removal of convention based loading not only makes everything explicit, but also removes all the boilerplate needed to get a view from the container.
I don't particularly like ROR exactly for the same reason for which I don't like this convention driven way just explained.
And as you can see removing conventions and using different techniques can actually reduce the boilerplate if done properly.
Perl's terseness is about ripping meaning away from the code's symbolic representation to save a few characters. That's a related issue to boilerplate, but it's not the same thing at all.
Boilerplate is usually maximized in projects which try to give users a huge amount of flexibility that they don't need or want.
This is partly why minimizing boilerplate is so hard - one thing that continually surprises me is what users want flexibility over and what they don't. I've lost count of the number of times I've given users flexibility to do something they don't give a shit about and not given them the flexibility to do something they really wanted.
On a related note: The single thing I absolutely love in Visual Studio is how it handles C#'s #region statements. While they traditionally seem to be used mostly to group class members (which doesn't really benefit the reader much, IMHO), one can use them to hide the fact that C# still has the tendency to be a bit verbous by reducing complicated methods to pseudo code.
I tend to use this pattern quite a lot:
#region Find the first Foo with Bar
var foo =
foos.Where(f => f.Bar)
.Distinct()
.OrderBy(f => f.Baz)
.FirstOrDefault();
#endregion
I still miss a similar feature in Emacs and Vim. There are similar folding markers one could use (say {{{ }}}), but they don't seem to work together with syntax-based folding.
If you know how to do this, I would indeed be interested.
I already pondered if it wouldn't be possible to merge multiple "folding strategies" if one would build some kind of interface with which multiple routines could basically communicate folds in a buffer.
Say you have one routine scanning the syntax of the according language, one looking for folding markers and one for manual (Vim-style) folds. I guess it technically should be trivial to use all of them provided they are nested and don't intersect in weird ways (the latter could be resolved with some simple priorities).
I wound up not getting the computer out. Looking at it right now to see if it would be as easy as I was thinking.
Simply put, knowing how advanced org-mode can get with folds, this is likely trivial to do. (If you haven't played with it, I'd recommend taking a look. I don't have perfect examples, but http://taeric.github.io/DancingLinks.org is an example. Open that in emacs with org-mode and you will see that all of the source blocks can be folded together. For me, the source blocks are correctly colored, as well.)
The basic idea I had was to just make a few functions that would:
- Fold from current location to next #endregion
- List all region/endregion's in the current buffer
- Fold all region/endregion's in the current buffer
The second of those felt trivial, since it is basically M-x occur #region\|#endregion. On top of that, I just need to learn how to actually make overlays and then the rest feels like it would fall into place easily enough.
Edit: It occurs to me that I didn't show how to actually hide text, which I think is your question. Fastest way I can see is this (run this in your scratch buffer to see it in action):
(let ((o (make-overlay 5 120)))
(overlay-put o 'display "---hidden---"))
To easily delete this particular overlay, if you make it, use:
I feel that this sort of 'cleverness' in packing a large amount of logic onto a small amount of code is treated quite differently between Python and C-type languages.
In C++, would you write the same structure using a single line for-loop and ternary operators?
I tend to avoid both 'cleverness' and writing ultra-terse code for a couple of reasons.
The first is that somebody else will eventually have to work with it, decipher what the heck I was trying to achieve, and generally get tripped up and slowed down by it.
The second is that eventually I may have to work with it again, and sit there scratching my head and wondering what I'd been smoking when I wrote instead of focussing on the changes I need to make.
The point is to code for maximum clarity to humans whilst still achieving the desired functional and non-functional outcomes on the machine. You will use varying terseness as part of doing this because terseness isn't the point: it's just one of many means to your end.
I completely agree. Writing a loop with several useless variables in scope, increment operators, exploding computational complexity is for sure more difficult for a human to understand compared to declaring what your code ought to do in a concise and well understood way.
The maximum clarity for humans is always the goal and bug-prone, verbose code is for sure not the way to go.
If I were asked to do a similar thing in C it would probably look something like this --- I've renamed some of the variables to match a hypothetical situation and show that naming can have a significant effect on understanding what the code could be doing:
for(i=0,total=0;i<resultlen;i++) {
int price = results[i].price;
total += results[i].selected && (price&1) ? price : 0;
}
In C++, someone would probably do the same using the standard library functional programming templates etc., approaching the conciseness of the Python (and coincidentally, likely generating even more code than the simple loop above.)
Yes, I'd write that code in C++ in a single like if I could. At the moment it's not quite possible, but in C++17 and with ranges library the expressiveness of C++ will get quite close to the expressiveness of Python.
It's not "cleverness", but rather the ability to express the intent as succinctly and clearly as possible. Higher level languages allow you to do that, lower level languages don't. Obviously, Go is not a very high level language because it forces you to write explicit loops instead of using comprehensions or functional algorithms. Loop is a primitive concept. All other factors being equal (i.e. performance) you should always prefer comprehensions or algorithms over explicit loops.
Your comments clearly ring true, for the example code snippets shown. But i think you have missed the point of the article, which is the benefit of the Go version when iteratively changing the structure.
The author could have brought the examples forward, instead of just providing links to urllib3 and the Set type.
>Your comments clearly ring true, for the example code snippets shown. But i think you have missed the point of the article, which is the benefit of the Go version when iteratively changing the structure.
You can iteratively change a smaller, terser code, must faster. There's a reason prototypes are often written in languages like Python.
Honestly I can't really see how anyone can prefer code that is more verbose and bug-prone to code that is more concise, clear and with less bugs.
I agree with you, but seeing the number of proponents of Enterprise Java there are, it's clear that there are a significant number of programmers who prefer verbosity and extra abstractions --- although IMHO not necessarily because it's actually better.
In fact a trend I'm seeing with a lot of programming languages is more verbosity and unnecessary abstraction, leading me to wonder whether programmers actually know the languages they're using or are barely scraping by with the very minimal basics (and in the case of some languages, a bit of OOP cargo-cult dogmatism.)
Although I'm not really familiar with Python, and the order of the clauses its conditional expression syntax is unusual and a little surprising, I think that second line probably took longer to write than it did for me to read.
>I agree with you, but seeing the number of proponents of Enterprise Java there are, it's clear that there are a significant number of programmers who prefer verbosity and extra abstractions --- although IMHO not necessarily because it's actually better.
Even Enterprise Java wasn't verbose for the sake of it (I say "wasn't" because modern enterprise Java can be more like modern functional style than Go).
It was more verbose because it added tons of options and flexibility while keeping with the OO style, so everything was pluggable and configurable and factorable. But that's something else than Go style verbosity.
I don't know people who like verbosity per se.
But I can symphatize with people who think that if removal of verbosity introduces other burdens that possibly affect the understandability or refactorability of the code, it is not clear-cut if it's worth it.
Say, with Scala implicit parameters, you get to remove code from call-site. Does it make the code better? Depends on the case, in my opinion. So I don't get the point to say some people categorically want verbosity. They may prefer explicitness and unsurprising code which may require verbosity. Or perhaps the removal of verbosity requires code that allocates more memory -- in that regard Kotlin seems to be a good citizen. It has ways to make code concise without making the byte code entirely different.
> unnecessary abstraction, leading me to wonder whether programmers actually know the languages they're using or are barely scraping by with the very minimal basics
It is more that they have good abstract thinking, so abstraction is not an issue for them. Abstractions are issue only for some people.
> I think that second line probably took longer to write than it did for me to read.
If it takes more time to read then write, then it is not done yet needs to be refactored. Good code is as obvious to read as possible.
To take an analogy, it is similar to saying that we should use small words when trying to communicate. But once you understand the meaning of the more complex words, it is more fluent to communicate in terms of the longer words which represent the abstract concept more concisely and correctly. Of course it actually holds true to some extent in case of spoken languages, but that is because your intended audience is not expected to understand all the words in the dictionary. But the programmer who is reading the code is expected to understand the whole dictionary of python, including all the keywords, constructs and idioms.
It gave me some thought, but ultimately I think don't think it is a correct argument.
To take your analogy to its conclusion, we don't want to communicate only using small words, true, but we don't want to communicate only using big words, either.
Ain't nobody (= most people) got time for Hegel :)
I suspect that's why some of the very "expressive", i.e. dense languages, aren't widely adopted.
When I see the error-handling examples, I think that the problem with this kind of "boilerplate" is that it's easily forgotten. This results in errors being eaten and that's bad. But I agree with the main point that "proportionality" of code is a good thing.
It seems he have been writing code so clever and compact that he didn't dare to change it when it came to refactoring time:
> I’d take any opportunity to write terse code. It feels good to be clever
> I noticed that when I wrote Python code, I would often go out of my way to avoid messing up a beautiful terse set of lines by moving logic to places other than where it might naturally belong.
I can see how a less expressive language might help with this problem, but I think a better solution would be to rein in the cleverness and focus on readability rather than shortness.
I think it's related to the iron law of abstractions. It's easier to change code that has no abstraction than the wrong abstraction. The right abstraction is better than both those, of course, but is often difficult to identify upfront.
Succinctly put. I have observed a tangentially related correlation:
The code that needs least understanding to change will be the code changed, until it isn't the code needing the least understanding any more.
In unison they sort of tells us why pragmatism is often good when we start writing code, but how we must always try to push towards finding (good) abstractions as the work progress.
https://vimeo.com/9270320 (Greg Wilson - What We Actually Know About Software Development, and Why We Believe It’s True).
It seems we have scientific data showing that the number of code lines one can review per day doesn't depend on the programming language.
This means one would spend a lot more time reviewing this C++ code if you looking at the manually-rewritten-to-assembly version instead.
This is a very good argument supporting programming at the highest possible level of abstraction, which is the exact opposite of boilerplate and verbosity.
So please let's not rationalize/stockholm-syndrome over the limitations of any programming language ; boilerplace and verbosity are always bad.
From the maintenance perspective I would prefer the more verbose code in most cases, because it is easier to debug. Moving the verbose code block to a method or function for reuse will fulfill the DRY principle as well. Then you can decide during debugging to step over or step into the method. Doing to much in one step would also contradict another rule: the single responsibility principle.
A counterpoint is that more declarative code can often remove the need for tedious step by step debugging because the behavior of the code is more transparent. I do most of my work in Haskell and find I don't typically need to debug it like I would for a less declarative language because the meaning of the code is much more clear.
I come from a C# background and have little experience with Haskell, so YMMV. C# got many new features over the years like LINQ or Lambda Expressions. My experience is that you get most of them if used in a moderate way. Like splitting complex LINQ into smaller parts and introduce intermediate results instead of one big query. They are also easier to test this way. Another important aspect of this topic is if you are maintaining your own code or code others have written. I used to maintain code written by another team, which only did feature development and barely any code maintenance. While doing only code maintenance on code written by others for a longer period I gained a new perspective on what's good code in the long term.
This is a hilariously bad take. If we follow the argument to its natural conclusion then go is itself a poor choice of language. After all, go removes memory management boilerplate in favor of an automated gc system. By the author's argument go should force you to manage your memory yourself. In fact, we really should all just code in assembly because it has the clearest relationship between code and what is going on under the hood. True, coding in assembly requires a lot of needless boilerplate but by this article's argument that's actually a feature.
Go has so many truly bad design decisions (lack of generics being the most damning) and it's sad to see its fanboys try to argue that they're actually good.
All of the examples are far better than the code I normally deal with.
That being said, the most tiresome code is this kind of step-by-step error checking.
if (! is_numeric($id)) {
exit('The ID is not a number.');
}
$db = open_database();
if (! $db) {
exit('There was an error connecting to the database.');
}
$query = db_query($db, 'select * from widgets where id = $1', $id);
if (! $query) {
$db_error = get_db_error_msg($db);
exit('There was something wrong with the query: ' . $db_error);
}
Zzzzzzz. It triples the code size. One third is the normal code path. Two thirds is all the error checking. I'm sorry, I just don't write it:
$db = open_database();
$query = db_query($db, 'select * from widgets where id = $1', $id);
I let errors bubble up to some general error handler, which stops the script in its tracks and gives the user a vague and unhelpful error message. It doesn't matter. Even if I gave them a very specific error message, all that they would do is log a ticket, and I would still go into the server log to uncover the line number the error was on, debug, and fix it.
Don't get me wrong. I write JavaScript to constrain user input, with friendly directions and error messages. And I set tight types in the database and add database constraints where I can. But I don't write all this step-by-step checking in the middle layer (PHP, Python, Go, what have you). So if the user somehow bypasses the JavaScript and tries to insert text into an integer column, the database will simply refuse, and the resulting web page will be a very ugly error message, which is what they deserve.
If the database is down, it's going to be ugly, regardless.
I admit that this strategy works okay for non-life-threatening, SQL-backed web apps and such. You may need to do something tighter in your work. But I wonder if there would still be something more elegant than the every-other-line error checking shown in the article.
The catch-all top level error handler isn't typically a hallmark of good design but I think it's pretty relatable as a quick fix when you need to get something working on a deadline.
This is one of the situations where the difference between scripting and "real" development becomes obvious.
Yes, it makes you zzzZZZz. If you don't care for errors anyway. However, when developing a large reliable software project you need to think through the possible error cases and handle them appropriately. Even if that just means logging them correctly and exiting so the error can be easily tracked down later (for example: was it expected/unexpected? What where the parts of the context that can't easily be collected by a runtime?).
And in a large project, it's not that bad. How many places are there where you setup the database connection, an OpenGL context, a network connection, etc? Just build the abstractions appropriate for your project once. Simply crashing backwards through the function calls with a generic exception and a stacktrace is not always good enough.
Given that software systems are at their heart gate configurations on silicon, and given that those gates need to interact with mechanical and electrical systems, there will always be different classes of recoverable error for which the recovery procedure differs.
So yes, outside of pure functions you will always need special-snowflake error handling.
You only write it once in your Database class. The only thing that changes from call to call is the query string and parameters. Handle all errors gracefully there, throw an error to the calling code and let it handle it to the user with a 500 while the support channel gets alerted with the real error.
My dogma around this sort of pattern is similar, except that I don't just use one global error catching statement.
Basically, every module or function or whatever has a client. Sometimes those clients are other parts of your own system, sometimes they're end users outside of your control. Regardless, at significant barriers of abstraction, I try to catch errors, and translate them into new error types/representations that would be relevant or actionable by the intended client of that code, with the original error provided as a member. If you follow this pattern, the logic you write is free of error handling code, error handling code is centralized at abstraction barriers, and at any point you should have few errors that you wouldn't know how to respond to and probably just give up and bubble to the top.
But I wonder if there would still be something more elegant than the every-other-line error checking shown in the article.
Factor out the error checking into a function:
onerrexit(is_numeric($id), 'The ID is not a number.');
onerrexit($db = open_database(), 'There was an error connecting to the database.');
...
I admit you still have to explicitly state you're checking, but that's not much overhead, makes the code somewhat clearer, and gets rid of all the if(...). If the error-action is more involved, like your "third paragraph", then you can make an errcheck that takes a function reference for the error-action instead (not sure if that's possible --- haven't used PHP in a long time --- but there's probably a similar method otherwise.)
Sometimes when I'm being very lazy writing C, I define a macro of this kind which doesn't even take any error message argument but just prints line number, textual representation of the code which failed and errno. Huge debugging aid at negligible cost.
Oh, well, Apache and PHP give me this out of the box. So even though there is not a great deal of custom error checking in my code as I said before, I always have enough information in Apache's logs to track down the problem. I guess with C and Go you have to roll your own.
I think most of the complaints about the python version go away when you do something like this....
is_even = lambda n: n % 2 == 0
score_result = lambda j, k: j
is_interesting_result = lambda j, k: is_even(j) and k
a = some_variable + 42
b = sum(map(score_result, filter(is_interesting_result, results)))
or
b = sum(j for (j, k) in results if is_even(j) and k)
List comprehensions only become difficult when you start doing conditional execution within the result side. Use if for filtering or turn your complex operation into a meaningfully named function/lambda expression.
The first one is horrible and super un-pythonic (it even violates PEP8[1]). But perhaps even more important, it doesn't even work since a map-function can only take one argument (not two).
Using functional programming paradigms like this often advised against in Python.
> The first one is horrible and super un-pythonic (it even violates PEP8[1]).
I'm not really concerned with PEP. I'm concerned with if the idea being expressed is cleaner then the original. My first iteration was cleaner then the original comprehension (despite the comprehension being PEP8). It also let me see what I was really doing and I was able to turn my map and filter into a much nicer list comprehension.
Also, pointing out lambdas instead of defs is silly. It was done for an example. Most of the time you'll be operating on data structures with built in functions or supporting functions you will have written making it cleaner to use the map and filter paradigms.
money = sum(user.get_payed_balance() for user in users if
user.has_paid_bills() and
user.is_still_subscribed())
I much prefer the filters to the list comprehension in this case. I'll think "User's money for user in users if user has paid their bill and subscribed". I'm going to think "Sum the get_payed_balance for every user who is_still_subscribed and who has_paid_bills" for the functional implementation. If I had complex tuple arraignments then (like the OP's post) then I agree LCs will be the better way forward. If you're dealing with objects or functional-fitting problems I prefer map and filter.
> But perhaps even more important, it doesn't even work since a map-function can only take one argument (not two).
The concept still stands. Just use `from itertools import starmap`. Starmap is like map that calls the * operator on arguments.
>>> a = lambda a, b: 10
>>> list(map(a, ((1, 2), (3, 4))))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: <lambda>() missing 1 required positional argument: 'b'
>>> list(starmap(a, ((1, 2), (3, 4))))
[10, 10]
> Using functional programming paradigms like this often advised against in Python.
I'm not concerned with if it's "Advised against". I'm concerned with a) will someone be able to understand this better then the implementation that currently exists) and b) will I be able to come back in 10 years and tell what this is doing if need be.
You should check out this talk [1] before going around trying to use PEP8's inconsequential clauses to call something "horrible" code.
The proportionality argument is bunk, straw-man level bunk. Consider what goes on every time go accesses an object, casts to an interface or writes to a channel. Huge, complex code under the covers, way more than the Python examples he's given. What this article really says is "There are abstractions I'm not as comfortable processing as others, therefore those abstractions are bad."
If having code be proportional to complexity was a good idea, we'd all be writing in assembly and subroutines would be frowned on.
I have a Lua module I wrote that allow me to do stuff like:
process.limits.soft.cpu = "4m" -- limit to 4 minutes of CPU time
process.limits.soft.core = 0 -- and no core file
Yes, ultimately, it ends up in setrlimit(), but the amount of code that goes into backing this up is a bit daunting. But it more than makes up for in showing the intent (and it's not like this is done in the hot path of a program).
I think having code be proportional to 'complexity' is good, but having it be proportional to 'work' makes no sense. Personally I find things 'simple' when they only require one level of abstraction, such as accessing an object, or performing arithmetic operations on algebraic objects (e.g. multiplying matrices). And complex when they require more (e.g. manipulating the coefficients of two matrices to define their matrix product).
Boilerplate is when you requires more code than the complexity requires, usually because the language doesn't provide a sufficient opportunity for abstraction.
Note that, if we define a '->' operator that roughly expands to the following:
(x, err) -> f :
if (err != nil) {
return nil, err
}
return f(x);
Then the entire code he wrote as an example of boilerplate can be written as
DoSomething -> SomethingElse -> MoreWork;
with some minor differences that could be optimized away in the right language.
I always write terse code, not only because it reduces the number of lines (which is important to me) but also because:
- You clearly state the atomicity, that this thing happening in 1 long line is specific thought, intent, desire - it shouldn't really be on multiple lines - its easier to remove/comment out intent (as it is always self contained), replace it, optionalize it and reuse it.
- Scrolling the code is damaging to the focus.
- Once you learned the specific methods used to create terse code seeing it repeated all over again you will become more proficient in reading the code all around.
- Its way more fun. Its like poetry in programming - why would we all write dummy boring stories. Language is everything (obligatory babel 17 mention: https://en.wikipedia.org/wiki/Babel-17)
Yes but it is avoidable if you use a Lisp. Every part of the code can be compacted into a smaller form. I know when I write in Clojure it's always the most concise code I've ever written, with minimal effort too.
This is true in Elixir as well, which has hygienic macros very much like a Lisp. There's a balance, of course, but it really is quite nice to be able to just rewrite the AST as you need. And one must always remember the adage data > functions > macros.
130 comments
[ 3.2 ms ] story [ 201 ms ] threadIf you abstract things but don't give meaningful enough names, you loose everything (because your callers are still forced to lookup the definitions of your abstractions, and now this definition can be hard to search for).
This point makes sense, however I think I like to have my cake and eat it too. Java 8's stream constructions can be relatively terse, but if you need to switch back to a for loop, IntelliJ has nifty right-click menu items for doing so. I think the argument made by TFA is at least related to John Carmack's take on inlining vs. abstraction: http://number-none.com/blow/john_carmack_on_inlined_code.htm...
It's very unfashionable advice in some circles, though. It sounds just enough like "go back to writing unstructured copy-paste spaghetti" to trigger some people's kneejerk. But the ground rules of "don't take subroutines, don't jump backwards" actually keep it from getting too far out of control, because they align you towards code that only uses enough flexibility for the task at hand, hence there's a lower likelihood of missing something. Very good practice for greenfield code where you might reflexively add a generalization that you don't use. I can still get better at it.
Edit: And one of the most important things it taught me, is that I can use scope blocks and a comment instead of a function, and it'll be sufficient.
is what i'd call it.
That's not what boilerplate code is, and DRY is not the way you get rid of boilerplate (the article claims it is a part of the solution).
boilerplate is a repeated preamble that's necessary and unchanging in order to get down to the business of actual logic.
An easy example of boilerplate would be include guards in a C/C++ header file, or 'use strict' in perl.
For example, the author says this:
> When I write Go, I noticed that my tweaks almost always go exactly where they naturally belong. My boilerplate doesn’t stay untouched!
If you're "touching your boilerplate" while writing code that affects the logic of your system, then it isn't boilerplate.
Those examples are just clear indications that languages sometimes have really terrible default behaviors. Namely, Perl should have been strict by default (with a "no strict" if this didn’t perform well enough), and C++ should have at least added #import.
> Maybe this is subjective but I posit that the code’s shape looks more like what it is.
Indentation isn't merely stylistic or syntactic, but it tells us useful things about code at these lower levels. A more complex code shape should convey more complex logic, and we should want that complexity apparent to us...at least sometimes. It looks great to have a nice, terse one-liner, but visually flattening possible control flow paths feels really nasty.
Objective thinkers tend to go looking for existing, established boilerplate or frameworks because of the immediate leverage that is provided. They may think of too much subjective thought or originality as ridiculous NIH or wheel-reinvention. Their struggle is in the weighty task of grokking and maintaining third-party resources.
Scott Adams has kind of unwittingly uncovered this dichotomy with his own inventive, subjective thinking. He considers more objective thinkers as "word thinkers" or some name like that, because they go looking up the established names of things rather than understanding the things themselves.
Also Chris Langan similarly trashed Jeopardy contestants in a news interview, because they aren't subjective thinkers like he is; their minds are just filled with objective facts about various things. Well, true for him; his bias dictated the way he developed his own unique universal model--he prefers and has a gift for subjective thought.
Really though, we need both kinds of approaches.
The thing with code is that you can take those existing patterns and make them a first-class concept in your program. You're still in "objectively thinking" - still reusing existing code - just in a more compact and safe way.
And words are amazing. All these complex and nuanced concepts we're talking about can be expressed and shared almost instantly because of the words we share. If we had to invent words as we communicated, or worse yet, had no words, there would be no discourse because the sun would go down before we got anywhere.
It is true that words are set in their definitions and intuitions, much in the way that scientific models are and common sense is. These can be considered objective facts.
But challenging these facts and inventing new words is just an important part of the process. Some have more success with it than others, but not looking up what is already known is just wasting your time. Same with not using words that already mean what you mean.
Subjective thought leads to objective facts with fact checking. Objective facts are then better shared, and better learned, because learning is far easier than arriving at them yourself. Learning just takes communication, which is proportional to the complexity of the information. Inventing or discovering new facts requires the entire lengthy and laborious process which is will inevitably include experiments and fact checking and evidence validation if we are to be scientific. If the problem can be solved with pure thought, chances are it was a philosophical one, although philosophy too highly relies on existing language.
Generalisation is always[tm] a bad idea. Taken to illogical extremes, at the other end of the spectrum we have the monstrocities of old Perl. After all, with Perl it was often said that you could fit most solutions to a oneliner.
In fact, I would go as far as to claim that using Python for n-tuple iterators inside branched comprehensions are getting awfully close to the worst aspects of what Perl scripts were commonly criticised for.
1. RESTful routes leading to implied controller actions that render a view without any controller declaration at all. I've seen more than one codebase with routes that could be invoked by users despite being never linked in production and having no test coverage. It's basically undocumented code due to a total absence of declaration, and a case of needing to know the implied framework behaviour to understand a program.
2. I have an issue with ActiveRecord's schema behaviour. By avoiding attribute declaration in models and just defining ORM methods by asking the database for column names, it only takes one poorly handled migration or a DBA oops to lead to some very strange production behaviour. Situation is only partially offset via the (new) attributes API and/or a schema cache dump during CI but one can never fully do away with the hazard. Philosophically speaking, sometimes Rails thinks it is dictating the database schema, sometimes Rails accepts what the database tells it, and out-of-the-box rails has no model boilerplate that bridges the gap to guarantee a specific DB schema.
In both cases I add my own safeguards for production environments, but that's a matter of programmer discipline rather than mandatory declaration.
Boilerplate is usually maximized in projects which try to give users a huge amount of flexibility that they don't need or want.
This is partly why minimizing boilerplate is so hard - one thing that continually surprises me is what users want flexibility over and what they don't. I've lost count of the number of times I've given users flexibility to do something they don't give a shit about and not given them the flexibility to do something they really wanted.
That's a great way to put it. I'll steal this one.
I tend to use this pattern quite a lot:
I still miss a similar feature in Emacs and Vim. There are similar folding markers one could use (say {{{ }}}), but they don't seem to work together with syntax-based folding.I already pondered if it wouldn't be possible to merge multiple "folding strategies" if one would build some kind of interface with which multiple routines could basically communicate folds in a buffer.
Say you have one routine scanning the syntax of the according language, one looking for folding markers and one for manual (Vim-style) folds. I guess it technically should be trivial to use all of them provided they are nested and don't intersect in weird ways (the latter could be resolved with some simple priorities).
Simply put, knowing how advanced org-mode can get with folds, this is likely trivial to do. (If you haven't played with it, I'd recommend taking a look. I don't have perfect examples, but http://taeric.github.io/DancingLinks.org is an example. Open that in emacs with org-mode and you will see that all of the source blocks can be folded together. For me, the source blocks are correctly colored, as well.)
Quickly searching, I found https://www.emacswiki.org/emacs/FoldingMode, which seems to be a large start to this. There were also some other ideas that seem to get close.
The basic idea I had was to just make a few functions that would:
The second of those felt trivial, since it is basically M-x occur #region\|#endregion. On top of that, I just need to learn how to actually make overlays and then the rest feels like it would fall into place easily enough.Edit: It occurs to me that I didn't show how to actually hide text, which I think is your question. Fastest way I can see is this (run this in your scratch buffer to see it in action):
To easily delete this particular overlay, if you make it, use: So, the basic task that will need to be completed is finding the point position for the start and end of make-overlay.// Find the first foo with bar foos.map(f => f.bar).uniq().orderBy(f => f.baz)[0]
does the same basic thing, the ability to collapse and display intent is incredibly rad for lack of a better word
In C++, would you write the same structure using a single line for-loop and ternary operators?
The first is that somebody else will eventually have to work with it, decipher what the heck I was trying to achieve, and generally get tripped up and slowed down by it.
The second is that eventually I may have to work with it again, and sit there scratching my head and wondering what I'd been smoking when I wrote instead of focussing on the changes I need to make.
The point is to code for maximum clarity to humans whilst still achieving the desired functional and non-functional outcomes on the machine. You will use varying terseness as part of doing this because terseness isn't the point: it's just one of many means to your end.
It's not "cleverness", but rather the ability to express the intent as succinctly and clearly as possible. Higher level languages allow you to do that, lower level languages don't. Obviously, Go is not a very high level language because it forces you to write explicit loops instead of using comprehensions or functional algorithms. Loop is a primitive concept. All other factors being equal (i.e. performance) you should always prefer comprehensions or algorithms over explicit loops.
The author could have brought the examples forward, instead of just providing links to urllib3 and the Set type.
You can iteratively change a smaller, terser code, must faster. There's a reason prototypes are often written in languages like Python.
I agree with you, but seeing the number of proponents of Enterprise Java there are, it's clear that there are a significant number of programmers who prefer verbosity and extra abstractions --- although IMHO not necessarily because it's actually better.
In fact a trend I'm seeing with a lot of programming languages is more verbosity and unnecessary abstraction, leading me to wonder whether programmers actually know the languages they're using or are barely scraping by with the very minimal basics (and in the case of some languages, a bit of OOP cargo-cult dogmatism.)
Although I'm not really familiar with Python, and the order of the clauses its conditional expression syntax is unusual and a little surprising, I think that second line probably took longer to write than it did for me to read.
Even Enterprise Java wasn't verbose for the sake of it (I say "wasn't" because modern enterprise Java can be more like modern functional style than Go).
It was more verbose because it added tons of options and flexibility while keeping with the OO style, so everything was pluggable and configurable and factorable. But that's something else than Go style verbosity.
That's the point. Reading is far more important.
Say, with Scala implicit parameters, you get to remove code from call-site. Does it make the code better? Depends on the case, in my opinion. So I don't get the point to say some people categorically want verbosity. They may prefer explicitness and unsurprising code which may require verbosity. Or perhaps the removal of verbosity requires code that allocates more memory -- in that regard Kotlin seems to be a good citizen. It has ways to make code concise without making the byte code entirely different.
It is more that they have good abstract thinking, so abstraction is not an issue for them. Abstractions are issue only for some people.
> I think that second line probably took longer to write than it did for me to read.
If it takes more time to read then write, then it is not done yet needs to be refactored. Good code is as obvious to read as possible.
It gave me some thought, but ultimately I think don't think it is a correct argument.
Ain't nobody (= most people) got time for Hegel :)
I suspect that's why some of the very "expressive", i.e. dense languages, aren't widely adopted.
> I’d take any opportunity to write terse code. It feels good to be clever
> I noticed that when I wrote Python code, I would often go out of my way to avoid messing up a beautiful terse set of lines by moving logic to places other than where it might naturally belong.
I can see how a less expressive language might help with this problem, but I think a better solution would be to rein in the cleverness and focus on readability rather than shortness.
In unison they sort of tells us why pragmatism is often good when we start writing code, but how we must always try to push towards finding (good) abstractions as the work progress.
It seems we have scientific data showing that the number of code lines one can review per day doesn't depend on the programming language. This means one would spend a lot more time reviewing this C++ code if you looking at the manually-rewritten-to-assembly version instead.
This is a very good argument supporting programming at the highest possible level of abstraction, which is the exact opposite of boilerplate and verbosity.
So please let's not rationalize/stockholm-syndrome over the limitations of any programming language ; boilerplace and verbosity are always bad.
Go has so many truly bad design decisions (lack of generics being the most damning) and it's sad to see its fanboys try to argue that they're actually good.
That being said, the most tiresome code is this kind of step-by-step error checking.
Zzzzzzz. It triples the code size. One third is the normal code path. Two thirds is all the error checking. I'm sorry, I just don't write it: I let errors bubble up to some general error handler, which stops the script in its tracks and gives the user a vague and unhelpful error message. It doesn't matter. Even if I gave them a very specific error message, all that they would do is log a ticket, and I would still go into the server log to uncover the line number the error was on, debug, and fix it.Don't get me wrong. I write JavaScript to constrain user input, with friendly directions and error messages. And I set tight types in the database and add database constraints where I can. But I don't write all this step-by-step checking in the middle layer (PHP, Python, Go, what have you). So if the user somehow bypasses the JavaScript and tries to insert text into an integer column, the database will simply refuse, and the resulting web page will be a very ugly error message, which is what they deserve.
If the database is down, it's going to be ugly, regardless.
I admit that this strategy works okay for non-life-threatening, SQL-backed web apps and such. You may need to do something tighter in your work. But I wonder if there would still be something more elegant than the every-other-line error checking shown in the article.
Yes, it makes you zzzZZZz. If you don't care for errors anyway. However, when developing a large reliable software project you need to think through the possible error cases and handle them appropriately. Even if that just means logging them correctly and exiting so the error can be easily tracked down later (for example: was it expected/unexpected? What where the parts of the context that can't easily be collected by a runtime?).
And in a large project, it's not that bad. How many places are there where you setup the database connection, an OpenGL context, a network connection, etc? Just build the abstractions appropriate for your project once. Simply crashing backwards through the function calls with a generic exception and a stacktrace is not always good enough.
So yes, outside of pure functions you will always need special-snowflake error handling.
Basically, every module or function or whatever has a client. Sometimes those clients are other parts of your own system, sometimes they're end users outside of your control. Regardless, at significant barriers of abstraction, I try to catch errors, and translate them into new error types/representations that would be relevant or actionable by the intended client of that code, with the original error provided as a member. If you follow this pattern, the logic you write is free of error handling code, error handling code is centralized at abstraction barriers, and at any point you should have few errors that you wouldn't know how to respond to and probably just give up and bubble to the top.
Factor out the error checking into a function:
I admit you still have to explicitly state you're checking, but that's not much overhead, makes the code somewhat clearer, and gets rid of all the if(...). If the error-action is more involved, like your "third paragraph", then you can make an errcheck that takes a function reference for the error-action instead (not sure if that's possible --- haven't used PHP in a long time --- but there's probably a similar method otherwise.)Using functional programming paradigms like this often advised against in Python.
[1] https://www.python.org/dev/peps/pep-0008/#programming-recomm... (search for "Always use a def statement")
I'm not really concerned with PEP. I'm concerned with if the idea being expressed is cleaner then the original. My first iteration was cleaner then the original comprehension (despite the comprehension being PEP8). It also let me see what I was really doing and I was able to turn my map and filter into a much nicer list comprehension.
Also, pointing out lambdas instead of defs is silly. It was done for an example. Most of the time you'll be operating on data structures with built in functions or supporting functions you will have written making it cleaner to use the map and filter paradigms.
bs I much prefer the filters to the list comprehension in this case. I'll think "User's money for user in users if user has paid their bill and subscribed". I'm going to think "Sum the get_payed_balance for every user who is_still_subscribed and who has_paid_bills" for the functional implementation. If I had complex tuple arraignments then (like the OP's post) then I agree LCs will be the better way forward. If you're dealing with objects or functional-fitting problems I prefer map and filter.> But perhaps even more important, it doesn't even work since a map-function can only take one argument (not two).
The concept still stands. Just use `from itertools import starmap`. Starmap is like map that calls the * operator on arguments.
> Using functional programming paradigms like this often advised against in Python.I'm not concerned with if it's "Advised against". I'm concerned with a) will someone be able to understand this better then the implementation that currently exists) and b) will I be able to come back in 10 years and tell what this is doing if need be.
You should check out this talk [1] before going around trying to use PEP8's inconsequential clauses to call something "horrible" code.
[1] - https://www.youtube.com/watch?v=wf-BqAjZb8M
If having code be proportional to complexity was a good idea, we'd all be writing in assembly and subroutines would be frowned on.
Boilerplate is when you requires more code than the complexity requires, usually because the language doesn't provide a sufficient opportunity for abstraction.
Note that, if we define a '->' operator that roughly expands to the following:
Then the entire code he wrote as an example of boilerplate can be written as with some minor differences that could be optimized away in the right language.This is the design philosophy of Go, as far as I can tell.
- You clearly state the atomicity, that this thing happening in 1 long line is specific thought, intent, desire - it shouldn't really be on multiple lines - its easier to remove/comment out intent (as it is always self contained), replace it, optionalize it and reuse it.
- Scrolling the code is damaging to the focus.
- Once you learned the specific methods used to create terse code seeing it repeated all over again you will become more proficient in reading the code all around.
- Its way more fun. Its like poetry in programming - why would we all write dummy boring stories. Language is everything (obligatory babel 17 mention: https://en.wikipedia.org/wiki/Babel-17)