I've been using this for a couple of weeks and its a been really nice to remove the constant mental overhead of structuring code. it also makes you think about structuring functional blocks into more manageable, bite-sized chunks which is easier to understand. 10/10 would recommend
I use black every day. It's pretty good. The hours you save on pointless arguments e.g. ' or " is better as default for strings. I wish it'd come with standard python distribution `python fmt`
yeah, someone above mentioned that with Go as the example. It's pretty nifty in some cases but think there are two sides to the story. I know folks who write lovely python code everytime. I know others at my gig who even after 3 years are not putting spaces after equals sign (and other PEP/flake8 blunders) and every PR is littered with syntactic errors. The code formatter is brilliant for this case.
Nothing stops a reviewer from reformatting the code before reading it (probably using an autoformatter).
But I'm not sure if making the code easier -- and thus faster -- to read is necessarily a good thing. If the brain parses and analyzes the code in parallel, making it harder to parse could give the analyzing process more time and thus make the review more thorough.
Of course, nothing stops a reviewer from being thorough either way. But making it the path of least resistance makes it more likely.
Agreed. yapf has soo many config options, and I want to say some of those options are heuristics (like `SPLIT_PENALTY_AFTER_OPENING_BRACKET`) which can have some "unpredictable" side effects (at least to the eyes of a new user).
I prefer Black, which has minimal configuration. There's no debate to be had over details that don't matter much in the grand scheme of things. Yeah, maybe it doesn't always look _just_ how I'd like, but I get used it.
And while Black may not be 100% compatible with flake8 out of the box, they are aware of flake8 and document places where it may not be flake8-compatible.
This is a case where I'm completely decided. Everybody should use an autoformatter. The minimal benefit you get from custom formatting is completely outweighed by the uniformity, the consistency and readability of autoformatted code.
I also switched to mandatory clang-format for a big C project of mine last month. Just the PR hooks and linter integration into github not yet. Saves away a full round of nit picking.
I think a nontrivial portion of Go’s success can be attributed to having a standard format and treating deviation as compiler errors.
It is immensely valuable to be able to look at any Go code, whether in the toolchain, the standard library, or a random stackoverflow snippet, and not have to think about formatting at all.
I think you're conflating two concepts a little bit - deviation from `gofmt` isn't a compiler error, but lots of things (like unused variables or imports) are. I think they're both great.
You are corrrect. I guess I’ve learned to treat deviation from gofmt as error because of the go vim plugin. It runs goimports on save, which fixes a bunch of real errors (like unused imports).
That's not the case at all. The Go compiler treats an arbitrary few lint issues as compiler errors (e.g. unused imports), code which is not gofmt-formatted isn't a compiler error.
We have been using this for almost 7-8 months and even though I don't personally like the style of it very much, it has helped keep everything consistent. You can set it up as a pre-commit hook to automatically format the code too
How do you enforce the use of a formatter in a project -- pre-commit hook? pre-receive hook? both? And is there a convention to put a specific dotfile at the project root to hint at your IDE/editor that it should use a certain tool for automatic formatting?
We are using a pre-commit hook but I also have an on-save hook in my editor. Further, we also have a CI job that runs `black --check` which returns 1 (failing the build) if the files weren't completely formatted with black. In Go, the convention is that everyone's editor runs gofmt on change, and because the convention is so strong, that's usually enough (although many larger projects also have a CI job similar to the one described above).
Having used both extensively, my biggest grievances (in order) are that there aren't more editors that with good on-save support, that black is relatively slow (compared to gofmt), and that black isn't more opinionated and less configurable.
One, that shouldn't matter on CI anyway. Two, you can tell black to just print the difference with --diff and tell it to just check the files with --check.
The first one shows you what's different (so you have logs in your build job), the second one fails your job if there are any diffs. You could just do the second one if you're a fan of the whole brevity thing.
We use pre-commit.com – configure your project with dotfiles for various tools so things like editor auto-format & lint, pre-commit, etc. can pick them up, and then have a Git pre-commit hook which is also run by your CI tool for consistency.
Neat. I've been using an autopep8 pre-commit hook for a while, even though I kinda hate it (but nobody complains about noncompliant code anymore). I'll have to give this one a shot
I've been using it for a few months now, but I don't really like it. It does remove quabbles about formatting, but I've personally never felt that as a problem and it just replaces them with constant frustration.
I think code formatting is very important for readability, and I think many (subtle) choices about how to make a piece of code more readable are very subjective. These types of tools are just incapable to make those choices because they have no concept of intention.
And some rules make this painfully obvious even in less subjective cases. Eg: black formats dictionaries into a single line if they fit one line, which makes nested structures unreadable if they combine bigger and smaller sub-dictionaries.
In the end, I don't have problems with other peoples' formatting choices; as long as they are sufficiently consistent within their own style, which they usually are, I can live just fine with them even when they go against how I'd do things.
I do have problems with tools that reduce code readability for consistency's sake and argue that these provide little benefit, and may even be detrimental to code quality.
No, because I believe this is fundamentally unfixable. It's not a matter of changing this or that behavior, it's a matter of (apparent) formatting inconsistencies being important to convey intention and distinguish more important from less important bits.
In the example I mentioned, I may sometimes choose to put a small dictionary initialization into a single line if it's just a detail, or may split it into multiple lines if it's important enough for someone reading the code to pay attention to it before going further.
This simply cannot be decided by an automatic formatter without it knowing the code's purpose which, for now, requires a human brain.
Someone mentioned elsewhere that automatic formatters remove the cognitive load of formatting code, but I argue that load is an intrinsic part of programming, because it is (for the most part) the effort of communicating to the next person that comes along.
I write a lot of scientific code with python and in those cases you really want your equations and matrix columns aligned for readability. These matrices appear so often adding fmt:off destroys the readability/elegance gain from the clever formatting.
Here's an example snippet by Peter Norvig that is beautifully formatted:
def neighbors4(point):
"The four neighboring squares."
x, y = point
return ( (x, y-1),
(x-1, y), (x+1, y),
(x, y+1))
def neighbors8(point):
"The eight neighboring squares."
x, y = point
return ((x-1, y-1), (x, y-1), (x+1, y-1),
(x-1, y), (x+1, y),
(x-1, y+1), (x, y+1), (x+1, y+1))
Well sure, but an automatic formatter can still be better or worse. Maybe it won't be as good as hand-crafted formatting, but more bad examples can be avoided.
You could configure your IDE to re-format files into your favorite format and just have a pre-commit hook that formats them back into the official format. That's how I work. No arguments about style and formatting, yet I get to make the code look how I personally prefer it.
There is no such thing as "my favorite format" in that sense. Also, I don't mind reading code in other people's "favorite format" nor do I have any problem in following another person's style if that's the style agreed upon by the team.
No, we can't. My (and, by the sound of it, CrLf's) favorite format relies on information that your pre-commit hook has artificially removed from the code. Eg:
(Most cases are more subtle (and thus less amenable to "oh, you just need to build a complete static type checker into the formatter") than this, but I wanted an obvious example.)
It seems like the signature of the functions should be different, especially the second example. I would have written the cmplt function to take tuple pairs:
cmplt(
(type_a, value_a),
(type_b, value_b),
)
That's much more clear about the relationship between each pair of values either way, and would get formatted nicely by Black.
Sorry, should have clarified: my example was written in C[0][1]; cmplt is something like (Dtyp* ,Dval* ,Dtyp* ,Dval* ), which is what I was alluding to with "just need a type checker". Also, in the code I anonymised that from, cmplt and munge are imported (well, #included) functions from a different library.
0: which doesn't have tuples anyway; you could use structs, but it doesn't really work well in context/practice.
1: I don't use python frecently enough to have ready examples of autoformatter stupidity on hand for it.
> And some rules make this painfully obvious even in less subjective cases. Eg: black formats dictionaries into a single line if they fit one line, which makes nested structures unreadable if they combine bigger and smaller sub-dictionaries.
Prettier for JS has a simple solution to this specific problem, and that is to follow the code's lead. If the first element of the object or array starts on the same line as the opening bracket, keep it as a single line if it fits. Otherwise, if the first element starts on a new line, put every element on a new line (explode the collection), even if they would all fit on a single line.
Black should consider doing this instead of the open PR to explode collections if there is a trailing comma[0], which feels... wrong.
We use Black in a pre-commit hook for all our python code. It makes legibility and formatting a non-issue in the team. Yes it can look strange at first if you're not used to it, but after a while it makes formatting invisible (and it's odd reading non-black python). Would recommend.
how do you do this? do you have a jenkins server running that does the formatting? i would love to set this up because while i use black religiously i cannot for the life of me convince my boss get in the habit of using it (which is crazy because your commits are always fighting on whitespace).
Use pre-commit [0] so people can run it client side for quick feedback, and then use whatever CI you have (Jenkins, Travis, Circle, Azure) to enforce this. The important parts are the majority of people agree to do it, and that you have CI to be the "bad cop"/fall guy so it never becomes personal.
(edit: black has pre-commit support [1], and it's easier than setting up and maintaining your own hooks IMO)
I super dislike black's formatting, and I think it's really rare to actually see it in codebases. It wraps weirdly (sometimes not at all). I'd prefer to use yapf, but last I checked it still crashes on "f-strings".
Here's a small example:
basket.add({
apple.stem
for satchel in satchels
for apple in satchel
})
Black formats this as:
basket.add(
{
apple.stem
for satchel in satchels
for apple in satchel
}
)
I've never seen Python code like that.
I totally believe using a formatter is good practice. Black is in a challenging position of coming into a community with a lot of existing code and customs, and I get that. But I also think that's an opportunity, rather than having to guess at what is good, there's a wealth of prior art to look at. I wish it had done this, rather than essentially codify the author's style.
While I agree that the formatting choice looks a little weird. The primary benefit of a formatter is that the formatting is always the same not. It's less important that it matches everyone's preferences and more important that it always formats the code the same way.
It must be nice to work on a team where everyone agrees on the formatting. I don't think I've ever worked with a team where that was true. The exception being languages where the formatting was enforced by the language tooling itself. Those occasions no one even discussed it so I wouldn't exactly know if they agreed with it or not.
It's a very valid point. If there is a universal format, it should not do things so rare. The above example should be easy to address, check if the method argument is a single dictionary or multiple.
I completely agree that black does not format all examples in a nice manner, and your example is something I see pretty often. Whenever black adds too many indentation levels and line breaks in what should be a simple-enough statement, I simply refactor into multiple statements, e.g.
stems = {
apple.stem for satchel in satchels for apple in satchel
}
basket.add(stems)
I concede that without black it wouldn't have made sense to make that refactoring, but since I really want to use an autoformatter, and black is the best autoformatter I have, I'd rather restructure my code slightly so that I can just use black 100% all the time.
It's an interesting meta thought that using the autoformatter changes the way you write your code. Not that that's implicitly a good or a bad thing, I just find it interesting.
I think it’s under-appreciated when comparing tools like this. I’ve noticed that a bit since adopting Prettier and Black and one thing I’ve noticed during code review is that usually when I end up changing code structure to get better results from the tool, it’s a refactor that I probably would felt like I should have done but might have put off due to laziness even though it’s really not that much work.
I’m kind of thinking of it as something like baking some exercise into your day by putting a walk into your commute: the little nudge of not fighting the tool on something minor means you don’t skip little things, and over time that adds up more than it seems at first.
Oh that's interesting. I looked up the Issue [1] and it looks like it was an upstream bug, but also if you're using that version of Python (which, of course, we are) you'll still have that problem. Thanks for chiming in, you're gonna make my life a lot better haha.
if the progress of prettier (js) over the past few years is any indicator of what will happen with black, there will likely be incremental improvements in black that address the poor formatting cases you’re concerned about. i remember when prettier first came out, i was not convinced until my “standard” for formatting was met. but it was met eventually.
I think that's why it self-describes as an opinionated formatter vs YAPF where one has very granular control.
My understanding is that by using Black you're saying I choose to not express my opinions about formatting aesthetics and delegate that decision to Black instead.
I'm not a python person (for many years, at least), but I'd choose black for a completely different reason. By using Black, I'm choosing standardization over my whims.
Which, as a longtime Go dev and now Rust dev, I love formatters that are opinionated. I don't always love what Gofmt or Rustfmt do, but I definitely like consistency that the community has in code style.
So I don't care what Black thinks - I care what, hopefully, everyone uses. Though, I could see a situation where formatting can be altered and the project has a format config. Meaning that while code fmt differs between projects, it would be consistent among all devs working on X projects.
Our test pipeline fails any Python service trying to deploy if the black style check fails.
I was initially grumpy about my org adopting black because I preferred single quotes, but the level of standardization is a huge win in my book. I never even think about my code style anymore, I just write it and then run black.
I've always been a double-quoter, in contrast to most Python devs I know/work with. After we standardized on black, I got some satisfaction seeing all of my coworkers bend to my will (/s).
How narrow those two choices are! The correct answer is obviously to use single-quotes for keys and key-like cases and double-quotes for values. Make formatters impossible to write! (last sentence very /s, but that style actually exists)
You can disable string normalization with Black, and then continue using single quotes all day. But then, yeah, you give up enforced standardization on that.
That one actually makes a lot of sense to me, even if I've got to admit that I tend to go with the first option in my own code. It looks to me like a result of a couple rules that, in general, are sound: First, if an argument list can't fit all on one line, then every argument needs to go on a new line. And all the arguments need to be indented to the same level. The argument to that function includes the braces, so the opening one needs to go on a new line, and the closing one needs to be indented as well.
In this particular case, that may result in code that breaks with tradition. But I can't see a way to preserve the tradition without creating special or edge cases. For example, we can't follow the first option and get clean formatting with a function like zip. You'll have a train wreck of bad options about where to put the braces and how to place the comprehension's body in relation to the braces that enclose it. Versus, with Black's style, the answer is easy and straightforward, because you just do it the same way you would anywhere else:
zipped = zip(
{
apple.stem
for satchel in satchels
for apple in satchel
},
{
apple.core
for satchel in satchels
for apple in satchel
}
)
IMO, that's good, even if it isn't what we're all used to. Having a bunch of special cases just to match what someone might think is more aesthetically pleasing in specific situations is not a desirable feature in a set of autoformatting rules.
There was a proposal a few years ago to add array comprehensions to the language, but it was ultimately rejected in favour of using .map/.filter instead.
Not sure what came about of object comprehensions though.
That works if they're both comprehensions, but starts looking more gross when the 2nd argument isn't a comprehension, and now you're looking at deciding among options like
zipped = zip({
apple.stem
for satchel in satchels
for apple in satchel
},
someList
)
or
zipped = zip({
apple.stem
for satchel in satchels
for apple in satchel
}, someList)
(Which admittedly looks reasonably tidy, but starts to get gross again if we start looking at 3-ary functions.)
You've also got to contend with the first not being a comprehension meaning that the comprehension's indenting can't so easily be kept the same:
zipped = zip(
someList,
{
apple.stem
for satchel in satchels
for apple in satchel
}
)
Which is where I was going with the comment about edge cases. Personally, I don't want formatting rules where you might decide to format the arguments to a function in different ways depending on the specifics of what other arguments the function has. I like simple. Give me one rule for when it all fits on one line, and another rule for when it doesn't. And make sure neither of the rules causes me to have to re-indent things just because a function picked up an additional argument. And make sure that the rules are completely oblivious to the function's arity.
Honestly, I write my code like that. It could be because I write in other languages but I find that the right amount of indentation and vertical white space just makes the code infinitely better to read.
> I'd prefer to use yapf, but last I checked it still crashes on "f-strings".
There is a great autoformatter that does not have this issue. Using it guarantees that you'll never have to stick with an outdated version of a language because your autoformatter of choice hasn't been updated for the current one, or even worse, is not maintained anymore.
Because codebases are rarely even consistent amongst themselves. Getting good examples would be harder than just making the decisions yourself the way you like them. There aren't that many structures to consider.
> I wish it had done this, rather than essentially codify the author's style.
Yeah, I keep seeing people singing the praises of black and I'm really dreading the inevitable future where people start acting like this dude's opinion is Correct Official Python Style.
However the pandas source is not blackened, and I really miss not be able to just fix all formatting issues while working on a PR. After just a few months I am so used to black it just feels weird to not have it everywhere
This is a trend that I believe started with Go's code formatter, but such formatters differ in how much reformatting they will do.
Go's formatter adjusts whitespace within a line and will add or remove blank lines, but it doesn't change line breaks and has no opinion on the maximum length of a line. Other formatters work differently.
I often dislike autoformatter output too, but then I remember that while no-one likes what the autoformatter does to their code, everyone likes what the autoformatter does to their coworkers' code, and then I chill out about it.
Having a standard is more important than the standard being excellent.
Because you don't know if they've done the trick. That's why natural language is padded with extra words to add context and help rule out misunderstandings. The grammatical rules of agreement are one mechanism to do this.
And you can see the phenomenon of over-abbreviation by reading a few debates on twitter and repeatedly seeing, "how can you not understand what I tweeted?!"
I remove semicolons from JS b/c I feel like they are a bit of a hack. Except where it's required by the language (for loops, which are becoming less used quite rapidly), your code generally shouldn't be so complex as to require semicolons for readability.
Having said that, I've been introducing a lot more colons, since I use TypeScript.
I was referring to punctuation that is, in some way, actually necessary. No one ends their lines with semicolons in python although it's valid. The rules when the interpreter will "insert" them are obvious. Not so in Javascript.
They're not very important if you're the only developer on your team.
If you're working with others, though, then it becomes very important. I don't think I'm being entirely hyperbolic when I say that inconsistent or poorly-chosen formatting rules are the death of 1,000 cuts for a team's productivity.
There's a tiny but existent cost that's incurred every time formatting rules that aren't diff-stable result in a noisy code review that takes longer to read, or makes it harder for reviewers to discern the real changes from the formatting junk. There's a tiny but existent cost when excess delta makes it harder to gitblame. There's a tiny but existent cost when people have to stop and think about how to format their code manually. Or when they have to stop and debate formatting. Or when they read someone else's code slightly more slowly because different formatting rules make it harder for them to skim it or rely on pattern recognition instead of careful reading to understand its structure.
All those tiny little costs add up to something that's not so tiny. And it's so easy to make it just disappear, for the low low cost of swallowing one's pride, by simply adopting an opinionated autoformatter.
> There's a tiny but existent cost that's incurred every time formatting rules that aren't diff-stable result in a noisy code review that takes longer to read, or makes it harder for reviewers to discern the real changes from the formatting junk. There's a tiny but existent cost when excess delta makes it harder to gitblame.
Don't reformat code you didn't otherwise touch. That's just common sense. Common sense autoformatters lack.
> There's a tiny but existent cost when people have to stop and think about how to format their code manually.
I rarely think about how I format my code. When I do, it's because the code is hard to format in a readable way, in which case an autoformatter will produce garbage.
> Or when they have to stop and debate formatting.
"Doctor, it hurts when I do this."
> Or when they read someone else's code slightly more slowly because different formatting rules make it harder for them to skim it or rely on pattern recognition instead of careful reading to understand its structure.
Neglible. To the contrary, different formatting reminds you that you did not write this code and you should read it more carefully because you can't expect its creator to think the way you do.
I've never been able to wrap my head around people having strong opinions on style issues. I'll defer to whoever cares the most on the team and then just do that. When I look at the problems in code bases, rarely has "slightly inconsistent formatting" been at the top of the list.
I don't universally love what autoformatters do to code my coworkers wrote. I've worked with plenty of people who manage to do fewer weird formatting things than what mediocre autoformatters do.
My sense is that configurable autoformatters are doomed to suck.
Because an autoformatter with 10 simple yes/no options has 1024 different ways those options can interact, and an autoformatter with 20 yes/no options has 1,048,576 different ways that they can interact. It's simply not possible to make sure that you're going to get reliably good results in the face of that kind of combinatorial explosion.
Versus, if there's only one way that it will format things, then the people designing the rules have a single stationary target that they can aim at.
This isn't really the case, though. Lots of options are completely orthogonal (e.g. tabs vs spaces and function brace style don't really affect each other). My experience with clang-format has been that each option (mostly) only affects a single token type, which is determined using clang as a backend. I've only seen satisfactory results.
If you don't have as powerful a backend as the full clang paraer/lexer on the other hand, I could quickly see things breaking as you described.
We spend years peering at code hunting for tiny, miniscule mistakes. Thus we're training ourselves, quite rigorously, to spot minor deviations.
We're also irrational in the moment: our aesthetic sense is bothered by certain patterns, and our social sense wants to assign blame for this "wrongness" to individuals.
An auto-formatter removes a ton of deviations that don't matter, and desocializes the aesthetics.
This saves code reviewers time and stress and helps them focus on what actually matters in the code.
And it only has to save more time than it takes to run "pipenv run black" to be measurably worth it.
Agree 100%, except for a minor quibble at the end. I've tried a few small projects with pipenv and black recently, and though I love black, I'm still struggling to accept pipenv as good. It's so slow so often, and I can't understand why.
I'm coming from managing virtualenvs with s*ty bash scripts on a complex application, so pipenv got rid of a ton of jank, even though it's slow and has issues resolving certain dependencies.
I like it because I can document most maintenance tasks as "pipenv sync && pipenv run X" and they Just Work with exactly the library versions specified for that commit.
But definitely look into poetry if you're packaging a library.
I have similar experience, pipenv is not bad but it's definitely not great. Personally I'm starting to use poetry in new projects and it's being great, much better than pipenv.
Whenever I have time I want to migrate all my pipenv projects to poetry.
I've previously profiled pipenv and found it to be slowed down massively due to launching pip for each package it was working on. Unfortunately the maintainers think the progress bar is more important than performance: https://github.com/pypa/pipenv/issues/2207
I've been meaning to update that post to mention that Poetry has been releasing fixes and improvements more frequently than Pipenv. Poetry is great for developing packages, too.
(Showing up in an HN thread is a neat little milestone for my blog and me. Thanks for sharing!)
I use poetry. It has bugs (`poetry install/add/update` treats git-URL dev-dependencies as end-user dependencies). I sent a bug fix and it was not merged for months. I now use my own fork of Poetry with this fix added.
It's not you. There are plenty of people who dislike pipenv. Another commentor suggested poetry, which is what I favor when 'python3 -m venv .env' isn't appropriate.
I'm not looking for an "excellent" standard. I'm looking for something that isn't a complete outlier.
I feel like in every one of these formatter discussions people are waiting to pounce on anyone who takes issue with the formatting. I'm totally down with formatting: prettier, dart_style, rustfmt, gofmt, uncrustify, I use and love them all both professionally and personally (well, gofmt is bad at wrapping but the language is bad at wrapping in general). But in all these cases, these tools are either configurable or they set a standard for how to format code for $lang. Black does neither, which is fine, but its options then are "pick a common convention" or "pick an uncommon convention". All I'm saying is that I wish it had done the former.
> no-one likes what the autoformatter does to their code, everyone likes what the autoformatter does to their coworkers' code
It always strikes me as strange that we spend our own effort and time on systems that mandate code style when my unambiguously correct style and my coworkers obviously incorrect style both end up converted to the same AST for any useful work. Why isn't style an entirely local choice, with a higher-level representation of the code stored canonically?
Better question: What wheel did I point towards and suggest be reinvented?
Wow this is a killer feature. If you're really zealous about how things "ought to be", now you can just write your preferences out once and be done with it. Stinks that it locks you into JetBrains, presumably: unless, is this part open-sourced?
Seems like the underlying format is unrelated to editing with customizable AST—in fact, that seems like it'd make things like code review pretty miserable. You'd have to upgrade all tooling to be ast-aware.
That said, once upgraded, that would be far, far, far superior to text. Our tooling hit the limits of text a long time ago: consider how terrible diffs are at communicating simple operations like indentation changes, or how basic our refactoring is even in the best case scenarios.
> I often dislike autoformatter output too, but then I remember that while no-one likes what the autoformatter does to their code, everyone likes what the autoformatter does to their coworkers' code, and then I chill out about it.
I'm completely indifferent to autoformatters turning my coworkers' code from one perfectly readable format into another. The "problem" they solve is a ridiculous thing to worry about. Having a standard is not important, it's petty.
What I do dislike is having to fight the autoformatter because for some reason some teams use different settings. I dislike autoformatters turning code into something that is objectively harder to read -- sometimes it's not a matter of taste. Even when they work, I dislike even a second on something that only satisfies other peoples' pettiness.
I also dislike git blame being useless because someone reformatted all the code for no good reason.
That and it also saves our fingers' lives. Seriously, formatting code "by hand" just hurts. I highestly recommend anyone concerned about developing rsi to start autoformatting their code.
I have some issues with aesthetic decisions used in Black but the degree of head-nodding on the top answer in the SE post you linked is really disconcerting. The person basically says "well you're the boss so you should kindly remind your employees that its your way or the highway" in several different sentences.
It's pretty disturbing that people think this is how a manager behaves. Turns out there are reasons beyond aesthetics that people should apply strict code formatting practices.
You can find someone who’ll complain about anything. You have to evaluate whether they’re actually complaining about something valid or just reacting to change.
In the example cited, notice how insubstantial the complaints are? That’s a dead giveaway for someone who is reacting rather than thinking rationally about the goals and benefits. An important distinction to remember here is that we’re talking about something which is safe, fully automatic, and easily integrated into most editors, so the effort to follow it is a few seconds the first time you set it up.
I’ve generally found three classes of reaction to standardizing formatting, linting, and similar style checks: most people just roll with it, some people work through the initial “this is different!” reaction and realize how much easier consistency makes things (it’s been months since code review wasted time on formatting!), and a much group never get over it. I’ve only seen that a couple of times in a couple of decades and those were people who were net losses overall because this was just a symptom of a larger unwillingness to work well with others. The same guys refused to test their code, committed syntax errors or failed merges, wasted time reporting problems due to incredibly hacked up local build environments, etc. The important thing to remember is that those are a vanishingly small part of the community and I would make policy around them only to the extent of figuring out how to keep them from dragging your project down.
For anyone who doesn't get it, the name and logo comes from the famous story where Henry Ford, maker of the Model T -- one of the first affordable automobiles, because it was made on an assembly line -- said "Any customer can have a car painted any color that he wants so long as it is black." This quote and assembly-line idea translate into the branding of this project. (The project logo is visually similar to the modern Ford company logo, but with the word Black instead of Ford.)
I love black: formatting isn't a _big_ thing to think about, but it's always there. Until black. For [current project] it's hooked into VS Code as the default formatter, and (inspired by simonw) we've incorporated a black check into the test suite. It's simple:
> Black is the uncompromising Python code formatter. By using it, you agree to cede control over minutiae of hand-formatting. In return, Black gives you speed, determinism, and freedom from pycodestyle nagging about formatting. You will save time and mental energy for more important matters.
I run into this philosophical debate at work. In my mind, I don't care much what the resulting code looks like as long as
- it's somewhere in the reasonable spectrum of readablility
- it's consistent and unambiguous
- it never fails for a syntactically valid source file (if it's doing something for syntactically invalid source files, even better, formatting helps me find the error)
There are other opinions though that emphasize minimizing diffs or extensive customization to fit somebody's favorite style (I'm looking at you, rustfmt). Black seems to be right down my alley.
250 comments
[ 2.8 ms ] story [ 247 ms ] threadThough I think I prefer autopep8
But I'm not sure if making the code easier -- and thus faster -- to read is necessarily a good thing. If the brain parses and analyzes the code in parallel, making it harder to parse could give the analyzing process more time and thus make the review more thorough.
Of course, nothing stops a reviewer from being thorough either way. But making it the path of least resistance makes it more likely.
Edit: Previous discussion: https://news.ycombinator.com/item?id=17155048
I prefer Black, which has minimal configuration. There's no debate to be had over details that don't matter much in the grand scheme of things. Yeah, maybe it doesn't always look _just_ how I'd like, but I get used it.
And while Black may not be 100% compatible with flake8 out of the box, they are aware of flake8 and document places where it may not be flake8-compatible.
That's usually a good clue that you've hit real middle-ground.
I blackify my projects once we hit 3 contributors.
I write some garbage-formatted code, press ctrl-s, and the code is formatted automatically.
Even on a single-dev project, I would take a config and just put it there for the convenience it provides me.
If I ever were to code python again, I would probably also use black.
It is immensely valuable to be able to look at any Go code, whether in the toolchain, the standard library, or a random stackoverflow snippet, and not have to think about formatting at all.
That's not the case at all. The Go compiler treats an arbitrary few lint issues as compiler errors (e.g. unused imports), code which is not gofmt-formatted isn't a compiler error.
you can't even comment out your code while debugging. annoying as hell.
- reformat the whole codebase
- build (if needed, e.g a C++ project)
- run all tests
The programming workflow is then:
- edit code
- run the 'check' script
- repeat.
Having used both extensively, my biggest grievances (in order) are that there aren't more editors that with good on-save support, that black is relatively slow (compared to gofmt), and that black isn't more opinionated and less configurable.
black --diff
black --check
The first one shows you what's different (so you have logs in your build job), the second one fails your job if there are any diffs. You could just do the second one if you're a fan of the whole brevity thing.
Here’s a reference:
https://github.com/LibraryOfCongress/coding-standards
The CI check can be pretty simple — for example: https://github.com/LibraryOfCongress/concordia/blob/2f813f18...
Here's an example with Prettier:
https://coolaj86.com/articles/server-side-git-hooks-for-code...
I think code formatting is very important for readability, and I think many (subtle) choices about how to make a piece of code more readable are very subjective. These types of tools are just incapable to make those choices because they have no concept of intention.
And some rules make this painfully obvious even in less subjective cases. Eg: black formats dictionaries into a single line if they fit one line, which makes nested structures unreadable if they combine bigger and smaller sub-dictionaries.
In the end, I don't have problems with other peoples' formatting choices; as long as they are sufficiently consistent within their own style, which they usually are, I can live just fine with them even when they go against how I'd do things.
I do have problems with tools that reduce code readability for consistency's sake and argue that these provide little benefit, and may even be detrimental to code quality.
In the example I mentioned, I may sometimes choose to put a small dictionary initialization into a single line if it's just a detail, or may split it into multiple lines if it's important enough for someone reading the code to pay attention to it before going further.
This simply cannot be decided by an automatic formatter without it knowing the code's purpose which, for now, requires a human brain.
Someone mentioned elsewhere that automatic formatters remove the cognitive load of formatting code, but I argue that load is an intrinsic part of programming, because it is (for the most part) the effort of communicating to the next person that comes along.
I have not used Black, but considering it so I am curious about these occasional situations where you want to override auto-formatting
Here's an example snippet by Peter Norvig that is beautifully formatted:
I you would be fine with 95% of what it does and just want it to ignore the 5% where you care the most, I think the tool could benefit you.
At least that’s how I feel about autoformatting in general.
0: which doesn't have tuples anyway; you could use structs, but it doesn't really work well in context/practice.
1: I don't use python frecently enough to have ready examples of autoformatter stupidity on hand for it.
A better example is a matrix laid out as a grid, which auto-formatters always destroy
what do you think the result of
is? Now granted, you shouldn't do that, but what you think it is? a dict with 4 keys, right? {'type_a': ..., 'value_b': ...}.That's wrong. It has those four keys, and two more: '.0' and .1', whose values are the packed tuples.
!?!?!
Prettier for JS has a simple solution to this specific problem, and that is to follow the code's lead. If the first element of the object or array starts on the same line as the opening bracket, keep it as a single line if it fits. Otherwise, if the first element starts on a new line, put every element on a new line (explode the collection), even if they would all fit on a single line.
Black should consider doing this instead of the open PR to explode collections if there is a trailing comma[0], which feels... wrong.
[0] https://github.com/python/black/pull/826
https://github.com/python/black/blob/master/plugin/black.vim
1. black is not perfect, but it is pretty great
2. vscode is a much better IDE than Atom
(edit: black has pre-commit support [1], and it's easier than setting up and maintaining your own hooks IMO)
[0] https://pre-commit.com/
[1] https://github.com/python/black/blob/master/README.md#versio...
Run that target as part of the test
I super dislike black's formatting, and I think it's really rare to actually see it in codebases. It wraps weirdly (sometimes not at all). I'd prefer to use yapf, but last I checked it still crashes on "f-strings".
Here's a small example:
Black formats this as: I've never seen Python code like that.I totally believe using a formatter is good practice. Black is in a challenging position of coming into a community with a lot of existing code and customs, and I get that. But I also think that's an opportunity, rather than having to guess at what is good, there's a wealth of prior art to look at. I wish it had done this, rather than essentially codify the author's style.
black isn't an uglifier/minifier. it's undoubtedly readable.
>predictable
why?
You always end up adapting to the tool, while adapting the tool to you
Until you oneday reach a perfect harmony with your tool... until you introduce another one, and the equilibrium must once again be found.
Also of note, this is a general principle/behavior (eg you and your furniture..!)
I’m kind of thinking of it as something like baking some exercise into your day by putting a walk into your commute: the little nudge of not fighting the tool on something minor means you don’t skip little things, and over time that adds up more than it seems at first.
[1]: https://github.com/google/yapf/issues/342
My understanding is that by using Black you're saying I choose to not express my opinions about formatting aesthetics and delegate that decision to Black instead.
Which, as a longtime Go dev and now Rust dev, I love formatters that are opinionated. I don't always love what Gofmt or Rustfmt do, but I definitely like consistency that the community has in code style.
So I don't care what Black thinks - I care what, hopefully, everyone uses. Though, I could see a situation where formatting can be altered and the project has a format config. Meaning that while code fmt differs between projects, it would be consistent among all devs working on X projects.
I was initially grumpy about my org adopting black because I preferred single quotes, but the level of standardization is a huge win in my book. I never even think about my code style anymore, I just write it and then run black.
To be clear though, in your example, it would format it as:
It might also format it as It only formats like your example if the statement is much further indented I believeIn this particular case, that may result in code that breaks with tradition. But I can't see a way to preserve the tradition without creating special or edge cases. For example, we can't follow the first option and get clean formatting with a function like zip. You'll have a train wreck of bad options about where to put the braces and how to place the comprehension's body in relation to the braces that enclose it. Versus, with Black's style, the answer is easy and straightforward, because you just do it the same way you would anywhere else:
IMO, that's good, even if it isn't what we're all used to. Having a bunch of special cases just to match what someone might think is more aesthetically pleasing in specific situations is not a desirable feature in a set of autoformatting rules.Not sure what came about of object comprehensions though.
You've also got to contend with the first not being a comprehension meaning that the comprehension's indenting can't so easily be kept the same:
Which is where I was going with the comment about edge cases. Personally, I don't want formatting rules where you might decide to format the arguments to a function in different ways depending on the specifics of what other arguments the function has. I like simple. Give me one rule for when it all fits on one line, and another rule for when it doesn't. And make sure neither of the rules causes me to have to re-indent things just because a function picked up an additional argument. And make sure that the rules are completely oblivious to the function's arity.Maybe that would fail to put long strings on their own line?
There is a great autoformatter that does not have this issue. Using it guarantees that you'll never have to stick with an outdated version of a language because your autoformatter of choice hasn't been updated for the current one, or even worse, is not maintained anymore.
It's called "cat".
Precisely. I wonder why ML has not been applied here.
Yeah, I keep seeing people singing the praises of black and I'm really dreading the inevitable future where people start acting like this dude's opinion is Correct Official Python Style.
If you use Pandas you should be using method chaining and therefore also black.
Go's formatter adjusts whitespace within a line and will add or remove blank lines, but it doesn't change line breaks and has no opinion on the maximum length of a line. Other formatters work differently.
Having a standard is more important than the standard being excellent.
Neither is very important, though. It's just formatting and your code will run the same regardless.
And you can see the phenomenon of over-abbreviation by reading a few debates on twitter and repeatedly seeing, "how can you not understand what I tweeted?!"
https://medium.freecodecamp.org/codebyte-why-are-explicit-se...
> Javascript developers
Oh, of course.
Having said that, I've been introducing a lot more colons, since I use TypeScript.
If you're working with others, though, then it becomes very important. I don't think I'm being entirely hyperbolic when I say that inconsistent or poorly-chosen formatting rules are the death of 1,000 cuts for a team's productivity.
There's a tiny but existent cost that's incurred every time formatting rules that aren't diff-stable result in a noisy code review that takes longer to read, or makes it harder for reviewers to discern the real changes from the formatting junk. There's a tiny but existent cost when excess delta makes it harder to gitblame. There's a tiny but existent cost when people have to stop and think about how to format their code manually. Or when they have to stop and debate formatting. Or when they read someone else's code slightly more slowly because different formatting rules make it harder for them to skim it or rely on pattern recognition instead of careful reading to understand its structure.
All those tiny little costs add up to something that's not so tiny. And it's so easy to make it just disappear, for the low low cost of swallowing one's pride, by simply adopting an opinionated autoformatter.
Don't reformat code you didn't otherwise touch. That's just common sense. Common sense autoformatters lack.
> There's a tiny but existent cost when people have to stop and think about how to format their code manually.
I rarely think about how I format my code. When I do, it's because the code is hard to format in a readable way, in which case an autoformatter will produce garbage.
> Or when they have to stop and debate formatting.
"Doctor, it hurts when I do this."
> Or when they read someone else's code slightly more slowly because different formatting rules make it harder for them to skim it or rely on pattern recognition instead of careful reading to understand its structure.
Neglible. To the contrary, different formatting reminds you that you did not write this code and you should read it more carefully because you can't expect its creator to think the way you do.
I've never been able to wrap my head around people having strong opinions on style issues. I'll defer to whoever cares the most on the team and then just do that. When I look at the problems in code bases, rarely has "slightly inconsistent formatting" been at the top of the list.
Because an autoformatter with 10 simple yes/no options has 1024 different ways those options can interact, and an autoformatter with 20 yes/no options has 1,048,576 different ways that they can interact. It's simply not possible to make sure that you're going to get reliably good results in the face of that kind of combinatorial explosion.
Versus, if there's only one way that it will format things, then the people designing the rules have a single stationary target that they can aim at.
[1]: http://universalindent.sourceforge.net/screenshots.php
If you don't have as powerful a backend as the full clang paraer/lexer on the other hand, I could quickly see things breaking as you described.
Let me take a shot at why it's important.
We spend years peering at code hunting for tiny, miniscule mistakes. Thus we're training ourselves, quite rigorously, to spot minor deviations.
We're also irrational in the moment: our aesthetic sense is bothered by certain patterns, and our social sense wants to assign blame for this "wrongness" to individuals.
An auto-formatter removes a ton of deviations that don't matter, and desocializes the aesthetics.
This saves code reviewers time and stress and helps them focus on what actually matters in the code.
And it only has to save more time than it takes to run "pipenv run black" to be measurably worth it.
I like it because I can document most maintenance tasks as "pipenv sync && pipenv run X" and they Just Work with exactly the library versions specified for that commit.
But definitely look into poetry if you're packaging a library.
Whenever I have time I want to migrate all my pipenv projects to poetry.
It would be nice if pip could act more as a base tool and pass information back to a wrapper.
I'm no longer using it for Python projects.
(Showing up in an HN thread is a neat little milestone for my blog and me. Thanks for sharing!)
The Python community is victim to pipenv's creator's marketing and shoehorning, that's why.
I feel like in every one of these formatter discussions people are waiting to pounce on anyone who takes issue with the formatting. I'm totally down with formatting: prettier, dart_style, rustfmt, gofmt, uncrustify, I use and love them all both professionally and personally (well, gofmt is bad at wrapping but the language is bad at wrapping in general). But in all these cases, these tools are either configurable or they set a standard for how to format code for $lang. Black does neither, which is fine, but its options then are "pick a common convention" or "pick an uncommon convention". All I'm saying is that I wish it had done the former.
It always strikes me as strange that we spend our own effort and time on systems that mandate code style when my unambiguously correct style and my coworkers obviously incorrect style both end up converted to the same AST for any useful work. Why isn't style an entirely local choice, with a higher-level representation of the code stored canonically?
Better question: What wheel did I point towards and suggest be reinvented?
https://www.jetbrains.com/mps/concepts/
That said, once upgraded, that would be far, far, far superior to text. Our tooling hit the limits of text a long time ago: consider how terrible diffs are at communicating simple operations like indentation changes, or how basic our refactoring is even in the best case scenarios.
https://en.wikipedia.org/wiki/Structure_editor
Pretty printing.
I'm completely indifferent to autoformatters turning my coworkers' code from one perfectly readable format into another. The "problem" they solve is a ridiculous thing to worry about. Having a standard is not important, it's petty.
What I do dislike is having to fight the autoformatter because for some reason some teams use different settings. I dislike autoformatters turning code into something that is objectively harder to read -- sometimes it's not a matter of taste. Even when they work, I dislike even a second on something that only satisfies other peoples' pettiness. I also dislike git blame being useless because someone reformatted all the code for no good reason.
It's pretty disturbing that people think this is how a manager behaves. Turns out there are reasons beyond aesthetics that people should apply strict code formatting practices.
In the example cited, notice how insubstantial the complaints are? That’s a dead giveaway for someone who is reacting rather than thinking rationally about the goals and benefits. An important distinction to remember here is that we’re talking about something which is safe, fully automatic, and easily integrated into most editors, so the effort to follow it is a few seconds the first time you set it up.
I’ve generally found three classes of reaction to standardizing formatting, linting, and similar style checks: most people just roll with it, some people work through the initial “this is different!” reaction and realize how much easier consistency makes things (it’s been months since code review wasted time on formatting!), and a much group never get over it. I’ve only seen that a couple of times in a couple of decades and those were people who were net losses overall because this was just a symptom of a larger unwillingness to work well with others. The same guys refused to test their code, committed syntax errors or failed merges, wasted time reporting problems due to incredibly hacked up local build environments, etc. The important thing to remember is that those are a vanishingly small part of the community and I would make policy around them only to the extent of figuring out how to keep them from dragging your project down.
Happy Black user over here!
https://github.com/simonw/sqlite-utils/blob/master/tests/tes...
I run into this philosophical debate at work. In my mind, I don't care much what the resulting code looks like as long as
- it's somewhere in the reasonable spectrum of readablility
- it's consistent and unambiguous
- it never fails for a syntactically valid source file (if it's doing something for syntactically invalid source files, even better, formatting helps me find the error)
There are other opinions though that emphasize minimizing diffs or extensive customization to fit somebody's favorite style (I'm looking at you, rustfmt). Black seems to be right down my alley.