In other languages, such a change would only be possible with a major version bump, though I imagine that because of the Python 3 collective trauma, the language designers now are OK with breaking older code without calling it Python 4. Anything goes, as long as it's called Python 3.
(Python lost me in the 2->3 migration and I haven't used it in a decade, so correct me if I'm wrong)
Python doesn't use semantic versioning. The number after the first period is a major (annual) release and can and does contain breaking changes (though so far never on the scale of the 2->3 upgrade).
We may never see a 4.0 because of the scar tissue, but the language continues to evolve.
> We may never see a 4.0 because of the scar tissue, but the language continues to evolve.
They should do the opposite really. If it hurts, do it more often and get better at it. A perfect time would be when Python gets some nice JIT performance improvements which everyone will probably like.
I don’t think there have been significant backwards-incompatible changes in Python 3 the language. There have been some, e.g. async and await were first introduced as soft keywords and then switched to being actual ones. But that’s not all that different from the treatment of yield in Python 2.
(Recent stdlib changes have been much more destructive, but I’m assuming that, like in the original thread, we’re drawing a distinction between those and changes to the actual language.)
Full disclosure, I welcomed Python 3, because for me that was the first time (since 2.4 on Windows XP) that I could count on my programs not randomly shitting their pants upon encountering Cyrillic in files or filenames, which for a native speaker of Russian you can imagine is quite important. (The csv stdlib module in Python 2 did that, IIRC. Perhaps I was holding it wrong, but experience shows that absolutely everybody did.)
I think basically every new python version removes some standard libs and marks new ones as deprecated (at least 3.13 did), that’s potentially breaking.
The bytecode format changes, too, as does the C ABI (although they're making improvements on the latter front - there's supposed to be a more stable subset now). But more recently, they committed to a proper scheduling procedure for standard library deprecations and removals.
One that bit me was the change to StopIteration propagating through chained generators, following PEP 479. Nothing huge, but I had to patch some previously working code to accommodate the new language release.
It's still amazing to me that the old `csv` module expected you to open the file in binary mode. (This was fixed in 3.1 - the 3.0 release was honestly quite premature.)
> Requiring specific exception types makes the programmer’s intentions clear and encourages thinking about what exceptions might occur.
This reeks of the same rationale for checked exceptions in Java. That was a failed experiment. You can't force people to deal with exceptions. They end up just swallowing them instead. It's better to propagate an exception than do that almost all the time.
Like will we see except: replaced with except object:? I don't even know if that's valid. It's never come up.
Second, this would be a breaking change. I really feel like this is where Python 3 went off the rails. In the Python 2 days making breaking changes was essentially verboten. But ever since Python 3 decided breaking changes were OK< it's like there's little restraint now and minor releases seems to be far too comfortable with this.
I'm weakly opposed to the PEP, but if your concern is that you're going to lose the ability to catch all exceptions in new code, then that's wrong as discussed in the Backwards Compatibility section, i.e. do "except BaseException".
Going back more than a quarter century ago (!), I was one of those zealots in favor of checked exceptions in Java. It took a good 15 years to finally conclude the “checked” part is more trouble than it’s worth.
Given that there is inevitably a root context for any given language invocation (main() for a command li program, top of a thread for a multi threaded app, etc), unchecked exceptions and a top level handler have proven to be more than enough.
I won’t comment on Python backwards compat, will just shake my head.
If I’m understanding this correctly this proposal would fully break compatibility with many (most?) codebases, actually remove syntactic sugar and force more characters for the same functionality. I fail to see how this is even being considered.
I don’t understand the idiomatic viewpoint either here, I understand the author personally finds it confusing when excepts aren’t verbose but I think you would be hard pressed to find many python developers who agree. Even outside the ecosystem, most languages have been adding more support for bare excepts (like js with bare catch) so this feels like a step backwards.
But maybe I’m just not understanding this proposal!
Python feels like they are fixing the language the best they can by slowly and properly adding explicit types. Character count is not a valid argument when we have hard drives that are multiple terabytes and IDEs and LLMs that will gladly auto complete a full word with the import. Foo, bar variable names and i, n incrementers should be banished to freshman level undergrad tests meant to intentionally confuse the student.
I'm hoping Python 4 will be a big breaking change similar to the previous one and full support for explicit types will be one of the reasons.
Character count is a valid argument because a human is going to have to read the code at some point. Otherwise, Java style names like "countriesFromAsiaWhereAreTheMostPlanesAndBoats" would be just fine.
Edit: I don't get the hate for "i" or "j" as increment variables. When you're working with numerical data they closely match how you would express the same operation with mathematical notation and are idiomatic to the point that everyone with non-trivial experience in programming knows what they represent. There are better options in some cases (e.g. "for name in names:"), but there's nothing inherently wrong with i, j, k, etc.
It's not in bad faith to say that there's a cognitive load to having longer line lengths or, god forbid, wrapped lines when you're trying to figure out what the code is actually doing with that data the variables represent. Obviously the Python "except" case being discussed is not a big deal in this regard, but you made a blanket statement about character count being no big deal because of big hard drives and IDEs.
I work on code that uses `pid` pervasively, and it would drive me crazy if someone insisted I write out `processId` every time. Same with `tx` or `rx` for channel endpoints, `i` for loop variables, `txId` or `tid` for transaction ID in databases, etc. If something is very common in the domain you're working in it's more annoying _not_ to abbreviate it.
Recently, I saw somebody writing that in their projects, they define a Glossary.txt of terms, and then set up their tooling to flag any use of an identifier that isn't a word or defined in the Glossary. (Allowing also for compound words like camelCase and snake_case, of course). So in their project they could define `pid: process identifier [...]`, and then their tooling would be allow a variable named `new_pid`.
Upsides: consistent vocabulary in your project, a centralized place to look up jargon, and a subtle friction to avoid adding new terms that people might not immediately grok.
I wish I could remember the source; because it sounded like a nice setup to steal.
At my last job, there was a frontend developer who added a linter rule that variable names must be at least 2 characters long. The project already had 20,000 lines of code. Every time anyone made a change to a file, they would have to rename all the one letter variables. Usually, this meant all the for loops in the file. I tried to explain how pointless this rule was, but he wasn't having any of it.
Most people just renamed variables like i > ii, which was worse.
What is even the point of digging in deeper, did they not see everyone around them doing the i>ii workaround? You've lost, call it a failed experiment and move on.
a good linter rule would have exceptions for loop variables by context by just by name like i, j, k. Often just by name is good enough at least for a solo dev or small team. I require them to be at least 3 chars EXCEPT those.
Python 3 broke almost literally every non-trivial Python file on earth, and was saved only by `2to3` and `six` being able to automate or library-ify away 75% or so of the changes. The remaining 25% was make-work for teams needing to avoid the deprecation/EOL/vuln demons (or wanting to take up new language features that became Py3-only), and many teams took the opportunity to instead spend that time rewriting their codebases in Go or TypeScript+Node.
I don't know how much more breakage you really wanted out of Python 3 if "permanently scoured the public image of the language and caused many Python shops to, at least partially, stop being Python shops" wasn't enough.
(I say this as a still-fan of Python who has written quite a lot of it and contributed to MicroPython/CircuitPython's internals - I just also worked at a Python shop during the Py2->3 hell, and frankly, even my current dayjob still talks about that transition as a nightmare to watch out for if any other language starts doing similar talk.)
My experience was different. The single biggest change I had to deal with in a zillion places was change of str from bytes to unicode. That wasn't just a syntax change. It forced me to chase down all the places where I'd been handling treating strings like bytes because it worked 99% of the time as long as I was only dealing with ASCII. No amount of clever AST re-writing would save me from those earlier mistakes.
Python 3 made entire classes of errors impossible:
* Beginners don't introduce ACE exploits into their program from the get-go, because `input` is what `raw_input` was before and the "convenient" wrapper using `eval` was no longer available - so instructors were forced to teach students about explicit type conversions, like they should have been doing the whole time.
* You can't get `UnicodeDecodeError` from calling `.encode`, or `UnicodeEncodeError` from calling `.decode`, because you're never in the position of pretending that a sequence of bytes is a "string". There's no illogical "basestring" type and no `str` objects with ambiguous semantics - `.decode` and `.encode` do what they say, and belong to separate types. These problems resulted in a huge mess of duplicate Q&A on Stack Overflow full of incorrect advice from people with no clue what they were doing, and empowered people to ignore essential truths about Unicode until it blew up into a bigger problem.
* Similarly, when you read from a text file, you actually get text now.
* `print` used to be confusing and have weird ambiguities and tons of special syntax. Being a function means it can be taught the same way as any other function call; plus you get use as a higher-order function, unpacking operators (you can't do `print(*range(10))` in 2.x).
* `1000000000 in range(1000000000)` no longer hangs. `xrange` does not solve this problem (although it does improve matters quite a bit by avoiding high memory usage).
* Sorting heterogeneous lists correctly produces an error, rather than an ordering so strange as to merit its own Stack Overflow Q&A (https://stackoverflow.com/questions/3270680 and many duplicates). (You can still replicate the old order if you want.)
* `isinstance(x, int)` doesn't fail because of `x` being too large. This was a bizarre speed bump in such a high-level language.
* You can write '£' in your source code and you only have to declare an encoding if you use something other than UTF-8 (which Python correctly identified as the eventual winner).
When Python 3.2 came out I found it to be a breath of fresh air. By 3.4 I was already starting to wonder why so many others were dragging their feet on migrating.
> Python3 did not break enough to justify the jump from 2 to 3
I agree - it should have broken more (and thereby become able to fix more). It should also have been usable out of the gate and not majorly reworked (there was a serious battle over the syntax of byte and string literals, and possibly some other things, that resulted in 3.0 and 3.1 not seeing a full 5-year maintenance lifetime), and of course developers should have actually fixed stuff promptly and accepted that the obviously superior new ways of doing things were, in fact, obviously superior.
Unfortunately, a lot of other developers don't seem to agree.
And as for explicit types - I really wish people would stop trying to fight the type system - of Python, and of whatever other language. Python is not meant to support static types. It's not designed to reject your code at compile time for a type error and it isn't designed to take type information into account when generating bytecode. It's designed, instead, very explicitly, to let you care about what a given object can do, rather than about how it categorizes itself.
Javascript somehow manages to grow without breaking backwards compatibility. So does C++. Breaking countless packages (and forks of packages) in the pursuit of something as nebulous as language purity is a big mistake. I happen to like explicit typing but it's not the kind of thing you can graft onto a mature language without making awful compromises. Also, it pushes massive externalities onto the millions of people who rely on Python for their work.
By the time you've written a Python that has all explicit types in it, and harvested the speed advantages the interpreter can have when it can count on that, and all the other cascading changes that would result, and the fact that you would discard 100% of all previous modules both Python and C, you might as well just start using one of the existing statically-typed languages that already has a mature ecosystem.
Python should be working on being the best Python it can be, not being an adequate Python and an inadequate bodged-on static language on the side.
Python 3 does not, but the hypothetical Python 4 would be crazy to put types on everything and then not accelerate its interpreter with the resulting data.
The problem Python 3 has is dynamic types, as the dynamically-typed languages implement them, are viral; one little crack lets them in somehow and all the code operating on the data has to assume it's viral.
I would be pleasantly surprised if they manage to do that. The current type checkers of Python, as useful as they are, hit the limit pretty fast, because of its extremely dynamic nature and the type system struggling to type check valid expressions. The number of times I've seen common pandas and numpy code not type check
CPython doesn't, but there is Mypyc [1] which compiles statically typed Python to faster C extensions leveraging the type information. As usual, this comes with tons of limitations [2]
It's basically documentation to make it easier to use functions. After that people decided to enforce it basically as lints and only where possible. It feels like Python to me.
I'm hoping "Python 4" will be another language entirely that displaces it, Fixing some ecosystem problems upfront (packaging, concurrency/GIL). Nim is possible candidate, though Go is pretty popular w/ Pythonistas.
My personal unhappiness with how Py3K was handled, plus recent PSF events make me feel new leadership would also be a boon..
I have found Nim more useful/usable than Python + Cython since about 2014 for my personal use cases and almost always much more efficient on a "per unit of effort basis" than Go. At least if you are willing to write whatever code you need or link in C/C++ instead of relying on people gifting it to you.
I think the ecosystem leadership position Python finds itself in lately may well make Py-core silly enough to think "our users will tolerate our breaking-all-their-code antics no matter what". I suggest a more consenting-adults alternative here: https://news.ycombinator.com/item?id=41790766
FWIW I've been semi-actively designing a language of that sort, which I call Fawlty, since about February of this year. I was planning to start writing about it in July but then all the PSF drama stuff happened and I didn't want people to assume incorrectly that the idea was motivated by my disappointment with the community.
In fact, I'd been thinking about doing it since probably November of the previous year - and it's motivated by long-held beliefs that
Python's design falls short of the Zen;
several of the most common beginner pitfalls could and should be avoided by syntax changes and by a different approach to parsing the code; and
the standard library is full of ancient APIs that look terrible because they're motivated by C and Java designs that used to be the best we had but seem highly unidiomatic now.
It was somewhere in November or December last year that I first wrote those thoughts down more concretely (with details about what those pitfalls are etc.).
Given that pace of progress, however, I've more or less given up on ever expecting myself to publish something usable - by myself, at least. I've decided for now that it will be more practical to add blog posts about my ideas to the queue, and possibly see about my own implementation later.
I don't think so; if you recognize "Fawlty" as the name of a tower then you equally well recognize Fawlty Towers as a spiritual successor to Monty Python. The goal isn't to "escape" but to suggest that this is a new effort with the same background principles.
> force more characters for the same functionality
This is actually a good thing in some cases (possibly this one). Risky stuff should inherently be harder to do than safer stuff, otherwise people will reach for the risky alternatives when they don't need to, just to save time - or because they don't realize the risk.
Or at least, that's often the case. What's lacking here is evidence that this is actually happening. I can believe it, but evidence is necessary for breaking the language.
>> I fail to see how this is even being considered.
To me it stinks of an ego-centric person thinking they're a "language developer" and knowing better than the actual users of the language what's best for them. Just because something can be misused doesn't mean you have to take it away.
I haven't noticed, but since Rust came along is there a trend among languages to enforce "safer" programming at the language level? I could see that kind of thinking getting way out of hand. If that's the case, I would see this one as "I'm going to save the world with this dumb little change that breaks things for a bunch of people!"
I would hope a PEP like this came about from frequent user requests but that doesn't seem to be the case.
Rust had extremely successful marketing based on its security claims. It's no surprise that other languages jump on that bandwagon to not get left behind, is it?
It's interesting that you mention Rust, since Rust takes backward compatibility quite seriously; that's why it's still on version 1.x. Granted, sometimes there is a compiler bug that causes some old code not to compile with newer versions of Rust, but that is rare and never intentional like this PEP.
They do take it seriously, I agree. However, the commonly repeated meme that Rust only makes backwards-incompatible changes by mistake or to fix soundness issues is wrong. They allow themselves to make changes if they're judged to have a low (but nonzero) risk of causing backwards incompatibility in the wild. For example, adding a new function to a standard trait can be backwards incompatible but they do it all the time.
Indeed. Although it's worth noting that the same is true of e.g. stable enterprise favorites like Java, which regularly makes minor breaking changes that are judged to have little impact (which is why every Java release is accompanied by a compatibility guide; see the "Important Changes", "Removed Features", and "Other Notes" sections of the most recent release notes: https://www.oracle.com/java/technologies/javase/23-relnote-i...).
>> It's interesting that you mention Rust, since Rust takes backward compatibility quite seriously;
I mentioned Rust because the memory safety guarantees are a significant new thing for a language like that. I forgot about "managed" languages like C# because that's quite far from my mind, but that's another significant attempt at safety. This kind of little detail in the PIP is really insignificant by comparison, so I was speculating that it might be driven by some kind of "save everyone" mentality. If so, wondering if that's a trend lately and I hadn't noticed.
(Just to be sorta pedantic) I don't think it's the version 1.x that promotes backward compatibility, but editions.
In 2016, you could call your function `fn async(...) { ... }` without any issues, and you can still compile this code with the modern version of rustc. If you want to use the async features of Rust, you need to change your edition to at least 2018, which brings breaking changes in the language.
And you can have a project that mixes those editions, and cargo will correctly use the edition asked for each crate.
(So, I guess the ideal world would to first look at the package management in Python, and *then* try to introduce breaking changes. And I'm withholding how much I'm angry at the PSF for postponing having a decent package manager for 20 years and investing in removing the GIL and adding JIT.)
Not meaning to apologize for Python here, but you have significantly more ability to segregate “editions” when you statically compile code.
All the more reason to take a breakage very seriously. This is even worse than the walrus operator. At least I can ignore that. This breaks working code for some notion of purity.
Code compilation doesn't really have much to do with it. Python already has a somewhat similar ability - opting into certain language features of python on a file-by-file basis - using __future__[0]. It'd be pretty easy to add something like Rust editions by looking for a special statement in the file. And to make it more convenient, put it in the __init__.py and have it be transitive to the module.
There have been people trying to enforce safer programming at the language level at least since Java positioned itself as the safer alternative to C++ way back in the 90s.
I'm pretty sure Bertrand Meyer's OOSC[0] from 1988 had something like "if a language has a feature which comes with warnings that you shouldn't use it, it shouldn't have that feature" (paraphrasing).
No. A serious language designer will try to make things they like at the beginning, not trying to patch it later. Patching via breaking public API (language design) is never a good thing.
PEP contains lots of best practices that aren't enforced by the interpreter though, doesn't it? e.g. PEP 8. It's been a while so maybe PEP 8 is more unique than I realize? It seems like a pretty sensible recommendation that wouldn't necessarily need to change the way exceptions are handled by Python. Right there in PEP 8 it says in big text "A Foolish Consistency is the Hobgoblin of Little Minds." I imagine that enforcing this in the interpreter would fall under that, but it seems like a good piece of advice for folks new to the language, or more likely, new to coding.
Python is compiled to bytecode just like Java and C# - it just also happens to provide an environment out of box that will do it on the fly, and makes it much easier to access the "compiler services" in the standard library (and the built-in `eval`). And it's always been that way. The idea that some languages are "serious" and others are not is already suspect. The idea that being a "serious" language requires being "compiled", or that other languages are just for "scripting", will severely limit you as a developer.
>actually remove syntactic sugar and force more characters for the same functionality. I fail to see how this is even being considered.
Python is not APL. Getting at the functionality in fewer characters is not a design goal - it's just a usually consequence of the actual design goal.
This is being considered because "Explicit is better than implicit." and because it helps avoid a common class of error (e.g. `except: continue` prevents aborting a loop with Ctrl-C, which is often not intentional).
Or as the PEP puts it:
> While this syntax can be convenient for a “catch all” handler, it often leads to poor coding practices:
>
> 1. It can mask important errors that should be propagated.
> 2. It makes debugging more difficult by catching and potentially hiding unexpected exceptions.
> 3. It goes against the Python principle of explicit over implicit.
Python isn't any other language, either. It certainly isn't taking design guidance from JavaScript (which runs in an environment where the page is expected to show something coherent, and not a loud screaming error, no matter how absurd the input data and/or code).
As for how much code it would break, you made me curious:
But drilling down further, over 2/3 of those bare excepts are in local copies of Python itself (i.e., multiple versions of the standard library) that I built from source. Probably all the rest are in dependencies. I don't write code like that myself if I'm even remotely paying attention. (Of course, that only tells me how many occurrences there were, not how many files have them. But the first two results imply an average of about 2 `except`s per file, so.)
I don't know, but it seems like, as a language grows, the type of people working on improvements changes. In the beginning, the contributions come from people trying to solve application problems. In the end, the committees and contributors seem to be less connected to reality and more internal and isolated. They make changes that seem conceptually sound but aren't grounded in what the users of the language actually care about.
I'll add a 4th rationale:
4. It will create work for countless developers which is completely consistent with the python core value of disdain for other people's time.
If this pep were implemented I suspect it would result in forcing thousands of not tens of thousands of people to spend hours modifying perfectly working code and destroying the ability to run old scientific code without modification. Extremely effective industrial sabotage if it were to be accepted.
It is hard for me to articulate how much peps like this reinforce my desire to never start another python project. Even if this pep is rejected the fact that there are people who would put in the time and effort to write and submit such a PEP tells me that they will do it again, and eventually they might succeed.
On the bright side, turning bare exceptions into types exceptions is the kind of thing an llm is great for. It's also basically zero cost for new code.
On the other hand, I completely agree that it's not worth a breaking change.
There's some space for interpretation in picking exactly which exception type to use depending on context (value error vs runtime error vs not implemented error), and there may be package specific exceptions available.
Thank goodness there was 2to3 tool in the past. It made the migration to Python 3 so smooth and quick. /partial-s
I know it is not nearly on the same level, but people seriously overestimate the effort needed between not doing anything at all and even the slightest work, no matter how reliable and easy. The difference between nothing and anything is huge.
I feel like I've heard this argument countless times, and yet I'm never swayed by it. I've been using Python for about 20 years and I've never felt put out by the need to change anything to work with a new version of Python (or of a library). It simply hasn't caused significant pain - my memories are more filled with painful debugging sessions caused by overly-clever designs or trying to refactor too much at once.
That's besides the point. I don't want to muck about with tools on my Python scripts.
I have sometimes not run a Python script for a few years, and then when I need it, it stopped working and I need to track down what changed/broke or run some tool or whatnot. I don't keep track of the latest greatest Python changes – like most Python programmers it's not my "day job" to write Python code so I now need to track what changed between "the Python version I used about 3 years ago, whatever that was" and now. It's pretty annoying.
And that's assuming said tool will be fool-proof. Never mind of course that all my dependencies (if any) will need updating too.
What will happen in practice is that people will write "except Exception:" rather than "except:" and do nothing different. Basically nothing will change.
Meanwhile, I have C and Go programs that have worked without modification for about 10 years. Not that nothing ever breaks in C or Go, but it's the exception (hah!) rather than the rule.
> like most Python programmers it's not my "day job" to write Python code
I’d love to know whether that’s true, and to what extreme. I believe you’re right - that people using Python for a few hours a week (or less) greatly outnumber software developers using it as their primary language.
I think that’s a real issue for the evolution of Python, because updates to the language design (e.g. the makeup of the Steering Council) come almost entirely from the second group.
> I think that’s a real issue for the evolution of Python, because updates to the language design (e.g. the makeup of the Steering Council) come almost entirely from the second group.
Yes I agree, and it's disappointing to see some take such a narrow view of things.
A big part of maintaining and evolving a language is saying "no" a lot. There are a lot of people with ideas, almost always reasoned from their own use-case. That's okay, everyone does that to some degree, but there almost always trade-offs and such to consider.
Your job as Steering Council or Core Dev or BFDL or whatever governance you have is to consider all use cases and make a balanced decision. Reading that thread, some do. But unfortunately others don't.
Even for Python 3, Guido spent most of his time saying "no" to proposals. There were a lot of pretty wild ideas for Python 3000.
Which just underscores the point that this is mostly software engineering theater. If your goal is a system in which all exceptions are explicitly and appropriately handled, your first mistake was picking Python.
I propose a rider to the PEP in which implementation will be deferred until its proponents can correctly affirm that the library reference lists, for each function and method, every exception it might throw.
>If your goal is a system in which all exceptions are explicitly and appropriately handled, your first mistake was picking Python.
No, the goal is a system in which the code correctly indicates which exceptions it's intended to handle, and doesn't accidentally handle the wrong exceptions because the developer was either lazy or misinformed about the semantics (perhaps due to experience with a different programming language).
> ...and doesn't accidentally handle the wrong exceptions...
What support does Python offer, to the author of a function, for determining what would be the wrong exceptions for it to handle? How, in Python, does the author of a function signal, to the functions she is calling, which exceptions they should not handle? As I alluded to in my previous post, the Python language and documentation does not even give programmers a good accounting of what exceptions the library functions might throw.
The point was about how the set of exceptions handled by `except:` might not match what the programmer expects, especially a programmer coming from a different language. It's not about knowing what kinds of exception the code in the `try` could raise and it isn't about telling a called function what to do.
Other languages, notably Java, have tried the "functions document what they can raise" idea. Everyone seems to hate the result.
So it doesn't matter if it goes through or not, just that someone proposed a change like this is enough to steer you away from Python?
If the change goes through, couldn't you just use older Python versions for those specific projects, or has the Python ecosystem still not figured out how to do this without huge hassles?
Does that matter when you just want to run "old scientific code"? Old version of libraries like OpenSSL can still be run in that context, granted you don't expose that code to the internet at large.
Old scientific code broke for many people with the introduction of the mac m1. I would think this would be a continuing trend in the future. Staying on old versions simply isn't possible over a long period without keeping the hardware going with it too.
> Old scientific code broke for many people with the introduction of the mac m1.
How could the people maintaining Python possibly avoid that? It would be up to Apple to proactively reach out to affected projects, if Apple cares about that.
Who finds "old scientific code" and then exposes a server running that code to the internet without any changes? Sounds like asking for trouble, but I guess we all use computers differently...
I don't know why people do things without thinking them through, but they do. Regarding trouble, I don't think we've covered anything here that wouldn't be asking for trouble.
Either software is updated or it isn't. If you're worried about "bitrot" then you bear the responsibility for your end of keeping the system up to date. (Or finding a third party to do it.) API Changes occur for a reason, and it isn't reasonable to expect other developers to make security fixes to their older versions of code in perpetuity while guaranteeing that stable interface in perpetuity. They'd never get to fix anything that isn't a security issue that way. Programmer resources are limited - especially for Python, which doesn't pay the overwhelming majority of its devs (although it can afford to pay several PSF staff).
Python is open source. Nothing prohibits you from forking the 2.7 codebase and adding your own security patches (or more substantial things like back-porting new OpenSSL support, or even cherry-picking backwards-compatible features from 3.x that you do like), for example.
I'm happy when people criticize new features in Python. But I expect to read criticism of features based on their actual merits and consequences, not on the principle that it's new or backwards-incompatible or would cause "churn".
So we have this proposed change which will cause a bunch of needless churn.
Then we have people saying "just don't upgrade anything if you don't want to deal with the churn" but that too is a very large burden for all the reasons I mentioned.
All the reasonable adults in the room want option 3: don't do the thing that causes needless churn.
> But I expect to read criticism of features based on their actual merits and consequences, not on the principle that it's new or backwards-incompatible or would cause "churn".
I am honestly shocked to see this attitude. Churn is a very real consequence. Do you have no respect for the time of the countless devs who use python?
The entire point is that proponents of the change disagree that it's "needless churn".
It's bizarre to me that expecting people to improve their code incrementally is considered a disrespect for their time.
The proposed change is not arbitrary - which can be seen by trying to imagine the alternate universe in which the reverse change were proposed. One can imagine a world in which `except:` were added to a Python that didn't support it, but certainly not one where it were made mandatory (whether for the `except BaseException:` case or the `except Exception:` case).
I assume there are people out there who would, similarly, argue that it was "needless" to make `print` into a function (and thereby break users of the `>>file` syntax). But it demonstrably and significantly makes the language better.
If you want to talk incrementalism, then the right approach is making this an error in other parts of the Python ecosystem. Linters, mypy, pyright, etc, without make a source-breaking language change.
Imagine whatever universe you want, but we live in this one. In this universe, there is a shitload of old python. Python 2.7 isn't even completely gone. Literal lifetimes of collective time would be spent on the fallout of this change, which does not seem worth the reward.
> It is hard for me to articulate how much peps like this reinforce my desire to never start another python project
I completely understand this sentiment. Recent python events have made me wonder if there are some people intent on sabotaging the management of the language.
I loved the incremental improvements and thoughtful process involved up until a couple of years ago but it feels like python will become brittle and break badly if things continue the way they are. It feels like the adults have been driven out the room when it comes to stewardship. I'm not sure how recoverable the situation is.
Using "|" to merge dictionaries (which was possible in other ways before) instead of offering pipes as in bash and Elixir (a feature that's actually useful).
The “|” operator was already used for set unions and binary OR, so it’s a little late to reserve it for control flow. Personally I don’t mind having a “dict union” operator at all.
Any Lisp-ness comes from Elixir being a skin on Erlang, and so comes almost anything else good to say about Elixir. (And my comment was explicitly comparing Elixir to Erlang.)
The 'pipe' is hacky, for example, because it's just syntactic sugar that only works in one specific case, and not in general.
On the Python side, though, at least you can build your own pipes! You can define various helper classes that have, say, an `__rrshift__` method, to let you do the following with full type-checking support:
Like all of python, `a | b` operator is just `a.__or__(b)`. If you want that operator to do something different in a different context, just override __or__.
Nothing prevents you from defining the | operator for other user-defined types where that would actually make sense. A dictionary doesn't represent an ongoing process or stream. Lots of things are possible; that isn't a reason not to find better ways to do them (cf. Raymond Hettinger).
It wasn't recent by Internet time but when the debate on walrus operator drove out the BDFL that was the obvious break. Python has been circling the drain ever since. A lot of motion, yes, but to what end?
- - - -
Oh! How could I forget!? The creeps actually banned Tim Peters!
Yes? and as GP said, he stepped down because (as explained in the first line of his email) "Now that PEP 572 is done, I don't ever want to have to fight so hard for a PEP and find that so many people despise my decisions."
A lot of the time, people say things like "Python has been circling the drain ever since." referring to the implementation of the "walrus operator", to imply that they don't like the feature and that it was the first of a series of changes to the language that have made it progressively worse; and they often further imply that if only we still had the original leadership then we could avoid such damage to the language.
I was, in a sense, in that camp at the time, before I looked it up. I felt that the operator went against the spirit of the language by trampling on what was previously a strong, and clearly very conscious, separation between statements and expressions. And I misguidedly imagined, and lamented, that GvR was unable to keep it out of the language, being overruled by consensus.
I just want to make sure it's clear that things are not like that. Rather, the Python envisioned (nowadays, though not originally) by the original leadership includes PEP 572 - and probably also the large majority of what's been added since.
It's not the "walrus operator" per se that's the problem, it's the change in project governance.
Python has been captured by bureaucracy and corporate interests. The way it's being improved-to-death is a symptom. The unceremonious eviction of Tim Peters indicates to me that the take-over is complete: the new guard feels comfortable throwing out the old guard, they expect that their power is such that no one will blink, and no one did. (No corporation withdrew support for them, the blow-back was all hot air.)
It would be interesting to know GvR's opinions of some of the new cruft, but it's not really relevant.
I feel like as a scripting language Python excels. Glad to have this PEP, but it would be more pythonic have except be optional.
The reason I pick up Python for projects is because it grows with the application; opportunities to add typing etc. Who knows maybe in a few years Python will enforce all the types and it will be as verbose as Java. Personally I’d like to see how they handle declaring a method or function throws exceptions.
Pretty narly we have compiled Python apps with poetry, it’s starting to punch out of its weight class.
I think emitting a warning every time an unspecific exception is caught might be a better balance. That way, you could still do a quick “try: … except: …” when drafting new code, but the code might warn you if the bare except block is ever used (including what exception was caught, and a suggestion for how to catch only that specific exception).
Perhaps in a few years we can have another PEP, to require "except BaseException" to be replaced with bare "except:". Then we can all change our code back again.
Eh, while I sympathize with what you’re saying, PEPs get written and rejected all the time. I’ve gotten the impression that some were written for the main goal of documenting the reasons why a common request is a bad idea.
Like, I don’t know if there’s a PEP to use braces, but it wouldn’t surprise me if someone had made one so that from then on there’d be an official doc you could point people at when they ask about it.
Not saying this is one of those, and I see Brett Cannon’s on this one. I am saying not to get too worked up over the existence of a draft PEP.
I often find discussions of these sorts, whether for python or other open source projects, get so focused on purity of concept that they totally forget practicality
If someone on my team or in my company proposed to break most of our python code for no substantial reason, unless they were pretty junior I would count that as a real red flag against their judgement.
How do people land on the python steering council exactly?
Please don't cross into personal attack / name-calling, or post shallow dismissals of other people's work. As the site guidelines say, a good critical comment teaches us something.
The existence of two opposing PEPs says to me that there may be a case to be made for either side, so we should default to not changing things.
I used to think it’s likely that you can make a much stronger case for one side, but I personally feel that’s more for early language development phase. If it’s been a certain way for a long time, it ought to be overwhelmingly obvious and overwhelmingly supported by the community to change it. And even then I feel a bit of doubt.
If you're referring to this PEP's "twin", i.e. "PEP 758 – Allow except and except* expressions without parentheses", that is not an opposing PEP. These two PEPs are orthogonal. One does not contradict the other. They are twins only in the sense that they are both about exception handling syntax.
They really are hard to work with compared to languages where you declare which exceptions you'll throw,. I always get angry when pylint raises an error for catching over-broad exceptions[0]
Of course I'm catching broad exceptions because I have no idea what kind of exception is going to be thrown 12 dependencies deep and I don't want it to completely crash the program instead of letting me retry or do something else.
100% agree. Most of my bare except: are followed by import pdb;pdb.set_trace() so I can figure out what went wrong and then fix my code so that it never happens again, but I still leave it there because I I don't have time to consider the millions of ways my hastily thrown-together python script is going to fail nor do I want to game out how many different errors could happen. If Python would have been this hard to use 20 years ago, I wouldn't have been able to learn to program.
Yup this drives me crazy. I've been bitten by urllib3 or SSL exceptions being bubbled up by random libraries so many times that now I always include an except Exception: block just in case.
The way I was taught Python, you really, really don't want to use bare `except:`, because it catches _everything_: Ctrl-C interruptions, system exit, etc. Instead, you really ought to use `except Exception:` (where `Exception` is the base class for any "normal" runtime error).
So I definitely understand the rationale, but it's hard to say it's worth the pain of backward incompatibility - we have linters, style guides, etc. that can catch this.
Yes, I was bitten by it in the past. Still, it'd better be a lint, or at least a very very long deprecation period... like, deprecated and removed in Python 4 or something.
I steers people who don't know better away in numerous ways, e.g. most editors with language awareness will give some kind of visual indication like a strike-trough.
One example of a time I used a bare except was when I wanted a program to retry three times if it failed for any reason. I just wrapped everything in a for loop with a catchall except.
The problem occurred when our scheduling program (Airflow) noticed the program was taking too long to run and decided to kill it. It sent a kill signal to Python, which dutifully caught the exception, retried and continued to run. I had to add a special case to allow Airflow to kill the program.
This PEP just forced me to look up the difference between the classes Exception and BaseException. It turns out that BaseException includes every exception, whereas Exception excludes those that are trying to exit the program (like SystemExit and KeyboardInterrupt).
With a bare except, your code will continue to retry even if SystemExit or KeyboardInterrupt is raised. This is almost always a bug.
In other words, your comment is an argument for the proposal!
I don't think it's a good enough argument to make a backwards incompatible change. This is a wart Python has to live with now. But I do think it's a shame that bare excepts behave in a way that is almost always a bug.
Maybe bare excepts could be modified to just catch Exceptions. It seems like a reasonable expression of the idea: everything that could go with my program but not with the OS.
Personally I think that would have been a better choice in Python's original design, but to change it now would be a backwards-incompatible change, i.e. it suffers from the same big problem everyone is highlighting in the PEP.
That seems less bad in the sense it would affect fewer people, but the ones it did affect would likely be much more strongly affected. For instance, I could imagine someone with an old daemon that had a too-level loop like:
while True:
try:
serve()
except:
log(‘oops’)
so that it was more or less bulletproof. This might be a highly unpleasant change for those people who counted on it running 24/7 and never dying.
In other words, the current behavior is a minor hassle for many people. That change would be a major hassle for a few.
I’d be all for a deprecation warning on bare excepts. That might nudge a lot of people to fix their code without actively breaking anything.
Thank you for being honest about making the exact mistake they are trying to help out with this.
I’ll restate what I posted elsewhere, I have never come across anyone who actually wanted to write a bare except, only people who thought naively that it worked like “except Exception” actually does.
I understand people are grabbing pitchforks because everyone has at o e time or another written code using a bare except and they are thinking that this will break it, but they 100% intended that to work like “except exception” python could just make that the default for the bare exemption and keep the syntax.
Depreciating the behavior is a good choice, and anyone who actually wants to capture exit events or keyboard interrupt can explicitly capture those.
If code breaks with this depreciation it’s because someone is doing something extremely out of the ordinary but isn’t being explicit about it.
"our users are idiots and we need to keep them away from sharp edges" is exactly what keeps driving me away from Python and pip. It's why I wrote https://pip.wtf -- Python package management would be so simple if they'd just stop adding more and more seatbelts and cushions to Python.
No kidding about pip. The dependency resolver change several years ago was a similar terrible move to the PEP being considered here. It broke so much legitimately working code for no good reason; just paternalism from the core devs. The change pushed my team to stop using pip at all for dependency management.
Hard disagree there. It was way too easy to get yourself into an incompatibility hell with the old resolver, where package A relied on transitive dependency X v1.2 and package B needed X v2.1. Which version of X you got depended on whether you installed A or B first.
Yes, the new version did mean I had to straighten out a few projects that were already working before, but they were working by coincidence because my code paths weren’t stumbling across the incompatibilities. The problem already existed. The new resolver just exposed it.
My case was different from yours. Our project wasn't working by coincidence, the dependency resolver was flagging incompatibilities that simply didn't apply to our case, and began refusing to build a stable working project. Yes it's more risky to override that kind of guardrail, that's why I would only do it when I know the risks and tradeoffs and determine it's the best course of action on balance. I strongly believe that tools should ultimately work for the user, over dogmatic principles.
I'm fine with the those safety guardrails being the default behavior, but removing any sort of escape hatch because the pip devs think that they know better than the users of the tool 100% of the time is what I object to.
In the end we ended up ditching pip entirely for this use case and ended up with a much better system, with absolutely no disasters as a result, but we had a burn a lot of time and angst that could have been spent on actual problems we were trying to solve.
> If you don’t want pip to actually resolve dependencies, use the --no-deps option. This is useful when you have a set of package versions that work together in reality, even though their metadata says that they conflict. For guidance on a long-term fix, read Dealing with dependency conflicts.
Not just considered in PEP 722 - the uv feature is just an implementation of PEP 723 [1] (PEP 722's successor/competitor), which was accepted. Other tools like pipx support it as well.
"seatbelts and cushions" is not how I'd describe a package manager that can run arbitrary code from the downloaded package when you explicitly tell it "please only download this", simply because it wants to verify that building it will result in it having name and version metadata that matches what you asked for (https://github.com/pypa/pip/issues/1884).
This is not fixed in 24.2 btw, even if you do everything according to the latest standards - you're still allowed and expected to have a setup.py if you choose Setuptools as your backend and you release an sdist with a non-trivial build step. 24.3 should be out some time this month and I'll be interested to see if they've finally done something about this issue, which has existed for almost the entire lifetime of Pip.
One must imagine Sisyphus happy. Python just loves to break working code on a regular basis with its new releases. If your code is protected from untrusted user data and the internet, Python 2 is actually a really nice language that doesn't constantly force rewrites.
Oh, you want to know the naive UTC datetime in Python, to interface with something like PostgreSQL that recommends naive times? Back in the old days, a simple datetime.datetime.utcnow(). Now days, you need something like:
try:
from datetime import UTC as tz_UTC
except ImportError:
from pytz import UTC as tz_UTC
dt = datetime.datetime.now(tz_UTC).replace(tzinfo=None)
> If your code is protected from untrusted user data and the internet, Python 2 is actually a really nice language that doesn't constantly force rewrites.
If Python 2 is acceptable for your use case, then you could stay on an old version of Python 3 just fine as well.
I think the point was making sure that code won’t break in the future. If you tell someone „use python 2 to run my script“, you know it’s going to work basically forever because the latest python 2 won’t be changed. That’s not true for python 3. I still think it’s a bad argument, but that’s what I understood the idea as.
That argument still seems inconsistent to me, since saying "use python 2, pin your dependences and never upgrade python so your script runs forever" is the same as "use python 3.12, pin your dependences and never upgrade python so your script runs forever".
Although 3.1 also meets that criteria. And even with python 2, the dependencies could still update. The main thing is that it's the not-updating that will make your code run forever (on any system, language, version), not the fact that it's python 2.
No you can't. For example I use a script to compress scanned PDFs by combining individually processed JBIG2 images and that script hasn't been updated for more than a decade: https://github.com/agl/jbig2enc/blob/master/pdf.py It works and generates perfectly good PDFs. No it doesn't work with Python 3 because it mixes bytes and strings copiously. I could spend half an hour upgrading it to work with Python 3 but there's no reason to.
Don't forget the whole reason why Python 3 exists is because it broke compatibility with Python 2. Plenty of old unmaintained scripts were forever stuck in Python 2. Not to mention an old version of Python 3 actually performs worse than Python 2.
Nothing is untrusted. I trust my own scanner. It's just that it produces files that are too large. And when it is told to reduce file size, it reduces resolution instead of using good compression.
> I could spend half an hour upgrading it to work with Python 3 but there's no reason to.
How about empowering people other than yourself to understand how the code works, rather than relying on them to decipher the precise way in which you "mixed bytes and strings copiously"?
What if someone else did it for you? Would you reject the PR on principle?
The code was not written by me. I merely found this code useful.
If the author, Adam, or someone else upgraded it I would be happy using it with Python 3. But the code works as is. I'm also happy using it with Python 2.
> Oh, you want to know the naive UTC datetime in Python, to interface with something like PostgreSQL that recommends naive times?
Postgres never recommended naive datetimes. A TZ-aware datetime is semantiacally the same as a tuple of (<location/agreed offset>, <time in the moment since unix epoch defined in terms of UTC>). Those who recommended dropping the knowledge of the first part from that pair did it because they didn't know better.
It's a perfectly fine strategy in some situations to only store UTC in the database and handle time zones on display. It's your database, you know what's in there. As an added bonus it allows easily flagging non-UTC timestamps as errors on some level to make sure you don't get tangled up in time zones.
> It's your database, you know what's in there. As an added bonus it allows easily flagging non-UTC timestamps as errors on some level to make sure you don't get tangled up in time zones.
I think you're mixing rendering (and for that matter - time ordering) with the initial information providing, that is a precise data input that doesn't rely on anything external.
Postgres ALWAYS stores timestamps in UTC [1]. When you submit data for persistence from your application, you have a choice of either:
- informing the DB about the recorded location/offset at the point of persistence (even if it's the zero offset) so that the input conversion happens without any reliance on the underlying OS-level or configuration setting [2]
- or omitting that information and therefore 1) losing precision at the point of data input and 2) delegating the location/offset inference for the internal purpose of Postgres to the external global configuration: both side-effects are bad from consistency/reliability perspectives.
I've never seen a single application that would win anything from (2) compared to explicitly providing the offset of the observed moment, even if the entire business domain is in UTC at all times. In fact, I've seen many business applications that explicitly record the offset in a separate column next to the provided timestamps for any future analysis and retrieval. And it's partly by design of the underlying abstractions: the unix epoch is defined in terms of UTC too, so when you design your program around implicit UTC you are not gaining much - the offset information is still there, it's just your data types don't make it clearly visible to everyone. But the moment you start integrating your data with the real world that cares about global time ordering of your recorded data events, you get a whole bunch of silent mistakes and issues that you won't have enough data to fix reliably until you switch to explicit offsets in all timestamp evaluations.
You're describing a world that doesn't exist from my experience writing booking systems.
Getting time zone conversions right all the way through the stack is far from trivial, which is why I prefer to do it in one place and keep the rest of the system out of it.
They explicitly describe the PEP as evil, is there a tradition in the Python community for having obviously terrible PEPs, just to document the reasons for not doing something? Because that would make this a lot more understandable.
If it's not a serious proposal then that's even worse, because there's a long discussion on that thread. So this non-serious proposal is just wasting people's time.
> I figure if I make this PEP, we can then ask Guido to quickly reject it, and then when this argument next starts up again, we can say ‘Guido isn’t changing things to suit the tab-haters or the only-tabbers, so this conversation is a waste of time.’ ...
> This proposal, if accepted, will probably mean a heck of a lot of work for somebody. But since I don’t want it accepted, I don’t care.
> Any literal composed entirely of M, D, C, L, X, V and I characters that does not follow this format will raise a syntax error, because explicit is better than implicit.
Good as a reminder that rules like “explicit is better than implicit” should not be followed all the way to the most absurd possible conclusions.
There are two "problems" this PEP is trying to solve.
One is that bare excepts are permitted. The argument against this is that explicit is better than implicit. A matter of taste, but I don't find this convincing.
The other problem is what bare excepts mean. Bare excepts are syntactic sugar for `except BaseException`. This means that an application containing a bare `except` followed by the vast majority of real-world error handling will continue to run even if SystemExit or KeyboardInterrupt is raised. This is almost always a bug.
I do find this second argument convincing, and I wish Python did not contain this design wart.
If I could go back in time and change Python syntax, it would make it hard for people to silently treat these special interrupts as "handleable" like regular errors. The tiny set of applications that really can and should handle them (e.g. TUIs or the mailman example discussed in the final section of the PEP) can explicitly do so with e.g. `except KeyboardInterrurpt` or even `except BaseException`.
But I agree with the consensus here that this does not rise to the level of something being worth a backwards-incompatible change.
Anyone reading your code is going to assume this is a bug. The PEP is right that explicit is better than implicit. You should write `except BaseException` (whether or not this PEP is approved).
"except:" is explicit enough and "except BaseException" is redundant.
Moreover I think there is a real risk people are going to write "except Exception:" instead, which breaks in the fringe case an exception that derives from BaseException (enforced by interpreter) but not from Exception is thrown.
Even if catch Exception is what users usually mean, changing code from "catch (BaseException):" to "catch Exception:" may break some code if improperly reviewed.
It's also not worth breaking production code over this.
> "except:" is explicit enough and "except BaseException" is redundant.
Take that up with the consensus view of the python community, as reflected by python linters in their default configuration, almost all of which warn on bare except.
The debate in the PEP is whether this should be a syntax error. The debate about whether it is good style is over though.
> It's also not worth breaking production code over this.
> Take that up with the consensus view of the python community, as reflected by python linters in their default configuration, almost all of which warn on bare except.
I don't fully agree on it being a purely a style issue (though the warnings from linters are wholly justified). From what I understand, "except:" is a code smell because you don't actually want to catch BaseException instead of just Exception.
Linters wouldn't have warned about it if it meant "except Exception:". The real issue, IMO, is the fact non-exceptions (Ctrl-C, generator exit, program exit code, etc.) have been crammed into the exception system.
>which breaks in the fringe case an exception that derives from BaseException (enforced by interpreter) but not from Exception is thrown.
For many users, in many cases, this would be fixing the code rather than breaking it.
Forcing people to write either `except BaseException:` or `except Exception:` means forcing them to think about which one they actually mean. This is a good thing, just like the enforcement of proper separation between bytes and text is a good thing.
Only if they don't understand what 'raise' means. It's obvious this construct is just injecting some additional information in a passing exception, there's no issue if it catches everything.
It will change exception class if logging function will fail... I wouldn't call this "good chance", those kinds of things are pretty unlikely in my experience.
Bare except + reraise is a very common Python pattern, so no, it won't be assumed to be a bug. This was actually one of the major points in the discussion of the PEP that led to its rejection.
Just noting it here: your code is incorrect. In case of a KeyboardInterrupt error and another error raised by `log_tons_of_debug_info()` (there's no error free code, right?), KeyboardInterrupt would end up being masked (it would go into the __context__ attribute of another error). The program won't abort its execution. And it's just one example out of many where it's critical to not mask error types.
Correct code would be:
try:
something()
except BaseException as ex:
try:
log_tons_of_debug_info()
finally:
raise ex
But really, you don't want to mess with BaseExceptions at all, so just do `except Exception` instead of a bare `except:`.
Why wouldn't I want to mess with BaseExceptions? They are not magic, and add only 3 classes to the list:
SystemExit - You _definitely_ want to catch this one for logging. If a library (not top-level app) calls `sys.exit` you at least want to know what's happening, if anything so you can talk to author and get them to use proper exception types.
KeyboardInterrupt - I normally want to catch this one as well. If the program was taking too long and I hit Ctrl-C to stop it, I _do_ want to see all the debug output. And if I don't, for some reason, there is always Ctrl-\ which kills python immediately and unconditionally.
GeneratorExit - this one is tricky and I agree that in a lot of cases, you don't want to print logs on it. But it also very rare - it only appears in async functions (emitted by yield), and never propagated to caller. So as long as you are not doing async, you can simply ignore it, which covers majority of the the code I write.
Because accidentally masking some BaseExceptions like `asyncio.CancelledError` can lead to things like memory/resource leaks and potentially your production app going down in pretty hard to debug ways.
> Just noting it here: your code is incorrect. In case of a KeyboardInterrupt error and another error raised by `log_tons_of_debug_info()` (there's no error free code, right?), KeyboardInterrupt would end up being masked (it would go into the __context__ attribute of another error).
Your snippet has the opposite problem: if ex is a regular Exception, but then KeyboardInterrupt is raised by `log_tons_of_debug_info()`, then your snippet would mask that by re-raising ex in its place. I think the original snippet is probably better on balance, even ignoring difference in code complexity.
Java solved the problem by having Throwable as the root of all exceptions and not advertising that fact loudly. The derived Exception class is the root of all safely catchable exceptions. When someone catches a Throwable, something strange is going on.
> Bare excepts are syntactic sugar for `except BaseException`.
I'm guessing that a `3to4` script would be provided which replaces bare `except:` with `except BaseException:`. We have the experience of `2to3` to draw on with regards to how that might play out.
EDIT: Haha, I now see that this PEP proposes a change without advancing the major version. That surprises me.
I think Rich Hickeys advice of not breaking people applies here.
The anger from this potential change is that really all you are doing is taking something away that was working, and now people will need to review their code or keep python on a previous version which sucks.
I think that people who propose these kinds of changes don't appreciate the importance of the programming language being at the bottom of the stack so there's really never a good reason to break people even if you think it's nicer as you really can't appreciate how much work you are creating for people.
One painful one that is still reverberating a bit in some areas is the renaming of "SafeConfigParser" to just "ConfigParser" in the standard library (in 3.12). This caused a whole lot of breaking in some areas because versioneer (a package for determining a package version from git tags) used it (in code that was placed inside your package, and so couldn't be solved by just upgrading versioneer).
Python 3.7.9 (default, Aug 23 2020, 00:57:53)
[Clang 10.0.1 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cgi
>>>
Python 3.13.0 (main, Oct 8 2024, 01:04:00) [Clang 18.1.8 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cgi
Traceback (most recent call last):
File "<python-input-0>", line 1, in <module>
import cgi
ModuleNotFoundError: No module named 'cgi'
IMHO, working on a programming language is only for some, just like working on a database is only for some. The first rule should be: "you should never break the language, ever". Just like you should never break the database or kernel behavior.
This is why I like to stick to C, Common Lisp, Clojure, and (to some extent) Java/JVM. I don't know about Clojure's future, but C and Common Lisp have been fine for the last 40 years, and I'm not expecting the least in the upcoming years.
I half agree with this rule. I think that it's fine to break things as long as you make a semantic version change _and_ provide automated tooling for upgrading old code. If you can't build this tool, that is a strong negative signal for both versions of the language.
What I don't like about say, c, is that it has various backward compatible additive dialects like c11 vs c99. I personally don't agree that c11 and c99 are the same language in spite of the backwards compatibility and I think it makes the entire ecosystem worse. At some point there needs to be a successor rather than just piling on to old broken designs. I would prefer a better FFI or other tools to interface with legacy code in the new dialect.
> I half agree with this rule. I think that it's OK to break things as long as you make a semantic version change _and_ provide automated tooling for upgrading old code. If you can't build this tool, that is a strong negative signal for both versions of the language.
I'd be happy to find automated tooling for upgrading old code that actually works. Python 2to3 converter worked on simple code but would fail on anything mildly complex (what would that tell about the Python language then :P).
IMHO, breaking language could makes sense if you have built-in runtime (like golang) or you ship native binaries that depend solely on kernel or libc (C/C++). You build your program and ship it to the clients. But, when you have a runtime like Python, you must also force clients to upgrade it. And the problem with Python is that you can't easily do that because system stuff usually depends on a particular version (yum/dnf, ubuntu services, and so on). And, unlike Java, the Python community only advertises a little about having multiple Python versions installed simultaneously.
> What I don't like about say, c, is that it has various backward compatible additive dialects like c11 vs c99. ... At some point there needs to be a successor rather than just piling on to old broken designs.
This is why people are still using C. There are many successors with varying success (D, C++, Rust, Zig...) and much more added complexity to the language and runtime. Language simplicity, compilation speed, and compiler simplicity - all of that was lost.
That's funny, I code a lot of python but I don't participate in any large projects where a pipeline might fail due to this. So I don't follow the PEP news much.
And yet I have created my own habit of not using bare excepts. TIL what bare except even means, but I do not use them. Simply because I think it makes more sense to specify the exception I want to catch, and failing that I specify the base class Exception.
So I guess I understand the author of this PEP, we're of one mind on this. :)
Also thanks to this post TIL that a bare except might catch interrupts like ctrl-c. Even more justification for my new habit.
The proposal (not part of PEP 760) to add an implicit `raise` to the end of bare `except:` blocks would be far more damaging than making bare-excepts a syntax error.
> Now there’s an interesting idea: don’t make bare except illegal, make it have an implicit raise at the end (and disallow return, break, and continue).
...ahh, a bit of a misreading on my part. However, I was really looking for your explanation of the horrors that could happen rather than what the suggestion was. Thinking through a bit more, yeah, implicit re-raise does have some pretty bad outcomes if you can't change code in a dependency.
It still feels like `deno` is somewhat on the right track where permissions are dropped by default, but It'd Be Nice(tm) if programming languages enabled that a bit easier.
import sales_tax_calculator as xyz with [ cap.NETWORK, cap.FILESYSTEM, cap.USB, ...etc... ]
xyz.calculate( sales_price, state=user.address.state )
We're implicitly allowing imported libraries the full power of the containing programming language where with promiscuous code sharing (trending towards a low-trust environment), it'd be a lot better to _not_ give `cap.FS, NETWORK, USB, etc...` by default.
Bringing it back around: `import somelib with [ cap.BARE_EXCEPT, cap.RAISE ]` or something to control their handling of "unknown" exceptions is interesting. Let them handle any exceptions or interrupts they've authored, but let me explicitly have control over catching stuff that isn't "from them".
...an extended version of dependency injection or inversion of control.
Adding an implicit raise to the end of a bare-except would quietly break things, and is non-trivial to detect. Say you have a naive base-except:
def loop():
try:
check_service()
except:
logging.exception("Error while checking service.")
time.sleep(60)
Really you shouldn't be using a base-except here. You should at the bare minimum catch `Exception`. Adding an implicit `raise` at the end will break this function without so much as a warning. Instead of calling the function every minute, the loop is broken with an unexpected exception that was deliberately suppressed (and logged).
A more common scenario for myself is write a lot of my scripts in the style:
An implicit raise would obnoxiously break them when my bare-except is intentional, and effectively cause the error to be printed twice to the terminal. Now I'm not wholly opposed to forcing `except BaseException:` instead of `except:`, but an implicit raise would cause all sorts of subtle bugs.
I'm not categorically against it, but it needs to go into Python4, not a minor revision. It breaks too much.
(Plus, the suggested transition easement, "A tool will be provided to automatically update code to replace bare except: with except BaseException:", indicates a fundamental flaw in this approach: it's still trivial for developers to catch-and-kill exceptions. So we're strictly increasing the verbosity of the language without actually solving the problem. :( ).
Sane BDFL would have already stepped in with "Absolutely not" and PEP closed with "Will not implement"
EDIT: Add on, Private Email to Pablo going "Dude, why are you purposing stuff that will break code spectacularly? I think we need to talk about your approach to language design."
They are literally trying to depreciate a feature that regularly breaks user space because it’s unintentionally misused. I don’t think I’ve ever come across a use case where people actually intended to write a bare except. Anyone who ever does this are actually thinking that it’s effectively the same as “except Exception” which it isn’t, but that’s more obvious to most users.
397 comments
[ 1.4 ms ] story [ 427 ms ] threadIn other languages, such a change would only be possible with a major version bump, though I imagine that because of the Python 3 collective trauma, the language designers now are OK with breaking older code without calling it Python 4. Anything goes, as long as it's called Python 3.
(Python lost me in the 2->3 migration and I haven't used it in a decade, so correct me if I'm wrong)
We may never see a 4.0 because of the scar tissue, but the language continues to evolve.
Python: for the love of God [1] don't touch the version
I wonder if the scar tissue will ever heal and we'll see a Python 4 in two decades.
1: https://www.youtube.com/watch?v=asUyK6JWt9U
They should do the opposite really. If it hurts, do it more often and get better at it. A perfect time would be when Python gets some nice JIT performance improvements which everyone will probably like.
(Recent stdlib changes have been much more destructive, but I’m assuming that, like in the original thread, we’re drawing a distinction between those and changes to the actual language.)
Full disclosure, I welcomed Python 3, because for me that was the first time (since 2.4 on Windows XP) that I could count on my programs not randomly shitting their pants upon encountering Cyrillic in files or filenames, which for a native speaker of Russian you can imagine is quite important. (The csv stdlib module in Python 2 did that, IIRC. Perhaps I was holding it wrong, but experience shows that absolutely everybody did.)
[1] https://peps.python.org/pep-0594/
Yes, indeed it would.
> This change is not backwards compatible. Existing code that uses bare `except:` clauses will need to be modified. To ease the transition:
> * A deprecation warning will be issued for bare `except` clauses in Python 3.14.
> * The syntax will be fully disallowed in Python 3.17.
> * A `from __future__ import strict_excepts` will be provided to invalidate bare except handlers in earlier versions of Python.
> A tool will be provided to automatically update code to replace bare `except:` with except BaseException:
First, consider this motivation from the PEP:
> Requiring specific exception types makes the programmer’s intentions clear and encourages thinking about what exceptions might occur.
This reeks of the same rationale for checked exceptions in Java. That was a failed experiment. You can't force people to deal with exceptions. They end up just swallowing them instead. It's better to propagate an exception than do that almost all the time.
Like will we see except: replaced with except object:? I don't even know if that's valid. It's never come up.
Second, this would be a breaking change. I really feel like this is where Python 3 went off the rails. In the Python 2 days making breaking changes was essentially verboten. But ever since Python 3 decided breaking changes were OK< it's like there's little restraint now and minor releases seems to be far too comfortable with this.
Going back more than a quarter century ago (!), I was one of those zealots in favor of checked exceptions in Java. It took a good 15 years to finally conclude the “checked” part is more trouble than it’s worth.
Given that there is inevitably a root context for any given language invocation (main() for a command li program, top of a thread for a multi threaded app, etc), unchecked exceptions and a top level handler have proven to be more than enough.
I won’t comment on Python backwards compat, will just shake my head.
I don’t understand the idiomatic viewpoint either here, I understand the author personally finds it confusing when excepts aren’t verbose but I think you would be hard pressed to find many python developers who agree. Even outside the ecosystem, most languages have been adding more support for bare excepts (like js with bare catch) so this feels like a step backwards.
But maybe I’m just not understanding this proposal!
I'm hoping Python 4 will be a big breaking change similar to the previous one and full support for explicit types will be one of the reasons.
Edit: I don't get the hate for "i" or "j" as increment variables. When you're working with numerical data they closely match how you would express the same operation with mathematical notation and are idiomatic to the point that everyone with non-trivial experience in programming knows what they represent. There are better options in some cases (e.g. "for name in names:"), but there's nothing inherently wrong with i, j, k, etc.
I flag nearly all abbreviations in code reviews because code is meant to be read and the names of things should be clear.
That's a nice compromise where you need to learn a few core concepts, but the code itself is easier to scan for bugs in a lot of places.
* x, y, z (coordinates)
* r, g, b, a; c, m, y, k; y, u, v (colors)
* i, j, k matrices and such
* p, i, d; Kp, Ki, Kd in process control
* rv (return value), where it is clear what The Thing To Be returned is.
* common variables from eg physics
Generally because these are well known and defined in particular contexts.
I don't always make it, but sometimes an argument can be made that single letters are better known/recognizable by their single letter name.
Compare:
* total_energy = mass * light_speed_constant ^ 2
vs
* E=m*c^2
Or:
* process_control_setting = proportional_component + integral_component + derivative_component
vs
* q=p+i+d (for those of us who know PID control)
Upsides: consistent vocabulary in your project, a centralized place to look up jargon, and a subtle friction to avoid adding new terms that people might not immediately grok.
I wish I could remember the source; because it sounded like a nice setup to steal.
"Why don't you write out 'process_id'?"
"Because the people who maintain the sort of code that cares about pids all refer to it as 'pid'."
Most people just renamed variables like i > ii, which was worse.
For example: https://clang.llvm.org/extra/clang-tidy/checks/readability/i...
Python3 did not break enough to justify the jump from 2 to 3, IMHO.
>>> and full support for explicit types will be one of the reasons.
Fair point!
I don't know how much more breakage you really wanted out of Python 3 if "permanently scoured the public image of the language and caused many Python shops to, at least partially, stop being Python shops" wasn't enough.
(I say this as a still-fan of Python who has written quite a lot of it and contributed to MicroPython/CircuitPython's internals - I just also worked at a Python shop during the Py2->3 hell, and frankly, even my current dayjob still talks about that transition as a nightmare to watch out for if any other language starts doing similar talk.)
* Beginners don't introduce ACE exploits into their program from the get-go, because `input` is what `raw_input` was before and the "convenient" wrapper using `eval` was no longer available - so instructors were forced to teach students about explicit type conversions, like they should have been doing the whole time.
* You can't get `UnicodeDecodeError` from calling `.encode`, or `UnicodeEncodeError` from calling `.decode`, because you're never in the position of pretending that a sequence of bytes is a "string". There's no illogical "basestring" type and no `str` objects with ambiguous semantics - `.decode` and `.encode` do what they say, and belong to separate types. These problems resulted in a huge mess of duplicate Q&A on Stack Overflow full of incorrect advice from people with no clue what they were doing, and empowered people to ignore essential truths about Unicode until it blew up into a bigger problem.
* Similarly, when you read from a text file, you actually get text now.
* `print` used to be confusing and have weird ambiguities and tons of special syntax. Being a function means it can be taught the same way as any other function call; plus you get use as a higher-order function, unpacking operators (you can't do `print(*range(10))` in 2.x).
* `1000000000 in range(1000000000)` no longer hangs. `xrange` does not solve this problem (although it does improve matters quite a bit by avoiding high memory usage).
* Sorting heterogeneous lists correctly produces an error, rather than an ordering so strange as to merit its own Stack Overflow Q&A (https://stackoverflow.com/questions/3270680 and many duplicates). (You can still replicate the old order if you want.)
* `isinstance(x, int)` doesn't fail because of `x` being too large. This was a bizarre speed bump in such a high-level language.
* You can write '£' in your source code and you only have to declare an encoding if you use something other than UTF-8 (which Python correctly identified as the eventual winner).
When Python 3.2 came out I found it to be a breath of fresh air. By 3.4 I was already starting to wonder why so many others were dragging their feet on migrating.
I agree - it should have broken more (and thereby become able to fix more). It should also have been usable out of the gate and not majorly reworked (there was a serious battle over the syntax of byte and string literals, and possibly some other things, that resulted in 3.0 and 3.1 not seeing a full 5-year maintenance lifetime), and of course developers should have actually fixed stuff promptly and accepted that the obviously superior new ways of doing things were, in fact, obviously superior.
Unfortunately, a lot of other developers don't seem to agree.
And as for explicit types - I really wish people would stop trying to fight the type system - of Python, and of whatever other language. Python is not meant to support static types. It's not designed to reject your code at compile time for a type error and it isn't designed to take type information into account when generating bytecode. It's designed, instead, very explicitly, to let you care about what a given object can do, rather than about how it categorizes itself.
Python should be working on being the best Python it can be, not being an adequate Python and an inadequate bodged-on static language on the side.
Does the interpreter actually optimize code based on type information? My knowledge is that it does not
The problem Python 3 has is dynamic types, as the dynamically-typed languages implement them, are viral; one little crack lets them in somehow and all the code operating on the data has to assume it's viral.
[1]: https://mypyc.readthedocs.io/en/stable/
[2]: https://mypyc.readthedocs.io/en/stable/differences_from_pyth...
I'm hoping "Python 4" will be another language entirely that displaces it, Fixing some ecosystem problems upfront (packaging, concurrency/GIL). Nim is possible candidate, though Go is pretty popular w/ Pythonistas.
My personal unhappiness with how Py3K was handled, plus recent PSF events make me feel new leadership would also be a boon..
I think the ecosystem leadership position Python finds itself in lately may well make Py-core silly enough to think "our users will tolerate our breaking-all-their-code antics no matter what". I suggest a more consenting-adults alternative here: https://news.ycombinator.com/item?id=41790766
In fact, I'd been thinking about doing it since probably November of the previous year - and it's motivated by long-held beliefs that
Python's design falls short of the Zen;
several of the most common beginner pitfalls could and should be avoided by syntax changes and by a different approach to parsing the code; and
the standard library is full of ancient APIs that look terrible because they're motivated by C and Java designs that used to be the best we had but seem highly unidiomatic now.
It was somewhere in November or December last year that I first wrote those thoughts down more concretely (with details about what those pitfalls are etc.).
Given that pace of progress, however, I've more or less given up on ever expecting myself to publish something usable - by myself, at least. I've decided for now that it will be more practical to add blog posts about my ideas to the queue, and possibly see about my own implementation later.
If you're going to escape from Python, I think you need something more like Plissken.
History shows that's a good idea if and only if you use a new name for the new language.
cf Python 2 -> 3, Perl 5 -> 6 -> Raku, Javascript -> Typescript
You can keep Python as part of the name. Call it SuperPython or something. Just don't call it Python 4.
This is actually a good thing in some cases (possibly this one). Risky stuff should inherently be harder to do than safer stuff, otherwise people will reach for the risky alternatives when they don't need to, just to save time - or because they don't realize the risk.
Or at least, that's often the case. What's lacking here is evidence that this is actually happening. I can believe it, but evidence is necessary for breaking the language.
To me it stinks of an ego-centric person thinking they're a "language developer" and knowing better than the actual users of the language what's best for them. Just because something can be misused doesn't mean you have to take it away.
I haven't noticed, but since Rust came along is there a trend among languages to enforce "safer" programming at the language level? I could see that kind of thinking getting way out of hand. If that's the case, I would see this one as "I'm going to save the world with this dumb little change that breaks things for a bunch of people!"
I would hope a PEP like this came about from frequent user requests but that doesn't seem to be the case.
so that people can opt-in or opt-out selectively, per their own discretion, for these kinds of rules
I dont see the point of this being part of the language since it break compat and brings zero benefits at runtime
https://www.flake8rules.com/rules/E722.html
I mentioned Rust because the memory safety guarantees are a significant new thing for a language like that. I forgot about "managed" languages like C# because that's quite far from my mind, but that's another significant attempt at safety. This kind of little detail in the PIP is really insignificant by comparison, so I was speculating that it might be driven by some kind of "save everyone" mentality. If so, wondering if that's a trend lately and I hadn't noticed.
In 2016, you could call your function `fn async(...) { ... }` without any issues, and you can still compile this code with the modern version of rustc. If you want to use the async features of Rust, you need to change your edition to at least 2018, which brings breaking changes in the language.
And you can have a project that mixes those editions, and cargo will correctly use the edition asked for each crate.
(So, I guess the ideal world would to first look at the package management in Python, and *then* try to introduce breaking changes. And I'm withholding how much I'm angry at the PSF for postponing having a decent package manager for 20 years and investing in removing the GIL and adding JIT.)
All the more reason to take a breakage very seriously. This is even worse than the walrus operator. At least I can ignore that. This breaks working code for some notion of purity.
[0]: https://docs.python.org/3/library/__future__.html
I'm curious - if you had a `pub fn async(...){...}` in some 2016 crate, can you still call it from a 2024 codebase?
[0] https://en.wikipedia.org/wiki/Object-Oriented_Software_Const...
That's why I put "language developer" in quotes.
It's the enforcement by compiler, which will break lots of existing code, that makes people unhappy.
Python is not APL. Getting at the functionality in fewer characters is not a design goal - it's just a usually consequence of the actual design goal.
This is being considered because "Explicit is better than implicit." and because it helps avoid a common class of error (e.g. `except: continue` prevents aborting a loop with Ctrl-C, which is often not intentional).
Or as the PEP puts it:
> While this syntax can be convenient for a “catch all” handler, it often leads to poor coding practices: > > 1. It can mask important errors that should be propagated. > 2. It makes debugging more difficult by catching and potentially hiding unexpected exceptions. > 3. It goes against the Python principle of explicit over implicit.
Python isn't any other language, either. It certainly isn't taking design guidance from JavaScript (which runs in an environment where the page is expected to show something coherent, and not a loud screaming error, no matter how absurd the input data and/or code).
As for how much code it would break, you made me curious:
But drilling down further, over 2/3 of those bare excepts are in local copies of Python itself (i.e., multiple versions of the standard library) that I built from source. Probably all the rest are in dependencies. I don't write code like that myself if I'm even remotely paying attention. (Of course, that only tells me how many occurrences there were, not how many files have them. But the first two results imply an average of about 2 `except`s per file, so.)If this pep were implemented I suspect it would result in forcing thousands of not tens of thousands of people to spend hours modifying perfectly working code and destroying the ability to run old scientific code without modification. Extremely effective industrial sabotage if it were to be accepted.
It is hard for me to articulate how much peps like this reinforce my desire to never start another python project. Even if this pep is rejected the fact that there are people who would put in the time and effort to write and submit such a PEP tells me that they will do it again, and eventually they might succeed.
On the other hand, I completely agree that it's not worth a breaking change.
Even better/worse, could do it with regex on text streams.
> A tool will be provided to automatically update code to replace bare except: with except BaseException:.
I know it is not nearly on the same level, but people seriously overestimate the effort needed between not doing anything at all and even the slightest work, no matter how reliable and easy. The difference between nothing and anything is huge.
I have sometimes not run a Python script for a few years, and then when I need it, it stopped working and I need to track down what changed/broke or run some tool or whatnot. I don't keep track of the latest greatest Python changes – like most Python programmers it's not my "day job" to write Python code so I now need to track what changed between "the Python version I used about 3 years ago, whatever that was" and now. It's pretty annoying.
And that's assuming said tool will be fool-proof. Never mind of course that all my dependencies (if any) will need updating too.
What will happen in practice is that people will write "except Exception:" rather than "except:" and do nothing different. Basically nothing will change.
Meanwhile, I have C and Go programs that have worked without modification for about 10 years. Not that nothing ever breaks in C or Go, but it's the exception (hah!) rather than the rule.
I’d love to know whether that’s true, and to what extreme. I believe you’re right - that people using Python for a few hours a week (or less) greatly outnumber software developers using it as their primary language.
I think that’s a real issue for the evolution of Python, because updates to the language design (e.g. the makeup of the Steering Council) come almost entirely from the second group.
Yes I agree, and it's disappointing to see some take such a narrow view of things.
A big part of maintaining and evolving a language is saying "no" a lot. There are a lot of people with ideas, almost always reasoned from their own use-case. That's okay, everyone does that to some degree, but there almost always trade-offs and such to consider.
Your job as Steering Council or Core Dev or BFDL or whatever governance you have is to consider all use cases and make a balanced decision. Reading that thread, some do. But unfortunately others don't.
Even for Python 3, Guido spent most of his time saying "no" to proposals. There were a lot of pretty wild ideas for Python 3000.
I propose a rider to the PEP in which implementation will be deferred until its proponents can correctly affirm that the library reference lists, for each function and method, every exception it might throw.
No, the goal is a system in which the code correctly indicates which exceptions it's intended to handle, and doesn't accidentally handle the wrong exceptions because the developer was either lazy or misinformed about the semantics (perhaps due to experience with a different programming language).
What support does Python offer, to the author of a function, for determining what would be the wrong exceptions for it to handle? How, in Python, does the author of a function signal, to the functions she is calling, which exceptions they should not handle? As I alluded to in my previous post, the Python language and documentation does not even give programmers a good accounting of what exceptions the library functions might throw.
Other languages, notably Java, have tried the "functions document what they can raise" idea. Everyone seems to hate the result.
If the change goes through, couldn't you just use older Python versions for those specific projects, or has the Python ecosystem still not figured out how to do this without huge hassles?
How could the people maintaining Python possibly avoid that? It would be up to Apple to proactively reach out to affected projects, if Apple cares about that.
Python is open source. Nothing prohibits you from forking the 2.7 codebase and adding your own security patches (or more substantial things like back-porting new OpenSSL support, or even cherry-picking backwards-compatible features from 3.x that you do like), for example.
I'm happy when people criticize new features in Python. But I expect to read criticism of features based on their actual merits and consequences, not on the principle that it's new or backwards-incompatible or would cause "churn".
Then we have people saying "just don't upgrade anything if you don't want to deal with the churn" but that too is a very large burden for all the reasons I mentioned.
All the reasonable adults in the room want option 3: don't do the thing that causes needless churn.
> But I expect to read criticism of features based on their actual merits and consequences, not on the principle that it's new or backwards-incompatible or would cause "churn".
I am honestly shocked to see this attitude. Churn is a very real consequence. Do you have no respect for the time of the countless devs who use python?
It's bizarre to me that expecting people to improve their code incrementally is considered a disrespect for their time.
The proposed change is not arbitrary - which can be seen by trying to imagine the alternate universe in which the reverse change were proposed. One can imagine a world in which `except:` were added to a Python that didn't support it, but certainly not one where it were made mandatory (whether for the `except BaseException:` case or the `except Exception:` case).
I assume there are people out there who would, similarly, argue that it was "needless" to make `print` into a function (and thereby break users of the `>>file` syntax). But it demonstrably and significantly makes the language better.
Imagine whatever universe you want, but we live in this one. In this universe, there is a shitload of old python. Python 2.7 isn't even completely gone. Literal lifetimes of collective time would be spent on the fallout of this change, which does not seem worth the reward.
I completely understand this sentiment. Recent python events have made me wonder if there are some people intent on sabotaging the management of the language.
I loved the incremental improvements and thoughtful process involved up until a couple of years ago but it feels like python will become brittle and break badly if things continue the way they are. It feels like the adults have been driven out the room when it comes to stewardship. I'm not sure how recoverable the situation is.
The 'pipe' is hacky, for example, because it's just syntactic sugar that only works in one specific case, and not in general.
To be fair, the notion that pipes are lower-priority than other syntax needs is not exclusive to Python: in the JS world, discussion in https://github.com/tc39/proposal-pipeline-operator and specifically https://github.com/tc39/proposal-pipeline-operator/wiki/Bike... has been going on since 2018, with things like Tuple Literals taking precedence.
On the Python side, though, at least you can build your own pipes! You can define various helper classes that have, say, an `__rrshift__` method, to let you do the following with full type-checking support:
load_iterable_from_db() >> to_dict_by("id") >> tee(logger) >> call_(dict.values) >> to_dataframe
(With great apologies to FP folks who see a bind operator, and C++ folks who have seen enough operator overloading for a lifetime!)
Not necessarily something you want to use unless you want to confuse your team, but quite useful for fluent code in notebooks!
- - - -
Oh! How could I forget!? The creeps actually banned Tim Peters!
I was, in a sense, in that camp at the time, before I looked it up. I felt that the operator went against the spirit of the language by trampling on what was previously a strong, and clearly very conscious, separation between statements and expressions. And I misguidedly imagined, and lamented, that GvR was unable to keep it out of the language, being overruled by consensus.
I just want to make sure it's clear that things are not like that. Rather, the Python envisioned (nowadays, though not originally) by the original leadership includes PEP 572 - and probably also the large majority of what's been added since.
It's not the "walrus operator" per se that's the problem, it's the change in project governance.
Python has been captured by bureaucracy and corporate interests. The way it's being improved-to-death is a symptom. The unceremonious eviction of Tim Peters indicates to me that the take-over is complete: the new guard feels comfortable throwing out the old guard, they expect that their power is such that no one will blink, and no one did. (No corporation withdrew support for them, the blow-back was all hot air.)
It would be interesting to know GvR's opinions of some of the new cruft, but it's not really relevant.
The reason I pick up Python for projects is because it grows with the application; opportunities to add typing etc. Who knows maybe in a few years Python will enforce all the types and it will be as verbose as Java. Personally I’d like to see how they handle declaring a method or function throws exceptions.
Pretty narly we have compiled Python apps with poetry, it’s starting to punch out of its weight class.
TBH that kind of code barely survives minor Python version upgrades in my experience.
Caught and not re-raised
Like, I don’t know if there’s a PEP to use braces, but it wouldn’t surprise me if someone had made one so that from then on there’d be an official doc you could point people at when they ask about it.
Not saying this is one of those, and I see Brett Cannon’s on this one. I am saying not to get too worked up over the existence of a draft PEP.
The rejection of braces isn't in a PEP to my knowledge; it's only in the __future__ Easter egg.
How do people land on the python steering council exactly?
https://news.ycombinator.com/newsguidelines.html
I used to think it’s likely that you can make a much stronger case for one side, but I personally feel that’s more for early language development phase. If it’s been a certain way for a long time, it ought to be overwhelmingly obvious and overwhelmingly supported by the community to change it. And even then I feel a bit of doubt.
Of course I'm catching broad exceptions because I have no idea what kind of exception is going to be thrown 12 dependencies deep and I don't want it to completely crash the program instead of letting me retry or do something else.
0:https://pylint.readthedocs.io/en/stable/user_guide/messages/...
The way I was taught Python, you really, really don't want to use bare `except:`, because it catches _everything_: Ctrl-C interruptions, system exit, etc. Instead, you really ought to use `except Exception:` (where `Exception` is the base class for any "normal" runtime error).
So I definitely understand the rationale, but it's hard to say it's worth the pain of backward incompatibility - we have linters, style guides, etc. that can catch this.
All features of a language end up used, regardless of deprecation state.
The problem occurred when our scheduling program (Airflow) noticed the program was taking too long to run and decided to kill it. It sent a kill signal to Python, which dutifully caught the exception, retried and continued to run. I had to add a special case to allow Airflow to kill the program.
This PEP just forced me to look up the difference between the classes Exception and BaseException. It turns out that BaseException includes every exception, whereas Exception excludes those that are trying to exit the program (like SystemExit and KeyboardInterrupt).
In other words, your comment is an argument for the proposal!
I don't think it's a good enough argument to make a backwards incompatible change. This is a wart Python has to live with now. But I do think it's a shame that bare excepts behave in a way that is almost always a bug.
In other words, the current behavior is a minor hassle for many people. That change would be a major hassle for a few.
I’d be all for a deprecation warning on bare excepts. That might nudge a lot of people to fix their code without actively breaking anything.
The PEP proposes a deprecation timeline for exactly this.
I’ll restate what I posted elsewhere, I have never come across anyone who actually wanted to write a bare except, only people who thought naively that it worked like “except Exception” actually does.
I understand people are grabbing pitchforks because everyone has at o e time or another written code using a bare except and they are thinking that this will break it, but they 100% intended that to work like “except exception” python could just make that the default for the bare exemption and keep the syntax.
Depreciating the behavior is a good choice, and anyone who actually wants to capture exit events or keyboard interrupt can explicitly capture those. If code breaks with this depreciation it’s because someone is doing something extremely out of the ordinary but isn’t being explicit about it.
A bare except is something to be flagged up by tools, not disallowed by the language. It is definitely not worth a backward-incompatible change.
I am slowly going off Python.
Yes, the new version did mean I had to straighten out a few projects that were already working before, but they were working by coincidence because my code paths weren’t stumbling across the incompatibilities. The problem already existed. The new resolver just exposed it.
I'm fine with the those safety guardrails being the default behavior, but removing any sort of escape hatch because the pip devs think that they know better than the users of the tool 100% of the time is what I object to.
In the end we ended up ditching pip entirely for this use case and ended up with a much better system, with absolutely no disasters as a result, but we had a burn a lot of time and angst that could have been spent on actual problems we were trying to solve.
> If you don’t want pip to actually resolve dependencies, use the --no-deps option. This is useful when you have a set of package versions that work together in reality, even though their metadata says that they conflict. For guidance on a long-term fix, read Dealing with dependency conflicts.
Did that not work?
[1]: https://peps.python.org/pep-0723/
This is not fixed in 24.2 btw, even if you do everything according to the latest standards - you're still allowed and expected to have a setup.py if you choose Setuptools as your backend and you release an sdist with a non-trivial build step. 24.3 should be out some time this month and I'll be interested to see if they've finally done something about this issue, which has existed for almost the entire lifetime of Pip.
Oh, you want to know the naive UTC datetime in Python, to interface with something like PostgreSQL that recommends naive times? Back in the old days, a simple datetime.datetime.utcnow(). Now days, you need something like:
If Python 2 is acceptable for your use case, then you could stay on an old version of Python 3 just fine as well.
Don't forget the whole reason why Python 3 exists is because it broke compatibility with Python 2. Plenty of old unmaintained scripts were forever stuck in Python 2. Not to mention an old version of Python 3 actually performs worse than Python 2.
How about empowering people other than yourself to understand how the code works, rather than relying on them to decipher the precise way in which you "mixed bytes and strings copiously"?
What if someone else did it for you? Would you reject the PR on principle?
If the author, Adam, or someone else upgraded it I would be happy using it with Python 3. But the code works as is. I'm also happy using it with Python 2.
Postgres never recommended naive datetimes. A TZ-aware datetime is semantiacally the same as a tuple of (<location/agreed offset>, <time in the moment since unix epoch defined in terms of UTC>). Those who recommended dropping the knowledge of the first part from that pair did it because they didn't know better.
I think you're mixing rendering (and for that matter - time ordering) with the initial information providing, that is a precise data input that doesn't rely on anything external.
Postgres ALWAYS stores timestamps in UTC [1]. When you submit data for persistence from your application, you have a choice of either:
- informing the DB about the recorded location/offset at the point of persistence (even if it's the zero offset) so that the input conversion happens without any reliance on the underlying OS-level or configuration setting [2]
- or omitting that information and therefore 1) losing precision at the point of data input and 2) delegating the location/offset inference for the internal purpose of Postgres to the external global configuration: both side-effects are bad from consistency/reliability perspectives.
I've never seen a single application that would win anything from (2) compared to explicitly providing the offset of the observed moment, even if the entire business domain is in UTC at all times. In fact, I've seen many business applications that explicitly record the offset in a separate column next to the provided timestamps for any future analysis and retrieval. And it's partly by design of the underlying abstractions: the unix epoch is defined in terms of UTC too, so when you design your program around implicit UTC you are not gaining much - the offset information is still there, it's just your data types don't make it clearly visible to everyone. But the moment you start integrating your data with the real world that cares about global time ordering of your recorded data events, you get a whole bunch of silent mistakes and issues that you won't have enough data to fix reliably until you switch to explicit offsets in all timestamp evaluations.
[1] https://www.postgresql.org/docs/current/datatype-datetime.ht...
[2] https://www.postgresql.org/docs/current/runtime-config-clien...
Getting time zone conversions right all the way through the stack is far from trivial, which is why I prefer to do it in one place and keep the rest of the system out of it.
> I figure if I make this PEP, we can then ask Guido to quickly reject it, and then when this argument next starts up again, we can say ‘Guido isn’t changing things to suit the tab-haters or the only-tabbers, so this conversation is a waste of time.’ ...
> This proposal, if accepted, will probably mean a heck of a lot of work for somebody. But since I don’t want it accepted, I don’t care.
I don't know if there are others.
Good as a reminder that rules like “explicit is better than implicit” should not be followed all the way to the most absurd possible conclusions.
They're using "evil twin" in the sense of "it's closely related to 758, but goes in the opposite direction to it".
There are two "problems" this PEP is trying to solve.
One is that bare excepts are permitted. The argument against this is that explicit is better than implicit. A matter of taste, but I don't find this convincing.
The other problem is what bare excepts mean. Bare excepts are syntactic sugar for `except BaseException`. This means that an application containing a bare `except` followed by the vast majority of real-world error handling will continue to run even if SystemExit or KeyboardInterrupt is raised. This is almost always a bug.
I do find this second argument convincing, and I wish Python did not contain this design wart.
If I could go back in time and change Python syntax, it would make it hard for people to silently treat these special interrupts as "handleable" like regular errors. The tiny set of applications that really can and should handle them (e.g. TUIs or the mailman example discussed in the final section of the PEP) can explicitly do so with e.g. `except KeyboardInterrurpt` or even `except BaseException`.
But I agree with the consensus here that this does not rise to the level of something being worth a backwards-incompatible change.
(there is also coredump but most distros disable or hide them, so it's not a problem in practice)
Moreover I think there is a real risk people are going to write "except Exception:" instead, which breaks in the fringe case an exception that derives from BaseException (enforced by interpreter) but not from Exception is thrown.
Even if catch Exception is what users usually mean, changing code from "catch (BaseException):" to "catch Exception:" may break some code if improperly reviewed.
It's also not worth breaking production code over this.
Take that up with the consensus view of the python community, as reflected by python linters in their default configuration, almost all of which warn on bare except.
The debate in the PEP is whether this should be a syntax error. The debate about whether it is good style is over though.
> It's also not worth breaking production code over this.
Agreed.
I don't fully agree on it being a purely a style issue (though the warnings from linters are wholly justified). From what I understand, "except:" is a code smell because you don't actually want to catch BaseException instead of just Exception.
Linters wouldn't have warned about it if it meant "except Exception:". The real issue, IMO, is the fact non-exceptions (Ctrl-C, generator exit, program exit code, etc.) have been crammed into the exception system.
For many users, in many cases, this would be fixing the code rather than breaking it.
Forcing people to write either `except BaseException:` or `except Exception:` means forcing them to think about which one they actually mean. This is a good thing, just like the enforcement of proper separation between bytes and text is a good thing.
There is a good chance it will fail to do that. See elsewhere in this thread.
Correct code would be:
But really, you don't want to mess with BaseExceptions at all, so just do `except Exception` instead of a bare `except:`.SystemExit - You _definitely_ want to catch this one for logging. If a library (not top-level app) calls `sys.exit` you at least want to know what's happening, if anything so you can talk to author and get them to use proper exception types.
KeyboardInterrupt - I normally want to catch this one as well. If the program was taking too long and I hit Ctrl-C to stop it, I _do_ want to see all the debug output. And if I don't, for some reason, there is always Ctrl-\ which kills python immediately and unconditionally.
GeneratorExit - this one is tricky and I agree that in a lot of cases, you don't want to print logs on it. But it also very rare - it only appears in async functions (emitted by yield), and never propagated to caller. So as long as you are not doing async, you can simply ignore it, which covers majority of the the code I write.
Your snippet has the opposite problem: if ex is a regular Exception, but then KeyboardInterrupt is raised by `log_tons_of_debug_info()`, then your snippet would mask that by re-raising ex in its place. I think the original snippet is probably better on balance, even ignoring difference in code complexity.
Java Throwable ~= Python BaseException.
Java Exception ~= Python Exception.
The problem here is that a bare except catches something similar to Throwable, not something similar to Exception.
I'm guessing that a `3to4` script would be provided which replaces bare `except:` with `except BaseException:`. We have the experience of `2to3` to draw on with regards to how that might play out.
EDIT: Haha, I now see that this PEP proposes a change without advancing the major version. That surprises me.
2. Python does not use semantic versioning. 3.13 is a different major version to 3.12.
The anger from this potential change is that really all you are doing is taking something away that was working, and now people will need to review their code or keep python on a previous version which sucks.
I think that people who propose these kinds of changes don't appreciate the importance of the programming language being at the bottom of the stack so there's really never a good reason to break people even if you think it's nicer as you really can't appreciate how much work you are creating for people.
Also, I'm starting to get warning about something in tarfile that I will need to track down: https://peps.python.org/pep-0706/
This is why I like to stick to C, Common Lisp, Clojure, and (to some extent) Java/JVM. I don't know about Clojure's future, but C and Common Lisp have been fine for the last 40 years, and I'm not expecting the least in the upcoming years.
What I don't like about say, c, is that it has various backward compatible additive dialects like c11 vs c99. I personally don't agree that c11 and c99 are the same language in spite of the backwards compatibility and I think it makes the entire ecosystem worse. At some point there needs to be a successor rather than just piling on to old broken designs. I would prefer a better FFI or other tools to interface with legacy code in the new dialect.
I'd be happy to find automated tooling for upgrading old code that actually works. Python 2to3 converter worked on simple code but would fail on anything mildly complex (what would that tell about the Python language then :P).
IMHO, breaking language could makes sense if you have built-in runtime (like golang) or you ship native binaries that depend solely on kernel or libc (C/C++). You build your program and ship it to the clients. But, when you have a runtime like Python, you must also force clients to upgrade it. And the problem with Python is that you can't easily do that because system stuff usually depends on a particular version (yum/dnf, ubuntu services, and so on). And, unlike Java, the Python community only advertises a little about having multiple Python versions installed simultaneously.
> What I don't like about say, c, is that it has various backward compatible additive dialects like c11 vs c99. ... At some point there needs to be a successor rather than just piling on to old broken designs.
This is why people are still using C. There are many successors with varying success (D, C++, Rust, Zig...) and much more added complexity to the language and runtime. Language simplicity, compilation speed, and compiler simplicity - all of that was lost.
And yet I have created my own habit of not using bare excepts. TIL what bare except even means, but I do not use them. Simply because I think it makes more sense to specify the exception I want to catch, and failing that I specify the base class Exception.
So I guess I understand the author of this PEP, we're of one mind on this. :)
Also thanks to this post TIL that a bare except might catch interrupts like ctrl-c. Even more justification for my new habit.
https://discuss.python.org/t/pep-760-no-more-bare-excepts/67...
It still feels like `deno` is somewhat on the right track where permissions are dropped by default, but It'd Be Nice(tm) if programming languages enabled that a bit easier.
We're implicitly allowing imported libraries the full power of the containing programming language where with promiscuous code sharing (trending towards a low-trust environment), it'd be a lot better to _not_ give `cap.FS, NETWORK, USB, etc...` by default.Bringing it back around: `import somelib with [ cap.BARE_EXCEPT, cap.RAISE ]` or something to control their handling of "unknown" exceptions is interesting. Let them handle any exceptions or interrupts they've authored, but let me explicitly have control over catching stuff that isn't "from them".
...an extended version of dependency injection or inversion of control.
A more common scenario for myself is write a lot of my scripts in the style:
An implicit raise would obnoxiously break them when my bare-except is intentional, and effectively cause the error to be printed twice to the terminal. Now I'm not wholly opposed to forcing `except BaseException:` instead of `except:`, but an implicit raise would cause all sorts of subtle bugs.Why would you ever consider breaking everyone's throwaway scripts?? For what is already a universal linter rule?
(Plus, the suggested transition easement, "A tool will be provided to automatically update code to replace bare except: with except BaseException:", indicates a fundamental flaw in this approach: it's still trivial for developers to catch-and-kill exceptions. So we're strictly increasing the verbosity of the language without actually solving the problem. :( ).
Only proper response is private email from sane BDFL similar to Linus email of "WE DON'T BREAK USERSPACE"
EDIT: Add on, Private Email to Pablo going "Dude, why are you purposing stuff that will break code spectacularly? I think we need to talk about your approach to language design."