According to a quick `apt remove python` you still need Python 2 for (on Debian Stable, at least):
* minetest
* mpv
* npm
* qbittorrent
* smbclient
* virtualbox-6.1
These are the more popular ones. I write and don't hate (compared to Python2) Python3 but apparently if I remove Python2 I have to shed all these programs because they still depend on it!
Can you include versions of all these deps? For example, as of late 2019 npm does work on Python3. So maybe some of this is just Debian Stable being slow with new version releases as usual? npm is one of the few pieces of software I've had to install outside of the package manager recently because the version available was just too old.
mpv, really? That has nothing to do with python as far as I know. There are Python bindings to libmpv, which I maintain, but I doubt they're included in Debian's mpv package (and they're Python 3 compatible).
mpv seems to use python-docutils for generating the documentation, which has had a very long path towards full Python3 compatibility and only really "reached" it last year (after Debian's freeze).
Indeed. They haven't announced any dates for the next stable release, but looking at past cycle I think we can expect python3 to be the base python within a year, being conservative.
I have no experience of this myself, but I remember people saying that 3.0 had at least one major-yet-unnecessary incompatibility with 2.7, and that converting 2.7 to 3.2 or 3.3 was significantly easier than 2.7 to 3.0 (and so that could explain the speed it was dropped - the fix may have been 3.1 and not 3.2 like I have in my head)
It's not impressively long, it is artificially long. It was decided there would be no 2.8, so they kept 2.7 up for compatibility reasons for a long time to make it easier for people to transition.
Oh I see. I was looking at it on mobile yesterday and the dates were 2000, 2010, 2020, etc. Even rotated landscape. On desktop it expands to the full width and I get every year. Something to take note of perhaps.
You're getting downvoted for what seems like a harsh policy. I think it's definitely understandable that most people want to make the switch as easy as possible, and that's the reason for the delay in discontinuing Python 2. But I would make the case like this:
The slow approach failed. Chances are, most Python 2 code was written after 2010. By switching earlier, yes, you make the switch harder for the handful of people who actually did port their code, but on the other hand you avoid the exact same porting difficulties for many more because the majority of Python 2 code would never have been written! Someone upthread mentioned scientists, for example, who were still writing their analysis code in the late 2010s and are now confused about why none of it runs.
To be sure, it's worth thinking about why it failed, as many other projects have been able to get away with a longer deprecation periods. But its seems like there is a case to be made, in pure consequentialist terms, that Python would have been better off killing Python 2 earlier.
Also consider that a lot of hardest to port python code in various enterprise systems is going to be python 2 for as long as it takes to kill last RHEL version that defaulted to Python 2 (which I think, is the RHEL7, which is going to be supported till I think 2026?)
MISTER PROSSER: But Mister Dent the plans have been available in the planning office for the last nine months!
ARTHUR DENT: Yes! I went round to find them yesterday afternoon. You hadn’t
exactly gone out of your way to pull much attention to them have you? I mean, like actually telling anybody or anything.
MISTER PROSSER: The plans were on display.
ARTHUR DENT: Ah! And how many members of the public are in the habit of casually dropping around the local planning office of an evening?
MISTER PROSSER: Er – ah!
ARTHUR DENT: It’s not exactly a noted social venue is it? And even if you had popped in on the off chance that some raving bureaucrat wanted to knock your house down, the plans weren’t immediately obvious to the eye were they?
MISTER PROSSER: That depends where you were looking.
ARTHUR DENT: I eventually had to go down to the cellar!
MISTER PROSSER: That’s the display department.
ARTHUR DENT: With a torch!
MISTER PROSSER: The lights, had… probably gone.
ARTHUR DENT: So had the stairs!
MISTER PROSSER: Well you found the notice didn’t you?
ARTHUR DENT: Yes. It was on display in the bottom of a locked filing cabinet, stuck in a disused lavatory with a sign on the door saying “Beware of the Leopard”. Ever thought of going into advertising?
Q: I didn't hear anything about this till just now. Where did you announce it?
A: We talked about it at software conferences, on the Python announcement mailing list, on official Python blogs, in textbooks and technical articles, on social media, and to companies that sell Python support.
Where else would you like them to have made it clear?
One problem is that all those things are more likely to reach people interested in Python unto itself vs as a tool to accomplish some other goal, like scientific research. The bigger problem, if/when you reach all those people, is convincing them to put their work on hold while they deal with the fallout of nonfunctional changes to the language/ecosystem, making them jump through academic and aesthetic hoops.
If a journal is so out-of-touch with the practice of science it can't be bothered to update its readers on a major change in a widely-used tool, that's a problem with the journal.
It's not that nobody knew it's going to be EOL-ed, but I haven't met anyone taking it seriously. I've only seen a circular logic of "Python 2 is just fine, and nobody uses Python 3, so no rush". I've seen companies started and intentionally built on Python 2 after the original EOL deadline.
It baffles my mind, because I've lived through PHP 4→5 switch. It was a major breaking language change that affected every object assignment. It felt like it was dragging on forever, but in retrospect, compared to Python, that was a quick and successful upgrade.
It is probably there are more disposable PHP projects than Python ones, meaning that the PHP code base is more frequently replenished with new projects that usually start with the latest version of PHP.
Also plenty of time for python 3 to get a `from __past__ import py27` to run in backward compatibility mode so that python 2 code could be imported in python 3 modules.
About every other month I have to help a biologist who has pulled down an older program from Github debug why it broke in their data only to find that they were trying to run in Python 3 instead of Python 2. These are people that can write shell scripts, but don't program. It makes me aware of all the old code out there that is not used super frequently, but still solves people's problems, and would continue to if there hadn't been breaking changes.
As I write Python 3 now, I am less than impressed with the reasons for breaking compatibility. I'm told that it was necessary for proper bug-free Unicode code, but there's enough difficulty when using, say, gzip, and little enough documentation about the "proper" way to handle Unicode that I doubt that it will actually reduce Unicode-related bugs.
Python is still a great tool, but Python 3 really took the shine off for me.
> little enough documentation about the "proper" way to handle Unicode that I doubt that it will actually reduce Unicode-related bugs.
There are hundreds if not thousands of pages of unicode documentation. If anything the problem is that there is way too much for any normal person to actually understand it.
For most people though, knowing when to normalize strings to NFC versus NFKC is all they need to not unknowingly accumulate tons of technical debt.
You are right, there's tons of Unicode documentation out there. The problem is how to use encodings in Python. What parts of the standard library are going to be OK with Unicode, and which parts are OK with bytes, and more importantly which parts reject bytes/Unicode?
For example, I coded along in Python 3, not changing and IO handling for the longest time, but when I Tried to use the gzip module I suddenly had to learn a whole bunch more knowledge about Unicode. I looked around for about 10 minutes for a some sort of best practices guide, but ij lieu of that just mucked with encode/decode functions until it worked. I don't know if I did conversions at the right spot, or where the guidelines are for changing between bytes and characters, but it didn't matter for my data at the time and o had a deadline so I couldn't really justify spending another hour at the time to figure it out.
Maybe I'm just trying to justify my behavior to myself, but I think this sort of problem solving is the norm. I tend to try to adopt best practices and educate myself more than the other programmers I have worked with, if not more, and am the one introducing flake8 etc. to code reviews, so I'm not the worst sort of programmer, though I certainly wouldn't claim to be top-tier either.
> What parts of the standard library are going to be OK with Unicode, and which parts are OK with bytes, and more importantly which parts reject bytes/Unicode?
This is coming from someone who's written a lot of Java (Java's had the byte/char distinction since forever); the way Python 2 pretended byte strings are strings really was broken and not what you want. What the changes in Python 3 mean are that you have to actually think about if something is bytes or a string, and you have to be intentional in choosing encodings.
I've recently been using some other languages that use a Python 2-esque "strings are byte-strings that are probably UTF-8" model, and I actually think it's a better choice than what Python 3 went with.
Sure, you can argue that iterating over bytes is less "correct" than iterating over Unicode code-points. But iterating over code-points isn't very useful either -- maybe that code-point represents the letter B, but maybe it represents the third of seven characters and modifiers and zero-width joiners that make up one family emoji.
This is the worst part. 12 years to change the language’s Unicode model and, after all that, the new model turns out to be a bad design because in the meantime, everyone else has standardised on UTF-8.
What should `str.reverse()` do? It's very different for byte strings and UTF-8 strings. Or `str[4]`? `str[1:]` can corrupt a string for one definition of []. What about Windows users? There's a chance Powershell is using Windows-1252.
That's the thing, though -- even in Python 3's model, neither of those operations produce the expected result in all cases. If I reverse "réal", I might get "laér" as expected, or I might get "láer" -- it depends on if é is represented as one single character or as two characters using "latin letter e" + "combining acute accent".
The same thing applies to indexing and slicing. You might get an actual character, or you might get something invalid by slicing halfway into a multi-code-point sequence.
So Python 3's model isn't actually that much better than Python 2's at handling these cases. But it comes at a significant cost, introducing "unicode vs bytes" confusion, and making it more difficult to write correct programs for platforms like Unix where filenames aren't necessarily valid Unicode.
Actually it is. What you suggesting would completely break those operations. Compared just not getting what you want. In any way you can tell python how to behave through normalization of the unicode.
> What you suggesting would completely break those operations. Compared just not getting what you want.
I reversed a string several times in Python 2 and not once did my computer catch fire! I just got a string back that, in certain situations, didn't contain the characters I was expecting. And then my father had to have a talk with me about Unicode, and how computers sometimes use multiple bytes to represent a single character, and so if you rearrange the individual bytes you get different, usually more broken-looking characters.
And when my child runs "🇨🇦"[::-1] in Python 3, I'll still have to give them The Unicode Talk. I just need to use the word "codepoint" instead of the word "byte". The situation is basically the same--if you know enough about Unicode to understand Python 3's behavior on certain non-English text, you know enough to understand Python 2's behavior.
> And when my child runs "🇨🇦"[::-1] in Python 3, I'll still have to give them The Unicode Talk
What were you expecting to get when doing reverse on 🇨🇦? The flag symbol is encoded using regional indicator symbol letters: C, A. When you reverse it you get regional indicator symbol letters A, C which represents Ascension Island. In python2 you get a total garbage.
The flag representation is not part of unicode, unicode only specifies that you can encode country codes using ISO 3166-1 alpha-2 and it is up to the application how to present it.
> What were you expecting to get when doing reverse on 🇨🇦?
My future child sees only one character in that string, an emoji. (If you put the cursor to the right of 🇨🇦 and press backspace, the whole flag goes away, so that must mean it's only one character, right?) If they reverse that string, they'd expect to get the same result back.
> The flag symbol is encoded using regional indicator symbol letters: C, A. When you reverse it you get regional indicator symbol letters A, C which represents Ascension Island.
Right, this is exactly what I'll have to explain! Just like how in UTF-8, "é" is encoded using two bytes: 0xC3 0xA9. So if you reverse that, you get bytes 0xA9 0xC3, which do not represent "é". It's the exact same problem, you've just moved up one level of abstraction. (The Python 3 version of the problem isn't unique to flags -- there are all kinds of modifiers and ZWJ sequences and combining characters that get messed up if you rearrange individual codepoints.)
A few languages have tried to solve this problem in other ways, like how Swift considers each character of a string to be a "grapheme cluster", but that comes with a lot of its own drawbacks.
But Python is a duck-typed language. And most of the time, a bytearray's quack and a chararray's quack sound similar, if not the same [0]. Java can afford to distinguish between bytes and characters because it is strongly typed throughout the language. In Python, whether 2 or 3, as a developer you do not have the luxury of informed reasoning whether something is bytes or a string, because your functions' input parameters can be either.
In hindsight, I think the proper decision would have been to ensure every string has an accompanying encoding attribute, and refuse operations on two strings of different encodings (or even perform on-the-fly encoding changes). At least that wouldn't have created a source-incompatible language.
[0] Here's some of the operations that should absolutely be invalid on a byte array, but aren't because Python3's bytes object isn't actually bytes, it's 8-bit ASCII: lower(), title(), upper(), isalpha(), isdigit(), isprintable(), splitlines()
> Java can afford to distinguish between bytes and characters because it is strongly typed throughout the language. In Python, whether 2 or 3, as a developer you do not have the luxury of informed reasoning whether something is bytes or a string, because your functions' input parameters can be either.
Python is a strongly typed language, and you can check a function's argument types at runtime with isinstance(), and inspect those types' class hierarchies with issubclass().
The parent meant statically typed. Meaning it shows the errors at compile time.
Yes, you can check at runtime, the problem though is that especially on Python 2. Those checks don't happen very often.
Python 3 though will crash if you try to use bytes as text or vice versa. The python 2 though it will do an implicit casting and execute code, which will run and look like it runs correctly until you get an unicode characters, then it will crash.
Java won't even let you compile such program. You could get somewhat similar protection if you annotate types and use mypy to check them.
While I agree with you about some of those, `isprintable` only makes sense in a byte object, it doesn't make sense on text which is printable by definition. Perhaps it should be called isasciipritnable, but yeah.
> In hindsight, I think the proper decision would have been to ensure every string has an accompanying encoding attribute
This wouldn't have made things any more source compatible. Adding more types doesnt address the issue that python2 string was multiple things masquerading as one.
> In Python, whether 2 or 3, as a developer you do not have the luxury of informed reasoning whether something is bytes or a string, because your functions' input parameters can be either.
I guess they can be, but the code might also work on an int tuple, even though the operation doesn't make any sense. Here's a case where duck typing does something, but it doesn't make sense.
import functools
def sum2(vals):
return functools.reduce(lambda a, b: a + b, vals)
Python's built-in sum won't work on strings or lists of strings (so much for saying the language is duck typed), but this will. It's also weird because addition is associative, but not with strings, so changing a+b to b+a changes the behavior and gives you a very cute answer to the "reverse a string" algorithm:
reverse = sum2
In practice, every legitimate use I've seen for duck typing might as well use what would be an interface in Java. I see duck typing as interfaces without the formality of declaring them. If you violate the implicit contract, funny things happen, and this is why Typescript got so popular and Python added type annotations.
> little enough documentation about the "proper" way to handle Unicode that I doubt that it will actually reduce Unicode-related bugs
I won't claim here that the compatibility break was actually worth it, but I am sure that it reduces Unicode-related bugs.
It's easy to write Python 2 code that seems to work fine but immediately crashes when it has to process Unicode. But if your Python 3 code works at all it'll probably handle Unicode fine, or at least handle it better than naive Python 2 code would.
Whenever I have to write Python 2 code that has to handle Unicode, I mainly just try to write it so that it would be able to run in Python 3.
> It's easy to write Python 2 code that seems to work fine but immediately crashes when it has to process Unicode. But if your Python 3 code works at all it'll probably handle Unicode fine, or at least handle it better than naive Python 2 code would.
Yeah, that's the biggest benefit, and the reason python 3 handles it better, is because it crashes early. When it it sees that you are making an incompatible conversion, it refuses to run without waiting for some specific unicode character to show it.
> Whenever I have to write Python 2 code that has to handle Unicode, I mainly just try to write it so that it would be able to run in Python 3.
Thankfully I no longer have to write python 3 code, but from my experience, it works the best to write the code for python 3, then adapting it (sometimes with help of six package and imports from `__future__`) to work with 2.7. It is easier than do it the other way.
Production code must be maintained. I've seen many shops with legacy 2.7 code that is stuck due to breaking compatibility in 3.x. Put it this way, if my production code suddenly needs to be patched due to non-security changes then that has to come out of somebody's budget. What else could we be doing with that budget than patching code from 2.7 to 3.8. How do you justify that to your budget director?
As long as you are willing to spend some of that budget continuously patching and maintaining the old 2.x interpreter to address security vulns as they are discovered (IF they are discovered), sure. No need to upgrade!
If you haven't had the resources to port a codebase to a new language version over the course of twelve years, you clearly haven't had the resources to maintain it.
> What else could we be doing with that budget than patching code from 2.7 to 3.8. How do you justify that to your budget director?
Can't speak for smaller businesses, but in larger companies, there is so much "waste" that the cost in $$$ to do this is insignificant in comparison. And if you choose to spread this effort out over, say, 5 years the annual cost is miniscule.
Not only is this untrue in the case of Python, I think it's untrue in the case of basically every other successful language:
- ANSI C was incompatible with pre-ANSI C. Current C has various minor incompatibilities like the removal of gets().
- bash defaults to non-backwards-compatible behavior and lets you set various shell options like "compat31" if you want it.
- Newer version of the JDK generally require code fixes.
- PHP has made various backwards-incompatible changes, like the removal of magic quotes in 5.4.
There's also many more cases of languages getting forked in backwards-incompatible ways and going on to be successful. C++ is doing fine despite not being fully backwards-compatible with C; VB.NET broke compatibility with VB; zsh isn't POSIX-compatible by default; there were a million variants of BASIC back in the day, and still are a million variants of SQL; etc.
When I was first starting, I almost gave up on Python because the print "Hello world" script that I copy-pasted from an online tutorial failed to run.
The hardest part (for me) of getting started with a new language is setting up the environment to run properly... Installing libraries, configuring environment variables, pointing the IDE to the interpreter, etc. So of course that's where I assumed my mistake was.
It was only much later that I realized that I was running Python 3 (newer is better right?) and the tutorial was written when Python 2 was the latest version. So the print statements changed.
And of course, "print" is in everyone's first program. So it broke the experience for all first time users.
> So it broke the experience for all first time users.
To be fair, Python 3 provides a very clear error message if you try to use print as a statement, although I don't know when exactly that was added:
$ echo 'print "Hello World"' > test.py && python3 test.py
File "test.py", line 1
print "Hello World"
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print("Hello World")?
I agree with the parent actually. My first python program was a json read and print something. 4 lines, maybe 5. I had a need to parse json and print something specific and I suspected I would be able to do it in python. After 4 hours, I was able to and I felt happy. This is not to complain about that 4 hours.
But when I think back at that 4 hours and how much time will it take to do it today, it reminds me of all the learnings. Is it load or loads? Which one is from file? What is a json object? Do I need to quote? If so how? Why does it work with the small block but not the big block? Is it the nesting? Am I doing something wrong?
I think the same is true of hello world. Hello world is a newbie presumably who has not written a single line of code. Well you copy and paste and it does not run. Yes the error is there but hey I copied working code : did I install something wrong? Should I check some website? It says it is python but is it? You can't imagine the questions that will come to a newbie. We need to be mindful of this rather than have a stack overflow mindset.
This is largely due to conditioning of over 30+ years of bad error messages.
The program exited with error code -2
Error MS1337 occurred
Something went wrong. Contact your Administrator
Ideally; error codes should provide information about how to fix the problem (which python in this case does). That falls on the developer side of things (and may not be possible in many cases).
Learning that error messages are useful and can solve your problem, and de-training the reflexive "This message won't help me", lies on the end user side of things (assuming the devs did their part).
> Ideally; error codes should provide information about how to fix the problem (which python in this case does). That falls on the developer side of things (and may not be possible in many cases).
Rust generally tries to do that, although the fixes it proposes tend to lack context, and can thus lead you into the wrong direction entirely (e.g. once upon a time it would tell you to go and implement a trait on your type when the issue was that you'd forgotten a trait bound on a generic type, it still kinda says that but now there's a snippet which adds the trait bound so it's less error-inducing).
Python itself has a habit of contributing to this long history of extremely hostile messages.
If run in interactive mode, the Python interpreter takes its cues from vi and deliberately frustrates the obvious methods to quit. Ctrl-C results in Python whining KeyboardInterrupt but doing nothing helpful. Typing 'exit' or 'quit' both result in messages that make it clear that the interpreter knows what you want to do but won't comply out of spite.
More fun can be had with certain versions of Python 3 when running on Windows, where the Python print function would throw an exception if told to print Unicode characters. This made printing to the console non-portable because Unicode characters had to be stripped if running under Windows. This seems to have been fixed, however.
Still existing are the misleading warnings that Python produces if you forget to include the self parameter on a class method. Python emits an error that blames code that calls the method for passing one too many parameters instead of identifying the underlying problem. Python also gets similarly unhappy, and points fingers at the wrong code, if the brackets are omitted when instantiating a class.
Since Python is dynamically typed, these errors can't be identified until execution. Requiring all-paths testing to catch syntax errors is a special kind of user hostility in and of itself.
Python is an utterly horrible language and it's deeply unfortunate that it has become so popular when so many less error-prone, less obnoxious, alternatives exist.
> Typing 'exit' or 'quit' both result in messages that make it clear that the interpreter knows what you want to do but won't comply out of spite.
The interpreter knows no such thing. Both quit and exit are regular python objects that implement a __repr__ method, both of which return a string that inform the user how to exit. The __repr__ method could have implemented exit, but that would have been poor form (and create issues with tooling), set bad precedent, and go against python coding guidelines.
Ctrl-C giving a keyboard exception rather than exiting is a good thing, as it is far more likely that you are trying to terminate a long running statement, or exit out of sub code block, than trying to kill the interpreter process. This is also the standard behavior of almost any interpreter you care to name (irb, node, clisp, shell). This is because the interpreters try to mimic unix shell semantics (CTRL-D also kills your shell), since they are, after all, shells.
Since Python 3 can detect this case and accurately determine the user intent, it could have just executed the statement as if it were written with parentheses. Had they done that alone, they would have significantly decreased the number of short Python scripts that broke during the transition. Or they could have made a --permissive flag that would do this.
If you think that, do you think Word should start refusing to open .doc documents at some point?
Code is sometimes alive and changing, and sometimes an artifact. Requiring that code become alive after being an artifact carries significant risks. One might say "code should never be unmaintained" but I think that ignores the reality of limited resources that all of us deal with.
Well, we can either have continuous innovation and improvement with proper security, or we can have long-term support for code that is unmaintained, but it currently seems impossible to have both. I don’t think anyone would consider the .doc format a paragon of design elegance or security.
It would take quite a lot of effort to offer the new print function and the old print statement at the same time, and it would require either a lot of special cases or renaming the new print function.
And there's really no upside. If the only legacy thing in your script is the use of print statements, that's trivial to fix by hand or with one of the automatic converters.
Yes there is, the print function is the biggest syntactic incompatibility. Besides the (optional) u marker for unicode strings that they reintroduced later. Most of the other difference (like changed packages) are semantic.
You would have been able to import and run Python 2 in Python 3, maybe with a couple of "if"s or some magic in the module loader. Breaking compatibility like this felt petty and deliberate and made it really hard to have one codebase target both python 2 and 3 (temporarily).
But as the GP mentioned, it can be fixed automatically.
In this specific case I think it's a non-issue. If python3 was limited (or limited except in very niche cases) to changes that could be made programmatically, we wouldn't be in this situation. Everyone would just run 2to3.py and never look back.
The problem here is that print "statement" can't be upgraded to a log "function" easily as the programmer grows in experience.
A print "function" conversely is quite easy to upgrade to a log "function".
In addition, "print" is almost always a code smell in professional code. The moment you need to localize, change destination, etc., "print" goes right out the window.
I have lots of beefs with Python 3. Changing print to a function is NOT one of them.
That's a terrible message for people trying to run other people's code, unfortunately. It wouod be more useful to say "Are you trying to run Python 2 code in Python 3?", If Python detects enough "2isms" in the file.
Except that is imprecise and can be misleading. It is certain you called print incorrectly. It is not known if you just made a mistake or are trying to execute python 2 code.
This was a while back. In version 3.5 (5 versions after the change was made!) the message was just:
SyntaxError: Missing parentheses in call to 'print'.
This doesn't give the "did you mean" hint, and while it's perfectly understandable for someone who's gotten as far as learning exceptions in Python, that usually comes a few chapters after "hello world". It's definitely enough to throw a newbie off. And even if they're a veteran of other languages, they might have been exposed to a wide variety of error messages like this that commonly indicate a problem elsewhere, and having never yet successfully run a python script at this point, the prior for an error in configuration is much higher than for the tutorial being wrong.
Oh man I had the exact same experience about 10 years ago. I couldn’t get “hello world” to work despite doing everything the tutorial said. Disheartened me from trying any more programming for several years.
There were so many routes forward that didn't involve breaking everything but still would have allowed fewer unicode bugs (maybe). And they could have done a lot more to make the transition easier if they wanted to - some of it was just in your face type breakage to a prove a point without considering some underlying use cases (a lot of internet protocols, headers etc still ASCII).
What is nuts to think about - if tweaking some unicode handling is worth creating an incompatible version, what happens when some random python developer has another itch to scratch that they consider so important it is worth break a language over. The idea that these types of fixes require breaking compatibility is absolutely insane if you want to address a larger / longer term market.
Complain all you want about Microsoft, C etc. Even go of all languages (because google has an otherwise terrible rep in this area!!). But some of these languages and companies really do seem to be able to deliver or intend to deliver longer term compatibility.
When businesses and lifetimes are measured in decades (not months) it is VERY nice to build on stuff that has longer support horizons.
Python 2.7 was supported for 9+ years, python 3 was released in 2008, with development beginning before then. You had almost 15 years of warning from initial announcement to eol. Python3 predates golang.
> And they could have done a lot more to make the transition easier if they wanted to - some of it was just in your face type breakage to a prove a point without considering some underlying use case
Such as?
> (a lot of internet protocols headers etc still ASCII).
What do you mean here? Py3 correctly handles http headers encoded as ascii bytes. Yes the decision to make http headers ascii bytes and not basically anything else is bad in hindsight, but that ships sailed.
Maintaining python2 and python3 forever in perpetuity wouldn't be free either. Why should they provide free labor in perpetuity so that you need no provide any?
In theory if there was sufficient need the entire community of python2 users could collectively fund said support but I don't see how they can demand it for free.
You don't need to, provided you're willing to also maintain a Python 2.x fork on your own instead, or willing to pay someone else to do it for you, or willing to risk no more patches to the interpreter. You do you, boo.
I claim that this would have negative EV. Most exceptions aren't due to version mismatch. You could add a line for a very specific set of syntax errors, but those aren't very useful.
Also having your new version suggest using the old version is counterproductive.
Maybe... But there are two correct solutions (using an old version or upgrading the code) and such message would communicate clearly the need to decide between them.
I think you're misunderstanding what was meant by "longer term compatibility". The issue isn't that the migration period was too short, it's that there was a such a big discontinuity in the compatibility in the first place.
Python 3 provided no extra value at a high migration cost and painful library ecosystem issues. This is in contrast to typical programming language version migration, where breakages are either avoided entirely or are extremely localized. For example C++11 had some breaking changes compared to C++98. I don't know if I ever noticed them, but it didn't matter since C++11 added so much value that everyone wanted to migrated ASAP, and anyway there was no need to move the whole ecosystem at once.
No matter how long the transition period was for Python 3, there was not going to be a good time to migrate. So most people would leave it until the very last minute, resent every hour of time spent on it, and sometimes put the option of rewriting in another language on the table.
> Python 3 provided no extra value at a high migration cost and painful library ecosystem issue
I disagree with this. There was a lot of value add for specific (and relatively common) workloads. If you were doing unix automation you didn't gain much (but I argue this is because unix does unicode bad, not because python). But web development did, even if most of it wasn't obvious or was too soon to be obvious.
As for c++ the effort the language goes to to maintain backwards compatibility has costs, both in terms of syntax and semantics. Those costs push people to rust etc. Much the same that migration to py3 pushes some people to go. It's difficult to say which is right without the ability to time travel.
We have line of business code (not python) that still runs in production that was written over 20 years ago.
Hell Larges parts of Windows is still code that is decades old...
This "well you had plenty of time" shows just how much many modern / web developers do not understand Enterprise code, code that goes into manufacturing or other long running processes where "new" machines are 15 years old
I'm not clear what your point is. Much of the python3 code I use today was written 10 years ago (or more) and only barely had to be touched to work on 3.
> other long running processes where "new" machines are 15 years old
Then...pin to a 15 year old version and use that. No one is forcing you to upgrade.
Web developers re-write their entire stack all the time. In a business with tons and tons (and tons) of little apps and services and command line tools etc, NO ONE wants to (or can) go through all this stuff to update it because someone decided the way unicode should be handled should change. The value gain to those 100's or little apps and utils is essentially zero. I do think a LOT of people got a lot better at making sure to lock configs down - it used to be you could basically ship the python file and as long as you were not doing something weird the system python / distro python would work fine. Now you really need to version pin everything, because python 3 was a nightmare for all the pip libraries and the knock on effects just showed up everywhere.
Take a read here. Py3 was supposed to better - but in a LOT of use cases this better turned out to be "miserably slower". So you were asked to do a lot of work to get your code to run slower.
File handling in the new unicode world was something like 2-3x SLOWER. This is not percent, something which took 3 minutes all of a sudden took NINE minutes because of the io module.
URL parsers became a nightmare - dual implementations of codecs started poping up.
For a "better" python, I didn't really start using it until 3.5 or so. Now it's much more pleasant, but at the key point of transition when everyone was going to check out the new thing, it was slow, put lots of extra bumps that appeared unnecessary to solve the problems that needed solving etc.
I'm not sure what your point is. You still had 12+ years of warning. You didn't need to migrate in 2010. I'd guess that the majority of application code migrated in the past 2 years, but that doesn't mean that you didn't have time to plan, and prepare.
> File handling in the new unicode world was something like 2-3x SLOWER. This is not percent, something which took 3 minutes all of a sudden took NINE minutes because of the io module.
Yes, unicode is slower. If you weren't using unicode you could and can still do all your processing in bytes mode.
That's not the way it worked. You couldn't migrate your code until your libraries had migrated. They didn't always want to migrate because their userbase was generally mostly on 2.
This is what worries me - the lack of understanding of how broken these backward compat breaking changes are. Mercurial, built on python, took 10 years of work to get to python 3 - for basically no reward (they just need bytes and unicode is not super relevant to them) as just one example of MANY.
Let me give you an example. in Python 2 you could use u'' for a unicode string. Perfect, I could update python 2 code to use this or b'' if I needed bytes so in the python 3 setting the right things could happen. The python developers literally TOOK THIS AWAY in python 3. So now, for no reason we can ever hope to understand, you couldn't write code that work work in 2 and 3 by using u'' and b''. This was the type of screw the developer in your face attitude that developers got.
Let's be clear, library folks would need to write code that worked for python 2 and 3 for a long time. And instead of making this easy or at least reasonable, the python developers went out of their way to basically screw folks over here.
Others will remember that the very reasonable formating approaches to bytes (yes, those are useful) was trashed in python 3, again for no seemingly good reason. % operator .format etc. WHY? I'm serious.
"You had 12 years of warning" - hey how about python follows some other languages. Make the update compelling so folks are excited to move. Think about things that will make it easier instead of just pulling all these silly stunts. You couldn't practically move initially. Something as simple as doing u'' in python 2 that would alias properly in 3 - was a no go.
This is what worries me about posts like yours, python developers have really no clue how stupid this type of backward compat breaking approach is if you have limited upside and go out of your way to make it hard to support both versions. It is critical that lesson be learned - but it seems hopeless.
Anyways, go and others I think are going to make a reasonable run at this sort of utility language space with I think some stronger efforts around backward compatability. I think the next backwards incompatible change in python (python 4?) will probably see a much bigger migration off python.
> They didn't always want to migrate because their userbase was generally mostly on 2.
Yes, this doesn't change my argument. You had 12 years of warning. You couldn't migrate immediately, but you had 12 years of warning to plan your migration.
> I think the next backwards incompatible change in python (python 4?) will probably see a much bigger migration off python.
You aren't paying much attention to the python ecosystem then. I wouldn't expect a python3-like breaking change for another 10 years, if ever.
> Mercurial, built on python, took 10 years of work to get to python 3 - for basically no reward (they just need bytes and unicode is not super relevant to them) as just one example of MANY.
> The python developers literally TOOK THIS AWAY in python 3.
And then readded in python 3.3, around 7 years ago.
> . You couldn't practically move initially. Something as simple as doing u'' in python 2 that would alias properly in 3 - was a no go.
I agree you couldn't migrate immediately! But you still had 12+ years of warning! I'll repeat myself: I expect most application code was migratedn in the last 2 years. But the maintainers of those applications had 10+ years to prepare for the migration of their application. This prep took many forms. For some it was nothing, for some it was helping dependencies migrate to 2/3 compat. For some it was planning the eventual migration. But every single one had more than a decade of warning.
I think Perl 5 (despite the much bigger Perl 6 fork) used a language pragma for unicode. When Java introduced generics, they had to use type erasure.
It does seem like if you put in some effort and do some decidedly unaesthetic things, you can bolt on and backport a lot. No developer dreams of maintaining dirty hacks at scale, though.
Please note that Perl 6 did not start out as a fork, but over the years effectively was considered to be one. Which is why it changed its name to Raku (https://raku.org using the #rakulang tag on social media).
> When businesses and lifetimes are measured in decades (not months) it is VERY nice to build on stuff that has longer support horizons.
Indeed. I'm in the process of writing some simple web stuff. I considered python, but decided against it because I'm doing this for free, and don't want to be asked to come back and fix it in a decade because "Python 4.6 needed a feature, and it'll break compatibility with Python 3"
Yes, they could still just run it with Python 3, but do I really want to go through the effort of explaining how to configure that instead of just handing over a working binary?
Why would you worry about that for Python? The community learned their lesson. I'd worry more about other language maintainers who haven't had the learning experience.
Because with Python the possibility is still there. With C/Rust/Go, it's simple to statically compile and remove the possibility of that type of issue.
It's not a program that will be running on a server, so I'm not really concerned about security fixes/updates to libraries.
And then the distro they are using stops shipping Python 3 by default because "No one should be using that anymore.".
While I could bundle python with the program, that's getting to be more hassle then it really seems like it's worth. In addition, I'm not particularly comfortable with configuring python, so there's a non negligible chance I'd screw that up and be in the same position.
It's not that it's impossible for Python to work in this situation, it's that me doing it is a bad idea. I'm sorry if my original comment implied otherwise.
What you're talking about Have you tried to run VB6 code in latest VB?
Things are getting broken all the time, the major mistake IMO what PSF did is giving 10 years to migrate code. Everyone waited to middle of 2019 when they actually started to migrate and the 10 years only allowed to created FUD.
I used Python for a while years ago and asked a core developer about this, he said they’re not paid for support of that nature and to talk to Red Hat. For them it’s just a fun project, no matter who (foolishly?) built a product on it. They’ve made it out to look like a toy language, but it’s taken seriously.
So I went with C# instead. I’d choose it again over Python.
Did you end up talking to Red hat? I'm curious about how much "support" they'll provide. I can't imagine anything other than back porting security fixes, but even then it seems like they'd be pretty limited.
I struggle with this whenever I think about it. On one hand I agree with all of the points you make. But on the other hand, it didn't have to be this way. Python 3 has been out for 12 years now. If people hadn't dragged their feet on converting the mess wouldn't be nearly as large as it wound up being.
IMHO 12 years is a very short time! Of the user-facing programs I use (mostly CLI), the vast majority of the code That gets executed was written far longer ago than that.
Writing some code and not have it executable a dozen years later is a terrible terrible situation. And it wouldn't be this way if Python 2 wasn't EOL.
I'm going to always need to keep a copy of Python 2 around. Always. And if I'm going to change old, working code, I'm probably going to rewrite it in a new language.
As with my previous post I'm torn on topics like this as again I agree with everything stated. And yet, Python was relatively niche in the early aughts so had only really been in the mainstream for a handful of years before 3.0 came out. What %age of live python code in the wild was written after 3.0 was released? I'd wager that's a very large number.
But yet I get why things are the way they are. Heck, it was only last year that I switched to using Python 3. I use it so rarely that it was just easier to stick with the Python 2 which was already installed on my system.
They were spending the time doing other important things. They were making the rational decision to work on something that was important and WASN'T already working. The python 2 code was working fine, it didn't have bugs and it didn't need any new features. Other code either didn't exist and needed to be written or had bugs that needed to be fixed.
It makes sense that changing working code never made it high enough on the priority list to get worked on.
And on the third hand companies I've worked with had 25-year-old C code bases that seemed to compile and mostly work, but mostly only with -O1 or lower, because they've had UB since the beginning of time but only more recent compiler optimizations reveal the underlying issues.
And this old code is also written in a style (usually global variables and mixed concerns and years of sub-ideal bug fixes and patches) which makes it very hard to change or modernize (forget adding threads with all of this global mutable state).
The only point being that old code may superficially work, but in many cases (in my experience) and regardless of language, it all has technical debt and needs work to keep it relevant.
Does that matter? The code performs its duty and has for 25 years. If there are bugs in it they clearly haven't been important enough to be fixed. Technical debt? Maybe, but if noone is collecting the fees, who cares?
If the situation changes and the original code suddenly becomes a basis for further development then that's a different thing. But cleaning up the code without clear benefit is a waste of resources.
Hats off to original developers. Making code that serves its purpose for 25 years is quite an achievement.
The code gets bug fixes occasionally because things change (scaling past previous limits, changes in the environment, etc) and of course new features are required once in a while. So the debt usually can't be swept under any carpets.
All code rots. If you are lucky to never need to fix it or change it, that's great, but that is (in my experience) extremely rare over time spans measured in decades.
Sometimes a code base runs for decades not because it was particularly robust or well-written, but because it is too painful to replace it due to its complexity or the long ago loss of the knowledge needed to work on it.
Most of our C code actually doesn't compile and work, unmodified, under the latest GCC. There are many new warnings, and like any professional C shop, we build with -Werror (or at least don't ignore the warnings). Since I help with maintaining the build infrastructure at work but I don't own any of the C code, I've proposed setting the new warnings to -Wno-error, but I've gotten pushback from the code owners, saying that they expect the new warnings to catch actual bugs and they want to deal with the fixes.
Sure, there is some amount of C code from 25 years ago that works unchanged, but there's also a good bit of Python code from 25 years ago that works unchanged. (I just tried a few of the examples in the Python 1.3 tarball, released in 1995, and it seems like most of them will work with very minor changes.)
If you compile with -Werror, then you are intentionally running it in no-backwards-compatibility mode. That is not really comparable with Python situation.
Actually GCC is very careful about adding new warnings to Wall because of backwards compatibility. That's a reason why Wall doesn't actually contain all warnings.
Kind of, yes, but it's usually warning us about things that were actual bugs and reliance on unspecified/underspecified behavior that might be changing with newer optimizations.
Not really in my experience. I maintain some relatively old codebases and the most common warnings for old code with newer GCC generally are:
- Warnings recommending the use of parens in code like `if (a || b && c)`. The original code is (probably) not broken, it's just that GCC thinks that it's error-prone to write conditionals that way (I agree). There's also a similar warning for code like `while (a = foo())` where it asks you to add parens to make sure that you didn't use = instead of == by mistake.
- Warnings for unmarked fall-throughs in `switch`. Again, not broken if the fall-through was intentional. That's a fairly new warning, so a lot of all code will trigger it.
- Warnings for some const to non-const pointer conversions (in particular for literal strings in C++). There you could argue that the original code is bogus but as long as the pointer is not written to it's fine. Const-correctness wasn't really a thing in C and C++ for a long time, a lot of legacy code plays fast and loose with const qualifiers, but that doesn't necessarily mean that the code is broken.
- Warnings about unused variables, in particular variables being written to but not read. As compiler get smarter they tend to catch more of those.
Unfortunately, 30 year old Fortran code did not have such good luck in my company. VB6 compiler is hard to come by these days, and I have been tasked with rewriting it for "modern computers" ! The newest executable we have is from ~2008. Kudos to Microsoft to keep it running at least.
But 30 year old Fortran code should compile and run with no problems, unless it used something non-standard, shouldn’t it? In which case, the problem is not the evolving Fortran standard, but the use of non-standard compiler features.
“The Fortran standards committee generally refuses to break backward compatibility when Fortran is updated. This is a good thing (take that, Python), and code written decades ago can still be compiled fine today.” -- http://degenerateconic.com/backward-compatibility/
Not source-incompatible. Yes, if you have a binary linked against an old dynamic libc then you may have to give an alternative LD_LIBRARY_PATH or link it statically after the fact.
Python users are not C users. C users typically understand compiler flags and build tooling.
Some Python users understand compiler flags, but there's also a big community of scientific developers who just want a working language.
Python 2 -> 3 created a lot of problems for them without making their lives easier or more productive.
Perhaps Python 3 should have been called something entirely different, and P2 could have been handed over to a new set of maintainers who weren't going to EOL it.
Well, if you compile python 2 code using Cython you also can run it in Python 3.
But seriously:
- C never did such change, it doesn't make as much sense for it to do it since it is low level
- C has a type static typing, so if they did, they could make compiler refuse to compile until types are right. Python has dynamic typing so these kinds of things show up when running, you can use something like mypy, but unlike C it is optional.
BTW: I saw people using Go as an example that there was no need to add unicode support like in python, and simply store everything as UTF-8. Those people are forgetting that Go already has clear distinction between text and bytes (via string, bytes, runes types), how the language stores text is just an implementation detail, since normally there shouldn't there be access to the internal representation of the text anyway.
The problem is why should people be "converting the mess" in the first place. Backward compatibility, especially when it comes to programming languages, should be a given. If anything, this is reason that people should stop using python.
Is there really that much choice for alternatives though? What else is installed everywhere, has a repl, huge standard library and tons of learning resources, no boilerplate to get going. I don't really like python and still consistently find uses for it.
Ruby meets most of those criteria. It also had a more graceful (though not without its pains) Unicode transition, as it allowed you to mark file by file the status of string literals.
Ruby is a comical example, because they did breaking changes in minor version (I think it was 1.8 to 1.9?).
The thing I think they did better though, is they didn't give people 10 years to fix their sh*t. You either moved on or you stayed with an unsupported version, and people did move on.
For Python 3 the major shifts were in 2015 and 2020. The first year was when PSF announced they won't add any new features to 2.7, and that's when libraries started adding Python 3 support. And 2020 was EOL for Python 2.7.
It doesn't matter how much time you give people to migrate, they will always do it at the very last minute.
The amount of breaking changes was also much smaller than in Py3k.
Also, 1.8 marked the creation of complete compliance test suite (as Ruby 1.8 became an ISO Standard), and that test suite has been expanded since then allowing for alternative implementations.
Because the test suite is your target, not a hard to interpret 2kLOC spaghetti code C function.
> Ruby is a comical example, because they did breaking changes in minor version (I think it was 1.8 to 1.9?).
Ruby was then using a version numbering scheme similar to the old Linux version numbering scheme, where odd minor numbers were development releases and even minor numbers were stable releases. In that scheme a odd minor version is equivalent to a SemVer major release. (Ruby switched to SemVer-ish versions with 2.x)
perl5 meets that except for the REPL. Maybe elisp?
I feel like golang could take the place of a lot of Python out there, but not all, certainly.
It would be nice to have an alternative to Python that took backwards compatibility seriously. Might be easier to start from Python2 than to go from scratch. You might be able to add a REPL to perl5, but its syntax isn't the most beginner friendly I've ever seen.
If you replace "installed everywhere" with "reasonably easy to install", there are interesting alternatives among the Lisp dialects. Especially as Python has become a lot more similar to Lisp with many of the additions in Python3.
Things these languages do have in common with Python3 are:
- support for keyword arguments and optional arguments
- support for objects and interfaces (but not necessarily for inheritance)
- handling of names in scopes and name spaces
- closures and lambdas
- list comprehensions
- arbitrarily long integers, complex numbers, a numeric tower
- an empty sequence or empty string is logically false (with a few nuances)
- low-level bit operations (often down to a very low level such as popcount in C)
- easy to call into C code (or Java code if it runs on the JVM)
- full unicode support
- conditional expressions like (a if boolval else b)
- type annotations
- string formatting forms a mini-language
- print is a function
- hexadecimal, octal, and binary literals
- tools for iteration and first-class functions, like partial application
- support for OOP
In addition to this, the members of the Lisp family usually have as well:
- strong support for functional programming
- full and efficient garbage collection
- persistent data structures
- compile to machine code (native code generation or JIT compilation)
- a powerful macro system
- capability to define custom control constructs
- support for pattern matching to members of compound objects, similar to
(a, b) = name[:2]
- often very good support for parallelism
What exactly would be suited best depends IMO on the use case.
I think Racket is ideal for easy learning, a platform-independent GUI, and comprehensive documentation, and is also useful fir science applications. It can also interact well with C libraries (or, more generally native libraries with C interface), and it is good for scripting.
Clojure is ideal for server applications with a lot of concurrency and a high performance. Its concurrency support is brilliant and best-in-class, much better than Python.
Common Lisp / SBCL is somewhat under-hyped in comparison to Clojure, yet it has an excellent performance, integrates well with C libraries, has a highly interactive repl, a comprehensive library system (quicklisp), a standard build system (asdf), comprehensive documentation for beginners (common Lisp cookbook), it is very usable for scripting, has standard pthreads-like concurrency primitives, a powerful object system, and then some more.
Categorically demanding complete backward compatibility puts huge limitations on the development of the language. I agree that the barrier should be high, and I don't think many disagree that the Python transition was a mess, but for languages to evolve there needs to at least be some way to fix issues that are considered design errors.
There's no optimal or blameless solution here but I think in particular when it comes to the scientist use-case mentioned above, an entire decade is enough to address what is likely a relatively small codebase, it's not just a fault of the language, but also neglect or lack of resources in fields where programming is secondary in nature. (also btw detrimental to the quality of science, as recent cases have shown where programming errors basically invalidated entire studies)
>Categorically demanding complete backward compatibility puts huge limitations on the development of the language
Not really. Just introduce an optional version declaration at the top of the file, which guarantees the old behavior. Then feel free to tinker with the design as much as you want. It's a solved problem and it works well. Sure, you still break people's code if they don't put a version declaration in - but now the fix is a single line.
The fatal flaw in Python's approach was cleaving apart the Python2 and Python3 worlds, making it impossible to mutually import modules. Python2 and Python3 are practically speaking different languages - different syntax, different semantics, different libraries, and a different userbase. As far as the computer is concerned, they are separate entities, with the feeble "2to3" program being the only thing linking them.
Actually I did some small tests, and you could mix python2 and python3 code together through use of Cython.
You can use Cython to compile your python 2 code into .so objects and then include them in Python 3. Then you can progressively migrate code file by file to Python 3. I didn't do it on a larger project, so maybe there are some gotchas, but it seems to work as far as I could test it.
Regarding the differences in languages Python 2.7 was actually a bridge to migrate to Python 3. That's why all features of 2.7 were backported from 3 (this was also the reason people saying: "why should I migrate to 3, it doesn't offere anything new". It didn't because everything new was backported.
Anyway thanks to this it was actually possible to write code that worked on both Python version. I did it several times. It was actually the easiest when you wrote the code for Python 3 and only used features that 2 had. Add all the `__future__` imports and remaining parts filled with the `six` module.
Yes. "from __future__ import ..." could have had "from __past__ import ..." or some other such 1-line mechanism to maintain compatibility. Before the switch to P3, I argued vehemently with (some of) The Powers that Be that doing this was unnecessary and would break the community. The argument was, "Don't worry, P3 is so much better, people will want to change over." Well, yeah, maybe. Personally, I ignored P3 until 3.6 arrived. THAT was enough of a jump to abandon P2.
Similar story here, 3.5 was first version that was even practical to really consider (they kept on making the compat story a bit better because they FINALLY listened to folks). I really wish you and others had more luck earlier, the transition did not need to be nearly as painful.
> Python2 and Python3 are practically speaking different languages - different syntax, different semantics, different libraries, and a different userbase.
Yes, and tools and infractstructure should account for that: Python 2 and Python 3 should use different file extensions, and libraries written for python 2 should not just adapt to Python 3 and break backward compatibility, but come _at least_ as a new major version - better with a different library name, as Go lang requires.
And still it would have been possible to support Unicode in python 3 without breaking backwards compatibility. It was possible to support in Common Lisp which was defined as a ANSI standard in 1994, and Common Lisp implementations like SBCL or Schemes compile to far faster code than C Python (while providing many of its flow control affordances, like list comprehensions, or a numeric tower). Why should it not be possible for Python? Apart from the developers not wanting to do it?
> it's not just a fault of the language, but also neglect or lack of resources in fields where programming is secondary in nature.
A lack of resources for porting and maintaining code to new language versions is totally typical for most of science. There are simply no jobs or incentives to do that, especially if you want to have any kind of career in science.
Further, python developers seemed to think that the bulk of code and maintenance work is in the language implementation and that the many small libraries and scripts are not important. But this is not the case, similar to that in trees, a rather large part of the biomass is not in the trunk, but in the roots and leaves. And the small application scripts and libraries used in science change much more slowly than the main language revisions.
You could argue that this does not matter because most of python's applications are not in science, at least not if you look at the respective amount of money moved. But this fails to take into account that a lot of python's success was so far based on its use in science, especially numerical python and data analysis.
> If anything, this is reason that people should stop using python.
It's also the reason I never started in the first place. It wasn't in any way clear which the right option was during the period of schism, and it went on for an awfully long time.
With that being said I'm starting to tinker with Python now simply because it has a great ecosystem for NLP and ML. I certainly need the former, although how much of the latter I'll need (possibly none) is unclear.
Regardless, it seems to be the de facto standard language for anything that falls under the broad category of data science to the point where for me to pick up a different language would feel very much like swimming upstream.
No, they should have made the breaking changes more slowly, piece by piece, with a couple minor versions depreciation cycle. Python 3 was too many changes simultaneously. But it also coincided with dramatic changes in the size and variety of the programming community. It was hard to anticipate the troubles, of course easy in hindsight.
You fail to understand that they purposely broke compatibility for no benefit. That is not improving the language - that is being stupid.
In python 2 you could specify a unicode literal with u''. So a natural evolutionary option if you had a unicode addiction and wanted python 3 to be unicode native, would be to treat u'' as unicode literals in python 3 as well.
This let's the code run under python 2 still, and then you get unicode in 3. And if you are 3 only, just "blah blah" works as well - nice.
But they didn't do that. They purposely took u'' out to make it hard to support 2 and 3. That is absolute stupidity. In other words, they BROKE compatibility for no benefit - but just to break it.
They did the same thing with byte handling, jumped the io abstraction in - these things all destroyed performance and use cases.
Python is FOSS. If there was a big enough market for python 2 it would have been forked and maintained by a separate entity. That it didn't happen proves this is overblown
I guess there isn't really a propper need for a perpetuation of python27. This is mostly people who don't really see a difference between the versions other than that their print statements suddenly fail or that unmaintained dependency they used in that code half a decade ago complains. Who cares about the security pitfalls involved with using outdated python when tour code doesn't work?
Of course python could have dealt better with backwards compatibility, but the python code I wrote was transfered to version 3 pretty quickly.
It happened multiple times, and one of the times, the project needed to be renamed, causing a huge argument, lots of eyes on the project, and momentum to slow. There's a huge social cost to a fork.
The dependency web makes it really hard to upgrade one part. You have to upgrade or rewrite all your dependencies and then do the same to your code: I tried this several times in the ‘07-‘12 timeframe and gave up each time: the cost wasn’t worth the benefits.
Ultimately, I stopped writing Python: these days I write Common Lisp for personal projects and Clojure/JavaScript for everything else.
It was made a lot harder by their efforts around intentional breakage. You couldn't do import from past (ie, to get print "" working). You couldn't do u'' to get unicode compatible string handling between both versions.
The whole Python 2 vs 3 thing turned me off when the split first happened when it was “Python 3000” back in, what, 2008 or so. Every time I‘ve had to do something with Python since then there has inevitably been some sort of incompatibility issue that popped up, although it’s been quite a lot better these past few years.
It always left a bitter taste in my mouth and I always feel like I’m fighting with Python every time a project comes up where I have to use it. There are so many other languages that are a pleasure to use and I’ve always wanted to like Python, I just feel like I’m cursed to always find Python needlessly annoying.
Personally, I don't mind some inconvenience if language designers occasionally need to make breaking changes to allow improvements, and I've had to port enough scientific Fortran code to newer languages to limit my sympathy for porting from Python 2 to 3. Computer languages aren't static, and sometimes I need to do some work to keep up as they evolve.
I have compiled Fortran programs from the 70s on modern platforms without changing a line. The compiler, OS, and CPU architecture had all disappeared but the programs still worked correctly.
I've 'modernised' such an old program then come back a few years later, realised the 'modern' ones don't run anymore and reverted to the old Fortran version.
Well, it is open source. I'm not expecting Python 2 to get completely dropped from Linux distros in the near future. (I could be wrong; I will be disappointed if I am.)
There was an article the other day about problems mixing different python versions on Mac. I admittedly didn't read much of it—but I was somewhat baffled by the problem. I have Python 3 installed on my mac as "python3", and Python 2 (which comes with macOS) as just "python". I think that's the right way to treat them—as different, independent interpreters which can co-exist, as do, say, Python and Ruby.
> These are people that can write shell scripts, but don't program.
Being able to figure out that you're using a different major version of Python is definetly expected of today's biologist. This is part of the profession now.
As if all those shell scripts are portable across time and distributions. Or PHP code. Or really any other code. How many builds of your software are truly reproducible?
Python 2/3 incompatibility is always magnified for some reason, but it's nothing out of the ordinary in my experience.
Didn't we just recently have a highly upvoted/heavily debated HN submission on semver? Enough said...
> As I write Python 3 now, I am less than impressed with the reasons for breaking compatibility. I'm told that it was necessary for proper bug-free Unicode code, but there's enough difficulty when using, say, gzip, and little enough documentation about the "proper" way to handle Unicode that I doubt that it will actually reduce Unicode-related bugs.
This was absolutely essential. Python 2 tried to fix by introducing unicode type. It was hacky, and because they looked for a shortcut, they did a very bad thing by introducing implicit casting (so python 2 automatically converted between str and unicode). They did implicit casting because they didn't made the effort of fixing the stdlib to properly handle unicode.
At that point not only there was a problem that str represent two different things (bytes and text), but also authors that were using unicode (to make their code futureproof) make their life actually harder (in python2 even when you know how unicode works, implementing it correctly is a minefield, it is so easy to do it incorrectly and end up with code that works, but blow in special circumstances).
Python 3 basically decided to rip the bandaid off and do it correctly. And that's what happened. When you program in python 3 you no longer have to think much about unicode. To convert between byes you need to explicitly convert from text to bytes (.encode()) and bytes to text (.decode()) but this is not a concern unless you are doing some kind of serialization.
While the breaking unfortunately was necessary (a lot of python 2 code, while it runs it is broken), I'm wondering if it was possible to do it a bit more developer friendly.
I think things would be much easier if instead of changing the behavior of `str` type, Guido would introduce a `text` type (along with `bytes` type), then later deprecate it. I think that would help a lot. Although in that case modifying stdlib would be unavoidable, so it would be a lot of work for them.
BTW: MyPy was also a tool that could help with the migration, but once again they IMO dropped a ball. Basically when MyPy is used in Python 2 mode, it follows Python 2 typing system, which means it still allows for implicit casting. I think what was really needed is to highlight these issues to allow for fixing python 2 code before doing the migration. The way mypy worked required you to first migrate your code to Python 3 and only after that you could start fixing the text/bytes issues.
On a similar note, both Java and C# made an early and mostly incorrect decision to support utf-16 in the native string and char types. This choice also made some programmers to believe that every unicode character fits in one char, causing bugs, etc.
Yet, I have not seen a rush to "fix" any of them the Python way.
You are mixing the internal representation of text with the change to make distinction between text and bytes.
The python incompatibility wasn't really the Unicode it was that older versions of python mixed text with bytes, and because of that a lot of python 2 code is actually broken. It might run fine, but as soon as input are Unicode characters then many python 2 applications crash.
Python 2 tried to implement Unicode and implemented an `unicode` type. But it actually made things worse due to them adding implicit casting.
The proper way to fix the issue was actually clearly making the distinction what is text and what are bytes, something that was in both Java, C# (both of them are younger than Python BTW, the Unicode was in works when Python was released).
he UTF-16 used by Java and C# as I said is just an internal representation of the text, and that's actually trivially changed since normally those things are not accessible for developers. The only way where it could somewhat leak is default encoding, and maybe names of the methods from stdlib. But those things are really easy to change without much breakage.
Side note, Python internally stores text in many different ways if the text is only out bytes it uses traditional C-string (for performance). Otherwise it also stores as UTF-8, UTF-16, UTF-32 based on characters and requested conversion. So those conversion you said Java or C# did Python 3 does all the time (based on need) without breaking running code.
I don't think I am mixing it up. The char type is not a unicode character just like the byte is not. Yet, these are the characters in a string type. Python went to fix this, jvm and clr didn't.
I'm not that familiar with C#[1], but in Java char is absolutely an unicode character. That's why you have char and byte types. Java clearly separates them, and so is python since version 3.
[1] although from quick googling it looks like C# handles things the same way as Java.
As somebody who programs a lot of python for roughly a decade now I really like Python 3 (although I tone quite a while to switch over completely). Unicode handling really got much more straightforward (which you might notice only if you need to deal with Unicode, which many people in the anglosphere don't do all that often). I like the new formatting options, type annotations all great stuff.
And quite frankly most python 27 code I wrote worked after adding brackets to the print statements.
They are unsupported by the PSF, but still supported by Linux distributions. For example Ubuntu 16.04 uses Python 3.5 and is still supported until April 2021 (security updates until April 2024).
Yep. BTW an easy way to use more recent versions of Python (and multiple different versions) is to use a version manager. I like asdf which is multi language (and multi database)
Yes. I like asdf because most projects are multilanguage nowadays, usually something + Node. Asdf also manages databases so I don't have to keep track of docker containers for multiple versions of postgresql or mysql.
As someone interested in learning Python, does it mean I dont have to worry about version 2.x? Do I just skip any book or tutorial containing that version?
No, you don't need to worry about it. Do ask about it in interviews, though. There are still Python 2.7 apps running in production, and there's a good chance that companies are looking upgrade their code to Python 3.
If you're interested, this document[1] is a good resource should you want to port from 2.7 to 3.
Is there any programming language that's "done"? I feel like any programming language that's "done" is closer to "abandoned" than the developers saying "the language is perfect now, there's no need to add or change anything".
That doesn't really contradict me. You _can_ fix your python version. You can even continue using all the libraries you have installed with your python version. Of course you can't necessarily use newer versions of those libraries or possibly new that haven't been written yet, but of course you can keep using your python and whatever libraries you have installed for it.
But yes you can't expect new code to continue to support your fixed choices forever. Then again that's true of essentially every language out there so I don't really see what's special with python.
Unless things change now that there’s a “steering council” in charge rather than a BDFL, probably never.
Instead, I expect the language to continue along the path of FOMO-driven accumulation of features (and complexity), while still not addressing its major issues (e.g. performance and parallelism).
I believe the fundamental error was to try and break backwards compatibility in a dynamic language. I can see how this would be much safer for a strongly typed language to pull off. It would still be a PITA, but at least I'd feel a bit more save about it. I watched the transition of a NVDA from Python2 to Python3. It took a pretty long time, and some of the commit logs read like voodoo. I would really not want to be in charge of having to migrate a semi-largish codebase from Python2 to Python3, especially if there is only limited amount of automated tests.
In some contexts strong typing means that all values have a well-defined unique type that cannot be changed. For python this means that the language takes care of performing meaningful conversions when a value of a type A need to be turned into type B and also that there is no way to convince python that a value is a String (for example) if that value is not a valid string.
Or from wikipedia:
In Luca Cardelli's article Typeful Programming,
a "strong type system" is described as one in
which there is no possibility of an
unchecked runtime type error.
Since its original design and first release. Try to do 1 + 'a' in Python and see what kind of error you get. Now try it in Javascript, which is actually weakly typed. If you still don’t believe me, try reading the documentation.
It frustrates me that even now I regularly find out about some new thing being written at work, only a few months old but already fairly widespread among internal teams, and...oh, they chose Python 2.7. We not only have Python 3 available but several versions of it, too.
Python 2 doesn’t sound like an ancient, totally-different-language so people just adopt it because it was the version on their laptop or something and because it “works”. In reality, it’s inevitable that some major IT upgrade will finally throw out that interpreter (or, the existing one will fail to work for some obscure reason and no one will fix it because it’s “unsupported”). And at that time it will be non-trivial (to say the least) to fix every affected script and the entire downstream dependency chain because — as so many on HN know — Python 3 is a different language.
Any ideas on how to present this to management as the crisis that it should be? It’s really quite a risk but it is hard to demonstrate the risk when everything “works” so “well” right now.
Middle English still works too doesn't mean it's not a problem to try to use it for a new project. Everything has trade offs, both staying on the old AND moving to the new. At some point everyone does move and managing the size of that transitional tail well is part of running a business well. There is not a one stop answer to the problem of when to migrate to the new and stop using the old but "always bleeding edge" and "always what we started with" are unlikely answers for most places.
> The real crisis is all the lost time dealing with porting to Python 3.
You are agreeing with your parent:
> And at that time it will be non-trivial (to say the least) to fix every affected script and the entire downstream dependency chain because — as so many on HN know — Python 3 is a different language.
Why are people choosing 2.7 for new projects when there is a time bomb coming up? Several years ago, you could blame the Python folks for this break, but once 3.3 (or perhaps 3.5) was out, there was no good reason to start new projects in 2.7, especially when it was publicized that the language will be EOL.
I really think the "Python 3 is a different language" stuff is overblown.
str vs bytes isn't as bad as people make it out to be. You'd be used to it within a week. The other stuff is mostly just semantic changes like range vs xrange or dict.items() returning an iterator now vs a list.
At this point, the bigger issue might be learning all of the new features people were missing out on by not moving earlier, like asyncio.
The syntax differences are indeed minor. But dependencies are not.
If you have a large python2 project with 100 dependencies, will all of those have python3 versions? And will they be compatible with each other in the ways your current dependencies are?
Sure, but even that is dwarfed by the equivalent problems in the ruby community. Most gems die after X years. Ironically some part of this must be explained by Python's enormous popularity.
Not wanting to update dependencies is fine, but in that case I'm not sure how it's an issue that you also have to use a certain version of the interpreter?
There is a pretty good chance they all have a python 3 version.
It's not easy to determine which versions of the library to upgrade to however. What's the latest version compatible with both python 2.7 and 3.7. Would be nice if pip had a setting to get that.
At this point all packages work with Python 3. If you use ones that don't, they are not maintained, and you shouldn't use them.
[1] A one that I often seen was MySQL-python. That package hasn't been touched since 2014. Another one I've seen was PIL, that one was last released in 2009, and isn't even on PyPI, because it is that old, fortunately maybe because it's not on PyPI, people are more aware of Pillow which is a drop in replacement
The primary practical effects of staying on Python 2 are
- You won't be able to use newer versions of open-source libraries or newly-developed libraries (including both new features and security updates - note also that a project's developers won't be looking for security bugs in old versions of libraries and likely aren't supporting old versions, but attackers probably are looking for them!)
- Your company ends up with two sets of code in effectively two different languages, and you can't extract common libraries, reuse tested business logic, etc.
Those are things the business is likely to care about. It would be particularly helpful if you can show that you were unable to use the new thing because you had existing code in Python 3. or the new thing was unable to use new features of the libraries they depend on.
Do you have a security team who cares about running supported versions of software or vulnerabilities in deployed software? Can you find libraries in their `pip freeze` that have had security updates that weren't backported to the Python 2 version?
Have you had problems when a team using one Python interpreter needs to call code written by a team using another interpreter?
Writing new projects in Python 2 is rather like writing new projects in Visual Basic 6, in that it will certainly still work in 2020 and technically it's easy to get started - except with the advantage that 2 and 3 are actually quite similar languages. If your company would try to pressure folks writing in VB 6 to use something else (VB.NET? Qt? Electron?), imagine you're in that situation, except it's magically much much easier to do the port.
Also, and importantly, you can port file by file (if you use six and test on both versions). That's much more incremental and safer than a port by rewriting.
> Any ideas on how to present this to management as the crisis that it should be?
Show them the EOL document, and ask them if they can afford to have their systems owned when an exploit is released for Python 2.7 that goes unpatched because the project is unmaintained.
It's easier to start new projects in a supported version, than it is being forced to port your applications at the 11th hour after exploits in unmaintained Python versions are found.
From a developer's perspective, you're coding in a time machine with Python 2.7. Updates to community packages from the last decade haven't been backported to their Python 2.7 versions, so you'll run into an accumulation of bugs at every turn. If you're interfacing with an API, you probably won't find an off-the-shelf solution for Python 2.7 like you would Python 3. You are literally creating more technical debt with every line of Python 2 that you write.
For those that think migrating from python2 to 3 is easy: no. You’ve clearly never tried to migrate different companies/environments.
Sure, the code now runs in the py3 interpreter, but guess what!? That was the easy part! Now go hunting down Byzantine build scripts and environments to find out how it was deployed and tested. Tested lol? Ugh and guess what? The code base has commits that we’re never deployed to prod. What do you do now? You basically own a service that hasn’t been deployed in 4 years and a branch that was never deployed ever.
The things you describe are real problems but they are orthogonal to the Python migration. You have to pay that cost eventually, with or without migrating. It's not just Python that moves.
I don't really understand all the grumpiness around Python releases. There are lots of programming languages that depend on interpreters or VMs (Java, Erlang / Elixir, Go [kinda]). Those languages have versions which are no longer supported and, as far as I know, that's not controversial.
The old versions exist. They are not supported. You can choose to use them if you wish or use the latest. I don't see many complaints about the fact that Java SE 6 (or whatever) isn't getting updates anymore.
I think the Python team fucked up the 2->3 transition, but that was 12 years ago! Do other people have explanations as to why old versions of the Python interpreter are associated with so much emotional investment?
Go 1.15 can compile Go 1.0 programs. That's why I'm grumpy with the Python team but not grumpy with the Go team. There's simply no more to be said about the situation than that.
It's also "Go one point ten" and "Go one point fifteen". That's a minor version update. Backwards compatibility is common between minor version updates - in fact, it's part of a major version update that backwards compatibility breaks.
It is minor because the Go team understands their responsibility not to make major changes. Which they learned in part from watching Python screw it up.
I haven't spent a lot of time with Java, but you can generally compile old java source with new compilers as well as run old java bytecode with new runtimes. Old and new code can interact with reasonable limits. You may be limited by the available runtime depending on where you need to run code, of course.
The same is true for Erlang/BEAM/OTP. You can even dist connect old and new VMs as long as you're careful not to send things the old VM won't understand (like Maps to a BEAM from before Maps). Erlang does tend to actually remove deprecated things after a few releases, unlike Java though, so it can be difficult to support more than two or three releases.
In both Java and Erlang, you can upgrade before your dependencies, at least most of the time, but that doesn't seem to be the case for python 2 -> 3, which means people are stuck on 2 when they want to use 3, until their dependencies update or they take over or abandon them. That's not a quick process which is why people are still upset.
“The plan is not to do 4.0 until 2.7 is barely a faint memory, and then we will consider how to handle the transition very carefully.” - Guido van Rossum
On most Linux distros the `python` package corresponds to Python 2. Likewise, most distros have a `python3` package with a relatively recent Python 3 version. This is for compatibility reasons and won't change anytime soon.
The most notable exception is probably Arch Linux, which has Python 3 available in the `python` package.
This is kind of Debian stable's appeal, and shouldn't be a surprise. Python 3 is behind the python3 command.
Linux distributions that appeal to those who like to be on the cutting edge have pointed the python command to python3 already. Debian stable will be there one day, too.
The tools for upgrading code to Py3 paradigms started in 2.6 however. There was plenty of time to do it. Unless a project dealt heavily with unicode the transition was trivial if any effort was made.
Yes the translation tools existed, but they were not quite enough. Besides real world doesn't move in one swoop. You need co-existence.
I very much wanted to move to Python 3, but critical libraries I was using were not there. And running both 2 and 3 at the same time was very unergonomic until python-future [1] became stable and widely-used.
I started a couple of green field projects in Python 3.3 and Python 3.4, and have to disagree. By that time most third party packages had moved to Python 3, and I had no trouble building large applications with it.
Python2 aside, judging by the rest of these posts, maybe it's just me but I haven't seen any problems with forwards compatibility in Python 3. That is, stuff written 5 years ago runs fine on 3.8. The only issues I've had is with virtual environments and pip freeze because some older Python versions relied on packages for features now in the Python standard library, but since we know how to spot those now that's not too bad of a fix.
On the other hand, Python gives a growing and continuously maintained standard library and none of the headaches that, say the JS ecosystem gives.
That said, not all of the features are great all the time. Async/await comes to mind, which adds a lot of complexity without the performance gains even senior programmers might expect. But overall, we're already using features in Python3.8 for projects, and have upgraded our instances to Python3.8 without issue. I guess it's possible we leave some pip dependencies in the dust because they may have some incompatible code with a future 3.x release, but we haven't had a problem before. Tests permitting, we'll do the same for 3.9 if we find something we like or might wait a few years.
> and none of the headaches that, say the JS ecosystem gives.
ECMAScript is the language, and it is amazingly backwards compatible back 20 years to about the start of the century without any trouble, and plenty of tooling like TypeScript can target ECMAScript 3 released a score of years ago.
Python could have added a ‘use strict’; in 3.0 to avoid a breaking change!?
Sure, since then browser features have radically changed and frameworks have changed, but the language itself has been extremely solid. Conflating JS and the ecosystem is not a fair comparison.
Also there have been many competing different implementations of JS engines, with amazing levels of compatibility between them (even Internet Explorer 5.5 released 2000 only had a few very minor known quibbles in the ECMAScript implementation).
Edit: Of course Unicode is actually hard to deal with because ECMAScript uses UTF16 for strings and symbols, even though the source code is usually written in UTF8. And note that the u flag was added to RegExp expressions to deal with Unicode better in ES2015 (especially U+10000 and greater, which are generally difficult to deal with in ECMAScript code).
On a technical level, you're absolutely right, JS as a language has an amazing amount of backwards compatibility. So much that IE running on Win XP can run things that Babel compiles from ECMAScript X in 2020.
But, the ecosystem is really something different entirely, and this isn't meant to be a slight to JS at all. It's just reality when running production code in 2020. Python's standard library is comprehensive, well-maintained and well-documented throughout. JS, by comparison, relies on packages from npm for the same features. Which means the issue my org has with the JS ecosystem is the stuff that's not built into the language spec or otherwise maintained by a central body, not that the things built into the language isn't standing the test of time or something like that.
> But, the ecosystem is really something different entirely
Most of the ecosystem changes are driven by externalities - web-servers, browsers, developers. You can still choose to use an old web framework and it will work on modern browsers.
That is entirely different from the Python 2.x -> 3.x change.
The days of a simple CRUD web page delivered by a server are long gone. jQuery is still brilliant at adding adornment to HTML produced on a server. Putting your view and controller into JavaScript just makes sense.
SPAs need completely different frameworks to support the highly complex needs of client-side rendering and interaction. Ten years ago component based renderers were not common and developer-friendly syntax hardly existed. Frameworks are being pushed by the needs of huge systems (Facebook), server side pre-rendering, and partial updates (correct term? to improve first load and responsiveness).
Also the features available in a modern browser are radically different. Remember building up a string of HTML in JS for performance reasons? Avoiding the DOM because it was slow? So many features only started becoming reliable or available around Internet Explorer 11, yet now IE11 is hardly ever supported because it is missing critical features. An example is that in older Internet Explorer and Safari, you could theoretically update CSS rulesets from the DOM, but in practice doing anything dynamic risked crashes and memory leaks (even loading new CSS pages was difficult and error prone and browser specific e.g. detecting CSS load failures).
Note: I was supporting the same JavaScript codebase for over a decade (initially IE5.5). There were no breaking changes in the ECMAScript, and very very few in the DOM (mostly driven by dropping IE specific DOM). The changes in the codebase were mostly driven by wanting to support new features in browsers.
The Fortran spec was published in 1957. Today’s Fortran has features that were not dreamt of then, such as array operations with automatic parallel execution. The changes are a bit more drastic than improving Unicode support. But I can take a Fortran program written in 1960 to run on a computer that needed its own building, that was fed into a machine without an operating system using punched cards, and compile and run it on my phone.
I don't remember the details, it's been nearly 30 years, but ... I had to fix Fortran-II (1960 or so) programs so that the fortran-77 compiler (on the same IBM VM/CMS 390 system) would agree to compile them. The changes weren't big, but they were needed. (I didn't write the original - I was just the maintainer at the time)
I also needed to change a lot of things porting the same program a few years later to an IRIX based machine whose name I don't remember right now (Octane, perhaps?). And that was a well behaved program that did not try to change the value of 4.
Python 0.9.1 programs (Earliest I used) required much fewer changes to run on 2.7 -- partly because I never used stuff like field access control (which was luckily dropped from the language). Python 3 requires a few more changes, but I suspect I can still compile Python 0.9.1 on modern hardware and run the original code.
This is a common experience, but it arises because there were many slightly different versions of Fortran in actual use, taking advantage of the features of the many different compilers. But a program that stuck to the official standard generally never needed to be changed. (But I might be exaggerating about 1960. I don’t know when the standard settled down and the commitment to non-breaking changes was adopted. But it was a long time ago.)
And amazingly it compiles, and does so in just 1.5 seconds on an iMac (unlike my Typescript project which is compiling at 90 seconds and counting.... hmmm).
Unfortunately, even now with access to the source code I can't find anything about field access control.
Python had keywords that were supposed to make methods and fields private or read only or something like that. I remember they were gone by 1.4; I never liked or used them. I am on phone now, and all google results now are referring to name mangling and underscores. When I am on desktop, I’ll dig up the version and syntax.
Ok, I've just downloaded the 0.9.1 sources, and it's not there yet, and in 1.4 it's not there anymore, must have been somewhere in between, maybe 0.9.4 or so;
I'm not sure if they were ever properly implemented, but there was a point in time in which the grammar had some provision for declaring methods private and/or fields read-only or something like that. I do remember I disliked the idea and never used it, and likely it wasn't popular because it's gone.
I can't find any version other than 0.9.1 to download, and trying to do newgroup archeology through Google Groups is incredibly frustrating. I've given up after about an hour of trying to find old mentions.
I don’t use Fortran regularly, or much at all, any more, so I’ll defer to someone who knows. But I think that after some point, the standards committee made a commitment to backwards compatibility, and these deprecated features actually still work, although people would be unlikely to use them in new code, because they are now considered bad practice.
Virtually all compilers still support them, even the most modern compilers that support Fortran 2018 (Intel, GNU, IBM, NAG, ...). The source file extensions readily tell the compiler what sort of code the compiler is dealing with. In the worst case scenarios that I have seen, the compiler issues a warning when encountering obsolescent language features.
Up until the Coronavirus hit, I was a volunteer teacher at a Girls Who Code club, where we used Python. I spent our very last class before the universe collapsed introducing students to the concept of functions.
Once we'd finished going over some example programs, I asked the kids to modify a quiz program they'd written previously with a different teacher. That program used copy-pasted lines of code in order to ask multiple questions, and I wanted the students to re-implement it with a single function instead. This was a pretty challenging assignment—the kids had never encountered functions before today, and I was purposefully not quite telling them what to do, only giving examples of functions used in other contexts. But one of the girls managed to get it, and I was really proud of her.
Except her code wouldn't run because—I realized with horror—the original program she'd done with the other teacher was actually python 2 code.
I am so damn frustrated that this happened! I was trying so hard to make these kids understand the concept of functions; I did not want to explain the differences between Python 2 and 3. It is, of course, primarily my fault—I should have coordinated better with the other teacher—although in my defense, we were both volunteers, and as it is I spent most of my Friday and Saturday planning this lesson.
It also should just never have happened! I'm not aware of any other language where this type of confusion is so common, and it really makes me wonder if we'd be better off teaching something else. I'm not sure who is to blame for the situation, but it really sucks, and I don't think it's going to go away just because Python 2 is now technically EOL.
Ruby went through a similar period when transitioning from 1.8 to 1.9. Strings learned about encoding, breaking most libs handling strings, some syntax was added, some removed. It was possible to write 1.8 and 1.9 compatible code, but it took some effort. However, the big players in the community, rails etc. tackled that change head on. It was a point of pride for libraries to be 1.9 compatible. And everyone moved on. I’m always puzzled by how hard the python community ist stuck on 2.x
Python didn’t have a single big player like Rails to “lead the charge”. Instead, its ecosystem is made up of many smaller libraries.
Because it’s not possible to run Python 2 and Python 3 code together, none of those libraries could upgrade until its dependencies had done so. This just led to everyone waiting for everyone else.
Point 2 is False. It's easily possible to write code compatible with both 2 and 3. It's becoming less pleasant as one can't take advantage of the great new features from v3.6+, but I've done that for about a decade now.
I agree somewhat with Point 1, but the Python ecosystem is larger than Ruby's. Each domain could have moved separately, and did, to some extent.
Sorry, I meant that Python 3 code can’t use dependencies that are written in Python 2. Therefore a library couldn’t upgrade to Python 3 until all its dependencies had upgraded.
The scuipie od changes was also intentionally smaller. I've seen programs where all it took to fix changes was maybe 15 minutes if you actually replaced the code, instead of injecting few methods to String class.
I would posit this is an equally valuable teaching moment for different reasons. I completely empathize with you, but this shit happens all the time in our industry. A brief sidebar to tackle breaking API changes (even if it’s, hey this is a shit show we will circle back to later) would be a helpful seed to plant in the mind of a young engineer.
Anyone teaching Python 2 now is years out of date. It's not your fault that the other teacher isn't staying current. As a teacher, I made the switch around 2016 and I was slow to change.
While the Python core team made mistakes in community management, I wouldn't blame them in this case.
Back in high school, I volunteered to tutor math at a free summer camp for poor children. A professional high school teacher was also volunteering. I wound up tutoring her in math, so that she could teach the kids. Sometimes teachers need a little update.
I talked to her about it later. Without getting too far in the weeds, part of the issue is this browser environment we sometimes use, called Trinket.io. It seems to be very common in the educational space.
The paid version of Trinket let's you use Python 3, but the free version forces you into this hybrid interpreter which attempts to run both kinds of code, and it does a bad job.
The only reason that I can come up with for teaching Python 2 in 2020 is that the instructor only became familiar with Python because they were expected to teach it.
This is like complaining that calculus is too fractured because someone taught your students to use Newton's notation instead of Leibniz's.
I am soo glad, I pushed back on python migration in my team until 3.6 was released. We were running large amount of django code base in 2.X and 3.6 was the only version which had some reasonable improvements to upgrade to. I sincerely hope the python community has learnt it lesson and wont break backwards compatibility in 4.X releases.
I would have been okay if the only breaking change in python 2 -> 3 was strings are Unicode by default.
But they went with the logic of “if we’re gonna break, why not break a bunch of other things and take this opportunity for a different python”
Needless to say, the folks in charge of python didn’t think things though. It’s a big fat breaking change with no real automated way to upgrade. A whole bunch of apis are now different. At my company even after a year it’s still a pain having some scripts only work in py2, while others in py3.
317 comments
[ 3.7 ms ] story [ 290 ms ] thread* minetest
* mpv
* npm
* qbittorrent
* smbclient
* virtualbox-6.1
These are the more popular ones. I write and don't hate (compared to Python2) Python3 but apparently if I remove Python2 I have to shed all these programs because they still depend on it!
Maybe it's a VapourSynth thing?
1: https://release.debian.org/buster/freeze_policy.html
[0] https://www.python.org/dev/peps/pep-0414/
It's not impressively long, it is artificially long. It was decided there would be no 2.8, so they kept 2.7 up for compatibility reasons for a long time to make it easier for people to transition.
https://www.python.org/doc/sunset-python-2/
I expect the pains of dealing with 2vs3 to continue for few years at least, and that's with policy of avoiding and removing python were possible.
The slow approach failed. Chances are, most Python 2 code was written after 2010. By switching earlier, yes, you make the switch harder for the handful of people who actually did port their code, but on the other hand you avoid the exact same porting difficulties for many more because the majority of Python 2 code would never have been written! Someone upthread mentioned scientists, for example, who were still writing their analysis code in the late 2010s and are now confused about why none of it runs.
To be sure, it's worth thinking about why it failed, as many other projects have been able to get away with a longer deprecation periods. But its seems like there is a case to be made, in pure consequentialist terms, that Python would have been better off killing Python 2 earlier.
ARTHUR DENT: Yes! I went round to find them yesterday afternoon. You hadn’t exactly gone out of your way to pull much attention to them have you? I mean, like actually telling anybody or anything.
MISTER PROSSER: The plans were on display.
ARTHUR DENT: Ah! And how many members of the public are in the habit of casually dropping around the local planning office of an evening?
MISTER PROSSER: Er – ah!
ARTHUR DENT: It’s not exactly a noted social venue is it? And even if you had popped in on the off chance that some raving bureaucrat wanted to knock your house down, the plans weren’t immediately obvious to the eye were they?
MISTER PROSSER: That depends where you were looking.
ARTHUR DENT: I eventually had to go down to the cellar!
MISTER PROSSER: That’s the display department.
ARTHUR DENT: With a torch!
MISTER PROSSER: The lights, had… probably gone.
ARTHUR DENT: So had the stairs!
MISTER PROSSER: Well you found the notice didn’t you?
ARTHUR DENT: Yes. It was on display in the bottom of a locked filing cabinet, stuck in a disused lavatory with a sign on the door saying “Beware of the Leopard”. Ever thought of going into advertising?
A: We talked about it at software conferences, on the Python announcement mailing list, on official Python blogs, in textbooks and technical articles, on social media, and to companies that sell Python support.
Where else would you like them to have made it clear?
It baffles my mind, because I've lived through PHP 4→5 switch. It was a major breaking language change that affected every object assignment. It felt like it was dragging on forever, but in retrospect, compared to Python, that was a quick and successful upgrade.
As I write Python 3 now, I am less than impressed with the reasons for breaking compatibility. I'm told that it was necessary for proper bug-free Unicode code, but there's enough difficulty when using, say, gzip, and little enough documentation about the "proper" way to handle Unicode that I doubt that it will actually reduce Unicode-related bugs.
Python is still a great tool, but Python 3 really took the shine off for me.
Then it could tack on a "looks like you're trying to run a python 2 script on python 3" final error to stderr.
so “helpful”!
There are hundreds if not thousands of pages of unicode documentation. If anything the problem is that there is way too much for any normal person to actually understand it.
For most people though, knowing when to normalize strings to NFC versus NFKC is all they need to not unknowingly accumulate tons of technical debt.
For example, I coded along in Python 3, not changing and IO handling for the longest time, but when I Tried to use the gzip module I suddenly had to learn a whole bunch more knowledge about Unicode. I looked around for about 10 minutes for a some sort of best practices guide, but ij lieu of that just mucked with encode/decode functions until it worked. I don't know if I did conversions at the right spot, or where the guidelines are for changing between bytes and characters, but it didn't matter for my data at the time and o had a deadline so I couldn't really justify spending another hour at the time to figure it out.
Maybe I'm just trying to justify my behavior to myself, but I think this sort of problem solving is the norm. I tend to try to adopt best practices and educate myself more than the other programmers I have worked with, if not more, and am the one introducing flake8 etc. to code reviews, so I'm not the worst sort of programmer, though I certainly wouldn't claim to be top-tier either.
This is coming from someone who's written a lot of Java (Java's had the byte/char distinction since forever); the way Python 2 pretended byte strings are strings really was broken and not what you want. What the changes in Python 3 mean are that you have to actually think about if something is bytes or a string, and you have to be intentional in choosing encodings.
Sure, you can argue that iterating over bytes is less "correct" than iterating over Unicode code-points. But iterating over code-points isn't very useful either -- maybe that code-point represents the letter B, but maybe it represents the third of seven characters and modifiers and zero-width joiners that make up one family emoji.
What should `str.reverse()` do? It's very different for byte strings and UTF-8 strings. Or `str[4]`? `str[1:]` can corrupt a string for one definition of []. What about Windows users? There's a chance Powershell is using Windows-1252.
The same thing applies to indexing and slicing. You might get an actual character, or you might get something invalid by slicing halfway into a multi-code-point sequence.
So Python 3's model isn't actually that much better than Python 2's at handling these cases. But it comes at a significant cost, introducing "unicode vs bytes" confusion, and making it more difficult to write correct programs for platforms like Unix where filenames aren't necessarily valid Unicode.
I reversed a string several times in Python 2 and not once did my computer catch fire! I just got a string back that, in certain situations, didn't contain the characters I was expecting. And then my father had to have a talk with me about Unicode, and how computers sometimes use multiple bytes to represent a single character, and so if you rearrange the individual bytes you get different, usually more broken-looking characters.
And when my child runs "🇨🇦"[::-1] in Python 3, I'll still have to give them The Unicode Talk. I just need to use the word "codepoint" instead of the word "byte". The situation is basically the same--if you know enough about Unicode to understand Python 3's behavior on certain non-English text, you know enough to understand Python 2's behavior.
What were you expecting to get when doing reverse on 🇨🇦? The flag symbol is encoded using regional indicator symbol letters: C, A. When you reverse it you get regional indicator symbol letters A, C which represents Ascension Island. In python2 you get a total garbage.
The flag representation is not part of unicode, unicode only specifies that you can encode country codes using ISO 3166-1 alpha-2 and it is up to the application how to present it.
My future child sees only one character in that string, an emoji. (If you put the cursor to the right of 🇨🇦 and press backspace, the whole flag goes away, so that must mean it's only one character, right?) If they reverse that string, they'd expect to get the same result back.
> The flag symbol is encoded using regional indicator symbol letters: C, A. When you reverse it you get regional indicator symbol letters A, C which represents Ascension Island.
Right, this is exactly what I'll have to explain! Just like how in UTF-8, "é" is encoded using two bytes: 0xC3 0xA9. So if you reverse that, you get bytes 0xA9 0xC3, which do not represent "é". It's the exact same problem, you've just moved up one level of abstraction. (The Python 3 version of the problem isn't unique to flags -- there are all kinds of modifiers and ZWJ sequences and combining characters that get messed up if you rearrange individual codepoints.)
A few languages have tried to solve this problem in other ways, like how Swift considers each character of a string to be a "grapheme cluster", but that comes with a lot of its own drawbacks.
With combining character sequences there's at least a normalization and you can transform to the right representation based on your need.
As you mentioned that Swift allows one to operate on grapheme clusters, and it comes with advantages but also some issues as you pointed out.
The topic is complex but Python 3 does what unicode standard specifies, while Python 2 would output a garbage in similar scenario.
In hindsight, I think the proper decision would have been to ensure every string has an accompanying encoding attribute, and refuse operations on two strings of different encodings (or even perform on-the-fly encoding changes). At least that wouldn't have created a source-incompatible language.
[0] Here's some of the operations that should absolutely be invalid on a byte array, but aren't because Python3's bytes object isn't actually bytes, it's 8-bit ASCII: lower(), title(), upper(), isalpha(), isdigit(), isprintable(), splitlines()
Python is a strongly typed language, and you can check a function's argument types at runtime with isinstance(), and inspect those types' class hierarchies with issubclass().
Yes, you can check at runtime, the problem though is that especially on Python 2. Those checks don't happen very often.
Python 3 though will crash if you try to use bytes as text or vice versa. The python 2 though it will do an implicit casting and execute code, which will run and look like it runs correctly until you get an unicode characters, then it will crash.
Java won't even let you compile such program. You could get somewhat similar protection if you annotate types and use mypy to check them.
> In hindsight, I think the proper decision would have been to ensure every string has an accompanying encoding attribute
This wouldn't have made things any more source compatible. Adding more types doesnt address the issue that python2 string was multiple things masquerading as one.
I guess they can be, but the code might also work on an int tuple, even though the operation doesn't make any sense. Here's a case where duck typing does something, but it doesn't make sense.
Python's built-in sum won't work on strings or lists of strings (so much for saying the language is duck typed), but this will. It's also weird because addition is associative, but not with strings, so changing a+b to b+a changes the behavior and gives you a very cute answer to the "reverse a string" algorithm: In practice, every legitimate use I've seen for duck typing might as well use what would be an interface in Java. I see duck typing as interfaces without the formality of declaring them. If you violate the implicit contract, funny things happen, and this is why Typescript got so popular and Python added type annotations.I won't claim here that the compatibility break was actually worth it, but I am sure that it reduces Unicode-related bugs.
It's easy to write Python 2 code that seems to work fine but immediately crashes when it has to process Unicode. But if your Python 3 code works at all it'll probably handle Unicode fine, or at least handle it better than naive Python 2 code would.
Whenever I have to write Python 2 code that has to handle Unicode, I mainly just try to write it so that it would be able to run in Python 3.
Yeah, that's the biggest benefit, and the reason python 3 handles it better, is because it crashes early. When it it sees that you are making an incompatible conversion, it refuses to run without waiting for some specific unicode character to show it.
> Whenever I have to write Python 2 code that has to handle Unicode, I mainly just try to write it so that it would be able to run in Python 3.
Thankfully I no longer have to write python 3 code, but from my experience, it works the best to write the code for python 3, then adapting it (sometimes with help of six package and imports from `__future__`) to work with 2.7. It is easier than do it the other way.
Can't speak for smaller businesses, but in larger companies, there is so much "waste" that the cost in $$$ to do this is insignificant in comparison. And if you choose to spread this effort out over, say, 5 years the annual cost is miniscule.
- ANSI C was incompatible with pre-ANSI C. Current C has various minor incompatibilities like the removal of gets().
- bash defaults to non-backwards-compatible behavior and lets you set various shell options like "compat31" if you want it.
- Newer version of the JDK generally require code fixes.
- PHP has made various backwards-incompatible changes, like the removal of magic quotes in 5.4.
There's also many more cases of languages getting forked in backwards-incompatible ways and going on to be successful. C++ is doing fine despite not being fully backwards-compatible with C; VB.NET broke compatibility with VB; zsh isn't POSIX-compatible by default; there were a million variants of BASIC back in the day, and still are a million variants of SQL; etc.
The hardest part (for me) of getting started with a new language is setting up the environment to run properly... Installing libraries, configuring environment variables, pointing the IDE to the interpreter, etc. So of course that's where I assumed my mistake was.
It was only much later that I realized that I was running Python 3 (newer is better right?) and the tutorial was written when Python 2 was the latest version. So the print statements changed.
And of course, "print" is in everyone's first program. So it broke the experience for all first time users.
To be fair, Python 3 provides a very clear error message if you try to use print as a statement, although I don't know when exactly that was added:
To be fair, nobody reads error or warning messages.
But when I think back at that 4 hours and how much time will it take to do it today, it reminds me of all the learnings. Is it load or loads? Which one is from file? What is a json object? Do I need to quote? If so how? Why does it work with the small block but not the big block? Is it the nesting? Am I doing something wrong?
I think the same is true of hello world. Hello world is a newbie presumably who has not written a single line of code. Well you copy and paste and it does not run. Yes the error is there but hey I copied working code : did I install something wrong? Should I check some website? It says it is python but is it? You can't imagine the questions that will come to a newbie. We need to be mindful of this rather than have a stack overflow mindset.
Learning that error messages are useful and can solve your problem, and de-training the reflexive "This message won't help me", lies on the end user side of things (assuming the devs did their part).
Rust generally tries to do that, although the fixes it proposes tend to lack context, and can thus lead you into the wrong direction entirely (e.g. once upon a time it would tell you to go and implement a trait on your type when the issue was that you'd forgotten a trait bound on a generic type, it still kinda says that but now there's a snippet which adds the trait bound so it's less error-inducing).
If run in interactive mode, the Python interpreter takes its cues from vi and deliberately frustrates the obvious methods to quit. Ctrl-C results in Python whining KeyboardInterrupt but doing nothing helpful. Typing 'exit' or 'quit' both result in messages that make it clear that the interpreter knows what you want to do but won't comply out of spite.
More fun can be had with certain versions of Python 3 when running on Windows, where the Python print function would throw an exception if told to print Unicode characters. This made printing to the console non-portable because Unicode characters had to be stripped if running under Windows. This seems to have been fixed, however.
Still existing are the misleading warnings that Python produces if you forget to include the self parameter on a class method. Python emits an error that blames code that calls the method for passing one too many parameters instead of identifying the underlying problem. Python also gets similarly unhappy, and points fingers at the wrong code, if the brackets are omitted when instantiating a class.
Since Python is dynamically typed, these errors can't be identified until execution. Requiring all-paths testing to catch syntax errors is a special kind of user hostility in and of itself.
Python is an utterly horrible language and it's deeply unfortunate that it has become so popular when so many less error-prone, less obnoxious, alternatives exist.
The interpreter knows no such thing. Both quit and exit are regular python objects that implement a __repr__ method, both of which return a string that inform the user how to exit. The __repr__ method could have implemented exit, but that would have been poor form (and create issues with tooling), set bad precedent, and go against python coding guidelines.
Ctrl-C giving a keyboard exception rather than exiting is a good thing, as it is far more likely that you are trying to terminate a long running statement, or exit out of sub code block, than trying to kill the interpreter process. This is also the standard behavior of almost any interpreter you care to name (irb, node, clisp, shell). This is because the interpreters try to mimic unix shell semantics (CTRL-D also kills your shell), since they are, after all, shells.
Code is sometimes alive and changing, and sometimes an artifact. Requiring that code become alive after being an artifact carries significant risks. One might say "code should never be unmaintained" but I think that ignores the reality of limited resources that all of us deal with.
It would take quite a lot of effort to offer the new print function and the old print statement at the same time, and it would require either a lot of special cases or renaming the new print function.
And there's really no upside. If the only legacy thing in your script is the use of print statements, that's trivial to fix by hand or with one of the automatic converters.
Yes there is, the print function is the biggest syntactic incompatibility. Besides the (optional) u marker for unicode strings that they reintroduced later. Most of the other difference (like changed packages) are semantic.
You would have been able to import and run Python 2 in Python 3, maybe with a couple of "if"s or some magic in the module loader. Breaking compatibility like this felt petty and deliberate and made it really hard to have one codebase target both python 2 and 3 (temporarily).
In this specific case I think it's a non-issue. If python3 was limited (or limited except in very niche cases) to changes that could be made programmatically, we wouldn't be in this situation. Everyone would just run 2to3.py and never look back.
A "trivial" obstacle to a seasoned programmer can be an insurmountable obstacle to a beginner.
A print "function" conversely is quite easy to upgrade to a log "function".
In addition, "print" is almost always a code smell in professional code. The moment you need to localize, change destination, etc., "print" goes right out the window.
I have lots of beefs with Python 3. Changing print to a function is NOT one of them.
SyntaxError: Missing parentheses in call to 'print'.
This doesn't give the "did you mean" hint, and while it's perfectly understandable for someone who's gotten as far as learning exceptions in Python, that usually comes a few chapters after "hello world". It's definitely enough to throw a newbie off. And even if they're a veteran of other languages, they might have been exposed to a wide variety of error messages like this that commonly indicate a problem elsewhere, and having never yet successfully run a python script at this point, the prior for an error in configuration is much higher than for the tutorial being wrong.
There were so many routes forward that didn't involve breaking everything but still would have allowed fewer unicode bugs (maybe). And they could have done a lot more to make the transition easier if they wanted to - some of it was just in your face type breakage to a prove a point without considering some underlying use cases (a lot of internet protocols, headers etc still ASCII).
What is nuts to think about - if tweaking some unicode handling is worth creating an incompatible version, what happens when some random python developer has another itch to scratch that they consider so important it is worth break a language over. The idea that these types of fixes require breaking compatibility is absolutely insane if you want to address a larger / longer term market.
Complain all you want about Microsoft, C etc. Even go of all languages (because google has an otherwise terrible rep in this area!!). But some of these languages and companies really do seem to be able to deliver or intend to deliver longer term compatibility.
When businesses and lifetimes are measured in decades (not months) it is VERY nice to build on stuff that has longer support horizons.
> And they could have done a lot more to make the transition easier if they wanted to - some of it was just in your face type breakage to a prove a point without considering some underlying use case
Such as?
> (a lot of internet protocols headers etc still ASCII).
What do you mean here? Py3 correctly handles http headers encoded as ascii bytes. Yes the decision to make http headers ascii bytes and not basically anything else is bad in hindsight, but that ships sailed.
Why should I have to do labor to fix things that already work?
You don't right? Python 2 still works.
In theory if there was sufficient need the entire community of python2 users could collectively fund said support but I don't see how they can demand it for free.
Also having your new version suggest using the old version is counterproductive.
Python 3 provided no extra value at a high migration cost and painful library ecosystem issues. This is in contrast to typical programming language version migration, where breakages are either avoided entirely or are extremely localized. For example C++11 had some breaking changes compared to C++98. I don't know if I ever noticed them, but it didn't matter since C++11 added so much value that everyone wanted to migrated ASAP, and anyway there was no need to move the whole ecosystem at once.
No matter how long the transition period was for Python 3, there was not going to be a good time to migrate. So most people would leave it until the very last minute, resent every hour of time spent on it, and sometimes put the option of rewriting in another language on the table.
I disagree with this. There was a lot of value add for specific (and relatively common) workloads. If you were doing unix automation you didn't gain much (but I argue this is because unix does unicode bad, not because python). But web development did, even if most of it wasn't obvious or was too soon to be obvious.
As for c++ the effort the language goes to to maintain backwards compatibility has costs, both in terms of syntax and semantics. Those costs push people to rust etc. Much the same that migration to py3 pushes some people to go. It's difficult to say which is right without the ability to time travel.
Hell Larges parts of Windows is still code that is decades old...
This "well you had plenty of time" shows just how much many modern / web developers do not understand Enterprise code, code that goes into manufacturing or other long running processes where "new" machines are 15 years old
> other long running processes where "new" machines are 15 years old
Then...pin to a 15 year old version and use that. No one is forcing you to upgrade.
File handling in the new unicode world was something like 2-3x SLOWER. This is not percent, something which took 3 minutes all of a sudden took NINE minutes because of the io module.
URL parsers became a nightmare - dual implementations of codecs started poping up.
https://lucumr.pocoo.org/2014/1/5/unicode-in-2-and-3/
For a "better" python, I didn't really start using it until 3.5 or so. Now it's much more pleasant, but at the key point of transition when everyone was going to check out the new thing, it was slow, put lots of extra bumps that appeared unnecessary to solve the problems that needed solving etc.
> File handling in the new unicode world was something like 2-3x SLOWER. This is not percent, something which took 3 minutes all of a sudden took NINE minutes because of the io module.
Yes, unicode is slower. If you weren't using unicode you could and can still do all your processing in bytes mode.
This is what worries me - the lack of understanding of how broken these backward compat breaking changes are. Mercurial, built on python, took 10 years of work to get to python 3 - for basically no reward (they just need bytes and unicode is not super relevant to them) as just one example of MANY.
Let me give you an example. in Python 2 you could use u'' for a unicode string. Perfect, I could update python 2 code to use this or b'' if I needed bytes so in the python 3 setting the right things could happen. The python developers literally TOOK THIS AWAY in python 3. So now, for no reason we can ever hope to understand, you couldn't write code that work work in 2 and 3 by using u'' and b''. This was the type of screw the developer in your face attitude that developers got.
Let's be clear, library folks would need to write code that worked for python 2 and 3 for a long time. And instead of making this easy or at least reasonable, the python developers went out of their way to basically screw folks over here.
Others will remember that the very reasonable formating approaches to bytes (yes, those are useful) was trashed in python 3, again for no seemingly good reason. % operator .format etc. WHY? I'm serious.
"You had 12 years of warning" - hey how about python follows some other languages. Make the update compelling so folks are excited to move. Think about things that will make it easier instead of just pulling all these silly stunts. You couldn't practically move initially. Something as simple as doing u'' in python 2 that would alias properly in 3 - was a no go.
This is what worries me about posts like yours, python developers have really no clue how stupid this type of backward compat breaking approach is if you have limited upside and go out of your way to make it hard to support both versions. It is critical that lesson be learned - but it seems hopeless.
Anyways, go and others I think are going to make a reasonable run at this sort of utility language space with I think some stronger efforts around backward compatability. I think the next backwards incompatible change in python (python 4?) will probably see a much bigger migration off python.
Yes, this doesn't change my argument. You had 12 years of warning. You couldn't migrate immediately, but you had 12 years of warning to plan your migration.
> I think the next backwards incompatible change in python (python 4?) will probably see a much bigger migration off python.
You aren't paying much attention to the python ecosystem then. I wouldn't expect a python3-like breaking change for another 10 years, if ever.
> Mercurial, built on python, took 10 years of work to get to python 3 - for basically no reward (they just need bytes and unicode is not super relevant to them) as just one example of MANY.
Mercurial took ~4.5 years to migrate, from April 2015 to November 2019. Mercurial also took a roundabout way of doing the migration that likely made it take longer (https://gregoryszorc.com/blog/2020/01/13/mercurial%27s-journ...)
> The python developers literally TOOK THIS AWAY in python 3.
And then readded in python 3.3, around 7 years ago.
> . You couldn't practically move initially. Something as simple as doing u'' in python 2 that would alias properly in 3 - was a no go.
I agree you couldn't migrate immediately! But you still had 12+ years of warning! I'll repeat myself: I expect most application code was migratedn in the last 2 years. But the maintainers of those applications had 10+ years to prepare for the migration of their application. This prep took many forms. For some it was nothing, for some it was helping dependencies migrate to 2/3 compat. For some it was planning the eventual migration. But every single one had more than a decade of warning.
It does seem like if you put in some effort and do some decidedly unaesthetic things, you can bolt on and backport a lot. No developer dreams of maintaining dirty hacks at scale, though.
Indeed. I'm in the process of writing some simple web stuff. I considered python, but decided against it because I'm doing this for free, and don't want to be asked to come back and fix it in a decade because "Python 4.6 needed a feature, and it'll break compatibility with Python 3"
Yes, they could still just run it with Python 3, but do I really want to go through the effort of explaining how to configure that instead of just handing over a working binary?
It's not a program that will be running on a server, so I'm not really concerned about security fixes/updates to libraries.
While I could bundle python with the program, that's getting to be more hassle then it really seems like it's worth. In addition, I'm not particularly comfortable with configuring python, so there's a non negligible chance I'd screw that up and be in the same position.
It's not that it's impossible for Python to work in this situation, it's that me doing it is a bad idea. I'm sorry if my original comment implied otherwise.
Things are getting broken all the time, the major mistake IMO what PSF did is giving 10 years to migrate code. Everyone waited to middle of 2019 when they actually started to migrate and the 10 years only allowed to created FUD.
So I went with C# instead. I’d choose it again over Python.
Writing some code and not have it executable a dozen years later is a terrible terrible situation. And it wouldn't be this way if Python 2 wasn't EOL.
I'm going to always need to keep a copy of Python 2 around. Always. And if I'm going to change old, working code, I'm probably going to rewrite it in a new language.
But yet I get why things are the way they are. Heck, it was only last year that I switched to using Python 3. I use it so rarely that it was just easier to stick with the Python 2 which was already installed on my system.
They were spending the time doing other important things. They were making the rational decision to work on something that was important and WASN'T already working. The python 2 code was working fine, it didn't have bugs and it didn't need any new features. Other code either didn't exist and needed to be written or had bugs that needed to be fixed.
It makes sense that changing working code never made it high enough on the priority list to get worked on.
And this old code is also written in a style (usually global variables and mixed concerns and years of sub-ideal bug fixes and patches) which makes it very hard to change or modernize (forget adding threads with all of this global mutable state).
The only point being that old code may superficially work, but in many cases (in my experience) and regardless of language, it all has technical debt and needs work to keep it relevant.
If the situation changes and the original code suddenly becomes a basis for further development then that's a different thing. But cleaning up the code without clear benefit is a waste of resources.
Hats off to original developers. Making code that serves its purpose for 25 years is quite an achievement.
All code rots. If you are lucky to never need to fix it or change it, that's great, but that is (in my experience) extremely rare over time spans measured in decades.
Sometimes a code base runs for decades not because it was particularly robust or well-written, but because it is too painful to replace it due to its complexity or the long ago loss of the knowledge needed to work on it.
Sure, there is some amount of C code from 25 years ago that works unchanged, but there's also a good bit of Python code from 25 years ago that works unchanged. (I just tried a few of the examples in the Python 1.3 tarball, released in 1995, and it seems like most of them will work with very minor changes.)
- Warnings recommending the use of parens in code like `if (a || b && c)`. The original code is (probably) not broken, it's just that GCC thinks that it's error-prone to write conditionals that way (I agree). There's also a similar warning for code like `while (a = foo())` where it asks you to add parens to make sure that you didn't use = instead of == by mistake.
- Warnings for unmarked fall-throughs in `switch`. Again, not broken if the fall-through was intentional. That's a fairly new warning, so a lot of all code will trigger it.
- Warnings for some const to non-const pointer conversions (in particular for literal strings in C++). There you could argue that the original code is bogus but as long as the pointer is not written to it's fine. Const-correctness wasn't really a thing in C and C++ for a long time, a lot of legacy code plays fast and loose with const qualifiers, but that doesn't necessarily mean that the code is broken.
- Warnings about unused variables, in particular variables being written to but not read. As compiler get smarter they tend to catch more of those.
“The Fortran standards committee generally refuses to break backward compatibility when Fortran is updated. This is a good thing (take that, Python), and code written decades ago can still be compiled fine today.” -- http://degenerateconic.com/backward-compatibility/
Glibc and GCC introduce incompatible changes all the time.
Some Python users understand compiler flags, but there's also a big community of scientific developers who just want a working language.
Python 2 -> 3 created a lot of problems for them without making their lives easier or more productive.
Perhaps Python 3 should have been called something entirely different, and P2 could have been handed over to a new set of maintainers who weren't going to EOL it.
But seriously:
- C never did such change, it doesn't make as much sense for it to do it since it is low level - C has a type static typing, so if they did, they could make compiler refuse to compile until types are right. Python has dynamic typing so these kinds of things show up when running, you can use something like mypy, but unlike C it is optional.
BTW: I saw people using Go as an example that there was no need to add unicode support like in python, and simply store everything as UTF-8. Those people are forgetting that Go already has clear distinction between text and bytes (via string, bytes, runes types), how the language stores text is just an implementation detail, since normally there shouldn't there be access to the internal representation of the text anyway.
The thing I think they did better though, is they didn't give people 10 years to fix their sh*t. You either moved on or you stayed with an unsupported version, and people did move on.
For Python 3 the major shifts were in 2015 and 2020. The first year was when PSF announced they won't add any new features to 2.7, and that's when libraries started adding Python 3 support. And 2020 was EOL for Python 2.7.
It doesn't matter how much time you give people to migrate, they will always do it at the very last minute.
Also, 1.8 marked the creation of complete compliance test suite (as Ruby 1.8 became an ISO Standard), and that test suite has been expanded since then allowing for alternative implementations.
Because the test suite is your target, not a hard to interpret 2kLOC spaghetti code C function.
Ruby was then using a version numbering scheme similar to the old Linux version numbering scheme, where odd minor numbers were development releases and even minor numbers were stable releases. In that scheme a odd minor version is equivalent to a SemVer major release. (Ruby switched to SemVer-ish versions with 2.x)
It would be nice to have an alternative to Python that took backwards compatibility seriously. Might be easier to start from Python2 than to go from scratch. You might be able to add a REPL to perl5, but its syntax isn't the most beginner friendly I've ever seen.
$ perl -d
^d
DB<1> print 5*5
25
DB<2> q
$
Also, perl5 has excellent backward compatibility. I'm still running the same scripts since the late 90s in production.
Perl is superior to python in having forward references, compile and runtime type checking, and no significant whitespaces.
Whenever I have to use python, my first impression is always, "Gee, what a giant step backwards compared to Perl."
Things these languages do have in common with Python3 are:
- support for keyword arguments and optional arguments
- support for objects and interfaces (but not necessarily for inheritance)
- handling of names in scopes and name spaces
- closures and lambdas
- list comprehensions
- arbitrarily long integers, complex numbers, a numeric tower
- an empty sequence or empty string is logically false (with a few nuances)
- low-level bit operations (often down to a very low level such as popcount in C)
- easy to call into C code (or Java code if it runs on the JVM)
- full unicode support
- conditional expressions like (a if boolval else b)
- type annotations
- string formatting forms a mini-language
- print is a function
- hexadecimal, octal, and binary literals
- tools for iteration and first-class functions, like partial application
- support for OOP
In addition to this, the members of the Lisp family usually have as well:
- strong support for functional programming
- full and efficient garbage collection
- persistent data structures
- compile to machine code (native code generation or JIT compilation)
- a powerful macro system
- capability to define custom control constructs
- support for pattern matching to members of compound objects, similar to
- often very good support for parallelismWhat exactly would be suited best depends IMO on the use case.
I think Racket is ideal for easy learning, a platform-independent GUI, and comprehensive documentation, and is also useful fir science applications. It can also interact well with C libraries (or, more generally native libraries with C interface), and it is good for scripting.
Clojure is ideal for server applications with a lot of concurrency and a high performance. Its concurrency support is brilliant and best-in-class, much better than Python.
Common Lisp / SBCL is somewhat under-hyped in comparison to Clojure, yet it has an excellent performance, integrates well with C libraries, has a highly interactive repl, a comprehensive library system (quicklisp), a standard build system (asdf), comprehensive documentation for beginners (common Lisp cookbook), it is very usable for scripting, has standard pthreads-like concurrency primitives, a powerful object system, and then some more.
There's no optimal or blameless solution here but I think in particular when it comes to the scientist use-case mentioned above, an entire decade is enough to address what is likely a relatively small codebase, it's not just a fault of the language, but also neglect or lack of resources in fields where programming is secondary in nature. (also btw detrimental to the quality of science, as recent cases have shown where programming errors basically invalidated entire studies)
Not really. Just introduce an optional version declaration at the top of the file, which guarantees the old behavior. Then feel free to tinker with the design as much as you want. It's a solved problem and it works well. Sure, you still break people's code if they don't put a version declaration in - but now the fix is a single line.
The fatal flaw in Python's approach was cleaving apart the Python2 and Python3 worlds, making it impossible to mutually import modules. Python2 and Python3 are practically speaking different languages - different syntax, different semantics, different libraries, and a different userbase. As far as the computer is concerned, they are separate entities, with the feeble "2to3" program being the only thing linking them.
You can use Cython to compile your python 2 code into .so objects and then include them in Python 3. Then you can progressively migrate code file by file to Python 3. I didn't do it on a larger project, so maybe there are some gotchas, but it seems to work as far as I could test it.
Regarding the differences in languages Python 2.7 was actually a bridge to migrate to Python 3. That's why all features of 2.7 were backported from 3 (this was also the reason people saying: "why should I migrate to 3, it doesn't offere anything new". It didn't because everything new was backported.
Anyway thanks to this it was actually possible to write code that worked on both Python version. I did it several times. It was actually the easiest when you wrote the code for Python 3 and only used features that 2 had. Add all the `__future__` imports and remaining parts filled with the `six` module.
Yes, and tools and infractstructure should account for that: Python 2 and Python 3 should use different file extensions, and libraries written for python 2 should not just adapt to Python 3 and break backward compatibility, but come _at least_ as a new major version - better with a different library name, as Go lang requires.
And still it would have been possible to support Unicode in python 3 without breaking backwards compatibility. It was possible to support in Common Lisp which was defined as a ANSI standard in 1994, and Common Lisp implementations like SBCL or Schemes compile to far faster code than C Python (while providing many of its flow control affordances, like list comprehensions, or a numeric tower). Why should it not be possible for Python? Apart from the developers not wanting to do it?
A lack of resources for porting and maintaining code to new language versions is totally typical for most of science. There are simply no jobs or incentives to do that, especially if you want to have any kind of career in science.
Further, python developers seemed to think that the bulk of code and maintenance work is in the language implementation and that the many small libraries and scripts are not important. But this is not the case, similar to that in trees, a rather large part of the biomass is not in the trunk, but in the roots and leaves. And the small application scripts and libraries used in science change much more slowly than the main language revisions.
You could argue that this does not matter because most of python's applications are not in science, at least not if you look at the respective amount of money moved. But this fails to take into account that a lot of python's success was so far based on its use in science, especially numerical python and data analysis.
It's also the reason I never started in the first place. It wasn't in any way clear which the right option was during the period of schism, and it went on for an awfully long time.
With that being said I'm starting to tinker with Python now simply because it has a great ecosystem for NLP and ML. I certainly need the former, although how much of the latter I'll need (possibly none) is unclear.
Regardless, it seems to be the de facto standard language for anything that falls under the broad category of data science to the point where for me to pick up a different language would feel very much like swimming upstream.
(Related, but separate:) Would you have preferred if Python 3 was released under a different name that didn't include "python"?
In python 2 you could specify a unicode literal with u''. So a natural evolutionary option if you had a unicode addiction and wanted python 3 to be unicode native, would be to treat u'' as unicode literals in python 3 as well.
This let's the code run under python 2 still, and then you get unicode in 3. And if you are 3 only, just "blah blah" works as well - nice.
But they didn't do that. They purposely took u'' out to make it hard to support 2 and 3. That is absolute stupidity. In other words, they BROKE compatibility for no benefit - but just to break it.
They did the same thing with byte handling, jumped the io abstraction in - these things all destroyed performance and use cases.
[1] https://access.redhat.com/solutions/4455511
Of course python could have dealt better with backwards compatibility, but the python code I wrote was transfered to version 3 pretty quickly.
Perhaps some chose instead move to other languages.
https://github.com/naftaliharris/tauthon/issues/47
Ultimately, I stopped writing Python: these days I write Common Lisp for personal projects and Clojure/JavaScript for everything else.
It always left a bitter taste in my mouth and I always feel like I’m fighting with Python every time a project comes up where I have to use it. There are so many other languages that are a pleasure to use and I’ve always wanted to like Python, I just feel like I’m cursed to always find Python needlessly annoying.
There was an article the other day about problems mixing different python versions on Mac. I admittedly didn't read much of it—but I was somewhat baffled by the problem. I have Python 3 installed on my mac as "python3", and Python 2 (which comes with macOS) as just "python". I think that's the right way to treat them—as different, independent interpreters which can co-exist, as do, say, Python and Ruby.
Will there be Python 3.8 interpreters in 2100? F77 will still compile and run better than it did in the 1980s.
Being able to figure out that you're using a different major version of Python is definetly expected of today's biologist. This is part of the profession now.
Python 2/3 incompatibility is always magnified for some reason, but it's nothing out of the ordinary in my experience.
Didn't we just recently have a highly upvoted/heavily debated HN submission on semver? Enough said...
This was absolutely essential. Python 2 tried to fix by introducing unicode type. It was hacky, and because they looked for a shortcut, they did a very bad thing by introducing implicit casting (so python 2 automatically converted between str and unicode). They did implicit casting because they didn't made the effort of fixing the stdlib to properly handle unicode.
At that point not only there was a problem that str represent two different things (bytes and text), but also authors that were using unicode (to make their code futureproof) make their life actually harder (in python2 even when you know how unicode works, implementing it correctly is a minefield, it is so easy to do it incorrectly and end up with code that works, but blow in special circumstances).
Python 3 basically decided to rip the bandaid off and do it correctly. And that's what happened. When you program in python 3 you no longer have to think much about unicode. To convert between byes you need to explicitly convert from text to bytes (.encode()) and bytes to text (.decode()) but this is not a concern unless you are doing some kind of serialization.
While the breaking unfortunately was necessary (a lot of python 2 code, while it runs it is broken), I'm wondering if it was possible to do it a bit more developer friendly.
I think things would be much easier if instead of changing the behavior of `str` type, Guido would introduce a `text` type (along with `bytes` type), then later deprecate it. I think that would help a lot. Although in that case modifying stdlib would be unavoidable, so it would be a lot of work for them.
BTW: MyPy was also a tool that could help with the migration, but once again they IMO dropped a ball. Basically when MyPy is used in Python 2 mode, it follows Python 2 typing system, which means it still allows for implicit casting. I think what was really needed is to highlight these issues to allow for fixing python 2 code before doing the migration. The way mypy worked required you to first migrate your code to Python 3 and only after that you could start fixing the text/bytes issues.
Yet, I have not seen a rush to "fix" any of them the Python way.
The python incompatibility wasn't really the Unicode it was that older versions of python mixed text with bytes, and because of that a lot of python 2 code is actually broken. It might run fine, but as soon as input are Unicode characters then many python 2 applications crash.
Python 2 tried to implement Unicode and implemented an `unicode` type. But it actually made things worse due to them adding implicit casting.
The proper way to fix the issue was actually clearly making the distinction what is text and what are bytes, something that was in both Java, C# (both of them are younger than Python BTW, the Unicode was in works when Python was released).
he UTF-16 used by Java and C# as I said is just an internal representation of the text, and that's actually trivially changed since normally those things are not accessible for developers. The only way where it could somewhat leak is default encoding, and maybe names of the methods from stdlib. But those things are really easy to change without much breakage.
Side note, Python internally stores text in many different ways if the text is only out bytes it uses traditional C-string (for performance). Otherwise it also stores as UTF-8, UTF-16, UTF-32 based on characters and requested conversion. So those conversion you said Java or C# did Python 3 does all the time (based on need) without breaking running code.
[1] although from quick googling it looks like C# handles things the same way as Java.
And quite frankly most python 27 code I wrote worked after adding brackets to the print statements.
People forget that part of the reason why the Python maintainers were so snotty about Python 2 to Python 3 was because they went through this before.
Anyone remember the gnashing of teeth that occurred when trying to drag people off of Python 1.5.2? I sure do.
I want to throw anybody complaining about backward compatibility in 2020 into Python 1.5.2 because THAT'S what you're advocating for.
https://asdf-vm.com/
The Python plugin is at
https://github.com/danhper/asdf-python
You can use tools like six to port python 2 to python 3 which should cover 90% of the code in small scripts
IMO you should consider simultaneously learning mypy and Python type annotations. Python with some typing is a vast improvement.
If you're interested, this document[1] is a good resource should you want to port from 2.7 to 3.
[1] https://docs.python.org/3/howto/pyporting.html
But yes you can't expect new code to continue to support your fixed choices forever. Then again that's true of essentially every language out there so I don't really see what's special with python.
Instead, I expect the language to continue along the path of FOMO-driven accumulation of features (and complexity), while still not addressing its major issues (e.g. performance and parallelism).
Or from wikipedia:
Python 2 doesn’t sound like an ancient, totally-different-language so people just adopt it because it was the version on their laptop or something and because it “works”. In reality, it’s inevitable that some major IT upgrade will finally throw out that interpreter (or, the existing one will fail to work for some obscure reason and no one will fix it because it’s “unsupported”). And at that time it will be non-trivial (to say the least) to fix every affected script and the entire downstream dependency chain because — as so many on HN know — Python 3 is a different language.
Any ideas on how to present this to management as the crisis that it should be? It’s really quite a risk but it is hard to demonstrate the risk when everything “works” so “well” right now.
You are agreeing with your parent:
> And at that time it will be non-trivial (to say the least) to fix every affected script and the entire downstream dependency chain because — as so many on HN know — Python 3 is a different language.
Why are people choosing 2.7 for new projects when there is a time bomb coming up? Several years ago, you could blame the Python folks for this break, but once 3.3 (or perhaps 3.5) was out, there was no good reason to start new projects in 2.7, especially when it was publicized that the language will be EOL.
str vs bytes isn't as bad as people make it out to be. You'd be used to it within a week. The other stuff is mostly just semantic changes like range vs xrange or dict.items() returning an iterator now vs a list.
At this point, the bigger issue might be learning all of the new features people were missing out on by not moving earlier, like asyncio.
If you have a large python2 project with 100 dependencies, will all of those have python3 versions? And will they be compatible with each other in the ways your current dependencies are?
It's not easy to determine which versions of the library to upgrade to however. What's the latest version compatible with both python 2.7 and 3.7. Would be nice if pip had a setting to get that.
[1] A one that I often seen was MySQL-python. That package hasn't been touched since 2014. Another one I've seen was PIL, that one was last released in 2009, and isn't even on PyPI, because it is that old, fortunately maybe because it's not on PyPI, people are more aware of Pillow which is a drop in replacement
- You won't be able to use newer versions of open-source libraries or newly-developed libraries (including both new features and security updates - note also that a project's developers won't be looking for security bugs in old versions of libraries and likely aren't supporting old versions, but attackers probably are looking for them!)
- Your company ends up with two sets of code in effectively two different languages, and you can't extract common libraries, reuse tested business logic, etc.
Those are things the business is likely to care about. It would be particularly helpful if you can show that you were unable to use the new thing because you had existing code in Python 3. or the new thing was unable to use new features of the libraries they depend on.
Do you have a security team who cares about running supported versions of software or vulnerabilities in deployed software? Can you find libraries in their `pip freeze` that have had security updates that weren't backported to the Python 2 version?
Have you had problems when a team using one Python interpreter needs to call code written by a team using another interpreter?
Writing new projects in Python 2 is rather like writing new projects in Visual Basic 6, in that it will certainly still work in 2020 and technically it's easy to get started - except with the advantage that 2 and 3 are actually quite similar languages. If your company would try to pressure folks writing in VB 6 to use something else (VB.NET? Qt? Electron?), imagine you're in that situation, except it's magically much much easier to do the port.
From there, you use the built-in documentation, learn your way around the language, and end up with Python 2.7 as your daily driver.
You have to know to write "python3" to get Python 3. In effect, it's not the default, it's the special alternate version.
Show them the EOL document, and ask them if they can afford to have their systems owned when an exploit is released for Python 2.7 that goes unpatched because the project is unmaintained.
It's easier to start new projects in a supported version, than it is being forced to port your applications at the 11th hour after exploits in unmaintained Python versions are found.
From a developer's perspective, you're coding in a time machine with Python 2.7. Updates to community packages from the last decade haven't been backported to their Python 2.7 versions, so you'll run into an accumulation of bugs at every turn. If you're interfacing with an API, you probably won't find an off-the-shelf solution for Python 2.7 like you would Python 3. You are literally creating more technical debt with every line of Python 2 that you write.
Sure, the code now runs in the py3 interpreter, but guess what!? That was the easy part! Now go hunting down Byzantine build scripts and environments to find out how it was deployed and tested. Tested lol? Ugh and guess what? The code base has commits that we’re never deployed to prod. What do you do now? You basically own a service that hasn’t been deployed in 4 years and a branch that was never deployed ever.
The old versions exist. They are not supported. You can choose to use them if you wish or use the latest. I don't see many complaints about the fact that Java SE 6 (or whatever) isn't getting updates anymore.
I think the Python team fucked up the 2->3 transition, but that was 12 years ago! Do other people have explanations as to why old versions of the Python interpreter are associated with so much emotional investment?
major.minor.revision: https://semver.org/
They ticked the version at the right spot, which means it's (edit: not) backwards compatible.
The same is true for Erlang/BEAM/OTP. You can even dist connect old and new VMs as long as you're careful not to send things the old VM won't understand (like Maps to a BEAM from before Maps). Erlang does tend to actually remove deprecated things after a few releases, unlike Java though, so it can be difficult to support more than two or three releases.
In both Java and Erlang, you can upgrade before your dependencies, at least most of the time, but that doesn't seem to be the case for python 2 -> 3, which means people are stuck on 2 when they want to use 3, until their dependencies update or they take over or abandon them. That's not a quick process which is why people are still upset.
https://mobile.twitter.com/gvanrossum/status/128544327896012...
But probably not in terms of "Don't use it in production anymore". Because in many places, what is in the current stable Debian repos is what is used.
The most notable exception is probably Arch Linux, which has Python 3 available in the `python` package.
https://packages.debian.org/buster/python3
Linux distributions that appeal to those who like to be on the cutting edge have pointed the python command to python3 already. Debian stable will be there one day, too.
Most important third-party libraries were only ported Python 3.5 onward. So the real Python 3 start date should be 2015, not 2008.
I very much wanted to move to Python 3, but critical libraries I was using were not there. And running both 2 and 3 at the same time was very unergonomic until python-future [1] became stable and widely-used.
[1] http://python-future.org/
About five years ago I used six and it worked well enough. Now both obsolete.
On the other hand, Python gives a growing and continuously maintained standard library and none of the headaches that, say the JS ecosystem gives.
That said, not all of the features are great all the time. Async/await comes to mind, which adds a lot of complexity without the performance gains even senior programmers might expect. But overall, we're already using features in Python3.8 for projects, and have upgraded our instances to Python3.8 without issue. I guess it's possible we leave some pip dependencies in the dust because they may have some incompatible code with a future 3.x release, but we haven't had a problem before. Tests permitting, we'll do the same for 3.9 if we find something we like or might wait a few years.
The only exception to this that I can think of is the changes to generators introduced in PEP479.
https://www.python.org/dev/peps/pep-0479/
https://www.juandebravo.com/2018/11/09/python-37-stop-iterat...
ECMAScript is the language, and it is amazingly backwards compatible back 20 years to about the start of the century without any trouble, and plenty of tooling like TypeScript can target ECMAScript 3 released a score of years ago.
Python could have added a ‘use strict’; in 3.0 to avoid a breaking change!?
Sure, since then browser features have radically changed and frameworks have changed, but the language itself has been extremely solid. Conflating JS and the ecosystem is not a fair comparison.
Also there have been many competing different implementations of JS engines, with amazing levels of compatibility between them (even Internet Explorer 5.5 released 2000 only had a few very minor known quibbles in the ECMAScript implementation).
Edit: Of course Unicode is actually hard to deal with because ECMAScript uses UTF16 for strings and symbols, even though the source code is usually written in UTF8. And note that the u flag was added to RegExp expressions to deal with Unicode better in ES2015 (especially U+10000 and greater, which are generally difficult to deal with in ECMAScript code).
But, the ecosystem is really something different entirely, and this isn't meant to be a slight to JS at all. It's just reality when running production code in 2020. Python's standard library is comprehensive, well-maintained and well-documented throughout. JS, by comparison, relies on packages from npm for the same features. Which means the issue my org has with the JS ecosystem is the stuff that's not built into the language spec or otherwise maintained by a central body, not that the things built into the language isn't standing the test of time or something like that.
I hope that explains things.
Most of the ecosystem changes are driven by externalities - web-servers, browsers, developers. You can still choose to use an old web framework and it will work on modern browsers.
That is entirely different from the Python 2.x -> 3.x change.
The days of a simple CRUD web page delivered by a server are long gone. jQuery is still brilliant at adding adornment to HTML produced on a server. Putting your view and controller into JavaScript just makes sense.
SPAs need completely different frameworks to support the highly complex needs of client-side rendering and interaction. Ten years ago component based renderers were not common and developer-friendly syntax hardly existed. Frameworks are being pushed by the needs of huge systems (Facebook), server side pre-rendering, and partial updates (correct term? to improve first load and responsiveness).
Also the features available in a modern browser are radically different. Remember building up a string of HTML in JS for performance reasons? Avoiding the DOM because it was slow? So many features only started becoming reliable or available around Internet Explorer 11, yet now IE11 is hardly ever supported because it is missing critical features. An example is that in older Internet Explorer and Safari, you could theoretically update CSS rulesets from the DOM, but in practice doing anything dynamic risked crashes and memory leaks (even loading new CSS pages was difficult and error prone and browser specific e.g. detecting CSS load failures).
Note: I was supporting the same JavaScript codebase for over a decade (initially IE5.5). There were no breaking changes in the ECMAScript, and very very few in the DOM (mostly driven by dropping IE specific DOM). The changes in the codebase were mostly driven by wanting to support new features in browsers.
Ordered more efficient dicts was another win.
I also needed to change a lot of things porting the same program a few years later to an IRIX based machine whose name I don't remember right now (Octane, perhaps?). And that was a well behaved program that did not try to change the value of 4.
Python 0.9.1 programs (Earliest I used) required much fewer changes to run on 2.7 -- partly because I never used stuff like field access control (which was luckily dropped from the language). Python 3 requires a few more changes, but I suspect I can still compile Python 0.9.1 on modern hardware and run the original code.
This inspired me to download Python 0.9.1. to find out what field access control was.
I did find it. https://www.python.org/download/releases/early/
And amazingly it compiles, and does so in just 1.5 seconds on an iMac (unlike my Typescript project which is compiling at 90 seconds and counting.... hmmm).
Unfortunately, even now with access to the source code I can't find anything about field access control.
What was it? How did you use it?
I'm not sure if they were ever properly implemented, but there was a point in time in which the grammar had some provision for declaring methods private and/or fields read-only or something like that. I do remember I disliked the idea and never used it, and likely it wasn't popular because it's gone.
I can't find any version other than 0.9.1 to download, and trying to do newgroup archeology through Google Groups is incredibly frustrating. I've given up after about an hour of trying to find old mentions.
I haven't used Fortran in over 40 years. But, at the time, the following features were in very common use:
And probably a bunch more from the list: https://en.wikipedia.org/wiki/Fortran#Obsolescence_and_delet...These features are easily changed in the source code. So does the compiler still support them? Or is there the equivalent of the python 2to3 program?
Once we'd finished going over some example programs, I asked the kids to modify a quiz program they'd written previously with a different teacher. That program used copy-pasted lines of code in order to ask multiple questions, and I wanted the students to re-implement it with a single function instead. This was a pretty challenging assignment—the kids had never encountered functions before today, and I was purposefully not quite telling them what to do, only giving examples of functions used in other contexts. But one of the girls managed to get it, and I was really proud of her.
Except her code wouldn't run because—I realized with horror—the original program she'd done with the other teacher was actually python 2 code.
I am so damn frustrated that this happened! I was trying so hard to make these kids understand the concept of functions; I did not want to explain the differences between Python 2 and 3. It is, of course, primarily my fault—I should have coordinated better with the other teacher—although in my defense, we were both volunteers, and as it is I spent most of my Friday and Saturday planning this lesson.
It also should just never have happened! I'm not aware of any other language where this type of confusion is so common, and it really makes me wonder if we'd be better off teaching something else. I'm not sure who is to blame for the situation, but it really sucks, and I don't think it's going to go away just because Python 2 is now technically EOL.
Because it’s not possible to run Python 2 and Python 3 code together, none of those libraries could upgrade until its dependencies had done so. This just led to everyone waiting for everyone else.
I agree somewhat with Point 1, but the Python ecosystem is larger than Ruby's. Each domain could have moved separately, and did, to some extent.
While the Python core team made mistakes in community management, I wouldn't blame them in this case.
Back in high school, I volunteered to tutor math at a free summer camp for poor children. A professional high school teacher was also volunteering. I wound up tutoring her in math, so that she could teach the kids. Sometimes teachers need a little update.
The paid version of Trinket let's you use Python 3, but the free version forces you into this hybrid interpreter which attempts to run both kinds of code, and it does a bad job.
This is like complaining that calculus is too fractured because someone taught your students to use Newton's notation instead of Leibniz's.
But they went with the logic of “if we’re gonna break, why not break a bunch of other things and take this opportunity for a different python”
Needless to say, the folks in charge of python didn’t think things though. It’s a big fat breaking change with no real automated way to upgrade. A whole bunch of apis are now different. At my company even after a year it’s still a pain having some scripts only work in py2, while others in py3.
Python3 pulled a google.