That's modern python. There's really no single stable python to target anymore. Only pythons. Every project requires you to set up a container in which you just create a custom python specifically for that project.
Breaking feature removal is more of the same spirit. These are the consequences of popularity and rapid progress. Perl would never be forced into a corner like this and chose to break things.
Sure it does. I'm still dealing with the fallout of old projects built with old webpack which uses md4 (as a file name cache buster only!) which is gone in recent nodejs+openssl binaries.
You’re wildly overestimating the impact of this, but if you actually use Node you’ll learn two things: deprecations do happen (just look at the release notes!) but to the extent that you could say it doesn’t that would only be because Node doesn’t have a large stdlib and so the code you need to replace wasn’t deprecated by the core developers but rather the thousands of third-party modules’ maintainers. Either way you can’t ignore maintenance needs but I’ve found it to require more work in the latter model since the schedules aren’t synchronized.
Unironically yes - with a good shellcheck setup and a good local wiki of documented utility functions I’ve found bash to be an absolute powerhouse for writing portable code.
I’ve not used Perl as much in the last decade as I used to, but I’m always pleasantly surprised when I dig up some older stuff and it just works.
Ruby has managed to be incredibly self-compatible while adding new features for many many years now. Its main weakness compared to Python is its performance but that’s been steadily improving every year with each new (backwards compatible) Ruby release. And for 99% of apps it’s fast enough.
2.x era python also had rolling deprecations and removals, so I thought it was a bit similar. For example in 2.5:
> The old regex and regsub modules, which have been deprecated ever since Python 2.0, have finally been deleted. Other deleted modules: statcache, tzparse, whrandom.
After over a decade of Python as my language of choice, I have stopped developing new projects with it, directly as a result of its instability through releases. Bit rot is a thing, and Python routinely causes more than seems reasonable.
I begrudgingly have moved to Go. Mostly because my current day job uses it a lot, so I have been paid to pick up more knowledge about its ecosystem.
Don’t get me wrong: there are plenty of things to dislike about it (or any language, once you start pulling that thread). However, they have made a much stronger commitment to avoid breaking backward compatibility.
Really? I even ported 2.7 code back when 3.6 got good, and it was all far simpler than I had dreaded.
I might be not as eager as some to learn new features though, but I am also not so conservative that I won't do things some new way, especially if the old way is being deprecated (but so far warnings have always given me plenty of heads up).
Quite the opposite, I like Python as much as I did when we first met. Nothing is without warts, but they are rarely really problematic for me in Python.
If your energy levels drop, or you lose interest, ports will become harder.
When you write code that has a short life span none of this matters. But when your software has a several year or decade life expectancy it will matter.
Also economics plays a role. If we have a real economic downturn in IT bit rot will become a huge problem for many small businesses.
If any effort is too much, you're basically stuck on a very select set of dead-ish languages or dialects thereof. But I'm not sure that's a more viable approach then using a language that is still pretty great about backwards compatibility.
Out of curiosity, why are people storing SHA512 hashes in that format? Is there a protocol or tool that requires it? I'm going out on a limb and assuming they aren't using it to check passwords.
In my case (old style DES hashes) it’s to interact with some old crusty garbage.
Unfortunately a lot of that old crusty shit is still in production in many places, despite being EoL for years, with no plans to replace it this side of 2038.
* Puke warnings on the screen when using insecure algorithms
* Guarantee secure algorithms everywhere
These algorithms aren't hard to implement, and anything relying on DES will be faster in interpreted Python than in optimized code on the 80486 it was intended to run on.
I'd rather have this removed than broken, but it makes sense to just fix it.
As a footnote, a lot of code doesn't rely on this for anything high-stakes. If this is hashing passwords on my local machine, it's a backup to a backup for security. It should move to something modern, but it breaking in the meantime is a bigger failure than an obsolete crypto algorithm.
PyPi has robust cryptography libraries from likes of Google, others. You do not need to implement them yourself. You can just install a well-known library used by the best in the software development industry.
Cryptography should fail to operate rather than operate insecurely.
There's also a lot of precedent for removing these security footguns from the stdlib that no one actually uses. From the top of my head, the `imgop` library and the `cgi.escape` function. (My personal bugbear is `csv.Sniffer` which, as far as I can see, has no correct use case, it's just broken.)
I'm somewhat out of touch with Python, but as far as I know, it's been best practice to use `cryptography`[1] for a long, long time.
ETA: Just to note - DES is super, super broken. It shouldn't even be an option. DES was published in 1975. 3DES was published in 1981 - DES has been considered too small a keyspace since 1981. That's older than Python. Exporting DES in your crypto API is a bug. (Apparently, according to Wikipedia, NIST has deprecated all use of DES [including legacy applications] as of 2023. But they only deprecated new applications in 2017, so perhaps I overstate.)
"Guaranteed to be available" isn't the same as "available in practice". Linux's current crypt function provides yescrypt, gost-yescrypt, scrypt, bcrypt, sha512crypt, sha256crypt, sha1crypt, sunmd5, md5crypt, bsdicrypt, bigcrypt, descrypt and NT (ordered by decreasing strength).
So, if you're on an ancient unix box (or a Mac, apparently), then you're stuck with the obsolete standard library your OS vendor packaged, but that's true of other things.
Presumably, they'll remove SSL support soon too, since, when provided with a sufficiently old version of openssl support, it won't provide modern algorithms either. That means "only obsolete SSL modes are guaranteed to be available".
In this thread: People don't read the PEP and make knee-jerk assumptions.
In the PEP (https://peps.python.org/pep-0594/#crypt) it points out that this module didn't work on Windows at all and didn't provide any useful real-world functionality on Linux, BSD or macOS.
The use case mentioned in this blog (SHA512 password hashing) is considered a bad practice because unlike bcrypt, SHA512 is fast for an attacker to execute to check if a credential guess matches a hash.
> People don't read the PEP and make knee-jerk assumptions.
They just came over from the Java string interpolation thread, because after not reading the JEP they weren‘t sure how to spend the rest of the evening and thought „I know! Now I will not read the PEP.“ /s
Linux distros are converging towards using libxcrypt with yescrypt, which is a modern password hashing function. If you have a crypt function that wires back to libxcrypt's crypt, you can use that.
It's a bit unfortunate that people seem to be unable to agree on a modern password hash (other places tend to use scrypt or argon2), but well, I think yescrypt is fine, and having a way to generate and verify crypt-compatible hashes is certainly useful functionality.
I think this is a good decision -- crypto is not a standard library thing and including it in the first place was a mistake. I assume someone will take the liberty of mirroring the crypt library onto pypi and everyone can just install that if they're worried about their code breaking. In my experience upgrading python versions always has some friction so I don't think that's a big ask.
Technically yes, in the sense that libc also includes a libcrypt.so, but the one in glibc is hot garbage, so most distros disable the libcrypt part of glibc these days and ship libxcrypt instead.
Though in fairness it had been deprecated for 12 years at that point.
> We did not want to hurt the people using Python 2. So, in 2008, we announced that we would sunset Python 2 in 2015, and asked people to upgrade before then. Some did, but many did not. So, in 2014, we extended that sunset till 2020.
It blows my mind some people are still on Python 2, but I worked at a place that shifted an app to Python 3 in 2021, so I guess I'm one to talk. (Incidentally, I have written much more Python 3 than I ever wrote Python 2, but I still write print without parenthesis the first time, every time.)
A couple does mean 2, not "a few". It's just that people are often very inexact about times and numbers. So yeah it's silly to argue that "a couple of years ago" is wrong because it was actually 4 because the context clearly implied a degree of approximateness.
Inflammable means flammable. Couple means few. You can disagree with descriptivism, but you've lost the moment the issue was noticed if enough people keep doing it.
Yes, but the big thing which will make it truly EOL is when the long-term support Linux distributions drop it. Right now you can have a fully supported system from Red Hat, Amazon, Ubuntu, etc. with Python 2 on it. All of the slackers have been delaying upgrades because that extended support means they don’t immediately fail their compliance audits.
I'm not sure about the others, but Python 2 is not officially supported on Ubuntu. It was moved into Universe and is considered a community-supported package rather than supported by Canonical.
Yeah, but it's incredibly expensive. We got quoted a $$$$$$+ _monthly_ number several years ago, which helped us justify to the business to upgrade to Python 3.
Interesting, they must scale up with age. I had to deal with that recently and it was cheaper than trying to rush a late development project for a replacement.
Debian is quite useful here. They keep prebuilt packages in the archive, one just needs to disable PGP key expiration checking to install them. But otherwise old Debians still work.
IMHO, there should be a way to make de facto standard libraries included in python batteries. My (personal) short list:
- requests
- cryptography
- numpy
- pandas
- tornado
- plotly
at least half of the top10 https://pypistats.org/top should be candidates for being included in python
That won’t be good. Their release cycles need to be different for patches and upgrades. Being included in Python stdlib means they’ll only get updated when Python does, which won’t be good for neither maintainers nor users.
> Crypt is particularly painful, since removing it will effectively cryptoshred user password databases.
What do you mean by that? Anyone interacting with passwords on Linux, macOS, etc. has needed to use PAM anyway for decades. If you are building some kind of network app and just tossing passwords through crypt() you are going to have a much worse time on a security audit than you will installing a single python module.
The loss of telnet and crypt is going to mean I have to version pin a lot of older code.
I should look into which file formats are going away, I had some stuff that relied heavily on uuencode/decode (along with telnetlib) for bootstrapping file transfers into older systems…
That adds extra steps, I prefer (where possible) sticking with just the standard library for portability when working with other people’s horrendously broken systems that may not have outbound internet access.
Python core devs are succumbing to wikipedia editor disease it seems. Not the first bit of crypto functionality they've removed without a good alternative :(
"The original Enigma cipher was broken in 1944. The version implemented here is probably a good deal more difficult to crack (especially if you use many rotors), but it won't be impossible for a truly skillful and determined attacker to break the cipher. So if you want to keep the NSA out of your files, this rotor cipher may well be unsafe, but for discouraging casual snooping through your files, it will probably be just fine, and may be somewhat safer than using the Unix crypt command."
See https://docs.python.org/3/whatsnew/2.3.html for the deprecation notice: "The rotor module has been deprecated because the algorithm it uses for encryption is not believed to be secure. If you need encryption, use one of the several AES Python modules that are available separately."
It's a remarkably gentle deprecation schedule. PEP 594 has been worked on for four and a half years. Python versions since 3.11 (released over a year ago) printed warnings. Python 3.12, with crypt, will continue to be supported for nearly five years.
Between this and the telnetlib removal, a significant amount of code I have written is going to need to be pinned to a specific version, or vendor in those features.
A lot of that is code to interact with weird old shit, so I guess I can’t expect upstream to maintain old shit forever.
I do wish Python came with a solid cryptography library “out of the box” instead of having to use PyCrypto/Cryptodome/Cryptography/Whatever - and I also wish the way it’s ssl sockets module worked didn’t have weird unexpected things to do with how it handles file descriptors - you can’t just dup2 a ssl wrapped socket for example, unlike a normal socket
76 comments
[ 0.22 ms ] story [ 125 ms ] threadBreaking feature removal is more of the same spirit. These are the consequences of popularity and rapid progress. Perl would never be forced into a corner like this and chose to break things.
I’ve not used Perl as much in the last decade as I used to, but I’m always pleasantly surprised when I dig up some older stuff and it just works.
> The old regex and regsub modules, which have been deprecated ever since Python 2.0, have finally been deleted. Other deleted modules: statcache, tzparse, whrandom.
https://docs.python.org/3/whatsnew/2.5.html#new-improved-and...
Don’t get me wrong: there are plenty of things to dislike about it (or any language, once you start pulling that thread). However, they have made a much stronger commitment to avoid breaking backward compatibility.
I might be not as eager as some to learn new features though, but I am also not so conservative that I won't do things some new way, especially if the old way is being deprecated (but so far warnings have always given me plenty of heads up).
Quite the opposite, I like Python as much as I did when we first met. Nothing is without warts, but they are rarely really problematic for me in Python.
When you write code that has a short life span none of this matters. But when your software has a several year or decade life expectancy it will matter.
Also economics plays a role. If we have a real economic downturn in IT bit rot will become a huge problem for many small businesses.
What’s really funny is my older Python2 code doesn’t shit the bed half as much as the Python3 stuff due to it effectively being frozen in time.
Don’t do this. Instead, carefully analyse and evaluate well known and well tested packages to do crypto with.
Unfortunately a lot of that old crusty shit is still in production in many places, despite being EoL for years, with no plans to replace it this side of 2038.
Highlight:
> Only DES encryption is guaranteed to be available. DES has an extremely limited key space of 2*56.
A good solution would:
* Puke warnings on the screen when using insecure algorithms
* Guarantee secure algorithms everywhere
These algorithms aren't hard to implement, and anything relying on DES will be faster in interpreted Python than in optimized code on the 80486 it was intended to run on.
I'd rather have this removed than broken, but it makes sense to just fix it.
As a footnote, a lot of code doesn't rely on this for anything high-stakes. If this is hashing passwords on my local machine, it's a backup to a backup for security. It should move to something modern, but it breaking in the meantime is a bigger failure than an obsolete crypto algorithm.
As I understand it, they are difficult to implement securely. By that I mean avoid side channel (often timing) attacks.
There's also a lot of precedent for removing these security footguns from the stdlib that no one actually uses. From the top of my head, the `imgop` library and the `cgi.escape` function. (My personal bugbear is `csv.Sniffer` which, as far as I can see, has no correct use case, it's just broken.)
I'm somewhat out of touch with Python, but as far as I know, it's been best practice to use `cryptography`[1] for a long, long time.
ETA: Just to note - DES is super, super broken. It shouldn't even be an option. DES was published in 1975. 3DES was published in 1981 - DES has been considered too small a keyspace since 1981. That's older than Python. Exporting DES in your crypto API is a bug. (Apparently, according to Wikipedia, NIST has deprecated all use of DES [including legacy applications] as of 2023. But they only deprecated new applications in 2017, so perhaps I overstate.)
[1] https://cryptography.io/en/latest/
https://manpages.debian.org/unstable/libcrypt-dev/crypt.5.en...
So, if you're on an ancient unix box (or a Mac, apparently), then you're stuck with the obsolete standard library your OS vendor packaged, but that's true of other things.
Presumably, they'll remove SSL support soon too, since, when provided with a sufficiently old version of openssl support, it won't provide modern algorithms either. That means "only obsolete SSL modes are guaranteed to be available".
"The algorithms are mostly old, of poor quality and insecure. Users are discouraged from using them."
Why is this post being upvoted?
In the PEP (https://peps.python.org/pep-0594/#crypt) it points out that this module didn't work on Windows at all and didn't provide any useful real-world functionality on Linux, BSD or macOS.
The use case mentioned in this blog (SHA512 password hashing) is considered a bad practice because unlike bcrypt, SHA512 is fast for an attacker to execute to check if a credential guess matches a hash.
They just came over from the Java string interpolation thread, because after not reading the JEP they weren‘t sure how to spend the rest of the evening and thought „I know! Now I will not read the PEP.“ /s
Linux distros are converging towards using libxcrypt with yescrypt, which is a modern password hashing function. If you have a crypt function that wires back to libxcrypt's crypt, you can use that.
It's a bit unfortunate that people seem to be unable to agree on a modern password hash (other places tend to use scrypt or argon2), but well, I think yescrypt is fine, and having a way to generate and verify crypt-compatible hashes is certainly useful functionality.
2. No encryption in general, means also no http module, which means no REST calls, which means people would just use another language.
Technically yes, in the sense that libc also includes a libcrypt.so, but the one in glibc is hot garbage, so most distros disable the libcrypt part of glibc these days and ship libxcrypt instead.
> We did not want to hurt the people using Python 2. So, in 2008, we announced that we would sunset Python 2 in 2015, and asked people to upgrade before then. Some did, but many did not. So, in 2014, we extended that sunset till 2020.
https://www.python.org/doc/sunset-python-2/
It blows my mind some people are still on Python 2, but I worked at a place that shifted an app to Python 3 in 2021, so I guess I'm one to talk. (Incidentally, I have written much more Python 3 than I ever wrote Python 2, but I still write print without parenthesis the first time, every time.)
https://ubuntu.com/about/release-cycle
Thinking of Vernor Vinge's information archaeologists from A Fire Upon The Deep.
Crypt is particularly painful, since removing it will effectively cryptoshred user password databases.
Anyway, I'm not particularly surprised. I don't think I've ever written or encountered a python script that didn't bit-rot after six months.
What do you mean by that? Anyone interacting with passwords on Linux, macOS, etc. has needed to use PAM anyway for decades. If you are building some kind of network app and just tossing passwords through crypt() you are going to have a much worse time on a security audit than you will installing a single python module.
I should look into which file formats are going away, I had some stuff that relied heavily on uuencode/decode (along with telnetlib) for bootstrapping file transfers into older systems…
https://docs.python.org/3/library/crypt.html
"The original Enigma cipher was broken in 1944. The version implemented here is probably a good deal more difficult to crack (especially if you use many rotors), but it won't be impossible for a truly skillful and determined attacker to break the cipher. So if you want to keep the NSA out of your files, this rotor cipher may well be unsafe, but for discouraging casual snooping through your files, it will probably be just fine, and may be somewhat safer than using the Unix crypt command."
See https://docs.python.org/3/whatsnew/2.3.html for the deprecation notice: "The rotor module has been deprecated because the algorithm it uses for encryption is not believed to be secure. If you need encryption, use one of the several AES Python modules that are available separately."
This package says it's a drop-in replacement for the library. https://pypi.org/project/py-purecrypt/
A lot of that is code to interact with weird old shit, so I guess I can’t expect upstream to maintain old shit forever.
I do wish Python came with a solid cryptography library “out of the box” instead of having to use PyCrypto/Cryptodome/Cryptography/Whatever - and I also wish the way it’s ssl sockets module worked didn’t have weird unexpected things to do with how it handles file descriptors - you can’t just dup2 a ssl wrapped socket for example, unlike a normal socket