Yay! Glad to see this rather than the Red Hat approach of continuing to support dead versions (yes, I know that enterprises value stability blah blah blah)
The way I have always understood it - the Python that RedHat supplies is intended for it's scripts. If we install our own packages or remove any packages, then the risk destabilising the system. I have seen situations where people have removed python packages that resulted in yum no longer working, for e.g.
Hence, I have always (including from the RHEL 6.x days) provided my own local python version by compiling and packing as a custom rpm. I now use venv to do the same.
Actually RedHat no longer ships Python since RHEL 8, instead they have platform-python for internal use of the operating system and you need to explicitly install the Python version you want
This is a RHEL 8 only feature, RHEL 9 will have a global python3. Whether alternative version are offered as modules or as side-by-side installs like in Fedora, I do not know.
When I was at Apple, they discontinued bundling the command line version of emacs, and they sent a long internal email stating why. The stated reason was that any newer emacs would suffer from license incompatibility with macOS, and they didn't see the point of supporting a 10 year old version that the FSF doesn't even support when, if someone really wants emacs, it's just a `brew install emacs` away.
I think it was a "removing from future versions of macOS" situation, not removing from existing installs. If your mac is old enough to not have homebrew, it's probably too old to install Catalina due to the fact that they stopped supporting devices without Metal support.
Also, feel free to replace `brew install emacs` with the macports equivalent and the point still stands.
Well sure, there are alternatives. I was commenting on the casual assumption that everyone has brew. Brew has worse support for older Macs than a lot of other software.
As noted on the GNU emacs site[0], you can download prebuilt binaries here[1], if you really can’t be bothered with ‘make install’. Homebrew is hardly necessary (for most things.)
What I see when I open a terminal is this:
The default interactive shell is now zsh.
To update your account to use zsh, please run `chsh -s /bin/zsh`.
For more details, please visit https://support.apple.com/kb/HT208050.
I believe I switched from the default zsh to the bash that also came with the operating system. I don't think I installed a new bash.
I was just trying to add on to/respond to the question "Will they completely remove bash, rsync, eventually?" . My answer was too short and I didn't actually make it clear what I was saying. I think the answer to the question "Will they completely remove bash, rsync, eventually?" is yes, at least for bash, I think they will remove it. My evidence for thinking they'll remove it is if you switch from the default zsh to the bash that comes with the operating system, they'll kindly ask you to switch back to zsh every time you open the terminal. Yes, you can change the default from nagging you with BASH_SILENCE_DEPRECATION_WARNING, which is great to know.
I don't really see the problem with this though; people who are familiar with what bash and rsync are will probably understand computers well enough to install them with Homebrew or Macports or something.
> people who are familiar with what bash and rsync are will probably understand computers well enough to install them with Homebrew or Macports or something.
This is probably somewhat optimistic: there are a lot of people who are not developers who still use those tools — I've supported scientists, librarians, analysts, etc. who had a few scripts using tools like that but weren't really comfortable making changes. That doesn't mean it's _hard_ to install Homebrew but it's something which they'll need a little pointer and possibly institutional approval to do.
That's fair, I guess it has a potential to affect non-geek users at some level if they're using geek-supplied scripts.
I guess I just feel like the net good would be to de-bundle these non-compliant tools and get them to install upstream versions from Homebrew or MacPorts or something.
It's annoying, and I'm not one of those people that tends to be the "OMG JUST RTFM IT'S NOT THAT HARD!!", since if people are struggling with it then it is "that hard", but I feel like there isn't a solution that can satisfy all parties with this.
It seems that we either break the workflow, which makes a few of the aforementioned parties for a bit, keep bundling the old versions of these tools and leave potential security problems in there, or somehow forcing Apple to change their OS to be GPL3 compliant. For better or worse (probably worse), that last one isn't going to happen, so it feels like the lesser of the two evils is to stop bundling software that won't be getting future support.
I agree that plenty of people who are not developers use those tools but I also don’t think the lift of installing homebrew or macports is going to be too much for people who are already smart enough to run those scripts in the first place. Like you said, it just requires a little pointer, which hopefully people will be happy to provide.
The approval thing is valid, but I feel like that is likely going to be an issue that becomes more and more common as Apple moves to move more and more tools and utilities to the Xcode tools download anyway. Support/approval teams are just going to need to adjust to that reality, even if it’s an annoyance because I don’t see things shifting back, at least with Apple.
But issues of approval aside, I honestly think back of myself as a teenager, installing fink and Macports and figuring it all out as I went along and I wouldn’t have called myself a developer then. I was just a girl who wanted to tinker with stuff.
I think we (and I’m not talking about you specifically but the more general “we” as in the types of people that frequent HN) often underestimate the abilities of our non-dev brethren. We overestimate how much users know too, of course, but I do think we often slot people who have shown that they have the capabilities and desire to solve a problem into a “lesser” box just because they don’t carry the title of developer. And that affects those users too, who then assume they can’t do something when their own track record proves otherwise.
Definitely. I work a lot with librarians — that's a field which isn't considered “technical” but then you look and there are people writing shell scripts to process collections, writing tools in languages like Python to manipulate metadata or work with APIs, etc. and that distinction starts to look pretty arbitrary. (This is actually a big retention problem in the field: someone who learns that in college / on the job is going to look at the pay multiplier they can get by applying for a developer job)
It means I can’t go to help someone with their laptop and use emacs to look at their cruddy code :) but covid ended that sort of help anyways, now I have to watch them type in screen share and see if they know about tab completion and shell history thru the up arrow.
Part of the magic of macOS was a pretty UNIX that just works.
Sure, I can install and deal with the burden of a package manager, but I don’t want to and didn’t use to need to for basic tools. I want Apple to do this.
If you’re installing your own shell making sure it doesn’t conflict with anything else, you might as well run Linux.
Personally, I'd like to know why Apple decided to ban GPLv3 software in macOS. It's not locked down enough to trip the TiVo clause[0]. My ongoing guess has been "just in case we need this in iOS", but these tools are all things that Apple would never ship in iOS anyway - the whole point of that OS is that you don't get access to any of those tools[1].
Alternatively, they might have just taken the Linus Torvalds tact of "this changes the deal", even if that change is minor. But that's not so much a license compatibility problem as much as it is a difference of opinion. Which is something that might never actually be resolved. Thanks to the v3 split, "GPL" now means two different licenses based on RMS's personal opinions on what restrictions best preserved software freedom at that time[2]. Even a hypothetical GPLv4 that dropped the TiVo clause for the sake of appeasing Linus and Apple would be changing the deal for people who wrote v3 programs expecting them to be a bulwark against locked-down devices.
In either case, it does not make sense for Apple to ship decades-old utilities that it refuses to update for legal or moral reasons. At best, they're dead weight - it is very much common practice for people developing on macOS to install the newer versions of those utilities themselves. At worst, they are an actual security hazard, as they aren't sandboxed in any way. Bugs in those utilities would be very much exploitable on macOS. So if we've decided "we're not going to update this ever", then we might as well get rid of it and replace it with something else.
[0] The TiVo clause in GPLv3 is actually not that strong, BTW. It requires that, if you ship the software as part of some hardware, that you provide a way for the owner of that hardware to modify the GPLv3 bits and have the hardware still work. The only thing that would remotely trip the clause on macOS would be SIP, and that's also owner-overridable.
You will lose iOS app support in the process on M1 Macs, but the TiVo clause doesn't cover that scenario - which is ironic because that's exactly how TiVo worked around similar language in GPLv2.
[1] Apps like iSH or a-shell do exist, but they run x86 or WASM binaries within the App Store sandbox. They also don't trip the GPLv3 TiVo clause.
[2] At one point, RMS was seriously open to the idea of just rolling the Affero clause directly into GPL, which probably spooked people way more
I wonder if it has something to do with the Busybox lawsuit. I think that it might be more to do with "we don't want to rock the boat with licensing...GPL2 has worked for us, and we don't want to be forced into a corner like Linksys was with Busybox".
This is entirely speculation on my end, IANAL and I didn't work in that part of Apple.
I find it kind of frustrating that Apple would use that as a rationale without taking efforts to support homebrew. If they didn’t like the way homebrew was engineered, they could have afforded to port dpkg or something similar. If they did like homebrew, they should have hired on the core developers.
It’s less true today than it used to be, but Mac was the best Unix developer box in the corporate world. A huge portion of their sales (especially high end sales where they make the most profit) relied on these community efforts.
I’ve moved over to using nix instead of homebrew and it’s so much better that it makes me wonder how it ever got this way. I think this is how.
The funny thing is that Apple uses Homebrew internally. We had custom taps and it was part of a lot of setup documentation in our Confluence.
I think the issue is that Apple isn't "one" thing. There are thousands of engineers, each with their own opinion, but also a thousand lawyers and corporate executives who are very concerned with how every action will make Apple "look" as a company. Sometimes their decisions made sense, sometimes they don't.
They hired the original lead developer of Homebrew, which seems to me like support. He chose not to continue working there and seems to now be selling NFTs of his old tweets and git commits, but it seems hard to pin that on Apple.
I'm assuming, of course, the idea was to let him spend at least some of his company time working on Homebrew.
In perpetuity? Why? In forty years they should be using a 55 year old version of Emacs and a 45 year old version of Python? I respectfully but firmly disagree.
Very few are using it, it's not getting updates, and is a potential security hole because of the lack of updates, it seems to me to be smart to get rid of it. Most people wouldn't want to use that ancient version of Emacs, and would install a newer one via Homebrew or GNU Emacs for OS X.
What I don't get is why they bundled that version for so long. It would have made perfect sense to stop bundling it when emacs switched to GPL3. It just caused annoying issues where trying to use emacs in your terminal would bring up this ancient version that would potentially conflict with your .emacs.
RHEL and macOS are two very different products with two very different audiences and can hardly be compared like this.
That being said, Python 2 will last through the end of RHEL 7's lifespan through June 2024, and that includes the AppStream module in RHEL 8 which will similarly end support in June 2024. After that customers are recommended to upgrade to a supported version of Python or continue using Python 2.7 at their own self-supported risk, we will not support them in that endeavor.
RHEL 9 (due out in a few months) does not ship Python 2.7.
With Ubuntu, if you delete Python, it obliterates everything. Have fun re-installing your system. Learned that the hard way. You can bet I "use Arch btw" now.
I don't see how that's relevant here, unless you interpreted my statement to mean we will be removing Python 2.7 in RHEL 8. That isn't happening, the module will remain in the AppStream repositories but will not receive any support.
Removing the system Python is prevented on RHEL/Fedora (with an exception). The package manager, DNF and YUM, are marked as protected which prevent their removal. You'd need to remove the protection file from /etc/dnf/protected.d before you can do that. Removing the system python would remove the package manager which depends on it, therefore the transaction is stopped. Using the RPM utility directly also will not work unless you a) pass --nodeps or b) list out every single dependency at the same time manually. You can test this in the CentOS containers with the following:
It’s not just “value stability”, there’s a good chance that without them knowing some part of their billion dollar enterprise absolutely depends on an old python component that would break if 2.7 were removed.
They know that at some point, their RHEL version runs out of extended support. Two years before that they’ll experiment with the new RHEL version on their test environment and notice everything is totally broken. And then they’ll have two years to fix it which is just barely enough time, as “fixing it” requires buy in and planning and resources from dozens of divisions and hundreds of people.
To enterprises, having Red Hat support some ass backwards old version of whatever package is not just “valued”, it’s exactly why they pay millions to Red Hat instead of running some free Linux version.
Python 3 released in 2006. 2.7 was sunset two years ago (after a 5 years stay from execution). No one can claim they did not have enough time to make the switch.
Sorry for my short comment. I'm glad you understood what I meant.
For anyone else who reads this, strings and str() went from being arrays of bytes in Python 2 to Unicode strings in Python 3. That change broke a lot of programs and is one of the most criticized Python 3 changes.
I know it shouldn’t, but the python community’s bizarre behavior during the 2.X -> 3.X move honestly made me think less of the language.
Tech is supposed to just be tech, but when the community behaves this badly about adopting improvements how can that not influence your decision to invest in that tech? If 2.X -> 3.X was such a drama fest what’s gonna happen next time they need to upgrade? Etc
> Inertia seems to best describe the hesitancy to change.
I'd have gone for 'rigor mortis', but hey, we can't agree on everything ... like, say, a very minor change to string encoding with a 12-year managed rollout.
That's the problem - they literally do not. Technically better is not a good enough reason to change something. It has to provide some actual greater user value along axes that matter to users, and make the cost worth it. What you are seeing is that python3, well, doesn't. The options in that case are reduce cost or improve along axes that matter enough that cost is worth it.
> Tech is supposed to just be tech, but when the community behaves this badly about adopting improvements how can that not influence your decision to invest in that tech?
What you say was bad about it? And who were the bad people specifically? The people who were using python 2 or python 3?
For what it's worth, python3 >= 3.0 && python <= 3.2 were hideously broken in their unicode support. Arguably had worse/unusable uncode relative to python 2.6 or 2.7.
So there was a huge failure to launch type of problem, especially given how long python3 had been development.
It most definitely left a very sour taste in many people's mouth that didn't start dissipating till 3.5 or 3.6 when enough "killer" features had accumulated.
Even then, for a lot of usages, python 2.7 'just works'.
> the python community’s bizarre behavior during the 2.X -> 3.X move honestly made me think less of the language.
There weren't that many people who were outright against Python 3, other than Zed Shaw for a few years. It just took until Python 3.4 for the language to really be usable in production, and after that it took a couple more years for every library to be updated. But the community has been pretty unified for 5+ years at this point.
This is my experience as well. There were plenty of people online vocally against Python3, but once we hit 3.4 I never worked with anyone IRL that was against it. The mindset was almost always "this is the way forward", so that's how we moved with our projects.
If by unified you mean no longer arguing, sure I switched from Python 2 to Go lang. (I will concede if I am doing exploratory stuff, I will start new repos in Python 3, e.g. for boto quick exploration.) I am still bitter about the fine Python 2 code I supported and extended that is now all go or Java.
I used it production just fine before that. What didn't work for you? The unicode support was shoddy, but lots of other stuff worked fine.
The problem as I remember it was that there were some critical tools that just were not making the switch. The last one I could remember having to have a separate runtime for was graph-tool, but there were just a ton of them in the scientific computing community. Also, I don't think making print a function was worth it. So many people had muscle memory on that that personally I think it was worth just single casing it.
There definitely were, at least in the early versions. It would have been fine (I guess) if those early versions/iterations had been short-lived, so to speak, and for a stable and definitely faster version (compared to 2.7) to be made available shortly after that, but afaik that wasn't the case.
If it matters I've been writing Python professionally for more than 15 years and I started my career by seeing the (less famous) Zope2 to Zope3 botched migration. When Python3 was first announced I had hoped that the devs behind the project had learned from that related experience, guess I was wrong.
The "active community" is unified, I know a bunch of people with some personal piles of Python 2 who I don't expect to upgrade until they have no other choice. I also don't see why they should -- their code is running fine.
> But the community has been pretty unified for 5+ years at this point.
Sure, but they accomplished this mostly by displacing anyone who disagreed. That's fine, and Python 3 in a vacuum is a better language than 2. But I'd never make the mistake of choosing Python for a serious project going forward.
My company started life in about 2014 and for python is only on 3. And python 3 is great. Having written python 2 before and also perl, I really appreciate UTF-8 just works.
And I hope that python learned from 2 -> 3 and that in the future it will be better
The fact that you think this way shows why that community is so broken about things like this. You think the user community behaved badly? It was the developers.
For the vast majority of companies, python 3.x is not better enough for the cost. That is what happened. You may think it's awesome. It may be technically better it, etc. But the cost of moving was much more for most companies than the productivity gains. We've even measured it before. That is entirely on the python folks for not providing something people want enough to move to. All CSAT surveys of python i've seen at my company, and that my counterparts had ever seen, said literally the same thing.
So it seems like something the community could have discovered pretty easily in a rigorous way if they wanted. I used to spend a lot of time on python-dev. It's a group of good people and engineers for sure. There was always good rigor around technical reasons to do something, and around performance benchmarking. But i don't remember seeing a lot of productivity research or CSAT or .... It's fair to point out that most OSS communities don't do that, but that's a bad thing, not a good one, and one of the things that often leads to community trouble
One of the other major factors, besides the productivity issue, that makes it not better enough is the migration cost. Software of this kind needs to be built to be migrated from and to. If not, it's broken and you deserve all the animosity you will surely get from your users. It doesn't matter if you screwed up the language, etc. Tough crap.
In python, for lots of folk with substantial stacks, the amount that could be auto-migrated by the tooling provided was low - 40-50%. Yes, simple stuff could be auto-migrated, but not most real stuff.
We actually ended up building our own tooling to get closer to 70% because how expensive the overall thing. It processed many millions of lines of python code, automatically kept track of which things would be best to rewrite/finish migrating first (because it would unblock other things), and continuously tried to auto-migrate any non-migrated changed code in a loop to maximize automation.
The total engineer cost of moving was enormous. We are talking well over a thousand SWE years.
For what, exactly? As I said, we already measured (because plenty of new projects were python-3 only) and determined that all the improvements and hullaboo did not meaningfully improve productivity for people. Not enough. It was small, at best. It also didn't reduce the rate of production bugs, etc.
So all this work, for no meaningful value.
Because not enough were willing to do it, the result was to force it. Damn the users, full steam ahead. The users are just lazy.
I talked with lots of counterparts at other companies - most felt the same way.
The result for us is lots of teams gave up python forever.
The only reason it's growing at all is because of ML.
IMHO, python2->3 is an object lesson in doing it wrong and then victim blaming.
"We actually ended up building our own tooling to get closer to 70%. It processed maany millions of lines of python code, automatically kept track of which things would be best to rewrite/finish migrating first (because it would unblock other things), and continuously tried to auto-migrate any non-migrated changed code in a loop to maximize automation. The engineer cost of moving was enormous."
This was definately an opportunity for commercial software, i am not sure why companies worth hundreds of billions can't sort this out and expect a small nonprofit to provide this.
FWIW - i don't disagree, and I don't expect anything in the end, it's open source. People can do what thy want.
I was more pointing out the opinion given of how the user community sucked is very broken.
What the "larger user" (however you want to frame it) community would have preferred (IMHO) is much better migration tooling, even if it came at a cost of improvements.
The dev community chose to spend time on improvements instead. That's their choice, but it doesn't make the community wrong to be upset about it.
They forced incompatible changes on their users, all in the name of a Unicode fetish. They burned their credibility. The users acted rationally, it's the developers of Python who were abusive.
I think it’s less of a fetish and more of a “we made a huge mistake in the initial design of the language and confused text and the encoding of that text making a giant footgun that was hard to avoid.
The migration was difficult because you had to actually think about this stuff instead of it working by accident and living as a landmine if you encountered non UTF-8 text.
The fact that you actually have to decode your bytes and specify an encoding makes it really obvious when you’ve just assumed the world is UTF-8 without really backing it up.
What I don't get is it they break the language anyway, why they didn't fix classes. Most of python is pretty beautiful, but classes are so ugly, even basics like what is the Syntax for instance vs static cars, self/this, setting up constructors, calling inherited constructors. Like come on, how is it that they are uglier than Java in this regard.
The print() change drives me nuts: many scripts use the old form, and Python forces you to change it despite recognizing the old functionality. They could've simply allowed print expr to work as is and accepted it as a wart.
The async stuff feels a bit like a bridge too far for me WRT the language. Simplicity would be standardizing on a function color (as it were).
I don't envy their position of preserving backwards compat while evolving the language.
I think the person you're replying to was saying that there was no reason to remove the statement form. The only thing that removing the statement form of print did was break existing code. You could easily allow print to act as a function by removing it as a reserved word in expression contexts.
They made print a function out of a misplaced sense of language "purity", where they made a random ideal and broke code to support it.
Let's compare to JS: 20 years of new features and old code still works. Keywords were added, and old code still works. Even code that uses those keywords still works. It took more effort from the language implementers, but that's as it should be: it took a bit more effort for a few teams (who make the engines), instead of a large amount of effort for tens of thousands of groups.
I really don't know what you're talking about. I've written a lot of object-oriented code in Python and Java and there isn't a single thing I like better in Java (that doesn't come from the typing system). I don't know about 3.0, but it's been great for years. Take a look at some Django code.
While I happily use Python 3 now, the transition was very difficult, and I was grumpy about it at the time.
I work with code that uses a lot of files in a lot of different text encodings. Some are XML, some plain text, and some binary. Coming from other languages, Python 2's Unicode support was very difficult to work with, and my team was excited to move to Python 3, until we actually had to do the work.
Long story short, Python 3's str/bytes separation was a nightmare, especially when having to deal with third-party libraries that were expecting one or the other. This was especially true for libraries that came from Python 2, and were still expecting strs as function parameters when they should have switched to bytes. There was so much encode() and decode()ing going on that we occasionally caught ourselves getting them backwards in code reviews. We took a step back to see if we were just architecting things poorly, but no, we were doing things the best we could for our problem domain.
In the end, we traded one set of Python 2 text encoding footguns for a mostly different set of Python 3 text encoding footguns. It's not like Python 2 was great. Having a single str type instead of the str/bytes separation is worse in theory. But because Python 3 didn't design the separation of those types well enough, and still has a ton of text encoding footguns. It wasn't better enough to really justify all the work that went into the conversion.
What do you think the current text encoding footguns are?
In a different direction, I don't know what your problem domain is/was, but in general when I'm dealing with UTF8, I don't need to convert back to bytes very often. Was the need for conversion mostly due to the libraries that still expected strings instead of bytes?
It's been quite a few years since we went through the conversion, and at this point, working in Python 3 is natural to me, so I may not be able to recall all the footguns. I can say a lot of the difficulty was due to libraries, both third-party and standard, and that hasn't improved very much. I don't want to single anyone out here, because it's pervasive. In Pytuon 2, str was the bag of bytes type. I think a lot of libraries didn't want to change to accepting bytes types instead, because it broke API compatibility, but it caused a lot of issues.
I should also say that we were working with files in tons of encodings, not just UTF-8. We had UTF-16 and UTF-32, both little and big endian, with and without BOMs, but we also had S-JIS and a bunch of legacy 8-bit encodings. Often we wouldn't know what encoding a file was in, so we'd have to use the chardet library, along with some home-grown heuristics to guess.
Off the top of my head, the two biggest footguns are:
- There should be no way to read or write the contents of a file into a str without specifying an encoding. locale.getpreferredencoding() is a mistake. File operations should be on bytes only, or require an explicit encoding.
- .encode() and .decode() are very poorly named for what they do, and it wasn't that uncommon that someone would get them backwards. Sometimes, exceptions aren't even thrown for getting them wrong, you just get incorrect data.
Both of which were still issues with Python 2. There's a valid architectural argument to be had between the Python 2 way, where str was a bag of bytes, and the unicode type was for decoded bytes, and the Python 3 way, where the bytes type is your bag of bytes and str holds your decoded string. I favor Python 3's way of doing it, but it's almost six of one, half a dozen of the other. The advantages of one over the other are slight, and given how many library functions relied on the old behavior, it was probably a mistake to change it like that, rather than continuing the Python 2 way, and fixing issues like those above that caused problems.
I haven't done a ton of work with Python recently, but the problems I remember encountering came from the fact that python doesn't try to have encodings in any other part of the basic type system. So like, if you have an int or a float, you can pass those to any interface that takes a 'number-y' value and it will mostly work like you expect. That's also how strings worked in P2 - you could pass them around and things would accept the values (though you might get gibberish out the other side). Now, in P3, things will blow up (which is helpful for finding where you went wrong ofc - I understand the utility), but it means that your code handling things-that-might-be-strings-or-bytes often needs to have a different structure than the rest of your code.
I think the P3 string/byte ecosystem was made substantially weaker by P3 deciding not to lean more into types (something I have complained about on here before!). Like...they are the only values where the stdlib is extremely specific about you passing a value that has the exact right type, but the standard tools for tracking that are pretty poor.
> but it means that your code handling things-that-might-be-strings-or-bytes often needs to have a different structure than the rest of your code.
Isn't that the point? String and bytes are different beasties. You can often encode strings to bytes and just about anything accepting bytes will accept it, but the converse is not true. Bytes are more permissive in that any sequence of any 0x00-0xff is acceptable, but str implies utf8 (not always guaranteed, I've seen some shit), meaning e.g. you can dump it to json without any escaping/base encoding.
That was probably an inartful way of saying it on my part. Echoing what I said in another comment, I think the API for converting between bytes and str types is bad. .encode() and .decode() are bad names for what these functions do, and let to occasional incorrect code in both Python 2 and 3. I would have preferred something akin to:
b = b"some bytes"
s = u"some str"
foo = b.to_string(encoding)
bar = s.to_bytes(encoding)
or
foo = str(b, encoding)
bar = bytes(s, encoding)
I also mentioned that I think there shouldn't be any mechanism to read or write files to/from a string without specifying an encoding. locale.getpreferredencoding() is a mistake.
But one thing I didn't mention that I think would improve the design would be to make it harder to treat str as the bag-of-bytes type it was before. Swift took some steps down this path, and honestly, it made it much less pleasant to work with strings, but a bit safer. I would hope that a better design could be found.
This may be controversial, but I don't think you should be able to just subscript a str type. instead of s[1], you should be writing s.codepoints[1] or s.graphimeclusters[1], etc. depending on what you actually want. If str is truly a string type and not a bag of bytes, it should deal with the extra complexities that being a string brings to the table.
Oh, I see what you mean. Yeah, I have gotten tripped up wondering whether I should be calling encode() or decode() many times. Using the str() or bytes() constructor would be more intuitive for sure.
Interesting thought on subscripting strings. It would be hard to make it fly after all these years of habits, but I see your reasoning.
Transitioning was hard, especially with all the libraries in various state of support between 2 and 3, I had a similar experience.
But I disagree with you about the separation of bytes and string and the current state of the language. I write a lot of python that deals with bytes and text encoding, and now that all the libraries have caught up with 3, the situation is way better than it ever was. encoding, decoding, bytes manipulations are way less prone to errors.
I'm glad it's improved. I moved jobs a couple years ago, and since then, have only used Python for a few small personal projects. When I left, there were still some very rough edges around some very popular (and some not so popular) libraries.
And, as I said, I do prefer the str/bytes split over the unicode/str split, but I also wasn't doing a lot of raw byte manipulation. I agree that it was harder to do with the old str than bytes. I was mostly doing string operations on the grapheme cluster level, and then writing everything out as UTF-8, so I didn't see as much of the benefit.
I actually like Python a lot. Although I'm no longer using it professionally, it's my first choice for personal projects.
I'm also an advocate for using the right tool for the job, and in this case, Python may not have been the right tool for this job, but this was only one component in a much larger system. Sometimes you have to be suboptimal locally to be optimal globally.
And it's not like Python couldn't handle it, it was just that it had some design decisions that made things a bit harder than it would have been in some other languages. We got it working pretty well in Python 2, then the Python 3 transition happened, and it was a lot of work to get everything working as it had been, for only a small benefit to our team, but we got it working in 3 as well, and to my knowledge, it's still humming along.
If the reactions seem bizarre, maybe you're missing something.
For me it was more than "just the tech". Python 3.x was not the Python 3000 I was promised for a long time: many of the additions did not warrant breaking changes (as demonstrated by the backports) and many of Python's deeper warts still remain unaddressed, even today.
I think the deeper issue is that at the rate that 2.x got popular, the original motivation, principles and process behind the language's development got diluted. The orchestration of the push for 3.x adoption was not technically motivated in a convincing manner and signaled a change in the process. I stopped trusting the process.
I did a bit of contract work for a place with over a million lines of custom python 2.x scripts that ran inside of a python interpreter embedded in a proprietary product which made it difficult to do things like write and execute unit tests for the code. I think they were still writing new python 2.x code in 2018.
A lot of the scripts were supporting processes that had a finite lifespan during the initial stages when the company infrastructure was being built out, rather than ongoing operational processes that needed to be performed indefinitely, so hopefully they'll be able to set a lot of that codebase on fire instead of maintaining or porting it to python 3.
If that was the argument they'd have replaced it way sooner, because 3.x came out 14 years ago, 3.4 (when the 3.x branch could be called "actually ready for use") came out 8 years ago, 2.7 was announced to go EOL "soon" 5 years ago, and was properly killed off almost 2 years ago now.
We've seen an entire major version release of MacOS come out since Python themselves went "look, we're killing this thing off, deal with it", at which point Apple should have dealt with it, but didn't.
This is a replacement for the convenience of their own staff working on the OS, nothing more. It's good that it's finally happened, but it's a good five years overdue, and there's no motivation beyond "we no longer use it" at play here.
>Choosing to keep an unsupported piece of software in the system is a security liability.
It is the OS design that doesn't make it possible to support the principle of least privilege that is the security liability. Blaming users for wanting their working software to just keep working the same way is abuse!
If they want to run a piece of code written in HyperCard for the original Macintosh, they should be able to do so. This push towards ever more complexity while failing to fix the inbuilt security fault inherited from Unix is insane.
Why not? If you have a fault in a lamp switch, the circuit breaker keeps it from burning down your house.
Why can't an OS keep a rogue program from taking down your machine? You could specify what to run, and with what resources, and then even if Satan himself wrote the code, it wouldn't ever corrupt anything other than the resources specified.
This has to be one of the dumbest arguments i've ever seen on this hellsite.
Firstly, a circuit breaker does nothing but protect the wiring in your house. A faulty lightswitch can still throw sparks and start a fire as long as it never draws more than 10A or whatever your circuit breaker is rated for.
Similarly, operating systems can prevent a rogue process from taking too much CPU or RAM, but can't prevent a process from ransoming your files, at least not without a substantially different sandboxing regime to what exists in mainstream operating systems today. Why we don't/can't have this is an interesting disucssion that is much larger than the python2/3 question.
We all knew this was coming, but doing it in a point release seems like exceedingly bad form, particularly as Apple does major releases annually. I never expect point releases to break stuff, at least not on purpose.
The risk of unpatched vulnerabilities between now and the next "major release" far exceeds the risk of inconveniencing anyone foolish enough to rely on the system Python 2.7 installation.
I think this mindset is dangerous. There's a really long tail of niche things people do on their computers, such that I suspect a majority of users have at least one niche need. Especially today, when Chromebooks and iPads are available for anyone who only needs basic web browsing.
If you stop paying attention to the niches, they add up to a lot of people.
You are right, but I would say that this is quite extreme case. This is a programming language which support ended 2 years ago. Only reason to use it is to avoid turning old code base to new one. There are some serious questions if that is really a good approach. And if it is, why you need to use pre-installed Python 2 instead of installing it by yourself?
I expect that there are users, who are not themselves developers, who are using either apps or scripts which rely on the system-installed Python 2. They likely aren't aware that they're using Python, and they may not even know what Python is. It could be one line in one shell script block in one automator action that a coworker put together five years ago.
It's okay for that stuff to break. (I wish Apple would put more effort than they do into making it break less often, but that's neither here nor there.) But, it should break in a major release, when the user is prepared for things to break. It should not break in an update that was installed automatically overnight.
If they wanted to announce at the June 2022 WWDC that macOS 13 wasn't going to have it, that's fine. But we're several months into macOS 12.x and such a large change is bad form.
Apple supported Python 2 for two years after it was officially sunset by the Python Foundation. They've been advising that they will drop support for longer.
How long is long enough to avoid breaking something the users depend on? One more year? Five years? Fifty? I think we can all agree that it's reasonable to drop support at some point, making it just a question of when and at what cost.
"How long is long enough to avoid breaking something the users depend on?"
10 years - a piece of software compiled with 'supported' libraries, SDKs and what have you should function for 10 years. An appliance or a service should function for 10 years and have spare parts avaliable for that long.
That is the guideline I ise as a consumer, if I can see that something won't last a decade, I won't be spending money or time on it if i can avoid it. At that point it's no longer a durable good, it's a consumable.
If there are users that still depend on python 2.7, they can install it manually?
I mean I already think it's... interesting that an operating system comes with (multiple) programming language interpreters, available to the end-user.
That’s true if they’re using it from the command-line, but not if they’re using apps that linked against the system SDK’s libpython2.7.dylib (and assumed PyObjC would be present)
Is there any significant software still using that?
Apple is pretty aggressive about breaking backwards compatibility, as they did with x86, so it's no real surprise for unmaintained software to become completely unusable.
Based on what they’ve done, I guess the answer is no.
My company was shipping (somewhat niche) software linking with the built-in python2 up until last year though, and it’s been a scramble since Monterey to rebuild things to avoid the warning message. Definitely thought we would have python2 until 13.0 though (and part of me wonders whether this is just a “brown-out” for the beta and 12.3 release will be back to normal)
I think an end user that wants to automate things with a script can be expected to install their interpreters themselves or use an interpreter that builds on the OS's shell.
I am not to familiar with MacOS, but I imagine it comes with a bash/sh terminal? If so, then that is something that should be first choice for automation.
Everything else, be it python, perl, ruby or whatever should be installed by a user as needed.
And I think that's totally reasonable—as long as Apple makes the change in a major update, when users can expect (some) things to break and should be encouraged to make time to prepare.
Part of it is that it's just a mental burden to keep this kind of compatibility information in my brain. I have VMs set up of past macOS releases in case I need to run something old. I don't have VMs set up for earlier versions of a release, and I don't even know where I'd get an installer. Admittedly, in this case it would just be a matter of installing Python, but I don't necessarily know that as a user, or the software could need to be linked to Cocoa or some such.
Bingo.
This falls under the scope of support for an enterprise linux company-- orgs pay them to keep stuff backwards compatible long after the upstream deprecates it. Apple is decidedly not an enterprise support organization.
RHEL also doesn't want you using system python as your dev python. RHEL 8 uses 'platform-python' for the bits the system needs and if you install 3.6, they share the common bits
If your using a mac as a dev box, and really need python 2.7, you still have options.
Let's face it: at this point macOS version numbers are a marketing tool, and not much more. It's only 2 major releases ago that a "x.y" release *would* have been considered "major".
I don't think that's particularly relevant. It was pretty clear that between 10.0 and 10.15, the second number was the "major" version for all intents and purposes, and now it's the first number.
Just because someone has the resources to do something it doesn't mean that doing so is the right {business, technical} decision. You're confounding the two.
Customer service is still a business reason. It serves to keep your customers happy, and the calculus here is probably that there aren't enough customers to keep happy by maintaining python2.7 that would justify the costs and headache.
> Apple the corporation doesn’t have a package manager.
> MacOS, the ecosystem does.
> No different from Linux. There is no Linux package manager. There are numerous open source package managers for Linux.
macOS the OS has a package manager in terms of a tool to install packages. It does not have the ability to find/download packages to install though.
In the sense of a package manager being able to download and install additional software, macOS has several choices available but none of them are included with the OS, or have the ability to update the OS.
There is arguably no single "Linux ecosystem". There are numerous Linux-based distributions, which are largely comparable to macOS as an OS. One of the very key differences between each family of distro's is the package manager the OS ships with by default and uses to update itself.
So no. macOS does not "have" a package manager that's comparable to those included by default with linux distributions, but there are several third party package managers available for it.
> Another false statement. Homebrew is just as capable of searching as any Linux package manager.
I wasn't talking about homebrew.
It helps if you understand the comment before you reply to it and make accusations.
Edit to add, because fuck the HN timeouts. Heaven forbid someone make 6 comments in two fucking hours:
Try reading the whole fucking comment again, with the points being made really hammered home for you:
> macOS the OS has a package manager in terms of a tool to install packages. It does not have the ability to find/download packages to install though.
This is about the built in macOS package manager, that the OS itself uses to install packages. It's invoked via `installer` on the command line, or Installer.app in the GUI. It does not expose the ability to find/fetch packages from remote URLs.
> In the sense of a package manager being able to download and install additional software, macOS has several choices available but none of them are included with the OS, or have the ability to update the OS.
This is referring to homebrew, macports, etc.
We certainly do see things differently, because only one of us is repeatedly calling the other a liar due to poor reading comprehension.
If you don’t include homebrew, as a package manager for MacOS, then you aren’t having a serious conversation.
This whole line of conversation is about whether it’s necessary to manually install python.
If you claim it is you are lying, because it can easily be installed via homebrew, and you are clearly aware of that.
If you have some need to claim that homebrew isn’t a MacOS package manager or that MacOS doesn’t “have” homebrew, because Apple doesn’t ship it. Go right ahead, but it’s a red herring.
No, it means that macOS and its third party software aren't one in the same thing.
Is faker.js a macOS vulnerability because there were people using Macs who installed npm and then installed faker.js? This isn't a theoretical question—whenever you install something, you need to consider where it's coming from and whether they can be trusted.
It has a third party package manager, yes. I'm pretty sure your parent meant that there isn't a first party one.
On Debian, packages in the official repositories are considered to be literally part of the Debian project. The maintainers are considered Debian maintainers, and bugs go into the Debian bug tracker.
This matters due to the trust issues I alluded to above. It also means that on Debian, there is one definitive version of Python (or at least one per Python version). The Homebrew, MacPorts, and Python.org Python binaries are all slightly different, such that it's possible for software to work with one but not the other.
No two Linux package managers are identical either, and not all Linux distributions work like Debian. Debian is a collection of volunteers. The choice to trust it is no different to the choice to trust Homebrew. Considering the package manager to be part of Debian doesn’t change this.
However none of this has any relevance to the claim in question, which is that python must be installed manually.
MacOS certainly has a package manager that can install and maintain python.
Except that most other open source package managers strive to be good at what they do.
Brew has a long history of treating any criticism of their truly woeful approach to security, dependency management etc, with "well fuck you we dont want to hear your opinion".
There were other better macOS package managers before it; macports was sort of even Apple sponsored.
Everyone switched to brew even though it was the most poorly designed because it came from the Rails hype era where everyone wanted all their tools to be written in Ruby and didn’t care about anything else.
I used and still use macports, but there were valid reasons to switch to homebrew: macports did not have precompiled packages, did not use system libraries, and machines were not as powerful as they are now.
This meant if you wanted to install, say, ImageMagick, you would spend one day compiling stuff.
Also contributing a brew recipe was (and probably still is) easier than contributing to macports, and brew casks are pretty convenient.
I deeply dislike some of the choices of homebrew, but I can understand why it was popular, and it wasn't just because of the ruby hype.
It's funny how people think old, robust software is suddenly going to spring a bunch of vulnerabilities while sitting there, static, unchanging. It's far more likely the bleeding edge python(s) are going to be getting new bugs.
Of course it's not going to get new vulnerabilities, but that doesn't mean hackers won't find previously unknown vulnerabilities that have always been there. Looking through the CVE database, core python alone has had 15 CVE reports in 2020 and 2021, and for old pythons those will never be patched.
I'll never understand why people think they can write some software and it'll stay pristine forever. Even if it was had no known bugs at some point in the past, the world is a harsh place and your program, its dependencies, its programming language or even its OS can be found to be subtly broken at any moment. Even if it is not outright buggy, the world will change around you until the interfaces you depend on are no longer there. (looking at you, software program at $WORK that only works with PS/2 keyboards but not USB keyboards)
Many of those CVEs will not impact your application code, though. This is just a side effect of Python having a huge and somewhat crusty standard library.
If it is dynamically linked (probably the case, as that’s the norm on Mac OS), it might even get new vulnerabilities.
The combination (old Python, new shared library) could have vulnerabilities that neither (old python, old shared library) nor (new python, new shared library) has.
In the old days, software was like math - it had immutable, provable qualities. In the last several decades software has gradually become more and more like biology, where the core mechanisms are shrouded in mystery and we rely on observation and experiments to understand. The Star Trek game in Basic I typed into a computer is eternally true, a platonic form in a way software now no longer even aspires to be.
I'm sure it is a fine program, don't get me wrong. But citing a Basic I program as something that is eternally true is exactly what I mean when talking about the world moving on. Does it even run on 64-bit operating systems? Will it keep working correctly after the unix timestamp goes beyond its signed 32 bit maximum and overflows? I'll also eat my hat if there is anything you can prove about it in the modern meaning of formal verification.
There is plenty of software being written to very high standards of quality (formally verified real-time control software for rockets/power plants/etc comes to mind), but most of that is not available for free on the internet (or for $5/month as a SAAS) because you get what you pay for.
NASA software is very good, but not in general formally verified, I think (based mostly on reading postmortems on various NASA failures).
Even TLA+ analysis and proof of correctness doesn't stop things like "new zookeeper clients are pushing a lot of new writes => disk space needed for logs increases => across all the nodes in the zookeeper cluster => the entire formally correct cluster keeps falling over now but it didn't last week"
I meant more in the sense of the proofs in TAOCP - these are programs that don't interact with anything outside of the program itself, no non-specified input, so a lot simpler, no dates, no actual machine memory, just a little synthetic world defined by the language itself. Even TeX is imagined to be unchanging after Knuth's passing, and is basically unchanged since the 70s.
For sure the world has moved on, but we have lost the ability to have confidence in the software and have had to develop tools and ways of thinking inspired by biology.
I think, striving for software to not have a maintenance need if no new features need to be implemented is sound. I am very well aware, that the complexity of any but a trivial piece of software on a trivial piece of hardware is beyond human comprehension.
The general approach of most projects and vendors at the bottom of the stack to not accept full responsibility for their part is striking. The amount of lying, faking, leaky abstractions, adapting but not 100% etc. is so immense it is a wonder we get something done at the higher levels of the stack at all.
I cannot comprehend, how it is possible, to hammer the RAM in such a way, that you can affect neighboring cells charge until the information interpreted based on the amount of charge in the cells is changed. This should never ever be possible. I haven't seen any such disclaimer this was possible on the package or in the data sheet of any RAM module. The same is with changing the kind of magnetic recording in the same hard drive model without notification (or change in model name that would be more honest). And then there are "subtle" problems, like coil whine on mainboards or PSUs for no reason - we know how to fix this. These high pitched sounds can give you headaches without you hearing much or anything. There is no indication about it on the package and almost no review tests this. And the whole Meltdown and Spectre type of bugs is just the cherry on top with major performance regressions because of it. So how are you supposed to write reliable software on top of all this if just a tight loop might actually change the state of hardware in such a way that is becomes a problem? I mean, this is just crazy.
Why is this so downvoted? I really missed the days of python 2.7, I spent far less time debugging breaking version changes and spent more time on my actual work.
For security critical work, like web services, it makes more sense to constantly upgrade to the newest version. But for scientific work, which is a huge audience of python users, who often use obscure poorly supported libraries there is a lot of value in a stable foundation.
> I really missed the days of python 2.7, I spent far less time debugging breaking version changes and spent more time on my actual work.
This has been the Python 3 experience for years. If you had lots of previously ignored bugs in your Python 2 around string handling, it took longer but on a clean codebase it was often only a matter of minutes.
And yet just the other day on the python 3.10 migration in arch stuff broke and I wasted a good day hunting the error down. The solution was installing python 3.6, which should hopefully work for a year or two until THAT is left unsupported.
Kind of like how various Python 2 releases broke stuff and required time to fix? It's easy to forget that if it wasn't memorable but, for example, I remember the specific 2.7.9 release for that reason.
Vulnerabilities are going to come up, absolutely. Unless you believe that the older version is somehow 100% bug free and vulnerability free just by virtue of it being old. Being old doesn’t grant it any magic powers.
What’s old now is what was bleeding edge years ago - it hasn’t been updated (otherwise it wouldn’t be old), it has just gotten older.
And the problem with keeping old versions around is that when (not if) someone does discover a vulnerability, what are you going to do then?
If your answer is “we’ll patch it then”, then that’s not going to be an old version anymore, is it?
If your answer is “I guarantee it will 100% never have any vulnerabilities ever and so it will never need to be updated”, I don’t think we’re living in the same reality.
There was some recent news about a vulnerability in pkexec that's been around for 12 years. Who knows what Python 2 exploits are still undiscovered in a similar manner? Since it's no longer actively supported, attackers have all the time they want to find those problems.
* One of the libraries that old code depends on has a vulnerability discovered. They provide a patch, but it's only for the current version rather than the one used 10 years ago, which means that it's not as simple as just rebuilding Python 2.
* A new attack on a cryptographic component, network protocol, etc. is developed and suddenly defaults previously considered safe are no longer considered safe. As a great example, how many old things broke when major sites and services like PyPI or GitHub deprecated SSLv3, TLS 1.0, etc.?
* One of the Python libraries you use has had a vulnerability which was only recently discovered. The maintainer quite reasonably does not provide unpaid support for free software which they updated years ago.
What all of those have in common is that the problem is less the vulnerability than the fact that you are likely to need significant amounts of unrelated work getting other components updated to the point where you can install the patch. Given how often vulnerabilities are discovered which have been present for many years, this is likely to happen for any sufficiently large code base.
You leave out that even if software A itself doesn't change
1. The OS and hardware under it can change and lead to new vulnerabilities.
2. Dependencies of Software A can change, which can lead to new vulnerabilities in Software A.
3. It's easier to find more and more vulnerabilities on a target that does no longer move. And once they are found there will be fewer experts to detect/fix/report it as time goes on.
I've previously worked at a vendor using Python2. Someone brought up this concern. I looked up the number of security releases we had cut in the past due to Python2 vulnerabilities -- it was zero.
Doesn't need to be an RCE to be considered an exploit of the user's intentions. I can't fathom how people are still trying to use an EOL language or why they would even want to. Learn some new syntax, you'll live and the writing has been on the wall for like 20 years.
I don't think a whole lot of people are writing new software in Python 2, but I think a lot of people are still running old software via Python 2. Software they didn't write themselves and wouldn't know how to modify.
And that's not to say those users won't have to find something new eventually (or migrate the software to a contained environment). It just shouldn't happen in an automatically installed update.
>The risk of unpatched vulnerabilities between now and the next "major release"
His point doesn't necessitate waiting. You could just classify it as a major release instead. That is not to say it should be a major release necessarily.
Right, Apple could have removed Python 2 last fall with the release of Monterrey, and I would have been neither surprised nor unhappy. I was actually kind of surprised when it was still in Big Sur.
Then they could have done it in 2021 with the release of Monterrey. Heck, Monterrey was released before Omicron, so if we're concerned about COVID that should have been less disruptive than doing it now.
This wouldn't have worked either. Lots of countries, where vaccines weren't widely available yet, were going through a huge Delta wave before and after WWDC.
Now that most places have vaccines, it probably doesn't make sense to wait for the next major release because this has already been long delayed.
Other factors could have also been at play - perhaps Apple Inc had a lot of internal tools on 2.X and has only recently gotten around to migrating all of those.
The risk of unpatched vulnerabilities between now and the next "major release" far exceeds the risk of inconveniencing anyone foolish enough to rely on the system Python 2.7 installation.
You make an assumption that the way your system and method of working is the same as everyone else on the planet. It is not.
I have two tools that I use monthly which require python 2.7. The people who made the tools never updated them to work with 3. For this reason, I have a work box that I cannot upgrade.
I know that Macs are aimed at end users, and not developers, but it really is getting harder and harder for me to use my Macs for the kind of work I do. Each major software update borks my virtual hosts. PHP has been removed, and because of code signing requirements, adding it back is so complicated it takes an entire day and filtering through a dozen half-baked online tutorials. No, the ones that say "Just use brew!" are not the answer unless you have a very simple job. I don't use Docker, but from what I read on HN, that's six miles of bad road, too. But as I understand the situation, that's Docker's fault, not Apple's.
As a counter-point, my app was able to rely on the system Python 2.7 for FIFTEEN YEARS because it was an extremely stable component of the OS (and it allowed the app binary to be literally 10 megabytes).
Now? I honestly can’t think of a way to release without adding something like 100-200 MB to the app bundle, consisting of an entire Python installation. With a user-facing app it is not ideal to try to say something like “just download Python and tell me where it is”.
Yeah, and it’s been years or decades since you wanted to use the bundled software on MacOS for stuff that was important. Them removing Python 2 probably will fix more problems due to multiple Python installs than it will break anything. I tend to get a lot of support questions on “how to make this Python thing work on my Mac laptop” and I have seen some screwed up things.
It’s not semver, but I’m willing to bet (although I haven’t checked) that there were probably a few leftover system apps that didn’t get moved to Python 3 before 12.0 (many of the features were delayed, I’m guessing due to the pandemic their work schedule has been disrupted since Big Sur was feature frozen).
But see, if that is the case... Apple is a large company with lots of resources and knowledge of their internal roadmap. If it took Apple's own internal developers until now to remove their dependence on Python, then third-party developers and their users should get at least until next fall anyway.
(I'm not defending the use of Python 2. I'm arguing that, as a general rule, if something takes the vendor X time, users should be given X + Y time.)
Sibling post mentions the kext breakage happening as well as another example, but I feel like they've been slipping on this as they've had increasing difficulty matching major features to their unbending yearly release schedule. So more and more (and this is the case with macOS 12 also, see Universal Control) we've seen big headline features announced at WWDC keynotes not manage to make it into the .0, but rather trickle in over the .1 to .3 releases. Once that rubicon was crossed having more other major/breaking changes start sliding into point releases doesn't seem too surprising I guess, even if the cultural slippage is irritating.
Apple probably has to either give up strict semantic versioning for breaking changes, give up their strict release schedule, or be willing to defer announced features that don't make one major version all the way to the next. Something needs to give. Looks like they've decided to go with a "semantic-lite" where breaking changes and major features aren't the rule with point releases but can still happen in old enough/deprecated areas.
It does seem more needless though in cases like this granted, unlike other examples it's not really clear why they'd do this now rather than 12.0. Not as if Python 2.7 was a spring chicken last year either, support ended in 2020 and that was like 11 years after release already? Heck, why not 11.0?
It's not necessarily strict semantic versioning. However, Apple has always had a distinction between major and minor updates. Minor updates are installed on most Macs automatically, whereas major ones are not. Major updates have longer developer preview and public beta periods. Major releases continue to receive security patches for a couple of years.
If Apple isn't going to follow any types of rules, then what's even the point?
> It's implied in the fact that Apple has different types of updates in the first place.
As far as I understand, Semantic Versioning is not a synonym for "versioning scheme", but refers to a specific scheme (https://semver.org). So I don't understand how Apple having any versioning scheme implies that it must be semver.
I edited my post slightly. I never meant to imply that Apple follows strict semantic versioning—and in fact, they clearly don't, since they stayed on one major release for well over a decade.
It's okay if Apple's "minor releases" occasionally have breaking changes. But I don't think "wait until the next major release to remove an entire language runtime which has been built-in for 15+ years" is too much to ask.
Semver is a particular set of rules for defining what is a patch level, what is a minor update, and what is a major update. Before semver, these ideas still existed, but different groups had different ideas of what they meant.
That is perhaps still true in some large companies.
I kinda view them as using a modified version of semantic versioning where, if we take A.B.C, the normal roles of A and B are combined into B, and A is the big annual release with the media hype and advertising and shiniest features at the forefront.
Meaning A and B bumps could be breaking, but C is still just for bugfixes.
Sure it's not a great system from the technical side of things, but it shouldn't be shocking that Apple puts marketing first.
> it's not really clear why they'd do this now rather than 12.0. Not as if Python 2.7 was a spring chicken last year either, support ended in 2020 and that was like 11 years after release already? Heck, why not 11.0?
I don't think introducing breaking changes in 2020 (with the world turned upside down) for 11.0 would have been a good idea. The same reasoning applies to 12.0, many countries were going through a Delta wave without having much vaccine availability this past summer.
I think Apple has been very reasonable about this - they've been clear that 2.x is going away, they have been extremely accommodating of changing circumstances, and 2.x can still be installed manually after this change.
Let's say I have a little utility shell script I wrote ten years ago. I basically haven't touched it ever since. Inside the script is one line of python for fetching the current time with more precision than `date`, or something, which I forgot about a long time ago.
When I install a major update, I test all my scripts to make sure they still work. I do not test after every point release, and I shouldn't have to!
You should be testing early and often and not relying on arbitrary OS version numbers (Apple disagrees with you regarding the utility) to provide some kind of api stability or assurance.
This is why docker and other hermetically sealed packaging and languages (golang) systems work by avoiding the OS implementations which must change and update as quickly as possible.
The fact anyone would rely exclusively macOS level implementations and somehow expect them to remain stable and backwards compatible for any stretch of time blows my mind.
macOS isn't quite semver unfortunately. The mid year releases are usually quite significant.
While I'm not a fan of removing Python 2 mid major cycle, it's not without precedent, and they've had popups since the first Monterey release anytime anything linked against the system Python 2
Major release are already such a pain in terms of just how many dev ecosystem things break every time. Doing it in a minor release is really a blessing in disguise -- all you have to deal with is this.
It was deprecated back in 10.15 this didn't come completely unexpected. Using it since then always had the risk of it getting removed in a major or a point release.
Sure, and perhaps in that context "deprecation" is the right description, but in general I've only heard that word applied to an API being in-process for removal, not as the descriptor of the final removal itself.
Deprecate, from Latin deprecari, "to pray (something) away".
It would seem the gods eventually answered!
OT: it's the same precari that is in precarious, which originally meant something more like "dependent on the will of another". Both definitions have loosened nicely, but it's still fun to see how we got here, and to think of marking something Deprecated as an act of piety!
It's possible there was a script lurking somewhere under /System/Library/CoreServices that might have needed Python 2. The script got rewritten/snipped off, and there you go.
Now, it's included in the CLI tools. You know how when you run 'clang' or any of those, it's a wrapper that offers to download the real tool for you? That's how the 'python3' binary works now.
Is there any chance that they’ve stopped confusing everyone by aliasing gcc to clang transparently? That’s the change I really want to see. The programs inherently accept different arguments. The amount of GitHub issues that boil down to someone thinking they’re calling GCC but calling Clang instead is staggering. Let my build fail.
M1 MacOS 12.2, never installed any dev tools / homebrew
python --version
Python 2.7.18
python3 --version
xcode-select: note: no developer tools were found at '/Applications/Xcode.app', requesting install. Choose an option in the dialog to download the command line developer tools.
Thanks, but I was trolling the shameless, senseless and topically irrelevant plug for homebrew. Get homebrew! Don't miss out! Everyone's doing it! All the hip youngsters know!
They've done some contentious things in point updates but this is really going to put the cat among the pigeons, especially with the "alternative language" jab. Way to rub salt in the wounds :-)
EDIT: According to one of the commenters xattr is now a binary instead of a Python script, so that's probably the real reason why they got rid of 2.7 now!
About bloody time - the sooner we move on from the Python 2 to Python 3 fiasco, the better. I get hives every time I see a nice project on Github, recently updated, with "Tested on Python 2.7".
Only 14 years after 3 was released, and over 2 years after 2.x was truly killed off for good , but better late than never.
Bundling a dead technology that users couldn't remove or update made using actual normal python borderline impossible without being forced into a third party software manager.
Shipping with it is not the same as "if you uninstall it, which you can't even do unless you boot into safe mode and turn off kernel lever application protection, your OS is now broken beyond repair and you'll need a full reinstall", though.
427 comments
[ 3.2 ms ] story [ 134 ms ] threadHence, I have always (including from the RHEL 6.x days) provided my own local python version by compiling and packing as a custom rpm. I now use venv to do the same.
https://koji.fedoraproject.org/koji/search?terms=python%5B2-...
I can't say I disagree.
Also, feel free to replace `brew install emacs` with the macports equivalent and the point still stands.
[0]: https://www.gnu.org/software/emacs/download.html
[1]: https://emacsformacosx.com/
[0] https://github.com/zsh-users/zsh/blob/master/LICENCE
I believe I switched from the default zsh to the bash that also came with the operating system. I don't think I installed a new bash. I was just trying to add on to/respond to the question "Will they completely remove bash, rsync, eventually?" . My answer was too short and I didn't actually make it clear what I was saying. I think the answer to the question "Will they completely remove bash, rsync, eventually?" is yes, at least for bash, I think they will remove it. My evidence for thinking they'll remove it is if you switch from the default zsh to the bash that comes with the operating system, they'll kindly ask you to switch back to zsh every time you open the terminal. Yes, you can change the default from nagging you with BASH_SILENCE_DEPRECATION_WARNING, which is great to know.
Removing bash is a pretty safe bet. They’ve been making move after move in recent years preparing for it.
https://scriptingosx.com/2019/06/moving-to-zsh/
This is probably somewhat optimistic: there are a lot of people who are not developers who still use those tools — I've supported scientists, librarians, analysts, etc. who had a few scripts using tools like that but weren't really comfortable making changes. That doesn't mean it's _hard_ to install Homebrew but it's something which they'll need a little pointer and possibly institutional approval to do.
I guess I just feel like the net good would be to de-bundle these non-compliant tools and get them to install upstream versions from Homebrew or MacPorts or something.
It's annoying, and I'm not one of those people that tends to be the "OMG JUST RTFM IT'S NOT THAT HARD!!", since if people are struggling with it then it is "that hard", but I feel like there isn't a solution that can satisfy all parties with this.
It seems that we either break the workflow, which makes a few of the aforementioned parties for a bit, keep bundling the old versions of these tools and leave potential security problems in there, or somehow forcing Apple to change their OS to be GPL3 compliant. For better or worse (probably worse), that last one isn't going to happen, so it feels like the lesser of the two evils is to stop bundling software that won't be getting future support.
The approval thing is valid, but I feel like that is likely going to be an issue that becomes more and more common as Apple moves to move more and more tools and utilities to the Xcode tools download anyway. Support/approval teams are just going to need to adjust to that reality, even if it’s an annoyance because I don’t see things shifting back, at least with Apple.
But issues of approval aside, I honestly think back of myself as a teenager, installing fink and Macports and figuring it all out as I went along and I wouldn’t have called myself a developer then. I was just a girl who wanted to tinker with stuff.
I think we (and I’m not talking about you specifically but the more general “we” as in the types of people that frequent HN) often underestimate the abilities of our non-dev brethren. We overestimate how much users know too, of course, but I do think we often slot people who have shown that they have the capabilities and desire to solve a problem into a “lesser” box just because they don’t carry the title of developer. And that affects those users too, who then assume they can’t do something when their own track record proves otherwise.
That's a reasonable thing to be annoyed about, but I also think that progress dictates breaking workflows occasionally.
Sure, I can install and deal with the burden of a package manager, but I don’t want to and didn’t use to need to for basic tools. I want Apple to do this.
If you’re installing your own shell making sure it doesn’t conflict with anything else, you might as well run Linux.
Personally, I'd like to know why Apple decided to ban GPLv3 software in macOS. It's not locked down enough to trip the TiVo clause[0]. My ongoing guess has been "just in case we need this in iOS", but these tools are all things that Apple would never ship in iOS anyway - the whole point of that OS is that you don't get access to any of those tools[1].
Alternatively, they might have just taken the Linus Torvalds tact of "this changes the deal", even if that change is minor. But that's not so much a license compatibility problem as much as it is a difference of opinion. Which is something that might never actually be resolved. Thanks to the v3 split, "GPL" now means two different licenses based on RMS's personal opinions on what restrictions best preserved software freedom at that time[2]. Even a hypothetical GPLv4 that dropped the TiVo clause for the sake of appeasing Linus and Apple would be changing the deal for people who wrote v3 programs expecting them to be a bulwark against locked-down devices.
In either case, it does not make sense for Apple to ship decades-old utilities that it refuses to update for legal or moral reasons. At best, they're dead weight - it is very much common practice for people developing on macOS to install the newer versions of those utilities themselves. At worst, they are an actual security hazard, as they aren't sandboxed in any way. Bugs in those utilities would be very much exploitable on macOS. So if we've decided "we're not going to update this ever", then we might as well get rid of it and replace it with something else.
[0] The TiVo clause in GPLv3 is actually not that strong, BTW. It requires that, if you ship the software as part of some hardware, that you provide a way for the owner of that hardware to modify the GPLv3 bits and have the hardware still work. The only thing that would remotely trip the clause on macOS would be SIP, and that's also owner-overridable.
You will lose iOS app support in the process on M1 Macs, but the TiVo clause doesn't cover that scenario - which is ironic because that's exactly how TiVo worked around similar language in GPLv2.
[1] Apps like iSH or a-shell do exist, but they run x86 or WASM binaries within the App Store sandbox. They also don't trip the GPLv3 TiVo clause.
[2] At one point, RMS was seriously open to the idea of just rolling the Affero clause directly into GPL, which probably spooked people way more
This is entirely speculation on my end, IANAL and I didn't work in that part of Apple.
It’s less true today than it used to be, but Mac was the best Unix developer box in the corporate world. A huge portion of their sales (especially high end sales where they make the most profit) relied on these community efforts.
I’ve moved over to using nix instead of homebrew and it’s so much better that it makes me wonder how it ever got this way. I think this is how.
I think the issue is that Apple isn't "one" thing. There are thousands of engineers, each with their own opinion, but also a thousand lawyers and corporate executives who are very concerned with how every action will make Apple "look" as a company. Sometimes their decisions made sense, sometimes they don't.
I'm assuming, of course, the idea was to let him spend at least some of his company time working on Homebrew.
Very few are using it, it's not getting updates, and is a potential security hole because of the lack of updates, it seems to me to be smart to get rid of it. Most people wouldn't want to use that ancient version of Emacs, and would install a newer one via Homebrew or GNU Emacs for OS X.
That being said, Python 2 will last through the end of RHEL 7's lifespan through June 2024, and that includes the AppStream module in RHEL 8 which will similarly end support in June 2024. After that customers are recommended to upgrade to a supported version of Python or continue using Python 2.7 at their own self-supported risk, we will not support them in that endeavor.
RHEL 9 (due out in a few months) does not ship Python 2.7.
https://access.redhat.com/support/policy/updates/rhel8-app-s...
Removing the system Python is prevented on RHEL/Fedora (with an exception). The package manager, DNF and YUM, are marked as protected which prevent their removal. You'd need to remove the protection file from /etc/dnf/protected.d before you can do that. Removing the system python would remove the package manager which depends on it, therefore the transaction is stopped. Using the RPM utility directly also will not work unless you a) pass --nodeps or b) list out every single dependency at the same time manually. You can test this in the CentOS containers with the following:
quay.io/centos/centos:7:
quay.io/centos/centos:stream8 quay.io/centos/centos:stream9 Best practice policy: use yum/dnf, not rpm for any package management.They know that at some point, their RHEL version runs out of extended support. Two years before that they’ll experiment with the new RHEL version on their test environment and notice everything is totally broken. And then they’ll have two years to fix it which is just barely enough time, as “fixing it” requires buy in and planning and resources from dozens of divisions and hundreds of people.
To enterprises, having Red Hat support some ass backwards old version of whatever package is not just “valued”, it’s exactly why they pay millions to Red Hat instead of running some free Linux version.
For anyone else who reads this, strings and str() went from being arrays of bytes in Python 2 to Unicode strings in Python 3. That change broke a lot of programs and is one of the most criticized Python 3 changes.
I know it shouldn’t, but the python community’s bizarre behavior during the 2.X -> 3.X move honestly made me think less of the language.
Tech is supposed to just be tech, but when the community behaves this badly about adopting improvements how can that not influence your decision to invest in that tech? If 2.X -> 3.X was such a drama fest what’s gonna happen next time they need to upgrade? Etc
I'd have gone for 'rigor mortis', but hey, we can't agree on everything ... like, say, a very minor change to string encoding with a 12-year managed rollout.
That's exactly the point: it required a massive amount of effort for relatively minor benefits.
What you say was bad about it? And who were the bad people specifically? The people who were using python 2 or python 3?
For what it's worth, python3 >= 3.0 && python <= 3.2 were hideously broken in their unicode support. Arguably had worse/unusable uncode relative to python 2.6 or 2.7.
So there was a huge failure to launch type of problem, especially given how long python3 had been development.
It most definitely left a very sour taste in many people's mouth that didn't start dissipating till 3.5 or 3.6 when enough "killer" features had accumulated.
Even then, for a lot of usages, python 2.7 'just works'.
There weren't that many people who were outright against Python 3, other than Zed Shaw for a few years. It just took until Python 3.4 for the language to really be usable in production, and after that it took a couple more years for every library to be updated. But the community has been pretty unified for 5+ years at this point.
The problem as I remember it was that there were some critical tools that just were not making the switch. The last one I could remember having to have a separate runtime for was graph-tool, but there were just a ton of them in the scientific computing community. Also, I don't think making print a function was worth it. So many people had muscle memory on that that personally I think it was worth just single casing it.
Weren't there a lot of major performance regressions in the first couple versions?
If it matters I've been writing Python professionally for more than 15 years and I started my career by seeing the (less famous) Zope2 to Zope3 botched migration. When Python3 was first announced I had hoped that the devs behind the project had learned from that related experience, guess I was wrong.
Sure, but they accomplished this mostly by displacing anyone who disagreed. That's fine, and Python 3 in a vacuum is a better language than 2. But I'd never make the mistake of choosing Python for a serious project going forward.
And I hope that python learned from 2 -> 3 and that in the future it will be better
For the vast majority of companies, python 3.x is not better enough for the cost. That is what happened. You may think it's awesome. It may be technically better it, etc. But the cost of moving was much more for most companies than the productivity gains. We've even measured it before. That is entirely on the python folks for not providing something people want enough to move to. All CSAT surveys of python i've seen at my company, and that my counterparts had ever seen, said literally the same thing. So it seems like something the community could have discovered pretty easily in a rigorous way if they wanted. I used to spend a lot of time on python-dev. It's a group of good people and engineers for sure. There was always good rigor around technical reasons to do something, and around performance benchmarking. But i don't remember seeing a lot of productivity research or CSAT or .... It's fair to point out that most OSS communities don't do that, but that's a bad thing, not a good one, and one of the things that often leads to community trouble
One of the other major factors, besides the productivity issue, that makes it not better enough is the migration cost. Software of this kind needs to be built to be migrated from and to. If not, it's broken and you deserve all the animosity you will surely get from your users. It doesn't matter if you screwed up the language, etc. Tough crap.
In python, for lots of folk with substantial stacks, the amount that could be auto-migrated by the tooling provided was low - 40-50%. Yes, simple stuff could be auto-migrated, but not most real stuff.
We actually ended up building our own tooling to get closer to 70% because how expensive the overall thing. It processed many millions of lines of python code, automatically kept track of which things would be best to rewrite/finish migrating first (because it would unblock other things), and continuously tried to auto-migrate any non-migrated changed code in a loop to maximize automation. The total engineer cost of moving was enormous. We are talking well over a thousand SWE years.
For what, exactly? As I said, we already measured (because plenty of new projects were python-3 only) and determined that all the improvements and hullaboo did not meaningfully improve productivity for people. Not enough. It was small, at best. It also didn't reduce the rate of production bugs, etc.
So all this work, for no meaningful value.
Because not enough were willing to do it, the result was to force it. Damn the users, full steam ahead. The users are just lazy.
I talked with lots of counterparts at other companies - most felt the same way. The result for us is lots of teams gave up python forever. The only reason it's growing at all is because of ML.
IMHO, python2->3 is an object lesson in doing it wrong and then victim blaming.
This was definately an opportunity for commercial software, i am not sure why companies worth hundreds of billions can't sort this out and expect a small nonprofit to provide this.
What the "larger user" (however you want to frame it) community would have preferred (IMHO) is much better migration tooling, even if it came at a cost of improvements.
The dev community chose to spend time on improvements instead. That's their choice, but it doesn't make the community wrong to be upset about it.
Why didn't you just take over support of python 2.7?
The migration was difficult because you had to actually think about this stuff instead of it working by accident and living as a landmine if you encountered non UTF-8 text.
The fact that you actually have to decode your bytes and specify an encoding makes it really obvious when you’ve just assumed the world is UTF-8 without really backing it up.
The print() change drives me nuts: many scripts use the old form, and Python forces you to change it despite recognizing the old functionality. They could've simply allowed print expr to work as is and accepted it as a wart.
The async stuff feels a bit like a bridge too far for me WRT the language. Simplicity would be standardizing on a function color (as it were).
I don't envy their position of preserving backwards compat while evolving the language.
Old print isn't an expr at all, that was a major point of the change.
They made print a function out of a misplaced sense of language "purity", where they made a random ideal and broke code to support it.
Let's compare to JS: 20 years of new features and old code still works. Keywords were added, and old code still works. Even code that uses those keywords still works. It took more effort from the language implementers, but that's as it should be: it took a bit more effort for a few teams (who make the engines), instead of a large amount of effort for tens of thousands of groups.
I work with code that uses a lot of files in a lot of different text encodings. Some are XML, some plain text, and some binary. Coming from other languages, Python 2's Unicode support was very difficult to work with, and my team was excited to move to Python 3, until we actually had to do the work.
Long story short, Python 3's str/bytes separation was a nightmare, especially when having to deal with third-party libraries that were expecting one or the other. This was especially true for libraries that came from Python 2, and were still expecting strs as function parameters when they should have switched to bytes. There was so much encode() and decode()ing going on that we occasionally caught ourselves getting them backwards in code reviews. We took a step back to see if we were just architecting things poorly, but no, we were doing things the best we could for our problem domain.
In the end, we traded one set of Python 2 text encoding footguns for a mostly different set of Python 3 text encoding footguns. It's not like Python 2 was great. Having a single str type instead of the str/bytes separation is worse in theory. But because Python 3 didn't design the separation of those types well enough, and still has a ton of text encoding footguns. It wasn't better enough to really justify all the work that went into the conversion.
In a different direction, I don't know what your problem domain is/was, but in general when I'm dealing with UTF8, I don't need to convert back to bytes very often. Was the need for conversion mostly due to the libraries that still expected strings instead of bytes?
I should also say that we were working with files in tons of encodings, not just UTF-8. We had UTF-16 and UTF-32, both little and big endian, with and without BOMs, but we also had S-JIS and a bunch of legacy 8-bit encodings. Often we wouldn't know what encoding a file was in, so we'd have to use the chardet library, along with some home-grown heuristics to guess.
Off the top of my head, the two biggest footguns are:
- There should be no way to read or write the contents of a file into a str without specifying an encoding. locale.getpreferredencoding() is a mistake. File operations should be on bytes only, or require an explicit encoding.
- .encode() and .decode() are very poorly named for what they do, and it wasn't that uncommon that someone would get them backwards. Sometimes, exceptions aren't even thrown for getting them wrong, you just get incorrect data.
Both of which were still issues with Python 2. There's a valid architectural argument to be had between the Python 2 way, where str was a bag of bytes, and the unicode type was for decoded bytes, and the Python 3 way, where the bytes type is your bag of bytes and str holds your decoded string. I favor Python 3's way of doing it, but it's almost six of one, half a dozen of the other. The advantages of one over the other are slight, and given how many library functions relied on the old behavior, it was probably a mistake to change it like that, rather than continuing the Python 2 way, and fixing issues like those above that caused problems.
I think the P3 string/byte ecosystem was made substantially weaker by P3 deciding not to lean more into types (something I have complained about on here before!). Like...they are the only values where the stdlib is extremely specific about you passing a value that has the exact right type, but the standard tools for tracking that are pretty poor.
Isn't that the point? String and bytes are different beasties. You can often encode strings to bytes and just about anything accepting bytes will accept it, but the converse is not true. Bytes are more permissive in that any sequence of any 0x00-0xff is acceptable, but str implies utf8 (not always guaranteed, I've seen some shit), meaning e.g. you can dump it to json without any escaping/base encoding.
I'm curious to know what you mean. How would you improve the design?
b = b"some bytes" s = u"some str" foo = b.to_string(encoding) bar = s.to_bytes(encoding)
or
foo = str(b, encoding) bar = bytes(s, encoding)
I also mentioned that I think there shouldn't be any mechanism to read or write files to/from a string without specifying an encoding. locale.getpreferredencoding() is a mistake.
But one thing I didn't mention that I think would improve the design would be to make it harder to treat str as the bag-of-bytes type it was before. Swift took some steps down this path, and honestly, it made it much less pleasant to work with strings, but a bit safer. I would hope that a better design could be found.
This may be controversial, but I don't think you should be able to just subscript a str type. instead of s[1], you should be writing s.codepoints[1] or s.graphimeclusters[1], etc. depending on what you actually want. If str is truly a string type and not a bag of bytes, it should deal with the extra complexities that being a string brings to the table.
Interesting thought on subscripting strings. It would be hard to make it fly after all these years of habits, but I see your reasoning.
But I disagree with you about the separation of bytes and string and the current state of the language. I write a lot of python that deals with bytes and text encoding, and now that all the libraries have caught up with 3, the situation is way better than it ever was. encoding, decoding, bytes manipulations are way less prone to errors.
And, as I said, I do prefer the str/bytes split over the unicode/str split, but I also wasn't doing a lot of raw byte manipulation. I agree that it was harder to do with the old str than bytes. I was mostly doing string operations on the grapheme cluster level, and then writing everything out as UTF-8, so I didn't see as much of the benefit.
Reading your comment feels like "I don't like Python for being Python", more or less. Apologies if I misread.
I'm also an advocate for using the right tool for the job, and in this case, Python may not have been the right tool for this job, but this was only one component in a much larger system. Sometimes you have to be suboptimal locally to be optimal globally.
And it's not like Python couldn't handle it, it was just that it had some design decisions that made things a bit harder than it would have been in some other languages. We got it working pretty well in Python 2, then the Python 3 transition happened, and it was a lot of work to get everything working as it had been, for only a small benefit to our team, but we got it working in 3 as well, and to my knowledge, it's still humming along.
For me it was more than "just the tech". Python 3.x was not the Python 3000 I was promised for a long time: many of the additions did not warrant breaking changes (as demonstrated by the backports) and many of Python's deeper warts still remain unaddressed, even today.
I think the deeper issue is that at the rate that 2.x got popular, the original motivation, principles and process behind the language's development got diluted. The orchestration of the push for 3.x adoption was not technically motivated in a convincing manner and signaled a change in the process. I stopped trusting the process.
A lot of the scripts were supporting processes that had a finite lifespan during the initial stages when the company infrastructure was being built out, rather than ongoing operational processes that needed to be performed indefinitely, so hopefully they'll be able to set a lot of that codebase on fire instead of maintaining or porting it to python 3.
We've seen an entire major version release of MacOS come out since Python themselves went "look, we're killing this thing off, deal with it", at which point Apple should have dealt with it, but didn't.
This is a replacement for the convenience of their own staff working on the OS, nothing more. It's good that it's finally happened, but it's a good five years overdue, and there's no motivation beyond "we no longer use it" at play here.
It is the OS design that doesn't make it possible to support the principle of least privilege that is the security liability. Blaming users for wanting their working software to just keep working the same way is abuse!
If they want to run a piece of code written in HyperCard for the original Macintosh, they should be able to do so. This push towards ever more complexity while failing to fix the inbuilt security fault inherited from Unix is insane.
Why can't an OS keep a rogue program from taking down your machine? You could specify what to run, and with what resources, and then even if Satan himself wrote the code, it wouldn't ever corrupt anything other than the resources specified.
Firstly, a circuit breaker does nothing but protect the wiring in your house. A faulty lightswitch can still throw sparks and start a fire as long as it never draws more than 10A or whatever your circuit breaker is rated for.
Similarly, operating systems can prevent a rogue process from taking too much CPU or RAM, but can't prevent a process from ransoming your files, at least not without a substantially different sandboxing regime to what exists in mainstream operating systems today. Why we don't/can't have this is an interesting disucssion that is much larger than the python2/3 question.
Meanwhile, OSes like NixOS can easily deal with different versions of the same package (with security backports) ... Apple seems lightyears behind.
We all knew this was coming, but doing it in a point release seems like exceedingly bad form, particularly as Apple does major releases annually. I never expect point releases to break stuff, at least not on purpose.
If they need to do it for a year, it's not that hard.
Public webserver? Unlikely for sure.
Office fileserver? Absolutely.
Build server? Sure.
Now, what is the percentage of that among all MacBooks on latest MacOS out there.
Monthly, for me. Or more often if, like last week, a new customer required it.
If you stop paying attention to the niches, they add up to a lot of people.
It's okay for that stuff to break. (I wish Apple would put more effort than they do into making it break less often, but that's neither here nor there.) But, it should break in a major release, when the user is prepared for things to break. It should not break in an update that was installed automatically overnight.
If a company as 'small' as (pre-IBM) Red Hat can afford to support Python 2.7, then so can Apple:
* https://developers.redhat.com/blog/2018/11/14/python-in-rhel...
* https://access.redhat.com/solutions/4455511
Further, if Debian has the resources to support Python 2.x, then so does multi-trillion dollar company of Apple:
* https://packages.debian.org/search?keywords=python2
If they wanted to announce at the June 2022 WWDC that macOS 13 wasn't going to have it, that's fine. But we're several months into macOS 12.x and such a large change is bad form.
How long is long enough to avoid breaking something the users depend on? One more year? Five years? Fifty? I think we can all agree that it's reasonable to drop support at some point, making it just a question of when and at what cost.
They could drop it with v12, or wait for v13, both would be fine. Dropping it in the middle of release cycle what people are objecting to.
10 years - a piece of software compiled with 'supported' libraries, SDKs and what have you should function for 10 years. An appliance or a service should function for 10 years and have spare parts avaliable for that long.
That is the guideline I ise as a consumer, if I can see that something won't last a decade, I won't be spending money or time on it if i can avoid it. At that point it's no longer a durable good, it's a consumable.
I mean I already think it's... interesting that an operating system comes with (multiple) programming language interpreters, available to the end-user.
Apple is pretty aggressive about breaking backwards compatibility, as they did with x86, so it's no real surprise for unmaintained software to become completely unusable.
My company was shipping (somewhat niche) software linking with the built-in python2 up until last year though, and it’s been a scramble since Monterey to rebuild things to avoid the warning message. Definitely thought we would have python2 until 13.0 though (and part of me wonders whether this is just a “brown-out” for the beta and 12.3 release will be back to normal)
Yeah, fuck those end-users who might want to automate things with a script.
I am not to familiar with MacOS, but I imagine it comes with a bash/sh terminal? If so, then that is something that should be first choice for automation.
Everything else, be it python, perl, ruby or whatever should be installed by a user as needed.
Part of it is that it's just a mental burden to keep this kind of compatibility information in my brain. I have VMs set up of past macOS releases in case I need to run something old. I don't have VMs set up for earlier versions of a release, and I don't even know where I'd get an installer. Admittedly, in this case it would just be a matter of installing Python, but I don't necessarily know that as a user, or the software could need to be linked to Cocoa or some such.
But I really did think that since it was in Monterrey, it was going to stick around until at least macOS 13.
NOTE: Debian 11 (bullseye) has removed the "python" package and the '/usr/bin/python' symlink due to the deprecation of Python 2.
[0] https://wiki.debian.org/Python#Python_in_Debian
* https://packages.debian.org/bullseye/python2
There has been no dropping of support: if you want Python 2.7 you can install in.
Very HN: Business || technical. No concession for "Customer service."
You can't please everyone
macOS is going the same way, except that Apple doesn't have a package manager so it has to be a manual installation.
There is no need for a manual installation.
Just to quote the literal slogan of homebrew:
>The Missing Package Manager for macOS (or Linux)
I think it is fair to say that Apple does in fact not have a package manager. Unless they do bundle homebrew with their OS nowadays?
> Apple doesn't have a package manager so it has to be a manual installation.
The quoted statement is clearly false. To maintain that python has to be installed by hand is incorrect.
Apple the corporation doesn’t have a package manager.
MacOS, the ecosystem does.
No different from Linux. There is no Linux package manager. There are numerous open source package managers for Linux.
macOS the OS has a package manager in terms of a tool to install packages. It does not have the ability to find/download packages to install though.
In the sense of a package manager being able to download and install additional software, macOS has several choices available but none of them are included with the OS, or have the ability to update the OS.
There is arguably no single "Linux ecosystem". There are numerous Linux-based distributions, which are largely comparable to macOS as an OS. One of the very key differences between each family of distro's is the package manager the OS ships with by default and uses to update itself.
So no. macOS does not "have" a package manager that's comparable to those included by default with linux distributions, but there are several third party package managers available for it.
A red herring that is nothing to do with what we are discussing.
> It does not have the ability to find/download packages to install though.
Another false statement. Homebrew is just as capable of searching as any Linux package manager.
I wasn't talking about homebrew.
It helps if you understand the comment before you reply to it and make accusations.
Edit to add, because fuck the HN timeouts. Heaven forbid someone make 6 comments in two fucking hours:
Try reading the whole fucking comment again, with the points being made really hammered home for you:
> macOS the OS has a package manager in terms of a tool to install packages. It does not have the ability to find/download packages to install though.
This is about the built in macOS package manager, that the OS itself uses to install packages. It's invoked via `installer` on the command line, or Installer.app in the GUI. It does not expose the ability to find/fetch packages from remote URLs.
> In the sense of a package manager being able to download and install additional software, macOS has several choices available but none of them are included with the OS, or have the ability to update the OS.
This is referring to homebrew, macports, etc.
We certainly do see things differently, because only one of us is repeatedly calling the other a liar due to poor reading comprehension.
This whole line of conversation is about whether it’s necessary to manually install python.
If you claim it is you are lying, because it can easily be installed via homebrew, and you are clearly aware of that.
If you have some need to claim that homebrew isn’t a MacOS package manager or that MacOS doesn’t “have” homebrew, because Apple doesn’t ship it. Go right ahead, but it’s a red herring.
Is faker.js a macOS vulnerability because there were people using Macs who installed npm and then installed faker.js? This isn't a theoretical question—whenever you install something, you need to consider where it's coming from and whether they can be trusted.
MacOS has homebrew as an open source package manager.
On Debian, packages in the official repositories are considered to be literally part of the Debian project. The maintainers are considered Debian maintainers, and bugs go into the Debian bug tracker.
This matters due to the trust issues I alluded to above. It also means that on Debian, there is one definitive version of Python (or at least one per Python version). The Homebrew, MacPorts, and Python.org Python binaries are all slightly different, such that it's possible for software to work with one but not the other.
However none of this has any relevance to the claim in question, which is that python must be installed manually.
MacOS certainly has a package manager that can install and maintain python.
Brew has a long history of treating any criticism of their truly woeful approach to security, dependency management etc, with "well fuck you we dont want to hear your opinion".
Everyone switched to brew even though it was the most poorly designed because it came from the Rails hype era where everyone wanted all their tools to be written in Ruby and didn’t care about anything else.
This meant if you wanted to install, say, ImageMagick, you would spend one day compiling stuff.
Also contributing a brew recipe was (and probably still is) easier than contributing to macports, and brew casks are pretty convenient.
I deeply dislike some of the choices of homebrew, but I can understand why it was popular, and it wasn't just because of the ruby hype.
Edit: And it hardly came out of the blue. Python 2.7 was 10 years old then and people had known they needed to get off it for years.
Old software isn't inherently more risky.
I'll never understand why people think they can write some software and it'll stay pristine forever. Even if it was had no known bugs at some point in the past, the world is a harsh place and your program, its dependencies, its programming language or even its OS can be found to be subtly broken at any moment. Even if it is not outright buggy, the world will change around you until the interfaces you depend on are no longer there. (looking at you, software program at $WORK that only works with PS/2 keyboards but not USB keyboards)
The combination (old Python, new shared library) could have vulnerabilities that neither (old python, old shared library) nor (new python, new shared library) has.
There is plenty of software being written to very high standards of quality (formally verified real-time control software for rockets/power plants/etc comes to mind), but most of that is not available for free on the internet (or for $5/month as a SAAS) because you get what you pay for.
Even TLA+ analysis and proof of correctness doesn't stop things like "new zookeeper clients are pushing a lot of new writes => disk space needed for logs increases => across all the nodes in the zookeeper cluster => the entire formally correct cluster keeps falling over now but it didn't last week"
I meant more in the sense of the proofs in TAOCP - these are programs that don't interact with anything outside of the program itself, no non-specified input, so a lot simpler, no dates, no actual machine memory, just a little synthetic world defined by the language itself. Even TeX is imagined to be unchanging after Knuth's passing, and is basically unchanged since the 70s.
For sure the world has moved on, but we have lost the ability to have confidence in the software and have had to develop tools and ways of thinking inspired by biology.
The general approach of most projects and vendors at the bottom of the stack to not accept full responsibility for their part is striking. The amount of lying, faking, leaky abstractions, adapting but not 100% etc. is so immense it is a wonder we get something done at the higher levels of the stack at all. I cannot comprehend, how it is possible, to hammer the RAM in such a way, that you can affect neighboring cells charge until the information interpreted based on the amount of charge in the cells is changed. This should never ever be possible. I haven't seen any such disclaimer this was possible on the package or in the data sheet of any RAM module. The same is with changing the kind of magnetic recording in the same hard drive model without notification (or change in model name that would be more honest). And then there are "subtle" problems, like coil whine on mainboards or PSUs for no reason - we know how to fix this. These high pitched sounds can give you headaches without you hearing much or anything. There is no indication about it on the package and almost no review tests this. And the whole Meltdown and Spectre type of bugs is just the cherry on top with major performance regressions because of it. So how are you supposed to write reliable software on top of all this if just a tight loop might actually change the state of hardware in such a way that is becomes a problem? I mean, this is just crazy.
For security critical work, like web services, it makes more sense to constantly upgrade to the newest version. But for scientific work, which is a huge audience of python users, who often use obscure poorly supported libraries there is a lot of value in a stable foundation.
This has been the Python 3 experience for years. If you had lots of previously ignored bugs in your Python 2 around string handling, it took longer but on a clean codebase it was often only a matter of minutes.
What’s old now is what was bleeding edge years ago - it hasn’t been updated (otherwise it wouldn’t be old), it has just gotten older.
And the problem with keeping old versions around is that when (not if) someone does discover a vulnerability, what are you going to do then?
If your answer is “we’ll patch it then”, then that’s not going to be an old version anymore, is it?
If your answer is “I guarantee it will 100% never have any vulnerabilities ever and so it will never need to be updated”, I don’t think we’re living in the same reality.
* One of the libraries that old code depends on has a vulnerability discovered. They provide a patch, but it's only for the current version rather than the one used 10 years ago, which means that it's not as simple as just rebuilding Python 2.
* A new attack on a cryptographic component, network protocol, etc. is developed and suddenly defaults previously considered safe are no longer considered safe. As a great example, how many old things broke when major sites and services like PyPI or GitHub deprecated SSLv3, TLS 1.0, etc.?
* One of the Python libraries you use has had a vulnerability which was only recently discovered. The maintainer quite reasonably does not provide unpaid support for free software which they updated years ago.
What all of those have in common is that the problem is less the vulnerability than the fact that you are likely to need significant amounts of unrelated work getting other components updated to the point where you can install the patch. Given how often vulnerabilities are discovered which have been present for many years, this is likely to happen for any sufficiently large code base.
1. The OS and hardware under it can change and lead to new vulnerabilities.
2. Dependencies of Software A can change, which can lead to new vulnerabilities in Software A.
3. It's easier to find more and more vulnerabilities on a target that does no longer move. And once they are found there will be fewer experts to detect/fix/report it as time goes on.
What type of vulnerability with it just sitting there as an executable do you expect?
And that's not to say those users won't have to find something new eventually (or migrate the software to a contained environment). It just shouldn't happen in an automatically installed update.
His point doesn't necessitate waiting. You could just classify it as a major release instead. That is not to say it should be a major release necessarily.
Removing support in the middle of 2020 would have been very unpopular.
Now that most places have vaccines, it probably doesn't make sense to wait for the next major release because this has already been long delayed.
Other factors could have also been at play - perhaps Apple Inc had a lot of internal tools on 2.X and has only recently gotten around to migrating all of those.
You make an assumption that the way your system and method of working is the same as everyone else on the planet. It is not.
I have two tools that I use monthly which require python 2.7. The people who made the tools never updated them to work with 3. For this reason, I have a work box that I cannot upgrade.
I know that Macs are aimed at end users, and not developers, but it really is getting harder and harder for me to use my Macs for the kind of work I do. Each major software update borks my virtual hosts. PHP has been removed, and because of code signing requirements, adding it back is so complicated it takes an entire day and filtering through a dozen half-baked online tutorials. No, the ones that say "Just use brew!" are not the answer unless you have a very simple job. I don't use Docker, but from what I read on HN, that's six miles of bad road, too. But as I understand the situation, that's Docker's fault, not Apple's.
The patient says, "It hurts when I do this."
The doctor says, "Don't do that."
Now? I honestly can’t think of a way to release without adding something like 100-200 MB to the app bundle, consisting of an entire Python installation. With a user-facing app it is not ideal to try to say something like “just download Python and tell me where it is”.
See also the breaking change of 12.3 kernel extensions with Dropbox and Microsoft OneDrive:
* https://www.macrumors.com/2022/01/25/macos-12-3-cloud-storag...
* https://help.dropbox.com/installs-integrations/desktop/macos...
Apple OS naming isn't semver. They really have two OS major releases a year, Spring and Fall trains.
Brew, docker, no idea if they have working release on the site, but that’s another possible way.
(I'm not defending the use of Python 2. I'm arguing that, as a general rule, if something takes the vendor X time, users should be given X + Y time.)
Apple probably has to either give up strict semantic versioning for breaking changes, give up their strict release schedule, or be willing to defer announced features that don't make one major version all the way to the next. Something needs to give. Looks like they've decided to go with a "semantic-lite" where breaking changes and major features aren't the rule with point releases but can still happen in old enough/deprecated areas.
It does seem more needless though in cases like this granted, unlike other examples it's not really clear why they'd do this now rather than 12.0. Not as if Python 2.7 was a spring chicken last year either, support ended in 2020 and that was like 11 years after release already? Heck, why not 11.0?
If Apple isn't going to follow any types of rules, then what's even the point?
As far as I understand, Semantic Versioning is not a synonym for "versioning scheme", but refers to a specific scheme (https://semver.org). So I don't understand how Apple having any versioning scheme implies that it must be semver.
It's okay if Apple's "minor releases" occasionally have breaking changes. But I don't think "wait until the next major release to remove an entire language runtime which has been built-in for 15+ years" is too much to ask.
That is perhaps still true in some large companies.
Meaning A and B bumps could be breaking, but C is still just for bugfixes.
Sure it's not a great system from the technical side of things, but it shouldn't be shocking that Apple puts marketing first.
I don't think introducing breaking changes in 2020 (with the world turned upside down) for 11.0 would have been a good idea. The same reasoning applies to 12.0, many countries were going through a Delta wave without having much vaccine availability this past summer.
I think Apple has been very reasonable about this - they've been clear that 2.x is going away, they have been extremely accommodating of changing circumstances, and 2.x can still be installed manually after this change.
Bad form? Maybe. Good riddance? Yes.
Let's say I have a little utility shell script I wrote ten years ago. I basically haven't touched it ever since. Inside the script is one line of python for fetching the current time with more precision than `date`, or something, which I forgot about a long time ago.
When I install a major update, I test all my scripts to make sure they still work. I do not test after every point release, and I shouldn't have to!
It's not every point release - it's just this one, and it's because this change was (very reasonably) delayed from 2020.
2.X can still be installed manually, so you have that workaround.
This is why docker and other hermetically sealed packaging and languages (golang) systems work by avoiding the OS implementations which must change and update as quickly as possible.
The fact anyone would rely exclusively macOS level implementations and somehow expect them to remain stable and backwards compatible for any stretch of time blows my mind.
The fact it worked so well for so many years is a test that it’s needed and that it should be rewritten. All software should be kept up to date.
The same way software might have security issues and fixed, it can have other issues that need fixing.
While I'm not a fan of removing Python 2 mid major cycle, it's not without precedent, and they've had popups since the first Monterey release anytime anything linked against the system Python 2
Apple doesn't claim to follow semantic versioning, which is where your expectation comes from.
https://developer.apple.com/documentation/macos-release-note...
It would seem the gods eventually answered!
OT: it's the same precari that is in precarious, which originally meant something more like "dependent on the will of another". Both definitions have loosened nicely, but it's still fun to see how we got here, and to think of marking something Deprecated as an act of piety!
Now, it's included in the CLI tools. You know how when you run 'clang' or any of those, it's a wrapper that offers to download the real tool for you? That's how the 'python3' binary works now.
python --version
Python 2.7.18
python3 --version
xcode-select: note: no developer tools were found at '/Applications/Xcode.app', requesting install. Choose an option in the dialog to download the command line developer tools.
Python 3.8.9
macOS 12.2 (Intel), Xcode 13.2.1, no homebrew, no macports
Come on. ><
How is it going to be when that is removed? "python" will throw error and only "python3" will be available? "python" will run Python 3?
EDIT: According to one of the commenters xattr is now a binary instead of a Python script, so that's probably the real reason why they got rid of 2.7 now!
[0]: It consists of bash, zsh, php, ruby, python (v2), perl, osascript on my machine
python3 prompts to install command line developer tools
Bundling a dead technology that users couldn't remove or update made using actual normal python borderline impossible without being forced into a third party software manager.