Not that this has ever been a showstopper whenever this comes up, but just to put some chips on the table, I personally have no interest generally in CPython development on 3.X, but I would pledge some help wherever I could in a potential 2.8 release.
> It is a release goal for Ubuntu 14.04 LTS to have only Python 3 on the desktop CD images. Also, we won't be allowing Python 2 on the Ubuntu touch images.
... Why would you do that? Why is there an either/or choice? It's like if typing "python" suddenly started running ruby. They're different languages, they should have different names on the command line.
Possibly typing "python" should be deprecated, should issue a deprecation warning before starting Python2. It's also a pity Python2 and Python3 source files aren't easily distinguished and will generally produce obscure errors if run using the wrong interpreter.
OS X (at least the version on the Macbook I got from IT) also ships with a horribly outdated Bash, so I don’t think it healthy to put it up as a standard for anything…
Is there a particular reason for that? I can understand not wanting to ship GPLv3 code if it is embedded sufficiently deeply into the hardware, but it shouldn’t be a problem to ship GPLv3 Bash, as the user can change the binary delivered on the harddrive at any time?
I'm curious what specific issue Apple has with GPLv3 for something like bash? It seems to me that if they're providing the source, and not modifying the copyright/license, there would be no issue. The patent language only applies to the program itself. I guess one could view it as simple paranoia on the part of their lawyers...they took years to start shipping any GPL (v2) code on the Mac. It's actually news to me that it has bash out of the box, now; last time I used Mac OS X, it still had tcsh and an abominable development environment and I hated it.
Torvalds didn't put the "or any later version" clause in the Kernel's GPL license. And he hasn't required copyright assignments. Because of this, the kernel simply can't be converted to GPLv3. You would never be able to contact all the individual copyright holders and have them agree with complete consensus to re-licensing.
"I think it's insane to require people to make their private signing keys available, for example. I wouldn't do it. So I don't think the GPL v3 conversion is going to happen for the kernel, since I personally don't want to convert any of my
code."
He's spoken out against it personally - he cleary _doesn't like it_.
That quote referred to an interpretation of the initial public draft of GPLv3, which was radically different from the final version published in June 2007, including with respect to the so-called anti-Tivoization provisions. Of an interim draft of GPLv3 in March 2007, fairly close to what ended up being final, Linus said "Unlike earlier drafts, at least it does not sully the good name of the GPL".
However, you're correct that Linus clearly does not like even the released version of GPLv3.
> Likewise Torvalds refuses to let GPLv3 touch the kernel.
Please don't misrepresent other peoples view points out of context.
If you want to read Linux Torvalds opinion, I suggest that you start with either one of the many articles about it (like http://news.cnet.com/Torvalds-No-GPL-3-for-Linux/2100-7344_3...), or any of the many mails on the mailing list. You will notices that Torvalds criticism is mostly directed at the DRM provisions, since he don't think a copyright license should be able to dictate a political matter such as DRM restrictions on devices.
To quote:
"I also don't necessarily like DRM myself," Torvalds wrote.
"But...I'm an 'Oppenheimer,' and I refuse to play politics with Linux,
and I think you can use Linux for whatever you want to--which very much
includes things I don't necessarily personally approve of."
If Apple ships any OS X devices with DRM, they would have to have a separate build without the gplv3 bash in it. It would double support costs, maintenance and so on.
And patents. Any patents covering bash would loose value from using a GPLv3 bash. This also include any patent agreement Apple has, which include additional legal overhead.
GPLv2 vs GPLv3 is and always will be Patents and DRM.
Yes, it absolutely would. Particularly since that would mean that packages for 3.x would be available then; something that's not there now. If you want to migrate to Python 3, you have to re-compile any libraries you want to use.
Yes, I need a lot of my code to 'just work' out of the box on OSX, so I'm sticking with 2.7. I really wish Apple had put python3 in Mavericks, as /usr/bin/python3, rather than the default. sigh
I used Python 3 for the first time a few days ago (I've been programming in Python for 18 years). When I used python heavily (I've switched back to C++ and now Go) I depended a lot on a small number of really good libraries- ElementTree, NumPy, SciPy, etc. Unless/until all of those get ported to Python 3 (along with hundreds of other programs), and those ports are considered Correct (in the QA validation sense), it's hard for me to consider wholesale conversion.
I did it because I was trying to parse Unicode in XML (Apple iTunes music files) and Python 2 is completely broken when it comes to unicode.
I consider Python 3 a big "fuck you" from Guido to the community. I don't think he intended it to be so, but the effect of the long transition, and the lack of backported features in Python 2 (which could be easily accomodated), coupled with only limited adoption of Python3, demonstrate the leadership needs to pay closer attention to user pain.
Finally, I don't think Python will ever address the simple and most important core issue: lacking good multithreading support will make the language continue to be more and more marginal. CPUs aren't getting much faster, the CPython serial runtime isn't getting any faster. Multiple cores on a single chip, and multiple threads in a core, are getting more numerous- and not being able to take advantage of them using the obvious, straightforward memory-sharing, multi-threaded techniques the chips are design to run well- is just a clear ignorance of the future of computing designs.
I switched back from Python to C++ when shared_ptr and unique_ptr, along with std::map and std::vector became standard enough that programming in C++ became easy: convenient memory management, and maps/vectors are the two things that C++ adds to C that are valuable. Then I switched to Go because C++ compilation times are so bad, and writing high quality concurrent code is hard.
To demonstrate the vehemence which long-term experience programmers view the lack of ability with which Guido is running the python3 project.
If he had actually addressed the important questions in Python3 (first being getting the majority of people on 3 quickly by getting the major libraries ported quickly, second being addressing the GIl in either 2 or 3 (if the GIL had been removed in 3, I would have much more strongly considered it).
If either of those had been addressed, some of us (who are influential in the area) would have adopted 3 and prosetlyized it. However, I lost so much confidence in Python after the 2 to 3 transition that I've decided to actively non-prosetylize it.
Numpy supports Python 3 since version 1.5, released in August 2010. Ubuntu 12.04 and the current Debian stable carry this version. Scipy supports Python 3 since version 0.9, apparently released in 2010 as well with releases in the current Debian stable and Ubuntu 12.04.
‘Just now’ is hence a bit too late, but it is correct that many users have only been able to use Python 3 some four to five years after 2008.
I believe those versions were saddled with "experimental" wording and bugs, which made people steer away from them. This is all second-hand though, I personally never used them.
A lot of people cite that lack of NumPy support is what's keeping them from moving to Python 3, I've seen this comment more times than I can count. It seems most of them haven't done their research and found that NumPy/SciPy has had complete support for Python 3 for 4 years now.
Please don;t interpret my comments above as ignorance. I am quite aware of what is available (and was available, when I considered switching to python3). I did my research at the time (which was some 5 years ago). It looks like scipy was added about 3-4 years ago... at which point I don't care any more.
If it was valid at the time but your concerns have since been addressed, then your complaints aren't exactly relevant to someone who's reading this now and looking for information on Python 3, right?
Just out of curiosity, have you actually tried getting NumPy/SciPy working on Python 3? Because they've been fully supported for quite a while now. I don't understand how this myth keep perpetuating itself.
I'm not interested in even playing any more. Like I said, I haven't used Python 3 before today (after 18 years of programming Python). I would have cared, for example if Python 3 had any impact in the real world. Anyway, it took years for numpy support; it wasn't available until recently for python3. That's the problem: it took years before these common tools were proted.
I don't regard the lack of practical multithreading as a problem. We routinely use multiprocessing and IPC via queues for parallelism, and for me the main limitation is Python's excessive memory use: even atomic data types are objects which cause quite a bit of overhead (a memory efficient dictionary with support for a few basic data types would be great). Thankfully everything still fits in the server's memory, otherwise I would have to consider another language, such as Go.
Behind the scenes it uses multiprocessing and/or threading plus queues etc. I have a function that adds command line arguments (number of workers, use threads or processes, debug mode) and then another that returns an executor given the command line arguments.
The debug mode forces a single thread and is intended for when you want to run pdb as multiple threads/processes make things considerably more complicated.
I've been programming with import Queue, import threading, for years now (probably 8 or 9). I don't want to drop all my completely fine multithreading code for Yet Another Threading API.
Personally I think multiprocessing is just dopey. Why force me to program around two different address spaces when I can just have one?
I'd also done the Queue/threading thing for the longest time. I'd also use multiprocessing some of the time. But it was arbitrary, and I'd have to make sure that shutdown, exceptions etc are correctly handled.
> Yet Another Threading API
concurrent.futures is not yet another threading api. futures is a standard async work/result wrapper, while the process and thread executors are standard executor apis.
> Personally I think multiprocessing is just dopey
Then don't use it. But be aware that it works well for many people. Some of my code scales linearly with multiprocessing and all I had to do was use a process executor instead of a thread executor.
Well I do care. I think multiprocessing is a stupid abomination- it's like a person who never learned how to use threads found the threading library and decided to make a message-passing communication system with a thread-compatible concurrency library.
Message passing is fine (and it's the sort of thing that Go uses) but ultimately, leaving single-address-space multithreading completely on the table when it's the single most obvious and straightforward way to take advantage of modern processors is just dumb.
Fine, so you have found a language you prefer to Python. Go use it. Never mind about PyPy, which gives you compiled execution speeds, never mind about the fact that Python 3 fully supports NumPy, SciPy, matplotlib and much of the infrastructure you claim to need. If threading is your preferred solution to concurrent programming and it's working for you then there's no real problem, is there? I'm sorry Python doesn't work for you, but fortunately there's a rich toolset available of which Python is only a part. It's fine for many tasks, but it's by no means your only option. Isn't open source wonderful?
Python 3 came from a good place, and it definitely fixes many problems that sorely needed fixing, but it was doomed to failure from the start (and many developers said that in 2008 already).
For all intents and purposes, Python 3 is pretty much a new, separate programming language. Sure, it's very close to Python 2.x, but you don't have out of the box retro-compatibility so that pretty much kills it right there.
Python 2.x is so huge in terms of its use around the world and available libraries and resources for it that you just can't say "hey, the new version of Python will in practice break everything" and expect it to fly.
I love Python and the community around it (and have several packages on pypi myself), but Python 3 is a joke.
If we didn't want to kid ourselves, we'd kill Python 3 and back port the interesting features, like Alex suggests. At this point though, too much effort and egos are involved to make this realistic.
> but you don't have out of the box retro-compatibility so that pretty much kills it right there.
I don't know if that's necessarily a foregone conclusion. Scala releases aren't always/usually backwards-compatible, and you'd be hard-pressed to find people still using much older versions.
That's probably related to the user base of scale being orders of magnitude smaller. The network externalities of a large language user base give it incredible inertia that makes it very difficult to change.
> If we didn't want to kid ourselves, we'd kill Python 3 ...
Using Python both personally and professionally, I do not think it is necessary. I already came across a few 3.X features, that I miss in 2.X and I write most 2.X code "in preparation" for the jump. But that's it, it is a jump, still.
The most interesting part of the post is the feedback matter, which seems to be accidental, but isn't since an open source language is shaped by the people - something you can forget, when you are not a contributor (which most of the people might not be).
You're fine. 98% of PyPI downloads are not. That's a startling statistic, and argues that something IS necessary, rather than the status quo of shaming the top 1000 Python packages sans 3K support and telling people "it's really not that bad."
It is that bad, in terms of adoption of the first 5 years of 3K's release. And unless we want to spend the next decade trying to get to 50% mindshare with 3K, the community needs to suck it up and change tacks.
The definition of insanity is trying the same solution twice and expecting different results.
Perl 6 is vaporware. Python 3 is here, it's widely supported, it's faster than Python 2.7, and while Guido didn't do us any favors with his migration strategy, I don't see what the big deal is. Python 3.3 makes writing code that is compatible with both almost trivial.
People aren't using Python 3 enough because it's not the default in Debian/Ubuntu. That's about to change with Ubuntu 14.04. I expect that to tip the scales.
I have Ruby and Perl tattoos and now work at a Python (2.7) shop, so I have all of the horses in this race, but is there a significant difference between a language that's vaporware[1] and one that nobody uses?
The previous things I've seen also indicated that Python 3 _isn't_ widely supported, though I guess Django finally got support recently, and that it _wasn't_ faster than 2.7. Is this just old information on my part, or is it different for certain kinds of workloads?
> it's not the default in Debian/Ubuntu.
Yeah, that should have a large impact, for sure.
1: I have seen lots of "check out this Perl 6 code" blog posts, but I don't know to what degree it's ACTUALLY vaporware.
> but is there a significant difference between a language that's vaporware[1] and one that nobody uses?
There's a significant difference between a language for which a complete implementation does not exist such that you cannot use it, and one for which an implementation exists but few people currently choose to use it because there is a closely-related predecessor language that still has a stronger ecosystem.
As an stage in the development of a language out of an existing, widely-used predecessor, the "vaporware" stage tends to precede the "available but not popular due to predecessors ecosystem advantage" stage, which tends to precede the "displaces predecessor" stage.
Its obviously possible for a language to stop progressing at any of the earlier stages before reaching the last stage.
Well, if they are programming languages, are still unstable 10 years late, have near zero adoption and are missing half of the promised features, then yes.
Unfortunately languages like Rust perfectly fit your definition. Because Rakudo was started around the same time as Rust and Rust is aiming for may be 1% of the feature set Perl 6 aims to achieve.
That's moving the goalposts a bit! What's currently called "Rakudo" is at least the sixth or seventh attempt at an implementation, not counting Pugs, viv, v6, or Niecza.
Rust appeared publicly as a project to build a language in 2010 (it was a personal, private, project kept under wraps by Graydon a few years before which is irrelevant).
Perl 6 was first started on 2000. I've been reading Larry's apocryphal descriptions of its "features to be" for 4 times more years than Rust exists.
Rakudo is just a particular attempt at Perl 6, not the first and neither it consists of first time the language was announced publicly.
I remember Larry also saying(On a youtube video, may be at the O'reilly conference) that he has been working on Perl 6 since 1987. That's how he wishes to describe it.
Therefore I'm not sure what you've been reading, or if you are even reading them. Because if you would- you would know, Rakudo covers much of the Perl 6 specification.
By the way. Rust is still not complete. The wikipedia article says, work started in 2006- Which makes it 8 years and still incomplete. And this is for a project which has by far may be even 1000x modest goals. Python 3 was itself 10 years in development, and that is for small modifications to print statement and iterators. And even after 5 years since that date, it doesn't seem to have come any where closer to achieving good production scenario adoption.
And. These are- I said by far extremely modest goals compared to the Perl 6 project.
>I remember Larry also saying(On a youtube video, may be at the O'reilly conference) that he has been working on Perl 6 since 1987. That's how he wishes to describe it.
That's all well, but he announced it circa 2000. I don't care how he feels about it or how long he hacked it alone, I care since when the language was expected.
Therefore I'm not sure what you've been reading, or if you are even reading them. Because if you would- you would know, Rakudo covers much of the Perl 6 specification.
In a half-arsed form and with 1/100 the community of Perl doing anything with them. And still not all of them.
Personally I stopped caring somewhere around 2006. And I've read all of Larry's "apocalypses" back when they used to be on Oreillynet, as well as followed the internal implementation politics for a few years.
>By the way. Rust is still not complete. The wikipedia article says, work started in 2006- Which makes it 8 years and still incomplete.
No, it says that it started as a "part-time side project in 2006". That could be 2 weeks total spend in those years writing a list of desired features in a napkin and getting a hello world compiled for all I know.
I only care about the time since the project was publicly announced and the community started working on it.
If Rust had been hyped as much as Perl6 I'd call it vaporware too. It may yet turn out to be vaporware (in that it's not done), but right now I'm willing to give it the benefit of the doubt that they will actually deliver on their promises.
Perl 6 and Duke Nukem Forever are the geriatric poster children of vaporware, and always will be, even after they finally shipped. Being able to pick up a turd floating in a urinal does not in any way deliver on all the broken promises, rehabilitate the ruined reputation, or make up for all the blustering hype that was spewed for so many years about how superior Duke Nukem Forever's gameplay would be to all the other games that were already on the market. The damage was done a long time ago, and can never be undone. And Perl 6 is perfectly analogous to Duke Nukem Forever. Fortunately, nobody ever made such ridiculous claims about Python 3 as were made about Duke Nukem Forever and Perl 6.
> People aren't using Python 3 enough because it's not the default in Debian/Ubuntu. That's about to change with Ubuntu 14.04. I expect that to tip the scales.
I doubt it will tip that much. The enterprise customers we develop for still ship on CentOS and have auditing departments that have to scour each library and package a system depends on. Adding a dependency hurts in this environment. These institutions only upgrade their interpreters once every 5 - 8 years if you're lucky.
Comparing Perl 6 with Python 3 is a big joke. Even a cursory look at the Perl 6 specification and implementations like rakudo will tell you Perl 6 is attempting to go to a place where Python may like likely go in the next 40 years, with 10 Python 3 like disasters.
Its an exceptionally ambitiously project which they are in no hurry in finishing. And that is for a good reason.
Its an exceptionally ambitiously project which they are in no hurry in finishing. And that is for a good reason.
Way back in the early days, the goal was to release a usable 6.0 in a couple of years. The "there's no hurry" line is a post hoc justification from around 2006 or so.
But has the goal of Perl 6 as a project remained static? I'm relatively new to the Perl community. I know you are around since a very long time.
But I think the goal has changed from "Let's fix Perl 5" to being a more bigger goal of developing a language what Perl 5 would likely look after say 3 decades of iterations. I can't say if that is good from a purely practical perspective. But it does sound like worthwhile goal to chase.
Lets take a look at Python 3 itself. Yes they finished the project, but 5 years after that- Now what? Users don't seem to a have a valid reason to move away from 2.x series for merely small time incremental improvements, while forced to migrate bulk of the infrastructure and code for nearly ~0% productivity gains.
Perl 6 could have come out by 2005-2006, and if it were to be merely a incremental non-backwards compatible change over p5. We would be having similar posts about p6 as we are having now about Python 3. Imagine a refined, yet not that beneficial p5 breaking CPAN. I am sure you wouldn't like such a situation.
I suspect but can't entirely prove that the appearance, rapid ascent, and even more rapid burnout of Pugs made a lot of people realize that this would be a long slog. Even so, if you look back at mailing list messages or conference talks or blog posts in 2006, 2007, 2008, whenever, you'll see that the party line has always been "It's only a year or two away."
You're right that back then the expectations were a lot higher, but I don't think anyone even then expected it to replace Perl 5 that fast.
Also, Perl 6 turned out rather quickly to become a new language (although very much in the spirit of earlier Perls), while Python 3 was, as far as I can see (I'm not really a Python person) as an update to fix some important issues in the language, which despite by necessity being backwards incompatible, was intended to still be the same language.
You're right that back then the expectations were a lot higher, but I don't think anyone even then expected it to replace Perl 5 that fast.
The goal was to run existing Perl code in the same process, so that you could gradually adopt new features or use existing libraries with new code. If that had worked out, there'd have been less of the Python 2/3 gap.
Then again, the idea that Perl and P6 are different languages is something that's come up after the fact, after it became obvious that P6 wouldn't be ready for general use any time soon.
I don't share Alex's concern. The migration to Python 3.X is a slow, but in my opinion sure process. Already many of my small internal programs run on Python 3.4, and I believe that in 1-2 years from now I'll be writing most new Django client projects in Python 3.4 (hopefully running on Pypy3).
Agreed. More and more important libraries are being ported to Python 3. I'm working on a web app supporting Python 3.3. I keep it backward-compatible with Python 2.7 just in case I would badly need a library that hasn't yet been ported to Python 3 but it hasn't happened yet and I have good hope that we'll be able to deploy it on Python 3.3. I develop and test locally with Python 3.3 and travis runs my tests against both Python 3.3 and Python 2.7. Every time the Python 2 build breaks it's because of some silly unicode-related problem that has been fixed in Python 3, so it really motivates me to develop with Python 3 from now on.
I was even a bit surprized when he complained that people don't develop for Django using Python 3. There isn't even a year that Django supports it (with a year of "we support, but don't recommend"), and quite a few of its libraries already got ported.
Yep, people underestimated the time to migration. So what? I completely agree that migration is happening. Now, let's talk about IPv6...
Python3 is not exciting because well there is nothing to be excited about.
Consider me an average programmer, I have been using python for a year+ now. Most of the everyday stuff can be done in 2.7, some functionality I need / can't do I google and get a solution which works in 2.7. Why Py3 is not adopted is because there is not much benefit you get for doing the extra work (think chemistry - activation energy)
On another note, why can't we port it the little goodness back to 2.7 ?
> Python3 is not exciting because well there is nothing to be excited about.
I think this depends on whether or not you use, or will ever need to use, Unicode. I don't, and so Python3 is not only not exciting, but it creates a new problem for me: suddenly, I need to think about the difference between bytes and strings. Now maybe this is healthy for me, maybe it builds character. But it also adds effort and work that bring me no benefit. I'd like to see some of the smaller improvements of Python3--I'd like print to be a function, but in the big picture I'm better off with Python2.
I wonder if the Unicode problem could be solved in a way that I wouldn't need to think about it: perhaps utf-8 everywhere somehow.
Perhaps you misread my intended meaning, or I didn't state my position well. I don't have a Unicode problem. There is one (suppose HN forum software were written in Python2, for example). Python3's approach to solving an important problem that I don't have makes solving problems that I do have awkward. So I stick with Python2. I have to believe that there are people with both problems.
Static platforms are great for developers. The best years of WebObjects' life were after Apple had mothballed it. Returning to a project years later - all the old code, scripts, and patterns worked just the same. Nothing else in the java world was like that. Similar story with BSD. The python 2/3 migration has been well managed. There is no rush. Celebrate it.
I think this is exactly right. Often people rush to conclusions in favor of the new and shiny, but Python is a great language that works pretty darn well at the moment, even in v2.7, giving the community the freedom to fully vet their ideas and make a valuable product.
I remember reading commentary on HN when a new version of PuTTY was released, and there was general sentiment that PuTTY is - for all intents and purposes - a "finished" product in that it does what you need it to do.
Python v2.7 is similar in that it's a tool that does many amazing things and is very stable. Could it be improved? Sure! But for the most part it's a great piece of software.
The care and planning that went into creating the tool is also going into the upgrade process, and I think that's a good thing.
This is really a great point. Python 2 is stable and probably never going to die off. This implies that it can be stably and productively used without worrying about the language developers breaking it down the road. This is great. :-)
Python 3 is a huge distraction. It's hard to get people to move away from languages like java while there's a fragmented python world. If only we could just forget py3 the python community would be a better, more newbie friendly place. Every time I hire someone I have to explain why python 2 and why not "the latest"
I have been very impressed with 2to3, the amount of work it does is pretty impressive. But as somebody who tends and prefers to work in python 3 but often needs my scripts to run in python 2, I have no choice but write those in python 2. I can see the same dilemma for somebody who writes an open source library and hope for as much usage as possible.
No, I actually agree with a commenter on the page: what you need is a small compatibility layer to run 2.x code on 3.x, which can be slow and emit loads of warnings, and then stop making 2.x available AT ALL.
What 3to2 and six really do is keeping the 2.x runtime alive, and as long as the runtime sticks around (and is patched, and features are backported etc etc etc), people will keep using it.
That wouldn't solve the main issue, which is the [non-negative] amount of work in porting existing Py2 codebases.
Besides, 3to2 (which does exist but isn't maintained) and 2to3 are both bad ideas. They miss edge cases, even common ones, and they add even more complexity to the build/test/install process which people already find hard.
There is no meaningful performance increase to go to a backwards incompatible version? Three letters: DOA. if not performance then at least we'd need some crazy new feature like good multi threading or perhaps running on a new relevant platform (say ios or android). Otherwise we will be 2.X forever.
Actually, there are some minor performance increases, but there are tons of minor improvements, and the new async framework landing in 3.4 is very exciting.
It's only when you start doing unicode seriously that you see how broken python2 is. It is not unworkable - but it's a mess that is hard to resolve, and keeps biting you in old code paths that weren't tested with character from this subset rather than that subset.
So, Python 3's killer feature is reasonable Unicode support.
The problem is that reasonable unicode support is a non-issue in many areas were python had a strong competitive advantage over other languages (e.g. data analysis, scientific python).
My last few Python projects have started out as Python 3, but ended up as 2 due to missing library support.
Would it be at all feasible to enable Python 3 to import Python 2 code? I imagine this could be done without making Python 3 fully backwards compatible, but I might be wrong.
That'll be the crux. When you can run most/all existing python 2 code under python 3 we'll see the transition. Until then most of us have no plans to even look at python 3.
+1, but I'm ont sure how that's going to work. Even if you support Py2 syntax, some of the internals are different and might cause code to misbehave, especially the str/unicode/utf stuff.
I like to think of engineering as "solving problems within a system of constraints". In the physical world, engineering constraints are things like the amount of load a beam will bear. One of the primary easily-overlooked constraints in the software world is backwards compatibility or migration paths.
There are many examples of systems where many look at them today and say: "This is terrible, I could design a better/less-complicated system with the same functionality in a day". Some examples of this dear to my heart are HTML, OpenID, and SPDY. It's important to recognize the reason these systems succeeded is they sacrificed features, good ideas, and sometimes even making sense to provide the most critical piece: compatibility with the existing world or a migration plan that works piecemeal. Because without such a story, even the most perfect jewel is difficult to adopt.
The OP, about Python 3, is right on except for when it claims making Python 3 parallel installable with 2 was a mistake; doing that would make it even more impossible to migrate to 3 (unless the single binary was able to execute Python 2 code). (Also related: how Arch screwed up Python 3 even more: https://groups.google.com/d/topic/arch-linux/qWr1HHkv83U/dis... )
Exactly. This is why something like Java 8 is admirable. They've been able to introduce two major new features from newer "productive" languages – lambdas and traits – without breaking anything; not only that, old libraries will actually enjoy better APIs thanks to the new features without recompilation. Sure, there might be better ways to implement these features, but introducing them in a way that feels natural to a language with millions of professional developers without breaking source or binary compatibility is a commendable achievement.
Same with C#. That language has really exploded with new features, and still code and precompiled libraries written for 1.2 compiles and runs perfectly on 5.0.
HN loves to dump on C++, and in some senses I understand why, but Stroustrup had this figured out over 30 years ago. You don't break compatibility. For all the ugly warts in C++ due to the requirement for compatibility in C, I can take a C code base from the 80s (say a BLAS library or something), use it with my C++11 code, even using things like std::vector, std::string, or shared_ptr, without any real fuss. The idea that somebody will adopt a language very close to another language that they are already using is dubious at best. And that is what Python 3 is - a different language quite close to another perfectly good language. Why take on all the difficulties and heartache when I can just continue using this perfectly good language?
To be fair, Stroustrup made the wrong call on void* implicit casting. At least IME, it's the primary source of breakage when trying to incorporate C code into a C++ program --- but then again, we have extern linkage for a reason.
This seems like a reasonable perspective, but I wonder how much will change when Python 3 starts shipping with Linux distributions (and probably OS X eventually).
It won't change anything for the shop that has a million LOC, but it might start to budge that 2% number.
I don't know how one would do 1mil LOC in python. At 40K I'm having enough annoyance refactoring things.
The python heads say you need to write code to be readable.. ok so how do I read all the call sites of this method or function? An unforgivable contradiction on large code bases, I'm afraid.
It wasn't until 3.3 that py3 was really palatable. Easier to support unicode running the same codebase in py2 and py3. yield from -- look, a py3 feature worth porting for! 3.3 was released in late 2012, and so, we can probably shift this "5 year" expectation to start from there.
In fact, it's 3.4 that really starts to wet the beak with asyncio and enum. I'm not sure 2.8 needs to happen, if 3.x simply, and finally, has good reasons to get on board.
This. While there are many wonderful features in Py3 (I use it 100% in production at work) it still needs a killer feature to really inspire people to move over. Python's lack of solid asynchronous I/O is definitely one of those features I think.
I feel frustrated too, but I think Ubuntu 14.04 will tip the scales (it ships with Python 3 by default).
Also, the core devs got at least some things right with Python 3.3 by making it a lot easier to write code that targets 2.7 and 3.3 at the same time. In retrospect, that should have been the focus much sooner.
Windows doesn't ship with an OS-default version of Python and doesn't include OS features that depend on the OS-default version of python, so its not as influential in that regard. But, in that regard, the inclusion of the multiple-install-compatible python launcher in the basic Windows install of Python 3.3 probably improves the Python 3.x story on Windows, since it eases the multiple install pain and makes it easier to work with Python 3.x while having some Python 2.x software in use on Windows.
I also am sceptical that this will tip the scales, but for different reasons: you'll be able to install Python 2.7 with a single command. Plus, switching to Python 3 will still be difficult because many libraries still won't have been ported yet.
It's fascinating to compare this with ruby 1.9, released around the same time, but seemingly with a slightly better cost/benefit ratio, having nice new features and also significantly improved performance, and with ruby 1.8 being deprecated with a lot more speed and force. It got everyone to actually make the switch, and then ruby 2.0 came along, largely compatible and with a more improvements, and now ruby 2.1 seems to be an even smoother upgrade from 2.0.
The ability of the ruby core team to manage not just the technical aspect of making the language better, but smooth the transition in a way that actually succeeded in bringing the community (and even alternate ruby implementations) along with them, hasn't been given nearly enough credit. You could analogize it to Apple with OS 9 -> OS 10.9, versus Microsoft with people still running XP
Well, i mean, there were backwards incompat changes in ruby 2.0, which I think is after he said there woudln't be. They were fairly minor and easily dealt with, but there was certainly some software that had to be adjusted for ruby 2.0 from 1.9.3.
i feel ruby's syntax is mostly done. an addition here or there, but breaking shall be rare as it just feels so finished. performance wise i think it can use (and gets) a lot of love.
on the python side things are different. python seems quite a bit faster then ruby. same league, but faster. yet its syntax is what needs to be greatly improved upon as it is so full of surprises and dark corners.
a language's syntax it's like its API, and performance is an implementation "detail". breaking the API is much more painful then chaning the implementation -- therefor i prefer ruby's approach: begin slow but with a quite stable syntax.
p.s. i dont intend to hurt feelings with this comment
I've seen breaking changes between Ruby 1.8.6 and 1.8.7, so I'm really suspicious of this claim. Start with which version of Ruby are breaking changes not to be expected until 3.0?
Ruby doesn't follow SemVer, so expecting no breaking changes between 1.8.6 and 1.8.7 wouldn't necessarily be correct. I don't remember what the policy was at the time.
> You could analogize it to Apple with OS 9 -> OS 10.9, versus Microsoft with people still running XP
No, you couldn't. The difference in upgrade rates between Windows and OS X is primarily due to their differing customer bases. Windows is very popular in enterprise, which avoids unnecessary upgrades in order to ensure compatibility with in-house software.
OS X, however, has almost no presence in enterprise, and consumers don't mind upgrades nearly as much, since the consumer software they use has to be compatible with all OS versions. Also, you can't buy Macs with old versions of OS X, whereas Microsoft makes it trivial to buy a new machine and install an older version of Windows.
Yes, Macs have a decent presence in enterprises these days--although it's easy to overstate it; Mac overall market share is still pretty low. Somewhere in the 10% percent range I believe. Furthermore. and to the original point, a lot of those Macs are BYOD or otherwise not managed as a corporate desktop/laptop. (Where I work is a case in point. You see a fair number of Macs but IT doesn't formally support them.)
According to this site[0], it's at 7.54%. Out of that, only 32% are running the latest version of OS X (10.9). Another 24.5% are running 10.8. So almost half of OS X users are at least 2 versions behind. Not nearly as up-to-date as you might think.
Well, it's a good thing then that you can't make a direct comparison between the two OSes. Apple doesn't care about developers, quickly deprecating APIs. Therefore, targeting an Apple OS that's 1.5 years old is much more tedious than targeting a Microsoft OS that's 1.5 years old.
My last two jobs were at fortune 100 companies, and both had IT supported macs and a lot of them. A work with a lot of people from other large enterprises and the same is true. This is anecdotal, but more evidence than the guy claiming they don't have a presence
As well as the enterprise, don't forget the non-technical home market, and non-technical small businesses. Windows is much more common there, and they also don't have a short upgrade cycle.
I just want to chime in and say that Apple has done an abysmal job with backward compatibility, and it's not just due to the enterprise vs consumer market. I would wager that if you looked at the total list of apps ever released for Mac OS, the majority of them would not run today. That's because Apple's primary strategies are 1) innovate and 2) put the user first. Putting the developer first is not part of their profit motive like it would be for say Oracle or MathWorks. So without constant recurring effort, developers and the apps they create get left behind.
Apple frequently deprecates APIs that they endorsed just a few years ago. And they claim that apps can be rewritten to work with new APIs in a few hours, when in reality it can take weeks, months or even longer due to refactoring issues. If you want to be an Apple developer, you will likely be rewriting a portion of your code at some point to run on a new OS release. I was optimistic that the practice might end but history is already repeating itself with Apple insisting on iOS 7 compliance. The kicker is that Apple could have ported a lightweight version of Mac OS to run directly on arm for iOS, but they didn't, and I disagree with it being a performance issue (iPads are orders of magnitude more powerful than NeXT machines, or even the early PowerPCs that ran OS X). They created a vast array of nearly duplicate code prefixed with UI instead of NS. This looks like more of an anachronism every day to me with tablets running at multiple GHz with GBs of ram, when the only major difference between desktop and mobile is the touch interface.
Contrast this with Microsoft, where I am finding very old examples designed for Windows XP that still run today. Now Microsoft is certainly burning its developers with the major breaks in various versions of DirectX, or subtle differences in APIs between desktop/mobile/Xbox, but in my opinion this isn't happening nearly to the extent that it is with Apple.
> Contrast this with Microsoft, where I am finding very old examples designed for Windows XP that still run today.
xp? Up until a couple years ago when I switched to 64bit windows (first computer I had with more than 4gm ram) I could still run win3.1 and dos programs in (32-bit) windows 7. Including dos programs designed for the very original ibm pc (1981!).
Even today with 64bit windows I can still run most windows 95 programs.
I'm glad they didn't try too hard in porting MacOS to iOS. The touch/mobile platform is significantly different than the desktop platform. It required a huge rethink in design of the API and the UI. Is the Windows Desktop API similar to the Windows Phone API?
But were the changes between 1.8 and 1.9 in Ruby as significant as 2 to 3 to Python? Or even 1.8 to 2.0?
I know in Ruby 1.8.x to 1.9, some major changes in my day-to-day coding involved Hash (elimination of hash-rocket, and ordered key-value pairs by default)...but I can't think of anything off hand that required me to rewrite my own libraries. And between 1.9 and 2.0...I've been switching between machines that have 1.9.3x and 2.0x and can't even tell a difference. Obviously, I'm not doing any legacy-maintenance in this situation, but it seemed that Python's changes, while breaking, were also significant improvements and changes to the API that mandated changes in implementation?
Speaking as a non-Python-dev...I've been wanting to get more into Python, at least to write routines that take advantage of scipy and numpy and all that goodness...but the process of deciding between 2.x and 3.x and keeping the steps/compromises in order can be beguiling.
> but I can't think of anything off hand that required me to rewrite my own libraries.
There were a lot of gems that weren't 1.9 compatible for quite a while, so there must have been something that caused people to rewrite libraries. I think that one disadvantage for Python 2->3 was an advantage Python had over Ruby -- the bigger base of libraries, many of which were widely used but basically in very-low-effort maintenance mode.
1.8 to 1.9 was a HUGE release in terms of breaking changes. Probably the most notable change was the introduction of proper unicode support.
Unlike Python, which changed syntax at the same time, Ruby tried to maintain compatibility with existing syntax. In practice, this allowed Ruby libraries (including Rails; I did the bulk of the encoding work for Ruby 1.9 in Rails 3) to do runtime introspection to support 1.8 and 1.9 at the same time.
In my view, the best, most underrated thing that Matz did in Ruby 1.9 was to make all of the breaking changes detectable and shimmable at runtime.
Ah yes, it's fitting that I would forget about the Unicode change...the fact that I did so is just further affirmation of why I still get myself into encoding problems...all the time. But yes, the Unicode support did seem to be a frequent reason for library overhauls.
With Python, one must take a conscious effort to write code that is both 2.x and 3.x, but it is entirely possible (also with some detection/shimming at runtime). Some of that is easier if you stick to 2.7 and 3.3+ only, because then you can do things like "from future import print_function", use b"..." and u"..." literals etc.
Rename it as something else. Call it Cobra or something. Also remove all backward compatibility features. Maybe by taking away it's association with python, it will have a better chance.
I like Python 3. I prefer it. It is better to program in than 2.x. Iterators everywhere, no more unicode/encoding vagueness, sub-generators and more. It is a much better language and it's hard to see how it could have evolved without a clean break from its roots.
However it has been interesting to follow over the last five years. It has been a sort of, "what if p5 broke the CPAN," scenario played out in real-life. Breaking compatibility with your greatest resource has a painful trade-off: users.
Everything I work on is not even considering a migration to Python 3. OpenStack? A tonne of Django applications that depend on Python 2-only libraries? A slew of automation, monitoring and system administration code that hasn't been touched since it was written? Enterprise customers who run on CentOS in highly restrictive environments? A migration to Python 3 is unfathomable.
However my workstation's primary Python is 3. All of my personal stuff is written in 3. I try to make everything I contribute to Python 3 compatible. I've been doing that for a long time. Still no hope that I will be working on Python 3 at my day job.
Sad state of affairs and a cautionary tale: "Never break the CPAN."
But some things are lacking. Django is there, and it works great, but you hit a dead end with some libraries (like MySQL - even though it seems there are alternatives).
So in the end you don't make the move because you're afraid of needing something that may not be there.
Yeah, but one of the issues is that some very popular packages for Python were already development-dead for years.
PIL is a good example; the last actual PIL release was in 2009. We now have a great fork that works (Pillow), but PIL is not the only dead package that's still in active use. There are quite a few others, too.
This is one of the issues I see that need to be corrected. I have attempts at migrating to 3, that failed miserably. One happened to be while teaching my son Python with "Python for Kids" that ended up making the experience difficult and the others that ended up causing conflicts in various OS dependencies.
I have to say, Python is my favorite language still (at least 2.7), and I hope the community can get it back in line. I can see JS, NodeJS and Python being my core technologies, but this issue with 3 is squashing my dream.
Had a discussion at work a few weeks ago about this. Roughly, it came down to this - if you need any backwards compatibility whatsoever (Do you ever expect your code to ever run on machines with operating system releases younger than 2013ish OR do you ever plan to interop with code older than 2013ish), Python 2 is the way to go. Otherwise, feel free to use Python 3.
This effectively means that only for purely greenfield projects for machines under your team's control and possibly even new teams can Py3 be used confidently. It's a shame - there are a number of nifty cleanups in Py3, but I simply don't see a compelling reason to use Py3 over Py2. This is particularly true given PyPy being fast and Py2. Given the triple of Python/Perl/Ruby for a greenfield project, I'd probably explore Ruby; I haven't been jaded by it yet, unlike Perl or Python.
> Had a discussion at work a few weeks ago about this. Roughly, it came down to this - if you need any backwards compatibility whatsoever (Do you ever expect your code to ever run on machines with operating system releases younger than 2013ish OR do you ever plan to interop with code older than 2013ish)
Python 3 runs on operating systems older than 2013 and, while it might not be the OS default, there is no reason a non-library project can't bundle its own interpreter in the distribution and use that instead of the OS default interpreter.
> Python 3 runs on operating systems older than 2013 and, while it might not be the OS default, there is no reason a non-library project can't bundle its own interpreter in the distribution and use that instead of the OS default interpreter.
You are absolutely right, but the overall cost of deployment skyrockets at that point. Not to mention pernicious bugs from grabbing the wrong library & whatever other weirdness happens. Unless the project is reasonably significant (deployment cost is insignificant to development cost), it's much easier just to work from the system python.
Projects 'of a given size', yes. But there are small projects, projects for Windows, projects where the fleet is managed and deploying full venvs isn't done... etc.
The niceities of greenfield work and what ought to be done start breaking down as you get into heterogeneous systems where total system stability (lack of change ) is a big deal, and these fleets have been running for a decade or so. Imagine having to figure out deployments to a Python 2.4 environment in 2012 (I had this problem).
To misquote a famous statement, you fight the battle with the troops you have, not the troops you want, against the enemy you have, not the enemy you want. The reality is that it's often (usually?) not worth it to ship full venvs with Py3. It's often much better to ship Py 2.7 code (just your own code) and install your own libraries in the system library location.
Yeah, unfortunately CentOS users don't think this way. It's a serious question whether Py3 will _ever_ become the default in server distributions. I think we'll see more significant adoption once that happens.
This seems like it depends on what kinds of OS you regularly use. On Linux the deployment situation isn't quite that dire. Python3 has been in Debian stable since Debian 6 (aka Squeeze, released February 2011), so running Python3 code isn't a problem if the machine is even close to up to date. On Ubuntu the situation is even reversed: Python3 is default, and starting in 2014 it will be the only supported version, with Python2 support being dropped. It will still be installable out of "universe", but the goal is to drop Python2 from "main" by the release of 14.04 LTS in 2014, because LTS releases promise 5 years of support, and they don't want to be supporting Python2 into 2019.
Iterators everywhere are incredibly annoying, especially with my development workflow, where I don't put a line of code into a file before I run it manually in the interpreter. When I run a map over a list, I just want to see the freaking results.
Default unicode strings are obscenely annoying to me. Almost all of my code deals with binary data, parsing complex data structures, etc. The only "human readable" strings in my code are logs. Why the hell should I worry about text encoding before sending a string into a TCP socket...
The fact that the combination of words "encode" and "UTF8" appear in my code, and str.encode('hex') is no longer available, is a very good representation of why I hate Python 3.
In Python 2, the rule of thumb was "If it makes sense, it's going to work". In Python 3, this isn't true. Not often, but often enough to annoy. This makes Python "feel" like Java to me.
And worst of all, Python 3 has so many excellent features, like sub-generators, a better performing GIL, etc. These sometimes force me into coding with Python 3. I hate it every freaking time.
I said to myself that with Python 3.4, due to great stuff like the new asyncio module, I'll have to make the switch. It's really sad that this is because I "have to" do it, and not because I "want to".
this post is a great example of why python 3 didn't go far enough. it's too close to python 2 to be still called python and too far from python 2 to be still called python 2.
personally, coming from a country that needs to deal with non-ascii, i love unicode by default and there are b'' strings if you need them. str.encode is a non-issue - you wasted more words on it than the two-line function enc() it takes to fix it.
i have a program in python 2 that uses this approach, because i have a lot of decoding from utf and encoding to a different charset to do. python 3 is absolutely the same for me.
If all your code is dealing with binary data, why the heck are you using strings for it? There's a bytes type there for a reason, which doesn't deal with encodings, and you won't accidentally try and treat a blob like a string.
> Default unicode strings are obscenely annoying to me. Almost all of my code deals with binary data, parsing complex data structures, etc. The only "human readable" strings in my code are logs. Why the hell should I worry about text encoding before sending a string into a TCP socket...
If your Python 3 code is dealing with binary data, you would use byte strings and you would never have to call encode or touch UTF-8 before passing the byte string to a TCP socket.
What you're saying about Unicode scares me. If you haven't already, please read The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) before writing any libraries that I might depend on.
> you would use byte strings and you would never have to call encode or touch UTF-8 before passing the byte string to a TCP socket.
I'll start by adding that it's also incredibly annoying to declare each string to be a byte string, if this wasn't clear from my original rant.
Additionally, your advice is broken. Take a look at this example, with the shelve module (from standard library).
s=shelve.open('/tmp/a')
s[b'key']=1
Results in:
AttributeError: 'bytes' object has no attribute 'encode'
So in this case, my byte string can't be used as a key here, apparently. Of course, a string is expected, and this isn't really a string. My use case was trying to use a binary representation of a hash as the key here. What's more natural than that. Could easily do that in Python 2. Not so easy now.
I can find endless examples for this, so your advice about "just using byte strings" is invalid. Conversions are inevitable. And this annoys me.
> What you're saying about Unicode scares me.
Yeah, I know full well what you're scared of. If I'm designing everything from scratch, using Unicode properly is easy. This, however, is not the case when implementing existing protocols, or reading file formats that don't use Unicode. That's where things begin being annoying when your strings are no longer strings.
> Additionally, your advice is broken. Take a look at this example, with the shelve module (from standard library).
His advice was sound, and referred to your example of TCP stream data (which is binary). Your example regards the shelve library.
> So in this case, my byte string can't be used as a key here, apparently.
shelve requires strings as a keys. This is documented, though not particularly clearly.
> so your advice about "just using byte strings" is invalid.
Let me attempt to rephrase his advice. Use bytestrings where your data is a string of bytes. Use strings where your data is human-readable text. Covert to and from bytestrings when serializing to something like network or storage.
> Conversions are inevitable.
Absolutely, because bytestrings and (text)strings are two different types.
> And this annoys me.
There is no real alternative though, because there is no way to automatically convert between the two. Python 2 made many assumptions, and these were often invalid and led to bugs. Python 3 does not; in places where it does not have the required information, you must provide it.
> when implementing existing protocols, or reading file formats that don't use Unicode.
I'm afraid it's still the same. A protocol that uses unicode requires you to code something like "decode('utf-8')" (if UTF-8 is what it uses), one that does not requires "decode('whatever-it-uses-instead')". If it isn't clear what encoding the file format or protocol stores textual data in, then that's a bug with the file format or protocol, not Python. Regardless though, Python doesn't know (and can't know) what encoding the file or protocol uses.
Well I don't think I will be able to change your opinion if you feel so strongly about it. However some pointers perhaps might make it less painful:
> Iterators everywhere are incredibly annoying, especially with my development workflow, where I don't put a line of code into a file before I run it manually in the interpreter. When I run a map over a list, I just want to see the freaking results.
That's what list() is for:
>>> list(map(lambda x: x * x, xs))
> Default unicode strings are obscenely annoying to me. Almost all of my code deals with binary data, parsing complex data structures, etc. The only "human readable" strings in my code are logs. Why the hell should I worry about text encoding before sending a string into a TCP socket...
> The fact that the combination of words "encode" and "UTF8" appear in my code, and str.encode('hex') is no longer available, is a very good representation of why I hate Python 3.
I'm afraid I don't understand your complaint. If you're parsing binary data then Python 3 is clearly superior than Python 2:
>>> "Hello, Gådel".encode("utf-8")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in range(128)
Because they're not the same thing. Python 2 would implicitly "promote" a bytestring (the default literal) to a unicode object so long as it contained ASCII bytes. Of course this gets really tiresome and leads to Python 2's, "unicode dance." Armin seems to prefer it to the extra leg-work for correct unicode handling in Python 3 [0] however I think the trade-off is worth it and that pain will fade when the wider world catches up.
> In Python 2, the rule of thumb was "If it makes sense, it's going to work". In Python 3, this isn't true. Not often, but often enough to annoy. This makes Python "feel" like Java to me.
Perhaps this is because you're a used to Python 2 and anything else is going to challenge your perceptions of the way it should work?
I don't understand the Java comparison.
> And worst of all, Python 3 has so many excellent features, like sub-generators, a better performing GIL, etc. These sometimes force me into coding with Python 3. I hate it every freaking time.
> I said to myself that with Python 3.4, due to great stuff like the new asyncio module, I'll have to make the switch. It's really sad that this is because I "have to" do it, and not because I "want to".
Well you can always attempt to backport these things into your own modules and keep Python 2 alive.
I think going whole-hog and embracing Python 3 is probably easier in the long run. I'm not sure how long the world is going to tolerate ASCII as the default text encoding given that its prevalence has largely been an artifact of opportunity. Unicode will eventually supplant it I have no doubt. It's good to be ahead of the curve.
> Why the hell should I worry about text encoding before sending a string into a TCP socket...
A string represents a snippet of human readable text and is not merely an array of bytes in a sane world. Thus is it fine & sane to have to encode a string before sticking it into a socket, as sockets are used to transfer bytes from point a to b, not text.
Not arguing that you're wrong, but Unix/Linux is not a sane world by your definition. Whether we like it or not (I do like it), this is the world many of us live in. Python3 adds a burden in this world where none existed in Python2. In exchange, there is good Unicode support, but not everyone uses that. I can't help but wonder if good Unicode support could have been added in a way that preserved Python2 convenience with Unix strings.
(Please note that I'm not making any statement as to what's appropriate to send down a TCP socket.)
ASCII by default is only an accident of history. It's going to be a slow, painful process but all human-readable text is going to be Unicode at some point. For historical reasons you'll still have to encode a vector of bytes full of character information to send it down the pipe but there's no reason why we shouldn't be explicit about it.
The pain is painful [in Python 3] primarily for library authors and only at the extremities. If you author your libraries properly your users won't even notice the difference. And in the end as more protocols and operating systems adopt better encodings for Unicode support that pain will fade (I'm looking at you, surrogateescape).
It's better to be ahead of the curve on this transition so that users of the language and our libraries won't get stuck with it. Python 2 made users have to think (or forget) about Unicode (and get it wrong every time... the shear amount of work I've put into fixing codebases that mixed bytes and unicode objects without thinking about it made me a lot of money but cost me a few years of my life I'm sure).
I was careful to say "Unix strings", not "ASCII". A Unix string contains no nul byte, but that's about the only rule. It's certainly not necessarily human-readable.
I don't think a programming language can take the position that an OS needs to "adopt better encodings". Python must live in the environment that the OS actually provides. It's probably a vain hope that Unix strings will vanish in anything less than decades (if ever), given the ubiquity of Unix-like systems and their 40 years of history.
I understand that Python2 does not handle Unicode well. I point out that Python3 does not handle Unix strings well. It would be good to have both.
This is the first time I encounter the idiom Unix strings. I'll map it to array of bytes in my table of idioms.
> I don't think a programming language can take the position that an OS needs to "adopt better encodings".
I do think that programming languages should take a position on things, including but not limited to how data is represented and interpreted in itself. A language is expected to provide some abstractions, and whether a string is an array of bytes or an array of characters is a consideration of a language designer, who will end up designing a language takes one or another of the sides available.
Python has taken the side of language user: enabled Unicode names, defaulted to Unicode strings, defaulted to classes being subclasses of the 'object' class... Unix has taken the side of machine (which was the side at the time of Unix's inception.
> [...] probably a vain hope that Unix strings will vanish [...]
If only we wait for them to vanish, doing nothing to improve.
> Python must live in the environment that the OS actually provides.
Yes, Python must indeed live in the OS' environment. Regardless, one need not be a farmer because they live among all farmers, need they?
> This is the first time I encounter the idiom Unix strings
The usual idiom is C-strings, but I wanted to emphasize the OS, not the language C.
>> [...] probably a vain hope that Unix strings will vanish [...]
>If only we wait for them to vanish, doing nothing to improve.
The article is about the lack of Python3 adoption. In my case, Python3's poor handling of Unix/C strings is friction. It sounds like you believe that Unix/C strings can be made to go away in the near future. I do not believe this. (I'm not even certain that it's a good idea.)
I do not insist that C strings must die, I insist that C strings are indeed arrays of bytes, and we cannot use them to represent text correctly at present. I fully support strings to be Unicode-by-default in Python, as most people will put text in between double quotes, not a bunch of bytes represented by textual characters.
I do not expect C or Unix interpretations of strings to change, but I believe that they must be considered low-level and require higher-level language user to explicitly request the compiler to interpret a piece of data in such fashion.
My first name is "Göktuğ". Honestly, which one of the following is rather desirable for me, do you think?
I'm not arguing against you. I just don't write any code that has to deal with people's names, so that's just not a problem that I face. I fully acknowledge that lack of Unicode is a big problem of Python2, but it's not my problem.
A Unix filename, on the other hand, might be any sort of C string. This sort of thing is all over Unix, not just filenames. (When I first ever installed Python3 at work back when 3.0 (3.1?) came out, one of the self tests failed when it tried to read an unkosher string in our /etc/passwd file.) When I code with Python2, or Perl, or C, or Emacs Lisp, I don't need to worry about these C strings. They just work.
My inquiry, somewhere up this thread, is whether or not it would be possible to solve both problems. (Perhaps by defaulting to utf-8 instead of ASCII. I don't know, I'm not a language designer.)
> I insist that C strings are indeed arrays of bytes, and we cannot use them to represent text correctly at present
OK, maybe I do see one small point to argue. A C string, such as one that might be used in Unix, is not necessarily text. But text, represented as utf-8, is a C string.
It seems like there's something to leverage here, at least for those points at which Python3 interacts with the OS.
Keyme, you and I are the few that have serious concern with the design of Python 3. I started to embrace it in a big way 6 months ago when most libraries I use are available in Python. I wish to say the wait is over and we should all move to Python 3 then everything will be great. Instead I find no compelling advantage. Maybe there will be when I start to use unicode string more. Instead I'm really annoyed by the default iterator and the binary string handling. I am afraid it is not a change for the good.
I come from the Java world when people take a lot of care to implement things as streams. It was initially shocking to see Python read an entire file into memory, turn it into list or other data structure with no regard to memeory usage. Then I have learned this work perfectly well when you have a small input, a few MB or so is a piece of cake for modern computer. It takes all the hassle out of setting up streams in Java. You optimize when you need to. But for 90% of stuff, a materialized list works perfectly well.
Now Python become more like Java in this respect. I can't do exploratory programming easily without adding list(). Many times I run into problem when I am building complex data structure like list of list, and end up getting a list of iterator. It takes the conciseness out of Python when I am forced to deal with iterator and to materialize the data.
The other big problem is the binary string. Binary string handling is one of the great feaute of Python. It it so much more friendly to manipulate binary data in Python compare to C or Java. In Python 3, it is pretty much broken. It would be an easy transition I only need to add a 'b' prefix to specify it as binary string literal. But in fact, the operation on binary string is so different from regular string that it is just broken.
In [38]: list('abc')
Out[38]: ['a', 'b', 'c']
In [37]: list(b'abc') # string become numbers??
Out[37]: [97, 98, 99]
In [43]: ''.join('abc')
Out[43]: 'abc'
In [44]: ''.join(b'abc') # broken, no easy way to join them back into string
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-44-fcdbf85649d1> in <module>()
----> 1 ''.join(b'abc')
TypeError: sequence item 0: expected str instance, int found
I don't have enough experience with either version to debate the merits of the choice, but the way forward with python 3 is to think of bytes objects as more like special lists of ints, where if you want a slice (instead of a single element) you have to ask for it:
Which is obviously worse if you are doing it a bunch of times, but it is at least coherent with the view that bytes are just collections of ints (and there are situations where indexing operations returning an int is going to be more useful).
In Java, String and Char are two separate types. In Python, there is no separate char type. It is simply a string of length of 1. I do not have great theory to show which design is better either. I can only say the Python design work great for me in the past (for both text and binary string), and I suspect it is the more user friendly design of the two.
So in Python 3 the design of binary string is changed. Unlike the old string, bytes and binary string of length 1 are not the same. Working codes are broken, practice have to be changed, often it involves more complicated code (like [0] becomes [0:1]). All these happens with no apparent benefit other than it is more "coherent" in the eye of some people. This is the frustration I see after using Python 3 for some time.
All the other commenters here that are explaining things like using a list() in order to print out an iterator are missing the point entirely.
The issue is "discomfort". Of course you can write code that makes everything work again. This isn't the issue. It's just not "comfortable". This is a major step backwards in a language that is used 50% of the time in an interactive shell (well, at least for some of us).
The converse problem is having to write iterator versions of map, filter, and other eagerly-evaluated builtins. You can't just write:
>>> t = iter(map(lambda x: x * x, xs))
Because the map() call is eagerly evaluated. It's much easier to exhaust an iterator in the list constructor and leads to a consistent iteration API throughout the language.
If that makes your life hard then I feel sorry for you, son. I've got 99 problems but a list constructor ain't one.
The Python 3 bytes object is not intended to be the same as the Python 2 str object. They're completely separate concepts. Any comparison is moot.
Think of the bytes object as a dynamic char[] and you'll be less inclined to confusion and anger:
>>> list(b'abc')
[97, 98, 99]
That's not a list of numbers... that's a list of bytes!
> The converse problem is having to write iterator versions of map, filter, and other eagerly-evaluated builtins
Well, in Python 2 you just use imap instead of map. That way you have both options, and you can be explicit rather than implicit.
> That's not a list of numbers... that's a list of bytes!
The point being made here is not that some things are not possible in Python 3, but rather than things that are natural in Python 2 are ugly in 3. I believe you're proving the point here. The idea that b'a'[0] == 97 in such a fundamental way that I might get one when I expected the other may be fine in C, but I hold Python to a higher standard.
> >>> list(b'abc')
> [97, 98, 99]
>That's not a list of numbers... that's a list of bytes!
No, it's a list of numbers:
>>> type(list(b'abc')[0])
<class 'int'>
I think the GP mis-typed his last example. First, he showed that ''.join('abc') takes a string, busts it up, then concatenates it back to a string. Then, with ''.join(b'abc'), he appears to want to bust up a byte string and concatenate it back to a text string. But I suspect he meant to type this:
>>> b''.join(b'abc')
That is, bust up a byte string and concatenate back to what you start with: a byte string. But that doesn't work, when you bust up a byte string you get a list of ints; and you cannot concatenate them back to a byte string (at least not elegantly).
Well, yes. Python chooses the decimal representation by default. So? It could be hex or octal and still be a byte.
My example was merely meant to be illustrative and not an example of real code. The byte object is simply not a str; so I don't understand where this frustration with them not being str is coming from. If you use the unicode objects in Python 3 you get the same API as before. The difference is now you can't rely on Python implicitly converting up ASCII byte strings to unicode objects and have to explicitly encode/decode from one to the other. It removes a lot of subtle encoding bugs.
Perhaps it's just that encoding is tricky for developers who never learned it in the first place when they started learning the string APIs in popular dynamic languages? I don't know. It makes a lot of sense to me since I've spent years fixing Python 2 code-bases that got unicode wrong.
You are not making useful comment because you don't understanding the use case. Python 2 is very useful in handling binary data. This complain is not about unicode. This is about binary files manipulation.
I'm thrill about the unicode support. If they only add unicode string and leave the binary string alone and just require an additional literal prefix b, it will be an easy transition. Instead the design is changed for no good reason and the code are broken too.
I have a hard time believing that the design was arbitrarily changed.
The request to add string-APIs to the bytes object have been brought up before [0]. I think the reasoning is quite clear: byte-strings are not textual objects. If you are sending textual-data to a byte-string API, build your string as a string and encode it to bytes.
For working with binary data there's a much cleaner API in Python 3 that is less prone to subtle encoding errors.
edit: I realize there is contention but I'm of the mind that .format probably isn't the right method and that if there were one it'd need it's own format control string syntax.
What you are looking for is imap(). In Python 2 there are entire collection of iterator variants. You can choose to use either the list or the iterator variants.
The problem with Python 3 is the list version is removed. You are forced to use iterator all the time. Things become inconvenient and ugly as a result. Bugs are regularly introduced because I forget to apply list().
>>> "".join(map(lambda byte: chr(byte), b'abc'))
Compares to ''.join('abc'), this is what I call fuck'd. Luckily maxerickson suggested a better method.
I think, one mistake that was made with Python 3 is, that compatibility had initially very limited attention. They solved many problems of Python 2, but left the legacy behind ... and there was the trouble:
* Many libraries where limited to Python 2, because the effort converting them seamed to high
* Because of minor problems (like the infamous u"-stuff), the overhead converting simple Python 2 programs was to high.
Some of the problems where fixed later (e.g. infamous u"- is now legal in Python 3 and ignored -- why not before??), but I think that than it also was a little late ... Python 3 has evolved further and many people just got into the habit to ignore Python 3.
Not caring about compatibility can be necessary, but also can be a burden (that hurts a long time)!
From where I sit, it seems like the 2/3 schism is the result of "one and only one way to do something". While is sounds like a good slogan and I was on board with this party line for a long time, breaking perfectly good features in pursuit of a more perfect adherence to "only one way" does nothing except alienate the community.
Python3 is just now becoming the default for many Linux distributions. Once that adoption took place the adoption of Python3 will increase very much. It's as simple as this.
Once this milestone is hit, the remaning incompatbible libraries will see fixes for python3.
357 comments
[ 2.9 ms ] story [ 114 ms ] threadOS X ships with 2.7.5. For a casual python user, sticking with what is there and working is safe, especially when the benefits of 3.x are unclear.
> It is a release goal for Ubuntu 14.04 LTS to have only Python 3 on the desktop CD images. Also, we won't be allowing Python 2 on the Ubuntu touch images.
Possibly typing "python" should be deprecated, should issue a deprecation warning before starting Python2. It's also a pity Python2 and Python3 source files aren't easily distinguished and will generally produce obscure errors if run using the wrong interpreter.
Likewise Torvalds refuses to let GPLv3 touch the kernel.
He's spoken out against it personally - he cleary _doesn't like it_.
However, you're correct that Linus clearly does not like even the released version of GPLv3.
[edited slightly]
Please don't misrepresent other peoples view points out of context.
If you want to read Linux Torvalds opinion, I suggest that you start with either one of the many articles about it (like http://news.cnet.com/Torvalds-No-GPL-3-for-Linux/2100-7344_3...), or any of the many mails on the mailing list. You will notices that Torvalds criticism is mostly directed at the DRM provisions, since he don't think a copyright license should be able to dictate a political matter such as DRM restrictions on devices.
To quote:
If Apple ships any OS X devices with DRM, they would have to have a separate build without the gplv3 bash in it. It would double support costs, maintenance and so on.
And patents. Any patents covering bash would loose value from using a GPLv3 bash. This also include any patent agreement Apple has, which include additional legal overhead.
GPLv2 vs GPLv3 is and always will be Patents and DRM.
I did it because I was trying to parse Unicode in XML (Apple iTunes music files) and Python 2 is completely broken when it comes to unicode.
I consider Python 3 a big "fuck you" from Guido to the community. I don't think he intended it to be so, but the effect of the long transition, and the lack of backported features in Python 2 (which could be easily accomodated), coupled with only limited adoption of Python3, demonstrate the leadership needs to pay closer attention to user pain.
Finally, I don't think Python will ever address the simple and most important core issue: lacking good multithreading support will make the language continue to be more and more marginal. CPUs aren't getting much faster, the CPython serial runtime isn't getting any faster. Multiple cores on a single chip, and multiple threads in a core, are getting more numerous- and not being able to take advantage of them using the obvious, straightforward memory-sharing, multi-threaded techniques the chips are design to run well- is just a clear ignorance of the future of computing designs.
I switched back from Python to C++ when shared_ptr and unique_ptr, along with std::map and std::vector became standard enough that programming in C++ became easy: convenient memory management, and maps/vectors are the two things that C++ adds to C that are valuable. Then I switched to Go because C++ compilation times are so bad, and writing high quality concurrent code is hard.
If he had actually addressed the important questions in Python3 (first being getting the majority of people on 3 quickly by getting the major libraries ported quickly, second being addressing the GIl in either 2 or 3 (if the GIL had been removed in 3, I would have much more strongly considered it).
If either of those had been addressed, some of us (who are influential in the area) would have adopted 3 and prosetlyized it. However, I lost so much confidence in Python after the 2 to 3 transition that I've decided to actively non-prosetylize it.
‘Just now’ is hence a bit too late, but it is correct that many users have only been able to use Python 3 some four to five years after 2008.
Python 3.0 release date: December 3rd, 2008. Numpy 1.5 (Full 3.0 support) release date: August 31st 2010.
It took 21 months.
> Like I said, I haven't used Python 3 before today
Well, which is it? Neither makes you much of an authority on the subject.
Behind the scenes it uses multiprocessing and/or threading plus queues etc. I have a function that adds command line arguments (number of workers, use threads or processes, debug mode) and then another that returns an executor given the command line arguments.
The debug mode forces a single thread and is intended for when you want to run pdb as multiple threads/processes make things considerably more complicated.
Personally I think multiprocessing is just dopey. Why force me to program around two different address spaces when I can just have one?
> Yet Another Threading API
concurrent.futures is not yet another threading api. futures is a standard async work/result wrapper, while the process and thread executors are standard executor apis.
> Personally I think multiprocessing is just dopey
Then don't use it. But be aware that it works well for many people. Some of my code scales linearly with multiprocessing and all I had to do was use a process executor instead of a thread executor.
Message passing is fine (and it's the sort of thing that Go uses) but ultimately, leaving single-address-space multithreading completely on the table when it's the single most obvious and straightforward way to take advantage of modern processors is just dumb.
For all intents and purposes, Python 3 is pretty much a new, separate programming language. Sure, it's very close to Python 2.x, but you don't have out of the box retro-compatibility so that pretty much kills it right there.
Python 2.x is so huge in terms of its use around the world and available libraries and resources for it that you just can't say "hey, the new version of Python will in practice break everything" and expect it to fly.
I love Python and the community around it (and have several packages on pypi myself), but Python 3 is a joke.
If we didn't want to kid ourselves, we'd kill Python 3 and back port the interesting features, like Alex suggests. At this point though, too much effort and egos are involved to make this realistic.
I don't know if that's necessarily a foregone conclusion. Scala releases aren't always/usually backwards-compatible, and you'd be hard-pressed to find people still using much older versions.
Using Python both personally and professionally, I do not think it is necessary. I already came across a few 3.X features, that I miss in 2.X and I write most 2.X code "in preparation" for the jump. But that's it, it is a jump, still.
The most interesting part of the post is the feedback matter, which seems to be accidental, but isn't since an open source language is shaped by the people - something you can forget, when you are not a contributor (which most of the people might not be).
You're fine. 98% of PyPI downloads are not. That's a startling statistic, and argues that something IS necessary, rather than the status quo of shaming the top 1000 Python packages sans 3K support and telling people "it's really not that bad."
It is that bad, in terms of adoption of the first 5 years of 3K's release. And unless we want to spend the next decade trying to get to 50% mindshare with 3K, the community needs to suck it up and change tacks.
The definition of insanity is trying the same solution twice and expecting different results.
> Perl 6 is currently being developed by a team of dedicated and enthusiastic volunteers. (http://perl6.org/)
People aren't using Python 3 enough because it's not the default in Debian/Ubuntu. That's about to change with Ubuntu 14.04. I expect that to tip the scales.
The previous things I've seen also indicated that Python 3 _isn't_ widely supported, though I guess Django finally got support recently, and that it _wasn't_ faster than 2.7. Is this just old information on my part, or is it different for certain kinds of workloads?
> it's not the default in Debian/Ubuntu.
Yeah, that should have a large impact, for sure.
1: I have seen lots of "check out this Perl 6 code" blog posts, but I don't know to what degree it's ACTUALLY vaporware.
There's a significant difference between a language for which a complete implementation does not exist such that you cannot use it, and one for which an implementation exists but few people currently choose to use it because there is a closely-related predecessor language that still has a stronger ecosystem.
As an stage in the development of a language out of an existing, widely-used predecessor, the "vaporware" stage tends to precede the "available but not popular due to predecessors ecosystem advantage" stage, which tends to precede the "displaces predecessor" stage.
Its obviously possible for a language to stop progressing at any of the earlier stages before reaching the last stage.
Perl 6 is not vaporware, you can run it today:
http://perl6.org/compilers/features
Is it 100% feature complete or fast? No. But there working code and new, stable releases on a monthly basis.
If it's not feature complete it's vaporware.
That's moving the goalposts a bit! What's currently called "Rakudo" is at least the sixth or seventh attempt at an implementation, not counting Pugs, viv, v6, or Niecza.
Perl 6 was first started on 2000. I've been reading Larry's apocryphal descriptions of its "features to be" for 4 times more years than Rust exists.
Rakudo is just a particular attempt at Perl 6, not the first and neither it consists of first time the language was announced publicly.
Even beyond that those are not "features to be". They are already available for use - http://perl6.org/compilers/features
Therefore I'm not sure what you've been reading, or if you are even reading them. Because if you would- you would know, Rakudo covers much of the Perl 6 specification.
By the way. Rust is still not complete. The wikipedia article says, work started in 2006- Which makes it 8 years and still incomplete. And this is for a project which has by far may be even 1000x modest goals. Python 3 was itself 10 years in development, and that is for small modifications to print statement and iterators. And even after 5 years since that date, it doesn't seem to have come any where closer to achieving good production scenario adoption.
And. These are- I said by far extremely modest goals compared to the Perl 6 project.
That's all well, but he announced it circa 2000. I don't care how he feels about it or how long he hacked it alone, I care since when the language was expected.
>Even beyond that those are not "features to be". They are already available for use http://perl6.org/compilers/features
Therefore I'm not sure what you've been reading, or if you are even reading them. Because if you would- you would know, Rakudo covers much of the Perl 6 specification.
In a half-arsed form and with 1/100 the community of Perl doing anything with them. And still not all of them.
Personally I stopped caring somewhere around 2006. And I've read all of Larry's "apocalypses" back when they used to be on Oreillynet, as well as followed the internal implementation politics for a few years.
>By the way. Rust is still not complete. The wikipedia article says, work started in 2006- Which makes it 8 years and still incomplete.
No, it says that it started as a "part-time side project in 2006". That could be 2 weeks total spend in those years writing a list of desired features in a napkin and getting a hello world compiled for all I know.
I only care about the time since the project was publicly announced and the community started working on it.
I doubt it will tip that much. The enterprise customers we develop for still ship on CentOS and have auditing departments that have to scour each library and package a system depends on. Adding a dependency hurts in this environment. These institutions only upgrade their interpreters once every 5 - 8 years if you're lucky.
Its an exceptionally ambitiously project which they are in no hurry in finishing. And that is for a good reason.
Way back in the early days, the goal was to release a usable 6.0 in a couple of years. The "there's no hurry" line is a post hoc justification from around 2006 or so.
But I think the goal has changed from "Let's fix Perl 5" to being a more bigger goal of developing a language what Perl 5 would likely look after say 3 decades of iterations. I can't say if that is good from a purely practical perspective. But it does sound like worthwhile goal to chase.
Lets take a look at Python 3 itself. Yes they finished the project, but 5 years after that- Now what? Users don't seem to a have a valid reason to move away from 2.x series for merely small time incremental improvements, while forced to migrate bulk of the infrastructure and code for nearly ~0% productivity gains.
Perl 6 could have come out by 2005-2006, and if it were to be merely a incremental non-backwards compatible change over p5. We would be having similar posts about p6 as we are having now about Python 3. Imagine a refined, yet not that beneficial p5 breaking CPAN. I am sure you wouldn't like such a situation.
The idea that it would be a series of incremental improvements to Perl 5 ended around 2001, and certainly by 2002 at the latest:
http://www.perl.com/pub/2000/11/perl6rfc.html
The grand unification of runtimes (Ponie) had failed long before the announcement of its demise in 2006:
http://www.nntp.perl.org/group/perl.ponie.dev/2006/08/msg487...
I suspect but can't entirely prove that the appearance, rapid ascent, and even more rapid burnout of Pugs made a lot of people realize that this would be a long slog. Even so, if you look back at mailing list messages or conference talks or blog posts in 2006, 2007, 2008, whenever, you'll see that the party line has always been "It's only a year or two away."
Also, Perl 6 turned out rather quickly to become a new language (although very much in the spirit of earlier Perls), while Python 3 was, as far as I can see (I'm not really a Python person) as an update to fix some important issues in the language, which despite by necessity being backwards incompatible, was intended to still be the same language.
The goal was to run existing Perl code in the same process, so that you could gradually adopt new features or use existing libraries with new code. If that had worked out, there'd have been less of the Python 2/3 gap.
Then again, the idea that Perl and P6 are different languages is something that's come up after the fact, after it became obvious that P6 wouldn't be ready for general use any time soon.
Yep, people underestimated the time to migration. So what? I completely agree that migration is happening. Now, let's talk about IPv6...
Consider me an average programmer, I have been using python for a year+ now. Most of the everyday stuff can be done in 2.7, some functionality I need / can't do I google and get a solution which works in 2.7. Why Py3 is not adopted is because there is not much benefit you get for doing the extra work (think chemistry - activation energy)
On another note, why can't we port it the little goodness back to 2.7 ?
I think this depends on whether or not you use, or will ever need to use, Unicode. I don't, and so Python3 is not only not exciting, but it creates a new problem for me: suddenly, I need to think about the difference between bytes and strings. Now maybe this is healthy for me, maybe it builds character. But it also adds effort and work that bring me no benefit. I'd like to see some of the smaller improvements of Python3--I'd like print to be a function, but in the big picture I'm better off with Python2.
I wonder if the Unicode problem could be solved in a way that I wouldn't need to think about it: perhaps utf-8 everywhere somehow.
Until there's something that means I'll get work done faster using 3, there's really no reason for me to try and adopt it.
I remember reading commentary on HN when a new version of PuTTY was released, and there was general sentiment that PuTTY is - for all intents and purposes - a "finished" product in that it does what you need it to do.
https://news.ycombinator.com/item?id=2758696
Python v2.7 is similar in that it's a tool that does many amazing things and is very stable. Could it be improved? Sure! But for the most part it's a great piece of software.
The care and planning that went into creating the tool is also going into the upgrade process, and I think that's a good thing.
I have been very impressed with 2to3, the amount of work it does is pretty impressive. But as somebody who tends and prefers to work in python 3 but often needs my scripts to run in python 2, I have no choice but write those in python 2. I can see the same dilemma for somebody who writes an open source library and hope for as much usage as possible.
What 3to2 and six really do is keeping the 2.x runtime alive, and as long as the runtime sticks around (and is patched, and features are backported etc etc etc), people will keep using it.
Besides, 3to2 (which does exist but isn't maintained) and 2to3 are both bad ideas. They miss edge cases, even common ones, and they add even more complexity to the build/test/install process which people already find hard.
this is the killer feature that would move me to python 3. alas, not released yet. And I took a rails job :(
And apparently some fairly major ones (e.g., decimal in Python 3.3.)
So, Python 3's killer feature is reasonable Unicode support.
I don't know much about unicode myself, but some people I trust like Armin Ronacher have showed how some python3 are still heavily broken w.r.t unicode (http://lucumr.pocoo.org/2013/7/2/the-updated-guide-to-unicod...).
Would it be at all feasible to enable Python 3 to import Python 2 code? I imagine this could be done without making Python 3 fully backwards compatible, but I might be wrong.
There are many examples of systems where many look at them today and say: "This is terrible, I could design a better/less-complicated system with the same functionality in a day". Some examples of this dear to my heart are HTML, OpenID, and SPDY. It's important to recognize the reason these systems succeeded is they sacrificed features, good ideas, and sometimes even making sense to provide the most critical piece: compatibility with the existing world or a migration plan that works piecemeal. Because without such a story, even the most perfect jewel is difficult to adopt.
The OP, about Python 3, is right on except for when it claims making Python 3 parallel installable with 2 was a mistake; doing that would make it even more impossible to migrate to 3 (unless the single binary was able to execute Python 2 code). (Also related: how Arch screwed up Python 3 even more: https://groups.google.com/d/topic/arch-linux/qWr1HHkv83U/dis... )
It won't change anything for the shop that has a million LOC, but it might start to budge that 2% number.
The python heads say you need to write code to be readable.. ok so how do I read all the call sites of this method or function? An unforgivable contradiction on large code bases, I'm afraid.
My current projects are currently compatible with Python 3 and it's my main target whenever possible (depending on the dependancies).
But all in all this is one of these little things that make developping in Python less fun than before. This is not my preferred language anymore.
In fact, it's 3.4 that really starts to wet the beak with asyncio and enum. I'm not sure 2.8 needs to happen, if 3.x simply, and finally, has good reasons to get on board.
Samefag detected. Go back to 4chan, faggot.
Also, the core devs got at least some things right with Python 3.3 by making it a lot easier to write code that targets 2.7 and 3.3 at the same time. In retrospect, that should have been the focus much sooner.
How many people are still on Windows? How many are on OSX? How many are comfortable in 2.x and couldn't care less about new features?
Python 3.x may be better in some ways, but it just doesn't look better enough.
I wonder how long it will be before the 2.x community goes its own way like Perl 5.
Windows doesn't ship with an OS-default version of Python and doesn't include OS features that depend on the OS-default version of python, so its not as influential in that regard. But, in that regard, the inclusion of the multiple-install-compatible python launcher in the basic Windows install of Python 3.3 probably improves the Python 3.x story on Windows, since it eases the multiple install pain and makes it easier to work with Python 3.x while having some Python 2.x software in use on Windows.
The ability of the ruby core team to manage not just the technical aspect of making the language better, but smooth the transition in a way that actually succeeded in bringing the community (and even alternate ruby implementations) along with them, hasn't been given nearly enough credit. You could analogize it to Apple with OS 9 -> OS 10.9, versus Microsoft with people still running XP
I don't expect this to actually be true, but it speaks to an attitude.
https://gist.github.com/nathany/6046171
on the python side things are different. python seems quite a bit faster then ruby. same league, but faster. yet its syntax is what needs to be greatly improved upon as it is so full of surprises and dark corners.
a language's syntax it's like its API, and performance is an implementation "detail". breaking the API is much more painful then chaning the implementation -- therefor i prefer ruby's approach: begin slow but with a quite stable syntax.
p.s. i dont intend to hurt feelings with this comment
http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...
Ruby doesn't follow SemVer, so expecting no breaking changes between 1.8.6 and 1.8.7 wouldn't necessarily be correct. I don't remember what the policy was at the time.
https://www.ruby-lang.org/en/news/2013/12/21/semantic-versio...
Well now, that's not exactly semver, is it…?
No, you couldn't. The difference in upgrade rates between Windows and OS X is primarily due to their differing customer bases. Windows is very popular in enterprise, which avoids unnecessary upgrades in order to ensure compatibility with in-house software.
OS X, however, has almost no presence in enterprise, and consumers don't mind upgrades nearly as much, since the consumer software they use has to be compatible with all OS versions. Also, you can't buy Macs with old versions of OS X, whereas Microsoft makes it trivial to buy a new machine and install an older version of Windows.
That's in the US. In the rest of the world, it's around 5%.
0: http://www.netmarketshare.com/operating-system-market-share....
God I wish the numbers were like that for Windows.
Apple frequently deprecates APIs that they endorsed just a few years ago. And they claim that apps can be rewritten to work with new APIs in a few hours, when in reality it can take weeks, months or even longer due to refactoring issues. If you want to be an Apple developer, you will likely be rewriting a portion of your code at some point to run on a new OS release. I was optimistic that the practice might end but history is already repeating itself with Apple insisting on iOS 7 compliance. The kicker is that Apple could have ported a lightweight version of Mac OS to run directly on arm for iOS, but they didn't, and I disagree with it being a performance issue (iPads are orders of magnitude more powerful than NeXT machines, or even the early PowerPCs that ran OS X). They created a vast array of nearly duplicate code prefixed with UI instead of NS. This looks like more of an anachronism every day to me with tablets running at multiple GHz with GBs of ram, when the only major difference between desktop and mobile is the touch interface.
Contrast this with Microsoft, where I am finding very old examples designed for Windows XP that still run today. Now Microsoft is certainly burning its developers with the major breaks in various versions of DirectX, or subtle differences in APIs between desktop/mobile/Xbox, but in my opinion this isn't happening nearly to the extent that it is with Apple.
xp? Up until a couple years ago when I switched to 64bit windows (first computer I had with more than 4gm ram) I could still run win3.1 and dos programs in (32-bit) windows 7. Including dos programs designed for the very original ibm pc (1981!).
Even today with 64bit windows I can still run most windows 95 programs.
I know in Ruby 1.8.x to 1.9, some major changes in my day-to-day coding involved Hash (elimination of hash-rocket, and ordered key-value pairs by default)...but I can't think of anything off hand that required me to rewrite my own libraries. And between 1.9 and 2.0...I've been switching between machines that have 1.9.3x and 2.0x and can't even tell a difference. Obviously, I'm not doing any legacy-maintenance in this situation, but it seemed that Python's changes, while breaking, were also significant improvements and changes to the API that mandated changes in implementation?
Speaking as a non-Python-dev...I've been wanting to get more into Python, at least to write routines that take advantage of scipy and numpy and all that goodness...but the process of deciding between 2.x and 3.x and keeping the steps/compromises in order can be beguiling.
There were a lot of gems that weren't 1.9 compatible for quite a while, so there must have been something that caused people to rewrite libraries. I think that one disadvantage for Python 2->3 was an advantage Python had over Ruby -- the bigger base of libraries, many of which were widely used but basically in very-low-effort maintenance mode.
Unlike Python, which changed syntax at the same time, Ruby tried to maintain compatibility with existing syntax. In practice, this allowed Ruby libraries (including Rails; I did the bulk of the encoding work for Ruby 1.9 in Rails 3) to do runtime introspection to support 1.8 and 1.9 at the same time.
In my view, the best, most underrated thing that Matz did in Ruby 1.9 was to make all of the breaking changes detectable and shimmable at runtime.
More likely, if there indeed is never a 2.8, is that Python 2 would eventually be forked and renamed. "Rattler" would make a better pun.
And while doing so I will listen to this track from the Cobra soundtrack.
http://www.allmusic.com/song/skyline-mt0010878632
Preferably to its C64 rendition.
http://ftp.df.lth.se/pub/media/soasc/soasc_mp3/MUSICIANS/D/D...
However it has been interesting to follow over the last five years. It has been a sort of, "what if p5 broke the CPAN," scenario played out in real-life. Breaking compatibility with your greatest resource has a painful trade-off: users.
Everything I work on is not even considering a migration to Python 3. OpenStack? A tonne of Django applications that depend on Python 2-only libraries? A slew of automation, monitoring and system administration code that hasn't been touched since it was written? Enterprise customers who run on CentOS in highly restrictive environments? A migration to Python 3 is unfathomable.
However my workstation's primary Python is 3. All of my personal stuff is written in 3. I try to make everything I contribute to Python 3 compatible. I've been doing that for a long time. Still no hope that I will be working on Python 3 at my day job.
Sad state of affairs and a cautionary tale: "Never break the CPAN."
I agree 100% with your first paragraph.
But some things are lacking. Django is there, and it works great, but you hit a dead end with some libraries (like MySQL - even though it seems there are alternatives).
So in the end you don't make the move because you're afraid of needing something that may not be there.
A lot of those libraries have Python 3 port branches where you can help the effort. For example: https://github.com/boto/boto/tree/py3kport/py3kport
PIL is a good example; the last actual PIL release was in 2009. We now have a great fork that works (Pillow), but PIL is not the only dead package that's still in active use. There are quite a few others, too.
I have to say, Python is my favorite language still (at least 2.7), and I hope the community can get it back in line. I can see JS, NodeJS and Python being my core technologies, but this issue with 3 is squashing my dream.
This effectively means that only for purely greenfield projects for machines under your team's control and possibly even new teams can Py3 be used confidently. It's a shame - there are a number of nifty cleanups in Py3, but I simply don't see a compelling reason to use Py3 over Py2. This is particularly true given PyPy being fast and Py2. Given the triple of Python/Perl/Ruby for a greenfield project, I'd probably explore Ruby; I haven't been jaded by it yet, unlike Perl or Python.
Python 3 runs on operating systems older than 2013 and, while it might not be the OS default, there is no reason a non-library project can't bundle its own interpreter in the distribution and use that instead of the OS default interpreter.
You are absolutely right, but the overall cost of deployment skyrockets at that point. Not to mention pernicious bugs from grabbing the wrong library & whatever other weirdness happens. Unless the project is reasonably significant (deployment cost is insignificant to development cost), it's much easier just to work from the system python.
The niceities of greenfield work and what ought to be done start breaking down as you get into heterogeneous systems where total system stability (lack of change ) is a big deal, and these fleets have been running for a decade or so. Imagine having to figure out deployments to a Python 2.4 environment in 2012 (I had this problem).
To misquote a famous statement, you fight the battle with the troops you have, not the troops you want, against the enemy you have, not the enemy you want. The reality is that it's often (usually?) not worth it to ship full venvs with Py3. It's often much better to ship Py 2.7 code (just your own code) and install your own libraries in the system library location.
This seems like it depends on what kinds of OS you regularly use. On Linux the deployment situation isn't quite that dire. Python3 has been in Debian stable since Debian 6 (aka Squeeze, released February 2011), so running Python3 code isn't a problem if the machine is even close to up to date. On Ubuntu the situation is even reversed: Python3 is default, and starting in 2014 it will be the only supported version, with Python2 support being dropped. It will still be installable out of "universe", but the goal is to drop Python2 from "main" by the release of 14.04 LTS in 2014, because LTS releases promise 5 years of support, and they don't want to be supporting Python2 into 2019.
Iterators everywhere are incredibly annoying, especially with my development workflow, where I don't put a line of code into a file before I run it manually in the interpreter. When I run a map over a list, I just want to see the freaking results.
Default unicode strings are obscenely annoying to me. Almost all of my code deals with binary data, parsing complex data structures, etc. The only "human readable" strings in my code are logs. Why the hell should I worry about text encoding before sending a string into a TCP socket...
The fact that the combination of words "encode" and "UTF8" appear in my code, and str.encode('hex') is no longer available, is a very good representation of why I hate Python 3.
In Python 2, the rule of thumb was "If it makes sense, it's going to work". In Python 3, this isn't true. Not often, but often enough to annoy. This makes Python "feel" like Java to me.
And worst of all, Python 3 has so many excellent features, like sub-generators, a better performing GIL, etc. These sometimes force me into coding with Python 3. I hate it every freaking time.
I said to myself that with Python 3.4, due to great stuff like the new asyncio module, I'll have to make the switch. It's really sad that this is because I "have to" do it, and not because I "want to".
personally, coming from a country that needs to deal with non-ascii, i love unicode by default and there are b'' strings if you need them. str.encode is a non-issue - you wasted more words on it than the two-line function enc() it takes to fix it.
And while 2 lines are not worth my rant, writing those 2 lines again and again all the time, is.
If your Python 3 code is dealing with binary data, you would use byte strings and you would never have to call encode or touch UTF-8 before passing the byte string to a TCP socket.
What you're saying about Unicode scares me. If you haven't already, please read The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) before writing any libraries that I might depend on.
I'll start by adding that it's also incredibly annoying to declare each string to be a byte string, if this wasn't clear from my original rant.
Additionally, your advice is broken. Take a look at this example, with the shelve module (from standard library).
s=shelve.open('/tmp/a')
s[b'key']=1
Results in:
AttributeError: 'bytes' object has no attribute 'encode'
So in this case, my byte string can't be used as a key here, apparently. Of course, a string is expected, and this isn't really a string. My use case was trying to use a binary representation of a hash as the key here. What's more natural than that. Could easily do that in Python 2. Not so easy now.
I can find endless examples for this, so your advice about "just using byte strings" is invalid. Conversions are inevitable. And this annoys me.
> What you're saying about Unicode scares me.
Yeah, I know full well what you're scared of. If I'm designing everything from scratch, using Unicode properly is easy. This, however, is not the case when implementing existing protocols, or reading file formats that don't use Unicode. That's where things begin being annoying when your strings are no longer strings.
His advice was sound, and referred to your example of TCP stream data (which is binary). Your example regards the shelve library.
> So in this case, my byte string can't be used as a key here, apparently.
shelve requires strings as a keys. This is documented, though not particularly clearly.
> so your advice about "just using byte strings" is invalid.
Let me attempt to rephrase his advice. Use bytestrings where your data is a string of bytes. Use strings where your data is human-readable text. Covert to and from bytestrings when serializing to something like network or storage.
> Conversions are inevitable.
Absolutely, because bytestrings and (text)strings are two different types.
> And this annoys me.
There is no real alternative though, because there is no way to automatically convert between the two. Python 2 made many assumptions, and these were often invalid and led to bugs. Python 3 does not; in places where it does not have the required information, you must provide it.
> when implementing existing protocols, or reading file formats that don't use Unicode.
I'm afraid it's still the same. A protocol that uses unicode requires you to code something like "decode('utf-8')" (if UTF-8 is what it uses), one that does not requires "decode('whatever-it-uses-instead')". If it isn't clear what encoding the file format or protocol stores textual data in, then that's a bug with the file format or protocol, not Python. Regardless though, Python doesn't know (and can't know) what encoding the file or protocol uses.
> Iterators everywhere are incredibly annoying, especially with my development workflow, where I don't put a line of code into a file before I run it manually in the interpreter. When I run a map over a list, I just want to see the freaking results.
That's what list() is for:
> Default unicode strings are obscenely annoying to me. Almost all of my code deals with binary data, parsing complex data structures, etc. The only "human readable" strings in my code are logs. Why the hell should I worry about text encoding before sending a string into a TCP socket...> The fact that the combination of words "encode" and "UTF8" appear in my code, and str.encode('hex') is no longer available, is a very good representation of why I hate Python 3.
I'm afraid I don't understand your complaint. If you're parsing binary data then Python 3 is clearly superior than Python 2:
Seems much more reasonable than: Because they're not the same thing. Python 2 would implicitly "promote" a bytestring (the default literal) to a unicode object so long as it contained ASCII bytes. Of course this gets really tiresome and leads to Python 2's, "unicode dance." Armin seems to prefer it to the extra leg-work for correct unicode handling in Python 3 [0] however I think the trade-off is worth it and that pain will fade when the wider world catches up.> In Python 2, the rule of thumb was "If it makes sense, it's going to work". In Python 3, this isn't true. Not often, but often enough to annoy. This makes Python "feel" like Java to me.
Perhaps this is because you're a used to Python 2 and anything else is going to challenge your perceptions of the way it should work?
I don't understand the Java comparison.
> And worst of all, Python 3 has so many excellent features, like sub-generators, a better performing GIL, etc. These sometimes force me into coding with Python 3. I hate it every freaking time.
> I said to myself that with Python 3.4, due to great stuff like the new asyncio module, I'll have to make the switch. It's really sad that this is because I "have to" do it, and not because I "want to".
Well you can always attempt to backport these things into your own modules and keep Python 2 alive.
I think going whole-hog and embracing Python 3 is probably easier in the long run. I'm not sure how long the world is going to tolerate ASCII as the default text encoding given that its prevalence has largely been an artifact of opportunity. Unicode will eventually supplant it I have no doubt. It's good to be ahead of the curve.
[0] http://lucumr.pocoo.org/2013/7/2/the-updated-guide-to-unicod...
A string represents a snippet of human readable text and is not merely an array of bytes in a sane world. Thus is it fine & sane to have to encode a string before sticking it into a socket, as sockets are used to transfer bytes from point a to b, not text.
(Please note that I'm not making any statement as to what's appropriate to send down a TCP socket.)
The pain is painful [in Python 3] primarily for library authors and only at the extremities. If you author your libraries properly your users won't even notice the difference. And in the end as more protocols and operating systems adopt better encodings for Unicode support that pain will fade (I'm looking at you, surrogateescape).
It's better to be ahead of the curve on this transition so that users of the language and our libraries won't get stuck with it. Python 2 made users have to think (or forget) about Unicode (and get it wrong every time... the shear amount of work I've put into fixing codebases that mixed bytes and unicode objects without thinking about it made me a lot of money but cost me a few years of my life I'm sure).
I don't think a programming language can take the position that an OS needs to "adopt better encodings". Python must live in the environment that the OS actually provides. It's probably a vain hope that Unix strings will vanish in anything less than decades (if ever), given the ubiquity of Unix-like systems and their 40 years of history.
I understand that Python2 does not handle Unicode well. I point out that Python3 does not handle Unix strings well. It would be good to have both.
This is the first time I encounter the idiom Unix strings. I'll map it to array of bytes in my table of idioms.
> I don't think a programming language can take the position that an OS needs to "adopt better encodings".
I do think that programming languages should take a position on things, including but not limited to how data is represented and interpreted in itself. A language is expected to provide some abstractions, and whether a string is an array of bytes or an array of characters is a consideration of a language designer, who will end up designing a language takes one or another of the sides available.
Python has taken the side of language user: enabled Unicode names, defaulted to Unicode strings, defaulted to classes being subclasses of the 'object' class... Unix has taken the side of machine (which was the side at the time of Unix's inception.
> [...] probably a vain hope that Unix strings will vanish [...]
If only we wait for them to vanish, doing nothing to improve.
> Python must live in the environment that the OS actually provides.
Yes, Python must indeed live in the OS' environment. Regardless, one need not be a farmer because they live among all farmers, need they?
The usual idiom is C-strings, but I wanted to emphasize the OS, not the language C.
>> [...] probably a vain hope that Unix strings will vanish [...] >If only we wait for them to vanish, doing nothing to improve.
The article is about the lack of Python3 adoption. In my case, Python3's poor handling of Unix/C strings is friction. It sounds like you believe that Unix/C strings can be made to go away in the near future. I do not believe this. (I'm not even certain that it's a good idea.)
I do not expect C or Unix interpretations of strings to change, but I believe that they must be considered low-level and require higher-level language user to explicitly request the compiler to interpret a piece of data in such fashion.
My first name is "Göktuğ". Honestly, which one of the following is rather desirable for me, do you think?
orA Unix filename, on the other hand, might be any sort of C string. This sort of thing is all over Unix, not just filenames. (When I first ever installed Python3 at work back when 3.0 (3.1?) came out, one of the self tests failed when it tried to read an unkosher string in our /etc/passwd file.) When I code with Python2, or Perl, or C, or Emacs Lisp, I don't need to worry about these C strings. They just work.
My inquiry, somewhere up this thread, is whether or not it would be possible to solve both problems. (Perhaps by defaulting to utf-8 instead of ASCII. I don't know, I'm not a language designer.)
OK, maybe I do see one small point to argue. A C string, such as one that might be used in Unix, is not necessarily text. But text, represented as utf-8, is a C string.
It seems like there's something to leverage here, at least for those points at which Python3 interacts with the OS.
I come from the Java world when people take a lot of care to implement things as streams. It was initially shocking to see Python read an entire file into memory, turn it into list or other data structure with no regard to memeory usage. Then I have learned this work perfectly well when you have a small input, a few MB or so is a piece of cake for modern computer. It takes all the hassle out of setting up streams in Java. You optimize when you need to. But for 90% of stuff, a materialized list works perfectly well.
Now Python become more like Java in this respect. I can't do exploratory programming easily without adding list(). Many times I run into problem when I am building complex data structure like list of list, and end up getting a list of iterator. It takes the conciseness out of Python when I am forced to deal with iterator and to materialize the data.
The other big problem is the binary string. Binary string handling is one of the great feaute of Python. It it so much more friendly to manipulate binary data in Python compare to C or Java. In Python 3, it is pretty much broken. It would be an easy transition I only need to add a 'b' prefix to specify it as binary string literal. But in fact, the operation on binary string is so different from regular string that it is just broken.
(This narrowly addresses that concern, I'd readily concede that the new API is going to have situations where it is worse)
Here is another annoyance:
So in Python 3 the design of binary string is changed. Unlike the old string, bytes and binary string of length 1 are not the same. Working codes are broken, practice have to be changed, often it involves more complicated code (like [0] becomes [0:1]). All these happens with no apparent benefit other than it is more "coherent" in the eye of some people. This is the frustration I see after using Python 3 for some time.
All the other commenters here that are explaining things like using a list() in order to print out an iterator are missing the point entirely.
The issue is "discomfort". Of course you can write code that makes everything work again. This isn't the issue. It's just not "comfortable". This is a major step backwards in a language that is used 50% of the time in an interactive shell (well, at least for some of us).
If that makes your life hard then I feel sorry for you, son. I've got 99 problems but a list constructor ain't one.
The Python 3 bytes object is not intended to be the same as the Python 2 str object. They're completely separate concepts. Any comparison is moot.
Think of the bytes object as a dynamic char[] and you'll be less inclined to confusion and anger:
That's not a list of numbers... that's a list of bytes! And you get a string!Well, in Python 2 you just use imap instead of map. That way you have both options, and you can be explicit rather than implicit.
> That's not a list of numbers... that's a list of bytes!
The point being made here is not that some things are not possible in Python 3, but rather than things that are natural in Python 2 are ugly in 3. I believe you're proving the point here. The idea that b'a'[0] == 97 in such a fundamental way that I might get one when I expected the other may be fine in C, but I hold Python to a higher standard.
Well, yes. Python chooses the decimal representation by default. So? It could be hex or octal and still be a byte.
My example was merely meant to be illustrative and not an example of real code. The byte object is simply not a str; so I don't understand where this frustration with them not being str is coming from. If you use the unicode objects in Python 3 you get the same API as before. The difference is now you can't rely on Python implicitly converting up ASCII byte strings to unicode objects and have to explicitly encode/decode from one to the other. It removes a lot of subtle encoding bugs.
Perhaps it's just that encoding is tricky for developers who never learned it in the first place when they started learning the string APIs in popular dynamic languages? I don't know. It makes a lot of sense to me since I've spent years fixing Python 2 code-bases that got unicode wrong.
I'm thrill about the unicode support. If they only add unicode string and leave the binary string alone and just require an additional literal prefix b, it will be an easy transition. Instead the design is changed for no good reason and the code are broken too.
The request to add string-APIs to the bytes object have been brought up before [0]. I think the reasoning is quite clear: byte-strings are not textual objects. If you are sending textual-data to a byte-string API, build your string as a string and encode it to bytes.
[0] http://bugs.python.org/issue3982
For working with binary data there's a much cleaner API in Python 3 that is less prone to subtle encoding errors.
edit: I realize there is contention but I'm of the mind that .format probably isn't the right method and that if there were one it'd need it's own format control string syntax.
The problem with Python 3 is the list version is removed. You are forced to use iterator all the time. Things become inconvenient and ugly as a result. Bugs are regularly introduced because I forget to apply list().
Compares to ''.join('abc'), this is what I call fuck'd. Luckily maxerickson suggested a better method.* Many libraries where limited to Python 2, because the effort converting them seamed to high
* Because of minor problems (like the infamous u"-stuff), the overhead converting simple Python 2 programs was to high.
Some of the problems where fixed later (e.g. infamous u"- is now legal in Python 3 and ignored -- why not before??), but I think that than it also was a little late ... Python 3 has evolved further and many people just got into the habit to ignore Python 3.
Not caring about compatibility can be necessary, but also can be a burden (that hurts a long time)!
Python3 is just now becoming the default for many Linux distributions. Once that adoption took place the adoption of Python3 will increase very much. It's as simple as this. Once this milestone is hit, the remaning incompatbible libraries will see fixes for python3.