Ask HN: Why do Python libraries make so many breaking changes?
Today I was working with Pydantic and I realized that I am version 2, and so many things are not the same anymore. Of course, it's good that they have put warnings in there but I still don't like it.
I am also a heavy user of Poetry and I am in dread of updating it. Everytime an update happens, something breaks. Do other people have the same experience? Is this a Python only thing?
6 comments
[ 10.4 ms ] story [ 185 ms ] threadJava developers, for example, tend to be slightly older, and therefore have had more painful experiences upgrading stuff, but they also tend to work in large enterprises where breaking stuff has an associated cost, that you do not experience when you are hacking on your own. Not breaking stuff requires an insane amount of energy and discipline.
For the technological argument, I like to think back to the 1990s, when I experimented with Common Lisp, and was amazed at how long one could work on one piece of software, without having to restart any of it. Oftentimes, the runtime state and the source code on disk would get terribly out of sync. It was even a best practice to simply store the memory image to disk, and restart from that later. I envisioned a future in which these problems would be solved, and instead of writing new programs, we would merely polish and improve running software, keeping the data alive for eternity.
Unfortunately, it's now 30 years later, and the only movement in that direction I see is that data remains somewhat active while being shared by multiple services. Unfortunately, the services themselves are still blobs that must be restarted over and over again, losing all intermediate state.
I think we could do better, for example by having a programming language that takes immutability to the next level, i.e. by not allowing a restart, or by requiring upgrade functions to go from old data formats to newer ones. Perhaps something may have come out of the field of reversible programming languages?
I think there are two things going on:
1. Python: Python's laxness doesn't make it easy to define strict APIs or know when you're breaking them. The lack of export controls also means defining what's in the API and what's not is hard. Also Python's strong introspection makes it possible to do crazy things further blurring the API boundary.
2. The ecosystem: with pydantic as an example - the library has grown enormously, more than I ever imagined from an experiment in a file in another project to a major library used by thousands of millions of developers, for a very wise range of applications. I never imagined that happening, do I didn't think all that hard about the interface at the beginning, or even at the v1 release. I also knew much less back then.
I'm not sure this kind of growth is very common in other languages except JavaScript.
(Written on my phone, sorry for typos)
That is poppycock. Every time you make any change, you have to ask the questions:
1. Can the application use the API to tell the difference? (As in, tell whether the proposed change has been implemented or not?) (If this is not directly used by the API, what intermediate layers use this, which are connected to what APIs, and to which of those does the change matter, and how?)
2. Could an application so affected by the difference plausibly exist today, and would that application be doing something that is documented as valid? (Or that is not documented as valid only by neglect, and that should be valid the way it de facto works now?)
2. b) Do we have a test case for this (either way)?
This is not rocket science. Safe, high-level languages mostly make it easier, not harder, whether they are dynamic or not. You can do this. You pretty much have to do this. How can you not know if you're breaking an API; it's ridiculous.
Imagine if Intel said, oh, assembly language's laxness makes it hard for us to know when we're breaking the instruction set. Sorry your old Core 2 Duo stuff doesn't run on Core i9.