If you are working in a really shitty software shop with no discipline or process control, this can be a great way to start establishing both, so I say no.
If you are developing your own personal website with a static page and bootstrap, probably over kill.
We don't need to have a polarized opinion about everything.
i don't like the scheme described in the article, but i do think that X.Y.Z is overkill. X.Y provides plenty of information and doesn't make me second guess if i should upgrade or not. Z often creates a false sense of confidence that shit wont break. changes in X always mean breakage, changes in Y mean potential, but unintended, breakage.
The patch version is there for very minor things. Adding docs, adding tests, changing metadata, _maybe_ minor bug fixes. Hopefully no one has to depend on a patch version, and hopefully they're usually ".0". I've seen arguments that patch versions should go away, but revving minor versions for some things is often too much churn - people try to understand what's in the new release when it's not even worth their time.
Minor versions are things you reasonably might depend on like a new feature or significant bug fix. You _should_ be able to upgrade seamlessly though one person's bug fix is another's breaking change. I don't think it's reasonable to get rid of these and treat everything as a breaking change.
So I don't know what's simpler for libraries. Semver seems to have boiled it down pretty well.
you can also just git tag 1.0.0 and git push --tags. Then when you git describe you get a 1.0.0-###-gHASH where ### is the number of commits past 1.0.0 and HASH is the short hash of the last commit (I think). Its really useful for tracking versions across branches and also always having a unique version which points to an exact commit.
The comparison doesn't really make any sense - in what way is the Doge election protocol comparable to semver? It also seems weird to call something bad by comparing it to an election protocol that has been shown to be incredibly robust and was used in a prosperous city state that was successful for a very long time.
Yeah, I didn't get it either. It feels to me like he had this Italian election analogy in his pocket and couldn't wait to use it. It feels very forced.
@williamle8300
>He's making good points, but what alternative is he offering?
Your comment is dead, but to answer the question - the alternative proposed is to encode the contract of your library in itself, and never break it, only extend it.
If you need to change the API, you don't change behaviour or signatures of your functions while preserving the names; you just add a completely new mainspace, while letting people use the old functions if they still have reason to. You don't have to keep actively supporting the old namespaces (i.e. no new bugfixes), but just need to not break the stuff that already worked.
There's some scenarios where this is hard - for example it's a web api and supporting it is costing you $$. But for things like that devs are generally aware that what they're consuming might get shut down at some point; so doing a new api, in a new namespace, and shutting down the old one at some point is not completely out of the question.
The reasons that Rich Hickey tore into SemVer seem to be issues SemVer weren't concerned about solving and his alternatives doesn't solve the main issue SemVer was created for. The problem of "can I reasonably upgrade this library without breaking my own code, and if possible to so in an automated fashion" isn't solved by chronological versioning. If I need to upgrade my application due to a security vuln, SemVer lets me know if I can just "drop-in" the upgrade, or if I need to work more.
The problem of knowing whether 1.3 vs 3.7 was written 10 days ago or 10 years ago doesn't seem useful. Rarely has software I have written depended on how recent the library was released.
Ultimately, the talk tears into version numbers in general, but not semantic versioning - he doesn't discuss anything particular to SemVer. He could have been talking about Postgres' versioning conventions and the talk would still hold.
I think this talk makes his opinion on how breaking changes in libraries should be handled (in the context of the JVM ecosystem) very clear:
A) avoid breaking API changes
B) if you have to do large breaking API changes, that will probably affect alot of dependent projects, make it a new artifact / namespace / library that can live side-by-side with the old version
B is actually pretty common in large java library projects (rxjava -> rxjava 2, retrofit -> retrofit 2, dagger -> dagger ‡ all have independent maven artifacts and I think also packages) and imho this approach makes a lot of sense. It's also the more important part of this talk compared to his critique of semver.
Isn't semver the best way to do what he's advocating, then?
I mean, it's not like people delete 1.3.0 from their packaging system when they release 2.0.0. Incrementing the major version number is semver's way of declaring a new namespace, and once declared, it lives alongside the old ones.
It is about treating new versions with major API breakage as if they were completely new libraries, not as a "maybe, possibly drop-in replacement, iff we use just the right feature subset". E.g. RxJava changed their maven artifact ('io.reactivex:rxjava:1.y.z' -> 'io.reactivex.rxjava2:rxjava:2.y.z') and the package name where in which their code lives ('rx' -> 'io.reactivex'). This makes it possible for both versions to be used at the same time while transitioning, without breaking dependencies that rely on the older version and without having to bother with automatic multi-versioning in buildtools.
With that in place it is questionable what the actual advantage of semver vs a simpler monotonous versioning schema (like build number or UTC date) is.
> Ultimately, the talk tears into version numbers in general, but not semantic versioning
He does talk explicitly about the trouble of making major changes in SemVer[0]. The gist of his argument was that minor changes in semver are relatively useless while major changes have a high probability of breaking your software. Major changes in semver are backwards incompatible and update the program's API in place. This leads to dowstream breakage and fear around doing upgrades.
> If I need to upgrade my application due to a security vuln, SemVer lets me know if I can just "drop-in" the upgrade, or if I need to work more.
I think the point he was trying to make was that upstream developers could change internals of a library but keep the API consistent so that downstream devs would never have to worry about scary updates. As you said with SemVer, if the security upgrade is a significant change, then you can expect breakage in the library. What he was advocating was patching issues like security vulns under the hood while keeping everything backwards compatible. Major upgrades could even add new namespaces, functions, and arguments but there's no real point to deleting or mutating old code, that just creates breakage. He wants software libraries to be immutable to take care of dependency issues and versioning to better reflect the changes made in the code.
> As you said with SemVer, if the security upgrade is a significant change, then you can expect breakage in the library.
In practice that's very uncommon. If someone is actually doing security releases, then either they release a minimal change to supported versions, or the distributions do that for them. Actual security upgrades are normally single patches which take great care not to do any API or behaviour changes.
Create a new namespace for your new API, or your new Interface, or your new version, or whatever you want to call it. But don't delete your old namespace. If your code is someone else's dependency, you should leave the old namespace in place for those who need it.
Left unresolved is how to deal with the accumulation of old code. I assume there will eventually be automated ways of turning it into a library, so the old code no longer needs to appear in the code that you are actually working on. Some automated solution seems like the kind of thing that the Clojure crowd will come up. I think it would only take me a day to write a macro to recreate namespaces from a library. Zach Tellman has some code that does half of this (Potemkin).
IMO, semantic versioning (at least the x.y part) makes sense for libraries but not for applications but this distinction is rarely made and sometimes blurry.
This post completely ignores one of the most important features of semver, which is dependency management. Being able to do this is really great:
some-library>=1.0.0,<2.0.0
It means I can include a library and get non-breaking changes and all security updates until the next major release without worrying that a change is going to break me randomly. It doesn't mean I don't need to do testing, but it makes it a lot more likely that I won't accidentally break in some way I didn't expect due to a change upstream.
When I want to upgrade to a breaking change release I know I need to go look at the release notes to see what the breaking change is, and then make modifications to my application.
If a breaking change is made in a minor or point release I can open a bug with the project so they can revert this change in the next minor or point release.
(edited from <=2.0.0 to <2.0.0, as this was my original intent :) )
Possibly if you have very few dependencies, and those also have very few dependencies it will work out OK. But in a lot of circumstances, this is a recipe for shooting your foot off. For example, if you are building a server application in Node and the total number of included packages can reach into the thousands. And then if you deploy to your server and for some reason it has to update NPM, suddenly you have a potential of any and all of those thousands of packages to break, or interact poorly, or whatever. Since you have specified that anything between 1.0.0 and 2.0.0 is perfectly fine, you have absolutely no idea what secret sauce is necessary to get a working application again. So your website is down for hours while you struggle to figure out a working configuration.
and your application breaks - then some-library's developer isn't following SemVer anyways, and you are in dependency hell. If some-library's developer doesn't care about compatibility, then it doesn't matter what versioning scheme you use as you will have to whip up that secret sauce anyways.
However, that your issue isn't a problem with SemVer, but a problem with NPM/node. SemVer is working as intended, but getting the "secret sauce" to get everything working can be confusing as you try and figure out the differences between "devDependencies" and "peerDependencies". If every module just managed their dependencies, it could be easier (on your development experience, but not on your hard disk and build times).
> If some-library's developer doesn't care about compatibility, then it doesn't matter what versioning scheme you use as you will have to whip up that secret sauce anyways.
It does. Semver implies semantics arbitrary versioning is not.
This. You define your loose dependencies, then you compile them down to a lock file that you use to test your service and you deploy with the completely locked down set of dependencies. When you want to upgrade your deps, you re-run the compilation and get updated libraries, which you test again. If you have floating deps in production you're doing it wrong.
IME this problem is particular to NPM, due to JavaScript's historically-anemic standard library which made a culture of microlibraries inevitable. The lesson is that programming languages should provide rich operations on their standard types so that users can avoid having to transitively depend on ten different versions of left-pad.
Dependency hell isn't unique to NPM, but it's also not unique to semver. The primary driving feature of semver is that it provides the framework for a social contract between library author and library user about how automatic upgrades can be performed. This works great until you the number of transitive dependencies is "in the the thousands", which I've never seen on any project save for Javascript projects.
If you deploy unknown and untested packages to production, semver is not your problem. The utter lack of testing is. When you actually push to production, your packages should be locked to what you tested.
This doesn't prevent the use of version ranges during development or testing.
Edit: To be clear, I do not work with Node. If this is standard practice for Node shops, that's terrible.
This often results in security patches not being pushed. IME most companies are awful when it comes to maintaining dependencies, things are all to often locked until someone adds a package.
I understand that view but I think that the reality is that for many customers, downtime is worse than a potential security breach. Customers would rather downtime than a definite breach, but the chance of a breach is not always as bad as definite downtime.
I'm not saying that view is necessarily good, but I think it's common. I also think it's quite common that people assume this is the client view and act accordingly.
I definitely think skipping testing with the goal of getting security patches out sooner is a terrible plan except maybe in the case of an active exploit. You can get both frequent patches and high availability.
There is also lots of Node software running in environments where downtime is unacceptable. And having uncontrolled dependency churn is a great way to break your CI and demoralize your team. I know I've spent way more time than I care to admit debugging failures in test and production because for some reason there was no subdependency freeze. It's the kind of problem that makes you very frustrated.
This discussion is based on a false dichotomy. Yes, shops can be bad at updating dependencies. That's a matter of culture. Technically, I shouldn't have to constantly scan my dependencies for updates, much less security updates. That's what CVEs and release notes are for. Not freezing the dependency tree in CI and production on account of security updates is a bad pattern.
Your problem has nothing to do with semver. The solution to your problem is to use your package manager's dependency freeze utility (npm shrinkwrap in your case), and also to select your dependencies carefully, so you don't end up with thousands of subdependencies (I know some npm packages suffer from dependency incontinence - this is a good reason not to use them).
> And then if you deploy to your server and for some reason it has to update NPM, suddenly you have a potential of any and all of those thousands of packages to break, or interact poorly, or whatever. Since you have specified that anything between 1.0.0 and 2.0.0 is perfectly fine, you have absolutely no idea what secret sauce is necessary to get a working application again.
That's not how you're supposed to use npm. You should never perform an npm update directly in production exactly because of what you describe.
The proper way to manage dependencies update in npm is to perform the dependencies' updates in your build process when building your pre-production/staging build and then use the shrinkwrap[1] command of npm to generate a file with all dependencies pinned for production. This way, for a given production deployment, you know exactly what version of which dependencies was being used and you can rollback easily if an update break something.
to defend the author here, I think they were very clear that semver may not make sense for a very specific type of CD, always-on application. The article supports the thesis as stated in text, but the title is too broad.
semver conveys information about a released product. If you're running an internal project that isn't being released to the outside world and that project isn't a library being used by other services in your environment, then you can number the project any way you want. In fact, you don't even need to number it (you can just use the git SHAs, for instance).
semver is useful even if you're releasing something that isn't a library because it can be used to provide information about backwards incompatibility in things like database schema, cache incompatibilities, configuration file incompatibilities, that'll cause a service to break if updated.
That's a fair counterpoint to the article but your example above only demonstrates the library case. You should provide an example that demonstrates the value of semver in a CD, always-on top-level application to give information about schema, cache and configuration incompatibilities.
To use the web-service example: if you present an API for others to call, even if you never version yourself, you probably still version your API. Other posts on that blog mention doing so.[1]
Well, if you release a ton of versions of your API, some of them are going to be easier to transition between than others. As soon as that occurs, you conceptually have at least major and minor versions, but in a url-based API instead of in code. Or you choose to not hint your consumers if moving from v23 to v24 is likely to be simple or not.
Ehhh... at best it argues that it's over-complicated for something that only ever has a single version running everywhere. Like a web service that you control, for which there will never be a way to request "use that old version of the code instead".[1]
Which is true. But then why even bother with YYMM.xxxx? Just use a single incrementing integer. Or none - nothing can ever go backwards, so callers are required to ignore the version in practice.
[1]: an API presenting an old external interface is something different - there's still only a single "version" of the code that's actually running.
Yeah okay. I think there is a very large group of developers who work in just such an environment: developing web applications that have a "production" version whether they are public internet or internal to companies where the previous version would literally _never_ be consumed. The reason to track a increment in that kind of environment is as a code to align feature requests to a particular release (usually by date), this is useful for all sorts of internal tracking questions like: was this feature I requested live by 4/1/2016.
I personally dislike using semvar for these instances (my own opinion only) because it feels overcomplicated as you stated. I also learned a long time ago that _any_ kind of date designator was a recipe for answering why the "April" release was actually live on June 1st ;). Recently, I've dropped back to Git SHA-1 which has alignment to a specific version of code released AND can be tracked by date and deploy tools.
So I agree with the thesis, disagree with the solution, but still think ryan_lane did not effectively refute it above.
Yeah, the blog's context isn't about libraries. And agreed, tons of developers work in environments like this.
I pretty much exclusively deal with SHAs too, since tons of tooling understands it, and there's no implied order just by looking at it - which is a good thing. A SHA might be out, but waiting to ramp up to 100%. It does tend to mean "end of (nearly) all dev work" though, which is the actually important part for day-to-day developing.
In the past I've used YYYYMMDDXX as a version number. In my experience there's no real problem with confusion about what the version number means as long as it's automatically generated so there's a clear definition. For me it was "feature merges to master, and master has passed tests". Nothing about actual releases at all, because the version of the code says nothing about what you do with it.
The version deployed is 20160215.2.142001 (on 2016-02-15, there were three deploys, with the latest build affecting them being #142001). There's your date, which is additionally free of the uncertainty whether you meant mm/dd/yyyy or dd/mm/yyyy. How is that incompatible with semver? (Nowhere does it say that the numbers have to be single digits. Integers.)
My biggest beef with semver is that it is pushed by people encouraging this concept that additions are always safe. Or the lie that people will do security and other updates on older versions.
By and large, especially in an org, if you make a change that requires another place to change, you should go ahead and make that change. Period. Thinking people can stay on the old version till they want to update is BS. They will update when they have to. And it is rarely easy, then.
I agree, SemVersion is a technical solution for what is often a human/corporate problem.
A good solution to this is to have an "edge" build that will upgrade dependencies and let you know about breakages early, but that still requires companies to acknowledge that all software needs maintaining.
It's more of a technical tool for implementing a certain human/corporate behavior. If your organization is actually following a versioned process, then semver-aware tools are really handy; if you're note, then those tools are at best irrelevant and at worst just get in your way.
It used to work that way, though; software in the 90s and early 2000s did have sane semantic versioning, updates were backported to earlier major versions, and revision updates generally didn't break anything in production.
I bet semantic versioning is mostly championed by older veterans who have worked in maintenance or systems administration roles and are trying to get the industry to return to a versioning system that made sense once upon a time.
For my part ... while I'd dearly love to see semantic versioning come back, along with a few other good habits from a long time ago, I've given up on that and now just accept that updates will break things sometimes and there's always something to update and so something will be broken most of the time and that's just how it is so charge accordingly.
I think there used to just be less things versioned. Conway's law is probably hiding here. If there is an org to maintain it, it will be maintained. If you have more things to maintain than you do org structures, something is not getting maintenance.
I half-agree with you, because at some point, software versioning and the notion of stable apis went all post-modern (especially with open source).
But we used to pay software vendors a lot of money to care about boring things like backwards-compatibility, legacy support, and etc. Semantic versioning as a standard is a way of applying social pressure to authors to do the 'dirty work', regardless of whether they are being paid to do so. And if they don't feel like it, the result is usually "too bad for you". There's no real good answer for this that I can think of. Nobody wants to pay for leftpad.js Enterprise Edititon.
A lot of people are still doing that, including Linux distributions.
Keeping earlier major versions alive and backporting security fixes is very useful.
I'm not sure I understand your argument. If someone's not going to backport a v2.1 fix to v1.6, why would they backport a v20170106-1009 fix to v201610-1005?
I'm not sure what you are saying. My argument is that it is a myth that most things will get only security updates.
That is, the version scheme States the point is to somehow be in a position to get only security updates. However, unless there is an org specifically for maintaining a library, your only hope of getting updates is to take all updates. So, to stay on the latest.
I realized too late for editing it in, but I do want to add that I do not think it is solely pushed by people encouraging this. I honestly think that most proponents of this are incredibly well intentioned.
Which just highlights, to me, that semver is ultimately good intentions masqueraded as a mechanism.
Have you got an example where pure addition breaks the existing compatibility?
> the lie that people will do security and other updates on older versions.
Semver does not say anything will be supported. You still have to know what's going on with your dependencies as far as big releases and patches go. It simply says that is old versions are still supported, they're updates will preserve compatibility.
> Have you got an example where pure addition breaks the existing compatibility?
With c/c++ additions can break binary compatibility can't they? Adding a field to a struct for instance will mean existing code that does a malloc for that struct won't allocate coorectly. I think there are a million other cases to consider too.
Normally in a library you'd provide functions that allocate and free structures defined by that library. That's how successful C libraries do it.
I think semver covers this pretty well actually. The public API is not just the function signatures, but basically anything that makes your API - whether it's code or documentation. So either:
- your structs in the library are opaque and the library handles all the memory management itself - addition of fields is backwards compatible, or
- your structs are open and your functions expect the structs from outside - addition of fields is not backwards compatible
The second one is still falls under "Major version X (X.y.z | X > 0) MUST be incremented if any backwards incompatible changes are introduced to the public API."
To make a comparison in a dynamic language: you've got public function `f(d)` where `d` is some dictionary with elements `a` and `b`. Now in the next version you start to require `c` - same signature in the code, but it's not backwards compatible. It's still a public API, even if defined by the documentation.
I don't do C++ normally, but the situation seems similar. There's http://wiki.c2.com/?PimplIdiom for opaque objects, so you don't have to make their members part of the public API. But for things you do want to make public it's the same solution - make it a your public API by documenting it. If you know it breaks compatibility, it requires major version bump.
And no, after compilation, I don't believe the symbol order matters.
Qt uses it extensively. For big, complex objects like QTextEdit it's probably the least of your concerns, but they do avoid it for simple value types like QPoint.
A good chunk of these questions I'm asking are because I half remember Qt's binary compatibility guidelines from when I did an internship at Trolltech many years ago :) A similar list is here: https://community.kde.org/Policies/Binary_Compatibility_Issu... . It looks like the virtual function restrictions are what I was talking about, that's where pure additions can break compatibility.
It's an issue if the methods on that object are short-lived enough that the overhead of indirection is significant.
It's also an issue if very performance-sensitive routines need to read data from the object. In which case they already know exactly what's in there, so "pimpl" hiding has no benefit in the first place.
Implementations should be hidden if they are likely to change. In other words, there are are non-obvious data or method members and/or it's likely that functionality will be added in the future. These are the high-level architectural situations, not the leaf level ones where you optimize out every cyle possible.
your structs are open and your functions expect the structs from outside - addition of fields is not backwards compatible
Sometimes it's possible to handle this case (if you've planned ahead) by including unused space within the original struct defintion that you can then repurpose later.
Linux's support for ELF symbol versioning makes maintaining binary compatibility of C libraries easier than perhaps any other platform or language. It's patterned after Solaris' ELF symbol versioning, but more powerful.
glibc uses it extensively, but alas few other people know about it. Theoretically languages, like Rust, which compile to ELF executables could leverage the capability. But none do.
There are a series of posts from a libvirt developer which explain some of the mechanics:
I personally don't bother with semantic versioning at all. Where ABI compatibility matters, I'll commit to maintaining backward ABI compatibility in perpetuity using symbol versioning.[1] And maintaining backwards API compatibility is something I try to commit to as a matter of course; if I can't then I make sure builds will break loudly.
I don't begrudge people using semantic versioning, I don't just don't think it's the best approach for most cases. It's just one of several crappy choices. But at least with C on Linux, ELF symbol versioning is a slam dunk when you're serious about the matter.
[1] I usually avoid depending on proprietary platform features. But this is the one area I'm comfortable and eager to lean on Linux.
> Have you got an example where pure addition breaks the existing compatibility?
Surely they're mutually exclusive terms. I think GP was talking about cases where the developer releases a change thinking it's just an addition, but it actually breaks compatibility.
Of course, that boils down to saying that software updates sometimes have bugs. What that has to do with semver, I have no idea.
Sorry for the very late response. I should have written it as "additions are safe." I am more reflecting on additions being more liability (commonly called "tech debt") that you have.
I think others covered the edge cases where additions literally break. The sibling that said I was likely referring to just plain bugs hits it pretty well, I think. I was not going for esoteric scenarios.
And yes, semver doesn't say anything about if a version is supported. However, it is a strong indication that it is. In a social expectations sort of way. If you don't plan on supporting old versions of what you are doing... use a new name for the new stuff. Nobody expects google guava to support apache commons, even though they are doing a lot of the same things. We do expect, however, that apache commons will make sure any bugs in old versions are dealt with.
IIRC, Amazon deprecates old versions after some time period. That solves the "no one wants to update" problem. It also shows clearly that when a major version will not be maintained regularly.
> Or the lie that people will do security and other updates on older versions.
counterpoint: rails. At least two versions are supported at any time, sometimes older versions still receive fixes for critical security issues. Rails follows semver and they're doing fairly well. I rarely see breakages from upgrading anything but major releases.
For a smaller org, sure, especially if you have a monolithic build that would break if you made a change like that.
For a larger org, it's basically impossible to do that if you want to make forward progress.
But very much agreed that it's usually not easy to upgrade farther down the road when many things have changed. I've been trying to spread a culture of forethought and not changing things for the sake of "making things pretty", but it's an uphill battle.
> Thinking people can stay on the old version till they want to update is BS. They will update when they have to.
You can just about get away with this within an organisation, but you cannot do it to your customers if you have a competitor which doesn't impose this workload on them.
I'd like to kill off a decade-old toolchain I have to target, but people are still using the hardware it's required for. They don't want to spend the money on replacements when the current hardware still works, and why should they?
> Being able to do this is really great: some-library>=1.0.0,<2.0.0
No, no, no, no it isn't great; it's terrible.
It's makes the version used non-deterministic, and that is a recipe for build hell. The whole reason we have lockfiles, shrinkwrap, etc. is because people are specifying version ranges.
You may lock library versions for a build pretty strictly. But the more loose is the coupling, the more elbow room you have in this regard, so you can upgrade your ngnix without breaking your wsgi apps, or your database without breaking anything — when done within a reasonable range of versions.
Let's just say it: a lot of web development is done by producing a blob of code and frameworks of the week, getting paid, and not caring about security and maintainability.
The poor practices and tooling are a consequence of this mindset.
You're still allowed to use lockfiles and shrinkwraps.
Anyone deploying an application to a server or distributing an application to end users should have the dependencies shrinkwrapped and the application tested with them.
Upgrading dependencies to get bug and security fixes in minor and point releases is as easy as deleting the shrinkwrap, installing the dependencies again, and making a new shrinkwrap. Then you put your application through all of its tests with the new dependencies, commit the new shrinkwrap, and then you can release/deploy a new version of the application.
Having a non-deterministic version when you build is terrible. So don't do that. Having deterministic build dependencies is also a goal, but a different goal.
Having a non-deterministic version, so you can easily upgrade all your dependencies and get the latest and greatest compatible version of everything, is great.
Based on reading this it seems like you haven't worked on a large application where there are lots of modules and libraries developed by independent teams all coming together to form a final product (shipping with CD or not).
I work on a platform team developing many different libraries used by many different teams throughout the company. If we didn't leverage semver (or some versioning scheme that at least differentiates between breaking and non-breaking changes) I don't know how we would do it. We either a) wouldn't be able to release 'patch' updates to a particular library without consumers getting it for free/automatically without changing anything or b) wouldn't be able to release breaking changes without it automatically breaking consumers builds.
Semver may not be useful for the final build or end product that you end up shipping. But it is a very useful tool for all the parts (dependencies) that make up that final product.
I feel like semver works great in the right context, for example if I'm building an API/Library which other deveopers are using, it helps them to know if there's been a major version bump, breaking change or just general patches.
However if it's just software for the end user, such as a web browser, it doesn't make as much sense, what is defined as a breaking change for someone browsing the web? And will they care? I've seen it be used for websites, ok great, but who's on the receiving end of these version change numbers?
At the moment everyone is trying to do semver even in situations when it's not needed.
As a developer I really like to know which version of a browser a user is running to be able to reproduce a specific bug. It does matter if they are using a five year old browser or the latest evergreen update of Firefox. Having the version number available somewhere (Help » About …) facilitates communication a lot.
Also, for corporate customers it is sensible to agree on the version range supported for a web browser if you are offering a SaaS solution, especially if security matters.
Sure, I'm not saying remove version numbers as a whole, more that semver isn't needed for something like a browser.
What you've described doesn't need semver specifically
For dependencies semantic versioning is great because it gives a reasonable basis of compatibility. I say "reasonable basis" because just because it's a minor/patch change doesn't mean it won't break your code. It just means it's not supposed to!
For end user software, i.e big $$$ enterprise software, there's no point in using it. Unless the app itself is a platform for other apps (in which case you're really an API/library anyway), it's beyond overkill. Heck it's downright confusing to most business users. Arguably any sizable UI change would be a "major version" because you moved a button across the screen. For that type of software I go for <major>.<minor> which are chosen to coincide with marketing campaigns.
SemVer is sometimes very annoying (pre-release and metadata rules specifically) but also just solves versioning. You have to jump through some hoops to use `git describe` in a sane way with it, but it's worth it.
Semantic versioning works great for software like Java. When you deliver an application with java, you want to ensure that the vast majority of your users have a compatible Java version. You also want to make sure that a new version of Java doesn't break your software. Semver encodes both these properties. Since the first number in Java's semantic version is always one, you know it is always backwards compatible. Whenever the second number increments, you know new features have been added that require a new VM. The last number changing is mostly irrelevant to most people, but it might signify a bug fix or optimization.
When both those pieces of information - backwards compatibility and the need to upgrade - are important, semantic versioning should be used. If you are comfortable removing features without leaving an out for users, and you can reliably upgrade all users at the same time, encoding this information is much less important.
Tangentially relatedly, this recent r/haskell thread [1] contains a lot of discussion on the differences between Haskell's Package Versioning Policy and SemVer, and particularly downsides of SemVer.
This is punishingly stupid. If you are working on a large enough project (many libraries, many developers, many organizations, daily or nightly builds because it takes 4 hours to run all the unit tests even in parallel due to dependencies) you use semantic versioning.
If you're some pissant shop writing CRUD apps then yeah, don't bother. As soon as you outgrow that stage, there won't be any more questions.
Monolithic standalone apps or tiny projects don't need semver. Practically all collaborative software in between benefits massively from it.
I feel like this is a bunch of hand waving and does not really address anything but saying because we have CD we do not need to care about versions. This shows a lack of experience in a solid development process in large scale code bases.
CD itself posits that a lot of what was previously assumed to be a necessary part of large scale software delivery is just cargo culting that gets in the way.
I feel like this is a bunch of hand waving and does not really address anything but saying because we have CD we do not need to care about versions. This shows a lack of experience in a solid development process in large scale code bases.
Semver is a sweet spot for libraries (tools, utilities, APIs, etc.). When a library I use isn't on board with Semver, I am often skeptical, and almost always disappointed.
For programming languages it is less clear to me that there are benefits. The same goes for living production end-services, which are the topic of the article. Said production system might (should?) be separate in version churn from any API built on top of said system, or client libraries used to interoperate with said API.
It is likely that I agree with Mr. Gillespie on the topic with respect to "always on" systems, but I do not feel strongly about that agreement? I tend to look for and focus on Semantic Versioning in its sweet spot, that is, libraries.
Having said where I agree, there was one quote of the article that stood out to me:
>it feels a little bit overly pedantic
In my experience, that's the best part, perhaps even the entire point, at least with libraries. Semver is just a method of communication, but its standardization and lack of ambiguity (relative to other systems), means that communication is all the more clear. I need a new feature and so am bumping a library to a new minor version? I'll be sure to check for other new things and for any deprecations, but won't need to worry that existing behavior was removed. Very clear.
When I am engineering a system on top of various libraries/tools, then growing and changing said system over time, I want to be able to pedantically assert what is known (as well as what is not known!) about the system's dependencies. Semver is a good tool for this job.
The clarity also makes it possible to build linters that check whether you accidentally broke your server guarantees (at least for detectable API changes). That's awesome.
Funny you should mention Windows - while only the major version is used, there's always a build number associated with a RTM version: 7.0.7601, 8.0.9200, 8.1.9600, 10.0.14393
;)
I agree that an average user only cares for the major component.
Given that continuous deployment doesn't have anything like traditional releases, so why bother with semantic versioning at all. You're typically just dealing dates or monotonically increasing build numbers. There's no reason for anything else. Public APIs (including libraries), have traditional releases, and so the distinction between backwards compatible and incompatible versions really do matter.
That said, a three part number isn't really useful. Just increment the minor version number in all not incompatible cases.
Also, there is an antipattern with releases, it's naming them. It's complete bullshit because opaque with rarely any rhyme or reason. It's branding plain and simple. If you need to upgrade anything, you end up having to googling what the actual version number or release history is and then going off of that. It's a complete waste of time, and so twee.
SemVer has nothing to do with CD. It's for package management. With a package or lib I don't need (multiple) daily increments as I would not update a lib I use that frequently.
It’s tricky. On the one hand, if you could RELY on sane definitions of “breaking changes”, it can be quite simple to set up flows that evolve quickly and safely based on little more than the versions of dependencies. On the other hand, by nature software has too many combinations to test completely so you can never be sure that a change is going to be non-breaking for YOU; and ultimately you are responsible for anything that does break.
I find the right thing to do is to set up a completely parallel “beta flow” that mirrors production almost exactly, allowing you to freely dump in new changes and see what happens. Have a way to swap working beta-flow changes into production, and a way to swap it back. Then you can do anything you want in beta.
Sometimes being in a silo warps your view of the world. While semver isn't necessarily the most optimal for a given type of development, and there are many alternative ways to version, it works pretty well for most all kinds of software development which is why it is a pretty good standard, no matter what software you end up doing it should work for you.
218 comments
[ 2.2 ms ] story [ 242 ms ] threadIf you are developing your own personal website with a static page and bootstrap, probably over kill.
We don't need to have a polarized opinion about everything.
Minor versions are things you reasonably might depend on like a new feature or significant bug fix. You _should_ be able to upgrade seamlessly though one person's bug fix is another's breaking change. I don't think it's reasonable to get rid of these and treat everything as a breaking change.
So I don't know what's simpler for libraries. Semver seems to have boiled it down pretty well.
Three-part is unambiguous.
I was very disappointed to learn that Italy did not, in fact, elect a Shiba Inu to office.
Previously discussed: https://news.ycombinator.com/item?id=13085952
Your comment is dead, but to answer the question - the alternative proposed is to encode the contract of your library in itself, and never break it, only extend it.
If you need to change the API, you don't change behaviour or signatures of your functions while preserving the names; you just add a completely new mainspace, while letting people use the old functions if they still have reason to. You don't have to keep actively supporting the old namespaces (i.e. no new bugfixes), but just need to not break the stuff that already worked.
There's some scenarios where this is hard - for example it's a web api and supporting it is costing you $$. But for things like that devs are generally aware that what they're consuming might get shut down at some point; so doing a new api, in a new namespace, and shutting down the old one at some point is not completely out of the question.
The problem of knowing whether 1.3 vs 3.7 was written 10 days ago or 10 years ago doesn't seem useful. Rarely has software I have written depended on how recent the library was released.
Ultimately, the talk tears into version numbers in general, but not semantic versioning - he doesn't discuss anything particular to SemVer. He could have been talking about Postgres' versioning conventions and the talk would still hold.
A) avoid breaking API changes
B) if you have to do large breaking API changes, that will probably affect alot of dependent projects, make it a new artifact / namespace / library that can live side-by-side with the old version
B is actually pretty common in large java library projects (rxjava -> rxjava 2, retrofit -> retrofit 2, dagger -> dagger ‡ all have independent maven artifacts and I think also packages) and imho this approach makes a lot of sense. It's also the more important part of this talk compared to his critique of semver.
I mean, it's not like people delete 1.3.0 from their packaging system when they release 2.0.0. Incrementing the major version number is semver's way of declaring a new namespace, and once declared, it lives alongside the old ones.
What is Hickey suggesting be done differently?
With that in place it is questionable what the actual advantage of semver vs a simpler monotonous versioning schema (like build number or UTC date) is.
He does talk explicitly about the trouble of making major changes in SemVer[0]. The gist of his argument was that minor changes in semver are relatively useless while major changes have a high probability of breaking your software. Major changes in semver are backwards incompatible and update the program's API in place. This leads to dowstream breakage and fear around doing upgrades.
> If I need to upgrade my application due to a security vuln, SemVer lets me know if I can just "drop-in" the upgrade, or if I need to work more.
I think the point he was trying to make was that upstream developers could change internals of a library but keep the API consistent so that downstream devs would never have to worry about scary updates. As you said with SemVer, if the security upgrade is a significant change, then you can expect breakage in the library. What he was advocating was patching issues like security vulns under the hood while keeping everything backwards compatible. Major upgrades could even add new namespaces, functions, and arguments but there's no real point to deleting or mutating old code, that just creates breakage. He wants software libraries to be immutable to take care of dependency issues and versioning to better reflect the changes made in the code.
[0]https://youtu.be/oyLBGkS5ICk?t=13m45s
https://youtu.be/oyLBGkS5ICk?t=30m6s
https://youtu.be/oyLBGkS5ICk?t=46m28s
In practice that's very uncommon. If someone is actually doing security releases, then either they release a minimal change to supported versions, or the distributions do that for them. Actual security upgrades are normally single patches which take great care not to do any API or behaviour changes.
Left unresolved is how to deal with the accumulation of old code. I assume there will eventually be automated ways of turning it into a library, so the old code no longer needs to appear in the code that you are actually working on. Some automated solution seems like the kind of thing that the Clojure crowd will come up. I think it would only take me a day to write a macro to recreate namespaces from a library. Zach Tellman has some code that does half of this (Potemkin).
some-library>=1.0.0,<2.0.0
It means I can include a library and get non-breaking changes and all security updates until the next major release without worrying that a change is going to break me randomly. It doesn't mean I don't need to do testing, but it makes it a lot more likely that I won't accidentally break in some way I didn't expect due to a change upstream.
When I want to upgrade to a breaking change release I know I need to go look at the release notes to see what the breaking change is, and then make modifications to my application.
If a breaking change is made in a minor or point release I can open a bug with the project so they can revert this change in the next minor or point release.
(edited from <=2.0.0 to <2.0.0, as this was my original intent :) )
/me displays closet full of T-shirts.
However, that your issue isn't a problem with SemVer, but a problem with NPM/node. SemVer is working as intended, but getting the "secret sauce" to get everything working can be confusing as you try and figure out the differences between "devDependencies" and "peerDependencies". If every module just managed their dependencies, it could be easier (on your development experience, but not on your hard disk and build times).
It does. Semver implies semantics arbitrary versioning is not.
That's what .lock files are for.
This doesn't prevent the use of version ranges during development or testing.
Edit: To be clear, I do not work with Node. If this is standard practice for Node shops, that's terrible.
Not taking security patches is bad. Taking down production is also bad.
I won't say it's ever good, but for most software some downtime isn't the end of the world, certainly better than having security exploits available.
I'm not saying that view is necessarily good, but I think it's common. I also think it's quite common that people assume this is the client view and act accordingly.
I definitely think skipping testing with the goal of getting security patches out sooner is a terrible plan except maybe in the case of an active exploit. You can get both frequent patches and high availability.
This discussion is based on a false dichotomy. Yes, shops can be bad at updating dependencies. That's a matter of culture. Technically, I shouldn't have to constantly scan my dependencies for updates, much less security updates. That's what CVEs and release notes are for. Not freezing the dependency tree in CI and production on account of security updates is a bad pattern.
Nodejs is the first time I've encountered a deployment problem whose resolution was "you have to upgrade your package manager to a beta version".
You know, this should be a hint that the problem is perhaps not with the versioning system.
That's not how you're supposed to use npm. You should never perform an npm update directly in production exactly because of what you describe.
The proper way to manage dependencies update in npm is to perform the dependencies' updates in your build process when building your pre-production/staging build and then use the shrinkwrap[1] command of npm to generate a file with all dependencies pinned for production. This way, for a given production deployment, you know exactly what version of which dependencies was being used and you can rollback easily if an update break something.
[1] https://docs.npmjs.com/cli/shrinkwrap
semver is useful even if you're releasing something that isn't a library because it can be used to provide information about backwards incompatibility in things like database schema, cache incompatibilities, configuration file incompatibilities, that'll cause a service to break if updated.
Well, if you release a ton of versions of your API, some of them are going to be easier to transition between than others. As soon as that occurs, you conceptually have at least major and minor versions, but in a url-based API instead of in code. Or you choose to not hint your consumers if moving from v23 to v24 is likely to be simple or not.
[1]: https://surfingthe.cloud/ideal-api-versioning/ is mentioned in the article, for example.
Which is true. But then why even bother with YYMM.xxxx? Just use a single incrementing integer. Or none - nothing can ever go backwards, so callers are required to ignore the version in practice.
[1]: an API presenting an old external interface is something different - there's still only a single "version" of the code that's actually running.
I personally dislike using semvar for these instances (my own opinion only) because it feels overcomplicated as you stated. I also learned a long time ago that _any_ kind of date designator was a recipe for answering why the "April" release was actually live on June 1st ;). Recently, I've dropped back to Git SHA-1 which has alignment to a specific version of code released AND can be tracked by date and deploy tools.
So I agree with the thesis, disagree with the solution, but still think ryan_lane did not effectively refute it above.
I pretty much exclusively deal with SHAs too, since tons of tooling understands it, and there's no implied order just by looking at it - which is a good thing. A SHA might be out, but waiting to ramp up to 100%. It does tend to mean "end of (nearly) all dev work" though, which is the actually important part for day-to-day developing.
By and large, especially in an org, if you make a change that requires another place to change, you should go ahead and make that change. Period. Thinking people can stay on the old version till they want to update is BS. They will update when they have to. And it is rarely easy, then.
A good solution to this is to have an "edge" build that will upgrade dependencies and let you know about breakages early, but that still requires companies to acknowledge that all software needs maintaining.
I bet semantic versioning is mostly championed by older veterans who have worked in maintenance or systems administration roles and are trying to get the industry to return to a versioning system that made sense once upon a time.
For my part ... while I'd dearly love to see semantic versioning come back, along with a few other good habits from a long time ago, I've given up on that and now just accept that updates will break things sometimes and there's always something to update and so something will be broken most of the time and that's just how it is so charge accordingly.
But we used to pay software vendors a lot of money to care about boring things like backwards-compatibility, legacy support, and etc. Semantic versioning as a standard is a way of applying social pressure to authors to do the 'dirty work', regardless of whether they are being paid to do so. And if they don't feel like it, the result is usually "too bad for you". There's no real good answer for this that I can think of. Nobody wants to pay for leftpad.js Enterprise Edititon.
That is, the version scheme States the point is to somehow be in a position to get only security updates. However, unless there is an org specifically for maintaining a library, your only hope of getting updates is to take all updates. So, to stay on the latest.
Which just highlights, to me, that semver is ultimately good intentions masqueraded as a mechanism.
> the lie that people will do security and other updates on older versions.
Semver does not say anything will be supported. You still have to know what's going on with your dependencies as far as big releases and patches go. It simply says that is old versions are still supported, they're updates will preserve compatibility.
With c/c++ additions can break binary compatibility can't they? Adding a field to a struct for instance will mean existing code that does a malloc for that struct won't allocate coorectly. I think there are a million other cases to consider too.
I think semver covers this pretty well actually. The public API is not just the function signatures, but basically anything that makes your API - whether it's code or documentation. So either:
- your structs in the library are opaque and the library handles all the memory management itself - addition of fields is backwards compatible, or
- your structs are open and your functions expect the structs from outside - addition of fields is not backwards compatible
The second one is still falls under "Major version X (X.y.z | X > 0) MUST be incremented if any backwards incompatible changes are introduced to the public API."
To make a comparison in a dynamic language: you've got public function `f(d)` where `d` is some dictionary with elements `a` and `b`. Now in the next version you start to require `c` - same signature in the code, but it's not backwards compatible. It's still a public API, even if defined by the documentation.
What about c++? Object allocation is always handled by new on the consumer side isn't it? Also in C, does the order of function declarations matter?
And no, after compilation, I don't believe the symbol order matters.
It's also an issue if very performance-sensitive routines need to read data from the object. In which case they already know exactly what's in there, so "pimpl" hiding has no benefit in the first place.
Implementations should be hidden if they are likely to change. In other words, there are are non-obvious data or method members and/or it's likely that functionality will be added in the future. These are the high-level architectural situations, not the leaf level ones where you optimize out every cyle possible.
Sometimes it's possible to handle this case (if you've planned ahead) by including unused space within the original struct defintion that you can then repurpose later.
It's hard, but certainly not hopeless.
glibc uses it extensively, but alas few other people know about it. Theoretically languages, like Rust, which compile to ELF executables could leverage the capability. But none do.
There are a series of posts from a libvirt developer which explain some of the mechanics:
I personally don't bother with semantic versioning at all. Where ABI compatibility matters, I'll commit to maintaining backward ABI compatibility in perpetuity using symbol versioning.[1] And maintaining backwards API compatibility is something I try to commit to as a matter of course; if I can't then I make sure builds will break loudly.I don't begrudge people using semantic versioning, I don't just don't think it's the best approach for most cases. It's just one of several crappy choices. But at least with C on Linux, ELF symbol versioning is a slam dunk when you're serious about the matter.
[1] I usually avoid depending on proprietary platform features. But this is the one area I'm comfortable and eager to lean on Linux.
Really? Do you ever call `malloc(4)` instead of `malloc(sizeof(int))` or `malloc(sizeof(*ptr))`?
Surely they're mutually exclusive terms. I think GP was talking about cases where the developer releases a change thinking it's just an addition, but it actually breaks compatibility.
Of course, that boils down to saying that software updates sometimes have bugs. What that has to do with semver, I have no idea.
I think others covered the edge cases where additions literally break. The sibling that said I was likely referring to just plain bugs hits it pretty well, I think. I was not going for esoteric scenarios.
And yes, semver doesn't say anything about if a version is supported. However, it is a strong indication that it is. In a social expectations sort of way. If you don't plan on supporting old versions of what you are doing... use a new name for the new stuff. Nobody expects google guava to support apache commons, even though they are doing a lot of the same things. We do expect, however, that apache commons will make sure any bugs in old versions are dealt with.
counterpoint: rails. At least two versions are supported at any time, sometimes older versions still receive fixes for critical security issues. Rails follows semver and they're doing fairly well. I rarely see breakages from upgrading anything but major releases.
For a larger org, it's basically impossible to do that if you want to make forward progress.
But very much agreed that it's usually not easy to upgrade farther down the road when many things have changed. I've been trying to spread a culture of forethought and not changing things for the sake of "making things pretty", but it's an uphill battle.
You can just about get away with this within an organisation, but you cannot do it to your customers if you have a competitor which doesn't impose this workload on them.
I'd like to kill off a decade-old toolchain I have to target, but people are still using the hardware it's required for. They don't want to spend the money on replacements when the current hardware still works, and why should they?
No, no, no, no it isn't great; it's terrible.
It's makes the version used non-deterministic, and that is a recipe for build hell. The whole reason we have lockfiles, shrinkwrap, etc. is because people are specifying version ranges.
You may lock library versions for a build pretty strictly. But the more loose is the coupling, the more elbow room you have in this regard, so you can upgrade your ngnix without breaking your wsgi apps, or your database without breaking anything — when done within a reasonable range of versions.
Let's just say it: a lot of web development is done by producing a blob of code and frameworks of the week, getting paid, and not caring about security and maintainability.
The poor practices and tooling are a consequence of this mindset.
Libraries should depend on ranges (though they can also pin internally for development).
Anyone deploying an application to a server or distributing an application to end users should have the dependencies shrinkwrapped and the application tested with them.
Upgrading dependencies to get bug and security fixes in minor and point releases is as easy as deleting the shrinkwrap, installing the dependencies again, and making a new shrinkwrap. Then you put your application through all of its tests with the new dependencies, commit the new shrinkwrap, and then you can release/deploy a new version of the application.
Having a non-deterministic version, so you can easily upgrade all your dependencies and get the latest and greatest compatible version of everything, is great.
I work on a platform team developing many different libraries used by many different teams throughout the company. If we didn't leverage semver (or some versioning scheme that at least differentiates between breaking and non-breaking changes) I don't know how we would do it. We either a) wouldn't be able to release 'patch' updates to a particular library without consumers getting it for free/automatically without changing anything or b) wouldn't be able to release breaking changes without it automatically breaking consumers builds.
Semver may not be useful for the final build or end product that you end up shipping. But it is a very useful tool for all the parts (dependencies) that make up that final product.
However if it's just software for the end user, such as a web browser, it doesn't make as much sense, what is defined as a breaking change for someone browsing the web? And will they care? I've seen it be used for websites, ok great, but who's on the receiving end of these version change numbers? At the moment everyone is trying to do semver even in situations when it's not needed.
Also, for corporate customers it is sensible to agree on the version range supported for a web browser if you are offering a SaaS solution, especially if security matters.
"That version has Flash, that version does not; doesn't make sense to the user to tell them apart." Riiight.
For end user software, i.e big $$$ enterprise software, there's no point in using it. Unless the app itself is a platform for other apps (in which case you're really an API/library anyway), it's beyond overkill. Heck it's downright confusing to most business users. Arguably any sizable UI change would be a "major version" because you moved a button across the screen. For that type of software I go for <major>.<minor> which are chosen to coincide with marketing campaigns.
When both those pieces of information - backwards compatibility and the need to upgrade - are important, semantic versioning should be used. If you are comfortable removing features without leaving an out for users, and you can reliably upgrade all users at the same time, encoding this information is much less important.
[1] https://www.reddit.com/r/haskell/comments/5iok3l/haskell_pac...
If you're some pissant shop writing CRUD apps then yeah, don't bother. As soon as you outgrow that stage, there won't be any more questions.
Monolithic standalone apps or tiny projects don't need semver. Practically all collaborative software in between benefits massively from it.
For programming languages it is less clear to me that there are benefits. The same goes for living production end-services, which are the topic of the article. Said production system might (should?) be separate in version churn from any API built on top of said system, or client libraries used to interoperate with said API.
It is likely that I agree with Mr. Gillespie on the topic with respect to "always on" systems, but I do not feel strongly about that agreement? I tend to look for and focus on Semantic Versioning in its sweet spot, that is, libraries.
Having said where I agree, there was one quote of the article that stood out to me:
>it feels a little bit overly pedantic
In my experience, that's the best part, perhaps even the entire point, at least with libraries. Semver is just a method of communication, but its standardization and lack of ambiguity (relative to other systems), means that communication is all the more clear. I need a new feature and so am bumping a library to a new minor version? I'll be sure to check for other new things and for any deprecations, but won't need to worry that existing behavior was removed. Very clear.
When I am engineering a system on top of various libraries/tools, then growing and changing said system over time, I want to be able to pedantically assert what is known (as well as what is not known!) about the system's dependencies. Semver is a good tool for this job.
I really enjoy how the package-manager for the Elm programming language automates semantic versioning by diffing your apis based on the type system.
I think a bit part of the real criticism of SEMVER stems from this inability to either automate it or use it correctly.
Hickey's discussion of the subject obviously has a lot more great points to it, and even when I don't agree, I can't help to enjoy listening :)
Enterprise software sells for even more money, and the above is even more true, usually backed up with lawyers, PR campaigns, etc.
Windows 7, 8, and 10, sound to the average person more reasonable that:
(from the article): 1612.0009, 1612.0013, 1701.0253
Thus, they are not only a rock solid pattern, they are the bloody unwritten standard and more so everybody's daily expectation.
I agree that an average user only cares for the major component.
That said, a three part number isn't really useful. Just increment the minor version number in all not incompatible cases.
Also, there is an antipattern with releases, it's naming them. It's complete bullshit because opaque with rarely any rhyme or reason. It's branding plain and simple. If you need to upgrade anything, you end up having to googling what the actual version number or release history is and then going off of that. It's a complete waste of time, and so twee.
The article is just bizarre. It's like ranting about how knives suck because they can't easily transport rocks.
I find the right thing to do is to set up a completely parallel “beta flow” that mirrors production almost exactly, allowing you to freely dump in new changes and see what happens. Have a way to swap working beta-flow changes into production, and a way to swap it back. Then you can do anything you want in beta.