The v2 library suffix on the package path seems rather hackish.
Feels like it would have been better to use a different delineating character instead of a slash to make it abundantly clear at a glance that it's effectively a major version tag and not a subpackage.
Other than that, seems relatively straightforward. Hope this is the last "new and completely different" attempt to tackle dependency management in Go.
It's kinda both. The point is, that major versions should (by the philosophy of vgo) effectively be different packages and be handled like different packages.
I personally like that philosophy, but I wonder if it's in Go's best interests to ignore the desires of the people who don't like, or perhaps aren't even aware of, that reasoning.
Particularly the fact that it only applies after the second version. If we're going to require /v2, /v3, etc - shouldn't we also require /v1 for the first?
Also it's unclear how initial development (as per server) is to be handled. If the first few versions are v0.1.0, v0.2.0, etc. - how are these handled on the import paths.
What if a package URL actually contains "/v2" at the end, but this isn't actually referring to the version and should be used when fetching the package. Seems very odd. Agree that it should have been more like "@2".
What if you want to include two different minor versions, for example, as you can include two different major ones. Is that possible/allowed?
Requiring v1 would break backwards compatibility. If you create a new library, feel free to do that. The point is to treat different major versions as different packages, enough obviously lets you use multiple major versions in your project.
v0 and v1 are treated as the same package, you're supposed to use v0 as long as you're adapting to vgo / potentially doing breaking changes and use v1 when you know you'll keep a stability guarantee.
As I mentioned in the sibling comment, it's all described in Russ's blog post series, long, but well worth the read: https://research.swtch.com/vgo
> If the first few versions are v0.1.0, v0.2.0, etc. - how are these handled on the import paths.
This was answered in the post. It's handled in the mod file and not via import path. Import path only changes when there is breaking changes, ie: a major bump and a new /v# for the import path.
I agree. It seems to be based on having quite specific information about the structure of the package path based on the service it references. For instance, I assume that github.com/user/v2 will refer to version 0 or 1 of the 'v2' repository. When I wrote my own vgo-inspired package manager[0] I changed the handling of major versions in package paths such that they are separated by an @-sign. While github.com/user/v2@v3 still looks silly, I think it is less confusing.
I was very surprised at the way they decided to go for `v2`. It seems kind counter to the Go philosophy so far of simplicity, predictability, etc. Now I can't just look at the imports list and apply a very simple mental rule to know what names are in scope - there's this hacky exception which magically changes the rule for very specific name schemes.
If you're implying that "import foo" always puts foo in scope, that's never been true -- the package name doesn't need to match the last element of the path. Granted, you can count on it being true in 90% of cases, but I think "/v2" will quickly become unsurprising, the same way we expect "import go-foo" to add the name foo, not go-foo.
Do I have the ability to somehow specify "use git+ssh for this dependency" with the new modules system?
Right now it seems near impossible with go to do that other than manually cloning the repositories into the correct path. We can't host our things publicly and have to use SSH to clone the repositories at my company.
It is especially frustrating in our CI/CD process if we need to manually clone our packages for setting everything up.
I currently use submodules inside of vendor/ and set each dependency's remote to the SSH URL. You can then simply use git clone --recursive to get everything, including via SSH.
That being said, I share your hope that this can remain smooth in the new Go modules implementation. $GOPATH is always a hurdle for new-comers to wrap their heads around in my experience.
That's one thing that frustrated me about dep. go get is deficient in this area too. I understand the need to namespace packages, but requiring them to be hosted (or have metatags on a page) at the location the import path specifies in order to be able to pull them down is insanity.
To make matters worse, dep tried to stuff too much of a DSL into the package specification on the command line. example.com/path/pkg@hashish made it impossible to specify git@example.com/path/pkg as the location because the parser wasn't robust enough, and the package location parser wasn't/isn't smart enough to honor ssh://git@example.com/path/path as a way to be explicit about how you wanted this done.
dep did work for our use case if you edited the toml file directly, once I made a 2 character change to a regular expression in v0.3.0. We use dep and stopped upgrading with that version; I'm hoping go modules make non-public repos easier, but I'm not holding my breath.
> Go’s module system should not have to give consideration to the particular transport you want to use.
> Its references are canonical and it’s up to you to set up the relevant process for it to retrieve the source for those references.
Git has a proper notion of references; it properly separates transport from references of remotes. Go uses just `https://` links, and expects the website to have a single `<meta>` tag containing a vcs clone URL (i.e., transport) to use. VCSes have had separation and multiplicity of transport for a long time, but Go will deal with only one URL (i.e., transport), with no way for the user/distributor to specify preferences otherwise.
For example, everything from git.kernel.org to github.com allows the user to chose which URL to clone with. The remote is not supposed to know or dictate a single transport.
I think `go get`'s limited method of transport discovery is just a hack that got released into production and stagnated in its form. It was a perfectly fine hack for public repos (on the internet or on Google's intranet), but it just never got any features (let alone documentation) specifically for repos that need an authenticated way to clone them. The fact that git has a nifty way to rewrite URLs is just a luck.
My entire point is that VCSes support multiple transports since before Go came about. I never claimed Go should support HTTP and SSH only. In fact, I never claimed it should support either. I claimed it shouldn't force the VCS host to chose for the user which clone URL (and thus transport) to use.
It’s hardly magic. It’s a defined feature of git. If one couldn’t avail himself of the defined features of git, then there’d be no reason to build on top of it in the first place.
> If one couldn’t avail himself of the defined features of git, then there’d be no reason to build on top of it in the first place.
Sure build on top of it, but don't expect your users to implicitly know the defined, yet obscure, features of git you build on.
> It’s hardly magic.
I didn't claim this is a magic feature in git. I meant the way of `go get`-ing a private library by letting `go get` try and clone from https URLs while silently changing them to git:// URLs in a completely separate layer under it is magic.
How is the first application developer to know this git (not go) feature exists in the first place? The application developer is looking to use `go get`, and is frustrated that `go get` only downloads via https, asking them for a username and password combo every time. `go get` has no documentation to make it use git URLs.
Only when the application developer looks online for a solution to this do they find some StackOverflow post detailing this workaround. Or other workarounds, like the insecure option of telling git to save your https credentials.
Sure, a knowledgeable application developer can put whatever quirks their build system needs in their documentation; but a developer ignorant of these workarounds shouldn't have to go beyond `go get`'s documentation in the first place.
Unfortunately, the GOPATH structure makes it harder to mix golang projects with other languages when you need to. And it does not support package versions. I think the time has come to say goodbye to GOPATH.
Does not mean of course that include paths are better.
I'm not sure I see the difference. $GOPATH can be a list, all this tells you is that C and C++ libraries come from various sources (system, compiler, package manager, …), you could have the exact same crap in your system-provided GOPATH if there was such a thing.
Well, you have a system-provided GOPATH, in GOROOT (on my system, the stdlib source is in /usr/lib/go/src). But you just surprised me: I never knew that GOPATH is a list.
Isn't it a little odd that these are called modules when they sound very much like packages? In many languages, a module is just a source code file that can be imported. Why is Go redefining this term?
Go's terminology matches java, and is the exact reverse of your comments: a package is a namespace, a module is a bundle of 1..n namespaces (packages).
I don't think there's a universal definition (as showed by the other replies to you, saying, actually it's the oppisite...) I think you just have to learn the meaning language by language. Here is someone asking about the nuance in Python:
The only reason is to be backwardly compatible with the Go community calling what they have at the minute packages. It is foreign to D developers too where packages are at the top level, then modules, etc.
go mod edit -replace example.com/repo@version=example.com/fork@version
It seems like the second command doesn't understand branches, so you'll need to specify a "pseudo-version" like "v0.0.0-20171125154426-754f7301a386". Or maybe my go1.11 is just out of date.
Happens automatically when you run 'go mod init' in a directory containing Gopkg.lock. Caught me by surprise the first time as I had no idea it did that :)
So, if I want to use the new code in v2 for my v1 legacy consumers, would I let v1 (or a new v1.1) depend on v2, and wrap the new hi-method passing in a default parameter of "en" for language?
Or is my only "supported" option to push consumers to the new major version - with the possible bit-rot of the old code?
[ed: that is manual backport fixes/improvements to a v1.x version]
Russ has explicitly written that yes, redefining v1 in terms of v2 is not only the way to go, but in the future it might help automatise the switch to v2 with the "go fix" command.
Recently found a command line library written in go that was so useful to me in a python webapp I'm building, I decided to just wrap the go app and call it from a python subprocess. It is a bit slow to do it this way, but premature optimization, and all that. Anyhow, the bottom line is due to this I was forced to get a handle on Go this week so I could at least build the go library and make a few changes as needed. Wow, was I absolutely shocked to find out current package management in this go app is just pulling libraries from master on github. It really made me feel like go is far behind python in maturity. Pulling all these dependency libs from master definitely does not feel production ready. Maybe this will help resolve things.
The fact that new go developers do that tells a lot about the state of documentation, tutorials, tooling etc. A language and its ecosystem should encourage best practices from the start. Not as some kind of late afterthought.
Yeah thats the problem they have been looking to solve for a couple years now, but for what its worth the https://github.com/golang/go/wiki has information on dep tooling. My experience is that most people don't read it. Any tutorial worth its salt will mention this as well. Go has made it possible to pull right from master on deps fairly easily so many new developers do that so they don't have to learn dependency management. Proper dependency management is a skill that must be learned in any language, and often separates an entry developer from a mid level one.
There are many languages that make it explicitly difficult to pull directly from master. Nodejs packages are near impossible to install directly from a repository, you always have to go through npmjs.com. Haskell too (before stack became widespread), but even now most people install packages from hackage/stackage, where packages are strongly versioned.
This is where opinions will differ, I quite like go's ability to pull straight from a repository, and see outcomes like this as an unfortunate side effect of that feature. If a developer doesn't understand proper git and package versioning they will have a hard time in any language producing quality software. I'm not sure I want to give up the speed and ease of `go get` just to protect entry developers.
> This is where opinions will differ, I quite like go's ability to pull straight from a repository
Having the ability to do so does not mean it should be the easiest and simplest way to get from A to B.
Both pip and cargo can pull from repositories, but it's not the default and it's basically not covered in any tutorial. It's easy to do, but a developer is unlikely to find the information before they're actively looking for it.
> If a developer doesn't understand proper git and package versioning they will have a hard time in any language producing quality software.
I don't understand why you're using Go if your teaching methodology is to throw live chainsaws at people and berate them for not understanding proper power tool safety, why are you not just cutting out the middleman and using C++?
clearly you just don't like Go, which is fine. Of all the languages I've picked up Go was far and away the simplest and most productive, which is why I continue to use it. When I wrote python we were constantly using pip to pull from repos, so moving to Go I liked their repository centric view. Just because python recommends pip, doesn't mean junior devs don't totally botch dependency management with it. Dep management and git are core skills developers need to learn well to produce good software. There is simply no substitute for a bit of elbow grease on the matter. I'm not berating anyone, software is hard and being good at it means taking the time to learn some core tools.
Well at least you're not trying to defend the indefensible, even if you refuse to be honest about it.
> When I wrote python we were constantly using pip to pull from repos, so moving to Go I liked their repository centric view.
That is a rather peculiar situation, I believe in the 15 years I've been using Python I've had one dependency (and a rather outdated one at that) I had to pull from a repository.
> Just because python recommends pip, doesn't mean junior devs don't totally botch dependency management with it. Dep management and git are core skills developers need to learn well to produce good software.
I obviously agree with that complete platitude, the question is whether you think people learn better and are more likely to do so if you trick them and lay down traps for them. And whether you think your amusement is worth some out entirely.
> software is hard and being good at it means taking the time to learn some core tools.
And how is making those core tools harder to use and more likely to be misuse conducive to that?
You've been breaking the site guidelines, such as by being uncivil in comments here. That's not ok. Would you please review https://news.ycombinator.com/newsguidelines.html and use HN as intended from now on? We'd appreciate it.
This is a poor characterization of the issue. First of all, "do the packages come from git?" is not a real concern (similarly, "but it came from Pypi!" is not itself a feature). The real concerns are all "what reproducibility/security guarantees does my package manager provide?". For `go get` and vanilla `pip install`, the answer is "mostly none, but this is more or less okay for newbies/prototyping--it's okay though because the packaging docs point them to the better-but-tedious things". Cargo and Go modules do a pretty good job about providing reproducibility and usability at the same time, Python's answer seems to be `pipenv` which makes some pretty great promises, but has been quite buggy/quarelsome for us so far (and it's not even "official" as far as I can tell).
To characterize Go's ecosystem (but not Python's) as "throwing live chainsaws at people and berating them..." is pretty blatant dishonesty, doubly so once Go modules makes its official debut.
I disagree with your characterization. `go get` is the right tool for new developers (hell, I've been using Go for 7 years and I still use `go get` for the initial phase of my projects); they don't need reproducible builds, they need to install third party libraries and compile code. The current official reproducible build tooling in Go is vendoring, and that's quite unnecessarily painful for the things new developers need to do. Fortunately, Go 1.11 will be out soon with support for modules, so new developers can learn how to install code with the added bonus of not needing to learn a new set of tools down the road when they need to care about reproducibility.
I would say the state of Go dependency management is somewhere between what GP implies: "everyone is using go get to run things directly from master and that's terrible" and what you imply: "everyone has standardized around dep and that's fine".
I picked 5 "important" Go projects without checking first, and here's the solutions they use:
Kubernetes: The deprecated tool Godep + a pile of Make and Bash (which is the Go-est thing I've ever heard in my life)
Docker (Moby): vndr
Hugo: dep
Etcd: A bit of a mish-mash of dep and vndr, but mostly dep.
Cobra: Doesn't vendor dependencies, isn't a reproducible build (which is somewhat okay, as Cobra is primarily a library, not a tool in and of itself, but it does also have a CLI that probably breaks a lot).
In short, things are not currently fine. Dep is an okay tool, but fails for some use cases, and the community has not really rallied around it. Lots of important projects are sticking with what they've got. Glide, gvt, vndr, and Godep all remain important.
We're deluding ourselves if we discard an outsider's view that Go's dependency management situation is a dumpster fire, because it is. However, to GP, it isn't quite as bad as you suggest, most projects using Go have found some way or another of getting reproducible builds, and don't just run everything from tip of everyone else's master.
I am cautiously optimistic that modules will finally solve this mess, but we'll see to what extent they win in the marketplace of dependency management solutions.
Note that the Kubernetes people tried to migrate to Dep, but weren't successful. As I recall, it was due to several blocking issues (e.g. see https://github.com/kubernetes/client-go/issues/7).
I was likewise shocked at how good go compared to python in this regard.
Resolve your deps once, at build time, and you're done.
Compare that to dealing with the externalized deps that is deploying a python app on a machine that might have the correct version of python and all dependencies, or not. virtualenv? good luck. You end up bringing in docker and other tools just to deal with it. But what do you do if the machine is locked down by corporate IT and they don't allow docker or the installation of other python deps?
We use pipenv, but it's been buggy and quarelsome so far. I like what it promises to do, but so far my experiences with Go modules have been better (to my surprise, given how much newer Go modules are than pipenv).
good? go package management is awful. it makes tools like searching through a git repo a chore because all of the source code for your module is in the repo too. the OP was pointing out how far behind go is behind something like python.
if you really wanted to, you could so something exactly like go is doing in python and create a virtual environment with all of your packages and version of python and ship it. there are also ways to build binaries from python.
if your machine is locked down by corporate IT, you use a virutal environment
> Resolve your deps once, at build time, and you're done.
I have ... rarely had this experience.
Every go package manager has found new and frankly fascinating ways to lead me into a corner where I think I have committed and pushed working code, but which causes highly-visible CI barfing.
I should add as an aside: I've done two tours of duty as a Cloud Foundry Buildpacks maintainer. Everyone's package system sucks, but some suck a lot less.
* Maven is basically sane if used with levelled starters. Otherwise it's a preview of fighting the Many-Tentacled Ones of the Deeply Nested Dependency Graph using nothing but a dull butter knife held upside-down.
* Gradle is the worst allergic reaction to XML in history.
* NPM used to have amazing bugs and missing features but has laboriously, slowly improved.
* Python is about whether you hit the sweet spot of the particular packaging system you use.
* PHP is a mess, unless you use Composer, in which case you're still stuck with the weird reality that PHP's package system is really a mix of in-process modules which need to be compiled and slabs of plain old PHP.
* .NET Core had what seems like a deliberate policy to come up with the most confusing naming and versioning scheme humanly possible and was wildly successful at doing so. Oh, and no canonical reference for versions. None. Apparently this improved but it was hell on earth.
* Golang. A new package manager every ten minutes. A different vendoring model, different assembly model, different bugs, every damn week.
Only Bundler is basically sane. We hit lots of corner cases but for the 20/80 cases it essentially worked.
I think the point above was that once the binary is built you don’t have to worry about maintaining the exact same libraries on the server where they might be competing/mixing with various incompatible versions of shared libraries for Python or Ruby or whatever.
That said, yes this is a shameful hole in the golang world (caused by the project leads’ inside-Google myopia) but I’m glad they are finally addressing it at this level of integration. And despite a few oddities it looks pretty solid.
My view is that if it can't be built from scratch in a cleanroom environment -- ie, in CI or a buildpacks stager -- it can't be built. This goes for container images too.
The amount of "works on my machine" and "where's Gerald? this module only builds on his laptop" I saw as a consultant was depressing.
Speaking of containers, though: they solved the contending-interpreters problem quite handily. Cloud Foundry has used containerisation since before Docker existed and it really shines for buildpack usage.
We started with Heroku's buildpacks, which had to deal with the nightmare of a pre-container world and showed all the pain of it. There was a great deal of stuff we just didn't have to care about and we later rewrote the bulk of the buildpacks to jettison all the stuff we didn't need.
It's all come full circle: Heroku now use containerisation and both Heroku and Cloud Foundry will soon be adopting jointly-developed buildpacks-lifecycle code.
I would agree except I find Maven to be easily the most sane and flexible and extensible build tool out there. I've used Maven to consistently build and deploy vast systems made up of hundreds of libraries. And this was ten years ago. Maven's only "flaw" is that it doesn't support incremental builds and, frankly, I've come to believe this isn't a flaw at all and incremental builds are an inherently unpredictable anti-pattern. (That might just be the Maven trauma speaking. There are tools out there like buck that claim to get incremental builds down perfectly but I have doubts and we've seen some strange bugs in buck builds.)
What I find so strange is how the systems that came long after Maven -- particularly NPM and .NET -- that they could be so much worse than Maven. Somehow these people weren't aware of the importance of reproducible builds... despite that being a "thing" in Maven almost 20 years ago. Then, as noted above, you have go devs building off of git master branches.
All of this suggests a real problem in software engineering. There's a distinct lack of knowledge sharing that does not lead to gradual, linear progress. It's quite disconcerting. Things are not getting better, every day, after all.
These days I appreciate Maven much more than I did at first (ugh, XML, lolenterprisejava amirite??).
But the whack-a-dep thing is a real, genuine painpoint. Spring Boot Starters make this fairly close to invisible, but there are still billions of lines of Java code where the POM is like the map of an incredibly fragile ceasefire between 813 variants of some 118 different dependencies at 18 levels of nesting.
Like Game of Thrones, except bloodier, more complicated and much more likely to cause everyone to give up.
Why can’t google just maintain a npm.js style repo for go packages (together with a package manager program)? And a proper simple to read manifest file for a go programmer to list out external dependencies with their versions
I really don’t see these git based hacks can improve situation much.
It worked out for the time being, but npm destroyed the illusion. I don't even see the problem. A package manager could rely completely on git with the advantage to not have an single authority that isn't giving a flying fuck about quality and security of the hosted packages. Yes it forces you to dig out a good package, evaluate it and look at the track record of the devs. Someone has to do it, but it is currently hardly happening.
Either you or I don't understand. Of course Git is a decentralized VCS but how would you make use of a software module without offering a repo on some server? As someone remarked in answer to another of your comments: Github is centralized as well - but it's doesn't specialize in offering versioned binary artifacts for ease use in a software project.
And if you rely on one of my packages and I delete it from github or make it private, your build stops working, right?
Everyone was up in arms about npm allowing this a few months back. Rightfully so, I think, but I feel like it's going to be easy to hit this issue when pulling straight from github.
I don’t understanding the distinction you’re making. Most package managers I’ve used can point at mirrors. Even NPM can point at arbitrary local git branches, file paths, and remote locations. So “centralization: bad, git: good” seems like an irrelevant point.
As someone who has programmed a number of languages and currently mostly programs either Ruby or Go I have to say I would greatly prefer a Rubygems/bundler type system to the one Go seems to be heading for.
"We want to make it possible, at some future point, to introduce a shared proxy for use by the Go community, similar in spirit to those used by Rust, Node, and other languages. At the same time, the design must work well without assuming such a proxy or registry."
You can have a decentralised package manager that has support for a centralised package listing. This is what Nimble (Nim's package manager) does, there is a packages repo[1] which simply maps package names to Git/Hg repo URLs. So you can easily depend on URLs or package names.
Of course, even this can be considered decentralised since different package repos can be used easily. This is especially useful for private packages.
Does vgo miss a nuance in the semantic versioning of 0.x.x branches in that any bump to minor or patch allows a breaking change? Specifically point 4 of the spec. Some projects or libraries have a huge psychological barrier to releasing a 1.0.0. React comes to mind when they finally went from 0.14.x to 15.0.0.
---
4. Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable.
Go developers made a reasonable choice. Unstable API is just another way to say that there is no API. And with zero API any change is compatible and cannot break anything. So there is no need for v0.
If there is no API, how can the library be used? Every function a library exposes is part of the library's API, so for a library to have no API, it cannot expose any function.
> Unstable API is just another way to say that there is no API
No, it doesn't mean there is no API, it means there is no compatibility promise; it means there is an API per version, and the relationship between them i s not known.
> And with zero API any change is compatible and cannot break anything
That would be true, but this isn't "zero API", it's "zero promise of API compatibility"
That makes your statement "With zero API compatibility any change is compatible"...
That's very obviously wrong.
Please don't jump through these bizarre logic hoops to try and justify what go does here.
If I remember right, in vgo v0.x is supported and breaking changes are allowed during that time, and v0 can be graduated to v1 without an import path change. So that seems to be in agreement with the semver spec.
105 comments
[ 4.6 ms ] story [ 174 ms ] threadFeels like it would have been better to use a different delineating character instead of a slash to make it abundantly clear at a glance that it's effectively a major version tag and not a subpackage.
Other than that, seems relatively straightforward. Hope this is the last "new and completely different" attempt to tackle dependency management in Go.
Check out Russ's blog post series: https://research.swtch.com/vgo
Also it's unclear how initial development (as per server) is to be handled. If the first few versions are v0.1.0, v0.2.0, etc. - how are these handled on the import paths.
What if a package URL actually contains "/v2" at the end, but this isn't actually referring to the version and should be used when fetching the package. Seems very odd. Agree that it should have been more like "@2".
What if you want to include two different minor versions, for example, as you can include two different major ones. Is that possible/allowed?
v0 and v1 are treated as the same package, you're supposed to use v0 as long as you're adapting to vgo / potentially doing breaking changes and use v1 when you know you'll keep a stability guarantee.
As I mentioned in the sibling comment, it's all described in Russ's blog post series, long, but well worth the read: https://research.swtch.com/vgo
This was answered in the post. It's handled in the mod file and not via import path. Import path only changes when there is breaking changes, ie: a major bump and a new /v# for the import path.
Upgrade minor or patch to most recent:
Upgrade patch to most recent: Upgrade to specific version:[0]: https://futhark.readthedocs.io/en/latest/package-management....
Right now it seems near impossible with go to do that other than manually cloning the repositories into the correct path. We can't host our things publicly and have to use SSH to clone the repositories at my company.
It is especially frustrating in our CI/CD process if we need to manually clone our packages for setting everything up.
That being said, I share your hope that this can remain smooth in the new Go modules implementation. $GOPATH is always a hurdle for new-comers to wrap their heads around in my experience.
To make matters worse, dep tried to stuff too much of a DSL into the package specification on the command line. example.com/path/pkg@hashish made it impossible to specify git@example.com/path/pkg as the location because the parser wasn't robust enough, and the package location parser wasn't/isn't smart enough to honor ssh://git@example.com/path/path as a way to be explicit about how you wanted this done.
dep did work for our use case if you edited the toml file directly, once I made a 2 character change to a regular expression in v0.3.0. We use dep and stopped upgrading with that version; I'm hoping go modules make non-public repos easier, but I'm not holding my breath.
Its references are canonical and it’s up to you to set up the relevant process for it to retrieve the source for those references.
I get the impression that that’s what the proxy stuff is about - you’ll just set up a proxy which deals with retrieving the code.
> Its references are canonical and it’s up to you to set up the relevant process for it to retrieve the source for those references.
Git has a proper notion of references; it properly separates transport from references of remotes. Go uses just `https://` links, and expects the website to have a single `<meta>` tag containing a vcs clone URL (i.e., transport) to use. VCSes have had separation and multiplicity of transport for a long time, but Go will deal with only one URL (i.e., transport), with no way for the user/distributor to specify preferences otherwise.
For example, everything from git.kernel.org to github.com allows the user to chose which URL to clone with. The remote is not supposed to know or dictate a single transport.
I think `go get`'s limited method of transport discovery is just a hack that got released into production and stagnated in its form. It was a perfectly fine hack for public repos (on the internet or on Google's intranet), but it just never got any features (let alone documentation) specifically for repos that need an authenticated way to clone them. The fact that git has a nifty way to rewrite URLs is just a luck.
I didn't claim Go supports only git.
> so HTTP and SSH aren't the only options.
My entire point is that VCSes support multiple transports since before Go came about. I never claimed Go should support HTTP and SSH only. In fact, I never claimed it should support either. I claimed it shouldn't force the VCS host to chose for the user which clone URL (and thus transport) to use.
Sure build on top of it, but don't expect your users to implicitly know the defined, yet obscure, features of git you build on.
> It’s hardly magic.
I didn't claim this is a magic feature in git. I meant the way of `go get`-ing a private library by letting `go get` try and clone from https URLs while silently changing them to git:// URLs in a completely separate layer under it is magic.
That's what documentation is for. Every application should be documenting (e.g. in README.md) how it can be built.
Only when the application developer looks online for a solution to this do they find some StackOverflow post detailing this workaround. Or other workarounds, like the insecure option of telling git to save your https credentials.
Sure, a knowledgeable application developer can put whatever quirks their build system needs in their documentation; but a developer ignorant of these workarounds shouldn't have to go beyond `go get`'s documentation in the first place.
git config --global url.ssh://git@bitbucket.org/.insteadOf https://bitbucket.org/
Hooray!
Holy hell, it's eight!
Does not mean of course that include paths are better.
I agree with the GP that go package/module are similar to Java package/module
Thankfully the whole split-package nonsense will eventually go away, since the JPMS doesn’t allow a package to reside in multiple modules.
That doesn't seem weird to me. Lots of languages can split their namespaces across packages.
https://stackoverflow.com/questions/19198166/whats-the-diffe...
So Go can do what it wants as long as it is clear and you can find its definitions easily.
1. How does one use a branch of a dependency in go.mod? (scenario: local dev where api & impl repos are separated)
2. How does one use branch from a fork in go.mod? (scenario: fixed bug in someone else's repo and now want to use my fork)
Or is my only "supported" option to push consumers to the new major version - with the possible bit-rot of the old code?
[ed: that is manual backport fixes/improvements to a v1.x version]
See https://research.swtch.com/vgo-import, section "Automatic API Updates".
https://golang.github.io/dep/docs/Gopkg.lock.html
Having the ability to do so does not mean it should be the easiest and simplest way to get from A to B.
Both pip and cargo can pull from repositories, but it's not the default and it's basically not covered in any tutorial. It's easy to do, but a developer is unlikely to find the information before they're actively looking for it.
> If a developer doesn't understand proper git and package versioning they will have a hard time in any language producing quality software.
I don't understand why you're using Go if your teaching methodology is to throw live chainsaws at people and berate them for not understanding proper power tool safety, why are you not just cutting out the middleman and using C++?
clearly you just don't like Go, which is fine. Of all the languages I've picked up Go was far and away the simplest and most productive, which is why I continue to use it. When I wrote python we were constantly using pip to pull from repos, so moving to Go I liked their repository centric view. Just because python recommends pip, doesn't mean junior devs don't totally botch dependency management with it. Dep management and git are core skills developers need to learn well to produce good software. There is simply no substitute for a bit of elbow grease on the matter. I'm not berating anyone, software is hard and being good at it means taking the time to learn some core tools.
Well at least you're not trying to defend the indefensible, even if you refuse to be honest about it.
> When I wrote python we were constantly using pip to pull from repos, so moving to Go I liked their repository centric view.
That is a rather peculiar situation, I believe in the 15 years I've been using Python I've had one dependency (and a rather outdated one at that) I had to pull from a repository.
> Just because python recommends pip, doesn't mean junior devs don't totally botch dependency management with it. Dep management and git are core skills developers need to learn well to produce good software.
I obviously agree with that complete platitude, the question is whether you think people learn better and are more likely to do so if you trick them and lay down traps for them. And whether you think your amusement is worth some out entirely.
> software is hard and being good at it means taking the time to learn some core tools.
And how is making those core tools harder to use and more likely to be misuse conducive to that?
To characterize Go's ecosystem (but not Python's) as "throwing live chainsaws at people and berating them..." is pretty blatant dishonesty, doubly so once Go modules makes its official debut.
I picked 5 "important" Go projects without checking first, and here's the solutions they use:
Kubernetes: The deprecated tool Godep + a pile of Make and Bash (which is the Go-est thing I've ever heard in my life)
Docker (Moby): vndr
Hugo: dep
Etcd: A bit of a mish-mash of dep and vndr, but mostly dep.
Cobra: Doesn't vendor dependencies, isn't a reproducible build (which is somewhat okay, as Cobra is primarily a library, not a tool in and of itself, but it does also have a CLI that probably breaks a lot).
In short, things are not currently fine. Dep is an okay tool, but fails for some use cases, and the community has not really rallied around it. Lots of important projects are sticking with what they've got. Glide, gvt, vndr, and Godep all remain important.
We're deluding ourselves if we discard an outsider's view that Go's dependency management situation is a dumpster fire, because it is. However, to GP, it isn't quite as bad as you suggest, most projects using Go have found some way or another of getting reproducible builds, and don't just run everything from tip of everyone else's master.
I am cautiously optimistic that modules will finally solve this mess, but we'll see to what extent they win in the marketplace of dependency management solutions.
Godep, dep, glide, make files, you name it. it's a total dumpster fire.
I haven't written Go in a couple years, but I lol'd. This exactly what we used to do, except replace Godep with glide.
Resolve your deps once, at build time, and you're done.
Compare that to dealing with the externalized deps that is deploying a python app on a machine that might have the correct version of python and all dependencies, or not. virtualenv? good luck. You end up bringing in docker and other tools just to deal with it. But what do you do if the machine is locked down by corporate IT and they don't allow docker or the installation of other python deps?
Unless you're using CGO, in which case you still have to make sure your target environment has the shared libraries your bindings will make calls to.
if you really wanted to, you could so something exactly like go is doing in python and create a virtual environment with all of your packages and version of python and ship it. there are also ways to build binaries from python.
if your machine is locked down by corporate IT, you use a virutal environment
I have ... rarely had this experience.
Every go package manager has found new and frankly fascinating ways to lead me into a corner where I think I have committed and pushed working code, but which causes highly-visible CI barfing.
I should add as an aside: I've done two tours of duty as a Cloud Foundry Buildpacks maintainer. Everyone's package system sucks, but some suck a lot less.
* Maven is basically sane if used with levelled starters. Otherwise it's a preview of fighting the Many-Tentacled Ones of the Deeply Nested Dependency Graph using nothing but a dull butter knife held upside-down.
* Gradle is the worst allergic reaction to XML in history.
* NPM used to have amazing bugs and missing features but has laboriously, slowly improved.
* Python is about whether you hit the sweet spot of the particular packaging system you use.
* PHP is a mess, unless you use Composer, in which case you're still stuck with the weird reality that PHP's package system is really a mix of in-process modules which need to be compiled and slabs of plain old PHP.
* .NET Core had what seems like a deliberate policy to come up with the most confusing naming and versioning scheme humanly possible and was wildly successful at doing so. Oh, and no canonical reference for versions. None. Apparently this improved but it was hell on earth.
* Golang. A new package manager every ten minutes. A different vendoring model, different assembly model, different bugs, every damn week.
Only Bundler is basically sane. We hit lots of corner cases but for the 20/80 cases it essentially worked.
That said, yes this is a shameful hole in the golang world (caused by the project leads’ inside-Google myopia) but I’m glad they are finally addressing it at this level of integration. And despite a few oddities it looks pretty solid.
My view is that if it can't be built from scratch in a cleanroom environment -- ie, in CI or a buildpacks stager -- it can't be built. This goes for container images too.
The amount of "works on my machine" and "where's Gerald? this module only builds on his laptop" I saw as a consultant was depressing.
Speaking of containers, though: they solved the contending-interpreters problem quite handily. Cloud Foundry has used containerisation since before Docker existed and it really shines for buildpack usage.
We started with Heroku's buildpacks, which had to deal with the nightmare of a pre-container world and showed all the pain of it. There was a great deal of stuff we just didn't have to care about and we later rewrote the bulk of the buildpacks to jettison all the stuff we didn't need.
It's all come full circle: Heroku now use containerisation and both Heroku and Cloud Foundry will soon be adopting jointly-developed buildpacks-lifecycle code.
Of course, "Gerald" is an anagram for "Gradle".
What I find so strange is how the systems that came long after Maven -- particularly NPM and .NET -- that they could be so much worse than Maven. Somehow these people weren't aware of the importance of reproducible builds... despite that being a "thing" in Maven almost 20 years ago. Then, as noted above, you have go devs building off of git master branches.
All of this suggests a real problem in software engineering. There's a distinct lack of knowledge sharing that does not lead to gradual, linear progress. It's quite disconcerting. Things are not getting better, every day, after all.
But the whack-a-dep thing is a real, genuine painpoint. Spring Boot Starters make this fairly close to invisible, but there are still billions of lines of Java code where the POM is like the map of an incredibly fragile ceasefire between 813 variants of some 118 different dependencies at 18 levels of nesting.
Like Game of Thrones, except bloodier, more complicated and much more likely to cause everyone to give up.
I really don’t see these git based hacks can improve situation much.
Everyone was up in arms about npm allowing this a few months back. Rightfully so, I think, but I feel like it's going to be easy to hit this issue when pulling straight from github.
"We want to make it possible, at some future point, to introduce a shared proxy for use by the Go community, similar in spirit to those used by Rust, Node, and other languages. At the same time, the design must work well without assuming such a proxy or registry."
https://research.swtch.com/vgo-module
[0]: https://github.com/gomods/athens
Of course, even this can be considered decentralised since different package repos can be used easily. This is especially useful for private packages.
1 - https://github.com/nim-lang/packages
---
4. Major version zero (0.y.z) is for initial development. Anything may change at any time. The public API should not be considered stable.
https://semver.org/#spec-item-4
No, it doesn't mean there is no API, it means there is no compatibility promise; it means there is an API per version, and the relationship between them i s not known.
> And with zero API any change is compatible and cannot break anything
That would be true, but this isn't "zero API", it's "zero promise of API compatibility"
That makes your statement "With zero API compatibility any change is compatible"...
That's very obviously wrong.
Please don't jump through these bizarre logic hoops to try and justify what go does here.