It would be helpful if people would use asciinema (they can be easily paused/resumed and the full ptty text is copy/pasteable) or even just webm (smaller than their equivalent GIF and seekable).
I guess those aren't easily embedded in a GitHub readme though. Alas.
Manul is one of the very limited set of tools which provide idiomatically go-gettable projects without need of vendoring tool be installed on the user machine.
Putting your dependencies in a folder called "vendor/" is standard, but how you get them there isn't.
There's a collection of tools for managing that, like this one , and others like "godep" and "glide". Which one you use for that doesn't seem to be settled.
Yes. It's a terrible mess. But as of GopherCon a couple of weeks ago, the way forward is becoming clear. Most important, the Go Team is now in direct communication with the authors of the most widely used vendoring tools, and it seemed to be widely acknowledged that this is a severe problem, that a standard is needed, and that even things like "go get" might have to change. I'm very hopeful.
I'd much rather them forget reverse compatibility and fix the core problem once and for all. I personally like gb and the ability for each project to be its own GOPATH.
Unfortunately, the discussion took place on the "Hack Day", and wasn't recorded (I think they're going to try to change that for next year). Chris Broadfoot took pretty comprehensive notes, though: https://docs.google.com/document/d/1xMJ0c-YxvcgNglzjbALzncs5...
I've found that git submodules rapidly become an unwieldy mess of dependencies that make developing and deploying applications and project quite difficult. Many vendoring solutions have started off using git submodules for convenience but they are almost always supplanted by a more robust solution e.g. composer, bundler, etc. I feel that it's a mistake to utilise git submodules for this reason and move straight onto a solution which provided true dependency resolution.
If you're not a fan of submodules, but are in need of a this kind of tool, I'd recommend looking at `glide` [1] . I've had much better experiences, as a user, with `glide` than other tools, and it supports package aliasing which is really helpful. (Package aliasing allows for satisfying an import path with a private fork of a package, without having to rewrite all the import statements inside my fork.)
I thought vendoring means, you are safe even if the original repo was removed from GitHub.
This tool still looks nice for a Wrapper to handle submodules for you in a Go way. But still you are only safe, if you fork and keep your fork up to date and "vendor" your fork using Manul.
This is true but it's only a problem if you don't have a copy of your repository checked out anywhere. You can always make a copy of the repository later as long as you have at least one checkout.
I'm using subtrees to vendor C dependencies and it works well. Submodules are problematic as they may disappear or the git repo might be down. If your dependency is read only, subtree is a better option IMO.
"Pros" are listed, but "cons" are not. While evaluating git submodules as a possible solution for vendoring at my work, I found the following disadvantages with this model:
- you can use only git dependencies (no bzr, hg, ...); though most stuff is on github nowadays (especially with closure of Google Code), still not necessarily everything. With pure submodules you're locking yourself in the git monoculture.
- you must trust your dependencies will not ever disappear (see "the leftpad problem", or the mentioned closure of Google Code).
That said, all approaches have pros and cons. Though I admit I usually don't like advertisements which only show advantages and don't list downsides.
Another con is that submodules will be missing in the tarballs that Github autogenerates for tags and releases. I initially used submodules for my own Go binaries, but moved to a custom vendoring solution since then.
Also, it is not possible to express constraints (eg. follow 2.x tags). A manifest file is handy for that.
Personally, I like glide better since it takes a lot from Ruby bundle. I commit only glide.yaml and glide.lock in the repository. It is not flawless either as each operation triggers an update of all dependencies.
I've used git subrepo [1] for vendoring and been quite happy with it. Doesn't seem to have the problems of git submodule and it's easier to use than git subtree.
If you depend on A and B and both depend on C, you now have the npm2 style heirarchy of nested dependencies.
As with all 'use submodules' solutions, the devil is in the detail with the nested heirarchy of dependencies and version resolution.
I mean, fwiw, its quite nice for simple 1-level-deep dependencies, but this isnt a simple problem. What if A -> C@1.1.5, and B -> C@1.1.9. Do you install both? Only one?
You might still be able to solve it somehow, if the submodules are required to have semantic version tags on their versions, you parsed the entire hierarchy and then resolved conflicts (somehow), based on the semver semantics... I guess?
That's outside the scope of this tool. Go has already adopted the npm style submodules that you mentioned (I.e. Answer = you install both versions in their respective vendor/ dirs).
This tool just tries to help you manage that. Not that I'm a big fan, either way, but still.
> Some copy the source code of dependencies into the vendor directory:
I don't agree with any of the stated problems there. Unless you are building a library you should commit your dependencies and not rely on third party repos being available.
I can't stress enough how useful govendor [1] is if you do Go professionally.
Does the go toolchain allow you to freeze your dependencies to specific versions like pip, etc? I think that could be preferable to committing all your dependencies.
The go toolchain doesn’t but many of these vendoring tools do something similar. But you still don’t want to depend on the origin repo being up and have missing/altered repos break your builds.
Copying the source code of the dependencies into your repo isn't the only alternative to relying on third party repositories, though.
You could fork the dependencies that you work with into locally managed repos and use submodules to include them in your own project. That way you can easily update your dependencies without breaking everything and also maintain their commit history.
That's how we started back in the day. It works but setting up everything and doing the rebase/commit dance to update can be quite cumbersome when you have more than a couple dependencies. Nothing wrong with it if you are willing to do the work though.
Glide isn't ideal either. It gets confused about vendored dependencies by 3rd party libraries and tries to download them separately.
Also, it's invented it's own format for the deps and lock files instead of trying to standardise on the ones already out there.
So much for reproducible builds. What if the internet connection flakes on your build server. Or if the source to the library is deleted from wherever you're pulling it from.
Why the reluctance to add ALL SOURCE REQUIRED TO BUILD YOUR APP to your source repository. I don't get it.
Also, no matter what vendor tool you use, please use one such that I can "go get" your command (main package).
Although we're really at the point where I don't care what tool is used, so long as the community can pick one.
> Why the reluctance to add ALL SOURCE REQUIRED TO BUILD YOUR APP to your source repository. I don't get it.
That would make the Git history incredibly messy to work with and makes the repository much larger in size, making work slower. This kind of "monorepos" are known to be a pain in the ass with Git.
However, Git submodules is pretty much adding all the required source code (or at least pointers to) in your repo, and pinning them to a specific version.
The network connectivity issue is solved by having local mirrors.
There are interop tools to/from Git to pretty much all the popular version control tools, including Mercurial. They might not work out of the box with Git submodules, but you should be able to set up a mirror of the original repo in Git.
Erm, as a developer if I use a package manager that requires me to mirror the original repo then I don't think I'm going to use it at all. I love the idea of git submodules for handling dependencies, but the assumption here is that all packages use git. This is a minor, but definitely a fallacy when it comes to using some of the packages from Google, mainly legacy.
It's a good idea to be doing mirroring anyway if your business depends on it. Otherwise the original repo disappearing (or corrupted otherwise) or network going down is going to hit your productivity. And things are faster with mirroring.
For projects where all of the dependencies are git, I use git-freeze[0] to vendor into submodules. Works great, and no external package manager dependencies for consumers.
`go get` has to change for proper vendoring in Go. Right now its such a mess. Go needs a bundler like functional. e.g go bundle which will read a Gemfile like file to install your external packages.
I hope this comes before the end of the year. This has been put on the back burner for too long and I can understand GC was priority but now GC is very good.
As someone who has thought Go was a poor choice for almost any project for a long time and argued that many times, I used to come up against the argument that Go package management was superior to other languages. Six months ago I said:
> Packaging and deployment aren't problems in Go OR Python if you're only solving trivial problems.
> With a larger project, you might have an easier time packaging and deploying Go code, but that's largely because there are no libraries to package. Admittedly packaging can be a pain in Python, but that's usually because of poor choices in dependencies. Packaging Python with a few mature dependencies isn't hard in my experience. It's the projects where some idiot has pulled in every 0.x versioned library in pip that are hard to package. When Go has as wide a variety of libraries as Python people will run into the same problems in Go.
> You could argue that at least for now Go doesn't allow you to shoot yourself in the foot that way, but I'd rather have the option.
> I'm not defending Python in particular here. I'd say the same things I've said here about any mature language with extensive libraries.
> If you're espousing Go because of easy packaging and deployment, I strongly suggest that you consider whether that's actually a feature Go will have long term, and whether you're currently paying for that feature by having no libraries available to you. The only lesson I would take away from Go's easy packaging is to only use mature dependencies that pull their weight.
And in a different post:
> give it a decade and Go packaging will be just as miserable as packaging in any other language.
I've been coming up against that argument a lot less lately. It looks like my prediction is coming true a bit ahead of schedule.
I'm still hoping for all the great ideas from nix [1] and guix [2] to catch on.
Package manager shouldn't be that thing, that tries to integrate into existing fragile environments. It should be the thing that manages those environments, that manages all the compilers and libraries in a reliable and reproducible way, with rollbacks and everything.
63 comments
[ 3.1 ms ] story [ 129 ms ] threadI guess those aren't easily embedded in a GitHub readme though. Alas.
There's a collection of tools for managing that, like this one , and others like "godep" and "glide". Which one you use for that doesn't seem to be settled.
I haven't found anything on the conference site.
I've found that git submodules rapidly become an unwieldy mess of dependencies that make developing and deploying applications and project quite difficult. Many vendoring solutions have started off using git submodules for convenience but they are almost always supplanted by a more robust solution e.g. composer, bundler, etc. I feel that it's a mistake to utilise git submodules for this reason and move straight onto a solution which provided true dependency resolution.
[1]: https://github.com/Masterminds/glide
(see: http://blogs.atlassian.com/2013/05/alternatives-to-git-submo... and https://codingkilledthecat.wordpress.com/2012/04/28/why-your... )
- you can use only git dependencies (no bzr, hg, ...); though most stuff is on github nowadays (especially with closure of Google Code), still not necessarily everything. With pure submodules you're locking yourself in the git monoculture.
- you must trust your dependencies will not ever disappear (see "the leftpad problem", or the mentioned closure of Google Code).
That said, all approaches have pros and cons. Though I admit I usually don't like advertisements which only show advantages and don't list downsides.
Personally, I like glide better since it takes a lot from Ruby bundle. I commit only glide.yaml and glide.lock in the repository. It is not flawless either as each operation triggers an update of all dependencies.
[1] https://github.com/ingydotnet/git-subrepo
If you depend on A and B and both depend on C, you now have the npm2 style heirarchy of nested dependencies.
As with all 'use submodules' solutions, the devil is in the detail with the nested heirarchy of dependencies and version resolution.
I mean, fwiw, its quite nice for simple 1-level-deep dependencies, but this isnt a simple problem. What if A -> C@1.1.5, and B -> C@1.1.9. Do you install both? Only one?
You might still be able to solve it somehow, if the submodules are required to have semantic version tags on their versions, you parsed the entire hierarchy and then resolved conflicts (somehow), based on the semver semantics... I guess?
This tool just tries to help you manage that. Not that I'm a big fan, either way, but still.
> Some copy the source code of dependencies into the vendor directory:
I don't agree with any of the stated problems there. Unless you are building a library you should commit your dependencies and not rely on third party repos being available.
I can't stress enough how useful govendor [1] is if you do Go professionally.
[1] https://github.com/kardianos/govendor
You could fork the dependencies that you work with into locally managed repos and use submodules to include them in your own project. That way you can easily update your dependencies without breaking everything and also maintain their commit history.
Why the reluctance to add ALL SOURCE REQUIRED TO BUILD YOUR APP to your source repository. I don't get it.
Also, no matter what vendor tool you use, please use one such that I can "go get" your command (main package).
Although we're really at the point where I don't care what tool is used, so long as the community can pick one.
That would make the Git history incredibly messy to work with and makes the repository much larger in size, making work slower. This kind of "monorepos" are known to be a pain in the ass with Git.
However, Git submodules is pretty much adding all the required source code (or at least pointers to) in your repo, and pinning them to a specific version.
The network connectivity issue is solved by having local mirrors.
Not ideal, but not a deal breaker either.
Edit: IMO it's a deal breaker
And yes, it's annoying, but a workaround exists.
[0] https://github.com/nicerobot/git-freeze
I hope this comes before the end of the year. This has been put on the back burner for too long and I can understand GC was priority but now GC is very good.
> Packaging and deployment aren't problems in Go OR Python if you're only solving trivial problems.
> With a larger project, you might have an easier time packaging and deploying Go code, but that's largely because there are no libraries to package. Admittedly packaging can be a pain in Python, but that's usually because of poor choices in dependencies. Packaging Python with a few mature dependencies isn't hard in my experience. It's the projects where some idiot has pulled in every 0.x versioned library in pip that are hard to package. When Go has as wide a variety of libraries as Python people will run into the same problems in Go.
> You could argue that at least for now Go doesn't allow you to shoot yourself in the foot that way, but I'd rather have the option.
> I'm not defending Python in particular here. I'd say the same things I've said here about any mature language with extensive libraries.
> If you're espousing Go because of easy packaging and deployment, I strongly suggest that you consider whether that's actually a feature Go will have long term, and whether you're currently paying for that feature by having no libraries available to you. The only lesson I would take away from Go's easy packaging is to only use mature dependencies that pull their weight.
And in a different post:
> give it a decade and Go packaging will be just as miserable as packaging in any other language.
I've been coming up against that argument a lot less lately. It looks like my prediction is coming true a bit ahead of schedule.
Package manager shouldn't be that thing, that tries to integrate into existing fragile environments. It should be the thing that manages those environments, that manages all the compilers and libraries in a reliable and reproducible way, with rollbacks and everything.
[1] https://nixos.org/nix/ [2] http://www.gnu.org/software/guix/