It can be used for that. You can configure the tag you want to fetch and build if the repository is version-controlled, effectively allowing you to follow an external project: update your local settings when bumping your requirements, and it should follow along.
That seems basically like you create a specification for what versions of which repos you want and it will create it.
Is that any different to just having a shell-script curl the repos and versioning that script?
What I had in mind would be effectively external dependencies that would appear within the source tree when cloned/manifested/whatever which would themselves be version-controlled.
It would effectively be a 'curl' or 'clone' of an external version of a child repo maintained by the VCS.
I think Nix does/comes close to what you're after.
Nix 'packages' are just text files describing the build/install process, so they can be version controlled (e.g. there's a big repo at https://github.com/nixos/nixpkgs ).
Dependencies, build environments, etc. are fully specified by the package and isolated from each other, and there is an emphasis on reproducibility, so building/installing a particular package should always give the same results. (This isn't enforced, but a non-reproducible package is considered buggy, and there are tools to check if a package can be reproduced)
Nix packages themselves can reference external dependencies, like git repos and URLs, and usually provide a fixed checksum to compare the output against; if it doesn't match, the build is aborted. Wrappers/translators are also available to integrate with other package manager ecosystems, e.g. cabal/hackage for Haskell packages.
Results are cached locally (packages are only built once, until they get garbage collected) and remotely (if a build server has already built a package, you can fetch it from there instead).
For development environments, you can write a Nix package for your project then run the 'nix-shell' command to enter its build environment (i.e. all dependencies available, environment variables set, etc.)
It works ok for me with docker images. It only works for languages storing dependencies in the environment (e.g. python), so it won't work as well for C++.
I can download any docker image. Docker image contains all external dependencies and I can inspect those dependencies.
I assign a version to each Dockerfile. Docker image version is in the source control. Image with each version is stored in docker image registry. Version bump triggers update of all packages (either system level or language packages). This lets me revert external dependencies to state from any point in time. Or check out any commit from source control and run it in the same state as it was running in prod at that time.
I think you're describing Homebrew. Actually whenever I'm building and installing something in ~/.local (which I do fairly frequently if I don't install it via apt-get, I'm not a /usr/local fan), I wish I were using something like Homebrew. Just something to manage whats in ~/.local, so I can remove things one package at a time. It'd be nice to have a lightweight version of checkinstall for non-root usage, for example.
I think mulle-bootstrap can do this for you using ".bootstrap/embedded_repositories".I would try it out, and if it doesn't work out and you think it's almost there but just needs a little fix, then let me know,
There are a number of issues/annoyances with the mechanism, especially when working with local modifications, but nothing to make me eager to add the anguish of bash+CMake onto the existing toil of CMake.
(Arguably this also bootstraps a build environment, whereas CMake requires itself and an SDK in order to bootstrap, but at the point where a build environment needs to be scripted/reproducible containerization seems preferable)
I remember being told the same in a project/company in Holland, Europe. Never have been told again.
Maybe being strict worldwide (as soon as something is published to internet), or as soon as there is Intellectual Property Agreement, they are required lines.
I use to avoid them in each single file for personal projects, unless it's a convention (for example in perl, license in POD format at the end of the CPAN class, is a community practice).
My suggestion is to put a single line copyright with the year in every file, like
// (c) 2016 txutxu
and a license inside the project. The default copyright license protects you from unwanted copies.
But when it comes to IP rights, a lot of practices are actually voodoo legalities: people do them out of habit and paranoia, not because it has ever been useful in court.
28 comments
[ 3.0 ms ] story [ 54.2 ms ] threadWe already have our source in version-control.... we just need a way of updating and maintaining a repository with external dependencies.
Do any such tools exist?
Ultimately its basically a set of external dependencies that you want to version-control and occasionally update.
I see no fundamental difference to what version-control is, its just not a feature built into most/any.
One such tool:
https://github.com/LLNL/spack
That seems basically like you create a specification for what versions of which repos you want and it will create it.
Is that any different to just having a shell-script curl the repos and versioning that script?
What I had in mind would be effectively external dependencies that would appear within the source tree when cloned/manifested/whatever which would themselves be version-controlled. It would effectively be a 'curl' or 'clone' of an external version of a child repo maintained by the VCS.
Does that make sense?
Nix 'packages' are just text files describing the build/install process, so they can be version controlled (e.g. there's a big repo at https://github.com/nixos/nixpkgs ).
Dependencies, build environments, etc. are fully specified by the package and isolated from each other, and there is an emphasis on reproducibility, so building/installing a particular package should always give the same results. (This isn't enforced, but a non-reproducible package is considered buggy, and there are tools to check if a package can be reproduced)
Nix packages themselves can reference external dependencies, like git repos and URLs, and usually provide a fixed checksum to compare the output against; if it doesn't match, the build is aborted. Wrappers/translators are also available to integrate with other package manager ecosystems, e.g. cabal/hackage for Haskell packages.
Results are cached locally (packages are only built once, until they get garbage collected) and remotely (if a build server has already built a package, you can fetch it from there instead).
For development environments, you can write a Nix package for your project then run the 'nix-shell' command to enter its build environment (i.e. all dependencies available, environment variables set, etc.)
Isn't this effectively equivalent to version-controlling a shell script that contains a set of 'curls' or 'clones'?
What I had in mind would be more invisible and self-contained within the VCS.
It would effectively be a meta-VCS.
I can download any docker image. Docker image contains all external dependencies and I can inspect those dependencies.
I assign a version to each Dockerfile. Docker image version is in the source control. Image with each version is stored in docker image registry. Version bump triggers update of all packages (either system level or language packages). This lets me revert external dependencies to state from any point in time. Or check out any commit from source control and run it in the same state as it was running in prod at that time.
My point is that this function should effectively be part of the VCS.
[1]: https://git-scm.com/book/en/v2/Git-Tools-Submodules
[2]: https://developer.atlassian.com/blog/2015/05/the-power-of-gi...
What I'm thinking of is a link to a version of an external repository.
(It may be working like that, but I've never used it like that if so).
Actually, yes... what I would like is an explicitly versioned "submodule" or "subtree" command where I can manage the exact versions within the VCS.
Subtree works a little bit different. It is essentially a copy of an external repository instead of a link.
I would not do that. I would fork the repositories I wanted and tag them along with your version. (mulle-bootstrap tag comes to mind :))
The embedding could be done with .bootstrap/embedded_repositories, which may or may not fit the bill.
http://www.kitware.com/media/html/BuildingExternalProjectsWi...
There are a number of issues/annoyances with the mechanism, especially when working with local modifications, but nothing to make me eager to add the anguish of bash+CMake onto the existing toil of CMake.
(Arguably this also bootstraps a build environment, whereas CMake requires itself and an SDK in order to bootstrap, but at the point where a build environment needs to be scripted/reproducible containerization seems preferable)
Is there a good reason to insert 30 lines of license repetition, in each script of between 4 and 10 lines of total code?
I see it as a maintenance/developer-usability burden (specially if the license is at the _beginning_ of each file).
It is not under the license of the project, if the license is not in each file?
Maybe being strict worldwide (as soon as something is published to internet), or as soon as there is Intellectual Property Agreement, they are required lines.
I use to avoid them in each single file for personal projects, unless it's a convention (for example in perl, license in POD format at the end of the CPAN class, is a community practice).
// (c) 2016 txutxu
and a license inside the project. The default copyright license protects you from unwanted copies.
But when it comes to IP rights, a lot of practices are actually voodoo legalities: people do them out of habit and paranoia, not because it has ever been useful in court.