Yes, and I've packaged that them that way for my own personal use, but it's really annoying to deal with because I have no idea which clients work with which revisions of the toxcore git repo. Some official alpha releases would make this much more manageable.
I honestly skimmed the article and didn't read it deeply because I have to run in a bit. But the fact that GUIX is a language-agnostic package manager sounds very neat. I write Go code and one of the things I had to cope with in the beginning was not having a really good package management system. Go still gets criticized for this (although we can always vendor our apps/packages to make sure all dependencies are included). It seems as though GUIX could be used as a package management solution in Go. It might be worthwhile for someone to write a tutorial on GUIX + Go dev.
Guix (and Nix, which came with the whole concept) is a great leap forward in terms of software deployment and sysadmin. It makes it very easy to manage your OS in a very declarative way, with many functional guarantees. Bonus points for being written and hackable in Scheme.
I'm working on the Go compiler support. A small hitch in cgo compilation and then it'll be done. Have to work with the Guix maintainers to get the package up to snuff as well.
This sounds great. For projects that use lxml/numpy/scipy/pandas this could be a real help for getting set up on different machines especially if there is some kind of caching that can be done for the built packages.
People often have this disappointment about Guix not being some sort of meta-package manager that can use any package format right off-the-shelf, which I think is caused by a misunderstanding of how the system works. Guix packages describe the precise dependency graph of any piece of software, all the way down to the C compiler's compiler. A package on PyPI, or anywhere else for that matter, does not. For example, Python eggs that build native extensions need certain shared libraries and C header files available on the system before they can be installed, but PyPI has no way of installing those things or otherwise capturing that information in a precise way.
However, us Guix devs can, and should, make it easy for users to take the metadata for a package available in another system such as PyPI and produce, to the best of our ability, a Scheme snippet that gets one most of the way to a working Guix package. We wrote the 'guix import' utility for this purpose. Right now, it can generate Scheme snippets based on packages at PyPI, CPAN, Hackage, ELPA, RubyGems, and CRAN. The user can take the output of the importer, and add the (hopefully few) missing pieces, such as the C library a Python native extension needs.
I welcome you to give packaging a shot. The IRC channel and mailing list are full of nice people that can help you if you struggle with something or encounter a bug.
Currently it's a GNU/Linux package manager, but there's a working GNU Hurd port that is being finished up, and I don't see a reason why someone couldn't port it to one of the BSDs if they wanted to put in the effort. Nix, the project Guix is based on, has been ported to OS X and there's even an experimental Windows port, so its possible to support many systems.
I've never understood why there isn't a "meta-package manager" that gives you one OS-and-platform-neutral CLI tool for invoking whatever the most relevant package management tooling would be, and then polyfills in the differences.
At a base level, I want to just not have to care whether I need to "brew install", "gem install", or "npm install -g" something: just give me one namespace and put everything I need in it once. If the tool could do just that, I'd be sold.
However, I can't see why there couldn't be a more-complex namespace-mapping database, such that I could e.g. export an installed-above-baseline package-list from Ubuntu and import it into OSX, or Windows+Cygwin, to get equivalent effects. I wouldn't need the same versions of anything, necessarily; just the same commands and libraries.
> At a base level, I want to just not have to care whether I need to "brew install", "gem install", or "npm install -g" something: just give me one namespace and put everything I need in it once. If the tool could do just that, I'd be sold.
That's why I tend to use Arch these days, almost all the npm, pip, etc. packages are in the AUR, and I can generally rely on yaourt and the AUR to compile and install it successfully about 90% of the time these days.
> I've never understood why there isn't a "meta-package manager" that gives you one OS-and-platform-neutral CLI tool for invoking whatever the most relevant package management tooling would be, and then polyfills in the differences.
That would be very nice. I remember something a few months ago about a windows CLI package manager called 'chocolatey', or something similar. I imagine that and powershell would enable us to finally have a true independent package manager at last.
>I've never understood why there isn't a "meta-package manager"
Because it's a fool's errand. The resulting mess would be the lowest common denominator of all the supported packaging tools. All of the various language-specific package managers do not touch things outside of their domain, leaving huge gaps to fill in. How would a meta-package manager know that installing the pg Ruby gem required having the PostgreSQL shared library and header files available at build time? What version of that library would it use? How would you reconcile where each package manager installs things? Traditional system package managers install to the global /usr directory. npm makes nested directory structures anywhere. Brew installs to /usr/local. How would you deal with the source vs. binary based package managers? Centralized vs. decentralized? How about package managers that require root privileges vs. unprivileged ones? What software bootstraps the entire system?
I could go on, but I hope you can see the futility. Even if you could somehow reconcile all of these things, what did it get you? You still wouldn't know the precise dependency graph of your software because most package managers can't tell you that. You wouldn't get build reproducibility because most package managers don't address it. You wouldn't get transactional upgrade and rollback of software because most package managers do not use atomic operations.
Guix can handle software written in any language, allows unprivileged package management, allows you to create isolated environments of packages (like virtualenv, bundler, etc.), deduplicates all packages system-wide via a content-addressable storage system, uses atomic operations for transactional upgrades and rollbacks, and more. A meta-package manager built on top the plethora of today's awful package management tools couldn't easily achieve any of these features.
> How would a meta-package manager know that installing the pg Ruby gem required having the PostgreSQL shared library and header files available at build time?
Because that dependency is in the meta-package's manifest. Realize that I'm talking here about an entire ecosystem of "meta-package formulae" that contain instructions to make particular tool-calls on particular systems.
> What version of that library would it use? How would you reconcile where each package manager installs things? Traditional system package managers install to the global /usr directory. npm makes nested directory structures anywhere. Brew installs to /usr/local. How would you deal with the source vs. binary based package managers? Centralized vs. decentralized?
You wouldn't reconcile any of these things, any more than things are reconciled now when you "brew install" one thing and "gem install" another and "npm install -g" a third. The result is a heterogenous multi-runtime system, but it continues to work, because each tool knows how to manage its own ecosystem. A meta-package manager would call through to the regular package managers as its API for each ecosystem.
> How about package managers that require root privileges vs. unprivileged ones?
The meta-package manager could assume the existence of a self-elevating wrapper tool like Debian's wajig(1), and provide such shims for package managers that don't have one.
> What software bootstraps the entire system?
The regular OS package manager.
---
I don't think I'm being clear here; the point wouldn't be to replace the system package manager. The point is to automate the use of all the various package-management tooling for configuration management in a unified, declarative manner.
I would expect the meta-package for e.g. Postgres to look a lot like a Chef cookbook for installing Postgres, but with the limitation that—where a Chef cookbook would first attempt to use a package manager, and then "fall back to" mutating the system itself in a way no tooling on the system is tracking—a meta-package would instead just fail the installation if there was no package-manager source satisfying its constraints.
Guix is great, but it is its own OS with its own ecosystem. The point here would be to unify the API to package management on the ecosystems that already exist, such that you could interact with the package manager in ignorance of its host OS the same way you can currently interact with e.g. a Docker daemon while ignorant of its host OS.
If this tooling was available and baked into server VM images and so forth, you could, for example, mostly replace the "cloudinit" instance-initialization system with a simple package list; or create something like a "virtual AMI" that takes a parent AMI ID and a "[{package, version constraint}]" list, and derives a new child image to launch on a set update schedule.
> Guix is great, but it is its own OS with its own ecosystem. The point here would be to unify the API to package management on the ecosystems that already exist, such that you could interact with the package manager in ignorance of its host OS the same way you can currently interact with e.g. a Docker daemon while ignorant of its host OS.
I think there has been a misunderstanding. Guix is a standalone OS-independent (Linux only) package manager. I am using it on Debian right now. The software installed by Debian has no knowledge of the Guix software and vice versa.
Conda (installed through miniconda or anaconda) is probably the closest thing in existence - it's a universal package manager (built with Python, and mostly for Python, but it is extremely general). But it doesn't co-operate with any underlying package manager - it manages everything on its own. It does compartmentalize "pip install" for Python, and I assume it can just as easily do the same for npm, gem, rocks etc (though it does not out of the box)
31 comments
[ 4.7 ms ] story [ 77.3 ms ] threadThey really are getting back on track this month with irungentoo working full time.
It's not only Go that miss a "complete package management system". IIRC guix-tox can replace both regular CI system and python-tox as test runner.
It's a shame it can't use all of PyPI though.
Yes once package is built, it's cached.
edit: the command name is `guix import pypi` http://www.gnu.org/software/guix/manual/guix.html#Invoking-g...
People often have this disappointment about Guix not being some sort of meta-package manager that can use any package format right off-the-shelf, which I think is caused by a misunderstanding of how the system works. Guix packages describe the precise dependency graph of any piece of software, all the way down to the C compiler's compiler. A package on PyPI, or anywhere else for that matter, does not. For example, Python eggs that build native extensions need certain shared libraries and C header files available on the system before they can be installed, but PyPI has no way of installing those things or otherwise capturing that information in a precise way.
However, us Guix devs can, and should, make it easy for users to take the metadata for a package available in another system such as PyPI and produce, to the best of our ability, a Scheme snippet that gets one most of the way to a working Guix package. We wrote the 'guix import' utility for this purpose. Right now, it can generate Scheme snippets based on packages at PyPI, CPAN, Hackage, ELPA, RubyGems, and CRAN. The user can take the output of the importer, and add the (hopefully few) missing pieces, such as the C library a Python native extension needs.
Is that also integrated with the guix command line tool? If not, why not? It would be super neat :)
- Clone the git repo, build Guix, and run the build daemon
- Add the Scheme code for the package recipe to the relevant module
- Run 'guix build' to make sure it builds and runs
- Run 'guix lint' to catch common style issues
- Make a commit
- Send patch file to our mailing list
Full details about contributing can be found here: http://www.gnu.org/software/guix/manual/html_node/Contributi...
I welcome you to give packaging a shot. The IRC channel and mailing list are full of nice people that can help you if you struggle with something or encounter a bug.
At a base level, I want to just not have to care whether I need to "brew install", "gem install", or "npm install -g" something: just give me one namespace and put everything I need in it once. If the tool could do just that, I'd be sold.
However, I can't see why there couldn't be a more-complex namespace-mapping database, such that I could e.g. export an installed-above-baseline package-list from Ubuntu and import it into OSX, or Windows+Cygwin, to get equivalent effects. I wouldn't need the same versions of anything, necessarily; just the same commands and libraries.
That's why I tend to use Arch these days, almost all the npm, pip, etc. packages are in the AUR, and I can generally rely on yaourt and the AUR to compile and install it successfully about 90% of the time these days.
> I've never understood why there isn't a "meta-package manager" that gives you one OS-and-platform-neutral CLI tool for invoking whatever the most relevant package management tooling would be, and then polyfills in the differences.
That would be very nice. I remember something a few months ago about a windows CLI package manager called 'chocolatey', or something similar. I imagine that and powershell would enable us to finally have a true independent package manager at last.
Because it's a fool's errand. The resulting mess would be the lowest common denominator of all the supported packaging tools. All of the various language-specific package managers do not touch things outside of their domain, leaving huge gaps to fill in. How would a meta-package manager know that installing the pg Ruby gem required having the PostgreSQL shared library and header files available at build time? What version of that library would it use? How would you reconcile where each package manager installs things? Traditional system package managers install to the global /usr directory. npm makes nested directory structures anywhere. Brew installs to /usr/local. How would you deal with the source vs. binary based package managers? Centralized vs. decentralized? How about package managers that require root privileges vs. unprivileged ones? What software bootstraps the entire system?
I could go on, but I hope you can see the futility. Even if you could somehow reconcile all of these things, what did it get you? You still wouldn't know the precise dependency graph of your software because most package managers can't tell you that. You wouldn't get build reproducibility because most package managers don't address it. You wouldn't get transactional upgrade and rollback of software because most package managers do not use atomic operations.
Guix can handle software written in any language, allows unprivileged package management, allows you to create isolated environments of packages (like virtualenv, bundler, etc.), deduplicates all packages system-wide via a content-addressable storage system, uses atomic operations for transactional upgrades and rollbacks, and more. A meta-package manager built on top the plethora of today's awful package management tools couldn't easily achieve any of these features.
Because that dependency is in the meta-package's manifest. Realize that I'm talking here about an entire ecosystem of "meta-package formulae" that contain instructions to make particular tool-calls on particular systems.
> What version of that library would it use? How would you reconcile where each package manager installs things? Traditional system package managers install to the global /usr directory. npm makes nested directory structures anywhere. Brew installs to /usr/local. How would you deal with the source vs. binary based package managers? Centralized vs. decentralized?
You wouldn't reconcile any of these things, any more than things are reconciled now when you "brew install" one thing and "gem install" another and "npm install -g" a third. The result is a heterogenous multi-runtime system, but it continues to work, because each tool knows how to manage its own ecosystem. A meta-package manager would call through to the regular package managers as its API for each ecosystem.
> How about package managers that require root privileges vs. unprivileged ones?
The meta-package manager could assume the existence of a self-elevating wrapper tool like Debian's wajig(1), and provide such shims for package managers that don't have one.
> What software bootstraps the entire system?
The regular OS package manager.
---
I don't think I'm being clear here; the point wouldn't be to replace the system package manager. The point is to automate the use of all the various package-management tooling for configuration management in a unified, declarative manner.
I would expect the meta-package for e.g. Postgres to look a lot like a Chef cookbook for installing Postgres, but with the limitation that—where a Chef cookbook would first attempt to use a package manager, and then "fall back to" mutating the system itself in a way no tooling on the system is tracking—a meta-package would instead just fail the installation if there was no package-manager source satisfying its constraints.
Guix is great, but it is its own OS with its own ecosystem. The point here would be to unify the API to package management on the ecosystems that already exist, such that you could interact with the package manager in ignorance of its host OS the same way you can currently interact with e.g. a Docker daemon while ignorant of its host OS.
If this tooling was available and baked into server VM images and so forth, you could, for example, mostly replace the "cloudinit" instance-initialization system with a simple package list; or create something like a "virtual AMI" that takes a parent AMI ID and a "[{package, version constraint}]" list, and derives a new child image to launch on a set update schedule.
I think there has been a misunderstanding. Guix is a standalone OS-independent (Linux only) package manager. I am using it on Debian right now. The software installed by Debian has no knowledge of the Guix software and vice versa.
GuixSD is the OS built around Guix.