61 comments

[ 3.9 ms ] story [ 110 ms ] thread
> The GNU Shepherd is developed jointly with the GNU Guix project

Wasn't it made for the hurd?

Shepherd should run on Hurd, since Guile does, but it was created as part of the GNU system, which is in reality usually run on the Linux kernel.
Yes it was, but more activity with Shepherd is occurring in the Guix SD project.

Guix has also been ported to Hurd, so it's all going to be one big happy family (one day when Hurd finally gets USB and 64-bit support).

Interesting,after all this criticism of systemd.

This makes me wonder about the GuixSD distribution. Is this distro ready for prime-time. Suppose I switched from Linux Mint to GuixSD, what would be the advantages and disadvantages?

I like Lisp and program in Racket, so Guile wouldn't be far from that. I just wonder about experiences of other. How much does switching to such a lesser known free distro as the main OS "constrain" you?

> Suppose I switched from Linux Mint to GuixSD, what would be the advantages and disadvantages?

You'd lose a lot of software choices. Actually, the restriction to "GNU-friendly software" was what made me skip GuixSD so far. I guess that my WiFi wouldn't work.

Not true. You can write your own packages in the same high level interface. Including the mainline Linux kernel and broadcom drivers. I have GuixSd running on my 2013 MBP
Is there a...non official repository of these somewhere?
None that I know of currently. The OS is version 0.15.0. It's very young and doesn't have a huge user base. I'm sure there will be non gnu repositories that will eventually spring up. I've even toyed with the idea of setting one up.
Even as a toy example, a blog post and code that demonstrates setting up a 3rd-party package repository would be helpful. I'm just afraid of the GUIX team deciding they don't want to support 3rd-party repos because it encourages non-free software use, and deliberately stonewalling development of related features.
Guix has had support for external package definitions for a long time (via GUIX_PACKAGE_PATH) and recently gained a channels feature, which makes this even easier.

At the institute where I work we use this feature to provide package variants that aren't going to be added to Guix proper (e.g. because they are no longer supported upstream or because they are only useful on obscure systems).

This mechanism can be used to include package definitions for non-free software, of course, but this doesn't mean that the Guix project encourages the use of non-free software.

You have the freedom to package up and use non-free software, but we won't make them part of Guix, build them on our build farms, and we won't hold back changes to the mechanism even if that might break 3rd-party collections of non-free software.

You don't really need a blog post about how to set up a package repository, because it's right there in the Guix manual.

PS: It's "Guix", not "GUIX" ;)

Thanks for the reply. As long as it's easy to mix free and non-free packages at my discretion, I'm willing to give it a shot.

Also, thank you for the capitalization correction. It grinds my gears when people capitalize things unnecessarily and I wouldn't want to be one of them!

There are a smattering of code snippets around

This[1] is the one I used as a reference for my kernel build which includes brcmfmac

There is also a feature called 'channels' (not widely documented AFAIK) which allows you to describe an outside repository of code you can reference to install packages. There is one[2] that provides Chromium (though I cannot get the build to succeed).

The downside to all of these not being provided in mainline is the amount of time and energy to build them. I suspect there are more people than I would like curating their build of the linux kernel, manually specifying drivers included in the kernel. I really wish the community were a little less hard-set on the principle of 'freedom' and a little more pragmatic; I am sure the community is smaller due to people being turned away at the sheer amount of effort required to get their machine to a usable state.

[1]: https://github.com/wingo/guix-nonfree [2]: https://gitlab.com/mbakke/guix-chromium

^ I stand corrected: My WiFi (ath10k) WOULD probably work...
As someone who is sympathetic to their cause and their stance, I think the extreme position of excluding all non-free software can actually hamper the movement by making it less accessible to people. I get you always have to draw a line somewhere, but FSF draw it too firmly.
> you always have to draw a line somewhere, but FSF draw it too firmly.

They actually don't draw it exactly where I would. I don't care so much for games if they're open/free or not (it's nice when they are of course), but I am concerned with unauditable code on, for instance, the SATA controllers in my machine, even if it 'read only' code. Where the FSF is more concerned with the first and less with the second.

On the other topic, I think the ability to create (and share) your own recipes/channels for software on Guix allows for people to create fairly easy ways to install non-free software (or using Nix on top of GuixSD) without the Guix team having to support unethical software.

(comment deleted)
can you tell me how you got that running in some more detail? I also have a 2013 MBP and would like to try out GuixSD
I don't have access to it now. When I do I can put the package definitions and OS config up somewhere.

When you first boot GuixSD most of the core stuff should work fine. Wireless will not work but you should have ethernet. I created mainline linux kernel and firmware packages [1]. Then you need to get the broadcom b43 firmware to get wireless support. I think I made 2 packages. One for b43-fwcutter and one the broadcom-wl firmware which uses b43-fwcutter to cut and install it. I think just about everything on my MBP was working after that.

[1] example linux package https://github.com/wingo/guix-nonfree/blob/master/gnu/packag... [2] http://linuxwireless.sipsolutions.net/en/users/Drivers/b43/

When possible, a lazy alternative to packaging non-free software in guile would be using containerization software. I run dropbox in docker under GuixSD and works flawlessly.
Can you dockerize drivers?
You can't dockerize kernel modules, no. Though Linux mainline wifi support is pretty great these days, coming from someone who vividly remembers using wpa_supplicant with Windows drivers :-)
> You'd lose a lot of software choices. Actually, the restriction to "GNU-friendly software" was what made me skip GuixSD so far.

Guix includes free software packages of all sorts (more than 8k today), be it "GNU-friendly" (whatever that means) or not; you can browse them at https://guix-hpc.bordeaux.inria.fr/browse .

> I guess that my WiFi wouldn't work.

Maybe, maybe not; see https://gnu.org/s/guix/manual/en/html_node/Hardware-Consider... (the same issue applies to other distros that value user freedom.) Note that you're talking about GuixSD here, not Guix, which can be installed on top of other distros.

In the end the choice is up to each one of us. It's also up to us to determine what to do with hardware companies that deny us, collectively and individually, the right to control the devices they sell us.

I use nixos, not GuixSD, but I can speak to "what would be the advantages and disadvantages", particularly since guix is a reimplementation of nix.

The benefits to me are immense, because I travel between around six machines on a daily or weekly basis. Prior to moving to NixOS, I'd have to manually keep those machines in sync in terms of software installs, etc. However, after switching to NixOS, I have a set of files I version control which will declaratively build the system the way I want it.

So, if I install a new package and I'd like it be the same across all machines, I simply configure it, commit to git, and then pull down the config on the other machines and do "nixos-rebuild switch". Everything becomes exactly the same.

It's saved me so much time in terms of managing my computing life. I really couldn't see going back to a regular distro after using one which is declarative.

The drawbacks? Well, for one, Nix and Guix have various tricks with symlinking, etc, to make it appear to applications as if it's a typical Linux filesystem, but it's not. E.g. there's no /lib, etc. It's a little weird getting used to.

And with nix, there's the nix language, which isn't that difficult to learn but it's not my favorite. If the underlying language needs to be functional, I'd prefer a lisp over nix. Also, with nixos, there are too many different commands required to learn when you get deeper into it, but work is underway to improve that story.

At least for me...the benefits far outweigh the risks. I don't think I can ever go back. I think there's a real opportunity for a commercially-backed declarative Linux os...it really is a game changer and would do well in the enterprise.

For me, the problem isn't so much with the nix language(it's very similar to haskell), but the lack of discoverability for docs of library. The manual is mostly incomplete and i often have to look at sources.

My most favourite thing is that i can just override the packages to compile and install a new version/extra configure flags of a package that's not yet upstream.

Interesting... I've found the manuals to be ok so far. Can you share what you've had problems finding?
Well, a lot of common functions like fetch* , write*, makeWrapper that are used in packages are undocumented. I usually dig into source files using nix repl to find the location.
Man, I’ve really wanted to take the plunge and dedicate some play time to GUIX for a while now. I feel like between the GUIX package system and Shepherd, they’re creating an authentically new and exciting thing in the universe of Linux distributions.

The fact that they’re using Scheme for so much of it is also pretty cool, as I think this sort of high level systems programming is a great potential use case for showcasing meta programming as well as the unique power of lisps for code-as-data.

Same here...as a NixOS user, I really wish it used scheme instead of the nix language.

However, I do like the pragmatic nature of NixOS (you're able to easily install non-free packages). GUIX doesn't make this easy at all from what I understand. I appreciate those virtues, but practically speaking, I have to use non-free software on the desktop all the time.

Guix does have the capacity to add 'channels', e.g. https://gitlab.com/mbakke/guix-chromium , so while non-free software while never be part of the distribution itself, it could be done unofficially. Also, you can run the Nix standalone on top of GuixSD, so you could run GuixSD and then use Nix to install whatever's missing.
> you could run GuixSD and then use Nix to install whatever's missing.

How much cooperation between the two is necessary? If I wanted to install CUDA drivers with Nix would it "just work" (i.e. if it works on NixOS it works on GuixSD) or could GuixSD make problems?

I found a comment from a year ago claiming that the kernel wouldn't load blobs: https://news.ycombinator.com/item?id=14630581

I'd like to understand this as well. It's a viable option for me if it truly works.
I'm not certain about CUDA drivers and whether this sort of thing work "just work" via the Nix package manager on GuixSD. That sounds like you might need to recompile kernel headers or the like.

Re: the comment about the Guix kernel and blobs: Guix uses the linux-libre kernel which has blobs removed. BUT you can use a custom kernel on Guix, so you could use a blobby kernel, but you would have to configure it. If you poke around, you can find examples of people doing this, e.g.: https://gist.github.com/TeMPOraL/306bbb7c627ab0d32fa84af8319...

But, in general, the standalone Nix package manager (and the standalone Guix package manager, for that matter) are designed to work as isolated units, not depending as much on the 'host' system, so lots of stuff does 'just work' regardless of whether the host distro is GuixSD or Debian or Void or Arch or whatever.

If you have a Guix package definition for CUDA then you can install it. There's no need to use Nix when Guix can be easily extended.
Do you have more detail/instructions on running Nix on GuixSD?
Just install the standalone Nix installer like you would on any other (non-Nix) distro (see this part of the Nix manual: https://nixos.org/nix/manual/#chap-quick-start ). Both Guix and Nix are designed to work on top of 'foreign' distros, and this includes using a standalone Nix package manager on GuixSD and a standalone Guix package manager on NixOS (or both the standalone Guix package manager and the standalone Nix package manager on top of Arch Linux, etc. etc.)
Having once spent a happy afternoon browsing the Guile info pages, but otherwise pretty unfamiliar with it, this seems to realize a lot of the promise that Guile has had from the beginning. Specifically, the idea that the language that ties together the GNU operating system is not C or bash, but Guile Scheme. A language that can do very low level stuff and GUI development and be used for scripting, empowering the users of a capital-F Free operating system that's hackable all the way down. That's my impression anyway. As I said, this is mostly gleaned from the Guile info pages, but I was impressed by the consistency of that vision with the GNU project's overall goals.
For anyone interested in guile, guile 3.0 will have a JIT, making it even faster than 2.2 (by quite a large margin).

Guile is becoming really nice, and I find myself reaching for other lisps less and less.

How far along is it towards becoming the underlying runtime for Emacs lisp?
With the bikeshedding we see for much less controversial changes in the Emacs codebase, I suspect it will be released around the release of Duke Nukem forever II.

Elisp code runs fine under guile, but it has seen virtually no optimisation work and does a lot of unnecessary work. I suspect there is an lot of low hanging fruit, but since there is virtually no interest in guile-emacs apart from in comments on HN and Reddit I doubt the situation will get much better.

The guile VM is getting better and better though. Maybe they'll come around by guile 4 :)

I've always thought someone should use guile to add support for elisp plugins to vim & neovim using their rpc interfaces. Then maybe if something like that existed the org-mode authors could be convinced to cleanly separate out the emacs-specific code, and we could have org-mode on vim using the same codebase as emacs.

I have no idea how viable that is, but it would be pretty sweet. Too much for me to take on right now, but the idea has been rattling around my brain for a few years.

Scheme is a neat little language, but it's really not ideal for writing full software systems. Indeed, Guile & Racket (back when it was PLT Scheme) are perfect exemplars of Greenspun's Tenth Rule: Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp. E.g. GOOPS is similar to CLOS, but missing a bit and unsupported other than in Guile.

Meanwhile, implementations like SBCL are free, compile to machine code and are competitive on some benchmarks with C/C++.

Common Lisp — while it does have some warts — is intended for building long-lived, durable, full-featured systems. It's a really good language, and it standardises quite a lot. Often, what look like warts are actually useful features in large systems.

It really saddens me that so much effort has been expended trying to make Scheme a reasonable language for large systems, when there already exists one which is. That doesn't even get to the bits of Scheme which I believe are mistakes (e.g. a single namespace, making NIL, () & #f different).

I find many common lisp users' attitude towards scheme condescending. It is as if we are completely oblivious to the CL ecosystem or outright stupid to prefer something else.

I assure you, we look in envy at CL and what a standardised module system, portable networking and CFFI achieves. However, we still prefer scheme. Some of us actually like a single namespace, and don't mind the difference between false and nil (I have never really met a case where that distinction mattered much).

> I find many common lisp users' attitude towards scheme condescending.

Are you familiar with the Blub Paradox[0]? I contend that the things you dislike about Common Lisp or don't mind about Scheme having or not having are precisely the things which make Lisp better for real-world systems.

My opinion — a professional, educated & considered opinion, but an opinion nonetheless — is that Scheme is simply not suitable for large-systems work, and that all attempts to take a Scheme core and make it suitable founder on the shoals of Greenspun's Tenth. The Scheme maintainers agree, because in R6RS they tried to build a more capable language. The Scheme community agrees, because it rebelled against R5RS, deeming a more capable Scheme an un-Scheme-like Scheme.

I like Scheme. It's a pretty, elegant language, with an exciting model. But the language as it exists simply isn't ready for systems work, and (due to its underlying philosophy) probably never will be. That's not a bad thing. Scheme is great for what it is. It's great for teaching; great for reasoning about computation; great for learning evaluators & compilers. But it isn't great for building systems like an OS, emacs, a distribution, a package manager, an init system or a window manager — and the effort spent working on making its inadequacies tolerable could instead be spent on implementing features in a more-than-adequate language.

Also, Lisp doesn't have portable networking. That's one of its warts (I did say it has them — and plenty of them! Lisp is not perfect!). It does have portable streams (and I think all implementations make sockets available as streams), but Scheme's ports are analogous.

Neither is CFFI standarised.

0: http://paulgraham.com/avg.html

Can you explain why it's not ready systems work?
I have written applications weighing in at about 10k lines of code (not huge, but big enough to have to manage some complexity) using LispWorks. I doubt I would have problems writing the same software in guile. I don't really understand which inadequacies you mean.

The only thing I really miss is conditions and restarts (which is a pretty big downside) and how integrated CLOS is (and the fact that LW pre-populates the CLOS caches is pretty neat!)

>But the language as it exists simply isn't ready for systems work, and (due to its underlying philosophy) probably never will be.

One of the flaws with this argument is insisting that comparisons be made between CL and standard Scheme. No one uses strictly standard Scheme. It's better to think of Scheme as a family of loosely connected languages. I know that both Guile and Racket are languages (and they are their own unique languages) capable of serious systems work. CL is great, too, but I don't know why you keep insisting that it's the only lisp worth using. The Guix developers have shown that Guile can be used successfully for the initial RAM disk, init system, package manager, configuration management system, continuous integration system, web site generator, etc.

Are there any plans to support AOT as well?
Yeah. Well, guile already does AOT, but not to native code. Now it compiles to guile bytecode that is then jitted.

The plan is to use the JIT as a first step to do AOT compilation to native code and thus come a bit closer to the speed of chez. Hopefully quite a bit closer.

This is what I have gathered from Andy Wingo's mailing list posts, so grains of salt for everyone!

How good is the concurrency support? a lot of schemes have poor support for it(basically roll your own via libev), except racket.
Pthreads and guile-fibers (a parallel concurrent ML implementation).

If you never used CML you should try it out.

This is awesome! I have really been impressed with gnu's use of guile, and I really like how much more quickly iteration is than many other gnu projects. For those heavy cron users, mcron (guile based cron replacement) has been my favorite so far.

With a good deamon manager, we're one step closer to HURD!

Pardon me, I'm sort of a casual Linux user and I have no idea what people are talking about. What are GNU Guix and Shepherd? A cursory glance at Wikipedia makes it seem like they're GNU alternatives to the apt-get and init/systemD I've been on for awhile.

I'm outside of my Debian comfort zone, apologies for my ignorance! What's significant about this?

Basically they follow similar ideas to NixOS, but using Scheme instead of a custom script language.
Does anyone know if they solved their "trust init script too much" problems?

Last time I checked, they were loading user-defined scripts right into PID 1, so a badly written script could bring down the whole system.

People blame systemd for pid 1 instability, but at least systemd crashes get fixed eventually. With Shepherd, any package on the system can cause PID 1 to crash, and this just seems to crazy for me.

Can you give any more detail about this? Or a reference to a bug report or mailing list thread?
The manual [1] starts by saying: "When shepherd gets started, it reads and evaluates a configuration file." That config is full-featured, seemingly unrestricted, lisp file. While I saw some mentions of handling syntax errors on the mailing list, I saw no mentions of what happens if there is an infinite loop or if a script just reads a huge files and OOM's the process.

Now, one could imagine that maybe configs are parsed in a separate process, and then serialized and shipped to the main process for safe execution. But it does not seem to be the case -- the manual explicitly says you can use any Guile code to start processes, not just a safe subset.

Another option is that maybe there is a thread or a forked process for parsing config. But this does not seem to be the case either -- the manual does not mention that, and searching of the source code for "fork" or "thread" or "timeout" does not seem to indicate anything related.

The only mention of timeness expectation seem to be a single sentence "Actions are expected to be non-blocking." in one of the mailing lists [2].

So together, it seems to me that any bad script will just cause the daemon, and by extension the whole system, to not start.

[1] "https://www.gnu.org/software/shepherd/manual/shepherd.html#S...

[2] http://lists.gnu.org/archive/html/guix-devel/2016-11/msg0094...

There's a quote from the GNU Shepherd manual I really appreciated:

> I decided to not make it too sophisticated by trying to guess what the user might want just to theoretically fulfill the request we are processing. If something goes wrong, it is usually better to tell the user about the problem and let her fix it, taking care to make finding solutions or workarounds for problems (like a misconfigured service) easy. This way, the user is in control of what happens and we can keep the implementation simple. To make a long story short, we don't try to be too clever, which is usually a good idea in developing software.