I think this is unfair - Arch is also meme'd by linux enthusiasts and 4chan.
Additionally, Gentoo is the largest distro with the source-based package manager USP. I suppose you could take issue with that approach for reasons like the systemd maintainer does where he claims it wastes CPU-cycles and time. Personally I disagree with that assessment since reproducible builds are a vital part of FOSS.
Fun fact is that Gentoo is the most normie distro. ChromeOS is based on Gentoo and all have coreboot. I'm not disparaging it as much as seeing it as a similar learning tool.
This is partially true but I believe that the only component ChromeOS uses from Gentoo is the portage package manager during some of its bootstrapping protocols.
I was never a gentoo user (I installed once, like I did for basically any distro back then), but I wouldn't knock the sort of "apparently-unproductive tinkering" that Gentoo epitomizes. Like building the umpteenth blog engine, or yet another Todo manager in $language_du_jour, that's how we learn.
I use gentoo on my desktop and debian on an old refurb thinkpad I got for £100. Both of them are amongst the last of the large independent linux distros and I think both approaches are valid.
Never had a bad experience with Arch but the way they use the AUR as a crutch is a bit off-putting. It's a little bit like a giant gentoo overlay but pushes all the complexity onto the user.
Years ago, somehow, I wasted hours and days of computer and my time, compiling and fine tuning my gentoo system, god knows why, when next day I anyway format it to install newly arrived Ubuntu cd.
I too used to obsess over customizing my OS. Now I just install Debian, a handful of programs I use daily and that's it. I can recreate my setup on another machine in 20 minutes.
Not sure if you mean that sarcastically, but it's a really boring and stable OS, the setup is just a few clicks and hitting enter a few times. I think I haven't even bothered to change the wallpaper on my newest laptop.
There's nothing really special about Debian as well, it could as well be Ubuntu, Mint or anything else that's plug n play. I'm just used to Debian, and it comes with less junk I don't need installed by default.
Of course you customize your Debian installation over time.
Over time is the key here. A package here, a small config there, and after some time, that installation becomes so unique that it starts reading your mind.
Non-breaking updates is the icing on the cake.
The biggest point is you install Debian once, or when your processor architecture changes.
They talk about the shell as an IDE. My entire desktop is a 14 year experiment in tuning productivity. My ~/bin/ folder has around 100 scripts and maybe 20 are little scripts i wrote in conjunction with i3. Pretty cool how it stacks up over time
The only cool thing about it is that it’s declarative: nix-darwin everything and a fully working and customized machine is up in 10 minutes with one command
You and me both. But, we learned a lot. Nowadays I feel like Linux is my super power, OS, VM, Container, Nix-shell, WSL2. It’s all Linux. And you can drop me on any command line (even a BSD one to some extent) and I will feel at home and can solve problems. I’d like to think that’s where my happy time with Gentoo led to.
And no matter if its true (and it might well be!) the overarching tendency to look for reasons to explain time spent with Gentoo should probably tell us something.
At best, we are at least a bit confused about it all.
I was confusing because you can’t install it without understanding stuff like fstab, grub, user creation etc. It sets you up to be a sysadmin, it requires you to be a sysadmin. Ubuntu on the other hand, looks and acts more like an iPad than Windows.
Same, this was around 2004-2006ish when I maintained a Gentoo build for my Pentium 4 box. There was this somewhat draw to me of compiling my own binaries highly optimized to my processor and that Portage mostly works. But my gosh, gcc build times are killing the fun. When Ubuntu arrived and saw my peers being productive, I switched.
I had toshiba satellite with some whooping 96MB RAM... as main computer even... happily ran Windows 98... them I got the book "Linux from scratch" the rest is history... now I am happy mac user.
When every ounce of power mattered, fine-tuning your OS made sense.
Nowadays most people are swimming in CPU cores and gigabytes of ram and terabytes of solid-state memory, so fine-tuning is a waste of time (unless you play bleeding-edge games). But it wasn't always such.
Yeah, I'm calling bullshit on this one. At least, it doesn't line up with my experience. In my experience, games are optimized _just enough_ for a decent playing experience (and not always then). Games devs, as a whole, are the worst offenders of expecting their users to just throw more money (hardware) at the software to achieve usable/enjoyable experiences. There are, of course, exceptions. But, for every Carmack, there's 10s of thousands of developers scrambling to make their deadline, doing just enough to ship.
I have heard of people recompiling the kernel to improve gaming performance (mostly to use a different scheduler or what be it), but don't recall seeing anything beyond single digit percentage improvements in performance. Which makes sense, since you can only recompile the kernel and a subset of open source libraries that the game may use. Those are going to be fairly well optimized to start with.
The games themselves though are a different story. Outside of open sources games (which are usually less demanding than commercial ones), you don't have the source code to rebuild it. Even if you did, enabling optimizations beyond what the developer used risks breakage so you will have to do your own testing. Even then, simply rebuilding the software wouldn't address the quality of the code created by those developers who are scrambling to meet a deadline with as little effort as possible.
I'll be the first to admit that I'm not a game developer and my exposure to commercial games' source has been very limited. The most exposure I've had was to Civ4 due to Firaxis releasing the source for the core game DLL for modding. Civ4 also used Python for scripting and Python (undeservedly, here) gets the blame for the game being slow, especially during late-game play.
Back in the day, I spent a fair amount of time working on gutting the DLL because frankly, it was atrocious. My memory is a little fuzzy as it's been +10 years since I've looked at it, but things I remember seeing:
* over use of C/C++ preprocessor macros where an inline function would have been more appropriate to say, get the array/list of CvPlots (tiles) that needed to be iterated over all the time.
* lack of hoisting loop invariants out of loops. It is common to see usages of the non-trivial macros above in the bodies of tight loops, often tight nested loops. Optimizing compilers are great, but they're not _that_ great.
* the exposure of the C++ types to Python was done...poorly. It was done using Boost Python (which, while a great library for its day had a _huge_ learning curve). For every Cv type in the C++ DLL, there was a corresponding Cy type to expose it to Python. Collection types were _copied_ every call into Python code, which was quite frequent. The collections should have been done as proxies into the underlying C++ collections, instead of as copies.
Most of the changes I made were quite small, but over the course of a month of part-time hacking on it, I'd gotten late game turns down to a couple of minutes from 30-minutes and memory usage was extremely reduced; and I never did get around to fixing the Python wrapper as it would have too intrusive to fix it properly. I could have made more aggressive changes if I had full access to the source, but being constrained by DLL boundaries and C++ types being exported limited what could be done w/o breaking the ABI (had to be extremely careful about not changing object sizes, affecting vtable layout, etc).
Frankly, I doubt the developers spent very much time at all, if any, with a profiler during the course of development with the game.
When every ounce of power mattered, fine-tuning your OS made sense
I used to believe that and was a huge Gentoo user for years back when it was initially released. Then one day I benchmarked Gentoo and a default RedHat install on every work load I cared about, and everything was within the margin of error.
I think the biggest case where Gentoo still makes sense is when you have a large fleet of identical machines. In that case, the effort put into tuning your installation will be multiplied across the number of machines it's applied to.
For a single machine home install, the biggest value Gentoo has to offer is the learning experience. I ran it for about a year like 4 years ago, and I definitely learned a lot in that time. Hopped around a bit and I've since landed on GNU Guix, and I'm probably set for life now in the GNU world.
Made an ancient computer with very limited CPU usable for my siblings with Gentoo. The secret is to do USE="-* minimal" and enable things that are required from there. Compiling a custom kernel was actually necessary because it had a really old NVIDIA card that was no longer supported and I had to patch something to do with MTTR. Installed XFCE 4 and it idled with 70 MB of RAM used. Could play Youtube videos without the whole thing freezing whereas Debian could not. Gentoo is great.
Gentoo’s biggest attraction for me was always the USE flags - being able to turn off the X integration of mpg123 where CentOS demanded an entire X install to get the command line player.
A lot of the reason depends upon what you hope to get from the labour and the overall environment that you are working within.
I was working with a 486 around 1995. Compiling your own software was the norm and compiling your own kernel could have significant performance benefits (even if it was just to conserve the limited memory supported by machines of the day, to head off some of the swapping). By the time I learned of Gentoo, that was not really the case: most of the software one could obtain was provided in binary form and compiler optimizations were much less relevant (unless you had a special workload).
The tooling provided is important too. I was using NetBSD for a while. For the most part you just started the compilation process and walked away until it was done. (I imagine Portage is similar.) You didn't get the instant gratification, but it was not time intensive in the sense that you had to attend to the process. That was very much unlike my earlier experiences in compiling software for Linux, stuff that isn't in the repos, since it did have to be attended to.
It surely wasn't the norm for me, in 1995's Summer, I got my first Linux distribution via Slackware 2.0, everything was already compiled and when chosing to download tarballs I would rather pick already compiled ones.
Later on, to take advantage of my Pentium based computer, I would get Mandrake, with its i585 optimized packages.
Most of my Linux based software would be sourced via Linux magazines CD-ROMs, or Walnut Creek CD-ROM collections.
For kicks and giggles, I just set it up on a new system a couple of weeks back.
It's really not much different than working with Arch in terms of complexity. Initial setup takes a bit, but if you've installed arch you are pretty familiar with everything you need (in fact, arch docs are helpful for a gentoo setup :D).
The docs are VERY good and easy to google.
Compilation time can be nasty depending on what you install but not terribly bad. I just rebuilt the world because a GCC update broke lto that I'm running. With about 2k packages that took about 6 hours to complete on a Ryzen 7950.
General updates take almost no time at all (especially using git for syncing). Usually less than 10 minutes often less than 1. As I write this, I'm currently rebuilding kde (if you are using your computer, rebuilding doesn't really get in the way. Especially if you are already working with a multicore system).
“But computing power is much higher now. The same compilation now would probably take 1-2 hours, max. Updates would be super fast.”
I’m not so sure. A lot of the power comes from multiple cores. Years ago I had one core, now I have eight. A lot of the compiles don’t use all the cores.
Software has also gotten bigger. rustc is huge, for example. It didn’t even exist when I used Gentoo years ago.
These days I’m on the Mac and I just switched to Homebrew after using Macports for years. It was for one of the same reasons I stopped using Gentoo: compiling takes too long. Whenever I upgraded Mac OS versions, Macports required me to recompile everything. This was no problem at all for, say, tree. But something was pulling in gcc (emacs needed it for some reason??) and this took ages to compile.
At least Macports worked though. When I used Gentoo, it took so long to compile things that I would leave it overnight, and of course often in the morning I would see that the compilation stopped halfway through because something was broken. Hopefully that’s improved. Or of course maybe the binary packages will help with this.
But if I wanted a build-your-own, rolling-release binary system, I don’t see why I wouldn’t just use Arch.
The time I "wasted" with Gentoo in 2005 taught me enough about how Linux works to land me my first real IT job. I will forever be grateful to that distribution.
Yea, same here back in the day. Stage 1 installations for Gentoo really made me interact with the kernel and software in a different way. It did not just work, but while solving the issues, I learned a lot on how things worked internally. It's a great thing to get really familiar with the workings under the hood.
On my first pc assembled from used parts, I was able to squeeze every bit of compute out of gentoo. Being able to build smaller binaries by excluding dependencies seemed to help a lot. I used it until the first ubuntu was released and it just worked and worked well. The only problem was that it was an ugly brown.
But then you have to deal with big upgrades that might break your system and old packages (or start randomly adding PPAs etc.) A rolling distro means you can continually keep up with small changes and only adopt big new pieces (like systemd, pipewire, wayland etc.) if/when you are ready to.
I've installed Gentoo literally two times. Once per PC. Been using it for years. It's not like you have to keep tweaking it. It does help if you run a basic system like me, though (no DE, simple tiling WM, don't use much apart from Emacs and Firefox).
Same, but I learned so much while doing it. Eventually I got tired and moved to an arch and got most of the same without always fighting broken packages. But I still use the knowledge I gained dealing with random low-level issues when they crop up.
I had a similar experience with the Gentoos/Arches of the world. I'd never use Gentoo or a Gentoo-like as my primary OS for anything, likely for the rest of my life, but it still ended up being one of the most valuable operating systems for me to spend some time on.
Everyone please realize that just because you did something and no longer do that thing does not mean it was wrong to have done that thing.
All of us who at one point compiled our own kernels and now no longer do, are the killers that we are partly because we did things like that at least for a bit. It only makes sense not to now, after having done it.
It's not true to suggest (or to read these stories as a new bystander and come away with the idea that) "if I were smarter I never would have wasted time on that"
I hear your point that the act or process of doing the learning is good, even if the end result is that you shouldn't do the thing again. Such as learning assembly but then only doing web development for a profession where you don't need to know assembly.
However, I think that the statement below might be better with a bit of nuance.
> It's not true to suggest (or to read these stories as a new bystander and come away with the idea that) "if I were smarter I never would have wasted time on that"
I would say its "not true always," in some cases doing the action really wasn't worth the time.
Related to this, I believe the sentiment people have about regretting wasting time on some endevour, is a misalignment of what their intention was to begin with.
For example, if someone wanted to compile their own kernal because they wanted to learn and understand more about their computer its unlikely that they would walk away from that experience with regret. However if they wanted to compile their own kernal because they believed that in doing so they would make 10x more money in the long run (through learning so much), and that goal failed to materialize. They would likely tell others to not waste their time learning to compile their own kernal.
Not trying to be pedantic, or argumentative, I aggree with your point deeply, however I wanted to discuss it a bit further. Let me know your thoughts.
There was this sweet spot for a while where Gentoo just worked like a breeze, Where there weren't too many and too few useflags and when I would recompile Open Office when my dorm room was too cold in cold winter nights.
Only on HN could any state that Gentoo was ever in be reflected upon as "just worked like a breeze" unironically and I mean that in a fond, loving way.
But it does! Even with the weird packages I have installed (ROCm) and a ton of accept_keywords unmasks, all I have to do is overnight updated twice a month. I haven't touched /etc/portage in months.
You could start from a later stage, I think they had a mostly binary stage 3, and emerge was generally solid, albeit slower due to the compilations.
So at least once upon a time (10+ years ago) there was this option of just using it as almost another regular distribution.
Slackware on the other hand... (I say this is in a bad way, and I think it's changed since; for Slackware for anything more complex you had to manage the entire dependency tree yourself, and it was a pain in the neck for anything not part of the not-that-many-regular-packages; nota bene: for the "beaten path" Slackware was more or less just another Linux distro, but the "beaten path" was quite narrow).
I have this botched up Debian desktop installation that I rarely use but never quite got around to make a clean install because I need an installation medium for that and i don’t have usb sticks anymore.
Do change root installs still work? I would probably give it a try again.
I know it's off topic, but I find it interesting how this sentence was so hard for me to understand.
I struggled for many seconds but couldn't go beyond "was ever" because it felt like there was some mistake, like a word was missing here or there.
In the first pass, my mind decided that "state" was a verb, and, therefore, there should be a subject appearing before it.
But I only found "any", instead of "anybody" or "anyone".
Then there is "was ever in be" which, by itself, is a weird construction.
It does makes sense in the sentence, because it is "[the] state that Gentoo was ever in" + "be reflected upon".
But since I was (unconsciously) dividing the sentence in smaller parts trying to identify the subject, the predicate, the verb, the object, or whatever would make sense for me,
cutting the sentence like that only confused me even more.
I kept going back and forth trying to imagine which word was missing,
and only after pushing through until the quotation, the whole sentence finally made sense.
Although I can't think of any example right now,
I know that it is common to use sentences with structure similar to this one,
and I see them almost daily, probably multiple times a day.
However, as a non-native speaker, this one was an actual struggle,
and I feel so good for having overcome it that I am willing comment on it.
For closure, if I was the one writing this sentence, I would probably use the active voice with an indefinite pronoun, which is also probably what my mind was expecting:
Only on HN would anyone reflect upon any state that Gentoo was ever in as "just worked like a breeze".
And I ask: were there native speakers that also couldn't understand it in a single reading?
Native English speaker here. The sentence was definitely missing a word. I read it multiple times before I got it too. Your rewrite was easier to understand.
[only on HN] could [any state [that Gentoo was ever in]] be [reflected upon [as "just worked like a breeze"] unironically]_,_ and [I] mean [that] [in a fond, loving way]
The functionality in portage was there since forever the standard packages (think of it like a standard ubuntu install) were there for forever. They only branched out and offer more binary packages. Which is absolutely great but not a new feature... The headline is still misleading.
> we’re now also offering binary packages for download and direct installation! For most architectures, this is limited to the core system and weekly updates - not so for amd64 and arm64 however. There we’ve got a stunning >20 GByte of packages on our mirrors, from LibreOffice to KDE Plasma and from Gnome to Docker. Gentoo stable, updated daily. Enjoy! And read on for more details!
They've had a binary version of the full fat linux kernel available for a few years now. Other packages like firefox-bin have been available since before I started using gentoo in 2017.
Edit: Actually looking into this more the headline is accurate. This is the first time they've provided official binaries aside from stage3s when doing the initial installation.
Gentoo's big attraction for me is Portage. It goes beyond just providing a build environment and dependency management. Ebuilds (Gentoo packages) are supported by great tooling and Eclasses that handle a lot of corner cases in builds. Developing Ebuilds feel like doing a real software project, and is great for anyone who wants to experiment with packages that are not in the official repository. Coincidentally, I just published a tool to manage unprivileged chroots for testing ebuilds.
This development will make Gentoo more accessible for a lot of people. But I guess this isn't for me. My build configuration (like CFLAGS) are never going to match the official binaries and so they will never get used.
For me this is great news for less powerful devices I use that I don't feel like setting up a binrepo for (especially when it involves cross-compilation), but still want to reuse my portage config and custom ebuilds for.
For less powerful devices it has always been possible to install Gentoo in a chroot directory on a powerful computer, using a configuration appropriate for the less powerful device, compile and install in the chroot environment any packages, and then just copy all the files from the chroot directory, through Ethernet or using a USB memory, over the root HDD/SSD of the less powerful device.
I have used Gentoo in this way on many small computers with various kinds of Atom CPUs and with a small DRAM memory and small and slow SSD/HDD.
With multiple computers of a compatible kind, the files from such a chrooted installation compiled on a powerful computer can be copied over all of them and everything will work fine. If the chrooted installation is preserved, it can be updated later with other software packages and all the changes can be propagated with rsync on all other computers.
Linux is not like Windows, which will complain when run on a different computer than that on which it was installed initially.
I am not sure what you mean, but in the chrooted environment where you compile for a distinct machine you obviously use configuration options, including compiler flags, appropriate for the target computer, not for the host computer, so "-mtune=native" cannot be used.
My point is that you do not need to setup a binrepo or any other complication like this.
You can install easily Gentoo on a very weak computer, by performing the installation on a typical desktop computer, which may run a different Linux distribution, not necessarily also Gentoo, and then just copying the files.
The Gentoo manual has always included information on how to install Gentoo inside a chrooted environment.
Although, I do somewhat think that working out good optimization flags for cross-platform compiles is a moderately unusual skill, even among people who compile things regularly. Hopefully I caveated that sufficiently, I’m not saying it is a hyper-advanced dark art or anything.
I have in the past set up code on a cluster to just compile on one node for the first run with -mtune=native because I’m lazy!
I know this, I mentioned this as "setting up a binrepo" in my comment, as I think the binrepo approach makes more sense (esp. with regards to easy updates).
It's just an unwieldy amount of extra overhead, disk space, and time, which I'd rather avoid, especially for devices I'm not fully committed to maintaining over a long period. I've tried what you mention, it's just never convenient enough to be worth the pain.
This tool [1] does the exact same thing - except that you nuke the chroot when you're done. And the reason is the same - to find all necessary dependencies. I had a small script that eventually became a Rust program. Then I kept adding features to it until it became what it is now. That's the reason why I never really got to explore the alternatives. Anyways, it's usable and nearly done.
Thanks for suggestion though. I'll take a look at it.
I agree on customizing the package flags, and features. When using Gentoo in production it became an important part of our security posture to omit the features and integrations with unused software.
That being said we've always had a build host dedicated to producing binaries, but the actual support for binaries in Gentoo hasn't been great. Unsigned serving over HTTP or NFS of compiled artifacts is about all you get. I'm really pumped to see that the new package format adds in cryptographic verification that really should have been there all along even for internal only serving.
> There are so many plugins, connectors, protocols, and often the old neglected ones turn into attack vectors.
Practically speaking this is probably true but theoretically a distribution's job should be to somehow guarantee that a specific package built their way gets the security fixes for the way they built it.
This is anyway tangential to the fact that in security "less is more".
Here you go [1], [2]. It's not completely ready yet - but it's usable. It should be OK if you plan to just modify or reuse parts of it. It currently supports btrfs backend. Plain directory backend and packaging of the tool are not done yet - but shouldn't be too hard. I was keeping it for tomorrow. Meanwhile, you can use asciidoctor to convert the docs if you need to refer it.
Yeah, my CFLAGS won’t either. But I have to say I’m tempted to script something up so I can override all the plasma-related packages to use the common ones, or something similar, so I build everything where I care about speed myself and let the gui stuff be binary packages. Would save a whole lot of build time, and I’m not sure it would be much of a loss.
I'll never forget my first time building Gentoo, in school in 2003, on the school laptop.
I started the build in the evening, and in the morning I waited for the current package to download, then closed the lid, put the laptop in the bag, took the train to school, connected to the school wifi, and continued the build in school.
It was fun to try but that's all I did, quickly moved on to something more sane.
Similar, I did this in my first year at uni, and I felt invincible. The whole idea of building your own OS distro from parts is the epitome of a so-called IKEA effect. Simply formatting that hard drive and saying goodbye to my “unique” setup (in reality as vanilla as they come) was difficult!
I tried Gentoo Prefix for a while on macOS, so I could use the same package manager across OSes, but always compiling from source got a bit tiresome on a laptop, so I went with MacPorts instead.
Does that mean they're building the package about 2 times and maybe even more if systemd support needs to be there? As far as I know, Gentoo supports musl/glibc and could support either systemd or openrc... (some packages link against systemd for some functionality)
They do mention that they support 3 different profiles - openrc, gnome/systemd and plasma/systemd. Nothing mentioned about glibc/musl split. But they seem to be aiming only for a reasonable coverage. So I'd speculate that only glibc is targeted.
Installing Gentoo was always fun for me, to get it working and the dream of a fully custom optimized machine (I guess Arch offers similar experience, minus the full optimization), but getting everything polished afterwards was always just too much for me and I switched to a packaged distro.
It seems like it would be cool for an SBC, but the compilation (or setting up cross-compilation) was always too much; now maybe it’s feasible again? But I’m too old to have the time to try!
Back when I used to mess around with this more, I never noticed much speed advantage from compiling my software vs installing binaries. What did help was understanding what was running and cutting out the things I didn't need. I'd be surprised if Gentoo offered any advantages over Arch, Slackware, etc. for that.
> It seems like it would be cool for an SBC, but the compilation (or setting up cross-compilation) was always too much;
Back in the day I had a few relatively slow machines and used to compile my kernels using distcc to offload the tasks to them. I never used cross compilation but I see it is supported so it may be a possibility for small SBCs.
Wondeful, that means that you can use binary for most part of the system and just rebuild the parts you want to customize. A great saving of compile time !
I installed gentoo on a workmachine at a new job, like a dual xeon something with 64gb ram back in a time where 8gb ram on a worstation was plenty. I had a blast. It took "only" like 3 hours to get my usual to go system.
Unfortunately I need haskell on my workstation (xmonad) and ghc can easily take alone like 5-20+ hours compile time on older computers. I compiled ghc a few times on an old t41 - it took more than a day.
Still, gentoo is my first choice. I run it on a few root servers and every workstation.
Yeah, IME it's always a tiny number of packages; IIRC my system only took a day to build because webkit took most of that time, and I didn't even try to compile firefox. (In fact, some of my setup retains the expectation of running firefox from a tarball from Mozilla specifically because I used to run gentoo)
Probably orders of magnitude less than the amount of wasted energy and greenhouse gases emissions of needlessly compiling the same javascript software over and over again on the billions of devices used to access the web.
Not to talk about all the batteries we burn out from it or the billions of devices that is replaced solely because we run so utterly inefficient software.
I was referencing directly the amount of compilation, not necessarily the power consumption. But with regards to ecological impact, I would guess you have Java on a server in your mind.
But, the biggest Java user in term of number of devices is Android. And every time you install an app, you compile it. Including every time you update it (which nowadays is... everyday?). Also it'll recompile in background to use pgo. Nowadays [1] Google could pretty much compile it server-side for most devices, and save a lot of battery wear (batteries wear when getting hot, and compiling, weirdly, heats), in addition to power consumption.
It depends on how you use it. For long running applications like services, yeah, the JIT will get the bytecode down to some very high quality machine code. If you write lots of small applications and compose them using something like Unix, then Java is very inefficient.
I heat my home. The difference between heating via compiling my kernel and heating via whatever heating you use is almost certainly negligible. But I get a custom built system out of it too.
Well, even if everyone did have heat pumps, let's think about it a bit. I upgrade my system about once a week. I reckon on average it's about 1 hour of compiling per week but let's say 2 to be safe. That's no more than 1 kWh per week. A fridge-freezer will use that in a day. An electric oven will use that in 15 minutes. An electric car will use that travelling 3-4 miles. Most households use something like 30 kWh per day to heat. Even if a heat pump made that like 10 kWh it's still a drop in the ocean. And don't forget the resulting binaries run slightly more efficiently because they are compiled specifically for my CPU, plus I enjoy it.
Do you think that many people were even using it and compiling regularly? I left it around 2010 for linux mint. Just last summer I tried it again just out of curiosity, and couldn't even get it installed. They had a broken release and of course there was some hack to re-update everything back to the last stable version, which gave me flashbacks of what Gentoo Life was all about and I stopped then and there.
As far as hobbies go, it's very far from the worst. Some people play games for 12 hours a day on kilowatt gamer PCs, or race their car around a track, or cruise around on a gas guzzling boat, etc.
probably less watts than the full screen videos and rendered play worlds that are drawn behind the main menu on any given AAA console game bought by millions of people.
given the behavior one can only think that software people generally don't care unless it bothers a user metric like 'battery life'.
Probably not that much. It's not like gentoo was widely used compared to Red Hat or Ubuntu. Also CPU power usage is a rounding error if the computer was on anyways compared to the spinning HDD, monitor etc.
you build "impure" (in nixos speak) packages for all your software? if not this is not a close comparisson. gentoo shines when you need system wide control of build flags (for perf or security)
It seems like a close comparison to me. The inputs to nix derivations aren't always directly named after the build flags they control, but ultimately they control build flags. If you disable the binary cache then you're building everything yourself with system wide control of build flags.
Purity has to do with whether you let those builds depend on things that found lying around on the system versus things that are explicitly in the derivation (i.e. a pure build can tolerate missing files by building them, an impure one may get stuck for lack of a dependency or because a dependency was not as expected).
And this page seems to talk about overriding settings for all of nixpkgs, and even something kind of like USE flags (although I agree that it's way less powerful and generic than USE flags; that's one place Gentoo wins): https://nixos.org/guides/nix-pills/nixpkgs-overriding-packag...
wow. slackware is around since 93 and loadable modules since 95... i always remembered compiling drivers in to save RAM (i.e. i was more removing drivers than adding). But somewhere around kernel 1.2 we did have to compile driver modules in, indeed.
Linux was very new. I was in the last years of high school, 93-95 exactly. I couldn't even afford a 386 :)
A friend of mine got us access to one at the work place of a relative of his and we spent nights fiddling with Slackware. We even made a serial cable so I could log in via serial from the other computer in the office, which was only a 286.
The Pi (at least 1-4, haven't looked at 5) has an extremely weird set up where the GPU actually boots the system using a binary blob before handing it over to the CPU. When mainstream linux was first beginning to support the Pi this caused a lot of issues. One of which was that GPU acceleration was not available. It was in the Pi foundation's own distro (Raspbian/RaspberryPi OS) but not in others. One of the first to get it working was Gentoo. I really, really wanted to get Open Morrowind working so I tried it for a while.
Then I had to re-emerge a software package to change my timezone and I was just done. Thats way more complicated then it should be or needs to be.
One of the reasons I moved from Gentoo 15+ years ago to FreeBSD was that it was mandatory to compile everything while FreeBSD provided binary packages.
It may be not that important today - but it was a game changer with single CPU core and 1GB RAM.
Sad to say that you are completely right, for the longest time after being on gentoo for a long time I switched to macos for a few years, and was using gentoo prefix, which is vastly superior than Homebrew. I added patches to fix upstream llvm to work in gentoo prefix on macos almost a decade ago [1]
I finally met someone at the GSoC reunion that wanted to get me maintainer status, but I never got them to follow through. He had already warned me that it would be a complicated task to accomplish. I kept mentioning prefix needing binaries as well. Imagine if gentoo prefix had been as easy to install packages as homebrew on a mac.
It's sad, but gentoo is a good example of why an open source project that is technically superior, cannot survive inferior solutions without good stewardship if they disregard some basic end user quality of life features. I would argue that that is also what killed opensolaris/illumos(which is basically on life support), because the people in charge could never get past their elitism and decide that for community engagement the kernel build needs something more simple that 100 layers of nested incomprehensible makefile/shell spaghetti.
Is it too late? Gentoo has always been a niche within the already niche Linux community, but they seem to keep chugging along happily. Are they having some problems?
They have had the binary packaging capability for a long time now. It just didn't make sense to use it on a global scale considering the vast combination of packages they could generate (due to USE flags, profiles, etc) and the infrastructure needed to distribute them. They seem to have decided to offer binaries for the most common configurations. This isn't a major change. Perhaps the infrastructure also makes more sense than before.
I guess what I’m trying to say is, in order to define “too late” we need to define an objective and then figure out if we’ve passed the point where it is possible.
From the outside it looks like the Gentoo community is happy and stable being small. And that’s good.
I’m not sure Google copying some of their software and putting it on Chromebooks is a huge win for the community, although I bet some of the Gentoo devs are proud.
Any background on why (now)? The post is a bit sparse.
(I'm also one of those who left Gentoo for Ubuntu, both because compilation made it needlessly slow to wait for a tool I needed, and because emerge was just so slow compared to apt. Ebuilds were awesome at a time when dpkg build tooling seemed to change completely once a year.)
So, they just hold more binary packages than before? How tame. They should push further and make it easy for people using the same CFLAGS to share their binaries through torrents based on decentralized trust.
284 comments
[ 4.6 ms ] story [ 90.4 ms ] threadAdditionally, Gentoo is the largest distro with the source-based package manager USP. I suppose you could take issue with that approach for reasons like the systemd maintainer does where he claims it wastes CPU-cycles and time. Personally I disagree with that assessment since reproducible builds are a vital part of FOSS.
Never had a bad experience with Arch but the way they use the AUR as a crutch is a bit off-putting. It's a little bit like a giant gentoo overlay but pushes all the complexity onto the user.
Well, it is unique. It isn't another copycat or cosmetic derivative of Ubuntu.
Actually, for the most part, it's in a league of its own.
Chrome, git, ssh, docker, netbeans
I know it's not mainstream to use NetBeans these days, but I don't care, I'm just used to it and it gets the job done. Maybe I'm just getting old.
There's nothing really special about Debian as well, it could as well be Ubuntu, Mint or anything else that's plug n play. I'm just used to Debian, and it comes with less junk I don't need installed by default.
After running my home servers on it, one release wasn't too far behind to run as a desktop, and I've been happy here ever since.
Xfce desktop, move the panel to the bottom, install applications. Use it for the next days/months/years until I decide to look around again.
Over time is the key here. A package here, a small config there, and after some time, that installation becomes so unique that it starts reading your mind.
Non-breaking updates is the icing on the cake.
The biggest point is you install Debian once, or when your processor architecture changes.
--Sent from my a 6 year old Debian installation.
The only cool thing about it is that it’s declarative: nix-darwin everything and a fully working and customized machine is up in 10 minutes with one command
At best, we are at least a bit confused about it all.
If you can, try to get access to OpenVMS or Cisco IOS, it's an entirely different world in terms of user experience.
Nowadays most people are swimming in CPU cores and gigabytes of ram and terabytes of solid-state memory, so fine-tuning is a waste of time (unless you play bleeding-edge games). But it wasn't always such.
Unless you run javascript 'applications'. Games are already optimized.
What AAA titles have you played around launch in the last 5 or so years?
Yeah, I'm calling bullshit on this one. At least, it doesn't line up with my experience. In my experience, games are optimized _just enough_ for a decent playing experience (and not always then). Games devs, as a whole, are the worst offenders of expecting their users to just throw more money (hardware) at the software to achieve usable/enjoyable experiences. There are, of course, exceptions. But, for every Carmack, there's 10s of thousands of developers scrambling to make their deadline, doing just enough to ship.
The games themselves though are a different story. Outside of open sources games (which are usually less demanding than commercial ones), you don't have the source code to rebuild it. Even if you did, enabling optimizations beyond what the developer used risks breakage so you will have to do your own testing. Even then, simply rebuilding the software wouldn't address the quality of the code created by those developers who are scrambling to meet a deadline with as little effort as possible.
Back in the day, I spent a fair amount of time working on gutting the DLL because frankly, it was atrocious. My memory is a little fuzzy as it's been +10 years since I've looked at it, but things I remember seeing:
* over use of C/C++ preprocessor macros where an inline function would have been more appropriate to say, get the array/list of CvPlots (tiles) that needed to be iterated over all the time. * lack of hoisting loop invariants out of loops. It is common to see usages of the non-trivial macros above in the bodies of tight loops, often tight nested loops. Optimizing compilers are great, but they're not _that_ great. * the exposure of the C++ types to Python was done...poorly. It was done using Boost Python (which, while a great library for its day had a _huge_ learning curve). For every Cv type in the C++ DLL, there was a corresponding Cy type to expose it to Python. Collection types were _copied_ every call into Python code, which was quite frequent. The collections should have been done as proxies into the underlying C++ collections, instead of as copies.
Most of the changes I made were quite small, but over the course of a month of part-time hacking on it, I'd gotten late game turns down to a couple of minutes from 30-minutes and memory usage was extremely reduced; and I never did get around to fixing the Python wrapper as it would have too intrusive to fix it properly. I could have made more aggressive changes if I had full access to the source, but being constrained by DLL boundaries and C++ types being exported limited what could be done w/o breaking the ABI (had to be extremely careful about not changing object sizes, affecting vtable layout, etc).
Frankly, I doubt the developers spent very much time at all, if any, with a profiler during the course of development with the game.
Civ had been a 2d game until then, it was their first 3d title.
Not to mention that it was turn based strategy, and the main performance problem was AI turn length in the endgame.
I used to believe that and was a huge Gentoo user for years back when it was initially released. Then one day I benchmarked Gentoo and a default RedHat install on every work load I cared about, and everything was within the margin of error.
For a single machine home install, the biggest value Gentoo has to offer is the learning experience. I ran it for about a year like 4 years ago, and I definitely learned a lot in that time. Hopped around a bit and I've since landed on GNU Guix, and I'm probably set for life now in the GNU world.
The flags were just icing on that cake.
A full day compiling stuff only for the base install, let alone everything else I would eventually need.
On my case, I decided to become another Scientific Linux tester.
But computing power is much higher now. The same compilation now would probably take 1-2 hours, max. Updates would be super fast.
Gentoo itself is considered generally stable and a pretty solid distribution, or it used to be.
I wonder if these days the flexibility and the engineering behind Gentoo might be worth taking another go at it.
I rather use programming languages ecosystems that favour binary libraries for a reason.
I was working with a 486 around 1995. Compiling your own software was the norm and compiling your own kernel could have significant performance benefits (even if it was just to conserve the limited memory supported by machines of the day, to head off some of the swapping). By the time I learned of Gentoo, that was not really the case: most of the software one could obtain was provided in binary form and compiler optimizations were much less relevant (unless you had a special workload).
The tooling provided is important too. I was using NetBSD for a while. For the most part you just started the compilation process and walked away until it was done. (I imagine Portage is similar.) You didn't get the instant gratification, but it was not time intensive in the sense that you had to attend to the process. That was very much unlike my earlier experiences in compiling software for Linux, stuff that isn't in the repos, since it did have to be attended to.
Later on, to take advantage of my Pentium based computer, I would get Mandrake, with its i585 optimized packages.
Most of my Linux based software would be sourced via Linux magazines CD-ROMs, or Walnut Creek CD-ROM collections.
It's really not much different than working with Arch in terms of complexity. Initial setup takes a bit, but if you've installed arch you are pretty familiar with everything you need (in fact, arch docs are helpful for a gentoo setup :D).
The docs are VERY good and easy to google.
Compilation time can be nasty depending on what you install but not terribly bad. I just rebuilt the world because a GCC update broke lto that I'm running. With about 2k packages that took about 6 hours to complete on a Ryzen 7950.
General updates take almost no time at all (especially using git for syncing). Usually less than 10 minutes often less than 1. As I write this, I'm currently rebuilding kde (if you are using your computer, rebuilding doesn't really get in the way. Especially if you are already working with a multicore system).
I’m not so sure. A lot of the power comes from multiple cores. Years ago I had one core, now I have eight. A lot of the compiles don’t use all the cores.
Software has also gotten bigger. rustc is huge, for example. It didn’t even exist when I used Gentoo years ago.
These days I’m on the Mac and I just switched to Homebrew after using Macports for years. It was for one of the same reasons I stopped using Gentoo: compiling takes too long. Whenever I upgraded Mac OS versions, Macports required me to recompile everything. This was no problem at all for, say, tree. But something was pulling in gcc (emacs needed it for some reason??) and this took ages to compile.
At least Macports worked though. When I used Gentoo, it took so long to compile things that I would leave it overnight, and of course often in the morning I would see that the compilation stopped halfway through because something was broken. Hopefully that’s improved. Or of course maybe the binary packages will help with this.
But if I wanted a build-your-own, rolling-release binary system, I don’t see why I wouldn’t just use Arch.
I used first Xenix in 1993-1994, and naturally wasn't compiling it from scratch.
But yea, now-a-days I'm on Ubuntu LTS.
I've installed Gentoo literally two times. Once per PC. Been using it for years. It's not like you have to keep tweaking it. It does help if you run a basic system like me, though (no DE, simple tiling WM, don't use much apart from Emacs and Firefox).
All of us who at one point compiled our own kernels and now no longer do, are the killers that we are partly because we did things like that at least for a bit. It only makes sense not to now, after having done it.
It's not true to suggest (or to read these stories as a new bystander and come away with the idea that) "if I were smarter I never would have wasted time on that"
However, I think that the statement below might be better with a bit of nuance.
> It's not true to suggest (or to read these stories as a new bystander and come away with the idea that) "if I were smarter I never would have wasted time on that"
I would say its "not true always," in some cases doing the action really wasn't worth the time.
Related to this, I believe the sentiment people have about regretting wasting time on some endevour, is a misalignment of what their intention was to begin with.
For example, if someone wanted to compile their own kernal because they wanted to learn and understand more about their computer its unlikely that they would walk away from that experience with regret. However if they wanted to compile their own kernal because they believed that in doing so they would make 10x more money in the long run (through learning so much), and that goal failed to materialize. They would likely tell others to not waste their time learning to compile their own kernal.
Not trying to be pedantic, or argumentative, I aggree with your point deeply, however I wanted to discuss it a bit further. Let me know your thoughts.
So at least once upon a time (10+ years ago) there was this option of just using it as almost another regular distribution.
Slackware on the other hand... (I say this is in a bad way, and I think it's changed since; for Slackware for anything more complex you had to manage the entire dependency tree yourself, and it was a pain in the neck for anything not part of the not-that-many-regular-packages; nota bene: for the "beaten path" Slackware was more or less just another Linux distro, but the "beaten path" was quite narrow).
Do change root installs still work? I would probably give it a try again.
In the first pass, my mind decided that "state" was a verb, and, therefore, there should be a subject appearing before it. But I only found "any", instead of "anybody" or "anyone". Then there is "was ever in be" which, by itself, is a weird construction. It does makes sense in the sentence, because it is "[the] state that Gentoo was ever in" + "be reflected upon". But since I was (unconsciously) dividing the sentence in smaller parts trying to identify the subject, the predicate, the verb, the object, or whatever would make sense for me, cutting the sentence like that only confused me even more. I kept going back and forth trying to imagine which word was missing, and only after pushing through until the quotation, the whole sentence finally made sense.
Although I can't think of any example right now, I know that it is common to use sentences with structure similar to this one, and I see them almost daily, probably multiple times a day. However, as a non-native speaker, this one was an actual struggle, and I feel so good for having overcome it that I am willing comment on it.
For closure, if I was the one writing this sentence, I would probably use the active voice with an indefinite pronoun, which is also probably what my mind was expecting:
And I ask: were there native speakers that also couldn't understand it in a single reading?[only on HN] could [any state [that Gentoo was ever in]] be [reflected upon [as "just worked like a breeze"] unironically]_,_ and [I] mean [that] [in a fond, loving way]
https://www.akhuettel.de/~huettel/plots/mirrors/binpackages-...
> we’re now also offering binary packages for download and direct installation! For most architectures, this is limited to the core system and weekly updates - not so for amd64 and arm64 however. There we’ve got a stunning >20 GByte of packages on our mirrors, from LibreOffice to KDE Plasma and from Gnome to Docker. Gentoo stable, updated daily. Enjoy! And read on for more details!
they did decades ago.
Edit: Actually looking into this more the headline is accurate. This is the first time they've provided official binaries aside from stage3s when doing the initial installation.
This development will make Gentoo more accessible for a lot of people. But I guess this isn't for me. My build configuration (like CFLAGS) are never going to match the official binaries and so they will never get used.
But to be honest, I haven't looked at binrepos so far. Perhaps your reply is a good reason to.
I have used Gentoo in this way on many small computers with various kinds of Atom CPUs and with a small DRAM memory and small and slow SSD/HDD.
With multiple computers of a compatible kind, the files from such a chrooted installation compiled on a powerful computer can be copied over all of them and everything will work fine. If the chrooted installation is preserved, it can be updated later with other software packages and all the changes can be propagated with rsync on all other computers.
Linux is not like Windows, which will complain when run on a different computer than that on which it was installed initially.
https://forums.gentoo.org/viewtopic-t-245041-start-25.html
My point is that you do not need to setup a binrepo or any other complication like this.
You can install easily Gentoo on a very weak computer, by performing the installation on a typical desktop computer, which may run a different Linux distribution, not necessarily also Gentoo, and then just copying the files.
The Gentoo manual has always included information on how to install Gentoo inside a chrooted environment.
Right, I am sure they intended it to be absurd in an amusing way.
Although, I do somewhat think that working out good optimization flags for cross-platform compiles is a moderately unusual skill, even among people who compile things regularly. Hopefully I caveated that sufficiently, I’m not saying it is a hyper-advanced dark art or anything.
I have in the past set up code on a cluster to just compile on one node for the first run with -mtune=native because I’m lazy!
It's just an unwieldy amount of extra overhead, disk space, and time, which I'd rather avoid, especially for devices I'm not fully committed to maintaining over a long period. I've tried what you mention, it's just never convenient enough to be worth the pain.
You should check out what ChromeOS is doing. They are using bazel to execute ebuilds inside an ephemeral chroot: https://chromium.googlesource.com/chromiumos/bazel/+/refs/he...
This way it's guaranteed that no undeclared dependencies get used.
Thanks for suggestion though. I'll take a look at it.
[1] https://crates.io/crates/genpac
[2] https://wiki.gentoo.org/wiki/Chroot_for_package_testing
That being said we've always had a build host dedicated to producing binaries, but the actual support for binaries in Gentoo hasn't been great. Unsigned serving over HTTP or NFS of compiled artifacts is about all you get. I'm really pumped to see that the new package format adds in cryptographic verification that really should have been there all along even for internal only serving.
That's one of the most compelling cases I've heard for running something like Gentoo in prod.
There are so many plugins, connectors, protocols, and often the old neglected ones turn into attack vectors.
Practically speaking this is probably true but theoretically a distribution's job should be to somehow guarantee that a specific package built their way gets the security fixes for the way they built it.
This is anyway tangential to the fact that in security "less is more".
[1] https://git.sr.ht/~gokuldas/genpac
[2] https://crates.io/crates/genpac
I started the build in the evening, and in the morning I waited for the current package to download, then closed the lid, put the laptop in the bag, took the train to school, connected to the school wifi, and continued the build in school.
It was fun to try but that's all I did, quickly moved on to something more sane.
It seems like it would be cool for an SBC, but the compilation (or setting up cross-compilation) was always too much; now maybe it’s feasible again? But I’m too old to have the time to try!
Back when I used to mess around with this more, I never noticed much speed advantage from compiling my software vs installing binaries. What did help was understanding what was running and cutting out the things I didn't need. I'd be surprised if Gentoo offered any advantages over Arch, Slackware, etc. for that.
Back in the day I had a few relatively slow machines and used to compile my kernels using distcc to offload the tasks to them. I never used cross compilation but I see it is supported so it may be a possibility for small SBCs.
https://www.distcc.org/
I installed gentoo on a workmachine at a new job, like a dual xeon something with 64gb ram back in a time where 8gb ram on a worstation was plenty. I had a blast. It took "only" like 3 hours to get my usual to go system.
Unfortunately I need haskell on my workstation (xmonad) and ghc can easily take alone like 5-20+ hours compile time on older computers. I compiled ghc a few times on an old t41 - it took more than a day.
Still, gentoo is my first choice. I run it on a few root servers and every workstation.
But, the biggest Java user in term of number of devices is Android. And every time you install an app, you compile it. Including every time you update it (which nowadays is... everyday?). Also it'll recompile in background to use pgo. Nowadays [1] Google could pretty much compile it server-side for most devices, and save a lot of battery wear (batteries wear when getting hot, and compiling, weirdly, heats), in addition to power consumption.
I don't cool my home, though.
given the behavior one can only think that software people generally don't care unless it bothers a user metric like 'battery life'.
(microsoft's anti-virus)
I don't personally believe in the CFLAGS "performance" micro-optimization though so that's not really something that matters to me. Security is pretty hardened by default on Nix: https://nixos.org/manual/nixpkgs/stable/#sec-hardening-flags...
Then you haven't read their own docs but are providing lessons online for it :/
I used quotes because it is their term for that procedure.
Purity has to do with whether you let those builds depend on things that found lying around on the system versus things that are explicitly in the derivation (i.e. a pure build can tolerate missing files by building them, an impure one may get stuck for lack of a dependency or because a dependency was not as expected).
I'm pretty sure nixos can do that, though?
There's this for CFLAGs on a single package:
And this page seems to talk about overriding settings for all of nixpkgs, and even something kind of like USE flags (although I agree that it's way less powerful and generic than USE flags; that's one place Gentoo wins): https://nixos.org/guides/nix-pills/nixpkgs-overriding-packag...I haven't done it since Slackware came on floppies and you mostly had to recompile your kernel to get the right drivers in.
Even did a linux from scratch once, later, to see how it goes. Then went along with my business.
A friend of mine got us access to one at the work place of a relative of his and we spent nights fiddling with Slackware. We even made a serial cable so I could log in via serial from the other computer in the office, which was only a 286.
Then I had to re-emerge a software package to change my timezone and I was just done. Thats way more complicated then it should be or needs to be.
One of the reasons I moved from Gentoo 15+ years ago to FreeBSD was that it was mandatory to compile everything while FreeBSD provided binary packages.
It may be not that important today - but it was a game changer with single CPU core and 1GB RAM.
I finally met someone at the GSoC reunion that wanted to get me maintainer status, but I never got them to follow through. He had already warned me that it would be a complicated task to accomplish. I kept mentioning prefix needing binaries as well. Imagine if gentoo prefix had been as easy to install packages as homebrew on a mac.
It's sad, but gentoo is a good example of why an open source project that is technically superior, cannot survive inferior solutions without good stewardship if they disregard some basic end user quality of life features. I would argue that that is also what killed opensolaris/illumos(which is basically on life support), because the people in charge could never get past their elitism and decide that for community engagement the kernel build needs something more simple that 100 layers of nested incomprehensible makefile/shell spaghetti.
[1] https://github.com/fishman/timebomb-gentoo-osx-overlay/tree/...
https://wiki.gentoo.org/wiki/ChromeOS
Sounds like they are just using parts of it and not Gentoo itself.
From the outside it looks like the Gentoo community is happy and stable being small. And that’s good.
I’m not sure Google copying some of their software and putting it on Chromebooks is a huge win for the community, although I bet some of the Gentoo devs are proud.
(I'm also one of those who left Gentoo for Ubuntu, both because compilation made it needlessly slow to wait for a tool I needed, and because emerge was just so slow compared to apt. Ebuilds were awesome at a time when dpkg build tooling seemed to change completely once a year.)