Ask HN: Linux from Scratch.. Should I try it?
For a long time now, I've been pondering over whether I should dive into Linux from Scratch (http://www.linuxfromscratch.org).
I've been a Linux user (Ubuntu) for over three years now, and the stuff I've learned is only by trying/troubleshooting and so on.. I've never really read a book about UNIX (even though we had the course in college).
So, have any of you HNers tried out the LFS project from start to finish? Is it something that'll teach me more about how the Linux operating system works (of which I have a pretty decent idea), or is it something that help me towards my goal of becoming a Linux power user ?
Any opinions or help would be awesome!
58 comments
[ 3.2 ms ] story [ 119 ms ] threadI never done anything else with my build at the time, but I did join the Arch Linux community which is a distro based on LFS with a package manager. So I effectively went from knowing very little about Linux to being a package maintainer for a distro over the course of a weekend.
Personally, I'm still interested in taking some time to do it, and I've actually been trying to convince some friends of mine to spend a weekend, with each of getting a system hacked together. There appears to be lots of interest, but getting everyone to agree on a time might just be impossible. Perhaps I'll dig that project out of the closet this weekend and get it done myself...
In any case, if you're looking to understand Linux better, it will probably help, but I would suggest that there are probably better avenues for learning it. Advanced Programming in the Unix Environment will teach you all the subsystems you have to interact with (albeit, typically from a programming view), and Understanding the Linux Kernel will teach you the ins and outs of the kernel like nothing else. There may be better books out there for what you're looking for, but those two I have direct experience and would highly recommend.
On the other hand, if this is something you're interested in doing, shouldn't that be reason enough? You'll likely learn something on the journey, though it may not necessarily be Linux-related, and you'll be doing something you want to be doing.
It depends what your goal is. Linux from scratch will mainly teach you (wait for it) how to build linux from scratch. Systems that are not quite "from scratch" but are lower level than Ubuntu (eg, Gentoo, Arch (others?)) will also teach you some of this. I used Gentoo for years, and after compiling about a dozen kernels in a row to get things to work how I wanted, I guess you could say I'd learned something, if only 'what kernel options I want to enable on this hardware'. :)
But building Linux won't really teach you to _use_ it. If you want to be a power user in the long run, your time (and LFS will take a _lot_ of time!) might be better spent learning your shell/package manager/scripting language of choice in greater depth, compiling your own kernels and adding them as options in grub/lifo, that sort of thing.
All that said, if the LFS process sounds really appealing to you, go for it. You'll definitely learn _something_, but just maybe not as much as you're hoping.
LFS is something which people can use to create their own distros (I think I read somewhere that ArchLinux was created using LFS), which reminds me you could even give ArchLinux a shot. Pretty nice distro. ;-)
Daniel Robbins is also working on new Gentoo kernels and portage trees (and has a _ton_ of Linux documentation/tutorials) over at http://funtoo.org
I used Gentoo back when they still had regular release numbers instead of dates (1.2 I think) and stage 1 installs. Back then the docs were complete and extremely comprehensive and the Gentoo Forums were amazingly resourceful. The distro and the community have slowly degraded in quality since drobbins left. (This is not necessarily related to his departure.)
Anyways, it was great for learning. Before that I tried Red Hat and was confused as hell. I couldn't troubleshoot anything. With Gentoo I constantly broke things but was able to find out how to fix them relatively easily. I quickly built up an understanding of how everything fit together.
Maybe Gentoo is still useful in this regard but it certainly is not what it used to be.
I installed it a long time ago, and I remember learning a ton. I suggest that you dig deeper into any topics you read about in the manual that you haven't heard before. I remember lovely all-nighters learning about dynamic loaders, binutils, object files, ELF and ld and so on :)
When I used to recompile the kernel I also learned a lot by trying to figure on my own whether some options should be enabled or not. This resulted in several unbootable kernels, but I also learned what was IPC, NUMA, the IO-APIC and other stuff that wouldn't come my way otherwise.
The downside is a lot of time spent waiting for stuff to compile before you can move on, so make sure you have complementary material for those times.
What would you recommend?
2) Why exactly have you been pondering over it for so long? The cost of trying is next to zero. At worst, you'll end up with an unusable test machine, give up, and try something else. Simply going ahead and trying is probably better use of your time and energy than pondering for a long time and looking for advice as to whether you should try.
Why not ? A lot of us like to learn just for the sake of learning. I say becoming a Linux power user is a worthy goal in itself.
I agree with your point 2), it's fair easier to dive in and try it, rather than ask in forums if you should do it. It's not like it's going to cost you anything. Just download Virtual Box and do it.
2) It's mainly because I'm halfway through the alpha-launch of my web-startup, and have a little time to spare, which I want to use productively. I haven't been 'actively wasting time pondering over it' , it's more like 'been at the back of my mind' sort of a thing.
//edited: making my point more clear.
Try living out of a TTY for about a month. You will come out with rustic beard, red eyes and a triumphant smile. No Gnome or gdm, screen to take care of your "windows", mpd (ncmpc) to play your music, elinks to browse the web, surfraw to search, vi to edit your files, irssi for irc, bitlbee for msn etc. Control your daemons with rc.conf and begin your journey :)
Such knowledge seems most useful if you're working on some hardware device with linux embedded -- where it will be interesting to know about all the parts you can leave off or replace with smaller alternatives.
1) The LFS manual will get you to a base Linux system. Think of it as the minimal system required to get you up and running and nothing more. You will need to use BLFS or something else to get yourself to point where you will most likely be comfortable (especially as an Ubuntu user).
2) There is no package management by default. The manual recommends some ways to handle this including rebuilding the entire system from scratch (!) when you want to upgrade. So, after all of the work you do (which is mostly watching things compile) you are still responsible for maintaining all of your installed applications. For the average Linux system, this is a lot of packages.
3) All of that said, it is an interesting process to go through if you would like a little more insight into how a Linux system works, assuming that you take the time to understand what you are doing (and why) and not just blindly follow the manual.
Shortly after completing the project (completed over a weekend), I found that I had learned a lot but had no desire to maintain the system. I believe (don't quote me on this) that the creator of Arch Linux started from LFS.
In the end, I use Linux Mint. It's like Ubuntu with a better organized interface (IMHO) and more attention to visual detail.
Should you try LFS? Sure, why not? It won't kill you and maybe it will spur some new project or idea in you. Worst case scenario, you spend a few hours rebuilding a machine.
Hope this helps.
However, you can learn linux pretty darn well without ever building a system from scratch. You don't need to do that to become a "Power User" (though, for some, it helps).
The thing is, a system can work in many wildly different ways: just look at the system init procedure: there's file-rc, traditional sysvinit, various paralellizations, dependancy based init, systemd, and a whole bunch of other stuff.
There's also distributions which do not follow the FHS, for one reason or the other, and those work differently from the ones that do.
And so on, and so forth...
But anyway, if you have a lot of time, and you think it would be useful for you - go ahead.
Myself, I never found it neccessary to indulge in building a system from scratch.
Personally, I would start with a target "result" in mind. Maybe you want to create a minimal "server" layout or "lean desktop". Once you hit that point and have documented what it took to get there, throw it away and start again. Think about how to automate some aspects of the process.
I'm actually doing this right now with my OH SO COPIOUS free time. Building a VM-only target distro that's designed to be minimal and run only in a VM. The rest is expected to be managed entirely from tools like Chef or Puppet.
It's sort of a "put up or shut up" to this blog post: http://goo.gl/rML9
Why? Well - unless you have a very specific reason [1] it seems to me that LFS is too much time invested for not enough return. As others commented, you can go a long way with a source based distribution. Learning a different distribution's way to do things might already be pretty interesting (and arguably more useful: You'll probably never hit a LFS system in real use that needs fixing, but you might stumble upon a solaris/freebsd machine or even just a heap of Suse/RedHat/Debian/Slackware machines) and challenging. Start with that!
LFS is a purely academic thing for me. You follow the guide, just to end up with a system that is really nothing for daily use (or - needs too much work).
All the above assumes that your definition of a linux poweruser is someone that understands the darkest secrets of the OS. Which - is not exactly necessary. I know plenty of users that I'd consider power users because they focus on things they actually need (and improve those, get better at those), not things that somehow make this possible.
Cue the car analogy: A race driver would be a power user for me, I don't require that he can build a race car from scratch..
[1]: Some reasons I could imagine to change the answer to a yes:
1) You want to really build your own distribution for an embedded device and don't like openembedded
2) You are (or plan to) a system administrator and want to learn more underlying details for your job.
3) You plan to get into system-level programming/contribution and want to understand the status quo first.
If you foresee building any custom embedded Linux systems, even if you like openembedded, it gives you a good picture of what's going on down there (and a good picture of what openembedded is doing for you).
If you want to learn system administration, I really don't see it being that helpful. You'll learn what all the components of the system are and how to build them, but not necessarily how to use them.
I'd suggest that if you're a hacker type, you might want to try it anyway, just for fun. :-) But in general the knowledge you get from building LFS or Gentoo or similar isn't actually that useful from a system administrator/power user perspective.
I'll also second the comments from other people who built it, and then abandoned it: LFS is too finicky to manage and too hard to upgrade for me to do any real work with it. For me, it's a toy system that I play around with, then reboot into Debian/Ubuntu/Fedora/whatever to go get real work done.
What may be more interesting is to try one of the minimalistic BSDs (NetBSD or OpenBSD). You will likely learn far more, e.g. that many GNU-isms that you may have gotten accustomed to are not UNIX-isms. You'll be able to build your own customized NetBSD systems using the excellent build infrastructure. You can play with some of the nice packet filters (ipf/pf). Or spend time fixing pkgsrc ports that fail ;).
Once you get your head around it you'll realise that Linux is a complete mess and you'll understand more about UNIX and UN*X.
If that's your goal, I recommend it.
Back then, I only had a single computer, so while I didn't really need your computer for day-to-day work (or at least, it was rarely required to maintain my income), it forced me to carry on with LFS long past the book, all the way up to installing GNOME and OpenOffice. It was very worthwhile, and the residual knowledge I have from that process is still used frequently.
As another commenter pointed out, given the zero cost (other than your time) nature of trying something out, why ponder? If your pondering is that you don't want to mess up your current system, then install Virtual Box and run whatever else you want to learn from in a virtual machine inside VB.
The problem you may encounter, given that you say you've been only using Ubuntu, is that LFS may be taking a dive from the high platform into the deep end of the pool when you are not ready yet for that experience. You may want to ease into that just a little bit more.
My suggestion for you is to go download yourself a copy of the latest Slackware distribution, and start there. You'll learn 95% of what LFS will teach you just from Slackware alone, without also taking that high dive before you are ready.
And, again, if you don't want to kill your current system, then just install Virtual Box and then install as many distro's as you please (including LFS) as VM's. You get the same experience, without breaking your current working system.
The basic process as I remember it was to bootstrap a system by making a new partition, setting up a minimal directory tree, building a new C compiler (& libs/associated tools) then chrooting to it and using it to build up the rest of the system.
It was a fun process but tiring and time consuming. I wouldn't do it again.
If you want to become a system administrator go for it. Otherwise probably no.
If you're thinking of a Linux power user as someone who lives closer to the bare metal, I'd highly recommend Arch (http://www.archlinux.org/). Some people recommend Gentoo, I personally prefer Arch. Arch is like LFS without (all of) the pain, plus you get package management.
Go for it!
Based on that experience I wouldn't recommend LFS. Either spend your time studying some prebuilt system (I remember one time installing Debian via chroot, which came out extremely minimal, and that was somewhat interesting), or actually design and build your own Linux-based system. Outline (maybe on paper) what parts you want (maybe you have heard of some trendy init replacement, or you just hate pulseaudio and love oss, etc). Then research what you need to run that stuff. It will take a lot more time, but it will also be much more rewarding
When I was a kid I was taking a sailing course when my uncle (a seasoned sailor) asked me what we've been doing. I told him we've been just sailing around as we pleased and he said that was pretty bad, like learning how to drive by driving in an empty parking lot. He said we should try to get to this buoy first, then to this one etc.
I suggest (unless you are very interested in desktops) you get a Linux server in the cloud (rackspace is $12/mo), this will get you connectivity, it will force you to use the command line and you can always re-install it with a click if you mess up.
Make a list of objectives. If you need some suggestions:
- create a mail server, add users, manage forwarders, aliases, spam, why your messages don't reach hotmail etc.
- play with the firewall, how to deal with a bit of DoS, what about those pesky ssh brute-force scans?
- install a honeypot
- create DNS server
- install monitoring tools
- get some web app (wordpress, drupal for example), benchmark/stress test the heck out of them, try changing caching and other things, how to optimize apache etc
- write some shell scripts to automate/check some of the things you've done
etc