As the author of Linux Factory, that’s exactly what I built it for. I would love to help guide you and get you started on your project. Please create an issue in the GitHub project and I’ll further guide you there.
Anything with a learning curve steeper than "throw your files in a Dockerfile and call it a day" unfortunately tends to be considered "not sane". There is a whole market when it comes to making new tools for people who "don't have the time to RTFM". And then for tools helping to use those tools.
Word of warning - I don't know how much you've looked into it yet but depending on your needs from Jetson the Nvidia stack gets really complicated really quickly.
I know people have frustrations with Nivida supplied Jetpack but when you end up trying to figure out how to roll your own support for TensorRT, DLA, cuDNN, etc (on ARM64, of course) many people bail and run screaming back to Jetpack.
IMO, start small, maybe with a RPi. Possibly Nvidia provides layers for the Jetson (I haven't seen that), but in my experience the risk is that those layers are super complicated (and fairly messy). I think it's better to learn how Yocto works first, rather than learning how to hack the layers provided by the manufacturer.
Also it helped me to learn a bit how other distros are built: I tried Linux from scratch, then got interested in Alpine, and then in how PostmarketOS got started based on Alpine tooling.
After all, a distro is "just" a bunch of packages, which are archives containing files that get decompressed in the right place. Yocto provides tools to create those packages (just like mainstream Linux distro). What Yocto adds on top is tools to manage cross-compiling (toolchains, a way to create an SDK, etc) and a system of layering that helps organizing them ("those packages go specifically with the RPi, so I will put them in a layer called "RPi" and I will call that a "BSP"", etc).
Don't get me wrong, there is quite a lot. But there is no magic, the goal is always to create packages that will put the right files in the right place :-).
I just started learning Yocto a few weeks ago, and just shipped my first embedded image. Here are a couple of learnings and thoughts:
1. Start with an empty image and build your way up. There can be some cross-effects between packages depending on what you exactly do, it's a lot easier to debug if you go step by step. The base image recipes are all listed in the documentation, start with one of them.[1]
2. Lean into Bitbake and the whole thing. The more you try to fight it, the more it fights back. I liked this guide as a starting point to get a feel for it.[2]
3. Find a way to quickly iterate on your images. Nothing worse than losing your train of thought! Since I wanted to include update functionality I added swupdate, which integrates reasonably well with Yocto[3] - although the documentation is a bit difficult to read at times. But being able to simply push an update through WiFi is really, really nice.
4. If you do some lower-level stuff you'll inevitably need to create configuration fragments. This process was REALLY opaque to me. Say you want to set build options for U-Boot - you'll have to: 1) run `bitbake -c menuconfig u-boot` to open the config menu; 2) when closing the menu, you save the configuration in the default location; 3) then you run `bitbake -c diffconfig u-boot` to generate a config fragment; 4) you copy the config fragment into your recipe-append-folder and add it to the SRC_URI variable; 5) you run `bitbake -c clean u-boot`. When U-Boot is built the next time it will apply your configuration fragment to the build process.
These were the most important things that came to mind. Hope they help, and good luck on your journey! :)
The real value with Yocto is the community and vendor supported/supplied BSPs (board support packages). The gains from these (typically extensive and robust) BSPs usually more than offsets other rough edges with Yocto.
I completely agree; I worked on a project with a custom board and we tried Yocto at first because the silicon vendor pushed it, and it was indeed a complete nightmare. We finally switched to Buildroot and life was much easier.
I prefer the approach of AntiX and MX Linux. Start from a clean live distro, mod it to your taste, master it to a new Live ISO. You can also start from a normal MX/antiX system and make a live ISO backup. You can also install those ISOs. Systemd is optional on MX and disabled by default. LOTS of live and remaster options on those systems. I find myself more on live than on installed systems now. It's basically like working with FS snapshots.
A similar distro with remaster tools based on Devuan is Refracta.
There's almost no circumstance I can imagine where giving up a solid usable well known OS sounds tempting to me. Getting Debian running on PogoPlugs and OpenWRT devices has been a repeated ordeal I'vd gladly traversed. Having a well behaved normal OS that meets many user's expectations (but has your special stuff regularly installed!) is divine.
I used to run Debian Live, and appreciated the tiny amount of shell hacking they have to layer persistent storage filesystem atop a ro filesystem- identical to how theres a ro container image then the mutable rw layer atop it... but a decade older!
I find that the pretense of running "live" is largely gone. It's basically become a meaningless difference of what medium you run on but I've been using the same scripts to create the necessary root OS & UEFI EFI filesystem (FAT32 with a magic partition-type, populated with the bootloader & config). often I spin up systems by just copying a btrfs snapshot & rsync'ing efi directory (and updating some partition uuid's, regenerate machine-id), then I have two systems. Live feels meaningless, non-distinct from regular to me.
All these attempts to treat things different & distinctly, to make special cases for ourself- oh we need a network/router oriented OS, we need a bare OS, we need a consumer OS, we need a NAS os- so rarely have I ever felt like these attempts to flee from the fold & venture out to special really have rewards. This behavior of making exceptions for your problem, talking yourself out of doing the easy normal base thing, seems so rarely to me to have rewards. Run Debian, run systemd, run normal well-known tools! Unless you're really certain getting off the base path really is essential, has huge specific merit.
That's why I like this effort. It allows customization & baking your own thing, but it's much more routine & regular & normal than so many "make your own OS" projects, where folks tend to be quite "back to the land" in motive. Let's make our own OS from sticks! Yeah! But no. You almost certainly should have an ok userland base in my view. Debian is a pretty great one.
If anyone wants to build with/atop Debian Live, I did a couple projects years ago where I wrote wrappers that might still be good starting points:
* LilDeb, a USB multi-tool designed for 256MB (like when you couldn't find a Linux box, but you could find a Windows PC), with careful separation between read-only OS, ephemeral data, and persistent data. One shell script builds the entire thing, including the various config files. https://www.neilvandyke.org/lildeb/
* Rackout, start to a living room media appliance project that was going to provide some examples for my (abandoned) Racket book. If you happen to want to use Racket to do things like repartition a disk or build a distro image, some of the libraries I wrote might come in handy. https://www.neilvandyke.org/rackout/
How do you maintain this as new base releases are made?
Ie: Do you just manually reapply your changes every time? Or just do in-place upgrade and then snapshot that?
Personally, by the second time I did this, I'd be writing a script. Then I'd want to script the running of the script.. and before you know it you end up with a framework..
- overlays (basically you can separate configuration into separate units)
- templating support
- configuration in yaml files
- it still fully supports live-build support, so it should be super easy to migrate. You just put the live build files in os/lb or in the lb folder of an overlay.
It’s not funny if you have to explain it but it was mostly a joke, without rolling eyes. Also making fun a bit about how easy it is to make a claim that something is better than another thing and leave it at that, which is very common in every population but in particular something you hear from the arch one. I use Arch btw.
> Like offering users a GUI click-through installer?
I can't say if it's a joke or if you are serious.
But if you like an installer, there's manjaro.
Personally, I think install shouldn't need a click-through installer: it should be done by a commandline, to have a finer control over all the details.
With Ubuntu, I only used the installer iso to type Control-Shift-A to get a terminal, install rsync and do my tinkering.
Then can you tell me how to use the ubuntu GUI install to a ZFS encrypted root, without zsys, without grub but with a .efi packed kernel?
Or how can you mount the efi partition to new systemd location /efi instead of /boot/efi (ex: if you're not using grub) without breaking the install?
There are just too many choices for too many people. It's best not to play, and let it be a commandline problem. You rarely have to reinstall without another OS or boot media, so doing a chroot and copy-pasting the right commands (or rsync or untar a working master image) is just simpler.
I was obviously being sarcastic, but the sarcasm was earnest. For the most part, there shouldn't be any details to control. Just pick defaults that work for 99% users and let them get on with their life.
>But if you like an installer, there's manjaro.
Manjaro is not Arch. My comment was a criticism of Arch Linux, not Manjaro.
I think I'd rather have one featureful well-maintained tool backed by a community of people than a hundred of them with different features and single maintainers for each one.
Congrats and thanks. Not sure people interested in Debian want to contribute to the borg producing copilot and chatgpt based on their work on gh, among other things.
This project is a product of 4 years of work, mostly learning the Debian ecosystem. I started with Ubuntu, I’ve tried Arch, I’ve tried Gentoo, but nothing beats the ease of Debian when it comes to custom distributions. Debian was almost built to fork. Their ecosystem is amazing. Their live build tool is super.
I built Linux Factory so everyone else can build custom Debian distributions easily without having to know the live-build tool. It’s really easy to configure and get started. Please give me feedback on how I can make it better.
1. Having N tools that are 95% the same is just frustrating for everyone trying to find a tool that will be maintained.
2. Contributing to a project that almost does what you want leads to better tools that are also maintained so they can be relied upon to build new things.
3. Instead of building the Nth version of something, you could be working on something novel.
Theres no value from building something new if you can just learn how something that exists works and contribute to it instead. NIH syndrome is actively harmful to open source.
I hate this reddit style comment of linking a barely related thread, not to make any sort of point, but to just try to show you are in the know of some in joke.
Since I've been looking at mkosi for building Debian images recently, I would love a section which clarifies/differentiates features (and goals/non-goals) of linux factory and mkosi.
Linux Factory sounds like a fantastic tool that could benefit a lot of people who want to build their own custom Debian-based distributions. It's awesome that you've made it accessible even for those who aren't familiar with the live-build tool.
In terms of feedback, it might be helpful to include some tutorials or guides for those who are completely new to the process of building custom distributions. Additionally, having a user community where people can share their experiences and ask questions could be really valuable.
Oh, hey!!! Did you know that RHEL, CentOS Stream, & Fedora has been providing an image builder for a couple of years? All of these links below demonstrate functionality that you can do with CentOS Stream & Fedora, and even with the no-cost developer subscription for RHEL which unlocks 16 RHEL subs with tons of extra functionality. Many links below to help you easily learn to use it. We are actually using this tooling internally to beging building all of the image artifacts for all three of these linux distros. Eating our own dogfood; this is the future.
Also, below are links and some screenshots of the wicked cool new customizations that just landed in CentOS Stream and should come to RHEL 9.2 and Fedora soon.
Please give us feedback, feature requests, tell us how you're using it, pull requests at our upstream project https://github.com/osbuild
Happy building!
Linux-factory is a powerful and innovative open-source project that is making it easier than ever before for developers and Linux enthusiasts to create custom Debian-based operating systems. Whether you're a seasoned developer or a newcomer to the Linux world, Linux-factory is definitely worth checking out!Great work @clayrisser sir...!! Hope you will reach many more milestones in future sir..!!
Hey groestel..! Iam not promoting any product or service, but rather expressing appreciation for an open-source project.I genuinely believes in the value of the project and wants to show my support. And if you want to support you can..!! No offense, Thank you...!!
87 comments
[ 0.24 ms ] story [ 137 ms ] threadIt'd be great if I could easily customize Debian (i3, vim, zsh, chrome, etc.) and distribute / share it with others.
But I'm not sure what this does and cannot tell even after reading the first few sections of the README.
(To be clear: I really like Yocto)
I know people have frustrations with Nivida supplied Jetpack but when you end up trying to figure out how to roll your own support for TensorRT, DLA, cuDNN, etc (on ARM64, of course) many people bail and run screaming back to Jetpack.
Also it helped me to learn a bit how other distros are built: I tried Linux from scratch, then got interested in Alpine, and then in how PostmarketOS got started based on Alpine tooling.
After all, a distro is "just" a bunch of packages, which are archives containing files that get decompressed in the right place. Yocto provides tools to create those packages (just like mainstream Linux distro). What Yocto adds on top is tools to manage cross-compiling (toolchains, a way to create an SDK, etc) and a system of layering that helps organizing them ("those packages go specifically with the RPi, so I will put them in a layer called "RPi" and I will call that a "BSP"", etc).
Don't get me wrong, there is quite a lot. But there is no magic, the goal is always to create packages that will put the right files in the right place :-).
Good luck!
1. Start with an empty image and build your way up. There can be some cross-effects between packages depending on what you exactly do, it's a lot easier to debug if you go step by step. The base image recipes are all listed in the documentation, start with one of them.[1]
2. Lean into Bitbake and the whole thing. The more you try to fight it, the more it fights back. I liked this guide as a starting point to get a feel for it.[2]
3. Find a way to quickly iterate on your images. Nothing worse than losing your train of thought! Since I wanted to include update functionality I added swupdate, which integrates reasonably well with Yocto[3] - although the documentation is a bit difficult to read at times. But being able to simply push an update through WiFi is really, really nice.
4. If you do some lower-level stuff you'll inevitably need to create configuration fragments. This process was REALLY opaque to me. Say you want to set build options for U-Boot - you'll have to: 1) run `bitbake -c menuconfig u-boot` to open the config menu; 2) when closing the menu, you save the configuration in the default location; 3) then you run `bitbake -c diffconfig u-boot` to generate a config fragment; 4) you copy the config fragment into your recipe-append-folder and add it to the SRC_URI variable; 5) you run `bitbake -c clean u-boot`. When U-Boot is built the next time it will apply your configuration fragment to the build process.
These were the most important things that came to mind. Hope they help, and good luck on your journey! :)
[1] https://docs.yoctoproject.org/ref-manual/images.html
[2] https://a4z.gitlab.io/docs/BitBake/guide.html
[3] https://sbabic.github.io/swupdate/building-with-yocto.html
If are building your own board and developing a BSP, Yocto is a nightmare peeling away the onion layers.
I find Buildroot much saner and easier to work with.
I see buildroot for making a firmware, in the sense that I ship a bunch of tools that will run on a device (and I can flash updates).
Yocto is there to build a distro, where you have to manage packages, and potentially other people will be developing for it.
Of course if Yocto is overkill in your case, then buildroot will probably be a better fit.
A similar distro with remaster tools based on Devuan is Refracta.
There used to be the Debian Live project but it looks obsolete. EDIT: sorry I misread the bug reports, it is OK https://live-team.pages.debian.net/live-manual/html/live-man...
I used to run Debian Live, and appreciated the tiny amount of shell hacking they have to layer persistent storage filesystem atop a ro filesystem- identical to how theres a ro container image then the mutable rw layer atop it... but a decade older!
I find that the pretense of running "live" is largely gone. It's basically become a meaningless difference of what medium you run on but I've been using the same scripts to create the necessary root OS & UEFI EFI filesystem (FAT32 with a magic partition-type, populated with the bootloader & config). often I spin up systems by just copying a btrfs snapshot & rsync'ing efi directory (and updating some partition uuid's, regenerate machine-id), then I have two systems. Live feels meaningless, non-distinct from regular to me.
All these attempts to treat things different & distinctly, to make special cases for ourself- oh we need a network/router oriented OS, we need a bare OS, we need a consumer OS, we need a NAS os- so rarely have I ever felt like these attempts to flee from the fold & venture out to special really have rewards. This behavior of making exceptions for your problem, talking yourself out of doing the easy normal base thing, seems so rarely to me to have rewards. Run Debian, run systemd, run normal well-known tools! Unless you're really certain getting off the base path really is essential, has huge specific merit.
That's why I like this effort. It allows customization & baking your own thing, but it's much more routine & regular & normal than so many "make your own OS" projects, where folks tend to be quite "back to the land" in motive. Let's make our own OS from sticks! Yeah! But no. You almost certainly should have an ok userland base in my view. Debian is a pretty great one.
* LilDeb, a USB multi-tool designed for 256MB (like when you couldn't find a Linux box, but you could find a Windows PC), with careful separation between read-only OS, ephemeral data, and persistent data. One shell script builds the entire thing, including the various config files. https://www.neilvandyke.org/lildeb/
* Rackout, start to a living room media appliance project that was going to provide some examples for my (abandoned) Racket book. If you happen to want to use Racket to do things like repartition a disk or build a distro image, some of the libraries I wrote might come in handy. https://www.neilvandyke.org/rackout/
Ie: Do you just manually reapply your changes every time? Or just do in-place upgrade and then snapshot that?
Personally, by the second time I did this, I'd be writing a script. Then I'd want to script the running of the script.. and before you know it you end up with a framework..
This means Linux Factory is future proof.
- templating support
- configuration in yaml files
- it still fully supports live-build support, so it should be super easy to migrate. You just put the live build files in os/lb or in the lb folder of an overlay.
A couple of things it supports that live build does not support.
- overlays (basically configuration modules)
- templating
- python hooks
FTFY
IMHO arch is far better, and comes with an extra bonus: you can use pacman just like on MSYS2 (Windows)
Here's a simple example: you want hyprland, patheon and interception-caps2esc on ubuntu? Good luck!
On arch, it's just a pacman away.
You don't like how arch is perceived as elitist? Use manjaro.
Ubuntu had its place in the last century, but with most development happening on git you need a rolling release.
Poe law strikes again! My apologies then, because I known how Arch can have this reputation.
But Arch is just so good! Ubuntu and RedHat seem to be stuck in the past: it's not just about rolling releases, but many small details that pile up.
FYI I also find Alpine very interesting.
Like offering users a GUI click-through installer?
I can't say if it's a joke or if you are serious. But if you like an installer, there's manjaro.
Personally, I think install shouldn't need a click-through installer: it should be done by a commandline, to have a finer control over all the details.
With Ubuntu, I only used the installer iso to type Control-Shift-A to get a terminal, install rsync and do my tinkering.
Or how can you mount the efi partition to new systemd location /efi instead of /boot/efi (ex: if you're not using grub) without breaking the install?
There are just too many choices for too many people. It's best not to play, and let it be a commandline problem. You rarely have to reinstall without another OS or boot media, so doing a chroot and copy-pasting the right commands (or rsync or untar a working master image) is just simpler.
>But if you like an installer, there's manjaro.
Manjaro is not Arch. My comment was a criticism of Arch Linux, not Manjaro.
For example, my personal computer is running Debian Sid (i.e bleeding edge, dev track) while my computer for work is using the latest Debian stable.
https://wiki.debian.org/SystemBuildTools
The feeling you get from it is great! Plus, this is often how new things (or ways to do things) are discovered.
I built Linux Factory so everyone else can build custom Debian distributions easily without having to know the live-build tool. It’s really easy to configure and get started. Please give me feedback on how I can make it better.
Do you mean SUSE Studio? That functionality has been integrated into the openSUSE Build Service at https://build.opensuse.org/ .
sometimes the answer is just, why not? - it's not forbidden to learn something or try a different approach.
2. Contributing to a project that almost does what you want leads to better tools that are also maintained so they can be relied upon to build new things.
3. Instead of building the Nth version of something, you could be working on something novel.
Theres no value from building something new if you can just learn how something that exists works and contribute to it instead. NIH syndrome is actively harmful to open source.
In terms of feedback, it might be helpful to include some tutorials or guides for those who are completely new to the process of building custom distributions. Additionally, having a user community where people can share their experiences and ask questions could be really valuable.
Also, below are links and some screenshots of the wicked cool new customizations that just landed in CentOS Stream and should come to RHEL 9.2 and Fedora soon.
- Install in your local Fedora/CentOS Stream box
dnf install -y osbuild-composer composer-cli cockpit-composer bash-completion systemctl enable --now osbuild-composer.socket systemctl enable --now cockpit.socketfirewall-cmd --add-service=cockpit && firewall-cmd --add-service=cockpit --permanent source /etc/bash_completion.d/composer-cli
- RHEL Image Builder | Into the Terminal 53 youtube podcast https://www.youtube.com/watch?v=H-mv_WLmQdA - Blog: https://www.redhat.com/en/blog/using-no-cost-developer-subsc... - Blog: https://www.redhat.com/en/blog/announcing-full-support-new-r... - Upstream docs https://www.osbuild.org/guides/blueprint-reference/blueprint... - RHEL 9 docs https://access.redhat.com/documentation/en-us/red_hat_enterp...
Screenshots - https://github.com/tabowling/Fedora-Virt-Lab/blob/master/cs9... - https://github.com/tabowling/Fedora-Virt-Lab/blob/master/cs9...
Please give us feedback, feature requests, tell us how you're using it, pull requests at our upstream project https://github.com/osbuild Happy building!