I am not an OpenWrt developer, but I develop software for OpenWrt based platforms.
In short: on a lot of these devices it would be difficult or impossible to have a stable auto update mechanism. This could be due to flash size (16MB is considered large for consumer devices), or due to the state of the kernel modules responsible for hardware (e.g. wireless drivers).
Also there isn't usually a backup to the kernel in flash, so if your auto update is interrupted by a power failure or the user pulling the cord, you will have a bricked device. Good luck explaining the limitations of SPI EEPROM and compressed filesystems to a pissed off user with a brick. After the update you have to reboot, how do you schedule this on a device which is typically invisible to the user? In networks it's extremely difficult for these embedded devices to guess when someone might not be using it.
Having seen my fair share of ancient OpenWrt devices deployed in the field, the industry as a whole definitely needs to focus more on auto updates to resolve security issues.
However as the author points out, the sheer number of chipsets out there, it would be very difficult to accomplish without extensive testing.
I hope that we can all make meaningful progress toward auto updates of critical issues, but there is a long road ahead to that.
If you're willing to pay "decent money" for a router with auto-update, get some PC hardware (even as small as a WRAP or Soekris) and run pfSense. You'll never look back. I love OpenWRT but a $100 Netgear or Linksys device just can't touch a real PC motherboard with Intel NICs.
My preferred pfSense platform at both home and the office is a server-grade chassis running ESXi. My router runs in a VM on this server and I use vSwitches to connect the VM to the various ports on the 4-port Intel NIC.
I just built it like this for convenience (I already had the ESXi server) but I later found a side-benefit: if you've set up a DMZ on pfSense, you can easily attach VMs to the DMZ by associating them with the DMZ vSwitch. So, I have some untrusted VMs sitting in my DMZ and trusted VMs sitting on my LAN...all on the same piece of hardware.
I've been running this way since 2013 and it's been flawless and fast. More details here:
How much power does that use? I run linux a firewall/router on an older laptop + usb ethernet(2) and it's about 20w. I'd prefer < 10.
Perhaps I need to try Raspberry Pi next.
I have been running an IPcop firewall on an old PC for ages with three network cards. Just about to replace it with this small thing, also with IPcop, and it runs on a 4 watt power supply. I have realised that by downsizing my servers etc in the basement, now when they are a bit overkill (moved most stuff to a cloud solution) I can save enough electricity to power an electric car for most of the use I would make of it with what I save.
The Raspberry Pi is gonna suck as a router due to the "Ethernet on USB". You'd be much happier with one of the boards that chrissnell mentioned above.
Personally, I use a "Maxxwave 1106" running BSD I had laying around but it's on the more expensive side. If I was putting something together for myself, I'd get one of the small Atom boards with a couple of onboard Intel NICs.
I've been running OpenWRT on an PC Engines APU board (4 core x86 based w/ gigabit ethernet). According to the website[1] the power is "About 6 to 12W of 12V DC power depending on CPU load." Before my APU I ran an Alix board[2] for years.
It's a little bit more complicated to get running since it requires a custom compiled OpenWRT, but it's great to have more memory and disk space than I'll ever need in my router. I used to run on OpenWRT on reflashed consumer routers and I always had to be real careful about what I installed and ran. With the APU I can just install real `bash` and `less` and not have to deal with the crappy busybox emulations.
As mentioned, the ethernet interface is too slow for this. One could use one of the BananaPi variants for that, afaik there is one which has four independent ethernet ports for use as a router.
I use a standard BananaPi as NAS and are very satisfied with it. It comes with a SATA interface and 1Gb ethernet.
Why don't you just head over to the linked Indiegogo project (Turris) and grab a router that is open hardware, using open source (openwrt based) and has automatic updates?
I was just checking this out, but I've obviously missed the $180 backer opportunity (which is quite a bit more than the $75 all-in I just spent on my 128MB memory OpenWrt). Are you a backer? I wonder if the smartphone app they are building for the Turris will work with other OpenWrt installs; do you happen to know?
I'm a backer - but I don't know if that app would work with any OpenWrt based installation.
No idea what your current setup is, but I'd guess that this router is more powerful - and comes with the advantages we discussed here: The hardware is open (you can even get one with exposed connectors/a serial cable to mess with ~everything~), the software is open, you get a 'distribution' with updates by a NIC.
It's worth the money for me, but I understand if you're considering a generic OpenWrt based setup good enough. That's what I'm on right now, what works for me until this box arrives. And for me, an upgrade to 11ac was planned anyway, so this just came up when I was looking at new hardware anyway.
At the "discounted" $275 (2GB version + EU shipping), it's more expensive than embedded x86 solutions (APU2b4 or Celeron N) while being less powerful and flexible (the power consumption is the same).
Still a nice toy to play with, especially with all the connectors it offers and I can see this being a hit with iOS fans since it's supposed to be super easy to administer using their custom GUI and video tutorials, so worth the asking price.
I'm not an OpenWrt developer either, but I was thinking about this recently (how to handle autoupdates on OpenWrt). How about if OpenWrt was running on a hypervisor? It'd have to be small enough to fit in the EEPROM constraints, which probably rules out Xen, but KVM could work.
With a hypervisor in place, you could install an image alongside a working one, then do a warm reboot to start the new image. The user would be presented with a website to confirm that they'd like to keep this new image. If the user doesn't confirm this within a fixed time period, the router would roll back to the old image.
As for the space requirements for keeping two images on the router, I wonder if anything could be done there. For example, I wonder if the new image could be applied as a 'patch' on top of the old one, stored as file diffs rather than as a standalone image. Does that sound workable?
Your hypervisor scenario wouldn't work out, assuming auto-updates. What happens when the image is updated (automatically), the router reboots, and the user isn't aware of that (and, therefore, can't log into the web interface to "confirm" the update). It'd just get rolled back and reboot again.
On another note, if I had something with a hypervisor available, I'd run something other than OpenWRT on it. The primary use case of OpenWRT is these small embedded devices where such a hypervisor isn't available.
> " the user isn't aware of that (and, therefore, can't log into the web interface to "confirm" the update)."
The website would be presented automatically to the user, similar to how a login page is displayed when you connect to some public WiFi networks.
> "On another note, if I had something with a hypervisor available, I'd run something other than OpenWRT on it. The primary use case of OpenWRT is these small embedded devices where such a hypervisor isn't available."
Yes, it'd almost certainly help with broadening the range of options. Until I realised I was too big I was drawn to the idea of using Xen as I'd like to see OpenMirage running on home router hardware, it seems like a natural fit. A minimal QubesOS image could also be good.
OpenWrt founder here; you've pretty much hit the nail on the head.
Due to size constraints most platforms use a split filesystem consisting of a large read only root filesystem and a smaller overlay for changes. The read only filesystem allows for much better compression ratios but it also means if you attempt to replace an individual package you'll end up with two copies, original in the RO and new in the overlay.
Reflashing an OpenWrt device generally consists of migrating a minimal environment over to a ramdisk, rewriting the kernel and read only portions and then writing a backup of the config files to the overlay. On failure you're likely to be stuck in whatever rescue mode the bootloader provides.
> Reflashing an OpenWrt device generally consists of migrating a minimal environment over to a ramdisk, rewriting the kernel and read only portions and then writing a backup of the config files to the overlay.
Yup. Which is why auto updates will probably never work well.
What if the user is running some custom software on the device and it doesn't have enough RAM to store the new compressed filesystem? These devices don't have swap.
Routers are a subset of embedded systems, which means lots of different platforms, possibly with vendor included quirks, AND then users who run their own junk on them (ahem).
Given that most manufacturers are loath to even release their uboot source (come on guys, it's a bootloader not missile launch codes) and most of the implementations I've seen are a ham fisted modification of an old revision of uboot; just enough to boot, but dragons exist in the code.
Short of an iPhone for routers moment (yes,the irony of Apple's extremely closed system is not lost on me) where some company creates a router which just revolutionizes the whole industry[1], I don't see it happening on a consumer level unless there is a legislative requirement.
[1] this won't happen because phones are fun and shiny and routers aren't. At least to your average consumer.
Also the internet is awesome. An OpenWrt founder replied to my comment! :D
Is the solution then to use an x86 PC for home routing? I have been considering doing that. What's a good distro for this particular purpose? As it happens I bought a fanless i3 5005U from aliexpress a week ago mostly as a media player but also as a vague "I need something silent and server-ish" thing.
Small Atom boards with multiple Intel NICs are absolutely awesome for this. I prefer OpenBSD myself but if you're looking for something with a pointy-clicky web interface, slap pfSense on it and call it a day.
I am an OpenWrt developer but not part of the core team by far. We've been developing a platform to manage OpenWrt devices from the cloud for the last 18 months.
We struggled with this since day zero. As a startup, we didn't have the luxury money to build our own boxes with loads of memory. Plus we we wanted to support all OpenWrt devices. Ultimately, we took the logic of auto upgrades (and everything else for that matter, separate story) off the aps directly and now do everything in our platform.
Currently we only support a minimal set of devices - about 15 in total. However building / maintaining the firmwares for all of these, making sure they're up to date etc. is a huge burden.
Upgrading a box for a user poses so many challenges, I can't see why this is actually OpenWrt's responsibility?
What to do about testing? New releases can (do) have bugs. Again, since we're bootstrapped, we don't have a lab so we have separate development modes and test these on customers willing to have the bleeding edge.
The Netflix Syndrome as we call it. What if someone/thing is connected - do we still upgrade? I work late, and before sleep, I usually watch some Netflix. However, our nightly upgrades pushed from our platform were ruining my enjoyment! Shock. We had to introduce a feature to disable auto upgrades if there is a device detected.
What about stragglers? Customers with very outdated firmwares were not only vulnerable but also holding our development back. In the end we upgrade all boxes that are 2 months old between 4am and 5am local time every day. Some people refused to upgrade, they like the status quo, they're used to the gotchas in a particular release.
Ok, OpenWrt is becoming hugely popular but there's still a huge knowledge gap. If people struggle to understand the difference between WiFi and broadband, how can we expect them to understand firmware let alone, why it needs to be updated.
My ISP auto updates ours (against my wishes). I've had to hardware reset it several times as it introduces bugs constantly. Sometimes it just goes bust as well.
The worst part is that they don't seem to test before pushing these updates. Im back on 3.2.2RC.
This is interesting, but it would be more so if you could tell us what hardware, what "3.2.2RC" refers to, and which ISP (I'm assuming somewhere in Europe?).
Also, I thought it was assumed that letting your ISP control your router is a 'bad idea'; modem, sure, but not the router. Perhaps this just isn't important enough for you to care, but this entire thread is full of people that feel differently.
If openwrt had an updating feature and I was running it, do you want it to brick my device when they decide they can no longer support my router?
That is exactly what happened when I tried out openwrt a few years ago. Nobody thought it was wise to say that I would need more than the 4M of flash that is in my original "the linux router". I had to use tftp to put ddwrt back on it. That requires a wired connection. Thank god I have plenty of those. People today with their wireless everything would be SOL.
34 comments
[ 3.4 ms ] story [ 86.7 ms ] threadIn short: on a lot of these devices it would be difficult or impossible to have a stable auto update mechanism. This could be due to flash size (16MB is considered large for consumer devices), or due to the state of the kernel modules responsible for hardware (e.g. wireless drivers).
Also there isn't usually a backup to the kernel in flash, so if your auto update is interrupted by a power failure or the user pulling the cord, you will have a bricked device. Good luck explaining the limitations of SPI EEPROM and compressed filesystems to a pissed off user with a brick. After the update you have to reboot, how do you schedule this on a device which is typically invisible to the user? In networks it's extremely difficult for these embedded devices to guess when someone might not be using it.
Having seen my fair share of ancient OpenWrt devices deployed in the field, the industry as a whole definitely needs to focus more on auto updates to resolve security issues.
However as the author points out, the sheer number of chipsets out there, it would be very difficult to accomplish without extensive testing.
I hope that we can all make meaningful progress toward auto updates of critical issues, but there is a long road ahead to that.
My preferred pfSense platform at both home and the office is a server-grade chassis running ESXi. My router runs in a VM on this server and I use vSwitches to connect the VM to the various ports on the 4-port Intel NIC.
I just built it like this for convenience (I already had the ESXi server) but I later found a side-benefit: if you've set up a DMZ on pfSense, you can easily attach VMs to the DMZ by associating them with the DMZ vSwitch. So, I have some untrusted VMs sitting in my DMZ and trusted VMs sitting on my LAN...all on the same piece of hardware.
I've been running this way since 2013 and it's been flawless and fast. More details here:
http://output.chrissnell.com/post/39550480075/the-jack-of-al...
http://www.firewallhardware.it/en/alix_pfsense_embedded.html
Personally, I use a "Maxxwave 1106" running BSD I had laying around but it's on the more expensive side. If I was putting something together for myself, I'd get one of the small Atom boards with a couple of onboard Intel NICs.
It's a little bit more complicated to get running since it requires a custom compiled OpenWRT, but it's great to have more memory and disk space than I'll ever need in my router. I used to run on OpenWRT on reflashed consumer routers and I always had to be real careful about what I installed and ran. With the APU I can just install real `bash` and `less` and not have to deal with the crappy busybox emulations.
[1] http://pcengines.ch/apu.htm
[2] http://pcengines.ch/alix.htm
I use a standard BananaPi as NAS and are very satisfied with it. It comes with a SATA interface and 1Gb ethernet.
No idea what your current setup is, but I'd guess that this router is more powerful - and comes with the advantages we discussed here: The hardware is open (you can even get one with exposed connectors/a serial cable to mess with ~everything~), the software is open, you get a 'distribution' with updates by a NIC.
It's worth the money for me, but I understand if you're considering a generic OpenWrt based setup good enough. That's what I'm on right now, what works for me until this box arrives. And for me, an upgrade to 11ac was planned anyway, so this just came up when I was looking at new hardware anyway.
Still a nice toy to play with, especially with all the connectors it offers and I can see this being a hit with iOS fans since it's supposed to be super easy to administer using their custom GUI and video tutorials, so worth the asking price.
With a hypervisor in place, you could install an image alongside a working one, then do a warm reboot to start the new image. The user would be presented with a website to confirm that they'd like to keep this new image. If the user doesn't confirm this within a fixed time period, the router would roll back to the old image.
As for the space requirements for keeping two images on the router, I wonder if anything could be done there. For example, I wonder if the new image could be applied as a 'patch' on top of the old one, stored as file diffs rather than as a standalone image. Does that sound workable?
On another note, if I had something with a hypervisor available, I'd run something other than OpenWRT on it. The primary use case of OpenWRT is these small embedded devices where such a hypervisor isn't available.
The website would be presented automatically to the user, similar to how a login page is displayed when you connect to some public WiFi networks.
> "On another note, if I had something with a hypervisor available, I'd run something other than OpenWRT on it. The primary use case of OpenWRT is these small embedded devices where such a hypervisor isn't available."
Yes, it'd almost certainly help with broadening the range of options. Until I realised I was too big I was drawn to the idea of using Xen as I'd like to see OpenMirage running on home router hardware, it seems like a natural fit. A minimal QubesOS image could also be good.
Due to size constraints most platforms use a split filesystem consisting of a large read only root filesystem and a smaller overlay for changes. The read only filesystem allows for much better compression ratios but it also means if you attempt to replace an individual package you'll end up with two copies, original in the RO and new in the overlay.
Reflashing an OpenWrt device generally consists of migrating a minimal environment over to a ramdisk, rewriting the kernel and read only portions and then writing a backup of the config files to the overlay. On failure you're likely to be stuck in whatever rescue mode the bootloader provides.
Yup. Which is why auto updates will probably never work well.
What if the user is running some custom software on the device and it doesn't have enough RAM to store the new compressed filesystem? These devices don't have swap.
Routers are a subset of embedded systems, which means lots of different platforms, possibly with vendor included quirks, AND then users who run their own junk on them (ahem).
Given that most manufacturers are loath to even release their uboot source (come on guys, it's a bootloader not missile launch codes) and most of the implementations I've seen are a ham fisted modification of an old revision of uboot; just enough to boot, but dragons exist in the code.
Short of an iPhone for routers moment (yes,the irony of Apple's extremely closed system is not lost on me) where some company creates a router which just revolutionizes the whole industry[1], I don't see it happening on a consumer level unless there is a legislative requirement.
[1] this won't happen because phones are fun and shiny and routers aren't. At least to your average consumer.
Also the internet is awesome. An OpenWrt founder replied to my comment! :D
As mentioned above, check out pfSense and OPNsense.
Kong's build for ddwrt does have command line update
OSMC has done a really nice job of an auto update mechanism and it would be great to have the same functionality for router firmware.
There is also nslug2 and opkg...
Whoa, people still use these in the days of Raspberry Pi's? Oo
We struggled with this since day zero. As a startup, we didn't have the luxury money to build our own boxes with loads of memory. Plus we we wanted to support all OpenWrt devices. Ultimately, we took the logic of auto upgrades (and everything else for that matter, separate story) off the aps directly and now do everything in our platform.
Currently we only support a minimal set of devices - about 15 in total. However building / maintaining the firmwares for all of these, making sure they're up to date etc. is a huge burden.
Upgrading a box for a user poses so many challenges, I can't see why this is actually OpenWrt's responsibility?
What to do about testing? New releases can (do) have bugs. Again, since we're bootstrapped, we don't have a lab so we have separate development modes and test these on customers willing to have the bleeding edge.
The Netflix Syndrome as we call it. What if someone/thing is connected - do we still upgrade? I work late, and before sleep, I usually watch some Netflix. However, our nightly upgrades pushed from our platform were ruining my enjoyment! Shock. We had to introduce a feature to disable auto upgrades if there is a device detected.
What about stragglers? Customers with very outdated firmwares were not only vulnerable but also holding our development back. In the end we upgrade all boxes that are 2 months old between 4am and 5am local time every day. Some people refused to upgrade, they like the status quo, they're used to the gotchas in a particular release.
Ok, OpenWrt is becoming hugely popular but there's still a huge knowledge gap. If people struggle to understand the difference between WiFi and broadband, how can we expect them to understand firmware let alone, why it needs to be updated.
The worst part is that they don't seem to test before pushing these updates. Im back on 3.2.2RC.
Also, I thought it was assumed that letting your ISP control your router is a 'bad idea'; modem, sure, but not the router. Perhaps this just isn't important enough for you to care, but this entire thread is full of people that feel differently.
This is what I am running EG300-WU21U_OWT3.2.22-151006_1056
That is exactly what happened when I tried out openwrt a few years ago. Nobody thought it was wise to say that I would need more than the 4M of flash that is in my original "the linux router". I had to use tftp to put ddwrt back on it. That requires a wired connection. Thank god I have plenty of those. People today with their wireless everything would be SOL.