I had this exact issue today. Starting in the wee hours of the morning, the router would just crash every 30-40 minutes. I had to reboot it to get the router back. I updated firmware and it seemed to fix it..
This happened to me too! After a day of dealing with the router crashing every 30 minutes, I finally updated the firmware and am hoping the issue is fixed now.
Hmm maybe not but reading all the linked forum posts and threads, people are at a complete lost for information.
"I installed version X.Y.Z and so far so good"
"I tried all available updates and it keeps crashing".
There is no transparency in the published updates. Take that firmware blob (which is probably 99% GPL licensed by the way) and shut up. That's basically what the vendors are saying.
What's ironic is that most OEM are using some sort of out of date openwrt derivatives as a base for their closed source firmwares, which are obvious GPL violations.
> What's ironic is that most OEM are using some sort of out of date openwrt derivatives as a base for their closed source firmwares, which are obvious GPL violations.
Do you have a source of that or am I out of date? Based on my knowledge, outdated, GPL-violation OpenWRT would still be a massive upgrade over what's usually developed in-house by OEMs.
Linksys WRT 1200/1900/3200/32X touted their OpenWRT based firmware as a feature (though those routers are about a decade old now, and their Wi-Fi chips have unfixed bugs due to Marvell selling their Wi-Fi division to NXP)
I use a WRT1200AC running openWRT as my router, wifi disabled, and it works pretty well. I use two WRT1900AC running openWRT to make a WDS bridge and can get about 400-500Mbps of iperf3 throughput, and one of the two serves as an access point. Even though the wifi does indeed have its limitations in the form of missing features, it's been reliable and performant for me.
Right, but they're mostly bugs in optional features like 802.11w and DFS. In my mind, there isn't really much of a practical distinction between a missing feature and a feature that doesn't work. The interoperability bugs are perhaps the most nefarious, but don't really impact most people. For my home wifi setup, they've been pretty decent. Their range and throughput are good. I've reliably streamed games over wifi with them. Their CPUs are also relatively powerful, making them good routers. They can be had on eBay for $30-$40.
Buying $30-$40 access points on eBay is a bit of a hobby of mine. I've been using a Netgear R8000 as an access point for a few months. There's no OpenWRT support, but it's supported by FreshTomato. Qualitatively, it seems to work well in terms of latency and throughput when the signal is strong. It seems to perform worse than the WRT1200AC that I was previously using at that location, when at the fringe. When I'm at the center of my house, my phone tends to associate with it but then struggle to get packets through. Maybe I just need to dial down the AP's TX power, though.
Many home routers/wifi access point/mesh wifi systems are qualcomm/atheros based.
These system are almost always built on the official SDK provided by Qualcomm, known as QSDK.
QSDK is a fork of an old OpenWRT version (chaos calmer, 2018). Unlike the open source version, it includes proprietary drivers and many changes. Openwrt devs refer to it as a "frankenfirmware".
> It seems that the appropriate place to deal with network traffic security issues is with the network bandwidth provider, which would be your ISP.
This would imply that the ISP has abilities to interpose between malicious traffic and the user that, with modern technologies like DNS-over-HTTP, they simply don’t have. To most ISPs (that don’t force you to install MITM CA certs), user traffic is increasingly a black box. Which means that attacker traffic can blend right in as part of said black box. The only thing that can see into the black box is the device on the end of the line.
In most cases the device is getting it's IP information on boot by negotiation or request from the ISP. All the ISP needs is a database of blacklisted equipment and some middleware box to receive connections from insecure home devices when it detects them.
It's not about catching related traffic when it occurs, more managing the immediate implications from insecure devices on the network. The ISP can see the device, it doesn't need to inspect the traffic.
I'm convinced there are now multiple generations of engineers that have no idea that it is possible to write software that doesn't require over the air updates nor telemetry.
Have we all forgotten how Nest thermostats turned off the heat in the middle of January 2016 because of a botched software update? (Probably. It was hard to get Google to surface a link without a date due to so much SEO bullshit).
It’s not necessarily the engineers. Product and Leadership needs the telemetry in order to justify the expenses for their department or team or whatever and therefore justify the existence of those engineers.
Note that this doesn’t always work with smart thermostats. From first hand experience, there are some models out there that will generally work fine but will not complete booting up if one of the outputs is detected as closed.
Maybe, I guess, but it’d be pretty weird to be an engineer of any kind and not know about local file systems. More likely, both companies and most users want remote updates, and companies certainly love as much telemetry as they can get.
At least in this case, ASUS routers all have local firmware install functionality, so this particular case is certainly not an example of engineers not knowing it’s possible to write offline updates.
Now if you’re really lamenting the lack of computing devices that cannot be updated remotely by design, that’s a different story, and might be on a ship that sailed a while back. The problem there is that local updates are inconvenient enough for most people that they aren’t done, which is problematic from support and security perspectives, even though there are legit problems with remote updates too…
That’s a totally reasonable point of view, would be nice. But, the days of fixed-function hardware are almost completely gone. These routers (and lots of electronics) are just Linux computers stuffed in small boxes, and come with a lot of the same kinds of dependencies as your desktop. As such, when a security vulnerability is discovered, for example, it’s kind-of a good idea to be able to accept a software update, and when people are discovering them in every corner of the OS and utilities and daemons they run, it suddenly makes sense from a completely practical perspective to update automatically on a regular basis, which is what most people need for their computers & phones. They all should accept and respect the ability to disable auto-updates though. Maybe this case is an exploit or something.
Security updates is the important part. Particularly when dealing with vulnerable equipment which can be compromised and botnet malware dropped on it. A 0 day in a popular DVR/NVR or home router can lead to tens of thousands of devices that can throw a lot of heat. ISPs have been not great in this space so it's left to a small community to chase down the manufacturers to push updates out. The tragic part is for some devices - the company has gone out of business.
The scary part of auto update is when a company does a bad job of it. For example: letting the auto update site domain expire or point to an IP at a hosting provider that someone might pick up and if the devices don't do proper endpoint validation folks can use it to force downloads of compromised images.
"Do all routers these days "phone home" anyway and modify settings on autopilot?"
Depends.
"Homemade" routers running open source OS that computer owner can compile themselves need not "phone home" against owner intent. "Home" is the computer owner, not some company.
Commercial routers running closed source OS that owner cannot edit and re-compile can be expected to try to phone home for something, IMHO. A disturbing trend certainly not started by ASUS but which seems to be infecting most hardware sold with pre-installed closed source OS.
> A disturbing trend
I think it's inevitable and probably not completely bad that hardware vendors are taking responsibility for vulnerabilities in their products.
I know I would prefer something like that for my non-savvy people. Considering there are smart light bulbs with vulnerabilities which are years away from the landfill. The only question is which botnet will they join the next time they power cycle.
Unfortunately it's becoming increasingly expensive to find routers for casual home usage that allow for alternative supported firmwares. A decade ago slapping tomato on something cheap would do the trick, but now with high speed internet and wifi 6 there aren't many devices having that flexibility within entry-level price ranges (at least from what I've seen in Europe and Canada).
[0] would seem to suggest there are 45 options for things with WiFi 6 that you can install OpenWRT on. 16 of them are supported in an actual release and don't require running a snapshot.
Not amazing, to be sure, and I don't know what those devices cost, but I think the situation is less dire than your post would suggest.
With the price of high end routers rising the last time I needed one (that could get near to 1gbit) I went with a mini board that was well supported by OpenWRT. Some of these can also support WiFi via laptop style PCIe add-in cards.
If you plan on using third party firmware you should look at compatibility with OpenWRT BEFORE buying the device. Router manufacturers are unwilling and unable to provide a future-proof software.
Latest proof of that in my case was with a new Wi-Fi 6 router Netgear WAX206. After indeterminate time clients would just stop receiving packets and would require reconnecting. WAX206 latest factory firmware is based on OpenWRT from 2016 (Netgear still refuses to open source parts of it). After flashing current OpenWRT the connection is rock solid.
It isn't that easy. Both ddwrt and openwrt are very optimistic about calling something 'working'. Fully working might mean: Only slow wifi and 100 mb ethernet, No use of most hardware acceleration.
These projects are great, but against the background of 100s of new crappy devices every year, nothing can ever be decently finished. Hardware is simply not sold anymore before any device fully matures, and vendors are very stingy with firmware.
Brainslayer is so obsessively paranoid about the idea someone might "steal" and "sell" DD-WRT devices that he obfuscates building the firmware as much as possible.
The devices also are just the stock garbage vendor GPL tarball with DD-WRT's UI on top. So you'll still be stuck with stuff like an ancient kernel full of bugs
Thanks, I wondered why my router kernel went from 4.xx to 6.1 after flashing OpenWRT, especially when MediaTek is notorious for poor kernel support. This actually makes me respect OpenWRT even more.
MediaTek is pretty decent in terms of open source support for their WiFi chips and SoCs, to the point of continuously supporting a long time OpenWRT developer (Felix Fietkau) to work on their OpenWRT device drivers. This is an effort that is independent of their closed source drivers for the same devices.
Doesn't mean everything MTK does shines under the sun, but in Linux WiFi, they are probably the most open and well supported in the modern age.
Brainslayer builds kernels for DD-WRT broadcom devices (I don't know about other SoC manufacturers).
My AC68U running DD-WRT is using the 4.4 kernel. Stock firmware is using 2.6 kernel.
The current big issue with DD-WRT broadcom devices is that Brainslayer has not got access to build Kernels for the newer SDKs that support AX routers. A quick glance suggests AX support is lacking on other SoC manufacturers that DD-WRT support.
I plan on moving to OpenWRT compatible hardware for my next device.
This seems to be related to “ASUS Healing System” which I don’t even know if I have enabled or not.
That name already sounds creepy enough, but searching for that string (with the quotes) currently returns only 4 results, of people asking what it is. My guess is some sort of hidden backdoor, disguised as an ostensibly useful feature.
I assume that ASUS routers are based on Linux, so shouldn't the source for these routers be readily available? I am able to find custom third party mods (asuswrt-merlin) but I can't actually find a clear copy of the original sources!
This is very common when I look to find source for embedded devices like this. What I expect is the next step is that you will find (or be given) a borderline useless blob of source that doesn't explain any of it's build process, which is absurd because the GPL clearly defines the build "glue" as part of the source.
Is ASUS another company that is doing a poor job of GPL compliance in this space?
> AsusWRT is a derivative of Tomato which is itself one of the descendants of HyperWRT, a Linux distribution for low-end network appliances such as routers.
I Just looked at it, I remember that the original source was available from the support page(don't remember if you had to have the product registered though), but I can no longer find it. Maybe they switched to a manual request process?
Userspace programs do not have to be GPL to comply with a GPL-licensed kernel or even with other GPL-licensed userspace programs that they interact with.
I'm going to take a wild guess that the "ASUS Healing System" periodically checks system health and reboots a Daemon or the whole system if stuff breaks.
That seems to be the way to keep consumer grade routers from requiring the user walk over and reboot them once a week...
Domestic routers, in particular, are infamous for their web interface and management daemons often calling command line programs through system() to configure the network (and other system management tasks), instead of directly using the APIs these command line programs use. Not only is this inefficient and fragile, it also not rarely leads to being vulnerable to shell injection attacks (if you're lucky, only exploitable by authenticated users of the web interface).
A lot of times its software errors caused by powerline ingress from poor electrical isolation at the plug. If the wrong bits get flipped from EM interactions, only quick way to fix it is to reboot and reload.
They’re underpowered devices with almost no RAM, being asked to push traffic for ever faster internet connections. It’s a miracle consumer grade routers as well as they do.
The configuration space of any complex system diverges exponentially with respect to time. This is as true for computers as it is for you. Biologically, you start life with one set of genes and end life as a chimera, full of mutations.
In this particular case, the set of states that the device is in when it boots is relatively small compared to the set of states that device may be in a week later. More generally, all individual complex systems need to be "restarted" periodically, it's just a question of how often.
A lot of routers have this healing system built in. I had a netgear at close to the end of life which was two years After I bought it (I think), would reboot every hour. It was ok for the most part until my uncle came from elsewhere and was working remotely on a video call. It drove him bonkers.
I have a set of ASUS mesh routers and have been experiencing this issue today. Glad I see it's not my specific problem although it looks like I have to watch for firmware updates.
This happened to me this morning. I even called Cox, since they were out just last week doing scheduled maintenance and the internet went down a few times. Cox said the line looks fine (and tried to sell me their cable modem).
I logged into my router and checked for new firmware, then installed it, and it started working again.
Oddly, I'm also running Merlin but I did have a problem where my laptop thought it was connected to the AP but couldn't get local network traffic routed to it. Easiest solution turned out to be to just reconnect, so I don't actually know the deeper problem.
I don't think it has any relation, but since it's the first time it's happened, it was kind of a freaky coincidence!
Never put a router without open source firmware on the open Internet. It's that simple. The right tool for the job. (OTOH I advice against using Linux as your daily driver on any modern laptop. Once again, the right tool for the job and that's Windows + WSL.)
Hard pass. Not interested in Microsoft's latest malware/adware of an OS.
I've been running Linux on laptops for nearly 20 years now. Sure, it was a pain in the ass in the earlier days, but nowadays things just work, as long as you're willing to do your research ahead of time to choose a laptop where the hardware is fully supported.
You're biased. Win 11 doesn't have third-party ads of any kind, unlike iOS's app store, for example. It was always news, videos etc. Recommended content.
Moreover, ubuntu famously had an Amazon button on the home page, yet no is going around touting "Linux is riddled with adware".
I looks like Ubuntu was created just in order to be able to dismiss Linux as "riddled with adware". It's just a single distribution out of a hundred, and far from the best, so it's completely wrong of course.
A wild strawman appears! But seriously, it isn't 3rd party ads that GP was complaining about.
That said, if the OS is aggressively pushing recommendations for their first party news, videos, and search engine, and every one of those things shows 3rd party ads, I don't think that single step removed absolves win 11.
If you want to argue that preinstalled links to 3rd party websites, prominently displayed in the Start menu, do not count as "ads", then you are acting in bad faith and your arguments can be dismissed.
A bundled link to external properties is an advertisement.
If you want high performance from your Linux code, and to use large amounts of memory, Linux on the metal is much better. Thinkpads have great compatibility in my experience.
I can't recommend the tp-link AX50 highly enough. It has been rock solid and stable now for probably around 4 years. I don't remember ever having to reset it.
I recommended it to my friends when they changed their routers over, and it fixed a lot of issues for them as well. The ones who regularly used VR reported that the stuttering issues they had on VR over WiFi with their Netgear WiFi 6 routers were solved.
Have owned both ASUS and Netgear in the past and the experience was terrible -- the high-end ASUS router regularly bricked itself during auto update. So I bought a more expensive ASUS router for around $600, and you guessed it, it did the same thing just less frequently. The NETGEAR routers never bricked themselves, but across the 3 models we've owned nearly all of them would lose most of their throughput (around 90%) on a daily basis and would need a daily reset to get proper ISP speeds again.
I had a Dream Machine Pro for around a year before I moved it to my office to run the network in the office. Funnily enough it has probably been less stable than the tp-link router, but nonetheless the UDM Pro is still a great piece of kit and probably the last router you'd ever need if it's in your budget and you have a place to put it (awkward form factor for home).
Re: Open WRT I ran this around 3 years ago on a D-Link router that I had, and the stability was unpredictable.. better than ASUS and NETGEAR but probably leaning on the side of poor compared to tp-link and UDM Pro. It would randomly get slow once per fortnight or so, and would need to be rebooted. Sometimes it would panic and completely lose internet connection. There are plugins designed to reboot the router as soon as a loss of connection is detected, but why is this even necessary? It's annoying too. Drops all your connections.
I followed one of the solutions in the comments and removed /jffs/asd/chknvram20230516 and the issue seems to have gone away. CPU usage went from 70-90% back down to 5%.
I have one of those small Intel based nuc like devices and run x86 openwrt on that. It works great on a gigabit ipv6/ipv4 line. I also replaced the proprietary bios with coreboot for complete security.
I would just pick one up available from this list that matches your manufacturer preference(brand loyalty), desired feature set, and availability in the electronic stores you usually make your purchases.
Depends on your needs of course. If you want something cheap that can handle moderate usage in a house, I had pretty good luck with https://openwrt.org/toh/linksys/e8450 recently (the Belkin one was much cheaper than the Linksys brand one, for whatever reason).
Is there a comparable replacement company yet? The APU2 is nearly end of life, and while my existing PC Engines setup is trouble free, I intend to live longer than it does.
As far as I could tell, they were the only source of medium-performance boards with excellent open source support that didn't change the design every 12 months for no f---ing reason.
I think NanoPi devices are neat little things. R4s has openwrt support, the newer r5s/r6s are still in progress but I'd expect them to get supported sooner or later. You can probably already find patches floating around for them.
I had issues too. Thought it was just my home network. After a few reboots of every device on ai mesh, was able to install the latest firmware. It mentioned it fixed a memory leak. I have 2 RT-AX55's.
He had auto updates turned off, so there was no "allowing" happening. It's not even clear yet what happened, but one possibility is that this was someone taking advantage of an exploit on routers that had NOT been automatically updated.
Oh, that explains why my connection was slow from some rooms in the house today.
I also have an ASUS RT-AC86U (running 3.0.0.4.386_51255), but as a node in the mesh, it's still online.
However, my RT-AX55 (3.0.0.4.386_50460) in the mesh has been down all day and I haven't noticed before this post because the other nodes picked up the slack (with spotty/slow signal).
Also have an RT-AX82U (3.0.0.4.388_22525) as the mesh leader and it also doesn't seem impacted.
The power of blogging and HN. Asus or my ISP didn't tell me why my router/internet went out twice today. I honestly thought thieves stole my copper again:
I love when thieves are trying to steal copper, but all they get is a broken fiber optic ;)
My dad works as a network engineer, and he told me a story that one of the banks in Poland lost one of the internet providers. They investigated and found out that thieves stole hundreds of meters of a fiber cable, because they thought that it's copper.
Unbelievable. I struggled with this the whole day. Checked the ISP status. Thought, it could be overheating (router is in the garage) or bad RJ-45. Had to reboot multiple times today. Never ever I would have guessed it was a firmware issue.
Thank you for writing this blogpost. I just upgraded firmware. So happy when something has an explanation. !
The garage could be the coldest area in the building, or the hottest. Things happen, too...dust clogging ventilation, animals build a nest around it, router placed near refrigerator coils, etc.
I appreciate that you must live in a cooler climate. This is not the case everywhere. My garage cooks on a bad day. Great for the hot water heater, not so great for everything else.
I'm actually not that worried about them secretly doing something malicious (although that's also a valid concern) but rather in the "given enough eyeballs, all bugs are shallow" and the general risk factors (MikroTik going bankrupt and we're left with obsolete bug-ridden hardware).
Sadly given the large amount of software that's closed source there I don't see them working towards open sourcing any of it as, even technically, it would be a massive effort.
Agreed, even if it was not their intent it being closed source is malicious itself - we need to promote this discourse in public conciousness so people are less afraid to question the unaudited, outsourced binary blobs that fill their day to day lives.
As usual those exploits require the management interface to be open to the internet. Which you should never do to begin with on any router, and it isn't setup that way by default. Mikrotik is fine to have on WAN if you don't purposely make it insecure. Even Cisco has had similar vulnerabilities when you configure them in such a wrong way.
Yes, but my point is that there is nothing unique to Mikrotik here that makes using them on WAN a bad idea. Configure an Ubiquiti, Cisco, Aruba, Palo Alto, PfSense, etc in the same way and you're inviting trouble.
I run opnsense, and it's great, but I've learned to wait a few days before installing new updates. Almost always they're followed in a day or two by a rapid hotfix that fixes an embarrassing breakage.
I switched from a `show configuration commands` router to a `sho run` router and personally can't go back to anything without at least the former. To my knowledge, OpenWRT, OPNsense, Mikrotik RouterOS, as well as basically every plastic shell routers don't have it, but it just make way too much sense compared to those.
I run an RT-AC87U (3.0.0.4.382_52545-ga0245cc) as the main router
I have an RT-AC86U (3.0.0.4.386_51255-g486ded6) and an RT-AC68U (3.0.0.4.386_51255-g486ded6) running in AP mode.
Tried checking whether there was an update available for the RT-AC87U, but it looks like the ASUS firmware site is unreachable (overloaded?) for me at the moment.
I don't have an ASUS router, but three things leap out at me: a string being logged over and over, running out of space on a filesystem, and rotated log files named something.1.
It is trivially easy to blow right past the size capping on systems that use the old "newsyslog" style of external logfile rotation from the 20th century, and something that is logging a short string "[chknvram_action] Invalid string" over and over very fast is exactly how to do this.
For those interested in investigation, therefore, I would suggest looking at logfile sizes, and seeing whether it was logs eating all of the free space on /jffs and /var .
The underlying cause would be whatever is logging "[chknvram_action] Invalid string" thousands of times over, but the mechanism would be log files filling the tmpfs that the article mentions, which would explain why the system had no memory for forking new processes.
My wild speculation about "[chknvram_action] Invalid string" is that something somewhere in whatever "chknvram" is, the name being suggestive of something checking non-volatile RAM, has either bad data or a broken parser, and the recovery semantics are to retry immediately, incessantly, as fast as possible.
Does it store timestamps for those repeated occurrences? I wouldn't want my logs to "helpfully" coalesce multiple identical messages into a single one. For example, if I saw the equivalent of the text below in a program/OS while facing some issues, I'd be remarkably pissed.
[2023-05-18 07:12:21] [E] Invalid event 0x1AF2 received from ...
[2023-05-18 09:44:01] [I] Last message repeated 10 times
I care less about how many times the message was repeated - I care about timestamps, which I might want to correlate to other activities.
No, it's just a plain syslogd dating back to the 4.2 days. It aggregates at 30, 120, and 600 second intervals according to the source. Within that threshold I wouldn't care too much. If I really needed timestamps with more than thirty second precision I probably wouldn't be using syslogd.
In any new-ish production system I'd probably want to use anything other than syslogd anyways.
That's... good to know. I never realized anyone is doing something like this, ever. It breaks my trust in software logs in general - I'll be sure from now on to understand how any given program handles logging, before making assumptions relevant to troubleshooting.
In many cases, logs are asynchronous so depending on many factors among which are utilization of the host, you might get them with a delay and the ordering of events might not make sense because of that when read from the logs. If you need that precision you can surely engineer/ configure your system for that.
If I'm running my own distributed system for some business reasons, sure.
If I'm dealing with equipment failures, bugs in third-party software, or other such random tech bullshit, as an individual or a team, then I don't know in advance when and what precision I'll need.
In a reasonable system, you might be able to change some function in one place, perhaps even in the running system and get the precision or detail you need. You might take out the big guns like e.g. dynamic tracing using BPF to attach probes at the right places.
Most of the time, I'd assume all of them I can store (deduplication is fine, if I can recreate the raw data afterwards). Which is a lot, because they should compress well (in the limit, approaching the same size as deduplication solution).
Sometimes those data points don't matter - like if they're generated by some program stuck in an infinite loop. But in other cases, they do - like e.g. if each message is caused by some event, like another program doing some processing, or user pressing a key, etc. - then timestamps will be useful to identify the exact cause (e.g. logs only happen when process X is processing mouse input, or when user presses one of 20 specific keys on their keyboard, or only when my microwave oven is running).
-c Disable the compression of repeated instances of the same line
into a single line of the form "last message repeated N times"
when the output is a pipe to another program. If specified
twice, disable this compression in all cases.
So some somewhat more informed speculation is that the new signature file either yesterday or today either broke a parser or was itself corrupt. The error-handling path for this is still poor.
In the comments of the article someone mentions that deleting the file solved the issue without a firmware update. Too bad they didn't save it before, a comparison with the newer working version would be nice.
I guess Asus quickly discovered their mistake and removed the faulty file from their servers, but affected devices never got to the point where they'd look for a newer file but just choked on the local one.
Why does a router need malware signature files? It has no business monitoring my traffic, except in accordance with the firewall rules that I set myself.
From the article:
> not keeping my firmware up to date
I've had this (non-Asus) router for three years. I've never updated the firmware.
I've never updated the firmware on any of my routers, and I'll be happy to give you a million dollars if you can prove that it's part of a botnet. This pathethic holier than thou attitude is obnoxious, you really think 95% of routers are part of a botnet? Or that they're even accessible from the internet?
> you really think 95% of routers are part of a botnet? Or that they're even accessible from the internet?
It's probably not 95%, but it's certainly significant. And yes, most consumer routers are obviously accessible from the internet, that's how they work (they're generally not behind a firewall, they are a firewall, and can have vulnerabilities). You might want to read up on it [1, key quotes below], and I know I certainly wouldn't be willing to wager my consumer router wasn't infected, because how would I ever have any idea?
> Out of the box, most routers suck when it comes to security. Vulnerable firmware is an easy target. Backdoors have been discovered in just about every brand of router... It's possible you will never even know your device is infected...
> When we're talking about routers, these aren't the enterprise-level gear made by the likes of Cisco, Sonicwall, or Palo Alto... We're talking about what you would get off the shelf at Wal-Mart or Best Buy — the D-Links, Netgears, and Linksyses of the world.
> The problems start with the cheap, antiquated MIPS architecture used by a lot of these devices' processors... One substantial problem comes from a security flaw that goes back to 2001. Per one deep yet fascinating paper, the chips lack basic defensive abilities against malicious code execution. Combine this with manufactures commonly implementing out-of-date, vulnerable Linux kernels in their devices, and then putting this device on the edge of your network completely exposed to the internet.
But to sum up -- not updating your router's firmware is a terrible idea. It's not something to be proud of.
If you don't expose any external services of the router then it's a bit hard for botnets to take advantage of your router.
Most of these routers are likely becoming part of the botnet for enabling external web management and/or using default creds (especially if SSH is exposed externally).
Security isn't exceptionally hard, if you actually put some effort into it.
There have been plenty of router exploits that begin with a pivot from a web browser to a poorly secured admin panel on the local network. Firewalling incoming traffic from the Internet to the router's management interface is no security panacea.
Not sure you'll see this, with HN's lack of notifications, and so much time elapsed but here's the answer:
Most consumer routers do not support disabling the web interface on the local network, as it's the primary (only?) means of administration for them. This attack relies on getting users to browse to an address with default creds by some means, with a URL prepended that will cause the desired action to occur. More often than not a popular action is to modify the DNS servers used, so that DNS traffic can then me manipulated to point to malicious servers used for the ultimate attack.
Don't forget the biggest botnet being ASUS who didn't disclose the scanning... or whatever else they happen to be doing without the explicit consent of the owner.
Issues like this don't happen across such a wide area by accident.
Basic professional practices have any updates tested beforehand on physical hardware at multiple stages before any push happens, and they leave it up to the user because pushing an update without the owner's explicit consent to a device they own, runs afoul of the same hacking abuse laws. The legal exposure is massive.
It also doesn't account for the fact that firmware was attempted to be updated on devices which were set to not update. ASUS has a lot to explain. They didn't release any kind of statement so the lawyers and computer forensics people will likely need to get involved to get to the bottom of it.
I look forward to eventually hearing about what really happened.
Don't think so. The management interfaces are only exposed on the local network. If you could hack into my local network some other way (e.g. through Apache), you could probably attack the web interface on the router; but in fact no local network services are exposed to the internet.
It's a device that's directly exposed on the public Internet. Anyone in the world can send any package they want to it. It's not that uncommon for people to find bad bugs in network protocol stacks. An attacker wouldn't have to go through a web interface.
There are a multitude of ways to trick consumer routers into letting an external request log into the web interface. That's also ignoring the multitude of ways to botnet them without using the web portal, as manufacturers suck at their job and often ship known exploited libraries and utilities. Some have had genuine back doors even.
Botnet operators are careful to not mess with my internet connection. If they want to use my router for other stuff while it’s not in use by me, who am I to complain.
Hopefully that clears up the first question of "why does it need malware signature files?"
As for your router firmware? You should seriously update that. New exploits get found all the time.
Hackers use compromised routers as parts of a botnet, a intermediary route, or as an access point with which to steal data with Man in the Middle attacks.
> Hopefully that clears up the first question of "why does it need malware signature files?"
The malware signature files really don't help prevent your router joining a botnet.
Firmware updates, maybe, maybe not. It is quite possible for other routers with less generally sloppy and advertised-feature-rich firmware to actually be more secure even without updates for 3 years. It's quite possible that they have no api endpoints available for super-easy mobile app integration remote management etc, just ssh from local subnet or physical serial console.
There have been multiple cases of market-leading antivirus engines (symanted, mcaffee, etc) having sloppy code running with the highest possible privilege, parsing any files appearing on the system anywhere, and e.g. crashing a mail server that would otherwise be unaffected by the PoC samples being emailed through it by researchers.
So, I also take some issue with people who have no understanding of how all this software around us is designed and built (in routers, in windows, on web servers) and thinking that just updating everything all the time and running antivirus is the best you can do. You really can do a lot better if you know what you're doing.
There can be driver specific remotely exploitable issues that might not be widely communicated. Until operating systems are written more robustly, just having admin level stuff set up robustly isn't always enough. Of course, updates can add bugs too.
When I hear “signature file” I think of a list of signatures of known viruses and malware.
These types of signature files aren’t meant to guard against exploits, SSH brute forcing, etc, even if the router applies them to inbound traffic in addition to forwarded traffic. To do that, you typically need a WAF or some clever fail2ban-like filtering rules. Even up-to-date signatures won’t prevent a router from getting 0wn3d if the ssh daemon has a security hole for example.
As sites move to HTTPS, routers can’t even really filter networking traffic anymore. I don’t see why a router needs signature lists at all
> Hopefully that clears up the first question of "why does it need malware signature files?"
Not really. The article doesn't mention signature files. It describes the operations of a certain malware once it has been executed on a router. But first it has to get onto the router; and the only way new software can get onto this router is via a firmware update.
>the only way new software can get onto this router is via a firmware update.
That has never been the case. Software from consumer routers is often still in the "a trivial buffer overflow lets a malformed packet insert a payload into ram and convince the PC to jump to it." phase of software security. They are very much still wormable systems, like Windows 2000 style. Ever have your router glitch out and stop working and you have to reboot it? That's probably an exploitable bug.
An extra fun part is that about 2/3rds of the way through in pops someone else pointing out that the closed source ASUS proprietary malware scanner decided that all of xyr /usr/lib/crt*.o files on a mounted disc volume were malwares and promptly deleted them. (-:
Why not just put the drive on the network? I'm getting downvoted but I'm just trying to understand how a router is able to delete things on an external disc.
How would you put the drive on the network? Attaching it to an always-on computer, or buy a nas device or something? It seems easier to plug it into your router if the firmware supports it.
When I was a broke college student sharing my neighbors open WiFi (with permission) I convinced him to attach a HD to his router. I ripped all our DVDs, Blu-ray’s, and put our torrents on the HD. We could watch videos from our laptops connected to TVs.
The sea gate drive failed soon after and I lost tons and tons of files. Not to mention it silently corrupted almost every file before biting the dust, videos stopped playing suddenly.
Many of the ASUS routers have a built in feature to use a usb drive's space as FTP/SMB share on the network. Can be handy, although slower than an actual NAS.
A parsing error due to a signature-based malware definition file update is a totally plausible suspect!
It would explain why the router is downloading “updates” but not firmware upgrades.
Also, these signature files contain tons of hex strings and unusual characters used to identify the actual malware (IOCs).
We rollback these updates all the time when a bad malware signature update pegs the AV scan daemon. They are released several times per month depending on the vendor.
Someone more knowledgeable about ASUS asd can probably confirm/deny.
What's your recommended alternative to the newsyslog style of external logfile rotation? I'm not much of a sysadmin but it might be useful to know at some point. Thanks in advance!
The one that people came up with in the 1990s. There are quite a number of implementations to choose from. The shame of this hitting ASUS in 2023 is that this is a long-known problem and a long-since solved one. I have vague memories of grumpy posts on Usenet about this. It's that old a problem; and it has been solved for nigh on a quarter of a century.
Awesome, thanks! I am in fact guilty of using logrotate but thankfully I haven't been burned yet. Although perhaps with the advent of containers piping logs to streams I've unknowingly absolved myself.
Also, you have an amazing website. Looking forward to reading more.
459 comments
[ 2.5 ms ] story [ 436 ms ] thread"I installed version X.Y.Z and so far so good"
"I tried all available updates and it keeps crashing".
There is no transparency in the published updates. Take that firmware blob (which is probably 99% GPL licensed by the way) and shut up. That's basically what the vendors are saying.
What's ironic is that most OEM are using some sort of out of date openwrt derivatives as a base for their closed source firmwares, which are obvious GPL violations.
Do you have a source of that or am I out of date? Based on my knowledge, outdated, GPL-violation OpenWRT would still be a massive upgrade over what's usually developed in-house by OEMs.
Musk's "Starlink" router is also OpenWRT based
https://openwrt.org/toh/linksys/wrt_ac_series#marvell_wifi
Buying $30-$40 access points on eBay is a bit of a hobby of mine. I've been using a Netgear R8000 as an access point for a few months. There's no OpenWRT support, but it's supported by FreshTomato. Qualitatively, it seems to work well in terms of latency and throughput when the signal is strong. It seems to perform worse than the WRT1200AC that I was previously using at that location, when at the fringe. When I'm at the center of my house, my phone tends to associate with it but then struggle to get packets through. Maybe I just need to dial down the AP's TX power, though.
These system are almost always built on the official SDK provided by Qualcomm, known as QSDK.
QSDK is a fork of an old OpenWRT version (chaos calmer, 2018). Unlike the open source version, it includes proprietary drivers and many changes. Openwrt devs refer to it as a "frankenfirmware".
May be i am old fashioned but shouldn't hardware appliances be designed as standalone devices that have minimal external dependencies.
If there was no firmware update or patch applied -- the functionality of the device shouldn't change.
Do all routers these days "phone home" anyway and modify settings on autopilot? Even if user has chosen to turn off updates?
Users won't apply fixes.
Therefore auto-updates became the norm.
You can trust the users implicitly, or you can trust ASUS implicitly.
It seems that the appropriate place to deal with network traffic security issues is with the network bandwidth provider, which would be your ISP.
This would imply that the ISP has abilities to interpose between malicious traffic and the user that, with modern technologies like DNS-over-HTTP, they simply don’t have. To most ISPs (that don’t force you to install MITM CA certs), user traffic is increasingly a black box. Which means that attacker traffic can blend right in as part of said black box. The only thing that can see into the black box is the device on the end of the line.
It's not about catching related traffic when it occurs, more managing the immediate implications from insecure devices on the network. The ISP can see the device, it doesn't need to inspect the traffic.
Have we all forgotten how Nest thermostats turned off the heat in the middle of January 2016 because of a botched software update? (Probably. It was hard to get Google to surface a link without a date due to so much SEO bullshit).
https://www.nytimes.com/2016/01/14/fashion/nest-thermostat-g...
(Though replacing a smart thermostat with a mercury switch one is a five minute job if you're comfortable with it.)
At least in this case, ASUS routers all have local firmware install functionality, so this particular case is certainly not an example of engineers not knowing it’s possible to write offline updates.
Now if you’re really lamenting the lack of computing devices that cannot be updated remotely by design, that’s a different story, and might be on a ship that sailed a while back. The problem there is that local updates are inconvenient enough for most people that they aren’t done, which is problematic from support and security perspectives, even though there are legit problems with remote updates too…
Google no longer does search, it does "recommendation" and it sucks hard.
Now get off my lawn ))
The scary part of auto update is when a company does a bad job of it. For example: letting the auto update site domain expire or point to an IP at a hosting provider that someone might pick up and if the devices don't do proper endpoint validation folks can use it to force downloads of compromised images.
All of these things have happened.
Depends.
"Homemade" routers running open source OS that computer owner can compile themselves need not "phone home" against owner intent. "Home" is the computer owner, not some company.
Commercial routers running closed source OS that owner cannot edit and re-compile can be expected to try to phone home for something, IMHO. A disturbing trend certainly not started by ASUS but which seems to be infecting most hardware sold with pre-installed closed source OS.
I know I would prefer something like that for my non-savvy people. Considering there are smart light bulbs with vulnerabilities which are years away from the landfill. The only question is which botnet will they join the next time they power cycle.
Not amazing, to be sure, and I don't know what those devices cost, but I think the situation is less dire than your post would suggest.
[0] https://openwrt.org/toh/views/toh_available_16128_ax-wifi
(Despite routers capable of gigabit speeds being commonly both free and available on eBay for pennies)
https://openwrt.org/toh/views/toh_single-board-computers
Latest proof of that in my case was with a new Wi-Fi 6 router Netgear WAX206. After indeterminate time clients would just stop receiving packets and would require reconnecting. WAX206 latest factory firmware is based on OpenWRT from 2016 (Netgear still refuses to open source parts of it). After flashing current OpenWRT the connection is rock solid.
These projects are great, but against the background of 100s of new crappy devices every year, nothing can ever be decently finished. Hardware is simply not sold anymore before any device fully matures, and vendors are very stingy with firmware.
Brainslayer is so obsessively paranoid about the idea someone might "steal" and "sell" DD-WRT devices that he obfuscates building the firmware as much as possible.
The devices also are just the stock garbage vendor GPL tarball with DD-WRT's UI on top. So you'll still be stuck with stuff like an ancient kernel full of bugs
This significantly shrinks the number of devices they support, but the devices that are supported have their lifespans increased significantly
Doesn't mean everything MTK does shines under the sun, but in Linux WiFi, they are probably the most open and well supported in the modern age.
6.x kernel is not expected to land anytime soon (late this year or early next year maybe?)
If you have a 6.x kernel, this is probably a fork of OpenWrt.
That name already sounds creepy enough, but searching for that string (with the quotes) currently returns only 4 results, of people asking what it is. My guess is some sort of hidden backdoor, disguised as an ostensibly useful feature.
So DSP magic?
This is very common when I look to find source for embedded devices like this. What I expect is the next step is that you will find (or be given) a borderline useless blob of source that doesn't explain any of it's build process, which is absurd because the GPL clearly defines the build "glue" as part of the source.
Is ASUS another company that is doing a poor job of GPL compliance in this space?
Is it intentional?
That seems to be the way to keep consumer grade routers from requiring the user walk over and reboot them once a week...
Domestic routers, in particular, are infamous for their web interface and management daemons often calling command line programs through system() to configure the network (and other system management tasks), instead of directly using the APIs these command line programs use. Not only is this inefficient and fragile, it also not rarely leads to being vulnerable to shell injection attacks (if you're lucky, only exploitable by authenticated users of the web interface).
In this particular case, the set of states that the device is in when it boots is relatively small compared to the set of states that device may be in a week later. More generally, all individual complex systems need to be "restarted" periodically, it's just a question of how often.
I logged into my router and checked for new firmware, then installed it, and it started working again.
I don't think it has any relation, but since it's the first time it's happened, it was kind of a freaky coincidence!
Hard pass. Not interested in Microsoft's latest malware/adware of an OS.
I've been running Linux on laptops for nearly 20 years now. Sure, it was a pain in the ass in the earlier days, but nowadays things just work, as long as you're willing to do your research ahead of time to choose a laptop where the hardware is fully supported.
Yes of course it was. The only reason.
Delusuional.
You think some conspiracy created Ubuntu so people can point to the Amazon thing?
That said, if the OS is aggressively pushing recommendations for their first party news, videos, and search engine, and every one of those things shows 3rd party ads, I don't think that single step removed absolves win 11.
This is flatly untrue.
It preinstalls Spotify, Disney+, Instagram, Facebook, Tiktok, and a tonne of other tools.
A list with screenshots:
https://www.digitalcitizen.life/windows-10-bloatware/
If you want to argue that preinstalled links to 3rd party websites, prominently displayed in the Start menu, do not count as "ads", then you are acting in bad faith and your arguments can be dismissed.
A bundled link to external properties is an advertisement.
Some of us like to keep our work device ad-free
I recommended it to my friends when they changed their routers over, and it fixed a lot of issues for them as well. The ones who regularly used VR reported that the stuttering issues they had on VR over WiFi with their Netgear WiFi 6 routers were solved.
Have owned both ASUS and Netgear in the past and the experience was terrible -- the high-end ASUS router regularly bricked itself during auto update. So I bought a more expensive ASUS router for around $600, and you guessed it, it did the same thing just less frequently. The NETGEAR routers never bricked themselves, but across the 3 models we've owned nearly all of them would lose most of their throughput (around 90%) on a daily basis and would need a daily reset to get proper ISP speeds again.
I had a Dream Machine Pro for around a year before I moved it to my office to run the network in the office. Funnily enough it has probably been less stable than the tp-link router, but nonetheless the UDM Pro is still a great piece of kit and probably the last router you'd ever need if it's in your budget and you have a place to put it (awkward form factor for home).
Re: Open WRT I ran this around 3 years ago on a D-Link router that I had, and the stability was unpredictable.. better than ASUS and NETGEAR but probably leaning on the side of poor compared to tp-link and UDM Pro. It would randomly get slow once per fortnight or so, and would need to be rebooted. Sometimes it would panic and completely lose internet connection. There are plugins designed to reboot the router as soon as a loss of connection is detected, but why is this even necessary? It's annoying too. Drops all your connections.
Sorry for long.
I would just pick one up available from this list that matches your manufacturer preference(brand loyalty), desired feature set, and availability in the electronic stores you usually make your purchases.
No problems so far at all after 4 months.
As far as I could tell, they were the only source of medium-performance boards with excellent open source support that didn't change the design every 12 months for no f---ing reason.
I also have an ASUS RT-AC86U (running 3.0.0.4.386_51255), but as a node in the mesh, it's still online.
However, my RT-AX55 (3.0.0.4.386_50460) in the mesh has been down all day and I haven't noticed before this post because the other nodes picked up the slack (with spotty/slow signal).
Also have an RT-AX82U (3.0.0.4.388_22525) as the mesh leader and it also doesn't seem impacted.
https://news.yahoo.com/rise-copper-theft-officials-concerned...
My dad works as a network engineer, and he told me a story that one of the banks in Poland lost one of the internet providers. They investigated and found out that thieves stole hundreds of meters of a fiber cable, because they thought that it's copper.
Thank you for writing this blogpost. I just upgraded firmware. So happy when something has an explanation. !
Sadly given the large amount of software that's closed source there I don't see them working towards open sourcing any of it as, even technically, it would be a massive effort.
I run an RT-AC87U (3.0.0.4.382_52545-ga0245cc) as the main router
I have an RT-AC86U (3.0.0.4.386_51255-g486ded6) and an RT-AC68U (3.0.0.4.386_51255-g486ded6) running in AP mode.
Tried checking whether there was an update available for the RT-AC87U, but it looks like the ASUS firmware site is unreachable (overloaded?) for me at the moment.
I still want to know what happened. It feels like a good excuse to get a new non-asus router barring any other explanation
It is trivially easy to blow right past the size capping on systems that use the old "newsyslog" style of external logfile rotation from the 20th century, and something that is logging a short string "[chknvram_action] Invalid string" over and over very fast is exactly how to do this.
For those interested in investigation, therefore, I would suggest looking at logfile sizes, and seeing whether it was logs eating all of the free space on /jffs and /var .
The underlying cause would be whatever is logging "[chknvram_action] Invalid string" thousands of times over, but the mechanism would be log files filling the tmpfs that the article mentions, which would explain why the system had no memory for forking new processes.
My wild speculation about "[chknvram_action] Invalid string" is that something somewhere in whatever "chknvram" is, the name being suggestive of something checking non-volatile RAM, has either bad data or a broken parser, and the recovery semantics are to retry immediately, incessantly, as fast as possible.
In any new-ish production system I'd probably want to use anything other than syslogd anyways.
https://github.com/freebsd/freebsd-src/blob/main/usr.sbin/sy...
If I'm dealing with equipment failures, bugs in third-party software, or other such random tech bullshit, as an individual or a team, then I don't know in advance when and what precision I'll need.
Sometimes those data points don't matter - like if they're generated by some program stuck in an infinite loop. But in other cases, they do - like e.g. if each message is caused by some event, like another program doing some processing, or user pressing a key, etc. - then timestamps will be useful to identify the exact cause (e.g. logs only happen when process X is processing mouse input, or when user presses one of 20 specific keys on their keyboard, or only when my microwave oven is running).
…by default, but it can be disabled:
* https://man.freebsd.org/cgi/man.cgi?query=syslogd* https://www.snbforums.com/threads/what-is-asd-process.76242/...
So some somewhat more informed speculation is that the new signature file either yesterday or today either broke a parser or was itself corrupt. The error-handling path for this is still poor.
I guess Asus quickly discovered their mistake and removed the faulty file from their servers, but affected devices never got to the point where they'd look for a newer file but just choked on the local one.
Why does a router need malware signature files? It has no business monitoring my traffic, except in accordance with the firewall rules that I set myself.
From the article:
> not keeping my firmware up to date
I've had this (non-Asus) router for three years. I've never updated the firmware.
Botnet operators are very grateful for your cooperation, you are helping them a lot
It's probably not 95%, but it's certainly significant. And yes, most consumer routers are obviously accessible from the internet, that's how they work (they're generally not behind a firewall, they are a firewall, and can have vulnerabilities). You might want to read up on it [1, key quotes below], and I know I certainly wouldn't be willing to wager my consumer router wasn't infected, because how would I ever have any idea?
> Out of the box, most routers suck when it comes to security. Vulnerable firmware is an easy target. Backdoors have been discovered in just about every brand of router... It's possible you will never even know your device is infected...
> When we're talking about routers, these aren't the enterprise-level gear made by the likes of Cisco, Sonicwall, or Palo Alto... We're talking about what you would get off the shelf at Wal-Mart or Best Buy — the D-Links, Netgears, and Linksyses of the world.
> The problems start with the cheap, antiquated MIPS architecture used by a lot of these devices' processors... One substantial problem comes from a security flaw that goes back to 2001. Per one deep yet fascinating paper, the chips lack basic defensive abilities against malicious code execution. Combine this with manufactures commonly implementing out-of-date, vulnerable Linux kernels in their devices, and then putting this device on the edge of your network completely exposed to the internet.
But to sum up -- not updating your router's firmware is a terrible idea. It's not something to be proud of.
[1] https://www.cbtnuggets.com/blog/certifications/security/your...
Most of these routers are likely becoming part of the botnet for enabling external web management and/or using default creds (especially if SSH is exposed externally).
Security isn't exceptionally hard, if you actually put some effort into it.
Many of the big manufacturers have been tackling that issue by forcing a password change at setup and not allowing an insecure default to be chosen.
Why is the web interface left on? Just turn that off, there shouldn't be much to do with that.
Most consumer routers do not support disabling the web interface on the local network, as it's the primary (only?) means of administration for them. This attack relies on getting users to browse to an address with default creds by some means, with a URL prepended that will cause the desired action to occur. More often than not a popular action is to modify the DNS servers used, so that DNS traffic can then me manipulated to point to malicious servers used for the ultimate attack.
Issues like this don't happen across such a wide area by accident.
Basic professional practices have any updates tested beforehand on physical hardware at multiple stages before any push happens, and they leave it up to the user because pushing an update without the owner's explicit consent to a device they own, runs afoul of the same hacking abuse laws. The legal exposure is massive.
It also doesn't account for the fact that firmware was attempted to be updated on devices which were set to not update. ASUS has a lot to explain. They didn't release any kind of statement so the lawyers and computer forensics people will likely need to get involved to get to the bottom of it.
I look forward to eventually hearing about what really happened.
https://www.trendmicro.com/en_us/research/22/c/cyclops-blink...
Hopefully that clears up the first question of "why does it need malware signature files?"
As for your router firmware? You should seriously update that. New exploits get found all the time.
Hackers use compromised routers as parts of a botnet, a intermediary route, or as an access point with which to steal data with Man in the Middle attacks.
Signature files are only useful to scan network traffic.
How is it not relevant? This person has not updated their router's firmware in over 3 years.
The viral traffic needs to get to the router in the first place. I assume that the means of reaching the router is literally via network traffic?
What am I missing?
The malware signature files really don't help prevent your router joining a botnet.
Firmware updates, maybe, maybe not. It is quite possible for other routers with less generally sloppy and advertised-feature-rich firmware to actually be more secure even without updates for 3 years. It's quite possible that they have no api endpoints available for super-easy mobile app integration remote management etc, just ssh from local subnet or physical serial console.
There have been multiple cases of market-leading antivirus engines (symanted, mcaffee, etc) having sloppy code running with the highest possible privilege, parsing any files appearing on the system anywhere, and e.g. crashing a mail server that would otherwise be unaffected by the PoC samples being emailed through it by researchers.
So, I also take some issue with people who have no understanding of how all this software around us is designed and built (in routers, in windows, on web servers) and thinking that just updating everything all the time and running antivirus is the best you can do. You really can do a lot better if you know what you're doing.
https://lwn.net/ml/oss-security/20221013101046.GB20615@suse....
There can be driver specific remotely exploitable issues that might not be widely communicated. Until operating systems are written more robustly, just having admin level stuff set up robustly isn't always enough. Of course, updates can add bugs too.
These types of signature files aren’t meant to guard against exploits, SSH brute forcing, etc, even if the router applies them to inbound traffic in addition to forwarded traffic. To do that, you typically need a WAF or some clever fail2ban-like filtering rules. Even up-to-date signatures won’t prevent a router from getting 0wn3d if the ssh daemon has a security hole for example.
As sites move to HTTPS, routers can’t even really filter networking traffic anymore. I don’t see why a router needs signature lists at all
Not really. The article doesn't mention signature files. It describes the operations of a certain malware once it has been executed on a router. But first it has to get onto the router; and the only way new software can get onto this router is via a firmware update.
That has never been the case. Software from consumer routers is often still in the "a trivial buffer overflow lets a malformed packet insert a payload into ram and convince the PC to jump to it." phase of software security. They are very much still wormable systems, like Windows 2000 style. Ever have your router glitch out and stop working and you have to reboot it? That's probably an exploitable bug.
They argue that he doesn't understand and he's stupid to want to be part of a botnet and that Asus obviously know what they're doing.
The sea gate drive failed soon after and I lost tons and tons of files. Not to mention it silently corrupted almost every file before biting the dust, videos stopped playing suddenly.
It would explain why the router is downloading “updates” but not firmware upgrades.
Also, these signature files contain tons of hex strings and unusual characters used to identify the actual malware (IOCs).
We rollback these updates all the time when a bad malware signature update pegs the AV scan daemon. They are released several times per month depending on the vendor.
Someone more knowledgeable about ASUS asd can probably confirm/deny.
See https://jdebp.uk/FGA/do-not-use-logrotate.html for everything from Bryan Cantrill to comments in GNU source code. (-:
Also, you have an amazing website. Looking forward to reading more.