HN Explorer

Ask HN: A network device that doesn't exist?

37 points by jaclaz ↗ HN
I am not a programmer, nor an IT or network expert, so please be kind if you think that what I am asking for is a solved, elementary problem.

I am struggling to find a programmable device that can switch among a few RJ45 sockets for failover functions.

The problem:

I have a LAN where connected devices are connected to the gateway, for the sake of the question, the gateway is 192.168.1.1.

For some reasons the devices connected to the LAN have static IP addresses and the gateway is also "fixed" to 192.168.1.1.

For some other reasons the actual DSL router(s) have all their IP address set as 192.168.1.1.

From time to time[1] the "main" DSL router connection may fail (due to the router itself, its power adapter, ISP, cable, whatever) and I have a "backup" DSL router/modem connected to a separate DSL line and a third one with a (slower, but good enough) 4G SIM.

My current (caveman) manual method is to keep the two backup routers not connected to the network and in case of failure of the main one simply disconnect the RJ45 cable from it and insert it in one of the two backup routers.

There are manual switchboxes that allow switching among 2 or 4 network sockets, example:

http://www.cablesonline.com/abrjswitbox3.html

Using one of them would simplify switching when there is a failure when I am not physically present, as the instructions on the phone would amount to "Rotate the selector to B, if you still have no internet rotate it to C." (as opposed to "Disconnect the yellow cable from the white router and insert it in the back of the blue one, if you still have no internet, disconnect the yellow cable from the blue router and insert it in the back of the smaller white one") .

I know I could use a small router such as the Microtik hEX (RB750gr3):

https://mikrotik.com/product/RB750Gr3

But that would imply that the router gets the 192.168.1.1 and that the three DSL routers get addresses in another range, let's say 192.168.2.1, 192.168.2.2 and 192.168.2.3 (and this change needs to be done on at least one of the routers by the ISP) and, if - for whatever reasons - the Microtik router itself fails I cannot bypass it without changing the gateway on all connected devices (doable, but complex and taking a lot of time) or changing the IP address of the (working) DSL router (which is anyway complex and at least in one case not possible).

The "ideal" solution (in my perverted mind) would be something like the manual switchbox (that knows nothing about IP addresses but simply connects the IN port to the chosen OUT one), but capable of having some sort of failover function and automatically switching to the next RJ45 in case of missing internet connection AND somehow notifying that the main router/modem/connection is down and the router/modem/connection in use is the first or second backup one.

Maybe something like this does actually exist but I cannot find one?

Or there are other possible solutions that you can suggest?

47 comments

[ 3.1 ms ] story [ 156 ms ] thread
mikewarot
I went down this road about 15 years ago as the IT person for a small firm. My first pass at it was a linux box with 2 network cards, and 3 addresses (2 public IP addresses on eth0, and a local address on eth1).

Linux only has one routing table, so I had to use ebtables, and ipchains (I think?) to mark packets as they came in, and route them back out correctly. It worked well, but I was worried about the long term stability of it.

Then I found out that SonicWall made a box that did failover automatically, and never looked back.

I strongly suspect there is an open source project which does this these days.

h2odragon
> Linux only has one routing table,

Not the case for 20 years+. see "TABLE" in `ip route help` and `ip ro ls table all` to see all the route tables you currently have active... you might not have seen the 'local' routes before

mikewarot
That's cool.... the situation we were in was that we had multiple public ip addresses in each of our class C static addresses, but no BGP routing because we were small fry. I used DNS round robin so that from the outside world, it would try either route in, then forward those packets in to the appropriate server.

Its good to see you don't have to mark those packets manually any more.

huhtenberg
A simple homebrew option would be to get an intermediary box @ 192.168.1.1, stick all three routers into it, but keep only one of three links up. Then have a script on the box ping outside and switch to the next router on ping failure. I.e. if (ping 1.1.1.1 -c 1) fails N times in a row, bring current router link down (ip link set <dev> down) and bring next one up (ip link set <dev> up). Rinse and repeat.

Alternatively, you can keep all router links up, NAT their 192.168.1.1 to unique IPs and then have the script change the default route. This setup will allow for checking if previously failed higher priority router regained its connectivity and switching back to it if it did.

jaclaz
Yes, that would be the solution with a "box" that needs to have at least 4 ethernet ports, to which this or that Linux mini distro is installed and configured, I was looking for something already made and tested/easily configurable.

As you detailed, the way of working of this hypothetical device is simple enough (maybe too simple) and I thought that something like that existed and I was simply unable to find it.

syntaxing
What you want is something called a WAN failover. There’s a bunch of routers that offer it but I know OPNsense (and pfsense) has it. The static IP address does not matter (to a certain extent since you have a double NAT and some router disable this by default for safety reasons). The WAN failover is usually defined by the NIC so you would say you want eno0 is primary, eno1 as secondary, and eno2 as tertiary which correspond to physical ports on the router.

If you want something plug and play, look at Firewalla. I haven’t used it but I heard that it’s so easy “even your parents can set it up”.

tikkabhuna
I haven’t heard of Firewalla but I’ve used UniFi products. The UniFi Dream Machine Pro can do WAN failover.
WarOnPrivacy
> What you want is something called a WAN failover. There’s a bunch of routers that offer it but I know OPNsense (and pfsense) has it.

First place my mind went for a recommendation - and then discarded for someone w/ their exp level.

I would guess they could set up a pfSense box. Netgate has pretty good online docs. But even after deploying pfSense for 15 years, I still approach CARP (WAN failover) with some trepidation. It doesn't always just go (or stay going).

> look at Firewalla. ... it’s so easy even your parents can set it up"

New to me too. It sounds promising.

[deleted]
(comment deleted)
04rob
I believe what you are looking for is called a multi-WAN router. More specifically, one with a failover function.
ssss11
Years ago I worked on similar problems with retail stores and dodgy pstn & dsl. If you’re worried about the links themselves you can have one router device with multiple links connected to it and it fails over. They also can have a SIM card inside to failover to (4g I guess these days) mobile networks.

If you’re specifically looking to fail over between router devices… I think the only thing I ever did on that was have multiple exactly the same routers with exactly the same config - one hot and others as spare cold devices on site lying in wait. Not sure I ever used it.

jaclaz
Yep, this is how I have the current setup:

1) "main" router connected to a FTTC line

2) "secondary" router connected to an older (and slower) DSL line

3) "tertiary" router connected via 4G SIM

What I do now is simply take the cable out of the "main" router and connect it to the "secondary" one, and if also this one has no connection (cut cable, it happened three or four times in the last couple of years) connect it to the "tertiary" one.

As said, the issue is only if for whatever reasons I am not there, the A/B/C/D rotary selector box would be a simple solution that anyone present could be instructed to use, while connecting and disconnecting cables can be more difficult to explain over the phone.

The hypothetical device I was looking for would be an automated version of that.

Stem0037
Have you considered reaching out to specialists who might be able to create a custom solution for you, such as a small device with a script that could automate the switch over based on the health of the connection?
jaclaz
How much would one of these specialists take to create a custom solution like this?

I would expect something in the thousands of dollars range, not exactly affordable.

sjsdaiuasgdia
VRRP [0] is what you're looking for if you want to minimize failure points and manual actions. This allows multiple devices to "share" ownership of an IP address, giving you a single gateway address you can point all the clients towards. The protocol can detect when the current active device is unresponsive and transfers ownership of the IP to an available device. VRRP can be used on Linux via keepalived [1].

Setting all that up with a couple Linux machines would require some networking knowledge, but there are tutorials out there for all the pieces you'd need to put together.

[0] https://en.wikipedia.org/wiki/Virtual_Router_Redundancy_Prot... [1] https://www.keepalived.org/

subhro
I think CARP might be a better solution.
reliefcrew
I am/was relying on UCARP for this but the problem was almost always the connection, not the machine. So instead of running two machines all the time I just ended up writing a custom script to handle failures.

This was a pretty long time ago and these days I'd consider an existing tool like: https://lstein.github.io/Net-ISP-Balance/

zamadatix
Do you expect the solution to fail back to the primary DSL when it comes back and to fail over on more than just physical interface up/down? If so things become significantly more complex to the point I'm not sure it's really making life any easier

Honestly, as massive a pain as it might be, manually going through all the sites to make the internal and external network contexts decoupled (I.e. clients not pointing directly at the IP of a single carrier) is going to be a lot more reliable than whatever hacked together solution sounds like a good idea now. Doubly so if you're looking to make that solution itself redundant.

necovek
In theory, you could have a relay for each of the 8 Ethernet wires, and programmatically control that.

But... you could also set up an OpenWRT router and if you can't configure IPs in non-conflicting way, you could turn particular network interfaces on and off as it's basically a Linux box. Note that most consumer routers compatible with OpenWRT usually have only two network interfaces, even if they have multiple ports for one of them.

digitalsushi
I have that general problem. My wife and I work at home full time (well, it was full time til RTO, sigh) and we live in the woods with lots of trees. So we have two ISPs.

We have a pfSense router made by netgate that we can tell our preferences for how to use the WAN connections. We plug them both in and we can tell it to use one as a backup, or use both equally, or with whatever affinity we like.

pfSense can also be installed on any old hardware - I just wanted a tiny silent one with some support because my day job of testing packets for other people seems to be over and the desire has faded to know how it works. (My first router had a cold solder joint and failed ... it was some work to get an RMA through because the Texas power grid, where the company is, was offline for a week when I issued it, but that's life)

woleium
most folks use opnsense now instead of pfsense
zooFox
Probably different solution, but my Amazon Eero mesh allows having a backup WiFi.

There's a primary network that's connected via ethernet and I can set up a secondary network (in my case, iPhone hotspot) as a backup. https://techcrunch.com/2022/09/28/amazon-expands-eero-line-a...

The advantage of it is that it's quite simple and plug-n-play.

Would that work?

SigmundA
You need a router with multiple wan fail over support. You could build your own with PFSense or Opensense etc or buy one the supports out of the box.

Best one to buy for your situation IMO is a Pepwave Balance 20x [1], they are rock solid and specialize in multi-wan with a bunch of options from simple fail over to bonding them all together with vpn tech.

With the Pepwave you could put the 4g sim directly in its modem then get 2 wan licenses and plug the two DSL modems into them and set priorities as needed. Ideally the DSL modems would be in passthrough so you don't double nat but should work either way.

I use one of their routers in my RV and it seamlessly and reliably switches between Starlink via ethernet, 5g cellular and WiFi WAN.

1. https://www.peplink.com/products/balance-20x/

ahoka
A simple mikrotik netwatch script can solve this. No need for routing protocols.
Brajeshwar
Talking about consumers/prosumers, If I understand the problem correctly, isn't this done by the likes of most common multi-wan routers? I remember it was costly and we used Mushroom[1] Routers in late 2000s.

Right now, I've three Internet (kinda bonded) with failover using a TP-Link ER606[2]. Here is a simplified picture[3] that I did on a Saturday night.

My eventual upgrade that is likely to happen is to overhaul the network and move to Ubiquiti[4] and use one of their Edgerouters[5] or at the Dream Machine/Router[6][7] level (if it has the option).

The WAN router is the DHCP provider and all other wi-fi extensions / access points, picks up and distributes the Internet. So, your internets from the ISP supplies to the WAN Router.

1. https://www.mushroomnetworks.com

2. https://www.tp-link.com/us/business-networking/omada-sdn-rou...

3. https://www.instagram.com/p/CUWeopdPVOp/

4. https://ui.com

5. https://store.ui.com/us/en/products/er-x

6. https://store.ui.com/us/en/pro/category/all-unifi-gateway-co...

7. https://store.ui.com/us/en/pro/category/all-unifi-gateway-co...

tbyehl
The part you're missing with your idea of putting another router in the middle is that the LAN clients won't know and don't care about the router's next-hop(s). The LAN clients will always talk to router on 192.168.1.1 and the router will decide how and where to forward the packets.

You'll definitely need to get the one WAN device re-numbered off of 192.168.1.1 and probably all of them should be on unique subnets.

From what I know of Mikrotik, I think it'd be much easier to configure on a Ubiquiti EdgeRouter product (not UniFi or UISP). There are also routers from TP-Link, Peplink, and others that specifically market themselves for multi-WAN and may be easier still to understand/configure.

tbyehl
Oh, and as others mentioned, VRRP or CARP is how you can get high availability for your new routers. Both Mikrotik and EdgeRouters support VRRP.
arraypad
Something which goes further than this, but works very well for my use case and would probably suit others in the WFH crowd: OpenMPTCProuter [1]

This handles failover between connections and also aggregates them using MultiPath TCP to maximize bandwidth & overall reliability at the expense of increased data usage and the cost of running a machine somewhere with a decent connection, even a cheap VPS.

I'm using it to aggregate ADSL, Starlink and 4G, resulting in a stable 250mbps/50mbps connection.

* [1] https://www.openmptcprouter.com/

woleium
but then you face the same problem as when you have a VPN to a VPS, sites will see your IP address, see you are coming from a cloud service provider, and block you.
jitl
My Synology wifi router can do WAN failover and vlan tagging
toast0
Ok, so part one, like everyone is suggesting is you can probably make this fully automatic. I say, you don't even need to change the router IPs, but it's more difficult if you don't (you might need to setup 1:1 NAT to get sensible addresses and go from there). A new router with 4 interfaces, one to each modem and one to your network switch (if you can change modem IPs and disable DHCP on them, you can do everything with one NIC and a single collision domain, although you may not want to). You could get two devices each with 4 interfaces and use pfsync and CARP to make failover nearly seamless. There's appliances for this, but I've setup something similar from scratch on FreeBSD, use DSL if it's up, fail over to LTE otherwise.

Part two: what you asked for exists. Look for bypass NICs. There's a couple different flavors, but the commonly available pattern is a dual-port NIC, which you run a utility to switch from bypass mode (the two ports are connected via relays, the NICs are disconnected) to regular mode (each port serves one NIC).

If addressing is compatible, you could have three bypass nics, plug one end of each to your switch, the other end to a modem. The selected modem would be in bypass mode, and unselected modems would be in regular mode. (You could also do this with software bridging, only bridge the selected modem). I've had success with silicom bypass NICs, for older models like PEG4BPI, the -SD model is going to be easier to use PEG4BPI-SD has PCI ids set so that standard OS nic drivers will load automatically for the nics, but PEG4BPI has PCI ids set up differently and drivers won't start unless you modify them or can force them. You can actually modify the eeprom and turn a regular into a -SD, with just one bit flip, but that's an ordeal.

jaclaz
These bypass NICs sound interesting, thanks, I'll have a look for them.
toast0
Plenty used on ebay at low cost. Quad port 1G (two sets of bypass) go for $25. Dual port (one set) 10g-baseT is maybe $30ish. Fiber bypass nics are also available, they'll have direct fiber connectors rather than SFP+ though (bypass will be optical).
zamadatix
At the last place I worked (3200 physical locations) we scrapped physical bypass ports/NICs because they seemed to create more site outage time than they ever solved (maybe there is actually a model that is reliable at scale, but we didn't find it).

A port physical state watch script tends to be better. I don't know what lower end gear has it though, some may even support that as a mode of their "WAN monitoring" instead of actively sending probes. Worst case I know Ubiquiti Edgerouters let you run your own scripts on the Linux base and it'd be a matter of running "primary port up ? remove vlan from backup port and add vlan to primary port : remove vlan from primary port and add vlan to backup port" in a loop every second (there may even be a native feature for it though so check that first).

jaclaz
I thank everyone that replied, but most if not all the answers do not take into account the limitations of NOT changing the current (192.168.1.1) address of the gateway and/or are anyway using the "added router in the middle" option where the proposed Microtik hex router is replaced by some other brand/make/model.

I understand (a little) the way multi-WAN routing works, but due to the exposed limitations that cannot work.

The whole problem revolves around the fact that the (hypothetical) device (if it is a sort of router) has an own address of 192.168.1.1 and that it should be able to route/switch to one of the three DSL/4G router/modems each one having the same 192.168.1.1 address.

I am now thinking of a ethernet relay like the ones used in home automation, similar to this one (example):

https://relaypros.com/Relay/Relay/ETHERNET_RELAY_CONTROLLERS

but I read somewhere that it is not a good idea to use the relay contacts on the actual ethernet cable, so I could use one of them to control the power of the three routers/modems, but this would introduce a (bearable, still ...) delay as the routers take some time (tens of seconds, one is particularly slow and takes over a minute) to boot and establish connection.

taneq
How fast does it need to switch? An alternate approach would be to have all routers physically connected, but only one of them powered. You could probably do this with a smart power board and some scripting to switch the power points over if pinging an outside server stops working.
jaclaz
Yes, I thought about this approach, but as said at least one of these dsl routers is very, very slow at booting, more than 1 minute, 2 or more if I recall correctly, in the "three level cascading/rotating" setup the pinging script would need to be timed (either in seconds or in number of failed pings) to allow the booting time for the selected router before deciding that even the router that was just connected has no internet and decide to connect to the third one. I could leave the "other" routers powered on at all times and use three el-cheapo unmanaged switches to connect them and power on/off these switches instead, this would be almost instantaneous.
wiml
As other posters say, there's probably a better solution, but to answer your direct question about how to automate the "unplug this cable, plug in that one"...

How about plugging all four cables into a managed Ethernet switch, and then tell it to disable the ports you don't want? That'll avoid any IP-layer shenanigans. You will need to have a box somewhere to run the script that pings things and then reconfigures the switch when it decides it's time to failover.

jaclaz
Yes, the managed switch could be a possibility, the "box somewhere" is an added complication, but sounds like easy enough.

Essentially what I was looking for would be an easily programmable managed switch capable of pinging and switching ports on and off, i.e. a self-contained one.

Maybe something like that can be done with an Arduino or a RaspberryPi or similar small boards, but I was hoping there was something similar already made.

mannyv
So really, what you need to do is put another box between your DSL/4G modems and your gateway. I'm assuming the gateway has NAT, and your DSL modem has to be NAT. So you'll go to a triple NAT, which is less than ideal.

You can get an edge router X and set up the wan on three of the ports, and set the last port to your gateway. ER-X can do failover WAN automatically.

You can also get rid of the gateway and connect your LAN devices (via a switch) to port 4 of the ERX. Just set its IP to 192.168.1.1. The ERX NAT is pretty fast, but you're on DSL so that probably doesn't matter.

Do you need the gateway box at all?

jaclaz
Maybe there has been a misunderstanding of sorts.

By "gateway" I meant the IP address set in the various LAN devices for "default gateway" that (for the sake of the question) is set to 192.168.1.1.

Imagine that there is only one device (a PC) on the LAN and one DSL router/modem.

The router/modem (the actual "box") has an address of 192.168.1.1 and the only PC has set as gateway 192.168.1.1.

Now, let's say that there are instead several devices on the lan (all set with default gateway 192.168.1.1) and three DSL routers (that all have address 192.168.1.1 and that cannot be changed) of which only one is connected to the LAN at any given time by physically connecting/disconnecting from the main LAN switch their RJ45 cable (this is the current situation).

Instead of connecting/disconnecting manually the cables, I could put a "switch box" that would allow switching among the three routers cables, that would be something that anyone can be told to do over the phone in case of need.

I was asking if there was something ( a simple device) like that but that could be operated remotely or programmatically.

I presume that almost any router or managed switch (as it has been suggested, probably much better) can put a port in "up" or "down" state, but in the case of a router there would be some IP conflicts.

I could change the IP subnet and default gateway on all devices of LAN to (say) 192.168.0.x and have the "added" router be the default gateway at 192.168.0.1, routing all traffic to 192.168.1.1 (the fixed IP address of the DSL routers) but doing this, in case of failure of the "added" router I would have to change the default gateway on all the connected to LAN devices.

The "managed switch" sounds much better, though still if there was a simpler device it would be preferrable.

spiderxxxx
make them all have different ip addresses, assign them all in the routing table as default route with the metric being different, whichever one has the lowest would be chosen first, if that one is down, it should choose the next one with the second lowest metric. So, you can set 192.168.2.1 as metric 100, .2 as metric 200, .3 as metri 300
jaclaz
The actual "limitation" is that I cannot change the addresses of the DSL/4G routers.
woleium
Use a CARP floating IP as 192.168.1.1
jaclaz
Somehow answering my own question (though not in a satisfying way, as after all the hypothetical device I was looking for seemingly does not exist).

It seems like Mikrotik routerOS (probably also other routers/OS's but I didn't find any evidence about that) through some "magic" tinkering with prerouting/mangle/connection marks:

https://forum.mikrotik.com/viewtopic.php?t=187178

allows addressing from a same router more than one device with the same IP address.

So with two routers in cascade it would be possible (maybe) to use the first as the failover engine (recursive) routing from itself (192.168.1.1) to three different IP addresses (on a different subnet, let's say 172.16.0.0) that are managed by the second router that "routes" 172.16.0.10, 172.16.0.20, 172.16.0.30 to the 192.168.1.1 on ports ether2, ether3 and ether4.

If this works, the two routers can be (in case any of the two fails) by-passed by manually reconnecting cables (to only one of the DSL/4G routers, as it is now) without changing any configuration on the network clients.