This stuff probably wasn't around when IP was designed, but in my own toy routing network protocols I use bloom filters and cuckcoo filters to remove duplicate packets.
Trivial amount of space and computation, easy fix for loop issues. Especially since I assume malicious actors can always connect to any point in my toy network.
I guess the amount of computation required is too much for backbone routers, but for my toy projects it works nice :)
There's no such thing as a duplicate packet. Older protocols may very well send keepalives as a (UDP) packet that repeatedly has the same contents. Removing/dropping such packets is not something a router is "allowed" to do.
Also, apart from that, all but the smallest routers do not look at the actual data packets. They set up some rules for what they want (i.e. traffic addressed to them, routing protocols, some ICMP, etc.) and everything else never leaves the hardware forwarding pipe. Last I checked, a fat x86 box clocks in at a few 100Gb/s of software forwarding while hardware routers are breaking the 10Tb/s barrier easily.
[Ed. to add:]
Actually, if something further down behind your "deduplicating" router is overloaded/dropping packets, plain TCP will resend the lost packets with little to no changes. If the deduplication drops the retransmits, you've now broken plain current TCP ;). And having dropped packets & retransmits is a fundamental part of how TCP congestion control works (less so with ECN, but still.)
Feels like an ad piece for <you know who if you read the article>.
Loops in IP routing are a pretty well researched topic and AFAIK practices to avoid them are on most network professional cerficiation exams. Microloops during convergence are even an active topic of research.
Also, just to be clear, a loop can only bring down systems involved in it, you can't use it as a "packet accelerator". (Unless something is seriously borked, or you have a multicast loop.)
3 comments
[ 3.4 ms ] story [ 10.3 ms ] threadTrivial amount of space and computation, easy fix for loop issues. Especially since I assume malicious actors can always connect to any point in my toy network.
I guess the amount of computation required is too much for backbone routers, but for my toy projects it works nice :)
Also, apart from that, all but the smallest routers do not look at the actual data packets. They set up some rules for what they want (i.e. traffic addressed to them, routing protocols, some ICMP, etc.) and everything else never leaves the hardware forwarding pipe. Last I checked, a fat x86 box clocks in at a few 100Gb/s of software forwarding while hardware routers are breaking the 10Tb/s barrier easily.
[Ed. to add:]
Actually, if something further down behind your "deduplicating" router is overloaded/dropping packets, plain TCP will resend the lost packets with little to no changes. If the deduplication drops the retransmits, you've now broken plain current TCP ;). And having dropped packets & retransmits is a fundamental part of how TCP congestion control works (less so with ECN, but still.)
Loops in IP routing are a pretty well researched topic and AFAIK practices to avoid them are on most network professional cerficiation exams. Microloops during convergence are even an active topic of research.
Also, just to be clear, a loop can only bring down systems involved in it, you can't use it as a "packet accelerator". (Unless something is seriously borked, or you have a multicast loop.)