I'm particularly excited about the retransmit tracepoints, as troubleshooting some of these issues on systems that can't afford the tcpdump overhead has caused me significant headache in the past.
tcpdump shows on-the-wire packet details only, and works by instrumenting all send/receive events. These tracepoints can expose a lot more information: kernel state (including congestion control metrics), process IDs, application stacks, etc, and do so with much lower overhead by instrumenting lower-frequency events only. This makes it much more suitable for production use. We'll be able to collect TCP session information 24x7 with negligible overhead, whereas tcpdump/libpcap would be too expensive in our environment (Netflix).
I haven't used Little Snitch, but if it's a firewall then yes, this work is related to the eBPF work in Linux, which has been used for custom firewalls. There's a lot of work in this space, so I don't know where best to start. One example is cilium: https://github.com/cilium/cilium
Aha, that looks great.
Yes, it's a firewall/network monitor, which monitors/controls network traffic tied to specific applications. That does appear to be the level 7 piece in cilium that, based on my naive reading of older attempts at Little Snitch-alike projects, was difficult, unreliable, and costly to do.
Now I can confirm my excitement about these developments, even if I don't really understand them. :)
A "Little Snitch"-like system for Linux would be pretty easy to develop without these changes as well. I've thought about building such a tool, but never really had the need or saw the point over my existing workflows.
I did once (over 10 years ago now) develop a tool that would, upon receiving a packet from a new source, perform an OS fingerprint on it and if it turned out to be Microsoft Windows, block it once the fingerprinting process returned... :-) This tool actually just fired off a script (JIT compiled) for every packet that matched pre-defined criteria and that script could return a verdict on what to do with that packet, which could involve inspecting userspace connections trivially
I think the cgroup BPF filters align more with what Little Snitch can do as opposed to the TCP tracepoints. In modern versions of systemd, you can restrict a given network application to only having access to specific IPs via this mechanism.
It's more about starting out with being able to see what network activity specific applications are causing before you try to define what's allowed. There are a lot of programs running on my linux desktop that I would first need to monitor before I can decide if they should be restricted (or simply uninstalled).
No. I always make sure I put Linux and BSD logos on my slidedecks when introducing Netflix. Linux on the cloud, BSD on the CDN. And also, the TCP tracepoint kernel commits was really done by multiple other companies. :)
17 comments
[ 9.3 ms ] story [ 51.9 ms ] threadI'm particularly excited about the retransmit tracepoints, as troubleshooting some of these issues on systems that can't afford the tcpdump overhead has caused me significant headache in the past.
For older kernels, there is a ftrace way for tracing retransmits: http://www.brendangregg.com/blog/2014-09-06/linux-ftrace-tcp...
The oldest are SNMP style counters used by "netstat -s" / "sar -n ETCP" for all retransmits on the system, if you don't need per flow stats.
Ten there's an application level getsockopt TCP_INFO, detailed at: https://linuxgazette.net/136/pfeiffer.html
Haven't used it though.
I did once (over 10 years ago now) develop a tool that would, upon receiving a packet from a new source, perform an OS fingerprint on it and if it turned out to be Microsoft Windows, block it once the fingerprinting process returned... :-) This tool actually just fired off a script (JIT compiled) for every packet that matched pre-defined criteria and that script could return a verdict on what to do with that packet, which could involve inspecting userspace connections trivially
Do you (or anyone else here) know if something like this exists already, or is in the pipeline, for raw sockets in Linux?