Interesting.
I thought about doing this immediately after reading their old blog[1] post on punching through NAT some time ago.
Just a combo of never getting around to it and friends talking me out of it b/c of existing alternatives such as wormhole[2].
Is NAT traversal actually that big of a feature? The category of people that would use a tool like this already knows many ways to do it without NAT getting in the way.
Unless you have some kind of dynamic DNS with a background daemon, you'll be bothered by NAT eventually. And even then it's hardly as convenient. With Tailscale, I can sit on a rooftop bar in the Medina of Marrakech, connect my phone to the public WIFI, and access my laptop in the hotel WIFI across town. Or production infrastructure on a highly secured server somewhere on the other side of the planet. All without compromising security, and all devices involved just pick the shortest physical link to each other.
So have you done NAT traversal, or does one end just not have that issue? Cause I've done SSH forwarding etc a million times but never done NAT traversal.
If you ever connect over a mobile hotspot, then you need NAT traversal. And if you want something that "just works" across various network environments, you need NAT traversal.
always have! our darwin and windows clients are closed source, but they wrap the oss implementation in github.com/tailscale/tailscale and you can see and even use all the same hooks yourself.
the control plane is closed source, but headscale is an open source alternative that we embrace and encourage people to use if it meets their needs/desires
Half of the Software offering nowadays seems to be selling vendor lock-in at no added value and then making a profit.
Sure there's always some negligible added value, and then they reinvent a whole stack for their ecosystem. In the case of tailscale the added value seems to be avoiding going into your router and activating port forwarding? It looks like negative added value to me. Someone that tells you "don't do NAT traversal, just open a port" would be highly valuable, but saying no doesn't seem to be trendy, and is certainly not an easy sell.
> In the case of tailscale the added value seems to be avoiding going into your router and activating port forwarding?
Well, ignoring the tone... kinda yeah? I mean, if port forwarding works for you, and you have public IPv4, and you have no CGNAT, and you need a connection only between 2 hosts, then port forwarding is actually a half-decent idea. However, it's not what Tailscale excels at.
You make it sound like both options are equally valid, and as if port-forwarding were some esoteric technology. It's not, one is good, the other is bad.
5 years ago I had a similarly bland position, live and let live, I'd go to a client that was using a Ddns (zapto.org) for production loads serving thousands of customers. 5 Years later, I see them constantly facing SSL issues, and network issues, and even if they weren't it's still the wrong way to do things.
We have to stand for something, get an IP address, get a domain. If you don't have either of those, raise funds, get a budget, speak to people, you can't tech your way out of some things, I know we are hackers, but sometimes the solution is not hacking. Yes we should develop the power to do things even without approval from institutions and without collaboration from the central NIC, or the domain holder, etc... But we should refrain from using that power and actually integrate with domain holders, actually get IPv4/IPv6 blocks.
There's many reasons why you would use a ddns or similar tech, lack of budget, lack of main domain holder approval, lack of NIC approval, lack of ISP approval. In all cases, you are way better off by actually going the other way instead of using a ddns.
1- no budget: Get a budget, a domain is like 15$/yr. a 1024 IPv4 block leased from a NIC is like 600-2500$/year, so the cost of a Host or ISP is 0.6$ to 2.5$, maybe they charge you a margin and you end up paying 5$ to 50$/yr? If this is an issue, maybe your project should die, how much is your time worth? Fund your projects accordingly, or let them die. If this is someone elses project, they cannot afford you.
2- ISP: Call your ISP, if they can't help you access your router, or port forward or whatever, hang up, call again. If your solution is to use a ddns instead of picking up the phone, you have overtrained your hard tech skills and need to work on your soft skills. Speak to a human, you won't get very far if you never speak to humans. This is not a niche thing, we call it port-forwarding but it's how TCP and IP work in general, this is the stuff of the internet, if you can't do something as basic as that, you are never gonna make it, sorry.
3- Domain holder: If you work for a client, and they have a domain, but you don't know who holds it, or where the passwords are. Don't jump to a ddns or buying another domain, integrate, investigate, do an inventory. At the very least buy another domain, a ddns is just cheap to the point of being unprofessional, see 1.
There's a 4th excuse that I'll handle as well:
4- I'm just testing, this is just a throwaway project: Even if it is, you are still underfunding your bets if you don't even have a domain and standalone non-firewalled IP, even if reusing an existing domain and host, we are talking about like 5$ per month. Imagine a professional of any class that cannot afford a tool worth 5$, even a plumber has a toolkit with a screwdriver in it.
Even if you share a link to your project to a friend, if it says hsabhsabdsa.tailscale.net or bjsdbsajdbsa.vercel.app , you are already neutering your project. You can skip this 'phase', it's not necessary for a project to go through all phases, especially if a vendor is trying to convinve you that you should go through this phase, skip it, tell them no, build it right even for your day 1 demo, it's better than having a pre-day 1 demo that has low quality and does things wrong.
A bit off topic: it’s just insane how I used to watch this guy’s http2 in Go yt video 10 years ago, and he’s still very relevant to this day!
Cheers Brad!
https://github.com/n0-computer/pigeons offers the same but with SSH.
I use it in my homelab extensivelly for access, tunneling smb, postgres and some other minor bits.
I'm a big fan of iroh, but I think iroh's holepunching algorithm is at least partly based on tailscale's, so it's not 100% accurate to say iroh got there first. (Just the "as a library without a control plane" bit.)
Yes, though MW has the important distinction that it uses short human-readable session identifiers. This means MW supports file transfer where the out-of-band channel is just a phone call. This is also why it needs PAKE as opposed to the simpler cryptography used in tailcat
Not a huge lift to stick a PAKE in this protocol, though I'm dubious that it'd be much of a win. The PAKE + nameplate system in Wormhole makes a lot of sense, because you're doing one very specific thing with that system (moving a file from point A to point B). Here you're booting up servers; you're almost always going to have a non-voice channel to set up with.
You also don't want adversaries to be able to disrupt long-lived streams with bad password guesses, since I think part of Wormhole's security model is it will terminate the session if the other side gets it wrong.
I enjoy that they supply a nix install/environment, similar to the main tailscale/tailscale repo. Is nix widespread or the standard dev environment at tailscale, or is it like a 10% option and most people just use Docker or whatever?
Maintaining and providing a deb is pretty easy too; the real killer for a nix flake I think it's how dirt simple it is to provide infinite testing builds for different branches.
Well, for .deb there is the dependency problem. Even if your deb is just a release of a project, you can only use versions of dependencies that are in debian repository. Or include it all in your .deb, which becomes hard to maintain.
By "infinite branches" you probably mean that a nix derivation / flake is a recipe that can build any version of the project?
Yeah, Nix flakes can be a static URL path, but they inherit your VCS refs directly, so
nix run github:tailscale/tailcat
Gets you the default branch, but it's trivially easy to specify an arbitrary branch or tag, so like if you're trying to get a colleague to check out your work it's just:
nix run github:tailscale/tailcat/my-fancy-branch
And obviously a flake can reference other flakes with the same flexibility, so your home manager config or your company's private app flakes can transparently pull the not-yet-merged version of a change you're proposing upstream without having to make special internal "patched" released with tortured 1.2.3~actually.1.4.6 type version numbers like in Debian. And of course that same flake reference policy applies all the way down, so you also don't need to worry about the private apt repo as a big piece of shared state that's going rev forward and break your older code branches or Dockerfiles, since each branch of your app flake has its own flake.nix and accompanying lockfile controlling which deps it is pinned to.
It really is an awfully nice system, and shockingly modern in its sensibilities considering how old the Nix fundamentals are.
When I was writing rust stuff a few years ago it seemed like nix was the hip cool thing all the up and coming rustaceans (mostly unemployed students trying to land faang jobs) were using. If you're not familiar with the rust community it's very different from other programming communities I've been in (python/go/ruby/etc). Almost felt like you'd be kicked out of their lunch table if you weren't using it, but I've only see it that ubiquitous there. The rust discords, for instance.
Always makes me chuckle when there's this uncountably large group of people, and someone meets a handful of them and then decides they can make broad, sweeping statements about the entire group, despite having not met 99.99...% of them XD
I suspect some of this goes the other way too, that the people who see the value in Nix and/or are willing to tolerate its idiosyncrasies in exchange for what is on offer there are are a lot of the same kinds of folks likely to take an interest in rust, ocaml, elixir/gleam, etc.
Loving this. Honestly I keep encouraging folks to just TRY native Nix for dev environments. So easy to set up and get value on day 1 that you'd be shocked. Also models/LLMs have made lightyear strides in the last few months to help with everything around it. We've been building up Nix based skill systems into our Flox skill and it's been super meaningful both on adoption and when we need to scale it up. (disclaimer I do Flox.dev which is Nix based and we do cool Nix stuff)
Just yesterday I was complaining that I wanted to SSH back to my homenet while at the office, on my office (not home) tailnet. I wrote something based on Iroh to do this, but it's one shot (ie not particularly generalized). Might be able to throw it away or redesign it with some inspiration from this! Thanks.
I was just wondering yesterday if it'd be possible to have a tailscale client, or multiple tailscale clients in their own network namespaces, that could connect to multiple networks.
The slightly tricky part would be managing a local NAT that blends the two networks, and also does MagicDNS to answer on those NATed IPs. If routes conflicted, I guess you'd have to pick one?
yeah that's kinda what it does. It sets up two tsnet servers connected to two different tailnets and exposes them through a shared TUN device. It does the magic of allocating safe IPs to each host so that they don't overlap and does some leasing so that IPs aren't accidentally reassigned. It also does the MagicDNS thing so that DNS returns the "effective IP".
I wanted it to work on macOS so couldn't rely on network namespaces.
This is so cool! I mean, we really wouldn't need it if we had 100% ipv6 (no cgnat), but this is the next best thing. I think people underestimate the innovation that could happen if we had trivial p2p.
Looks dope, though I'm surprised, is Tailscale fine with their DERP servers being used by non-customers like this? (Yes, it's a Tailscale project, but doesn't require login.) I vaguely assumed there was an auth step before you could use one as a relay. They mention a rate limit, but still
Tool like this exposes big problem with the current shape of the Internet, I believe.
Everything should be possible with just plain netcat and IP stack. Someone asks about NAT holepunching, encryption, static IDs, permissions, etc., yes this is what Internet lacks and why every P2P app reinvents it over and over again.
I've been making a mesh vpn like radmin vpn or hamachi that does not need a central authority server and most importantly, can run on all major OS thanks to wireguard-go.
How much Tailscale is this still if the transport is based on wireguard, the control plane is something new with the keys based on the wireguard keys? Genuinely confused.
I dug into it a bit because I had the same question. The network layer here is wrapping around the Tailscale daemon's magicsock, which is the thing that does all the DERPing and NAT hole punching. Tailcat builds sort of a fake control plane that does a one way key exchange over DERP (the Meow message type) between client and server, then both sides do the normal CallMeMaybe dance to connect.
Netbird is the more mature version of this: fully open-source, self-sufficient, works like a charm, supports reverse proxying out-of-the-box, and much more. No reason to ever return to the Tail family.
86 comments
[ 0.21 ms ] story [ 29.0 ms ] thread[1] https://tailscale.com/blog/how-nat-traversal-works [2] https://github.com/magic-wormhole/magic-wormhole
It's really pretty cool.
if that is goal then why not go 100% open source to eliminate the ts derp control as well and get full sovereignty?
like netbird, openziti, zerotier, etc.
the control plane is closed source, but headscale is an open source alternative that we embrace and encourage people to use if it meets their needs/desires
Half of the Software offering nowadays seems to be selling vendor lock-in at no added value and then making a profit.
Sure there's always some negligible added value, and then they reinvent a whole stack for their ecosystem. In the case of tailscale the added value seems to be avoiding going into your router and activating port forwarding? It looks like negative added value to me. Someone that tells you "don't do NAT traversal, just open a port" would be highly valuable, but saying no doesn't seem to be trendy, and is certainly not an easy sell.
Well, ignoring the tone... kinda yeah? I mean, if port forwarding works for you, and you have public IPv4, and you have no CGNAT, and you need a connection only between 2 hosts, then port forwarding is actually a half-decent idea. However, it's not what Tailscale excels at.
> port forwarding is actually a half-decent idea
You make it sound like both options are equally valid, and as if port-forwarding were some esoteric technology. It's not, one is good, the other is bad.
5 years ago I had a similarly bland position, live and let live, I'd go to a client that was using a Ddns (zapto.org) for production loads serving thousands of customers. 5 Years later, I see them constantly facing SSL issues, and network issues, and even if they weren't it's still the wrong way to do things.
We have to stand for something, get an IP address, get a domain. If you don't have either of those, raise funds, get a budget, speak to people, you can't tech your way out of some things, I know we are hackers, but sometimes the solution is not hacking. Yes we should develop the power to do things even without approval from institutions and without collaboration from the central NIC, or the domain holder, etc... But we should refrain from using that power and actually integrate with domain holders, actually get IPv4/IPv6 blocks.
There's many reasons why you would use a ddns or similar tech, lack of budget, lack of main domain holder approval, lack of NIC approval, lack of ISP approval. In all cases, you are way better off by actually going the other way instead of using a ddns.
1- no budget: Get a budget, a domain is like 15$/yr. a 1024 IPv4 block leased from a NIC is like 600-2500$/year, so the cost of a Host or ISP is 0.6$ to 2.5$, maybe they charge you a margin and you end up paying 5$ to 50$/yr? If this is an issue, maybe your project should die, how much is your time worth? Fund your projects accordingly, or let them die. If this is someone elses project, they cannot afford you.
2- ISP: Call your ISP, if they can't help you access your router, or port forward or whatever, hang up, call again. If your solution is to use a ddns instead of picking up the phone, you have overtrained your hard tech skills and need to work on your soft skills. Speak to a human, you won't get very far if you never speak to humans. This is not a niche thing, we call it port-forwarding but it's how TCP and IP work in general, this is the stuff of the internet, if you can't do something as basic as that, you are never gonna make it, sorry.
3- Domain holder: If you work for a client, and they have a domain, but you don't know who holds it, or where the passwords are. Don't jump to a ddns or buying another domain, integrate, investigate, do an inventory. At the very least buy another domain, a ddns is just cheap to the point of being unprofessional, see 1.
There's a 4th excuse that I'll handle as well:
4- I'm just testing, this is just a throwaway project: Even if it is, you are still underfunding your bets if you don't even have a domain and standalone non-firewalled IP, even if reusing an existing domain and host, we are talking about like 5$ per month. Imagine a professional of any class that cannot afford a tool worth 5$, even a plumber has a toolkit with a screwdriver in it.
Even if you share a link to your project to a friend, if it says hsabhsabdsa.tailscale.net or bjsdbsajdbsa.vercel.app , you are already neutering your project. You can skip this 'phase', it's not necessary for a project to go through all phases, especially if a vendor is trying to convinve you that you should go through this phase, skip it, tell them no, build it right even for your day 1 demo, it's better than having a pre-day 1 demo that has low quality and does things wrong.
Recently: https://github.com/golang/go/commit/128a36cf0367c46daff2528d...
https://github.com/golang/go/commit/3c0665e551be23e62167f625...
https://github.com/richlegrand/bitbang-cli
https://github.com/coder/wush
https://github.com/n0-computer/iroh
https://github.com/n0-computer/dumbpipe
https://github.com/cablehead/pai-sho
I'm glad there's a competing Rust and Go implementation now. I like Iroh as a Rust user and having tailcat around doesn't hurt me
May be my idea of simpler cryptography is incorrect, but PAKEs like CPace do seem simpler than public key primitives in Noise.
And it might be because it is easy to provide and maintain compared to a .deb package.
By "infinite branches" you probably mean that a nix derivation / flake is a recipe that can build any version of the project?
It really is an awfully nice system, and shockingly modern in its sensibilities considering how old the Nix fundamentals are.
The slightly tricky part would be managing a local NAT that blends the two networks, and also does MagicDNS to answer on those NATed IPs. If routes conflicted, I guess you'd have to pick one?
Seems very doable though.
I wanted it to work on macOS so couldn't rely on network namespaces.
edit:
Which are probably tailscale's, as it's under tailscale/ github org?
But these are a separate fleet (https://tailcat.dev/derpmap.json) separate from our usual ones, and not using "tailscale.com" in DNS or SNI anywhere.
Everything should be possible with just plain netcat and IP stack. Someone asks about NAT holepunching, encryption, static IDs, permissions, etc., yes this is what Internet lacks and why every P2P app reinvents it over and over again.
(just a cute demo, not intended for release or ongoing maintenance)
I've been making a mesh vpn like radmin vpn or hamachi that does not need a central authority server and most importantly, can run on all major OS thanks to wireguard-go.
What is a data plane? What is a control plane? I don’t genuinely understand what these words mean :(
I bet I could simplify out the WebSocket-to-UDP proxy by leaning into tailcat's dataplane and just speak straight UDP.
The only thing that convinces me that tailscale isn't a CIA op is how badly it works for real uses