Accidentally wiring everything to everything else sounds kind of scary.
There's 1 or 2 things I wouldn't mind securely exposing to the internet (like Plex) but nothing I need so desperately while I'm out and about that I'd even want to take that risk.
Speaking of SSH, Tailscale has special support for it whereby it handles any incoming connection to port 22 from the Tailscale network, and deals with authentication itself. No public keys or passwords: if you’re logged into Tailscale you can be logged into the machine. This is particularly handy when you SSH from a phone, as proper credential management is a bit of a nightmare there.
this has me worried. i would not want that. i use zerotier, not tailscale, but the principle is the same. i have my laptops and my phone connected to my servers. given that all of those machines are already on the internet, connecting them into a virtual network does not add any risk in my opinion. (at least as long as you don't use features like the above). all i get is a known ip address for all my devices, with the ability to connect to them if they have an ssh server running. when i am outside the primary benefit is that i can tell which devices are online.
this is for teams where you don't want to create passwords or keep track of ssh keys for everyone by hand. it greatly simplified our server usage as we can simply ssh user@machine and it just works. you can create access controls for it as well.
I use a similar setup, but for anyone following this guide i would not recommend hosting your custom oidc server behind the same tailnet it authorizes.
Any configuration issues will lock you out entirely and you will need to have tailscale support re-enable an oauth provider and its not reversible.
I use an oauth provider to log in to tailscale and keycloak internally as an oidc provider for service to service auth.
I tried using `tailscale funnel` against a dummy server `python -m http.server`, and within 10 seconds the bots started to check for vulnerabilities.
Tailscale warns you about how enabling it will issue an HTTPS certificate which will be in a public ledger. But I wasn't expecting it to be this quick.
> It’s a subscription product, but it has an insanely generous free tier that covers basically anything you’d ever want to do as an individual.
Tailscale do have a very nice product, but privacy-conscious users should be aware that you must disable Tailscale's real-time remote collection of your behavior on your “private” network. See KB1011: https://tailscale.com/kb/1011/log-mesh-traffic
“Each Tailscale agent in your distributed network streams its logs to a central log server (at log.tailscale.io). This includes real-time events for open and close events for every inter-machine connection (TCP or UDP) on your network.”
For an example of how invasive this is for the average user, this person discovered Tailscale trying to collect ~18000 data points per week about their network usage based on the number of blocked DNS requests for `log.tailscale.com`: https://github.com/tailscale/tailscale/issues/15326
“When you use the Tailscale Solution, we collect limited metadata regarding your device used to access the Tailscale Solution, such as: the device name; relevant operating system type; host name; IP address; cryptographic public key; user agent (where applicable); language settings; date and time of access to the Tailscale Solution; logs describing connections and containing statistics about data sent to and from other devices (“Inter-Node Traffic Logs”); and version of the Tailscale Solution installed.” (emphasis mine)
Anyway, the reason I quoted that part of your post is because Tailscale are using some Fear, Uncertainty, and Doubt tactics here by naming the privacy-preserving option “no-support”, and if you are a free user then you aren't getting support from them anyway, so there should be no downside to keeping your private network private :)
I think I "get" what tailscale is about, what I don't get is how much of it is re-implemented and available out of the box in headscale. I already do most of the things mentioned in the article (from hand-rolled WG, Apache and firewall configurations), so this level of centralised automation and orchestration has some appeal, but I'm not willing to hand over the keys to my entire network to them and would rather keep things in-house.
And on the topic of headscale, some people bring up netbird as an alternative. Netbird gets some immediate sympathy from me as they put lots of emphasis opensource and self-hosted, but I'd be curious to see how they compare for the use-cases described in the article.
I don't know enough about networking as I should, so to plug for my gap in knowledge, I generally prefer to use more comprehensible (to me) forms of security. And a feature like this:
> Speaking of SSH, Tailscale has special support for it whereby it handles any incoming connection to port 22 from the Tailscale network, and deals with authentication itself. No public keys or passwords: if you’re logged into Tailscale you can be logged into the machine.
kinda worries me (given also IP spoofing is possible?), compared to SSH keys whose mechanism is more obvious and thus easier to trust.
I definitely like the idea of Tailscale as an extra layer of protection, but I'm not sure I'd loosen existing protections while using it, whereas many Tailscale articles often present it as a panacea for internal-network-over-the-internet security. Are my concerns misplaced?
One useful additional aspect to Tailscale that I've not seen mentioned so far, is the integration with Mullvad.
Using that you can get the benefit of their network servers, appearing just as standard Tailscale exit nodes, which is handy if you need to geo-shift traffic at all.
For me, the benefit was simple: I blocked off ssh and Postgres on my production server from the public internet, making it such that these are only accessible from my development machine and other servers. Yet from inside the tailnet I can still talk to Postgres as if it was on LAN. I’m not sure of any easy way to do this without Tailscale.
> For a long time I didn’t bother with any kind of ACLs within my Tailscale network. […] Then one day a thought hit me. […] That means anyone with access to any of my machines (or who managed to get Tailscale credentials out of one of my apps) would be able to SSH into anywhere else on my network.
I'm a happy Tailscale user but I'll keep saying this whenever Tailscale comes up: We need a way to `tailnet lock` (sign) not just the tailnet nodes but also the tailnet config (ACLs). Otherwise the above scenario of an attacker taking over the entire network is still possible even if you set all ACLs correctly. All it takes is for an attacker to take over the coordination server (to manipulate ACLs) and a single tailnet node. (Which, if you run Headscale, might even be the same machine.)
Until this is fixed I am not going to trust Tailscale with authenticating connections too much and will trade in convenience for defense in depth.
My primary use of tailscale is as a VPN. I have exit VPSes in 4 different countries, so usually at least one of them is not blocked from or to where I am trying to connect.
22 comments
[ 2.6 ms ] story [ 54.6 ms ] threadAccidentally wiring everything to everything else sounds kind of scary.
There's 1 or 2 things I wouldn't mind securely exposing to the internet (like Plex) but nothing I need so desperately while I'm out and about that I'd even want to take that risk.
Sounds like this is just for self-hosting?
this has me worried. i would not want that. i use zerotier, not tailscale, but the principle is the same. i have my laptops and my phone connected to my servers. given that all of those machines are already on the internet, connecting them into a virtual network does not add any risk in my opinion. (at least as long as you don't use features like the above). all i get is a known ip address for all my devices, with the ability to connect to them if they have an ssh server running. when i am outside the primary benefit is that i can tell which devices are online.
https://github.com/jaxxstorm/tgate
Any configuration issues will lock you out entirely and you will need to have tailscale support re-enable an oauth provider and its not reversible.
I use an oauth provider to log in to tailscale and keycloak internally as an oidc provider for service to service auth.
I have a similar set-up, without authentication however, relying on Nebula! https://github.com/slackhq/nebula
Tailscale warns you about how enabling it will issue an HTTPS certificate which will be in a public ledger. But I wasn't expecting it to be this quick.
Tailscale do have a very nice product, but privacy-conscious users should be aware that you must disable Tailscale's real-time remote collection of your behavior on your “private” network. See KB1011: https://tailscale.com/kb/1011/log-mesh-traffic
“Each Tailscale agent in your distributed network streams its logs to a central log server (at log.tailscale.io). This includes real-time events for open and close events for every inter-machine connection (TCP or UDP) on your network.”
It's possible to opt out of this spying on Unix/Windows/Mac clients by starting Tailscale with `--no-logs-no-support` or `TS_NO_LOGS_NO_SUPPORT=true` environment variable (see https://tailscale.com/kb/1011/log-mesh-traffic#opting-out-of...), but it is not currently possible to opt out in the Android/iOS clients: https://github.com/tailscale/tailscale/issues/13174
For an example of how invasive this is for the average user, this person discovered Tailscale trying to collect ~18000 data points per week about their network usage based on the number of blocked DNS requests for `log.tailscale.com`: https://github.com/tailscale/tailscale/issues/15326
Also see their privacy policy: https://tailscale.com/privacy-policy#information-we-collect-...
“When you use the Tailscale Solution, we collect limited metadata regarding your device used to access the Tailscale Solution, such as: the device name; relevant operating system type; host name; IP address; cryptographic public key; user agent (where applicable); language settings; date and time of access to the Tailscale Solution; logs describing connections and containing statistics about data sent to and from other devices (“Inter-Node Traffic Logs”); and version of the Tailscale Solution installed.” (emphasis mine)
Anyway, the reason I quoted that part of your post is because Tailscale are using some Fear, Uncertainty, and Doubt tactics here by naming the privacy-preserving option “no-support”, and if you are a free user then you aren't getting support from them anyway, so there should be no downside to keeping your private network private :)
And on the topic of headscale, some people bring up netbird as an alternative. Netbird gets some immediate sympathy from me as they put lots of emphasis opensource and self-hosted, but I'd be curious to see how they compare for the use-cases described in the article.
> Speaking of SSH, Tailscale has special support for it whereby it handles any incoming connection to port 22 from the Tailscale network, and deals with authentication itself. No public keys or passwords: if you’re logged into Tailscale you can be logged into the machine.
kinda worries me (given also IP spoofing is possible?), compared to SSH keys whose mechanism is more obvious and thus easier to trust.
I definitely like the idea of Tailscale as an extra layer of protection, but I'm not sure I'd loosen existing protections while using it, whereas many Tailscale articles often present it as a panacea for internal-network-over-the-internet security. Are my concerns misplaced?
Using that you can get the benefit of their network servers, appearing just as standard Tailscale exit nodes, which is handy if you need to geo-shift traffic at all.
I'm a happy Tailscale user but I'll keep saying this whenever Tailscale comes up: We need a way to `tailnet lock` (sign) not just the tailnet nodes but also the tailnet config (ACLs). Otherwise the above scenario of an attacker taking over the entire network is still possible even if you set all ACLs correctly. All it takes is for an attacker to take over the coordination server (to manipulate ACLs) and a single tailnet node. (Which, if you run Headscale, might even be the same machine.)
Until this is fixed I am not going to trust Tailscale with authenticating connections too much and will trade in convenience for defense in depth.