HN Explorer

Show HN: Weron – A Peer-to-Peer VPN Based on WebRTC Written in Go (github.com)

204 points by pojntfx ↗ HN
Hey HN! I just released weron, a P2P VPN that uses WebRTC for transport which I've been working on for the last couple of months. It can create both layer 2/Ethernet and layer 3/IP overlay networks, and the underlying transport layer can be easily embedded to write your own P2P apps with Go. Compared to for example Tailscale, WireGuard and ZeroTier, its much harder to block on a network level and also significantly easier to set up, while not sacrifing much performance.

I'd love to get your feedback :)

51 comments

[ 2.6 ms ] story [ 108 ms ] thread
saurik
Is what makes this a "P2P" VPN--as opposed to merely VPN software that you might choose to use between peers (such as Wireguard)--that it supports the group/"community" network behavior? When you do that, are you building a fully-connected overlay network--where everyone has a separate WebRTC link to each of the other participants--or are you building some kind of organized P2P network topology out of the participants?

(Note: I work on Orchid, a decentralized market for bandwidth built on Ethereum and WebRTC, which has a multi-hop-capable VPN client for various platforms available.)

pojntfx
> Is what makes this a "P2P" VPN [...] that it supports the group/"community" network behavior?

More or less! Compared to say OpenVPN there is no central server that all traffic is routed through (unless you choose to activate TURN of course, see the `--force-relay` flag), so hosting it is much cheaper as the only public part of the infrastructure is the part that exchanges the candidates ^^ Its pretty close to how n2n[1] does it, but has proper NAT hole punching support due to the WebRTC backend.

> When you do that, are you building a full-connected overlay network [...]

Yes! Its a full mesh; it seems to scale pretty well to a fairly large number of nodes, although I haven't tested it with more than ~100 peers in a single community yet. If the overhead of maintaining all of the connections is too much or peering between nodes in an issue, TURN can help to connect those peers with the rest of the mesh. Orchid looks interesting!

[1] https://github.com/ntop/n2n

saurik
Wait, is the signaling server "public infrastructure"? Are you then having users pre-set/share all their DTLS keys and ICE passwords (to prevent the obvious MITM)? I don't remember seeing that in your documentation (but I am not reading it extremely carefully or anything). (This isn't an issue for Orchid as the endpoints are expected to expose signaling ports for WHIP.)
pojntfx
Oh the signaling server is but a simple ICE candidate/offer/answer relay :) The messages being sent there are encrypted using a PSK so that the signaling server can't access the ICE candidates etc.
ackbar03
Does this also work between two peers that are behind NAT's / routers?

From what I vaguely remember when I tried playing around with this stuff, some NAT's / routers have some sort of port randomization thing so that the same ports don't always reach the desired location behind it.

Does WebRTC make that port connection persistent due to the streaming nature of it; is that the idea? Does the stream ever break and does it require getting ip and port information from the STUN and ICE servers again? If so, how often does that happen?

pojntfx
From my experience it works basically everywhere, but if it doesn't - just specify a TURN server on `--ice` and you'll be able to connect to a relay ^^ The ports/IP can actually change - I was traveling through Germany by train recently for example and whenever my public IP changed it signaled the updated candidates using trickle ICE :)
[deleted]
(comment deleted)
1vuio0pswjnm7
Here is some initial feedback: Nice selection of static binaries!

However the choice to hardcode Google as a default STUN server is off-putting. IMHO, 2008's n2n would never set a third party server as a default. https://luca.ntop.org/n2n.pdf

pojntfx
Thanks for your feedback! Is there a publicly hosted STUN server that guarantees better privacy? I'd love to use it if there is an alternative. It is of course possible to use any other STUN/TURN server by changing the `--ice` flag and host your own with i.e. `coturn` ^^
jewel
Two users don't need to use the same STUN server in order for udp hole punching to worj, so choosing randomly from a list might be a good way to go.
1vuio0pswjnm7
One of the beautiful things about n2n was that the supernode program is relatively small and simple. (n2n does not require the use of STUN or TURN.) There is no need for a third party to be the one running the supernode/STUN/TURN server software. Users can run it themselves.

Just because home users must involve commercial third parties, e.g., ISP, cloud hosting, etc., in order to listen on globally reachable IP addresses does not also mean that users must also let those third parties have control over the software users run from those addresses.

tptacek
Why wouldn't you just tunnel WireGuard over WebRTC? There's already a very, very good implementation of WireGuard --- one of the two major production implementations --- in Go. We tunnel it over WebSockets, and it took like an hour to write that code.
saurik
That seems like it would offer no advantages, but would result in all of worse performance, smaller remaining available packet size, and "more code".
pojntfx
You could totally do that! See https://github.com/pojntfx/weron#8-write-your-own-protocol-w..., but I agree with saurik - if DTLS is secure enough for your usecase, that would just add overhead ...
tptacek
Yep, that's a good point. I jumped into the code (I just wanted to see what the encryption transform was) and should have zoomed out and looked more carefully at the whole design. Thanks!
VWWHFSfQ
So that's what, 4 layers of encapsulation now? Yikes.
braginini
that is indeed a performance decrease. But the idea of this project is just great!

There is another way of combining WebRTC and Wireguard.

Netbird uses ICE protocol (used in WebRTC) for NAT traversal and then reuses that punched holes to establish a direct Wireguard tunnel.

https://github.com/netbirdio/netbird

P.S. I'm author of Netbird (formerly know as Wiretrustee)

Sean-Der
This is so cool! I love the embedded aspect of it, that was a big goal of Pion. Really exciting for users to not have to setup/configure a service to get connectivity going.

re: concerns about STUN I would love to experiment with adding NAT-PMP/PCP support to pion/ice. STUN server usage seems to be a major sticking point for people. Talked about it a bit here [0]. I am all for doing it, just would like to know the code is actually wanted before writing it :)

[0] https://mobile.twitter.com/_pion/status/1522365018855165952

pojntfx
Thanks for your work on Pion! I really enjoyed working with it, even in other places (STFS[1], a file system for tape drives that I wrote in January, uses it to access remote servers behind NAT for example). NAT-PMP/PCP support would be awesome!

[1] https://github.com/pojntfx/stfs

ianlevesque
> Compared to for example Tailscale, WireGuard and ZeroTier, its much harder to block on a network level

Tailscale at least bundles in an HTTPS fallback for those situations. I’m not sure that claim holds up.

pojntfx
A HTTPS fallback is more or less what weron provides - if you want to block the "weron protocol", you either have to block access to the signaling service (which runs on 443, and its just WebSockets) or block the related protocols, but since Zoom etc. use the same protocols that would probably be pretty hard to implement without disrupting everyday operations.
linsomniac
This looks pretty slick, and the WebRTC seems like it might offer some choices of traversal that would work where others don't. The inbuilt tools for "chat" (seems a useful test), throughput and latency also seem nice. I've been running into some weirdnesses at times with Nebula and Zero Tier where it's unclear where my latency or throughput limits are coming from.

I'm looking forward to doing a lot more with overlay networks in the short term, I've been playing with a variety of them over the last 18 months or so and just need to pick something and start deploying it.

pojntfx
Thank you! From my experience if you can get say Zoom or Meet to work in the network you're trying to deploy weron in, you'll get weron working as well. And if the traversal still doesn't work - just specify a TURN server on the `--ice` flag and it will definitely work through relay ^^
a-dub
very cool! was recently musing about how the next frontier in vpns is steganography, and here we are.

are the webrtc streams running at a constant bitrate or can the streams be detected by doing traffic timing analysis?

(i once looked at adding constant bitrate support to wireguard but ultimately abandoned it as it was looking to be excessively complicated)

pojntfx
Thanks! Its using the Pion library[1] which doesn't seem to be using a constant bitrate; that would definitely be an interesting addition though. Sending e.g. zeroes when its not transmitting packets/frames is totally doable, but atm not implemented.

[1] https://github.com/pion/webrtc

a-dub
the other big upshot would be that it would never leak anything in the flow characteristics of the stream.

i suspect that it might be possible to identify major websites being loaded by simply analyzing packet sizes and timings, but maybe not in http/2 world.

saurik
This is the kind of thing I would expect to see soon from Orchid, if that's interesting, which isn't "P2P" in this sense of group-networking but that doesn't seem to be your specific interest. (I am in charge of technology at Orchid.)
Sean-Der
The constant bitrate idea is really interesting, I think we could do it! I wonder if we can publish a constant rate at the public API of if it would take some interaction with the Congestion Controller in SCTP.
a-dub
i agree! it's been percolating for a very long time in my head. i was envisioning a paper that shows how traffic analysis can be used to identify activities in the encrypted stream and then a cbr patch to ameliorate it.

where it gets complicated is avoiding creation of new side channels. you need a very stable clock for pumping out the data and as you hint at, congestion control to avoid latency by actively adjusting the bitrate.

i was looking at doing this in the kernel implementation of wireguard, but those parts of the kernel didn't seem a good fit for this kind of time sensitive constant spinning algorithm.

if anyone is interested in this, would love to collaborate.

depingus
This looks nice. Do you have Android/iOS mobile apps on the roadmap?
pojntfx
Thanks ^^ Android/iOS is not _yet_ on the roadmap (although weron works on the latter if your phone is rooted, since TUN/TAP is available natively) but totally possible! Both systems provide a L3 VPN API, and since Gomobile can generate bindings to the networking layer for Java/Swift that is definitely something I want to work on in the future ^^
Siira
Having a docker image that sets up a VPN server with ideally zero config, plus short instructions for setting up the clients in popular OSes would go a long way to help adoption.
1vuio0pswjnm7
No one is raising the question of how to handle community name/key management for persistent communities. With n2n, which long predates Zerotier, Wireguard and Tailscale, one of the authors thought there needed to be automatic key rotation and also communication between "signal" servers. Original n2n allowed tunneling over HTTP as a fallback. It should be noted that n2n was never meant to be used for large communities, say, over 100 peers. Sure enough, the author of weron is issuing that same warning here.

Many years ago I had each peer run their own smtpd listening on the n2n TAP interface. It was peer-to-peer, encrypted email. This can be done for many other protocols, too. With Layer 2 overlay, there are no middlemen. Direct connectivity to friends and family with no need for Google/Facebook.

Layer 2 overlay networks with n2n can be significantly faster if automatic encryption is turned off. In that case, the user can encrypt and sign sensitive files with a separate program before sending them through the TAP interface. It works quite well.

pojntfx
Hi! n2n was a huge inspiration for the project, in fact I wrote Go bindings for n2n before starting it: https://github.com/pojntfx/gon2n

Weron also allows tunneling - just specify a TURN server on `--ice` and enable `--force-relay`, in which case it will probably scale to well over 100 nodes ^^ The signaling servers are fully horizontally scalable, so that you can benefit from a faster backbone on a global scale scenario - Redis coordinates messages, kicks etc. between them and a Postgres database maintains central state, such as credentials for persistent communities and client counts.

In terms of key rotation & encryption - weron heavily depends on DTLS as provided by Pion/WebRTC and thus inherits similar security properties. It is not possible to disable encryption of WebRTC, but tbf the performance benefits of using plain SCTP don't seem to be worth it as the RTT latency and resulting decrease in throughput[1] is the dominant performance bottleneck.

[1] https://tuhat.helsinki.fi/ws/portalfiles/portal/167373638/Es...

IceWreck
Hey, I have been using Wireguard and Nebula to connect to my home network for a while now, but theyre blocked by my uni. I bypass the blocking using shadowsocks but its still a hassle.

I'll test out Weron and hopefully I dont need to tunnel it through shadowsocks.

cycomanic
A side note, but a university network blocking VPNs, that's ridiculous! Uni IT should really be hit over the head with a clue bat. I mean what is the motivation for this?
pojntfx
That is basically the usecase for weron, good luck ^^ Networks blocking anything but HTTP has been one of the reasons why I started writing this (train WLAN in Spain does it for example)
mib32
I somehow can't understand how to use it :) Sorry! But nice work.
braginini
Well done!

Disclaimer: I'm author of netbirdio - a similar project.

Using WebRTC makes it possible to have a tunnel from the browser directly. Combined with a virtual terminal it just makes it even cooler!

@pojntfx what do you think about that? Feasible with your implementation?

I've been playing with such functionality in Netbird but since it depends on Wireguard it is a bit tricky :)

Would love to have your feedback as well on our project

https://github.com/netbirdio/netbird

PLG88
Wow, some awesome work here. I would be interested to know your thoughts on OpenZiti (https://openziti.github.io/). Its an open source project that allows you to embed private connectivity into your app using one of the many SDKs together with strong identity allowing outbound only connections. It is a mesh overlay similar to TURN but uses a concept of smart routing to normally reduce latency through circumvention of BGP. Various superpowers can be seen here - https://www.youtube.com/playlist?list=PLMUj_5fklasKF1oisSSuL...