I think a better option would be to create a Wireguard tunnel between Raspberry Pi and the remote server instead of a SSH Tunnel. Then there is no need to add or change ports and restart the tunnel for every service.
OP has literally written in the gist about exploring a way to map entire port range and avoiding doing this, so the non hacky way of doing this is setting up something like a wireguard tunnel. That's the reason I suggested doing this instead of a tunnel which has other disadvantages like doing TCP on TCP.
while i think this is true (any other vpn software would work too though) i want to point out that you actually can bring up a tun interface using ssh with the "-w local_tun[:remote_tun]" flag somewhat easily if you want to. It is also possible to make forwarding work in either direction using an integrated socks proxy using "-R" or "-D" flags ...
Is your plan to run some sort of IOT application on the PI?
I'm a little confused as to why you would go through the trouble of running a reverse proxy on a remote server that is fully capable of hosting your web applications and or just setting up a port forwarding rule on your local wifi router to redirect 10080 to 80.
I do a similar thing so I don't have to store sensitive data on a VPS. The sensitive data is used to compute non-sensitive results that I'm happy to go via a VPS.
Is exposing one's residential IP address such a bad thing? Is the threat model just that someone might wish to target you individually, and that they'll have better luck finding a vuln into your LAN if your services are listening directly on public ports?
I'd think that using a remote VPN is a better/broader solution to that particular problem. People don't often self-host things at the home, so it's unlikely anyone out there would see my DNS record and assume that it's of value and a useful tool for correlating my IP with my identity (and this actor would need to have eyes on a lot of different services' server logs in order to have data to _maybe_ correlate with me in the first place).
I bet there are common non default port numbers that people subconsciously tend to use, it might be an interesting project to create a list file similar to a passwd list file to make a port scanner more efficient. GOD = 12345!? 1337
I have a similar setup but instead of SSH Tunnels i use wireguard.
In my case some services are not routed to the public internet and only accessible from within my privat wireguard network. E.g my smart home hub. I really like the flexibility to decide whats available for who.
I've started playing around with Tailscale [1], which is based on WireGuard. Free to use for a single user, and I've tried it out on Raspberry Pi, Linux x86_64, Windows and Android. Seems to work well, except there appears to be some issue with accessing Windows via the secure IP (neither ping nor Remote Desktop appear to get through the Windows firewall, despite having a specific firewall rule for Tailscale) [2]. Can SSH into e.g. Linux hosts using PuTTY from Windows without issues.
the idea is that your hetzner server acts as a gateway and you can communicate with your pi through the virtual private network, therefore removing any need to access through public ip.
I did the same. It works really well for hole-punching through my stupid Comcast router! And I anticipate it will continue to work really well as I add more devices on my parent's, sibings', etc networks. It's cool to not have to think at all about where my devices are, since they're just all directly accessible through their dedicated IP on the wireguard VPN.
I also have my phone, laptop, etc all on the wireguard VPN, so I can ssh to anything from anywhere. I also have my laptop open to password-based ssh only from my wireguard VPN, so I can ssh (via termux) into my laptop, which has been extremely convenient in addition to being just plain cool. I can't speak highly enough of wireguard!
I'm going to give this a try for personal use. For larger scale management alternative to ngrok I've been eyeing Shellhub for it's self-hosted capabilities and dashboard.
Caddy underwent a complete rewrite that also invalidated most of the existing config. Personally I don't see any benefits in using caddy or another non mainstream httpd as it adds more risk to my project with usually very little benefits. Nothing against using it for hobby projects but in production I've migrated back to nginx.
You should definitely take a look at it - Caddy works fantastically well.
To give you an idea about the compatibility differences, it's like the upgrade from Vue.js 2 to Vue.js 3. Some changes to format are necessary for the new architecture. But you can always just run the older version (that's what I do). And swapping to the new one isn't terribly painful if your just using vanilla (for Caddy 1 extensions, which weren't created by the developer himself most of the time, I'm sure Caddy 2 versions will pop up over time).
Personally I prefer software that improves (slowly) over time, for at least a couple iterations. It's a balancing act to be sure, and there are diminishing returns to changing things up, but I think very little software comes out of the gate in a >80% optimal form.
I haven't found the changes in Caddy's config format to be problematic in the least.
I mean, in 90+% of cases, you just put in your Caddyfile
*host* {
reverse_proxy 0.0.0.0:*port*
}
and you get a fast reverse proxy that just works, with letsencrypt enabled by default to boot, nothing to worry about.
However, if you find yourself against a service that requires a slightly more advanced configuration, good luck making sense of their opaque configuration, especially since they now went v2 so everything old is out of the window, so if you search for something, chances are it's no good anymore.
If you are on AT&T u-verse you can just assign a part of your IPv6 subnet (which appears to be static) to your RPi and disable the firewall to that host. Then you can accept incoming connections on ports 80, 22 and even 25 (for 25 I had to send a chat message in my AT&t account and it was opened the same day). Then you can have a full IPv6 web server at no additional cost to the price of your u-verse home internet.
One downside is the actual steps are going to be different for every ISP/router combination. Tunneling is a universal solution, and provides other interesting tradeoffs.
When you're tunneling VPS to a home connection and rpi, the performance of the rev-proxy is not going to be a concern. Anything including a trivial script implementation of the proxy will be able to saturate this setup.
The answer is no, if it wasn’t obvious I’ll make an edit there. Inlets is free and OSS, inlets PRO is not free and optional. You can decide what works for you. In any case it’s much simpler than said hacks with Caddy and ssh
The solution in TFA works behind NAT and routers/firewalls, and has the added benefit of encrypting all of the traffic between the two machines thanks to SSH.
I’m not sure people who aren’t network administrators should be putting public IPv6 all over their LAN behind their router, at least not until v6 is more common and routers have better inbound security features for such things (like how NAT functions today for v4).
NAT like security for IPv6 is just dumb. We should always configure firewall for v4 and V6. NAT is not for security!! It has different purpose which is now being taken care by V6 and NAT should just die.
Yes, BUT: for many many years, home networks have relied primarily on NAT for security. The firewall was only ever touched when doing something advanced and even people who were doing things like setting up home servers and such rarely had to deal with it.
Every single tutorial on the topic is IPv4-specific and until IPv6 isn't more common, people will not write guides and tutorials with IPv6 in mind. So until the knowledge of proper security on open networks is common enough among home tinkerers, we should not be recommending people just throw everything straight onto the open Internet.
i don't think the typical NAT setup differs too much from a typical filtering setup. That is, only allow return traffic from public interfaces. While that is kinda implicit using NAT the change to being explicit is trivial.
somewhat simplified like this using iptables on linux:
iptables -t nat -A POSTROUTING -o WAN -j MASQUERADE
vs
iptables -A FORWARD -i WAN -m state ! --state RELATED,ESTABLISHED -j REJECT
An esoteric alternative is to host a Tor hidden service.
Tor hidden services don't require you open ports, and you don't need a static IP address.
Your residential IP is protected by the onion router, and the .onion domain name is free!
I mean, you’re technically correct, but isn’t this kind of like saying that the .local TLD is free of charge? It’s “free” because it’s not a part of the “normal” internet.
But .local aren't resolved by any public DNS server, so when outside your bubble you won't have them pointing where you expect while Tor names resolve whenever you can get connected to Tor Network
.oarsinsync is probably a better comparison. It’s not explicitly scoped anywhere, and it works nicely in my oarsinsync bubble, much like .onion works nicely in the ToR bubble.
In both cases, you need to connect using some kind of tunnelling protocol to connect into another network.
What I’m trying to say is that if you’re going to have to install additional software, or make additional configuration on the requesting node, it doesn’t really matter if it’s .local or .tor or whatever, because the only reason it works is because you did that extra work: and when you’re in that situation, it doesn’t really make sense to compare it to “paid” domains because the reason you pay for a domain is just that, you want it to work everywhere without additional configuration.
For me, all my personal machines have TOR installed anyway and I mostly want public address for gluing things together or being able to access things remotely. It's definitely not usable for everything you normally use domain names for (like showing people the menu at a restaurant.)
I will point out that you don't need to do any extra work to access web servers on TOR at least. I think cloudfront (and I know others) run a proxy that lets you access them with just a normal URL via a web browser. Most OSes need no extra configuration for .local.
This is actually a cool idea and an example of thinking outside the box.
You run a Tor proxy (bind to 127.0.0.1:9050) on a server on the Internet, and then run a Tor hidden service inside your LAN.
Then, you acquire an .onion domain for the hidden service (running inside LAN), and configure the Tor proxy (running on a server on the Internet) to access the hidden service via the .onion domain (for example, use Nginx to reverse proxy the hidden service).
This should work. However, I imagine it could be quite a bit slower compare to "normal access", and requires extra steps to secure the transport connection between the hidden service and the proxy as you might not want to have everybody on Tor to be able to access that hidden service.
I was more thinking of directly accessing the hidden service through the tor network (without a proxy on a VPS), which is acceptable if you're the only user.
This is what I've always done when I was a poor student and didn't have a public IP.
The latency can be painful but it's way better than nothing.
I will say you should probably be careful about running a webserver on TOR. Almost all my machines have a basic web server just for copying static files to windows hosts I don't want to give my password to. People would find these on TOR and then find my resume, I got plenty of emails asking to set up "hidden sites on the dark web for sharing pictures."
Then I would have to generate certificates and leave them on random people's machines. Plus when I'm sharing things with other people I can't just send them a link, I have to explain how to install the certificate into their browser (or OS in some cases.) On top of all that openssl is very complex and has had serious CVEs before, I'd feel a lot less comfortable exposing that to TOR than a smaller web server.
I don't mean HTTP client cert in the web server/browser, I mean the TOR built-in option that you enable with `HiddenServiceAuthorizeClient stealth list,of,client,names`. Granted, this does mean that you have to distribute keys (but you were already distributing an onion address) and there's more setup complexity (although you already have to run TOR, so we're not in "normal user" territory anyways).
I install a reverse proxy on every remote machine (raspberry pi or odroid) I manage. These machines are usually tucked behind someone's router, and it is hard to tell every router owner to forward the ssh port. When I install a reverse proxy, the machines can proactively tunnel right into a beacon machine whenever they are online and I can ssh into them from the beacon machine directly. I usually set up frpc as a service on the remote machine. It is also possible to use autossh with a monitoring port for this, but I noticed that frpc can bypass firewall restrictions much easier. This scheme also lets me use tools like Cockpit to view the health and status of my remotes.
That's how I understand it. Basically you have a machine hosted somewhere with a dedicated IP so you can access all remote machines from anywhere at any time, as long as they are connected to it via reverse proxy.
It does require the beacon machine to be up, but the system easily recovers from a reboot of the beacon, since frpc will keep on trying to reconnect. It is a poor man's firewall-resilient one-way vpn.
> Its based on wireguard and seems even better performance wise.
I hadn't considered that, but it makes sense. Since ZeroTier is layer 2, I'd expect it to become a leaky abstraction (at least for performance) in certain cases where algorithms are expecting other layer 2 hardware to be essentially directly connected with a wire when in reality they're connected over the internet.
What algorithms you have in mind? That would be a bad assumption, since layer 2 can be ethernet, but it may be i2c, or slip, or spacewire... Layer 2 does not mean "close" or "fast".
At Narration Box we use Tailscale with Caddy's Cloudflare Let's Encrypt DNS-01 for internal DNS resolution + TLS for the apps that need it. In essence you get something like Google's internal "go/ links" except everything's hosted on the external DNS server and automatically protected by default by Tailscale since the IPs pointed to cannot be accessed publicly.
Does ZeroTier let you expose a specific machine publicly? ie if I have a raspberry pi behind a NAT, will ZeroTier provide me with a public IP anyone can use to connect to it?
No, you still need to find a machine with a public IP. The usecase in parent comment mentioned only the SSH part, which is easier. But if you want to do forwarding from a specific machine to the public, you can do it manually.
What next? Well I setup some handy shell functions:
1) copy the latest screenshot into webroot/screenshots/${randstr}.png and copy the url to clipboard. this replaces dropbox's cool screenshot sharing feature. love sharing screenshots with coworkers this way, the files stays with you, you can redact easily.
2) serve static files the same way, if needed too. the magic's all in the shell scripts.
3) explore new open source webapps running on local docker.
Further?
Build a sort of like a devtools backend server, and a mobile app paired to it to do all sorts of cool stuff: run crons, get notifications for events, monitor servers / ssl expiry etc.
Limit's your imagination. But, security is also very important.
One can also use the Diode P2P network to get the same without a server: https://support.diode.io/article/ss32engxlq works for ssh remote access, websites and stuff similar to ngrok just without the strings attached.
I just set up a raspberry pi with ZeroTier. So far it seems like a really easy way to make the pi accessible whatever network it is in, and took minimal configuration to have it forward requests to any of the other devices on the internal network.
The only drawback is I don't think it would be possible to connect to a peer from something like heroku where you don't have full control over the server..
I use Cloudflare Argo Tunnel for this purpose and am very satisfied. It's easy to setup and if you have TLD on Cloudflare, the daemon(cloudflared) creates new A record for each host and creates a tunnel between your host and the nearest Cloudflare data center. You can also run multiple services on a single host. And all these connections are secure.
That's what I understand. They could corner this market if they made it flat $5-$10/mo and finished getting their DNS registrar up and running. But I suspect they have bigger fish to fry.
Side conversation: I'm interested in being pretty good at this sort of thing: setting up servers, making them connect, understand the protocols enough to put them to practice. I believe this is in the realm of IT. What are some good resources to read to get to my goal? Thanks!
I'd recommend doing almost exactly what OP describes. Start on your local network first. Get a raspberry pi, and figure out how to set up a webserver on it you can access from other computers on your network.
I don't get it, I may be missing some context thought: why the author didn't go for running said webapps directly on the VPS in the first place? Why bother running it on the Pi and set up the tunnel (which is a nice hack in itself, I get that)?
Something that requires LAN connectivity: media library off a local NAS (e.g. Plex), generic storage (e.g. Nextcloud), or home automation (e.g. Home Assistant).
Interesting solution. Also as alternative to a VPN.
A VPN is often just too complex to set up, and also requires some hardware in your local network; though often your router will have that already. Off course a Rpi can act as VPN gateway/server.
But solving it by flipping the "servers" around and running the "appserver" in your network then tunneling it to remote seems a really neat hack to avoid going into VPN hell.
Perhaps the Raspberry Pi is used to control a robot, collect sensor data, whatever. Perhaps it was just for the fun of it.
I've used similar techniques for doing a wide variety of operational tasks, so it can be worth it to take the time to figure out how to easily set up tunnels and reverse proxies.
One use case is the cost of RAM when you buy a VPS. Low-end VPSes are cheap ($5 a month and under) but if you have multiple apps (or one big one), getting a decent amount of RAM in a VPS raises the price significantly.
Using Linode as a benchmark, a 512 MB VPS is $5/mo but an 8 GB VPS is $40/mo. But an 8 GB Rasberry Pi 4 is well under $100 with peripherals so after three months it will have paid for itself.
This of course assumes:
1. that the cost of powering the Pi is negligible
2. your home internet connection is stable enough and fast enough
3. your apps run as comfortably on the Pi as on VPS
1. I have much better hardware in my house than any VPS can offer at a competitive price. This is particularly important for storage. Object storage like S3 is relatively cheap, but I just want a file system I can slap a static server (or whatever else I want) in front of. If you go for full block storage the price is way higher, and performance still isn't as good as a local SSD.
2. I hope the future of the internet involves more people self hosting over IPv6. I see tunneling solutions as an intermediate step to that world. If we make it easy for people to self host, it might even be able to help drive IPv6 adoption.
Having used SSH for this purpose for some time, I decided to build an automatic service for easing reverse HTTP, HTTPS, and SSH ProxyJump through SSH tunnels, its available on Github as well, but if you'd like, try it out:
ssh -R80:localhost:80 remote.moe
it should output an URL where your local port 80 is now available
You could do same with free tier google instance micro.
The thing is, what do you get from exposing your own pi to internet ? Like you are not going to host websites and shit so what crude home automation and monitoring system ?
Why not? Maybe not for a client, but for a blog or static site it takes just about no compute to host that, especially if it's behind Cloudflare or something.
SSH tunneling as a Service (STAAS): holepunch.io announced itself on HN in 2019 [1]. For now its free so it comes slightly ahead of using the Hetzner server.
My personal assistant runs on my pi. When I'm out I can use its chat app to talk to it. The chat uses a cloud real-time db like firebase. If I want to connect I just ask it to setup a reverse proxy.
It can also turn on my machines via wake on lan. I'm quite happy with the setup since I can leave home anytime for long periods and always know I have access to the "home base".
There's a fairly easy way to do this with NGrok, and inlets makes it a little better by allowing you to have any port you like and a custom domain if needed. Checkout the tutorials at: https://docs.inlets.dev/
156 comments
[ 4.3 ms ] story [ 215 ms ] threadSolutions can always be improved, but it's not always worth doing that.
The only TCP in use is the TCP connection of the SSH connection between hosts.
Beware of TCP over TCP issues[0] when using SSH for tun.
[0]: http://sites.inka.de/bigred/devel/tcp-tcp.html
I'm a little confused as to why you would go through the trouble of running a reverse proxy on a remote server that is fully capable of hosting your web applications and or just setting up a port forwarding rule on your local wifi router to redirect 10080 to 80.
Yes there are privacy reasons but that is not mentioned.
Im guessing the other commentors did not view the OPs git.
In my case some services are not routed to the public internet and only accessible from within my privat wireguard network. E.g my smart home hub. I really like the flexibility to decide whats available for who.
https://www.pivpn.io
[1] https://tailscale.com/
[2] https://github.com/tailscale/tailscale/issues/854
I also have my phone, laptop, etc all on the wireguard VPN, so I can ssh to anything from anywhere. I also have my laptop open to password-based ssh only from my wireguard VPN, so I can ssh (via termux) into my laptop, which has been extremely convenient in addition to being just plain cool. I can't speak highly enough of wireguard!
I wonder how a somehow simpler config syntax (Caddyfile) compares to a simpler maintenance system (apt).
To give you an idea about the compatibility differences, it's like the upgrade from Vue.js 2 to Vue.js 3. Some changes to format are necessary for the new architecture. But you can always just run the older version (that's what I do). And swapping to the new one isn't terribly painful if your just using vanilla (for Caddy 1 extensions, which weren't created by the developer himself most of the time, I'm sure Caddy 2 versions will pop up over time).
I haven't found the changes in Caddy's config format to be problematic in the least.
I mean, in 90+% of cases, you just put in your Caddyfile
and you get a fast reverse proxy that just works, with letsencrypt enabled by default to boot, nothing to worry about.However, if you find yourself against a service that requires a slightly more advanced configuration, good luck making sense of their opaque configuration, especially since they now went v2 so everything old is out of the window, so if you search for something, chances are it's no good anymore.
H2O web server seems to be faster than nginx , litespeed and supports http1/http2/http3 . It's config file is yaml file.
For HTTPS, i recommend https://acme.sh . It's very very easy and autorenews certificates using cron job.
You failed to do that twice just in this thread.
For many of us, it's a trust thing.
--
ETA:
Speaking of trust, I just read this FAQ on inlets.dev:
> Does inlets PRO "call home"?
> We trust our users to purchase the correct license for their usage, and in return we make licensing simple with a fixed-term license.
So, a few minutes after mentioning that "it's a trust thing", I came across this very simple, to the point, "yes or no" question ...
... and yet I notice that you|your company) (couldn't|didn't), for whatever reason, actually answer the question.
That, of course, makes me even more distrustful than I was a few minutes ago!
I’m not sure people who aren’t network administrators should be putting public IPv6 all over their LAN behind their router, at least not until v6 is more common and routers have better inbound security features for such things (like how NAT functions today for v4).
Every single tutorial on the topic is IPv4-specific and until IPv6 isn't more common, people will not write guides and tutorials with IPv6 in mind. So until the knowledge of proper security on open networks is common enough among home tinkerers, we should not be recommending people just throw everything straight onto the open Internet.
somewhat simplified like this using iptables on linux:
vsThere's nothing wrong in using NAT to increase the cost of attack as a part of a larger defence strategy.
I mean, you’re technically correct, but isn’t this kind of like saying that the .local TLD is free of charge? It’s “free” because it’s not a part of the “normal” internet.
Sure you don't get an interesting name but you do get one.
.oarsinsync is probably a better comparison. It’s not explicitly scoped anywhere, and it works nicely in my oarsinsync bubble, much like .onion works nicely in the ToR bubble.
In both cases, you need to connect using some kind of tunnelling protocol to connect into another network.
For me, all my personal machines have TOR installed anyway and I mostly want public address for gluing things together or being able to access things remotely. It's definitely not usable for everything you normally use domain names for (like showing people the menu at a restaurant.)
I will point out that you don't need to do any extra work to access web servers on TOR at least. I think cloudfront (and I know others) run a proxy that lets you access them with just a normal URL via a web browser. Most OSes need no extra configuration for .local.
You run a Tor proxy (bind to 127.0.0.1:9050) on a server on the Internet, and then run a Tor hidden service inside your LAN.
Then, you acquire an .onion domain for the hidden service (running inside LAN), and configure the Tor proxy (running on a server on the Internet) to access the hidden service via the .onion domain (for example, use Nginx to reverse proxy the hidden service).
This should work. However, I imagine it could be quite a bit slower compare to "normal access", and requires extra steps to secure the transport connection between the hidden service and the proxy as you might not want to have everybody on Tor to be able to access that hidden service.
The latency can be painful but it's way better than nothing.
I will say you should probably be careful about running a webserver on TOR. Almost all my machines have a basic web server just for copying static files to windows hosts I don't want to give my password to. People would find these on TOR and then find my resume, I got plenty of emails asking to set up "hidden sites on the dark web for sharing pictures."
The daemon takes very little memory and CPU.
https://github.com/fatedier/frp
Do you have a static IP or a domain that your beacon uses?
I hadn't considered that, but it makes sense. Since ZeroTier is layer 2, I'd expect it to become a leaky abstraction (at least for performance) in certain cases where algorithms are expecting other layer 2 hardware to be essentially directly connected with a wire when in reality they're connected over the internet.
https://news.ycombinator.com/item?id=24436399
I will definitely do a writeup during/before the Christmas break.
https://tailscale.com/
What next? Well I setup some handy shell functions:
1) copy the latest screenshot into webroot/screenshots/${randstr}.png and copy the url to clipboard. this replaces dropbox's cool screenshot sharing feature. love sharing screenshots with coworkers this way, the files stays with you, you can redact easily.
2) serve static files the same way, if needed too. the magic's all in the shell scripts.
3) explore new open source webapps running on local docker.
Further?
Build a sort of like a devtools backend server, and a mobile app paired to it to do all sorts of cool stuff: run crons, get notifications for events, monitor servers / ssl expiry etc.
Limit's your imagination. But, security is also very important.
The only drawback is I don't think it would be possible to connect to a peer from something like heroku where you don't have full control over the server..
A VPN is often just too complex to set up, and also requires some hardware in your local network; though often your router will have that already. Off course a Rpi can act as VPN gateway/server.
But solving it by flipping the "servers" around and running the "appserver" in your network then tunneling it to remote seems a really neat hack to avoid going into VPN hell.
I've used similar techniques for doing a wide variety of operational tasks, so it can be worth it to take the time to figure out how to easily set up tunnels and reverse proxies.
Using Linode as a benchmark, a 512 MB VPS is $5/mo but an 8 GB VPS is $40/mo. But an 8 GB Rasberry Pi 4 is well under $100 with peripherals so after three months it will have paid for itself.
This of course assumes:
1. that the cost of powering the Pi is negligible 2. your home internet connection is stable enough and fast enough 3. your apps run as comfortably on the Pi as on VPS
1. I have much better hardware in my house than any VPS can offer at a competitive price. This is particularly important for storage. Object storage like S3 is relatively cheap, but I just want a file system I can slap a static server (or whatever else I want) in front of. If you go for full block storage the price is way higher, and performance still isn't as good as a local SSD.
2. I hope the future of the internet involves more people self hosting over IPv6. I see tunneling solutions as an intermediate step to that world. If we make it easy for people to self host, it might even be able to help drive IPv6 adoption.
ssh -R80:localhost:80 remote.moe
it should output an URL where your local port 80 is now available
https://github.com/fasmide/remotemoe
[1] https://news.ycombinator.com/item?id=20006785
It can also turn on my machines via wake on lan. I'm quite happy with the setup since I can leave home anytime for long periods and always know I have access to the "home base".