The overhead of layer 2/3 packet headers in OpenVPN did not cause such a huge slowdown, its much more likely that OpenVPN was operating in TCP mode, and thus subject to the tcp-in-tcp[1] problem with tunnel internal tcp sessions. Even an extraordinarily slow machine by modern standards can manage 100mbit openVPN without much of an issue.
EDIT: did not see UDP results. There is still something else going on here, thats still much too slow for openVPN (without tweaking).
Both of the comments about VPN support within SSH seem to be heavily downvoted with no reason why. As far as I can see, their points are correct:
> OpenSSH has built-in TUN/TAP support using -w<local-tun-number>:<remote-tun-number>. Here, a layer 3/point-to-point/ TUN tunnel is described. It is also possible to create a layer 2/ethernet/TAP tunnel.[0]
Just with invalid terminology (s/ssh/OpenSSH) and the likelihood of the added slowdown mentioned in the article.
This is one of those Arch wiki pages that I find lacking, so I personally always refer to the man pages:
man 1 ssh:
-w local_tun[:remote_tun]
Requests tunnel device forwarding with the specified tun(4)
devices between the client (local_tun) and the server
(remote_tun).
The devices may be specified by numerical ID or the keyword
``any'', which uses the next available tunnel device. If
remote_tun is not specified, it defaults to ``any''. See also
the Tunnel and TunnelDevice directives in ssh_config(5). If the
Tunnel directive is unset, it is set to the default tunnel mode,
which is ``point-to-point''.
man 5 ssh_config:
Tunnel Request tun(4) device forwarding between the client and the
server. The argument must be yes, point-to-point (layer 3),
ethernet (layer 2), or no (the default). Specifying yes requests
the default tunnel mode, which is point-to-point.
TunnelDevice
Specifies the tun(4) devices to open on the client (local_tun)
and the server (remote_tun).
The argument must be local_tun[:remote_tun]. The devices may be
specified by numerical ID or the keyword any, which uses the next
available tunnel device. If remote_tun is not specified, it
defaults to any. The default is any:any.
man 5 sshd_config:
PermitTunnel
Specifies whether tun(4) device forwarding is allowed. The argu-
ment must be yes, point-to-point (layer 3), ethernet (layer 2),
or no. Specifying yes permits both point-to-point and ethernet.
The default is no.
Independent of this setting, the permissions of the selected
tun(4) device must allow access to the user.
It doesn't support UDP in general AFAIK, i'm a frequent user of shuttle and find it indispensable, but if you need UDP it does fail, but then most VPN protocols only support a subset of IP layer protocols, e.g multicast doesn't generally work on any of them.
sshuttle is basically magic to me. Wherever someone with no idea about their threat model has deployed firewalls (ala every major business I've ever worked at) sshuttle is usually the key to being productive for me.
"Wow a little googling on this subject led me to sshuttle, which I've never heard of before but looks awesome"
sshuttle is indeed awesome - a viable VPN that requires nothing on the endpoint but a functioning ssh login (and python on the remote host).
So any host, anywhere, with no configuration (and no permission required by the operators) that you can ssh into ... is now a VPN endpoint for you.
Sibling comments in this thread are pointing out that ssh has this functionality already and that is true, but some configuration and maintenance (and understanding) is required. Not so with sshuttle.
ALSO:
We (rsync.net) sponsored the reworking of the ipfw target for sshuttle so that it now works properly on FreeBSD with the --dns option, etc. We also sponsored proper UDP tunneling for sshuttle on FreeBSD but I am not sure those patches are in a -release version of FreeBSD yet ...
You need to use FreeBSD 11.0 (not 11.1) and you need to:
git clone https://github.com/sshuttle/sshuttle.git
cd sshuttle ; git checkout c746d6f7db3efbad6caddea76bdf916c46cf5c6e
... and that will get you a working sshuttle on FreeBSD with --dns support.
If you use Windows and need to access in a browser some web services over ssh, then just run ssh -D and configure your browser to use PAC JavaScript file that redirects relevant host names to the ssh proxy.
What about using cygwin instead of a Linux VM? I didn't have the time to try specifically this, bit normally you can reach really far with Cygwin. Plus, no need for root access, and the installation is natively "portable" (everything sits inside a single directory).
I mention this every time VPNs come up... It's just so much better than any of the other VPN clients if you are only interested in TCP, I've been using it for over 6 years.
The key difference between it and VPN protocols for anyone trying to compare is: TCP only, and TCP decompile/recompile with SSH in the middle. This has many advantages: speed (TCP over TCP is megga sucky obviously), server setup (basically none, you just need non-root access to an ssh server with python).
It really is as simple as making an SSH connection on the command line and you can route all your TCP traffic immediately.
It also has some really nice fine grained subnet routing features... On some strange occasions when I have needed it i have merged multiple remote subnets into the same virtual one one my machine by selecting ranges. It also has an auto discovery feature built in if you just want to route the specific IPs that exist on the other side.
sshuttle is also in a number of package managers now, it's in apt at least.
I've found that sshuttle with the --dns option still leaks dns queries occasionally so I set my web browser and curl to use SOCKS and run sshuttle to try to catch anything else.
I found that OpenVPN benefited greatly from setting larger tunnel MTUs and then allowing the IP packets to be fragmented. I think it's encrypting per-packet which is somewhat inefficient.
1. Doesn't look like there were runs of iperf over UDP and TCP, with simulated buffer sizes, so hard to know what the network connection is even capable of.
2. Also, how reliable is the network connection? Maybe it fluctuates every minute in usable bandwidth due to congestion or jitter. VMs are subject to noisy neighbor issues.
3. It looks like the ciphers are different, which could of course affect throughput greatly if one is not hardware accelerated or if its CPU bound. Worth checking CPU and other system resource usage for differences.
4. You could try OpenVPN with encryption and/or compression disabled entirely in UDP mode for best possible performance IIRC. This, at least, you would expect to beat an SSH tunnel.
> 2. It looks like the cipher "AWS128-CTR" is being used for
> SSH, I'm not even sure what that is. I'd look at CPU and
> other system resource usage for differences.
This has been known forever. Stream-based packet tunneling absolutely does outperform packet wrapping in good network conditions, for the simple reason that small packets can be combined by the transport layer and avoid the overhead.
The problem is that in the presence of any packet loss at all, every packet lost causes a stall of the whole tunnel until it gets retransmitted, which even in the best case recovery conditions requires two (three? I forget my SACK details) round trips. If one side is using traditional TCP retransmit timers it can be much, much longer.
Well-known unpredictable TCP-over-TCP performance would only explain OpenVPN/TCP results.
But OP has posed also OpenVPN/UDP performance that is 2x...3x slower than SSH port forwarding, so I'd guess there is more to this story (see also andmarios's impromtu test results above).
Your downside I don't think applies in this case. Because it's a single application wanting to make a single reliable connection. So if there was any packet loss at all it would have to stall the whole connection anyways regardless of whether it's wrapped in SSH, OpenVPN, or nothing.
I've been using a VPN over TCP ssh for a year. The end point is across the world. I get excellent bandwidth and no stalls. Latency is less than anticipated. The thoretical arguments against TCP/TCP do not match my experience.
There should be some catch in the setup. Maybe the CPUs he used are old and don't support hardware AES acceleration?
I just run iperf3 over OpenVPN / UDP with AES-256-GCM between two servers in the same DC (but different rooms, through their public network, 1gE links) and got an average of 750MBits/sec. I don't have any special setting.
Just tested. Ssh has indeed better performance, at around 870Mbit/sec.
But the gap in my -anecdotal- testing is much smaller than OP's and maybe predictable due to the differences (as OP described) on how OpenVPN and ssh work.
fwiw OpenVPN has had more consistent performance over the years than SSH for me. my servers have always been half a world away tho. i can reliably saturate my 100mbit home connection over SSH now, so I'm happy.
A SOCKS5 tunnel supports DNS. This is good for web browsing and is natively supported by ssh. You can even use it for individual programs using the tsocks command line wrapper.
One thing I hate about OpenVPN is the ridiculous configuration. I'm a big fan of tinc[0] personally, although you have to use 1.1pre releases to get half decent encryption, it's configuration is wonderful.
tinc 1.1pre seems pretty good, but I do worry about the legacy fallback for encryption; There doesn't seem to be any way to e.g. --force-experimental to prevent encryption falling back to the old RSA mode.
A huge advantage of vpn tunnels is that they stay up indefinitely. I've had openvpn links stay up for months. With ssh, you're lucky to get a few days. I've tried scripts and even autossh, and ssh has never been as reliable as openvpn or tinc.
Does anyone have any experience with ipop/groupvpn[0]? It seems like an interesting alternative to tinc in that it can do turn/stun using libjingle to bust nat. I'm not clear on how well tested or secure it is though.
Worth looking at ZeroTier (commercial - actually running a hardware IndieGoGo right now) and WireGuard (IPSec - 'The Next Generation' / 'Voyager' or whichever you thought was best) as modern options.
I first tried to setup OpenVPN and only wanted to use the the SSH tunnel for DNS purposes (I couldn't use normal DDNS tools), but in the end, I ended up running a SOCKS proxy over the tunnel and it was stunningly effective, enough that watching Netflix despite intercontinental round trips.
I used to maintain an ipsec VPN for road warrior type scenarios, but I've found that mostly I just wanted to get a remote desktop on a computer within the network.
So I've stopped mucking around with VPN and just published Apache Guacamole HTML5 VNC/RDP tunnels. No plugins, no clients except a modern web browser and good security with certbot+auto redirect to https+simple auth to keep badguys away from possible exploits in the guacamole login form. Configs can be as simple as the NoAuth plugin, all the way to full integration with eg. AD.
I've implemented this at a number of businesses, and it does what almost everyone in the organization wants. It also supports things like printing (to a pdf that comes thru the browser as a download) and file transfer.
Super easy to set up and is much more responsive than VNC or RDP thru ssh.
The times when I actually want full IP access to a network from my laptop are so rare that it seemed silly to maintain IPSec.
What need do others have for full-connectivity VPNs? Not site-to-site VPNs of course, but these ad-hoc on the road type ones.
Being local in a country with a limited Flix library, I have a hard time still paying for the (most excellent!) service.
And since Flix has succesfully banned all commercial VPN's, I am more and more contemplating seting up a "share your own local IP for Flix affectionados" with a couple of people.
Problem is they not only banned most commercial VPNs but i believe they pretty much banned entire ASN's from watching their stuff. So it's not like you can just go ahead and and set a VPS up somewhere, because it'll be blocked too :(
I recently wrote about my home VPN setup in a previous HN thread [1]. But to answer your question, the additional use-cases I have are:
1. I connect to my home file server from all of my devices. This allows me to access my singular master file system from everywhere, which includes all of my documents and data. For example, I can reach my Keepass database from all devices without any need for clumsy synchronization or a third-party "cloud" provider.
2. I view my security camera streams by reaching my on-premises NVR from anywhere.
3. I can print to home from anywhere.
4. My entire media library is available from anywhere.
5. I can reach my family's home intranet web-app from all of my devices (e.g., my phone), natively.
It's true that all of this could be done in a remote desktop window, but it would be less smooth and less convenient. E.g., using Keepass in a remote desktop window wouldn't be as convenient as running it locally (with the kdbx loaded from the file server).
I have pretty much the same functionality, including the printing, plus I have a home IoT setup.
Through the MQTT app on my phone I can view and control the home heating, see the output from various sensors (like house temp and outside temp) and control some of the lights - all without having to send my data to a commercial product's cloud service to manage things.
The IoT hub is an Atom-based Mini ITX machine running Ubuntu server, Mosquitto (MQTT message broker), Node-Red (MQTT message handler) and Home Assistant to manage some of the sensors and automation (although Node-Red does most of the work). I'll probably move the whole lot to a Raspberry Pi at some time.
This setup is more of a base to exercise my electronics hobby skills than a serious attempt to automate everything - for example, the home heating IoT stuff overlays (rather than replaces) the traditional electronic timer and temperature sensors on the heating/hot water system, so if the IoT cleverness goes offline, watchdog relays will return control to the old timer/sensors.
VPN access is through a Draytek xDSL Router (2860) - they have good inbuilt VPN support.
For everyone else, I recommend https://pritunl.com or https://www.zerotier.com to get VPN access easily. Pritunl is more traditional client/server while ZeroTier creates 1 giant LAN between as many machines as you want.
Thank you for mentioning them!!! Was just looking for a solution for connecting my VPSs across DO, Vultr and Lightsale ZeroTier looks like a perfect solution
1. My country bans a lot of websites, so I'm often using netherlands VPN to circumvent this ban. Also internet provider at my work sends angry emails when I'm downloading torrents, so I'm downloading them over VPN as well. I don't really see any alternative to VPN.
2. My home server contains some interesting services for me (file storage, movies, music, passwords), but I don't want to expose them to the internet, so I exposed it to VPN and protected with very minimal authorization (nginx checking value of specific cookie acting as a password). So I'm using VPN as a main protection for weakly protected services.
3. I didn't implement it yet, but I want to receive my mail on my home server. Currently mail server operates on VPN server, so I can't really be sure that nobody reads my mail. But operating e-mail requires static IP and reverse-ptr record which isn't available from my home internet provider. So I want to use some kind of port mapping with VPN to transfer everything to my home server. Though ssh would work well here, but I would need to think about reconnecting if network is down, I expect VPN to be more stable.
Ssh to access various servers/git repos/my desktop. Ssh in a local window is always snappy. Terminal access in a RDP/VNC window is not. Not to mention modifier key issues etc...
Guacamole also supports SSH/telnet, and is very snappy thru html5.
I autogenerate a guac config file for all the switches on each site at work, and even on high-latency low-bandwith sites on wireless it still works great. Definitely better than a console session via RDP or VNC.
This has the extra benefit that you can lock your switches down to only be managed thru certain IPs and enforce session recording.
Many companies (including startups) have internal tools that are web applications. While the more adventurous firms put these on public IPs behind an employee login/SSO system, the more conservative will always have them behind VPN. VNC is a drastically worse UX than a local browser unless your endpoints are really bad.
Honestly, I think you can get a similar security posture with a reverse proxy enforcing client certificates and 2FA (this is what my employer does), but people feel more comfortable trusting VPN appliances from big-name vendors than their nginx configs. Harder to fail open, certainly.
It’s really not an apples to apples comparison. For my teammates on the development team, yes an SSH tunnel is going to be easier for our needs due to the reduced overhead. But if I want to give my customer support team access to private web services in our production cluster, a VPN is the perfect solution.
At FarmLogs, each of our Kube clusters has a bastion VPN host that puts you inside the VPC and handles DNS to Route53 (and the VPC) so <service>.farmlogs “just works”
We use Foxpass to handle this with LDAP bridged to Google Apps so when an employee joins or leaves their VPN access is immediately updated.
You have a TCP connection between local app and SSH client, then a second TCP connection between SSH client and SSH server, and a third TCP connection between SSH server and remote app.
There is no TCP over TCP when you use SSH tunnels. Lost packets between SSH client and SSH server do not cause retransmits on either application side.
I use both actually, where the best tool is applicable.
Some commenters have mentioned StrongSwan (or OpenSwan which is more known to me).
I've actually seen OpenSwan perform better than Cisco VPN tunnels. And I love pointing that out to our networking guys since we pay nothing for OpenSwan.
117 comments
[ 3.2 ms ] story [ 202 ms ] threadEDIT: did not see UDP results. There is still something else going on here, thats still much too slow for openVPN (without tweaking).
[1] http://sites.inka.de/bigred/devel/tcp-tcp.html
Possibly the SSH encryption could have been more optimal for their use case?
http://sshuttle.readthedocs.io/en/stable/usage.html
Basically "vpn over ssh" (not really, but close enough).
> OpenSSH has built-in TUN/TAP support using -w<local-tun-number>:<remote-tun-number>. Here, a layer 3/point-to-point/ TUN tunnel is described. It is also possible to create a layer 2/ethernet/TAP tunnel.[0]
Just with invalid terminology (s/ssh/OpenSSH) and the likelihood of the added slowdown mentioned in the article.
[0] https://wiki.archlinux.org/index.php/VPN_over_SSH
man 1 ssh:
man 5 ssh_config: man 5 sshd_config:I've never had to use them since as you say few are that hardcore in their lockdowns, but it's nice to know they exist.
sshuttle also claims to have avoided the "TCP over TCP" problem.
https://stackoverflow.com/questions/41427123/how-does-sshutt...
Edit: why the downvote? I thought I asked a legit question...?
Please don't comment about the voting on comments. It never does any good, and it makes boring reading.
Handy when you want to do something private on a machine you don't have a VPN setup on.
sshuttle is indeed awesome - a viable VPN that requires nothing on the endpoint but a functioning ssh login (and python on the remote host).
So any host, anywhere, with no configuration (and no permission required by the operators) that you can ssh into ... is now a VPN endpoint for you.
Sibling comments in this thread are pointing out that ssh has this functionality already and that is true, but some configuration and maintenance (and understanding) is required. Not so with sshuttle.
ALSO:
We (rsync.net) sponsored the reworking of the ipfw target for sshuttle so that it now works properly on FreeBSD with the --dns option, etc. We also sponsored proper UDP tunneling for sshuttle on FreeBSD but I am not sure those patches are in a -release version of FreeBSD yet ...
You need to use FreeBSD 11.0 (not 11.1) and you need to:
... and that will get you a working sshuttle on FreeBSD with --dns support.A shame it's a bit more involved to get working with Windows, it's a pain to have to start a Linux VM to be able to use it :(
The key difference between it and VPN protocols for anyone trying to compare is: TCP only, and TCP decompile/recompile with SSH in the middle. This has many advantages: speed (TCP over TCP is megga sucky obviously), server setup (basically none, you just need non-root access to an ssh server with python).
It really is as simple as making an SSH connection on the command line and you can route all your TCP traffic immediately.
It also has some really nice fine grained subnet routing features... On some strange occasions when I have needed it i have merged multiple remote subnets into the same virtual one one my machine by selecting ranges. It also has an auto discovery feature built in if you just want to route the specific IPs that exist on the other side.
sshuttle is also in a number of package managers now, it's in apt at least.
If you're using a windows client I recommend bitvise tunnelier.
2. Also, how reliable is the network connection? Maybe it fluctuates every minute in usable bandwidth due to congestion or jitter. VMs are subject to noisy neighbor issues.
3. It looks like the ciphers are different, which could of course affect throughput greatly if one is not hardware accelerated or if its CPU bound. Worth checking CPU and other system resource usage for differences.
4. You could try OpenVPN with encryption and/or compression disabled entirely in UDP mode for best possible performance IIRC. This, at least, you would expect to beat an SSH tunnel.
The problem is that in the presence of any packet loss at all, every packet lost causes a stall of the whole tunnel until it gets retransmitted, which even in the best case recovery conditions requires two (three? I forget my SACK details) round trips. If one side is using traditional TCP retransmit timers it can be much, much longer.
But OP has posed also OpenVPN/UDP performance that is 2x...3x slower than SSH port forwarding, so I'd guess there is more to this story (see also andmarios's impromtu test results above).
I just run iperf3 over OpenVPN / UDP with AES-256-GCM between two servers in the same DC (but different rooms, through their public network, 1gE links) and got an average of 750MBits/sec. I don't have any special setting.
But the gap in my -anecdotal- testing is much smaller than OP's and maybe predictable due to the differences (as OP described) on how OpenVPN and ssh work.
When you are comparing one thing to another and writing up your pearls of wisdom for the masses you need to control every variable (within reason).
I can't see a LAN based test acting as a baseline.
Why? I plunk down a lot of stuff on my blog, not everything is up to that pretty high standard you mention.
Its basically a script which takes care of ssh, the routing, and firewalling. Its crossplatform, should work on Windows/Linux/macOS and perhaps more.
[1] https://github.com/apenwarr/sshuttle
[2] https://sshuttle.readthedocs.io/en/stable/
https://github.com/apenwarr/sshuttle is the original, now abandoned, sshuttle. Development was taken over by:
https://github.com/sshuttle/sshuttle
... which is the current, maintained version that you should track.
[0] https://www.tinc-vpn.org/
[0] http://ipop-project.org/
AutoSSH plus some firewall scripts to ensure data only travels through the VPN would suffice.
https://news.ycombinator.com/item?id=15596963
https://vadosware.io/post/ssh-tunneling-using-an-intermediar...
I first tried to setup OpenVPN and only wanted to use the the SSH tunnel for DNS purposes (I couldn't use normal DDNS tools), but in the end, I ended up running a SOCKS proxy over the tunnel and it was stunningly effective, enough that watching Netflix despite intercontinental round trips.
So I've stopped mucking around with VPN and just published Apache Guacamole HTML5 VNC/RDP tunnels. No plugins, no clients except a modern web browser and good security with certbot+auto redirect to https+simple auth to keep badguys away from possible exploits in the guacamole login form. Configs can be as simple as the NoAuth plugin, all the way to full integration with eg. AD. I've implemented this at a number of businesses, and it does what almost everyone in the organization wants. It also supports things like printing (to a pdf that comes thru the browser as a download) and file transfer. Super easy to set up and is much more responsive than VNC or RDP thru ssh.
The times when I actually want full IP access to a network from my laptop are so rare that it seemed silly to maintain IPSec.
What need do others have for full-connectivity VPNs? Not site-to-site VPNs of course, but these ad-hoc on the road type ones.
Honest question
Wow, didn't know that. Impressive!
And since Flix has succesfully banned all commercial VPN's, I am more and more contemplating seting up a "share your own local IP for Flix affectionados" with a couple of people.
How do others solve / done / consider this?
1. I connect to my home file server from all of my devices. This allows me to access my singular master file system from everywhere, which includes all of my documents and data. For example, I can reach my Keepass database from all devices without any need for clumsy synchronization or a third-party "cloud" provider.
2. I view my security camera streams by reaching my on-premises NVR from anywhere.
3. I can print to home from anywhere.
4. My entire media library is available from anywhere.
5. I can reach my family's home intranet web-app from all of my devices (e.g., my phone), natively.
It's true that all of this could be done in a remote desktop window, but it would be less smooth and less convenient. E.g., using Keepass in a remote desktop window wouldn't be as convenient as running it locally (with the kdbx loaded from the file server).
[1] https://news.ycombinator.com/item?id=15586997
Through the MQTT app on my phone I can view and control the home heating, see the output from various sensors (like house temp and outside temp) and control some of the lights - all without having to send my data to a commercial product's cloud service to manage things.
The IoT hub is an Atom-based Mini ITX machine running Ubuntu server, Mosquitto (MQTT message broker), Node-Red (MQTT message handler) and Home Assistant to manage some of the sensors and automation (although Node-Red does most of the work). I'll probably move the whole lot to a Raspberry Pi at some time.
This setup is more of a base to exercise my electronics hobby skills than a serious attempt to automate everything - for example, the home heating IoT stuff overlays (rather than replaces) the traditional electronic timer and temperature sensors on the heating/hot water system, so if the IoT cleverness goes offline, watchdog relays will return control to the old timer/sensors.
VPN access is through a Draytek xDSL Router (2860) - they have good inbuilt VPN support.
Are you always connected to your VPN on your mobile device? How much network traffic overhead does it add?
http://packetpushers.net/podcast/podcasts/pq-134-meet-zeroti... <- Here's a Pod with the creator of ZT, once you've listened to it you'll realize how it fits everyone =)
2. My home server contains some interesting services for me (file storage, movies, music, passwords), but I don't want to expose them to the internet, so I exposed it to VPN and protected with very minimal authorization (nginx checking value of specific cookie acting as a password). So I'm using VPN as a main protection for weakly protected services.
3. I didn't implement it yet, but I want to receive my mail on my home server. Currently mail server operates on VPN server, so I can't really be sure that nobody reads my mail. But operating e-mail requires static IP and reverse-ptr record which isn't available from my home internet provider. So I want to use some kind of port mapping with VPN to transfer everything to my home server. Though ssh would work well here, but I would need to think about reconnecting if network is down, I expect VPN to be more stable.
Honestly, I think you can get a similar security posture with a reverse proxy enforcing client certificates and 2FA (this is what my employer does), but people feel more comfortable trusting VPN appliances from big-name vendors than their nginx configs. Harder to fail open, certainly.
As such, it is potentially subject to past and future vulnerabilities OpenSSL.
OpenSSH does not depend on OpenSSL.
OpenVPN is way more robust solution for tunneling though.
I've found that UDP tunnels work sometimes better in when there is considerable packet loss and high latency (poor radio links).
At FarmLogs, each of our Kube clusters has a bastion VPN host that puts you inside the VPC and handles DNS to Route53 (and the VPC) so <service>.farmlogs “just works”
We use Foxpass to handle this with LDAP bridged to Google Apps so when an employee joins or leaves their VPN access is immediately updated.
Setup the tunnel: ssh -D 8080 vv@xx.xx.xx.xxx
Change your system proxy settings: Socks proxy, 127.0.0.1, 8080
Done!
You have a TCP connection between local app and SSH client, then a second TCP connection between SSH client and SSH server, and a third TCP connection between SSH server and remote app.
There is no TCP over TCP when you use SSH tunnels. Lost packets between SSH client and SSH server do not cause retransmits on either application side.
Btw I use unix sockets on client and server to avoid the local connection TCP overhead.
Some commenters have mentioned StrongSwan (or OpenSwan which is more known to me).
I've actually seen OpenSwan perform better than Cisco VPN tunnels. And I love pointing that out to our networking guys since we pay nothing for OpenSwan.