HN Explorer

Ask HN: A software for forwarding data to a remote server?

2 points by sbr464 ↗ HN
Looking for a local agent (software) that would forward data to a remote endpoint (https etc), and/or act as a proxy to connect a local database to a remote api/server.

Needs to work behind a NAT firewall and not require a static ip/typical VPN.

Example, sending any new files in a directory, or forwarding data from a connected serial/usb device, such as a sensor.

Kind of similar to Elastic Beats, or a DataDog agent, but wasn't sure if those would be the best solution.

Semi-technical users would install and run, so prefer not to use manual solutions, (configuring ssh server or rsync directly etc).

Preferably open source so can make changes as needed. But would consider a closed source/retail agent if it was configurable and lightweight. Would also consider two different agents to handle different use cases.

1 comment

[ 3.0 ms ] story [ 14.5 ms ] thread
DLA
You could write a simple script/daemon to wrap rsync. To make the install, configure, run process dirt simple and reliable.

You could write a nice agent in Go (golang.com) that watches directories and sends files over your choice of protocols, maybe to an AWS S3 bucket. A little dev work could get you a single file program (zero dependencies, just copy program and run it on each host). Here's one example: https://github.com/Redundancy/go-sync

Here's a search for more similar libs: https://golanglibs.com/search?q=rsync&sort=top

I've used this method extensively and can tell you it works very well. You can add whitelist/blacklist filters, specify folders, recursively traverse a tree of folders/files, etc.

A single file binary with zero dependencies is a huge plus for a situation like this. You can cross compile the same source for Windows and nearly every flavor of Linux with ease. Just my $0.02.

Google turned up FileJuggler which looks nice: https://www.filejuggler.com (Disclaimer: No personal knowledge of this software).