Ask HN: How do you manage large files across multiple laptops/desktops?
My wife is a graphic designer and works on both a Mac desktop and laptop. The assets she generates for a single project can reach into gigabyte territory – multiple versions of a set of print layouts, RAW photos, etc. One problem she has is how to reliably sync large files between computers. She uses Dropbox currently, but sync over network can takes a long time or sometimes just doesn't finish depending on the file. And when she runs into space limits on one computer, she'll need to adjust Dropbox settings to selectively sync items.
I'm curious what people do in this case. I think she needs:
1) A large amount of storage, expandable as time goes on 2) Files accessible from multiple computers 3) Ability to access files securely when working from a coffee shop, traveling, etc.
21 comments
[ 4.9 ms ] story [ 67.9 ms ] threadIt isn't exactly syncing, but it solves the problem, at least for me.
Add an rsync server to get backup. Forget the cloud, most solutions are overpriced and insecure. Use TarSnap if you have absolute have to backup reasonable quantities of critical data and call it a day.
Of course, it only works if you’re online, and VDI-ish solutions aren’t really a thing on OS X, so YMMV, but I remote into my desktop while I’m at school because my laptop doesn’t have much ram.
I work on large video files that I transport between locations and had an external drive that I used for a while but would end up forgetting or not wanting to unplug everything. I have settled on a high speed 64gb SD card that fits in any pocket or even my wallet. Out of all the fancy network tools I set up, this option has proved to be the most enduring.
Doesn't help with synchronizing which files are most current though. You might look at Transmit 5 from Panic. It has lots of useful syncing features now.
If you primarily do it at home, the speeds of Nextxloud are typically limited by the network bandwidth, and the storage is limited by what server you have. I use it and manage 5 TB of data on it without problems.
- Pages load pretty slow over mobile networks. Every app page requires like 5 seconds to load, as it loads the page in 2 steps, first the page and then then the content. Not talking about navigating folders in Files but apps like Calendar and News are especially slow.
- Bookmarks app silently stopped working few weeks ago when I thought I kept adding Bookmarks through Android mobile app, so I lost 2 weeks worth of them. Not sure if the server or the client app went wrong but I switched to Pocket even though I run everything self hosted where I can.
- Contacts app previously had a problem where you couldn't save more than one information per page reload until recently when used through CloudFlare which was very confusing to see some saved data disappear.
- News app's UI is a bit of a joke in terms of speed and usability, so you need a mobile app to consume it than from the web.
- Frankly all of the apps need mobile apps because pages load slow and it's reluctant to go through them from the web but the mobile app quality varies so much as they're all dependant on third party developers.
- NextCloud talk, I had a high expectation of it but no matter how I tried, it never worked reliably and client apps were nowhere near the quality to recommend to non tech work people.
- Using an external storage for Files makes the performance and reliability quite low and when you think about using it for work, you need maximum reliability but I ended up using local storage as that seems the only way that's tested enough.
- You can't tell how much of a file is uploaded reliably at all. The "seconds remaining" display jumps all over every second as it's probably calculating from the speed of the moment instead of averages of the last minute or something and you can't cancel the individual files queue.
- I have basic auth on the NextCloud instance but the official NextCloud app refuses to work through it, even though the web version allows you to have yourself logged in via basic auth.
- Email app acted like an alpha version, so I instead use integrated RainLoop which is fine but I couldn't get people to login automatically using the NextCloud logged in user even though I have that option turned on, instead I had to edit some source code to make it happen.
- Logging out takes forever.
I am impressed that the look and feel is good and the caldav being so easy to use but I'd say it's close when it comes to overall quality to have work people use it than just for yourself.
- SpiderOak One
- Mega (very cheap)
- Syncthing (open source)
She should also verify her Time Machine configurations on both Macs are set up to back up her sneakernet drive fairly frequently. ... You only seem to need backups when you don't have them...
You'll want to have an external USB3.0 drive to get from one machine to another IMO, because you would have wanted it backed up ANYWAY right? ...so there's no need to try to copy directly from one machine to another machine, without using the USB drive.
Here's an example command:
sudo rsync -aAXv --delete --force "/home/clay/ferguson/" "/media/clay/BAK/ferguson"
because all it needs to do is prompt for source and destination folders, with a stern warning about what it DOES. ha.
If you have to share files off the LAN, that gets more involved.
We currently use a mix of syncthing and NAS for our internal purposes, and iCloud for external syncing.