That requires you to have always on, central server running and stroing things. Devices running syncthing are run as equals. (or they can be configured not to be)
One cool feature is also encrypted syncing where some nodes dont even have the key (ie VPS syncs but doesnt know what it syncs, and only your devices have the key)
NextCloud is great but like any webapp suffers from bloatedness and cryptic errors. If you've ever tried uploading terabytes with the web client, you know what i'm talking about.
Also, performance (read bandwidth usage) isn't all that great with NextCloud. PHP-FPM was more designed for concurrent execution of multiple stateless requests server-side (PHP) than for sending raw bytes as soon as possible to the disk.
Love this. As someone primarily from a web background I've been getting into binaries and low-end dev with minimal dependencies lately (basically a foreign concept a front-end native like me). It's really nice having something that won't disappear or change and you feel like you own.
Does Syncthing allow you to sync two differently-named folders as one folder while preserving the respective names? For example, your pictures folder may not be called the same thing on your phone and computer, or across Windows and Linux, or you want to sync game save files between Android and Windows.
I'm not sure it's supported in a GUI, but a simple symlink should do the trick. Say your photos folder is called Pictures on one machine, and Photos one another: just ln -s Pictures Photos (admitting you've moved all the contents of photos to Pictures and removed the Photos folder itself) and that should just work (didn't test).
You don't need to do this: when the folder is offered to share to another device, you can specify which folder it's going to go into locally. Internally everything is matched by folder ID codes.
Yes, it does. I do this for multiple folder- i.e. the pictures folder on my phone is called whatever android has decided to call it this year and it syncs with my pictures folder on my desktop.
I disagree about the Dropbox folder criticism. It's actually simplifies things a lot: whatever you put there is synced. Very straightforward. You won't break anything.
Syncthing, on the other hand, is far too geeky and easy to ruin things if you somehow sync a rapidly updated folder. I've moved a few people to Syncthing and in the end they all were not very happy with it for one reason or another. It also isn't as clear on the sync state of a file.
So the caveat is that concurrently writing to the same file on separate machines will lead to data corruption? Is that reproducible with other files than Minecraft saves, or can it be that Minecraft introduces corruption by amending the on-disk file with a delta of currently-running changes not accounting for the possibility that the file has been altered?
The same could happen with Dropbox/icloud, or even if you just ran two Minecraft instances with the same copy of the world on one machine - Minecraft worlds are not designed for concurrent modification by multiple processes
The same can't happen with Dropbox, because minecraft worlds are NOT stored in ~/Dropbox/ folder. That actually reinforces my point about a separate synced folder being a plus for most users, not a minus.
Third party launchers commonly used for modpacks, the first party dedicated server, changing the settings on the first party launcher are all ways you can put your saves in the Dropbox folder
> minecraft worlds are NOT stored in ~/Dropbox/ folder.
They certainly can be. They can be stored wherever you like, either if you are using a modding/hacking launcher (both are very popular for non-technical players, e.g. kids who aren't programmers) or if you are running a minecraft server (the vanilla server jar runs wherever you decide to put it, and doing this is not exactly an esoteric feat; again, nontechnical kids do it.)
Of course they can. However, person who knows simlink-fu and other tricks is unlikely to find himself in a situation when his data is corrupted by an incorrect Dropbox sync process. Syncthing's user, however, can screw himself with just a few mouse clicks, and I personally saw that happen.
So I myself simply sync just one folder with Syncthing, Dropbox style.
I don't think there's any meaningful difference in difficulty in finding the Minecraft folder in AppData to sync with Syncthing vs finding the option in the minecraft launcher to save to your Dropbox folder.
That doesn't argue that Dropbox won't also ruin actively updated folders. You're now talking about Minecraft configuration which is orthogonal to your original point.
This reinforces the original point, which was that Dropbox's constraints (only syncing one folder) are limiting to some while decreasing the probability of others getting themselves into problematic states.
And suffers from the same underlying problem as Dropbox and iCloud: it's profit-driven proprietary software. Are you sure your interests as a user are aligned with that of Microsoft as a company? Are you confident that will not change over time?
I'm confident they are at the moment. And i'm even more confident that if that ever changes, i'll find many people in a HN/Github thread whose interest are very much aligned with mine to fork the project and keep it going in a direction that suits us. Something you won't ever be able to do with a proprietary piece of software, unless of course you're willing to spend thousands of hours reverse-engineering a protocol they're going to change as soon as you achieve interop.
Im not, and thats why I don’t rely on it too much. For backup I rely on backblaze - and i pay them for the service. Microsoft OneDrive is clearly a loss leader as it is so cheap when used in a family plan it is crazy!
Im more worried about my gmail reliance to be honest.
Agreed. The recent Google Drive update which completely broke file metadata on Mac OS, along with it often times mounting twice, requiring a reboot to fix it, prompted me to test out all the different cloud storage solutions.
As stated in the OP Dropbox has become far too bloated, and just feels wrong. iCloud Drive only works 90% of the time, and doesn't play nice with apps that use absolute paths (even many Cocoa MacOS apps, which do this under the hood).
OneDrive is the only client that gets out of my way, just works, and even can automatically move unused files to the cloud without them disappearing from the filesystem entirely. It's crazy to think that Microsoft are making better Mac OS software than Apple.
I haven't considered going the self-hosted route yet, but Syncthing looks promising if you don't need folder sharing or robust iOS support.
Agreed. I use it as part of my personal Office 365 Enterprise account and it has been fairly annoyance free compared to Dropbox/iCloud.
I really only treat these cloud storage services as a sync and as an additional convenience backup. I don't ever think of them as primary storage.
Box is another popular one, which has SSO and other options to allow it's use in more restricted environments. It's at about the same annoyance level as OneDeive.
This guy mentions Drop Box being good in 2012...that's when I stopped using it due to the various bloat and annoyances.
The performance of the Mac client is terrible. Simply clicking the menu bar icon sometimes takes several seconds to bring up a terrible looking menu that smells like Electron.
I wouldn't necessarily call it terrible, as all the big 3 cloud storage services just wrap Web UIs in a window instead of doing a fully native app. The file syncing functionality works just fine for me every time, compared to iCloud, which regularly will stop syncing silently, despite it being better integrated into the Finder.
I'd rather have reliable syncing with an Electron wrapper than a fully native UI with broken sync.
I agree it's ok. One time it really bit me in the ass by silently deciding that some random old backup folder of my Desktop, residing on my Onedrive, should actually be my desktop folder and all of a sudden all this old crap appeared on my desktop. No fun was had undoing that. Even now my computer seems to be in some murky situation where my desktop is partly local and partly on my Onedrive, not cool, I haven't really looked into it and I really have no time for this.
I've been using Syncthing for about a year now, with multiple computers, and it's fantastic.
Dropbox has become a real pain over the years (I've also been a paid Dropbox customer for a long time):
* adding bloat and features that I don't want or need
* constantly nagging me for upgrades
* lying that the admin password is required (it isn't)
* consuming large amounts of CPU when anything changes anywhere in the filesystem
If Dropbox had a subscription plan where I could pay to get just the basic file sync functionality, with no nagging, no prompting, and optimized CPU usage, I would be happy to hand over my money. But it seems the company is run by the marketing team now.
I highly recommend syncthing as a very, very capable replacement.
Can you summarize what they maybe introduced/fixed to quell your old review?
> over the last week, syncthing has corrupted my git repository twice. I reported this as an issue, but the issue was immediately closed with an explanation I disagree with.
> I no longer trust syncthing until this gets resolved, unfortunately.
I sync a folder containing multiple git repos, never had a corruption happen.
@jwr can you link the issue you reported?
The only major issue I had was performance on TrueNAS/FreeBSD, Syncthing tends to keep a lot of files open. Increasing maxfiles/maxvnodes fixed it. [0]
On a sidenote, what's the point of these limits if your entire system becomes unusable if you reach them?
I had the exact same thing, granted, it was ten years ago already, with dropbox. Something something file versioning, ending up with neighboring files of conflicting revisions, the disease spreading over to all instances.
My takeaway was „never keep git repos in a system that does its own distributed versioning”. I just do git push around so my remotes are up to date. Git is sort of dropbox with a manual gearbox, after all (and more, but that’s not the point).
I’m really unsure that this is a kind of easily solvable problem for either dropbox or syncthing. Unless your syncing product gets smart enough to understand what a git repository is and how some files have to be checked in in lockstep, that is. Otherwise, expect race conditions.
For me, it solved all of the corruption problems with using Dropbox for git storage. In the end, I decided it was too heavy weight, and wanted to be able to clone my repo on machines that might not have anything other than "git" installed, but until I reached that point, I was a happy user of it.
I trust syncthing for singular/regular files but trusting something as complicated and interconnected as a git repo on it seems like a bad idea, especially if it's possible for two syncthing computers to act on the same git repo... I never really thought about trying it.
FWIW I sync my ~/code directory, which I work on quite regularly, between a few macs and linux machines using syncthing. I've never faced corruption and I've been using this setup for 2 or 3 years now. The directory holds about 150 git repos / 50GB / 500k files/folders.
Doing a quick search of reported issues around git repos, basically the trouble is that a git repo folder does not expect to be sync'ed at a file level, because repos are expected to be in different states on different devices, and the state of a repo is represented in the filesystem via the .git directory.
As an example problem-case, say your first device checks out branch A and your second device checks out branch B. Some files conflict, which is likely reasonably handled as any filesync application needs to handle conflicting updates. But any files that are changed in one and not in the other end up getting sync'ed over into the other, including inside your GIT directory, which will very likely lead to an illegal GIT directory state.
As a toy example, pretend that the .git internals require that at any given time, `.git/activebranch/` must contain exactly one file whose filename is the name of the active branch, and git maintains this invariant. By syncing changed files, any filesync utility would break this invariant by resulting in `.git/activebranch/` containing two files, due to each side seeing a new file that the other side needs to acquire. Depending on how such a problem might manifest in real life as a result of performing a file-based sync on two instances of a git directory, this could lead to your .git directory entering a state in which git doesn't know how to interpret it.
--
This isn't a syncthing problem as such, it's a problem of trying to use a file syncing application on the internal state of an application on two different machines with two different states, and expecting the result to be a valid state.
> it's a problem of trying to use a file syncing application on the internal state of an application on two different machines with two different states, and expecting the result to be a valid state.
I think that we can narrow the problem down to the fact that git's internal state is represented in a way that is transparent to syncthing (a folder of files), that it will attempt to sync, but only partially succeed. If git's internal state was a single database file, you wouldn't get corruption, just a syncthing conflict file.
I loved Dropbox. If they had a reasonably priced plan with like 50GB I would have subscribed in a heartbeat but they claimed it was impossible to do so all they offer now is like 2TB for $20 CAD a month which is far more than I need or want to pay. Then they neutered the free plan to 3 devices and I've mostly stopped using it.
I didn't know 1pass had changed. Glad I ditched them for Bitwarden a few years back. However I think bitwarden makes their backend complicated enough to scare off casual users of it like me who thought about hosting their own backend :) .
1password has effectively made 1password 7 the last version that will support local vaults on desktop, however, the mobile app has indefinite support for local vaults (since they don't want to segment their app users based on subscription / local).
Also: Questionable request for Accessibility permission. I was surprised to see Dropbox is still asking for this. I thought Apple put the hammer down on this behavior from apps long ago. You're supposed to only request Accessibility if it is for the purpose of supporting assistive technologies like VoiceOver, to help people with disabilities. Accessibility permissions have become something of a golden ticket or carte blanche for apps, allowing them to do just about anything on the operating system. Dropbox's dialog is vague about the purpose ("To get the most out of Dropbox") for requesting this superpower, so one can only assume that they are not doing it specifically to support assistive technologies like VoiceOver.
> consuming large amounts of CPU when anything changes anywhere in the filesystem
I don’t understand how this still hasn’t been addressed. On macOS at least (not sure about other platforms), rather than subscribing to filesystem events for the synced folders, it subscribes to FS events for literally everything and checks to see if each is relevant to the folders it should sync.
I can't exactly recommend to use Amazon services, but if you're doing it anyway something like s3fs-fuse [0] should do the trick so that S3 appears as a standard filesystem to syncthing.
I suppose I’ve just become used to saying no to a bunch of things when installing Dropbox, but once that’s done I have no interactions with it at all. I don’t remember being pestered at any point recently and I’ve got it on three devices. I keep waiting for it to annoy me in some way that causes me to find another solution but it’s yet to happen. I migrated several G Suite accounts to Fastmail recently so I suspect it’s my longest running paid subscription at this point.
Personally I've noticed how dropbox has gone from being a tool that fits well into a larger context, to instead trying to encompass the entire experience of touching synced files. For example, the number of clicks (and the precision of those clicks and the distractions around them) to open the dropbox folder in your OS's file explorer has gradually increased. And the amount of attention dropbox asks for, things like notifications and upsells, has also gradually increased.
The post aludes to this notion of not trusting dropbox to just do the one thing well, and instead expecting them to grab more and more (because they are financially driven to do so), which resonated very hard. It's so common for a good piece of software to get worse and worse, bit by bit, as the marketing and business downsides consume the technical upsides.
It’s one click to get to your Dropbox folder though, what’s changed there? Besides I just have some symlinks to point at it and I never think about manually going there again. I dunno if I’m just already on the top tier of subscriptions but I get zero upsell
or notifications after it’s installed and configured. I’d certainly love it if everything was off by default but it’s not really any hardship to go through the preferences.
Obviously the moment it stops just sitting there backing up my files and keeping them in sync across my devices I’d be bothered, but whatever they’re trying to force on people hasn’t really registered with me in a decade of use.
Coming from a similar background product-wise: the struggle is real, and while file syncing and sharing is great and achievable it only allows you N user retention/acquisition, and thus M profit potential. If you (or your investors) demand M*1000 of that, you start looking for a way to usurp not only the user's file syncing needs, but also document editing, calendars, email, contacts, backup... anything that is within reach. This is exacerbated by the fact that more and more vendors (such as Adobe and Figma) drive away from "files", which effectively forces you to use _their_ cloud storage solution. When you are Dropbox this is not a very sweet spot to be in, because Adobe has _both_ the apps for creating projects _and_ the storage/sync, but you only have storage/sync.
As long as cloud storage needs to grow exponentially at any cost, feature bloat of the kind Nikita is describing will continue. Most likely Syncthing is so good exactly because they do not have these extrinsic pressures.
Add to that the fact that in a bigger product org, the way you get promoted is by shipping "your" feature. Quite often it would be a feature no user would ever want, but it is much more the problem of selling "your" feature to management internally versus selling it directly to the user.
A great example of a solution which is local-first, allows you BYO cloud storage (of multiple kinds) and simply offers its own cloud storage at a small markup is Arq, and we are blessed that it does not have the pressures I have described above.
I'm about to drop Dropbox as well, due to how PITA it is to use.
Have a picture in your Dropbox you want to send as a message on Android? Well, just click Share button of course! Wait, no, that just shares a link not the image itself...
No, you gotta do "Open with...", find suitable image app (gallery or picture viewer), wait for it to download and open the image, and share from that app...
Though I imagine Dropbox would be glad to get rid of me, not being a paying customer. I don't need 2TB so $10/mo is way too much.
On my S8 it doesn't resize the images and instead just complains it's too large for MMS, so it's pretty useless like that. I know, who uses MMS these days...
All good and well, but isn't comparing Syncthing and Dropbox kind of apples to oranges?
Dropbox has a completely different scope that goes beyond what Syncthing is offering. If you don't require the features Dropbox offers, it's the wrong tool in the first place.
Both are solutions to sync a folder across multiple devices. The fact that dropbox added a bunch of extra features over the years is irrelevant to most users.
That's like arguing Photoshop is too complicated compared to MS Paint, because both can resize and crop images.
If all you want to do is syncing folders across devices, use a tool that does just that. Just to emphasize how this is relevant, here's what Dropbox says about what it is [0].
> What is Dropbox?
> Dropbox is a place where all your team’s content comes together.
There's nothing in that very first sentence of their own description of the product that suggests syncing folders across devices as a use case.
Just because you can do that, too, doesn't mean it's the primary use case.
Dropbox sees their product as a solution for collaborating across teams.
But that's only one side, so let's look at the "Personal Use"-section [1]:
> Back up your big ideas, your best memories and your family traditions.
And again, syncing folders across devices isn't mentioned - they see Dropbox as a centralised backup solution.
> Centralise your storage, declutter your life
So they tell the user exactly how they see their product and the laborious installation process is a consequence of that.
So while you see a product for syncing across devices, the product is advertised as a collaboration solution or a centralised storage for backup and device-independent global access.
That's not me claiming this either, this is straight from the horse's mouth and what users are told.
Dropbox wants users to see Dropbox as all these things because syncing is a table stakes feature for all their competitors now. So they add all these in search of a USP.
From my experience though, their users (and not just the tech bubble) don't care. Someone who even opens the Web interface to access a file without a client/app installed counts as a power user. In fact as more people use aaS that hide the concept of files, a lot of those users are dropping off as they have all their docs in Google docs or photos in iCloud or whatever
Huh. Perhaps. Last time I checked, Dropbox very much did claim that syncing data between devices was their primary use case. Perhaps they changed while I wasn't looking (I haven't been looking since they forced me to Syncthing by deciding that my filesystems weren't supported).
Perhaps the author is coming from a similar place. History matters. If someone sells you a product, and then completely changes the scope of said product, it's a bit odd for people to complain when you start looking for alternatives that fit the scope of the product you initially bought, no?
Is it apples and oranges or is it a nice pair of paper scissors compared to a multitool with the little fold-out scissors? Often we carry the multitool because it mostly meets lots of different needs. Sometimes we just want a single focused tool that does its one job really well.
There are lots of ways to sync or backup files and folders, each with their own quirks, advantages, and disadvantages. There's git, mercurial, OneDrive, Sharepoint, Dropbox, Google Drive/Google One, rsync, scp, NFS, Owncloud, Nextcloud, Crashplan, unison, snapback, Amanda, bacula, IDrive, Carbonite, Barracuda Backup, Veam, duplicity, DAR, bup, Acronis TrueImage, Veritas Backup Exec, HP Data Protector, Borg backup, Commodo Backup, and Windows/SAMBA shares over CIFS to name a minority of options. If every attempt met every need, there'd be far fewer tools for this.
The biggest gap I've noticed in SyncThing right now is that their iOS client doesn't work well. It's a third party solution you have to pay for, and it can't sync into app folders or the new iOS file paradigm introduced in recent versions of iOS.
I'd love to switch over to SyncThing, but it would be next to useless on my iPad Pro, so I'm stuck using the incredibly buggy NextCloud filesync solution.
After reading this, I did some digging, and found Mobius[1]. It seems to work really well, I just got the pro version. I think the last time I looked, I saw that Syncthing only officially had an android client and stopped looking after that.
To be fair, I still would really like an official, open source iOS client, instead of some closed source thing I need to trust; I'm still a bit uneasy about using it.
I guess you’re talking about the client “Möbius Sync”. I’ll add that it’s also not available in most countries, for some reason the developer restricted it in the App Store.
On MacOS, using Maestral.app made my Dropbox experience a lot better. Well, it's still dropbox behind, but the app itself is way lighter and nicer to use.
I love syncthing as much as the next guy on here, but this article feels very much like "why doesn't this square peg fit my round hole?!" Dropbox (and iCloud) are meant more for cloud file storage than pure device-to-device synce. I couldn't go to a less tech savvy friend and say, "Oh, you use Dropbox for backups? Here, use syncthing instead!" For one, most people don't have multiple computers they want to keep in sync, and even if they do it's unlikely that those multiple computers have overlapping periods of being online, which syncthing will need to keep back ups!
The author's baffling reaction to iCloud and Dropbox's warning just makes my point. Those warnings are super clear to people who use those services for backup - remove this file and it won't be backed up, hence not available elsewhere. That isn't those platforms begging for attention (???). Remove those warnings and see how many normies complain to Apple/Dropbox that they didn't know removing a file from their synced folder meant it wouldn't be synced anymore. This also applies to the bizarre complaint about cloud storage space limits - of course you need storage space on the service if those files have to live elsewhere!
Using "External File Versioning" on that page you could probably have it commit to a git repository. Then view that repository on GitHub or Gitlab to get nice diffs.
>Dropbox (and iCloud) are meant more for cloud file storage than pure device-to-device synce
Device to device sync was literally the value prop for Dropbox when it launched > 10 years ago. That's why I started using it.
Dropbox does now allow a more cloud-native file system / cloud backup usage pattern, but its roots are absolutely "keep my files in sync across several computers."
That was 10 years ago. Yes, I also started using it because of that. These days the the more important feature (at least for me) is being able to offload rarely used files from my drive. SSDs in MacBooks are still crazy expensive.
This comment thread is about how the author doesn’t seem to realize that the motive of these services has changed in response to fewer and fewer users wanting a sync solution versus a backup solution.
It's definitely not "fewer and fewer users wanting a sync solution" and instead "more and more computer users we can sell something to if we pile features into our overloaded app".
There's a criticism in there that I share because I miss when things were more modular, but I also understand and appreciate the value and opportunities that modern tech is bringing to people.
The back and forth chaos of progress and foundation-building is important but frustrating.
Some were. But your point is valid; a company will move to what _most_ customers want. A FOSS application can keep servicing the (perhaps smaller) original crowd of users.
Well, above all, it's super convenient. The files stay where they are, you just mark them as "online only" and they don't take up space on your disk.
When you click to open such file or folder, it will immediately download back to your disk and you can use it again. You will lose maybe a few seconds once in a blue moon when you need one of them. But you easily gain hundreds of gigabytes of space on your SSD. Just yesterday I right-clicked some folders to mark them "online only" and immediately gained +300GB free space..
Plus of course you can access those files from your phone or ipad too, when needed. Basically I personally have all my work files in Dropbox and don't use additional backup (used Backblaze in the past).
External drives are a thing of history. Too much hassle, too much inconvenience, can't access anytime and anywhere I want, etc. What's the benefit of external drive compared to Dropbox really?
Dropbox works well for small files that you need to share with others. However, if you use it to back up large amounts of data ... say >100GB, the experience of using it is very poor. It consumes large amounts of CPU continually. It takes forever to run to completion (literally in some cases, in my case it ran for weeks before I gave up on it). An external hard drive is of comparable cost to Dropbox, and a one-time cost, not something you need to pay for, forever. It runs to completion in 20 minutes. You can keep multiple versions of it in different physical locations. It's conceptually simple with fewer points of failure.
Personally I tried using several different cloud backup solutions, but I gave up on them. A few encrypted external hard drives, updated every month or two in a repeatable way (e.g. a bash script), one at home, one at work ... and backup is a solved problem. Of course, to each their own.
Then you had some problem. I have ~1TB in Dropbox, files small and large. No problems with sync. Once in a while it does go crazy, eats up a lot of CPU and takes 1 hour to get in proper sync. It’s annoying, but luckily it’s rare. And it goes with me everywhere. How do you access external drive from your phone anyway?
The point of the external hard drives is really just to have a backup. I don't have a 1TB collection of files that I need to access from multiple places. So I would never try to access my big backup from my phone. I don't do that kind of work from my phone, ever. If you're in that situation, something like Dropbox might help, provided that it doesn't eat all your CPU. Good for you! Glad it's working out.
It's cheaper and faster. What's the hassle or inconvenience of a separate drive? If anything, Dropbox is less convenient because it requires installing an application to use it.
The only drawback is that you have to carry it around anywhere you want to access it. A modern external drive is the size of a phone though, so that's hardly a dealbreaker even if you do want to carry it around. Although frankly, cloud storage is much better for filesharing.
I guess it’s just about everyone’s slightly different scenarios. My typical example: I have all my photos on Dropbox. Some of them a decade old. Don’t need them taking up SSD space. Don’t need to carry them around. But once in a blue moon I want to see a decade old photo from my phone while I’m on the other side of planet. Same for my ebooks, older work files, tax returns, etc, etc.
>>Dropbox (and iCloud) are meant more for cloud file storage than pure device-to-device synce
> Device to device sync was literally the value prop for Dropbox when it launched > 10 years ago.
It's what was advertised. I find it interesting that its initial reception on HN is so frequently slammed for saying "we could already do this, if we wanted to", and then... 10+ years later, Dropbox has had to switch to doing something people want.
The only value I ever saw people getting from Dropbox was in using it as a way to hand off files to someone else across the internet. Which is a use case Dropbox has spent many years fighting against.
It's crazy how much I enjoy simple software. No crazy complicated APIs and complex installs. Just give me a binary I can call with a command line interface. I can play with it interactivity and easily automated whenever needed.
The only thing I ever use Dropbox for is requesting files from people that are too large or that their corporate Exchange environment prevents from being sent over email.
>The only value I ever saw people getting from Dropbox
Your exposure to Dropbox users is very very limited, then. I only very, very rarely use it as a medium to share. OTOH, I use it CONSTANTLY as a every-device file system, i.e. across my main machine, my backup, my iOS devices, and remote systems I work from. The sync is everything to me.
10 years ago, Dropbox was still storing your files on their servers so that your devices did not need to be online and accessible at all times in order to sync. It still operated in a fundamentally different manner than Syncthing. All that has really changed about the app is the marketing around it.
Dropbox still has my files on its servers. Not sure what you mean here. If all my devices are off, I can access anything in my Dropbox account from any computer's browser.
Startup idea: provide cloud backup service through Syncthing. End users install Syncthing and then add a remote host provided by your company that is bound to their account. That instance just keeps a copy of everything it receives on some cloud infrastructure. Slap a web interface and some modest data retention on it to recover files.
That's software done _only_ with end users in mind. It's the same model as redhat uses. IT companies suffer from the fact that they resist changing the design of the product s.t. the company offers much less value in the equation.
That will probably ruin everything the author liked about it in this post, because then you will have to add account creation, verification, ans sign-in steps. The horror!
You can use Syncthing for "cloud" storage as long as you keep an instance online at all times. Syncthing's functionality is a superset of Dropbox etc. People absolutely use Dropbox for device-to-device sync. They are not necessarily interested in the "cloud" backup but just want to send a file to a mate.
Yes, except for "online-only" files and sharing files with others, which don't really work with Syncthing. But I use Syncthing heavily and have no use for those.
There is no reason why syncthing cannot work on both a server in the "cloud" and an end-user device. Syncing files is a well researched and solved problem, at this point dropbox et al are not selling syncing, they are reselling IaaS + integration and gaffataping on anything else tangentially related they can conceive of to squeeze money out of users.
The fundamental value is already there to have for free and at a much higher UX quality, while paying for the IaaS directly... if only non-technical users were aware of it, and could access it as easily.
Exactly. It worked perfectly to sync my Android tablet data on my NAS, but I never tried it as a cloud file server. However it should be relatively easy to set up a VPN on a broadband (non NATted) connection, if it has a dynamic IP hook it to a free domain name using for example DuckDNS (.org), then redirect the VPN traffic to a *PI or similar board and a USB disk that will stay up 24/7, so that no matter when or from where, any other device with the right credentials can use that space as cloud storage from anywhere. The big difference being that there are no strings attached, no size caps, no sign this and that, no download of closed apps, no ads etc.
Dropbox is the entity who tries to fit the "extract money from the users" peg to this file sync functionality.
The cloud aspect of their solution can also be interpreted as "we need an excuse to examine all of your files while we are at it". So we can also get money from NSA or other three letter agencies.
I tried many sync solutions (Syncthing, SparkleShare, Resilio, Mega, Seafile, NextCloud) and I almost always experienced some kind of data corruption.
Syncthing was no exception.
I could not reproduce the problem, so I never made a bug report.
The corruption was something like this:
- I recursively renamed all files from a folder from uppercase to lower case. Suddenly I duplicated files, uppercase and lower case.
- I did not use a device for a long time (a year or so). After switching the old device on, it "recovers" deleted files.
- I put a git repository into a sync folder. It stopped working after a while.
This, however, could be a solved problem by now, as it is 2+ years ago, that I tried syncthing.
Am I the only one with that kind of Problems?
Has the situation improved?
I tried Syncthing many years ago and had similar issues. I was going to try it again a few months ago but memory usage and initial scam speed weren't good.
Resilio worked pretty well for me for years but now gets stuck with a weird SQLite error
I really wanted to love syncthing a lot, I loved the idea, the interface, everything. But I would always get a number of errors on some files for whatever reason that I would have to then manually fix and sometimes I couldn't even do that. But it has been a few years, so maybe it's time to try again...
I've been using resilio sync for years without corruption. If you're seeing corruption across half a dozen different systems, I wonder if you might have a hardware problem.
If a synchronization software can't handle a git repository, what else is it not able to handle?
After all, a git repository is just a bunch of files and in my case not particular big once.
It’s a consistency problem. Syncthing provides reasonably solid single file consistency. Git repos and anything else that requires multi-file consistency is trickier, since Syncthing doesn’t know about the internal consistency requirements.
If you use Syncthing peer-to-peer and/or have concurrent modifications to different peers before a full sync, you can run into problems. It’s not really designed for that, it’s optimised from the single user sharing individual files on multiple devices case.
Doing anything better here is incredibly hard in a peer-to-peer system on the level of abstraction that Syncthing operates on.
I think renames might be/may have been problematic because it tries to be smart and handle them efficiently. If it did the naive thing of delete followed by creation, consistency would be better, but performance would be worse.
I have been using Syncthing for some time and it has been excellent. The only downside is it actively tries to establish P2P connection when I have a central server with a public IP. Other than that it's great.
I just keep all my text files in a git repository and use custom elisp on top of magit to quickly commit all of them, I have a custom go syncronizer that would symlinks the text files around my filesystems and makes sure all my system is sane.
for the binaries stuff, I would use maybe sometime syncthing but that's more a backup thing, i don't usually need all of them available on all my laptops/ws
for real backup i just incrementally dump my btrfs snapshot to a remote USB disk every hour.
I also absolutely love Syncthing. I have been using it to keep my keepass database synced between devices for a few years now, and have had zero issues.
edit: ignore this, the software can relay connections when direct connection is impossible.
Just found this in the FAQ:
> If you see outgoing connections to odd and unexpected addresses these are most likely connections to relay servers. Relay servers are run by volunteers all over the world. They usually listen on ports 443 or 22067, though this is controlled by the user running it. You can compare the address you are concernced about with the current list of active relays. Relays do not and can not see the data transmitted via them.
"Syncthing relies on a discovery server to find peers on the internet. Anyone can run a discovery server and point Syncthing installations to it. The Syncthing project also maintains a global cluster for public use."
264 comments
[ 3.1 ms ] story [ 48.7 ms ] threadOne cool feature is also encrypted syncing where some nodes dont even have the key (ie VPS syncs but doesnt know what it syncs, and only your devices have the key)
Also, performance (read bandwidth usage) isn't all that great with NextCloud. PHP-FPM was more designed for concurrent execution of multiple stateless requests server-side (PHP) than for sending raw bytes as soon as possible to the disk.
https://news.ycombinator.com/item?id=23537243 (June 2020, 159 comments)
---
Recent discussion about the software: "Syncthing – a continuous file synchronization program"
https://news.ycombinator.com/item?id=28859521 (85 days ago, 230 comments)
Syncthing, on the other hand, is far too geeky and easy to ruin things if you somehow sync a rapidly updated folder. I've moved a few people to Syncthing and in the end they all were not very happy with it for one reason or another. It also isn't as clear on the sync state of a file.
Do you have a link/resource elaborating on these questions?
They certainly can be. They can be stored wherever you like, either if you are using a modding/hacking launcher (both are very popular for non-technical players, e.g. kids who aren't programmers) or if you are running a minecraft server (the vanilla server jar runs wherever you decide to put it, and doing this is not exactly an esoteric feat; again, nontechnical kids do it.)
So I myself simply sync just one folder with Syncthing, Dropbox style.
Im more worried about my gmail reliance to be honest.
As stated in the OP Dropbox has become far too bloated, and just feels wrong. iCloud Drive only works 90% of the time, and doesn't play nice with apps that use absolute paths (even many Cocoa MacOS apps, which do this under the hood).
OneDrive is the only client that gets out of my way, just works, and even can automatically move unused files to the cloud without them disappearing from the filesystem entirely. It's crazy to think that Microsoft are making better Mac OS software than Apple.
I haven't considered going the self-hosted route yet, but Syncthing looks promising if you don't need folder sharing or robust iOS support.
I really only treat these cloud storage services as a sync and as an additional convenience backup. I don't ever think of them as primary storage.
Box is another popular one, which has SSO and other options to allow it's use in more restricted environments. It's at about the same annoyance level as OneDeive.
This guy mentions Drop Box being good in 2012...that's when I stopped using it due to the various bloat and annoyances.
I'd rather have reliable syncing with an Electron wrapper than a fully native UI with broken sync.
Dropbox has become a real pain over the years (I've also been a paid Dropbox customer for a long time):
* adding bloat and features that I don't want or need
* constantly nagging me for upgrades
* lying that the admin password is required (it isn't)
* consuming large amounts of CPU when anything changes anywhere in the filesystem
If Dropbox had a subscription plan where I could pay to get just the basic file sync functionality, with no nagging, no prompting, and optimized CPU usage, I would be happy to hand over my money. But it seems the company is run by the marketing team now.
I highly recommend syncthing as a very, very capable replacement.
> over the last week, syncthing has corrupted my git repository twice. I reported this as an issue, but the issue was immediately closed with an explanation I disagree with.
> I no longer trust syncthing until this gets resolved, unfortunately.
https://news.ycombinator.com/item?id=23660438
—
I would absolutely be syncing directories containing git repos, for better or worse
@jwr can you link the issue you reported?
The only major issue I had was performance on TrueNAS/FreeBSD, Syncthing tends to keep a lot of files open. Increasing maxfiles/maxvnodes fixed it. [0]
On a sidenote, what's the point of these limits if your entire system becomes unusable if you reach them?
[0] https://jira.ixsystems.com/browse/NAS-104534
My takeaway was „never keep git repos in a system that does its own distributed versioning”. I just do git push around so my remotes are up to date. Git is sort of dropbox with a manual gearbox, after all (and more, but that’s not the point).
I’m really unsure that this is a kind of easily solvable problem for either dropbox or syncthing. Unless your syncing product gets smart enough to understand what a git repository is and how some files have to be checked in in lockstep, that is. Otherwise, expect race conditions.
https://github.com/anishathalye/git-remote-dropbox
For me, it solved all of the corruption problems with using Dropbox for git storage. In the end, I decided it was too heavy weight, and wanted to be able to clone my repo on machines that might not have anything other than "git" installed, but until I reached that point, I was a happy user of it.
I'm not aware of a similar tool for syncthing.
As an example problem-case, say your first device checks out branch A and your second device checks out branch B. Some files conflict, which is likely reasonably handled as any filesync application needs to handle conflicting updates. But any files that are changed in one and not in the other end up getting sync'ed over into the other, including inside your GIT directory, which will very likely lead to an illegal GIT directory state.
As a toy example, pretend that the .git internals require that at any given time, `.git/activebranch/` must contain exactly one file whose filename is the name of the active branch, and git maintains this invariant. By syncing changed files, any filesync utility would break this invariant by resulting in `.git/activebranch/` containing two files, due to each side seeing a new file that the other side needs to acquire. Depending on how such a problem might manifest in real life as a result of performing a file-based sync on two instances of a git directory, this could lead to your .git directory entering a state in which git doesn't know how to interpret it.
--
This isn't a syncthing problem as such, it's a problem of trying to use a file syncing application on the internal state of an application on two different machines with two different states, and expecting the result to be a valid state.
I think that we can narrow the problem down to the fact that git's internal state is represented in a way that is transparent to syncthing (a folder of files), that it will attempt to sync, but only partially succeed. If git's internal state was a single database file, you wouldn't get corruption, just a syncthing conflict file.
I love it, and turned on in Signal as well. Otherwise I'll forget it eventually.
Not offering reasonable storage for a reasonable price is a way to keep users that they don’t want away from their service.
Just like 1Password. They are extremely non subtle in pushing users that want local vault away from their customer base.
I don’t understand how this still hasn’t been addressed. On macOS at least (not sure about other platforms), rather than subscribing to filesystem events for the synced folders, it subscribes to FS events for literally everything and checks to see if each is relevant to the folders it should sync.
Could it be configured to sync to blob storage on s3 since ingest is free?
[0] https://github.com/s3fs-fuse/s3fs-fuse
Location: Norway.
The post aludes to this notion of not trusting dropbox to just do the one thing well, and instead expecting them to grab more and more (because they are financially driven to do so), which resonated very hard. It's so common for a good piece of software to get worse and worse, bit by bit, as the marketing and business downsides consume the technical upsides.
Obviously the moment it stops just sitting there backing up my files and keeping them in sync across my devices I’d be bothered, but whatever they’re trying to force on people hasn’t really registered with me in a decade of use.
As long as cloud storage needs to grow exponentially at any cost, feature bloat of the kind Nikita is describing will continue. Most likely Syncthing is so good exactly because they do not have these extrinsic pressures.
Add to that the fact that in a bigger product org, the way you get promoted is by shipping "your" feature. Quite often it would be a feature no user would ever want, but it is much more the problem of selling "your" feature to management internally versus selling it directly to the user.
A great example of a solution which is local-first, allows you BYO cloud storage (of multiple kinds) and simply offers its own cloud storage at a small markup is Arq, and we are blessed that it does not have the pressures I have described above.
Have a picture in your Dropbox you want to send as a message on Android? Well, just click Share button of course! Wait, no, that just shares a link not the image itself...
No, you gotta do "Open with...", find suitable image app (gallery or picture viewer), wait for it to download and open the image, and share from that app...
Though I imagine Dropbox would be glad to get rid of me, not being a paying customer. I don't need 2TB so $10/mo is way too much.
Dropbox has a completely different scope that goes beyond what Syncthing is offering. If you don't require the features Dropbox offers, it's the wrong tool in the first place.
If all you want to do is syncing folders across devices, use a tool that does just that. Just to emphasize how this is relevant, here's what Dropbox says about what it is [0].
> What is Dropbox?
> Dropbox is a place where all your team’s content comes together.
There's nothing in that very first sentence of their own description of the product that suggests syncing folders across devices as a use case.
Just because you can do that, too, doesn't mean it's the primary use case. Dropbox sees their product as a solution for collaborating across teams.
But that's only one side, so let's look at the "Personal Use"-section [1]:
> Back up your big ideas, your best memories and your family traditions.
And again, syncing folders across devices isn't mentioned - they see Dropbox as a centralised backup solution.
> Centralise your storage, declutter your life
So they tell the user exactly how they see their product and the laborious installation process is a consequence of that.
So while you see a product for syncing across devices, the product is advertised as a collaboration solution or a centralised storage for backup and device-independent global access.
That's not me claiming this either, this is straight from the horse's mouth and what users are told.
[0] https://www.dropbox.com/features
[1] https://www.dropbox.com/plus
From my experience though, their users (and not just the tech bubble) don't care. Someone who even opens the Web interface to access a file without a client/app installed counts as a power user. In fact as more people use aaS that hide the concept of files, a lot of those users are dropping off as they have all their docs in Google docs or photos in iCloud or whatever
Perhaps the author is coming from a similar place. History matters. If someone sells you a product, and then completely changes the scope of said product, it's a bit odd for people to complain when you start looking for alternatives that fit the scope of the product you initially bought, no?
There are lots of ways to sync or backup files and folders, each with their own quirks, advantages, and disadvantages. There's git, mercurial, OneDrive, Sharepoint, Dropbox, Google Drive/Google One, rsync, scp, NFS, Owncloud, Nextcloud, Crashplan, unison, snapback, Amanda, bacula, IDrive, Carbonite, Barracuda Backup, Veam, duplicity, DAR, bup, Acronis TrueImage, Veritas Backup Exec, HP Data Protector, Borg backup, Commodo Backup, and Windows/SAMBA shares over CIFS to name a minority of options. If every attempt met every need, there'd be far fewer tools for this.
Asking as a Nextcloud (file only) user considering making a switch.
To be fair, I still would really like an official, open source iOS client, instead of some closed source thing I need to trust; I'm still a bit uneasy about using it.
[1] https://www.mobiussync.com/
[1]: https://syncthing.net/downloads/
The author's baffling reaction to iCloud and Dropbox's warning just makes my point. Those warnings are super clear to people who use those services for backup - remove this file and it won't be backed up, hence not available elsewhere. That isn't those platforms begging for attention (???). Remove those warnings and see how many normies complain to Apple/Dropbox that they didn't know removing a file from their synced folder meant it wouldn't be synced anymore. This also applies to the bizarre complaint about cloud storage space limits - of course you need storage space on the service if those files have to live elsewhere!
* keep x previous revisions, or
* keep old revisions for x time, or
* diy with hook script
https://docs.syncthing.net/users/versioning.html
Device to device sync was literally the value prop for Dropbox when it launched > 10 years ago. That's why I started using it.
Dropbox does now allow a more cloud-native file system / cloud backup usage pattern, but its roots are absolutely "keep my files in sync across several computers."
That's exactly what the article author is saying.
There's a criticism in there that I share because I miss when things were more modular, but I also understand and appreciate the value and opportunities that modern tech is bringing to people.
The back and forth chaos of progress and foundation-building is important but frustrating.
> SSDs in MacBooks are still crazy expensive.
It sounds like an archival usage rather than syncing or sharing usage.
When you click to open such file or folder, it will immediately download back to your disk and you can use it again. You will lose maybe a few seconds once in a blue moon when you need one of them. But you easily gain hundreds of gigabytes of space on your SSD. Just yesterday I right-clicked some folders to mark them "online only" and immediately gained +300GB free space..
Plus of course you can access those files from your phone or ipad too, when needed. Basically I personally have all my work files in Dropbox and don't use additional backup (used Backblaze in the past).
Personally I tried using several different cloud backup solutions, but I gave up on them. A few encrypted external hard drives, updated every month or two in a repeatable way (e.g. a bash script), one at home, one at work ... and backup is a solved problem. Of course, to each their own.
The only drawback is that you have to carry it around anywhere you want to access it. A modern external drive is the size of a phone though, so that's hardly a dealbreaker even if you do want to carry it around. Although frankly, cloud storage is much better for filesharing.
> Device to device sync was literally the value prop for Dropbox when it launched > 10 years ago.
It's what was advertised. I find it interesting that its initial reception on HN is so frequently slammed for saying "we could already do this, if we wanted to", and then... 10+ years later, Dropbox has had to switch to doing something people want.
The only value I ever saw people getting from Dropbox was in using it as a way to hand off files to someone else across the internet. Which is a use case Dropbox has spent many years fighting against.
A work of poetry.
> A major initial motivation for both the ARPANET and the Internet was resource sharing.
Your exposure to Dropbox users is very very limited, then. I only very, very rarely use it as a medium to share. OTOH, I use it CONSTANTLY as a every-device file system, i.e. across my main machine, my backup, my iOS devices, and remote systems I work from. The sync is everything to me.
It's great, it's even better than cloud storage, since the VPS never has access to the encryption key.
[1] https://docs.syncthing.net/branch/untrusted/html/users/untru...
Syncing is not a square peg, monetisation is.
There is no reason why syncthing cannot work on both a server in the "cloud" and an end-user device. Syncing files is a well researched and solved problem, at this point dropbox et al are not selling syncing, they are reselling IaaS + integration and gaffataping on anything else tangentially related they can conceive of to squeeze money out of users.
The fundamental value is already there to have for free and at a much higher UX quality, while paying for the IaaS directly... if only non-technical users were aware of it, and could access it as easily.
Dropbox is the entity who tries to fit the "extract money from the users" peg to this file sync functionality.
The cloud aspect of their solution can also be interpreted as "we need an excuse to examine all of your files while we are at it". So we can also get money from NSA or other three letter agencies.
The corruption was something like this:
- I recursively renamed all files from a folder from uppercase to lower case. Suddenly I duplicated files, uppercase and lower case.
- I did not use a device for a long time (a year or so). After switching the old device on, it "recovers" deleted files.
- I put a git repository into a sync folder. It stopped working after a while.
This, however, could be a solved problem by now, as it is 2+ years ago, that I tried syncthing.
Am I the only one with that kind of Problems? Has the situation improved?
Resilio worked pretty well for me for years but now gets stuck with a weird SQLite error
If a synchronization software can't handle a git repository, what else is it not able to handle? After all, a git repository is just a bunch of files and in my case not particular big once.
If you use Syncthing peer-to-peer and/or have concurrent modifications to different peers before a full sync, you can run into problems. It’s not really designed for that, it’s optimised from the single user sharing individual files on multiple devices case.
Doing anything better here is incredibly hard in a peer-to-peer system on the level of abstraction that Syncthing operates on.
I think renames might be/may have been problematic because it tries to be smart and handle them efficiently. If it did the naive thing of delete followed by creation, consistency would be better, but performance would be worse.
for the binaries stuff, I would use maybe sometime syncthing but that's more a backup thing, i don't usually need all of them available on all my laptops/ws
for real backup i just incrementally dump my btrfs snapshot to a remote USB disk every hour.
works well for me so far,
I've lost patience with such products, and instead I tradeoff fewer features for more control. This applies to hardware too.
Just found this in the FAQ:
> If you see outgoing connections to odd and unexpected addresses these are most likely connections to relay servers. Relay servers are run by volunteers all over the world. They usually listen on ports 443 or 22067, though this is controlled by the user running it. You can compare the address you are concernced about with the current list of active relays. Relays do not and can not see the data transmitted via them.
"P2P swarm" sounds about right?
https://docs.syncthing.net/users/stdiscosrv.html
"Syncthing relies on a discovery server to find peers on the internet. Anyone can run a discovery server and point Syncthing installations to it. The Syncthing project also maintains a global cluster for public use."