This seems superior as it provides more advanced configuration and no limits with the number of parties you can invite. It appears to be free. And no central server. Transfers were painfully slow with Aero, I assume this would be faster.
> A de facto standard,
> potential opensource clients,
> not tied to a specific company,
When the source is closed, and the protocol is closed + encrypted? I think you may be a little optimistic... (Though I do hope that either the source or the protocol get opened)
There are a few advanced configurations where BTSync trumps Aero (at least last time I used it - someone, please correct me if these have been implemented).
- One way sync
- Selective device LAN sync (so you can tell it to only sync over LAN)
- Self destructing one-time share keys
- Less-friction on sharing - using a key based copy and paste system
- Powered by BT (this may not be an advantage though since BT is throttled on quite a number of providers through DPI worldwide
Nor PowerPC, which apparently many NAS units are using. Mine is a WD My Book Live Duo running Debian/PowerPC.
Additionally, AeroFS relies on Java, which is pretty big dependency (read: pain) to have on resource-restricted environments like a tiny NAS box. The daemon process of AeroFS consumes about 110MB of memory (I guess due to JRE), but my NAS unit has only 256MB memory in total with less than 100MB free, which means even if AeroFS supports PowerPC, there is still no way I could run it smoothly on my NAS without killing its performance.
Bittorrent Sync supports all of x86/x64/ARM/PowerPC, and the download is just a single 3.7MB binary without dependency other than glibc. Running it on my NAS shows it consumes less than 10MB memory. This is just so much nicer for deployment on a wider selection of devices.
I believe the use cases are different, which drives a difference in at least the user experience. AeroFS is designed for business use, so we strive to balance simplicity in UI/UX with security/privacy while trying to avoid things like side-channel sharing of secrets. We focus on functionality like managing your team, the ability to remotely wipe devices, control sharing within your organization, and so on, while providing the "Dropbox experience" that end users within the organization ask for, and without us storing any of the data itself.
I agree that the main gripe with AeroFS that people have voiced in this thread is certainly valid. AeroFS _did_ slow down in performance for some users recently. We actually just released a version today that addresses this bug (0.4.181 - see http://ae.ro/Ln2YJJ for the release notes, but it specifically had to do with the way we were initializing our jingle library), and in our own internal tests the performance has improved dramatically.
Since there is no central server, doesn't that mean that one device will have to be always on? That would add a degree of overhead and complication that services like Dropbox don't have.
There's a rendezvous server that also serves as a fallback relay in case two devices that want to sync are behind highly-unfriendly NATs. In that sense, it's exactly like Dropbox.
But it doesn't store your data, so yes, one device must always be on in order to sync with a new device. However, if you have a device that needs changes made on another one, it will just wait around and sync whenever the device with changes comes back online.
There have been stories on a few of these 'unlimited sync' services in the past few months. While it's a good idea, I really would prefer to have a little space available on a central server.
For me to consider switching to this service from Dropbox, it would have to be able to temporarily store some of my files from my work computer so that they could be transferred to my home computer later on when that computer was started up (for example). That is the feature that needs to be implemented for this to truly become a Dropbox-killer.
That might very well rule Sync out for personal use by the general public. The benefit of Dropbox (and similar server backed services) is that the most up-to-date version of a file is simply there as soon as you are connected to the Internet. Most people who have two computers won’t be using them both at the same time. For example, a common use case for these services is syncing files while you are traveling. Requiring a user who edits files on a laptop to turn their laptop on when they get back to the home/office in order for the changes to be recognize on their desktop seems like a big step backwards. In that use case Sync would just be a glorified USB thumb drive.
Unless you have a tools folder with a ton of files and Dropbox has no way to know which ones are the ones you want now. It can take up to a day to sync a large amount of files in my experience.
If I have to devise some system to zip my important stuff and keep that up to date, then I may as well just use BTSync for free.
I haven't yet looked to see how this service works so correct me if I'm wrong, but it seems that you could solve this problem by either leaving the desktop turned on (so that it syncs immediately whenever the laptop changes), or coordinating with the IT department and having them do the same using a file server the desktop will have access to (which IT departments will like because it makes it easier to centralize backups).
This is indeed the solution. If you need the feature "needs to work without needing both computers on at the same time", add a third computer that IS always on. Might cost you a small amount of money on say EC2 or something, but probably cheaper than drop box.
That is an acceptable solution for the HN community, that is not a solution for the general public. Dropbox has reached wider success because you don't need to have an EC2 or know what one is in order to use Dropbox.
This project is useless if they don't figure out a way to serve files without both machines being turned on, there is also the problem of syncing with multiple clients beside that.
I see two key use cases that could cause trouble:
1)a co-worker turned his computer off and went home from work, the syncing couldn't complete even if only a single doc was left
2)Because there is no central server if all of my coworkers in a distributed team sync with my new shared folder at the same time, it is required that I own a connection capable of supporting that kind of traffic, because it would be almost the same as streaming a video to multiple users, not to mention that I would probably have to avoid to browse the web for pretty much anything else in the meanwhile.
I'm not sure what you're getting at with 1- I don't think any service ever can transfer files to a machine that isn't on. As soon as the co-worker shows up on Monday, the transfer will finish.
As far as 2 is concerned though, that is not the case. Don't forget, it's built on top of BT. As each piece of the file gets transferred to other clients by the original machine, the other clients will be able to send those pieces out too. In theory it will be faster than a transfer from a central server.
1-Actually Dropbox lets everyone do just that, I can access the shared folder via the dropbox website whenever I want, download files in there and even access the old or deleted versions..something I see very unlikely in this case for now
Oh I see what you're saying. Bottom line, there needs to be at least 1 copy of each piece available to pull. It also definitely lacks a lot of the features that Dropbox provides on top of file syncing.
No, it will only download copies of data you are interested in. All parties in possession of the same file act as distributed backups of that file, since it is bittorrent.
We're looking at AeroFS and BitTorrent Sync for sharing sensitive files with our team members. We're leaning towards BS mainly because this feels completely P2P, whereas AeroFS may become useless if the company shuts down. Is that right? I'd love to hear from someone who has used both of these about stability, sync conflicts, CPU load etc. Also would love to hear about plans to open source BS.
I'm looking for something like this too. My big concerns of a Dropbox alternative is:
* Ability to handle large files (300MB incompressible Photoshop) and able to handle large complete data stores. (1 TB, which is impossible with Dropbox)
* Selectability for certain folders on certain users (no need for all data to be on all machines)
* The ability to 'archive' files to only the host.
* Good LAN networking with host so that the portable machines (iPads, Retina Macbook Pros with puny 128GB HDD) can use it fine.
I think either of them would rapidly become useless if the company shut down -- if it's not being kept updated, the clients will cease to work on newer OSes, etc. I wouldn't worry too much about the differences.
Maybe both BT and AeroFS should help customers mitigate that risk by committing to open source their code if they shut down. AeroFS has a revenue model, though, so I'd be less concerned about them shutting down this product.
I did. But, some of our team-members are non-technical. And for work-reasons, some of them are going to be using Windows. Both of those reasons meant that git-annex wasn't a serious contender for our use-case (not yet, anyway).
>BitTorrent Sync’s functionality is comparable to services such as Dropbox and Skydrive, except for the fact that there’s no cloud involved. Users sync the files between their own computers and no third-party has access to it.
> It is an ideal tool for people who want to share large amounts of data between computers without going through third-party services.
>The Sync application is available for Windows, OSX, Linux and has the ability run on NAS devices through a web-interface. Readers who are interested in giving it a spin can head over to BitTorrent labs [1], where the Sync app can be downloaded.
I'm more interested in the TLDR of how this works. Due to firewalls and such, it's technically impossible to have a peer to peer network without requiring port forwarding on at least one side. I wonder if they're really requiring that from users that want to use this.
Another way would be to do TCP or UDP hole punching, but that involves a third party for initial setup. Probably possible and probably safe, but I'd like to see a security review of that.
Really nice, but the thing is... Dropbox has gone beyond just file sharing.
One example: I used to use Flickr for photo sharing, but cameras got better, images got bigger, and I have a lot of photos. I moved from Flickr to Picasa as it could cope with the directories full of photos and I didn't have to manually upload them and Google's storage space was cheaper. Then I ran out of space... over 100GB of photos, where next?
And I've told a few photographers about this, and a few weeks later a friend of a friend of a friend excitedly told me on a forum how you can photo share in Dropbox.
And what I'm basically seeing is that the problem of "file sync" is being considered as solved by lay consumers, who really aren't prioritising encryption, and the problems that they now have is "share this directory of photos", and "share that directory of videos", and "sync this music privately, but let me play it back".
Dropbox isn't just file sync anymore.
What it is, is a serious threat to Flickr, Picasa, YouTube, Amazon MP3 Locker, Google Play Music, iTunes, etc.
And consumers are not thinking in terms of encrypted sync, they're just thinking in terms of "I just want to do X, why is it so hard", and so I can't see this (very nice) solution really solving the problems that consumers have, that will make them prioritise security.
I would respectfully disagree with what you seem to be implying.
This service would mostly be used by us who are wary about trusting Google/Microsoft/Dropbox and the judgement of their automated decision systems with our life . Yesterday there was a HN story about a guy who got kicked off Google Docs because an automated program was trolling/mining through their users files and its judgement was not to be questioned by mere mortals.
What if I put a picture of me and my kid playing in the swimming pool -- what is to prevent Dropbox/Google/Skydrive from tagging it as offensive and setting an overzealous DA or social-worker upon my family ? People have been arrested/detained at border crossings for perfectly innocuous pictures like these.
Therefore if you think privacy is only important for people stealing stuff, your are being naive.
Drop a TrueCrypt partition into your Dropbox [1], and you get the Dropbox QoS as well as the encryption for your kids' swimming pool photos. Plus, for non-swimming photos, you can just use the regular Dropbox folder.
I am not looking to hide stuff; I merely do not want my stuff accessed by third parties without either a search warrant or my explicit consent on a file by file basis. BitTorrent Sync seems to have a cool solution for me; my home desktop is almost always on.
> Yesterday there was a HN story about a guy who got
> kicked off Google Docs because an automated program was
> trolling/mining through their users files and its
> judgement was not to be questioned by mere mortals.
Wasn't that the user who was using a Docs form/spreadsheet to collect passwords from his customers? Google has stated that such forms are reviewed manually after being reported (for example, http://productforums.google.com/forum/#!topic/docs/7pKj6aXBK...), which means there must have been at least two humans involved.
IMO the risk with cloud storage isn't in getting suspended by some daemon run amok, it's in data being too easy to leak. Files stored in Google Drive or Dropbox are just one accidental "share" click away from becoming public.
Actually I was using a google docs spreadsheet to store usernames/passwords for various forums I had registered on. I didn't want to use any of the various password storage systems, the usernames/passwords weren't something I really cared if someone got a hold of them..
I created the file.. added a few forum links with user/pass combos.. and within about 48 hours, the file was nuked and I had an email from google stating a document I had put on google docs broke their terms of service.
I went through the terms of service and couldn't find anything regarding passwords, but it did say that anything I put in google docs was owned by google and publishable by google, so I guessed that was the reason (they can't publish passwords?)
edit: I didn't get kicked off of google docs altogether though, just the one file got nuked
Also there was a SkyDrive user who got all his MS stuff linked to the account nuked because he had taken half nude photos of someone with his phone and put them in his private folder in SkyDrive.
The only things that really keeps me from sharing photos via dropbox is
a) I use my picassa photo albums for big dumps of regular photos I take when out and about, the storage is big enough I don't have to pay anything (yet) and I have thousands upon thousands of photos there
b) I use my flickr account (paid so unlimited) to put up photos I've curated and cleaned up and processed and stuck on a map someplace and given a detailed description of...basically I use it as a detailed travelog of places I've been. I'm about 4 years behind on it to be honest, but the flickr uploadr let's me bulk upload and getting things into the account currently isn't the bottleneck in my proces
Dropbox has your stuff stored away in a fairly secure facility, while BitTorrent Sync just syncs between machines you've installed it on. That's worth something. Of course, you can just rent a cheap VPS somewhere and have it sync your BTS folders, but I think you'll be hard pressed to find a VPS with 100 GB storage for less than $10/month.
Also, Dropbox is user friendly. I can recommend it to my family members, and they have a referral program to encourage me to do it. What does BitTorrent Sync do that rsync cannot?
On the other side of the argument, I bet BitTorrent Sync will become as friendly as Dropbox, and perhaps third-party backup services will offer what Dropbox does. At that point, rsync's only advantage would be an open-source license.
It seems to overcome NAT and dynamic addressing very well. All you need is to distribute the small shared secret once (for example, offline via USB, paper, or via the first letter of every headline in consecutive college newspaper publications), and you have the ability to transfer files (and do anything that can be boiled down to that; ex: chat, perform backups, publish videos of cats among your friends, distribute mostly static data between front-facing servers, etc...) without further coordination (for example, constantly renewing a DNS record, overcoming NAT, and keeping SSH or FTP permissions in order). Sure it may not be the best experience for all use-cases from a UX standpoint, but it is very general and, being serverless, it is better from other perspectives. I'm not boxing with you, only responding to your "?" and advocating for more such software.
That's easily solved with Backblaze backing up one of the synced machines. Then it's $5/month for unlimited with off-site backup.
I agree with you that this is a very important consideration: Sync services (even Dropbox) are not backup services and shouldn't be treated like they are. You are in an even worse predicament if you are using it to sync everything locally.
Perhaps that actually may be where BT's headed to once they decide to take this out of alpha. Paid storage service for BT Sync that augments sync service.
I particularly like BT Sync that it doesn't require "account" to use.
Maybe use s3fs to store one of your BTS copies on Amazon S3?
That'd cost you $9.50/month just for the storage, bandwidth and transfer costs are a bit hard to predict - storing your mostly-write-only archives are different from your mostly-read-only mp3 collection or your pirate-bay-sourced movie collection collection.
I'm quite excited by this - I've been planning-but-not-doing-anything-about a cluster of inexpensive processors (probably RaspberryPi's or TPLink WR703-Ns) running Tahoe/LAFS. If this busts through NATed internet connections like I suspect it will, I'll probably give up on that plan and just install a bunch of copies of this in various places.
Yeah, I'd like a little more info about the stem. It looks like a quill, but then there's that cam. I've had bad luck with those in the past, wondering how this one is done.
The stem is a fully bespoke (the whole bike is) ahead stem. It uses a barrel with a 6mm allen bolt through it. The barrel is split in two and the bolt squeezes the two halves to give a large contact area with the stem. It's worked well, no complaints.
One of the goals of that bike was to design it such that it looked aesthetically clean of bolts, fixings and braze-ons.
An example of that is the rear rack... remove the rack and mudguard and there is clearance for cyclocross tyres, and there are no braze-ons that give even a hint that the frame can take racks.
I was going for a single multi-purpose frame (touring, cyclocross, urban-fixed) by simply changing wheelsets and basic things... any change should be a 10 minute job. But yet, even with multiple uses I did not want to compromise the looks to achieve it.
The photos show the bike in the most complex configuration, the Rohloff hub, the dynamo front hub, the racks, the mudguards, the flat handlebars with Rohloff shifter. But basically you can strip it to a fixed gear bike with belt drive and just a front brake in 10 minutes, and then shove it in a suitcase (S&S couplers) and go travelling with it.
And even with all of that... the bike may be custom, but I kept everything compatible with standard parts so it's easy to get a long life from the bike as parts will be available. That stem... well, any stem can go on there.
Robin's a cool guy too. Which helps when you go this crazy on spec'ing a bike.
I work for a startup and want to sync my dev machines at home and work and also a build farm. Bt sync is a good solution.
I am a musician and want to sync my tracks with a few producers. Bt sync is a good solution.
I am a political activist and want to ensure my data will be available to my affinity group even if Dropbox is threatened with a state security letter. Bt sync is an excellent solution.
Why? How is the government going to shut down what is basically a private decentralized p2p network? There isn't any central server they can take out and the traffic over the network would look like any other encrypted traffic.
If they gained access to the sync group with full permissions they could destroy it and get IP addresses, etc. And the format itself might not lend itself to all the needs of a political activist group. But I don't see how the cute cat theory holds any water here.
I'm unsure that Tor networks are viable in the long term for the same reason - they're subject to traffic analysis. Japan's police have been asking for Tor networks to be blocked over there: http://www.bbc.co.uk/news/technology-22248692
> Why? How is the government going to shut down what is basically a private decentralized p2p network? There isn't any central server they can take out and the traffic over the network would look like any other encrypted traffic.
Many ISPs, schools, cafes and corporate networks do this today already. They only allow HTTP (which goes through monitoring) and HTTPS which is decrypted by a MITM proxy (you have to allow their CA cert on your machine, uncompromised HTTPS is blocked).
In many countries this type of shenanigans are forbidden by law (friendly governments exist too).
True, but adding cute cats to the mix isn't going to fix that either.
I wasn't trying to argue that it was technically infeasible for an Orwellian government to stop it, just that it would be near impossible to stop where this type of traffic isn't already blocked.
The cute cat theory posits the resource doesn't provably work at scale until it hosts a lot of simple, vacuous content. While 200 TB exceeds the number of cute cats I have ever seen, I'm afraid it may not exceed the youtube bandwidth.
It has to work before it can work well. Thus, while PGP technically works great, I haven't heard a lot of Egyptions, Syrians, etc, lauding Phil Zimmerman. Instead, they seem happier with Facebook.
If one member of your affinity group gets arrested, the police would have access to all of your files plus all of the IP addresses of everyone in your affinity group
And IMHO all of the scenarios above are why BT Sync is more a Box alternative rather than a Dropbox alternative. Or more accurately a Groove competitor, but they're not the cool kids on the block any longer.
Business scenarios make this good for us, but for consumers this isn't a Dropbox alternative.
Dropbox's core offering boils down to file sync + cloud sync. There's many use cases where that's what you want - access on whatever devices you have the app installed on, and also over the web.
Cloud sync leads to storage limits (which don't make sense as a consumer for file sync, but are necessary for large scale cloud sync) and centralisation (desirable for vast access, not so much for some situations).
BT sync isn't a Dropbox replacement, it's a file sync replacement that's superior in some ways, but lacks the benefits of being combined with cloud sync.
Is there anyone interested in cloud sync for its own sake? In other words, if the entire thing works seamlessly and identically in-house, and assuming a disciplined backup strategy, is there anybody going "But if it were in someone else's data center, that'd be so much better!"
BT sync sounds like a complete replacement for dropbox for people with the skills and willingness to set it up.
It's a fair point, but if we consider the GP's claims, they're pretty much all benefits of cloud sync. You get a web-based view for your albums, you can share easier outside of your own devices, you can - as he says, compete with the top services for photo, video and music sharing. Cloud sync makes you a media company.
Personally, I use cloud sync for a lot of backup stuff. Granted, it's mostly Git to BitBucket, but in school I used to push documents to Dropbox or Box just to have a backup. I wasn't interested in having my essays synced to my phone, just that if my other versions died I could download my work again to another device.
If you're referring to companies though, no I doubt they'd want a seamless and perfect system in somebody else's data center. That said, a data center and a perfect seamless cloud backup solution is a considerable cost, especially for a startup.
But yep, I agree, BT sync would be perfect if you didn't want/need Dropbox remote backups or web services.
I have taken advantage of Dropbox's cloud sync features quite often. For example, my school's printers are attached to library computers but not my laptop. So I can just log in to Dropbox from a library computer to download and print a file from their web interface. The only real alternative would be to first send myself an email with the file as an attachment, which would require me to have the forethought to send that email from my laptop first, instead of knowing I can print any file I want whenever I want.
Also, I often share individual files by using the "Get a public link" feature in Dropbox, and it's comforting to know they can just access those files whenever they want no matter what devices are attached and syncing.
I don't have both my laptop and desktop on at the same time most of the time. Therefore syncing between them would happen very intermittently if there wasn't a cloud in the middle.
Additionally, I wouldn't be able to get access to my files when I was at my parents without my laptop - which I can do with Dropbox.
I pretty much agree. Correct me if I'm wrong but Dropbox were the first to have come up with Camera Upload feature, which basically makes any average user go trouble free with their cellphones, tablets, snapping away photographs knowing that they will be ready on their desktop to view if they want to.
Google+ does that too now, but this is in fact a killer feature which has been adopted well by the average folks out there. So yes, Dropbox is not just about file sharing anymore.
Yeah, I don't remember when dropbox started prompting me for photos, but I know it was after my galaxy nexus started uploading them automatically for me... which I find to be quite convenient. I don't really use G+ for anything else, but I've grown quite reliant of that particular feature recently.
Though, I believe the use case GP is describing might be a bit different, depending on whether or not a dedicated camera is being used and how large the image files are...
Sharing photos was solved for the 99% by Facebook & Apple & Google. If you're a photographer with gobs of high resolution photos Dropbox is perfect for you, but I imagine that is a niche pain point.
What is a much bigger opportunity is a way that easily lets groups share music and movies. This cannot be solved by a centralized service like Dropbox for legal reasons. This is what would concern me if I was Dropbox; BTSync's killer feature is off limits to Dropbox without a complete change in architecture (and some soul searching about if they want to risk ending up like Kim Dotcom).
this was all well and good to me until I saw that Google shut down someones account for violation of TOS. Im already mulling over alternatives. This might be one of them.
Seems like it would've been best handled by an advance warning system, i.e., at least a few days or a week's notice that you're violating their TOS and need to take action.
> And my question was, if the files are private, why should their content be against the TOS?
They probably don't want to allow the services to be used to aid illegal activity. Say what you want about how technology should be completely agnostic to matters of culture and legality, but that isn't at all the case.
I'm pretty sure many people have had this fixed without writing blog posts or having friends in google.
Just like many people have had their hotmail, paypal, facebook, itunes developer account etc. suspended for automatic detection of breach of TOS, and they've appealed and had their access returned.
This is not so much a google problem, as a reliance on 3rd parties to store your important documents, when everyone is warned (in tiny print) as to what may happen when they sign up.
I'm not excusing google's behavior, it's as bad as the rest of them, but they are playing by the rules we all agree to when we use their services.
The killer feature for me with Google+ is Google Picasa. The fact that I can choose which of the zillions of photos are my favorites and only sync those to Google+, and then share them with my friends and family is awesome. Dropbox is missing the photo application on the user's computer.
It's nice to be able to rollback/undelete files. Plus everyone is forgetting that Dropbox snagged Audiogalaxy a while back, so I'm expecting to be able to stream music back to myself from my Dropbox at some point as a service.
I just moved a bunch of stuff to btsync to mess with it but I still need to find a place offsite to keep it in addition if I want it to replace Dropbox. I'll likely use both tools in combination.
> And consumers are not thinking in terms of encrypted sync, they're just thinking in terms of "I just want to do X, why is it so hard", and so I can't see this (very nice) solution really solving the problems that consumers have, that will make them prioritise security.
I have a use case that is pretty common among my peer group and BT Sync has been the best solution I've been able to find. In a nutshell, I need to sync large datasets across multiple computers.
Dropbox: expensive
AeroFS: buggy, used too much bandwidth, and slow
Sparkleshare: uses git which chokes on large files
Git Annex Assistant: didn't work reliably on mac
rsync/duplicity/unison: needs extra logic for detecting file changes
I also think the "Dropbox replacement" idea is a strawman created by the TorrentFreak article. I've never had the impression that BT Sync is trying to replace Dropbox. It is just trying to do p2p sync with a great interface and some nice features such as read-only and one-time secrets.
Isn't that a different use case? Crashplan creates a opaque encrypted binary blob on a remote computer which is useful for backup, but not really for syncing files around.
Crashplan is great as a backup plan and offers a web interface for times when everything else has failed. I have used it to retrieve things a few times and I highly recommend it as a backup plan. But their web interface isn't their selling point, and you can't manipulate the files. It's read only access. It does track diffs, though, so that can be pretty handy.
For the reasons that you've specified and others, I rolled my own very simple sync system based on Git.
The biggest problem I personally have with Dropbbox et. al, is that none of the commercial solutions sync symbolic links opaquely. ("Opaque" syncing of symlinks means to sync the links themselves and not what the symlinks link to.) I use symlinks heavily and so I absolutely require this feature. Most of the commercial solutions just ignore symlinks, but Dropbox does the worst thing possible and treats symlinks transparently. This is utterly wrong and, in fact, downright dangerous!
Another feature that I need for telecommuting software development is to be able to exclude artifacts from what is synced. Eclipse, for instance, constantly churns out artifacts, and I certainly don't want or need those artifacts to be synced.
Two problems with my roll-your-own little syncing system remain: (1) The large file issue that you mentioned. (Sparkleshare is much worse in this regard than my system, last I tried.) (2) I know of no way to have the Git instance that I use for syncing not pay attention to the .gitignore files that are used by the Git instance that I use for version control. Annoying!
In any case, I'm very glad to hear that BitTorrent seems to be coming up with a solution that will address all my issues, and I won't have to roll my own anymore.
I can answer this question since I accidentally created a circular reference the other day. It starts syncing the recursive structure until it runs out of quota.
> It starts syncing the recursive structure until it runs out of quota.
I was curious about this myself, but I never tried it, as I feared that this was precisely the badness that would occur.
Dropbox is so good in so many respects, I just can't understand how they could have made such an utterly terrible and wrong decision on this particular issue.
You are right: Dropbox's behavior with regard to symlinks is fundamentally wrong.
It is kinda lame that other services, e.g. Google Drive, just skip symlinks, but that isn't as dangerous. The hierarchical structure of a directory tree is often effectively part of an application's data, and Dropbox silently corrupts that data. This can cause crashes, or multiple conflicting copies of not-easily-merged data to be strewn about in different places.
It's also hard to initially notice how broken it is, since symlinks work on the first machine where Dropbox encounters them, but then on all subsequent client devices that Dropbox sees are replaced with some version of whatever data they point to.
> You are right: Dropbox's behavior with regard to symlinks is fundamentally wrong.
What's even worse, is that no matter how hard I try, I am not able to convince most people of how wrong this is--even those who claim to be and should be computer savy. E.g., on support forums for other sync services, users are typically clamoring for Dropbox-style syncing of symlinks, and they will not be convinced otherwise, in spite of all reason. And despite 30 years of hard-earned experience with symlinks that irrefutably demonstrates that transparent syncing of symlinks is nothing but badness.
Yeah but the real problem, the ones users are trying to solve is the limits on having to move everything into one sync folder. Transparent symlinks are the cheap fix for them so that they can keep their files where they want and still have them sync. The better solution would be to just allow any folder(s) to be synced but none of the major cloud sync providers seem to want to enable that.
Have a look at https://www.cubby.com/. It syncs existing folders you want to, and per synced folder you can decide to use their cloud or sync directly.
> The better solution would be to just allow any folder(s) to be synced but none of the major cloud sync providers seem to want to enable that.
This correct solution is precisely what BitTorrent Sync provides. So that's all the more reason to be thankful that a seemingly excellent sync solution has finally arrived.
Sounds like it's time for a "design pattern" or "best practice" -- that is, covering up for a tool's faults.
What if you treat DropBox as a raw, strict hierarchical store and never use it directly, but only link into it from elsewhere? Move all the links outside of it.
(1) You just put a symlink into your Dropbox folder with the above sequence of commands, which just serves to illustrate the danger of this approach.
(2) Your suggestion is extremely difficult or impossible to do in the general case.
(3) I have better things to do with my life than rearrange my entire filesystem to suit the ill-conceived whims of a product that does the wrong thing.
(4) I would like to sync parts of my filesystem as they currently are.
(5) I have my own syncing solution that works better for my needs, and it only took me a couple of hours to throw together.
(6) I'm going to use BitTorrent Sync in the future for most of my syncing needs (as long as it turns out to be reliable) since it does syncing properly in the face of symlinks.
(7) I'll still use Dropbox for simple sharing of files with friends and getting files onto my iPad. Apparently, Dropbox just doesn't care about the needs of more sophisticated users, as that isn't a very large market.
> The biggest problem I personally have with Dropbbox et. al, is that none of the commercial solutions sync symbolic links opaquely
The tradeoff is to have all your stuff in one place, with is not a good thing most of the time IMO.
A frequent problem I stumble on: there is a work folder where everything is already on svn/git, but I'd want to temporarily sync a few folders to dropbox only when needed, and get them out of dropbox when finished.
As dropbox works now, to do this without using symlinks would be a PITA (I'd have to put all my files in the dropbox folder by default, and then play with the selective sync ? I am not even sure there is a way to do it without moving files around).
If dropbox had support for arbitrary multiple folder selection (but the the complexity would go way up?) I'd agree with your stance on symlink, but now this is more of a necessary evil.
> If dropbox had support for arbitrary multiple folder selection (but the the complexity would go way up?) I'd agree with your stance on symlink, but now this is more of a necessary evil.
I think you already answered your own objection on the right way to handle this need. Other syncing services, including BitTorrent Sync handle your need in precisely this manner. I.e., the right way. Dangerous, frustrating, and incorrect abuse of symlinks is NOT the right way to do this.
> I used to use Flickr for photo sharing, but cameras got better, images got bigger, and I have a lot of photos.
Why is this a problem with Flickr? I am generally shooting high res full frame, and post to Flickr. I have about a terabyte of images. There's no 100GB storage limit, just an annual flat fee.
Family can download full resolution images for printing. My grandmother can have my Flickr Photo Stream as a screensaver in her TV. I also enjoy fantastic two way integration with photo management tools, with tagging syncing back.
I can't see why I'd pay Dropbox considerably more for less features.
(What's more, if Flickr doesn't like a public photo's content, the worst that will happen is getting marked "not in public search areas", with an easy redress to get reinstated. It's unclear to me what Dropbox will do. Meanwhile, if Google doesn't like a public photo, I can lose my Google Account, as photogs have found to their chagrin.)
I am using Everpix at the moment, but Flickr seems like a better deal and has much more features.
Except for a critical one....auto-uploading from pre-specified folders & Photo Stream. Do you manually upload all your photos, or is there a better way?
You can auto load with a number of third party tools. Early on, I had a Perl uploader that would space things out during the day to keep a steady stream of new files posting. That shot me into Explore top 500 frequently. These days I like FlickStacker on iOS and FlickrExport on OSX. I just post sets when I'm done with triage.
> I also enjoy fantastic two way integration with photo management tools, with tagging syncing back.
Which photo management tools do you use? I currently have around 45k photos (~200GB) and Picasa is sometimes just too slow. I am planning to get a DSLR soon, so total size is going to be expanding rapidly. What kind of tools would you suggest? I am on Windows mainly, so Mac-only is not an option. Thanks!
Try Adobe Lightroom [yes, our favorite whipping boy Adobe :)]. It's really really awesome as a photo management tool. IMHO. If you find it slow, try throwing in an SSD. Don't run it with less than 8 GB RAM.
I can second that. Lightroom is indeed an awesome tool. You should really check it out if you have large amounts of images to be organized and want to bulk-optimize them. It takes some dedication to get used to the workflow and the available set of tools (many!), but it's worth it. Also, Adobe dropped the price significantly some time ago, making this great product available for a bigger audience.
I use Aperture for most photos. I use Lightroom for slides. On the PC side, you should use Lightroom. It's just gone into beta for version 5, try it now for free.
I strongly prefer the Aperture workflow, but for slides Aperture won't recognize my 64 bit .DNG files with infrared channel as being part of a JPEG/RAW pair, while Lightroom will.
I manage photos on the latest Mac Mini with the Fusion Drive, but the photo libraries are actually stored on Western Digital MyBook Velociraptor Duo Thunderbolt drives[1] which are insanely fast in RAID0 mode. With Thunderbolt I can attach that to a laptop or the Mac Mini and Aperture is licensed for 5 computers. I had been using the built in Fusion drive for my latest project triage, but the Velociraptor drives in RAID0 are so fast it's not worth the hassle of splitting that out.
I use a nightly rsync to replicate the libraries onto a LaCie 4big via Firewire 800. I never have less than two copies of photos on two devices, because I use a Nexto DI[2] to import the photos from flash while on the go, then I import from that into my libraries, leaving copies on it until I need space. Or, for things like iPhone imports, I import to the LaCie RAID, then import into Aperture's RAID0 library for speed, and again, only delete from the RAIDs when I need space and after I know Aperture's backed up. I also have a Backblaze[5] job backing up everything offsite for their flat fee.
I have three libraries, one for 2000-2010, one for the current decade, and one for international travel. While travel is less frequent, a trip generates more photos, so my domestic and international libraries tend to be similar in size. Each is 50K to 150K photos, and in the 350GB - 750GB range.
I use Aperture's library in the fully imported mode, where photos are stored in the library. This way I'll never accidentally move or delete a photo I want to keep. Of course, the library is just a package folder, you can CTRL-Click it to open it up and get at any of your import sessions original or raw files. Even if the DB is completely destroyed, the photos are safe.
I use the incredible Nik Collection of plugins. They were worth it at $750, and so much more worth it at Google's new price of $149[3]. These plugins work with Lightroom or Photoshop as well.
With over a decade of DSLR photos under management, I recommend you use a folder and image naming scheme like this:
If I'm using generic file system tools rather than a true photo management app, I name the file in a way that lets me search, sort, and reconstruct the original file, regardless of file system capabilities:
Moving images around across file systems will likely eventually lose the date, with this you can use a simple shell script to put back the create time. Or use ExifTool to get the data back from inside the file[4].
If you are a photographer, Our Tonido (http://www.tonido.com) solution is simply perfect. It is not expensive as dropbox, and no storage limits, supports variety of photo formats (JPEG, PNG, GIF, CR2, NEF, CRW and more). As dropbox, we have good mobile apps for all the popular platforms. Pretty much majority of our paid users are photographers.
I think for photographers, the backup functionality of most "sharing" sites is pretty important... and Tonido seems to not address that at all (by design).
It still looks like an interesting product, mind you...
Nope, that's just one directory. I was just illustrating the one photo sharing piece... which is as simple as drag a folder into Dropbox, and go to the web interface and click share.
I have approx' 40k photos (not including the .raw files) over about 180GB (including the .raw files).
> Then I ran out of space... over 100GB of photos, where next?
Uhm... how about the 200 GB plan? Or the 400 GB plan. Or 1 TB for $49.99/month? Or you could contact sales about Google Cloud Storage: https://cloud.google.com/pricing/cloud-storage Or you could just buy S3 storage.
Actually, there are literally dozens (to maybe even hundreds) of alternatives for hosting static content for less money for more bandwidth.
FWIW this product fits my use case perfectly. I want to sync lots of data, but I don't want to pay for Dropbox because I really, really don't care if my stuff gets into the "cloud". I just want it across machines. Currently I use AeroFS, but this looks spiffy and might find a home on my hard drive.
I know this may not be practical depending on the amount of space required, but you could probably run a very cheap EC2 image with a huge EBS volume attached for a year for cheaper than the bigger Dropbox offerings.
For example, a micro instance, with 500GB of EBS storage, with allowance for 1% data growth per daily snapshot (S3 backed), and 50GB of transfer per month will apparently cost $175.68.
The 500GB plan from Dropbox will cost you $499 .
I'm thinking, with enough data, Bittorrent Sync wins the pricing war, and I've been doing some testing on the client and it seems really resilient to data changing while syncing etc.
They have a native linux 64bit client, that's great! The linux binary is cli only, but provides a web interface on port 8888. However, the daemon binds to address 0.0.0.0 and not just localhost (127.0.0.1 or ::1)! So make sure you have a firewall. Otherwise everybody can access the web interface and thus your secrets.
I got in on the closed Alpha just last week, and it immediately fit into a use case I had where I needed to sync a folder on an OSX machine with a remote linux server. SFTP would have worked in this use case just as well, except it can be janky to slot into a non-developer's workflow.
Long story short, I setup BS on both machines with an absolute minimum of fuss, copied the password over, and the folder synced. Definitely does one thing and does it well.
I've been using BitTorrent Sync for the past week. For the use case of sharing large files, it obliterates Dropbox, AeroFS, and Cubby.
It distributes the files intelligently and makes optimal use of everyone's bandwidth. Dropbox, for example, slowly uploads all files to the cloud before distributing them, plus there's the space issue. AeroFS allows unlimited space, but is far slower than my Internet and LAN speeds allow, and does things like trying to upload the same file, linearly, to every peer at once. Cubby has limited space and has the same slow syncing problems.
I'm running BitTorrent Sync on my 6 year old Windows 7 Thinkpad, a newer Windows 8 desktop, a Digital Ocean Ubuntu VPS, and a Synology DS110j NAS. It runs perfectly on all of the above, and provides a useful web interface for the VPS and NAS.
If you already have a Synology box, what do you get with BS versus using Synology's own CloudStation? I've looked into BS, but I guess I'm not getting what I gain running it on my DS410.
I've never used CloudStation, but with respect to BitTorrent Sync, the DiskStation would be just another device that syncs the files. So the main benefit is that it acts as another backup / seeder for your private p2p network.
Can someone give me a quick rundown on how this works, vs security of files? If it's using the bittorrent protocol, wouldn't that mean that all your files, even though encrypted, are easily gathered by a third party?
Or does this simply make direct connections between PCs you own and people you have authorized to share your files?
> While Sync uses BitTorrent technology, people’s files are not accessible to outsiders. Only those who have the unique private key can access the shared folder.
>
> “All the traffic is encrypted using a private key derived from the shared secret. Your files can be viewed and received only by the people with whom you share your private secret,” BitTorrent explains.
It’s equivalent of making a private torrent of your encrypted content then giving your friend the torrent metadata AND decryption key. That is not including additional features.
We (rsync.net) will implement this on our storage arrays, provided it can run over SSH in some capacity. That way you can target rsync.net space with BS.
Agreed. I think that is a pretty good move for adoption. My QNAP NAS has always played well with bittorrent and backups. Adding ARM support means you can use this in a bunch of low power devices.
Wish the source was made available, I have an OpenIndiana server with several 1 TB drives running ZFS and I would love to use something like this... but only binaries are available.
Looking at some of the examples, I'm starting to see a future where people trade entirely in hashes - send some amount of bitcoins to address X, receive access to read-only access to bittorrent folder Y which is automatically populated with the latest TV episodes for a series, etc. Any idea how to apply that sort of anonymity and convenience to the real world though? Scanning QR codes with a phone seems to be the best so far, still more fiddly than cash though...
You might think you could possibly have done this in the physical world for a few years now. Don't fool yourself. You have a face and DNA. You will not be anonymous (or even pseudonymous) as long as you carry a body.
This is great; however, folks can still securely store data on cloud services. They just need to take a Trust No one (TNO) approach and encrypt their data to disk.
For example you can use TrueCrypt and create/mount a drive volume that is fully encrypted while synced across a file share storage/sync service like DropBox.
The only downside is that you have to install the TrueCrypt application on your client device, which does limit is platform offering (currently, no mobile).
Good point, I only use TrueCrypt for sensitive documents (small in file size), and don't worry/notice the entire volume sync.
BoxCryptor does look like a great alternative; however, it's not open source and does come w/ a price (literally and figuratively - file names exposed if using free version or drop $50 USD).
Have you used BoxCryptor? Curious what your experience has been. I'll give it a try.
Dropbox does not upload your whole Truecrypt volume every time you change it. It just uploads the blocks that are changed. It's pretty efficient.
The real problem with using a Truecrypt volume on Dropbox is concurrent changes to the same volume. It happens if you mount the Truecrypt volume on 2 machines at the same time. It will mess up badly.
BitTorrent Sync synchronizes your files using a peer-to-peer (P2P) protocol. This protocol is very effective for transferring large files across multiple devices, and is very similar to the powerful protocol used by applications like µTorrent and BitTorrent. The data is transferred in pieces from each of the syncing devices, and BitTorrent Sync chooses the optimal algorithm to make sure you have a maximum download and upload speed during the process.
The devices you setup to sync are connected directly using UDP, NAT traversal and UPnP port mapping. We also provide such additional methods of ensuring connectivity as relay and tracker servers. If your devices are on the same local network, BitTorrent Sync will use your LAN for faster synchronization.
It doesn't feel like this overlaps with my need for Dropbox. I use Dropbox for file sharing, backup, version control, and the ability to access my files at any time from any computer (through the website). Actually syncing between computers is just a small part of it.
This seems ideal for the enterprise use case, in particular because it's extremely performant, secure, and doesn't require a central/third-party server (e.g. AWS). The 'must always be online' requirement is actually a plus, not a minus for that crowd. Enterprises are generally very concerned about being able to revoke access to a given resource instantaneously. Any app that stores local copies complicates that quite a bit.
People will dismiss this as 'a toy that geeks use', but as Chris Dixon has noted, 'what the smartest people do on the weekend is what everyone else will do during the week in ten years' [1]
So now you have in place all the bits you need for a "members only" netnews feed, someone should go dig up the code and create a 'news' sync'd document. If you played your cards right you could use off the shelf news readers that look into a local repo of the news.
With just a tiny bit of crypto code you could add a repudiation feature (keys signed by the secret vs the secret itself) and control access to both individual groups and individual users.
Just to make the connection for others, you are describing something that can be used for chat AND binaries. As ugly a hack as it might be, (speaking to the choir, I know) there is tremendous value in simply being able to move messages (in this case, as simple files), between buddies without needing an "account" with a "service" (of the type that motivates people to excessively use "quotes").
If they allowed us to pick and choose what we want to download from friends shares as well as search our friends shares, this could become an interesting tool.
Imagine a private p2p site that you and your friends use. You choose who to share with and what files you can see.
Then add links to give to friends so attaching files in emails would be a sinch.
Is this usable as a backup offsite/distributed store?
Put backup into TrueCrypt Container which itself is located in a BT Sync folder.
Invite other devices + "partners" to sync that folder ...
any thoughts on this usage scenario with BT Sync?
Does the traffic from this appear same as P2P traffic? I think my company might block my internet access if it finds out that I am doing p2p sharing; and I don't want that to happen.
289 comments
[ 3.5 ms ] story [ 264 ms ] threadWhen the source is closed, and the protocol is closed + encrypted? I think you may be a little optimistic... (Though I do hope that either the source or the protocol get opened)
There are a few advanced configurations where BTSync trumps Aero (at least last time I used it - someone, please correct me if these have been implemented).
- One way sync
- Selective device LAN sync (so you can tell it to only sync over LAN)
- Self destructing one-time share keys
- Less-friction on sharing - using a key based copy and paste system
- Powered by BT (this may not be an advantage though since BT is throttled on quite a number of providers through DPI worldwide
Additionally, AeroFS relies on Java, which is pretty big dependency (read: pain) to have on resource-restricted environments like a tiny NAS box. The daemon process of AeroFS consumes about 110MB of memory (I guess due to JRE), but my NAS unit has only 256MB memory in total with less than 100MB free, which means even if AeroFS supports PowerPC, there is still no way I could run it smoothly on my NAS without killing its performance.
Bittorrent Sync supports all of x86/x64/ARM/PowerPC, and the download is just a single 3.7MB binary without dependency other than glibc. Running it on my NAS shows it consumes less than 10MB memory. This is just so much nicer for deployment on a wider selection of devices.
I'm definitely going for BTSync now.
I agree that the main gripe with AeroFS that people have voiced in this thread is certainly valid. AeroFS _did_ slow down in performance for some users recently. We actually just released a version today that addresses this bug (0.4.181 - see http://ae.ro/Ln2YJJ for the release notes, but it specifically had to do with the way we were initializing our jingle library), and in our own internal tests the performance has improved dramatically.
[1] http://forum.bittorrent.com/topic/8816-will-syncapp-be-open-...
But it doesn't store your data, so yes, one device must always be on in order to sync with a new device. However, if you have a device that needs changes made on another one, it will just wait around and sync whenever the device with changes comes back online.
For me to consider switching to this service from Dropbox, it would have to be able to temporarily store some of my files from my work computer so that they could be transferred to my home computer later on when that computer was started up (for example). That is the feature that needs to be implemented for this to truly become a Dropbox-killer.
If I have to devise some system to zip my important stuff and keep that up to date, then I may as well just use BTSync for free.
I see two key use cases that could cause trouble:
1)a co-worker turned his computer off and went home from work, the syncing couldn't complete even if only a single doc was left
2)Because there is no central server if all of my coworkers in a distributed team sync with my new shared folder at the same time, it is required that I own a connection capable of supporting that kind of traffic, because it would be almost the same as streaming a video to multiple users, not to mention that I would probably have to avoid to browse the web for pretty much anything else in the meanwhile.
As far as 2 is concerned though, that is not the case. Don't forget, it's built on top of BT. As each piece of the file gets transferred to other clients by the original machine, the other clients will be able to send those pieces out too. In theory it will be faster than a transfer from a central server.
"We also provide such additional methods of ensuring connectivity as relay and tracker servers."
* Ability to handle large files (300MB incompressible Photoshop) and able to handle large complete data stores. (1 TB, which is impossible with Dropbox)
* Selectability for certain folders on certain users (no need for all data to be on all machines)
* The ability to 'archive' files to only the host.
* Good LAN networking with host so that the portable machines (iPads, Retina Macbook Pros with puny 128GB HDD) can use it fine.
Maybe both BT and AeroFS should help customers mitigate that risk by committing to open source their code if they shut down. AeroFS has a revenue model, though, so I'd be less concerned about them shutting down this product.
>BitTorrent Sync’s functionality is comparable to services such as Dropbox and Skydrive, except for the fact that there’s no cloud involved. Users sync the files between their own computers and no third-party has access to it.
> It is an ideal tool for people who want to share large amounts of data between computers without going through third-party services.
>The Sync application is available for Windows, OSX, Linux and has the ability run on NAS devices through a web-interface. Readers who are interested in giving it a spin can head over to BitTorrent labs [1], where the Sync app can be downloaded.
[1]: http://labs.bittorrent.com/
Another way would be to do TCP or UDP hole punching, but that involves a third party for initial setup. Probably possible and probably safe, but I'd like to see a security review of that.
One example: I used to use Flickr for photo sharing, but cameras got better, images got bigger, and I have a lot of photos. I moved from Flickr to Picasa as it could cope with the directories full of photos and I didn't have to manually upload them and Google's storage space was cheaper. Then I ran out of space... over 100GB of photos, where next?
Hello Dropbox: https://www.dropbox.com/sc/um5zf95urdk3zmg/2SaSCUIQd8
And I've told a few photographers about this, and a few weeks later a friend of a friend of a friend excitedly told me on a forum how you can photo share in Dropbox.
And what I'm basically seeing is that the problem of "file sync" is being considered as solved by lay consumers, who really aren't prioritising encryption, and the problems that they now have is "share this directory of photos", and "share that directory of videos", and "sync this music privately, but let me play it back".
Dropbox isn't just file sync anymore.
What it is, is a serious threat to Flickr, Picasa, YouTube, Amazon MP3 Locker, Google Play Music, iTunes, etc.
And consumers are not thinking in terms of encrypted sync, they're just thinking in terms of "I just want to do X, why is it so hard", and so I can't see this (very nice) solution really solving the problems that consumers have, that will make them prioritise security.
I guess the marketing is to provide them with a nice level of deniability. It's not a Dropbox competitor, it's just encrypted folder sharing.
This service would mostly be used by us who are wary about trusting Google/Microsoft/Dropbox and the judgement of their automated decision systems with our life . Yesterday there was a HN story about a guy who got kicked off Google Docs because an automated program was trolling/mining through their users files and its judgement was not to be questioned by mere mortals.
What if I put a picture of me and my kid playing in the swimming pool -- what is to prevent Dropbox/Google/Skydrive from tagging it as offensive and setting an overzealous DA or social-worker upon my family ? People have been arrested/detained at border crossings for perfectly innocuous pictures like these.
Therefore if you think privacy is only important for people stealing stuff, your are being naive.
[1] http://tonyonsecurity.com/2012/08/05/securing-your-data-on-d...
https://www.boxcryptor.com/
IMO the risk with cloud storage isn't in getting suspended by some daemon run amok, it's in data being too easy to leak. Files stored in Google Drive or Dropbox are just one accidental "share" click away from becoming public.
The link you submitted was a genuine scammer issue which was reported by users and dealt with correctly by Google.
The link in my post is just a citation as evidence that account suspension is a manual process.
I created the file.. added a few forum links with user/pass combos.. and within about 48 hours, the file was nuked and I had an email from google stating a document I had put on google docs broke their terms of service.
I went through the terms of service and couldn't find anything regarding passwords, but it did say that anything I put in google docs was owned by google and publishable by google, so I guessed that was the reason (they can't publish passwords?)
edit: I didn't get kicked off of google docs altogether though, just the one file got nuked
http://wmpoweruser.com/watch-what-you-store-on-skydriveyou-m...
a) I use my picassa photo albums for big dumps of regular photos I take when out and about, the storage is big enough I don't have to pay anything (yet) and I have thousands upon thousands of photos there
b) I use my flickr account (paid so unlimited) to put up photos I've curated and cleaned up and processed and stuck on a map someplace and given a detailed description of...basically I use it as a detailed travelog of places I've been. I'm about 4 years behind on it to be honest, but the flickr uploadr let's me bulk upload and getting things into the account currently isn't the bottleneck in my proces
On the other side of the argument, I bet BitTorrent Sync will become as friendly as Dropbox, and perhaps third-party backup services will offer what Dropbox does. At that point, rsync's only advantage would be an open-source license.
It seems to overcome NAT and dynamic addressing very well. All you need is to distribute the small shared secret once (for example, offline via USB, paper, or via the first letter of every headline in consecutive college newspaper publications), and you have the ability to transfer files (and do anything that can be boiled down to that; ex: chat, perform backups, publish videos of cats among your friends, distribute mostly static data between front-facing servers, etc...) without further coordination (for example, constantly renewing a DNS record, overcoming NAT, and keeping SSH or FTP permissions in order). Sure it may not be the best experience for all use-cases from a UX standpoint, but it is very general and, being serverless, it is better from other perspectives. I'm not boxing with you, only responding to your "?" and advocating for more such software.
I agree with you that this is a very important consideration: Sync services (even Dropbox) are not backup services and shouldn't be treated like they are. You are in an even worse predicament if you are using it to sync everything locally.
I particularly like BT Sync that it doesn't require "account" to use.
That'd cost you $9.50/month just for the storage, bandwidth and transfer costs are a bit hard to predict - storing your mostly-write-only archives are different from your mostly-read-only mp3 collection or your pirate-bay-sourced movie collection collection.
I'm quite excited by this - I've been planning-but-not-doing-anything-about a cluster of inexpensive processors (probably RaspberryPi's or TPLink WR703-Ns) running Tahoe/LAFS. If this busts through NATed internet connections like I suspect it will, I'll probably give up on that plan and just install a bunch of copies of this in various places.
The bike is made by a gent named Robin Mather who lives in the South West of England. His website is unfortunately a poor one, but he's got coverage elsewhere: http://apracticalguide.wordpress.com/ http://www.bespoked2012.co.uk/BespokedBristol/Robin_Mather.h... http://www.headsetpress.co.uk/features/robin-mather-building...
The stem is a fully bespoke (the whole bike is) ahead stem. It uses a barrel with a 6mm allen bolt through it. The barrel is split in two and the bolt squeezes the two halves to give a large contact area with the stem. It's worked well, no complaints.
One of the goals of that bike was to design it such that it looked aesthetically clean of bolts, fixings and braze-ons.
An example of that is the rear rack... remove the rack and mudguard and there is clearance for cyclocross tyres, and there are no braze-ons that give even a hint that the frame can take racks.
I was going for a single multi-purpose frame (touring, cyclocross, urban-fixed) by simply changing wheelsets and basic things... any change should be a 10 minute job. But yet, even with multiple uses I did not want to compromise the looks to achieve it.
The photos show the bike in the most complex configuration, the Rohloff hub, the dynamo front hub, the racks, the mudguards, the flat handlebars with Rohloff shifter. But basically you can strip it to a fixed gear bike with belt drive and just a front brake in 10 minutes, and then shove it in a suitcase (S&S couplers) and go travelling with it.
And even with all of that... the bike may be custom, but I kept everything compatible with standard parts so it's easy to get a long life from the bike as parts will be available. That stem... well, any stem can go on there.
Robin's a cool guy too. Which helps when you go this crazy on spec'ing a bike.
Good to know Dropbox can host some of the steamiest bike porn I've ever seen.
I am a musician and want to sync my tracks with a few producers. Bt sync is a good solution.
I am a political activist and want to ensure my data will be available to my affinity group even if Dropbox is threatened with a state security letter. Bt sync is an excellent solution.
I think it will need more cute cats before you can make that call.
http://en.wikipedia.org/wiki/Cute_cat_theory_of_digital_acti...
If they gained access to the sync group with full permissions they could destroy it and get IP addresses, etc. And the format itself might not lend itself to all the needs of a political activist group. But I don't see how the cute cat theory holds any water here.
Many ISPs, schools, cafes and corporate networks do this today already. They only allow HTTP (which goes through monitoring) and HTTPS which is decrypted by a MITM proxy (you have to allow their CA cert on your machine, uncompromised HTTPS is blocked).
In many countries this type of shenanigans are forbidden by law (friendly governments exist too).
I wasn't trying to argue that it was technically infeasible for an Orwellian government to stop it, just that it would be near impossible to stop where this type of traffic isn't already blocked.
It has to work before it can work well. Thus, while PGP technically works great, I haven't heard a lot of Egyptions, Syrians, etc, lauding Phil Zimmerman. Instead, they seem happier with Facebook.
If one member of your affinity group gets arrested, the police would have access to all of your files plus all of the IP addresses of everyone in your affinity group
IP addresses = VPN
Files = Truecrypted
Business scenarios make this good for us, but for consumers this isn't a Dropbox alternative.
Cloud sync leads to storage limits (which don't make sense as a consumer for file sync, but are necessary for large scale cloud sync) and centralisation (desirable for vast access, not so much for some situations).
BT sync isn't a Dropbox replacement, it's a file sync replacement that's superior in some ways, but lacks the benefits of being combined with cloud sync.
BT sync sounds like a complete replacement for dropbox for people with the skills and willingness to set it up.
Personally, I use cloud sync for a lot of backup stuff. Granted, it's mostly Git to BitBucket, but in school I used to push documents to Dropbox or Box just to have a backup. I wasn't interested in having my essays synced to my phone, just that if my other versions died I could download my work again to another device.
If you're referring to companies though, no I doubt they'd want a seamless and perfect system in somebody else's data center. That said, a data center and a perfect seamless cloud backup solution is a considerable cost, especially for a startup.
But yep, I agree, BT sync would be perfect if you didn't want/need Dropbox remote backups or web services.
Also, I often share individual files by using the "Get a public link" feature in Dropbox, and it's comforting to know they can just access those files whenever they want no matter what devices are attached and syncing.
Additionally, I wouldn't be able to get access to my files when I was at my parents without my laptop - which I can do with Dropbox.
Google+ does that too now, but this is in fact a killer feature which has been adopted well by the average folks out there. So yes, Dropbox is not just about file sharing anymore.
Though, I believe the use case GP is describing might be a bit different, depending on whether or not a dedicated camera is being used and how large the image files are...
What is a much bigger opportunity is a way that easily lets groups share music and movies. This cannot be solved by a centralized service like Dropbox for legal reasons. This is what would concern me if I was Dropbox; BTSync's killer feature is off limits to Dropbox without a complete change in architecture (and some soul searching about if they want to risk ending up like Kim Dotcom).
Automation of Google's algorithm thought he was a hacker, they fixed it and he got his account back.
You should have read the whole story ;-)
Where's the relevance? I asked why is Google, the entity, looking inside of private files?
> a probability of breaking the TOS
And my question was, if the files are private, why should their content be against the TOS?
Also, locking somebody out of their account because of a probability generated by a script, without manual confirmation by a human?
OMG. Some people can lose their jobs or money because of such flukes.
They probably don't want to allow the services to be used to aid illegal activity. Say what you want about how technology should be completely agnostic to matters of culture and legality, but that isn't at all the case.
They only fixed it because he pulled teeth, tapped friends on staff at google and generally had to scream and shout.
In no way was the trek he had to go through to fix an automation problem a good example of "Don't worry about it"
Plus the fact that if it happens to you and you don't have the inside contacts you have no guarantee of getting it fixed.
Just like many people have had their hotmail, paypal, facebook, itunes developer account etc. suspended for automatic detection of breach of TOS, and they've appealed and had their access returned.
This is not so much a google problem, as a reliance on 3rd parties to store your important documents, when everyone is warned (in tiny print) as to what may happen when they sign up.
I'm not excusing google's behavior, it's as bad as the rest of them, but they are playing by the rules we all agree to when we use their services.
Rubbish. Dropbox already has people's photos. It's easier and better to tap the screen a few times and share them there.
It's nice to be able to rollback/undelete files. Plus everyone is forgetting that Dropbox snagged Audiogalaxy a while back, so I'm expecting to be able to stream music back to myself from my Dropbox at some point as a service.
I just moved a bunch of stuff to btsync to mess with it but I still need to find a place offsite to keep it in addition if I want it to replace Dropbox. I'll likely use both tools in combination.
I have a use case that is pretty common among my peer group and BT Sync has been the best solution I've been able to find. In a nutshell, I need to sync large datasets across multiple computers.
Dropbox: expensive
AeroFS: buggy, used too much bandwidth, and slow
Sparkleshare: uses git which chokes on large files
Git Annex Assistant: didn't work reliably on mac
rsync/duplicity/unison: needs extra logic for detecting file changes
I also think the "Dropbox replacement" idea is a strawman created by the TorrentFreak article. I've never had the impression that BT Sync is trying to replace Dropbox. It is just trying to do p2p sync with a great interface and some nice features such as read-only and one-time secrets.
The biggest problem I personally have with Dropbbox et. al, is that none of the commercial solutions sync symbolic links opaquely. ("Opaque" syncing of symlinks means to sync the links themselves and not what the symlinks link to.) I use symlinks heavily and so I absolutely require this feature. Most of the commercial solutions just ignore symlinks, but Dropbox does the worst thing possible and treats symlinks transparently. This is utterly wrong and, in fact, downright dangerous!
Another feature that I need for telecommuting software development is to be able to exclude artifacts from what is synced. Eclipse, for instance, constantly churns out artifacts, and I certainly don't want or need those artifacts to be synced.
Two problems with my roll-your-own little syncing system remain: (1) The large file issue that you mentioned. (Sparkleshare is much worse in this regard than my system, last I tried.) (2) I know of no way to have the Git instance that I use for syncing not pay attention to the .gitignore files that are used by the Git instance that I use for version control. Annoying!
In any case, I'm very glad to hear that BitTorrent seems to be coming up with a solution that will address all my issues, and I won't have to roll my own anymore.
I was curious about this myself, but I never tried it, as I feared that this was precisely the badness that would occur.
Dropbox is so good in so many respects, I just can't understand how they could have made such an utterly terrible and wrong decision on this particular issue.
It is kinda lame that other services, e.g. Google Drive, just skip symlinks, but that isn't as dangerous. The hierarchical structure of a directory tree is often effectively part of an application's data, and Dropbox silently corrupts that data. This can cause crashes, or multiple conflicting copies of not-easily-merged data to be strewn about in different places.
It's also hard to initially notice how broken it is, since symlinks work on the first machine where Dropbox encounters them, but then on all subsequent client devices that Dropbox sees are replaced with some version of whatever data they point to.
What's even worse, is that no matter how hard I try, I am not able to convince most people of how wrong this is--even those who claim to be and should be computer savy. E.g., on support forums for other sync services, users are typically clamoring for Dropbox-style syncing of symlinks, and they will not be convinced otherwise, in spite of all reason. And despite 30 years of hard-earned experience with symlinks that irrefutably demonstrates that transparent syncing of symlinks is nothing but badness.
But thank goodness that BitTorrent got it right!
http://help.cubby.com/forums/169907-general/suggestions/3529...
This correct solution is precisely what BitTorrent Sync provides. So that's all the more reason to be thankful that a seemingly excellent sync solution has finally arrived.
What if you treat DropBox as a raw, strict hierarchical store and never use it directly, but only link into it from elsewhere? Move all the links outside of it.
etc.(2) Your suggestion is extremely difficult or impossible to do in the general case.
(3) I have better things to do with my life than rearrange my entire filesystem to suit the ill-conceived whims of a product that does the wrong thing.
(4) I would like to sync parts of my filesystem as they currently are.
(5) I have my own syncing solution that works better for my needs, and it only took me a couple of hours to throw together.
(6) I'm going to use BitTorrent Sync in the future for most of my syncing needs (as long as it turns out to be reliable) since it does syncing properly in the face of symlinks.
(7) I'll still use Dropbox for simple sharing of files with friends and getting files onto my iPad. Apparently, Dropbox just doesn't care about the needs of more sophisticated users, as that isn't a very large market.
The tradeoff is to have all your stuff in one place, with is not a good thing most of the time IMO. A frequent problem I stumble on: there is a work folder where everything is already on svn/git, but I'd want to temporarily sync a few folders to dropbox only when needed, and get them out of dropbox when finished. As dropbox works now, to do this without using symlinks would be a PITA (I'd have to put all my files in the dropbox folder by default, and then play with the selective sync ? I am not even sure there is a way to do it without moving files around).
If dropbox had support for arbitrary multiple folder selection (but the the complexity would go way up?) I'd agree with your stance on symlink, but now this is more of a necessary evil.
I think you already answered your own objection on the right way to handle this need. Other syncing services, including BitTorrent Sync handle your need in precisely this manner. I.e., the right way. Dangerous, frustrating, and incorrect abuse of symlinks is NOT the right way to do this.
Is it so difficult to add a one line configuration item to your inotify cron daemon (incrond) ?
/home/my/directory IN_CLOSE_WRITE,IN_CREATE,IN_DELETE /my/unison/sync/script.sh
It works.
Why is this a problem with Flickr? I am generally shooting high res full frame, and post to Flickr. I have about a terabyte of images. There's no 100GB storage limit, just an annual flat fee.
Family can download full resolution images for printing. My grandmother can have my Flickr Photo Stream as a screensaver in her TV. I also enjoy fantastic two way integration with photo management tools, with tagging syncing back.
I can't see why I'd pay Dropbox considerably more for less features.
(What's more, if Flickr doesn't like a public photo's content, the worst that will happen is getting marked "not in public search areas", with an easy redress to get reinstated. It's unclear to me what Dropbox will do. Meanwhile, if Google doesn't like a public photo, I can lose my Google Account, as photogs have found to their chagrin.)
http://ipont.jubilo.ca/ip/flickstackr/
http://connectedflow.com/flickrexport/
I like Everpix too, fun to see what it comes up with for its "moments". Cool idea.
Which photo management tools do you use? I currently have around 45k photos (~200GB) and Picasa is sometimes just too slow. I am planning to get a DSLR soon, so total size is going to be expanding rapidly. What kind of tools would you suggest? I am on Windows mainly, so Mac-only is not an option. Thanks!
Some detail for you:
I use Aperture for most photos. I use Lightroom for slides. On the PC side, you should use Lightroom. It's just gone into beta for version 5, try it now for free.
I strongly prefer the Aperture workflow, but for slides Aperture won't recognize my 64 bit .DNG files with infrared channel as being part of a JPEG/RAW pair, while Lightroom will.
I manage photos on the latest Mac Mini with the Fusion Drive, but the photo libraries are actually stored on Western Digital MyBook Velociraptor Duo Thunderbolt drives[1] which are insanely fast in RAID0 mode. With Thunderbolt I can attach that to a laptop or the Mac Mini and Aperture is licensed for 5 computers. I had been using the built in Fusion drive for my latest project triage, but the Velociraptor drives in RAID0 are so fast it's not worth the hassle of splitting that out.
I use a nightly rsync to replicate the libraries onto a LaCie 4big via Firewire 800. I never have less than two copies of photos on two devices, because I use a Nexto DI[2] to import the photos from flash while on the go, then I import from that into my libraries, leaving copies on it until I need space. Or, for things like iPhone imports, I import to the LaCie RAID, then import into Aperture's RAID0 library for speed, and again, only delete from the RAIDs when I need space and after I know Aperture's backed up. I also have a Backblaze[5] job backing up everything offsite for their flat fee.
I have three libraries, one for 2000-2010, one for the current decade, and one for international travel. While travel is less frequent, a trip generates more photos, so my domestic and international libraries tend to be similar in size. Each is 50K to 150K photos, and in the 350GB - 750GB range.
I use Aperture's library in the fully imported mode, where photos are stored in the library. This way I'll never accidentally move or delete a photo I want to keep. Of course, the library is just a package folder, you can CTRL-Click it to open it up and get at any of your import sessions original or raw files. Even if the DB is completely destroyed, the photos are safe.
I use the incredible Nik Collection of plugins. They were worth it at $750, and so much more worth it at Google's new price of $149[3]. These plugins work with Lightroom or Photoshop as well.
With over a decade of DSLR photos under management, I recommend you use a folder and image naming scheme like this:
If I'm using generic file system tools rather than a true photo management app, I name the file in a way that lets me search, sort, and reconstruct the original file, regardless of file system capabilities: Moving images around across file systems will likely eventually lose the date, with this you can use a simple shell script to put back the create time. Or use ExifTool to get the data back from inside the file[4].---
Links referenced:
1. http://www.storagereview.com/western_digital_my_book_velocir...
2. http://www.amazon.com/Nexto-Digital-Photo-Storage-ND2730/dp/...
3. http://www.niksoftware.com/
4. http://www.sno.phy.queensu.ca/~phil/exiftool/
5. http://www.backblaze.com
It still looks like an interesting product, mind you...
Dropbox for privacy minded people if you will.
I need a solution for on the order of 40,000... for which Flickr is perfect, since there is no storage limit, just a very small flat fee.
I have approx' 40k photos (not including the .raw files) over about 180GB (including the .raw files).
Uhm... how about the 200 GB plan? Or the 400 GB plan. Or 1 TB for $49.99/month? Or you could contact sales about Google Cloud Storage: https://cloud.google.com/pricing/cloud-storage Or you could just buy S3 storage.
Actually, there are literally dozens (to maybe even hundreds) of alternatives for hosting static content for less money for more bandwidth.
For example, a micro instance, with 500GB of EBS storage, with allowance for 1% data growth per daily snapshot (S3 backed), and 50GB of transfer per month will apparently cost $175.68.
The 500GB plan from Dropbox will cost you $499 .
I'm thinking, with enough data, Bittorrent Sync wins the pricing war, and I've been doing some testing on the client and it seems really resilient to data changing while syncing etc.
Long story short, I setup BS on both machines with an absolute minimum of fuss, copied the password over, and the folder synced. Definitely does one thing and does it well.
It distributes the files intelligently and makes optimal use of everyone's bandwidth. Dropbox, for example, slowly uploads all files to the cloud before distributing them, plus there's the space issue. AeroFS allows unlimited space, but is far slower than my Internet and LAN speeds allow, and does things like trying to upload the same file, linearly, to every peer at once. Cubby has limited space and has the same slow syncing problems.
I'm running BitTorrent Sync on my 6 year old Windows 7 Thinkpad, a newer Windows 8 desktop, a Digital Ocean Ubuntu VPS, and a Synology DS110j NAS. It runs perfectly on all of the above, and provides a useful web interface for the VPS and NAS.
Or does this simply make direct connections between PCs you own and people you have authorized to share your files?
> While Sync uses BitTorrent technology, people’s files are not accessible to outsiders. Only those who have the unique private key can access the shared folder.
>
> “All the traffic is encrypted using a private key derived from the shared secret. Your files can be viewed and received only by the people with whom you share your private secret,” BitTorrent explains.
There are relays if you need them due to firewalls:
"We also provide such additional methods of ensuring connectivity as relay and tracker servers."
But you can opt out of this config.
We already do this with git-annex.
I'm building a backup service for sql server, and wonder if this route could be better than rsync?
For example you can use TrueCrypt and create/mount a drive volume that is fully encrypted while synced across a file share storage/sync service like DropBox.
The only downside is that you have to install the TrueCrypt application on your client device, which does limit is platform offering (currently, no mobile).
http://www.truecrypt.org/
A better solution is to use .sparsebundles (Mac only) or something like BoxCryptor/encfs that encrypts files in the volume individually
I use encfs though, good suggestion.
https://www.dropbox.com/developers/core/api#chunked-upload
BoxCryptor does look like a great alternative; however, it's not open source and does come w/ a price (literally and figuratively - file names exposed if using free version or drop $50 USD).
Have you used BoxCryptor? Curious what your experience has been. I'll give it a try.
[1] http://en.wikipedia.org/wiki/Encfs
The real problem with using a Truecrypt volume on Dropbox is concurrent changes to the same volume. It happens if you mount the Truecrypt volume on 2 machines at the same time. It will mess up badly.
BitTorrent Sync synchronizes your files using a peer-to-peer (P2P) protocol. This protocol is very effective for transferring large files across multiple devices, and is very similar to the powerful protocol used by applications like µTorrent and BitTorrent. The data is transferred in pieces from each of the syncing devices, and BitTorrent Sync chooses the optimal algorithm to make sure you have a maximum download and upload speed during the process.
The devices you setup to sync are connected directly using UDP, NAT traversal and UPnP port mapping. We also provide such additional methods of ensuring connectivity as relay and tracker servers. If your devices are on the same local network, BitTorrent Sync will use your LAN for faster synchronization.
http://labs.bittorrent.com/experiments/sync/technology.html
The config file is versatile enough to allow you to turn off the relay servers / dht / upnp, etc and simply declare static peers, which is cool.
People will dismiss this as 'a toy that geeks use', but as Chris Dixon has noted, 'what the smartest people do on the weekend is what everyone else will do during the week in ten years' [1]
[1] http://cdixon.org/2013/03/02/what-the-smartest-people-do-on-...
With just a tiny bit of crypto code you could add a repudiation feature (keys signed by the secret vs the secret itself) and control access to both individual groups and individual users.
Imagine a private p2p site that you and your friends use. You choose who to share with and what files you can see.
Then add links to give to friends so attaching files in emails would be a sinch.