Received! Thanks!
I noticed from your profile you wrote LOGO for python — I did the same for processing! http://terrapin.sourceforge.net/ Great minds :)
I love AeroFS. Although it is missing many of Dropbox's features (e.g. web storage of files, better version history, etc), the fact that it is extremely fast and there is no cap on P2P sync makes it my weapon of choice when sharing files/folders with colleagues.
What would this get someone who already has Dropbox and Air Drop on OS X?
Answer: compatibility! Now that walled gardens are going up, there's pain in getting through the walls. If they can make this compatible with Dropbox, it would be a great combination. There's also the possibility that Dropbox could add Bonjour/Rendevous to their client side software, reducing AeroFS to just a feature.
Not really, the attraction of AeroFS is not having a connection/reliance on S3. So for something like 1Password syncing, it would be great not to have a copy sitting somewhere in the cloud.
If you use a strong master password, and given that 1Password uses a secure key-derivation function, does it really matter if your 1Password file is sitting somewhere in the cloud?
My take on it is if there is no/little benefit of putting something in the cloud, there is no reason to do so particularly if it's something security related.
Thanks for that tip. I disabled the limiting now and it sped up from 11:42 minutes to 01:40 minutes! Now Dropbox is only x12.5 slower than AeroFS.
The real solution, of course, would be for Dropbox to transfer the file over the LAN immediately before waiting for it to be uploaded to the Dropbox servers.
That's not a solution, it's a design tradeoff. Dropbox's servers are the "truth", and the software's clear goal is to get a copy on the servers. There are many reasons for this--weird conflicts, inconsistencies, and so on--that are much harder to resolve than if there is a copy on the servers.
Once that happens, other clients will ask to download from the servers, notice that it's on the LAN already, and take it from there. I bet if you install a new client on the same LAN, or share a folder between clients on the LAN, or add a folder to selective sync, once the files are already there... it will be a competitive speed.
The Java to be disabling is the Java browser plugin, which allows un-trusted web sites to run applets in a sandbox on your machine. As you know, there have been a lot of issues with Java sandboxing.
However, AeroFS runs locally (as far as I understand, I don't have an invite) so should be fine as long as you trust the provenance of the code.
I believe those exploits are Java applets breaking out of the browser-plugin sandbox, which is designed to limit application access to even simple things like files and class loading. To be clear, these aren't strictly JVM vulnerabilities, and more plugin security-model vulnerabilities.
Java "desktop" applications don't have such a sandbox and thus they already have access to things that the vulnerabilities enable (such as manipulating system properties). Really, we ought to treat Java desktop apps no differently than native binaries, bounded by the the OS' security model.
Funny, I'm actually biased towards Dropbox. I use it for most of my stuff because of the versioning and the Packrat feature. But there's no reason for Dropbox not to do direct LAN syncing, and no reason for AeroFS not to offer infinite versioning. It may be more complex to design, but definitely feasible. So I don't think that this is an apples-vs-oranges comparison.
> But there's no reason for Dropbox not to do direct LAN syncing, and no reason for AeroFS not to offer infinite versioning.
Really, no reason? None at all?
For "infinite versioning", it's obvious - there's no upper bound on the amount of data the server has to store. Storage space costs money.
As for no direct LAN syncing, someone's already mentioned the reason - consistency. Syncing is already a hard problem, and it's dangerous to allow more things to go wrong because the clients synced but the server didn't receive all the data.
People often misunderstand what the Dropbox LAN Sync does.
The file needs to be already in the cloud before it can be synced over LAN. This means that when you add a new file, it will first get uploaded to Dropbox servers then transferred to other computers over LAN.
Considering that most home connections are asynchronous (slow up, fast down) the entire processes gets bottle-necked by that slow upload rate.
You've given an excellent answer to the question Why is Dropbox's LAN Sync so slow? but the more interesting question is Why don't the Dropbox engineers change how LAN Sync works so it won't be so slow anymore?
No, it doesn't download it from the master server. It transfers directly between the computers on the LAN. The only thing that has to go through the Dropbox servers is the master file list.
I can understand why, particularly given Dropbox's aim of being as simple as it can possibly be. At the moment the rule is simple. At the moment the cloud can be relied on as the authorative copy which everyone syncs with. If I have 3 or more local clients, all talking and sharing over LAN, many issues get very complicated.
It seems like it could be mitigated by having the client hash the file and notify the cloud of the hash of the file it intends to upload first. But I think your explanation for why they don't currently do LAN-only sharing is correct.
It does do that; I don't understand what you think it'll fix when it comes to multiple clients.
Do you want Dropbox to make a ghost entry with the contents and the hash, then start transferring it over LAN right away? That's going to be confusing when a ghosted file doesn't behave the same as a normal unsynced file. Especially if you start doing collaborative work and the version history is broken because that's cloud-based.
This is also how SpiderOak's LAN sync works. Both Dropbox and SpiderOak guarantee that the single point of truth is the cloud, for good reason.
Making exceptions to that approach (like allowing individual machines to sync with eachother directly before transferring to the cloud) creates opportunities for conflicts and inconsistencies that are very difficult to resolve because all the information might not be available.
Your second paragraph reads like a defense of CVS. git-annex handles this fine for me on my network at home with multiple machines and multiple copies of the repository/archive/drop-box-folder on each machine.
While it looks cool, it seems to me that it isn't entirely automatic. Sadly the vast majority of people will not go for anything less automatic than Dropbox.
Note that technically the file contents don't need to be in the cloud, but rather the metadata and indexing do. I'm guessing that the indexing is how Dropbox breaks the file into blocks, checksums them and does some form of duplicate detection to save on storage costs.
What's the best way of writing this. Is it x87 or 87x faster? Thank goodness it wasn't x86 faster as that would have confused me.
On a related note, I've been using Livedrive for a number of years now and they have a lan transfer option to obtain a file over the network than over the internet. I suspect however a lot of use cases are separated by the internet and generally aren't on the same network.
What would be awesome however is if after the sync both computers were able to contribute to uploading the file to the live server. So I could sync my files over lan to my laptop, go to work, then have my home/work computer upload them to the net, thus 2x quicker to store it in the cloud. But that would be a bit of an edge use case I think.
I would love an invite if anyone could spare one. My email address is hotspam{at}gmail.com. I've been on the sign up list for over a year but haven't heard a thing. Thanks in advance!
61 comments
[ 2.7 ms ] story [ 133 ms ] threadI signed up a long time ago for an AeroFS invite but had no joy. Anyone got one they could share? I'd love to try this out.
If anyone could spare me an invitation...
Answer: compatibility! Now that walled gardens are going up, there's pain in getting through the walls. If they can make this compatible with Dropbox, it would be a great combination. There's also the possibility that Dropbox could add Bonjour/Rendevous to their client side software, reducing AeroFS to just a feature.
My take on it is if there is no/little benefit of putting something in the cloud, there is no reason to do so particularly if it's something security related.
Not sure why this is taking upwards of 12 minutes for the OP.
http://i.imgur.com/wTWzG.png
The real solution, of course, would be for Dropbox to transfer the file over the LAN immediately before waiting for it to be uploaded to the Dropbox servers.
Once that happens, other clients will ask to download from the servers, notice that it's on the LAN already, and take it from there. I bet if you install a new client on the same LAN, or share a folder between clients on the LAN, or add a folder to selective sync, once the files are already there... it will be a competitive speed.
The Java to be disabling is the Java browser plugin, which allows un-trusted web sites to run applets in a sandbox on your machine. As you know, there have been a lot of issues with Java sandboxing.
However, AeroFS runs locally (as far as I understand, I don't have an invite) so should be fine as long as you trust the provenance of the code.
Java "desktop" applications don't have such a sandbox and thus they already have access to things that the vulnerabilities enable (such as manipulating system properties). Really, we ought to treat Java desktop apps no differently than native binaries, bounded by the the OS' security model.
Really, no reason? None at all?
For "infinite versioning", it's obvious - there's no upper bound on the amount of data the server has to store. Storage space costs money.
As for no direct LAN syncing, someone's already mentioned the reason - consistency. Syncing is already a hard problem, and it's dangerous to allow more things to go wrong because the clients synced but the server didn't receive all the data.
Also, uploading the data to Dropbox server, and then downloading again, wastes valuable quota for those under a data cap.
Do you want Dropbox to make a ghost entry with the contents and the hash, then start transferring it over LAN right away? That's going to be confusing when a ghosted file doesn't behave the same as a normal unsynced file. Especially if you start doing collaborative work and the version history is broken because that's cloud-based.
Making exceptions to that approach (like allowing individual machines to sync with eachother directly before transferring to the cloud) creates opportunities for conflicts and inconsistencies that are very difficult to resolve because all the information might not be available.
I guess its not automatic in the sense that it does not install itself, but then again neither does dropbox...
[1] http://git-annex.branchable.com/assistant/
On a related note, I've been using Livedrive for a number of years now and they have a lan transfer option to obtain a file over the network than over the internet. I suspect however a lot of use cases are separated by the internet and generally aren't on the same network.
What would be awesome however is if after the sync both computers were able to contribute to uploading the file to the live server. So I could sync my files over lan to my laptop, go to work, then have my home/work computer upload them to the net, thus 2x quicker to store it in the cloud. But that would be a bit of an edge use case I think.
A confusion index of 1.0 would be described as perfectly confused. But, as is often said, nothing is perfect. :)