To be fair, that's a decent start. Especially if you're looking at prioritising where your customer base is. Linux is important, but nowhere near as much in comparison for the average consumer/business.
"different filesystems": maybe not. This new Dropbox thing is a network filesystem so it could just use normal syscalls to read and write files on any filesystem the ~/Dropbox directory and be (local) filesystem agnostic.
After all you can bypass the problem by running a Windows VM in VirtualBox with guest additions and a shared filesystem. Is another way to map the Windows Dropbox's network filesystem to the Linux one, whatever it is.
Still, very cumbersome and probably slow. I remember not stellar performances with sharing files between host and guest OSes in that way.
Hopefully Dropbox will release the API and somebody will write a user mode filesystem to interface this new service.
You're looking at the wrong end of it. Dropbox doesn't have to deal with any filesystem, Dropbox instead needs to build a filesystem.
It doesn't have to deal with any filesystem because Linux abstracts it. The same abstraction allows it to build its own filesystem really, really easily. How easy? I wrote, mounted, and used a filesystem in less than fifteen minutes. In node.js. It's that easy.
> desktop environments
Dropbox doesn't need to support various desktop environments, just a stable API for the desktop environments (actually, just file managers) to use. Isn't this how Nautilus, Thunar, Dolphin et al. already support Dropbox's existing features?
That may be true compared to other synchronization solutions on Linux, but I doubt Linux users make up a disproportionate amount of their customer base.
I think it's an unfortunate situation resulting from the need to be realistic about releases. If something can be released now for > 90% of the users then companies will prioritise doing that and then releasing the remaining 10% later.
The problem is when that remaining 10% takes way longer than expected.
This is huge. The biggest reason for me to not store more content on dropbox, is that it always comes with the trade-off of taking up space on my local drive (128gb) as well.
Sure there's selective sync, but that's an all-or nothing approach.
I like the way Google approaches this with their Google Photos service. X most recent photos are stored locally, and everything else is pulled down on the fly as you search/browse for them.
Not disagreeing, but I'm not seeing that on my wife's iphone. She got a 16 gig phone (big mistake), but we were thinking that getting everything on iCloud would make up for this. We pay every month for iCloud storage.
That's not what we're seeing. Her phone gets to be 100% full, and deleting a photo frees up space. When we delete a photo, we get dire warnings that "this will delete the photo on all of your devices, and in the cloud too." I download them to my computer before doing this, so nothing is really lost. But it's annoying.
In settings you can set it to "optimise iPhone space" (also possible on a Mac in the photos app) which will automatically remove local copies of old photos from the device if space runs low, and then download them again when you open them. For me it works very well, though I have had some trouble where it seems to need a few hundred free megabytes to start the process. If you delete photos with iCloud photo library enabled, they will be deleted from all your devices.
By the way, it works that way by design. iOS will attempt to fill all the storage, and will throw stuff away (essentially on an LRU basis) as needed. Free storage is wasted storage, just like having a lot of physical RAM free is wasted in a virtual memory system.
This is why I was amused by the breathless articles last week about how to "free space on your iPhone" -- essentially by clearing the cache by golly!
(If the 16GB is filled up 100% and things fail, that's an actual bug though. You might really have filled it up with stuff that doesn't use iCloud, leaving too little for the iCloud apps to work with)
BTW all of the above is orthogonal to "optimize for space / use lower resolution on portable devices" that others have helpfully mentioned.
My first guess is decoupled drivers with an abstraction that the actual dropbox client service uses. One driver for windows, one for mac. Though I don't know if you can do user mode filesystems on those, but there is this: https://dokan-dev.github.io/ Or else they may have used some fs event notification library to intercept access to files, but I think that would be harder (impossible? IDK win32/mac) to do than a filesystem.
From a first look on Windows, this is implemented with NTFS sparse files [1] and a file system minifilter. The sparse files act as placeholders so that you can browse your entire Dropbox structure. When you access a file, the minifilter sees that and starts fetching the data from Dropbox servers in the background. You can think of it as HSM [2]
The word 'revolutionary' definitely deprecated in the past years. Anyways, this is great news as I'll no longer have to default to the website search for selectively non-synced files.
This sounds like the exact feature I've been desperately looking for, for quite some time now, watching Microsoft discontinue online-only files in Onedrive because "it confused customers", and shaking my head at the blatant lack of this kind of thing everywhere...
Until just recently, when I found odrive[1]. Not affiliated with them in any way, just a fan - if you don't want to wait until Dropbox decides to launch this publicly, check it out. It works with Dropbox/Google Drive/OneDrive/CreativeCloud/what-have-you and does exactly what Dropbox Infinite promises to do, except that it's seasoned, working, practically bug-free (that I can tell), and, well, available now.
$99/year ($8.25/mo) to have "Premium" which offers the option to leave files in the cloud and sync on demand. This is too much money to gain a feature addition. No thanks.
Interesting... I just checked and apparently I'm on a grandfathered free plan that can still unsync. Didn't know they changed it.
For me, it's still more than just a feature addition though - I use odrive for all my cloud storage accounts. It's all under one roof. Your mileage may vary, of course.
Microsoft had what they called "placeholders" and it worked great, but it did confuse some users who looked for a file, saw it was "there" and happily went offline. Only to find it was only a placeholder. Furthermore, on devices with limited storage (phones, tablets) even the small size of the placeholders becomes a problem when the number of files is big enough. So Microsoft temporarily removed the placeholders functionality, promising to bring it back in the near future using some new (at the time not yet invented?) technique that would solve the problem.
Onedrive had similar same icon overlays to notify if the file was really downloaded or not. Dropbox is going to run into the same problem with placeholders where people go offline thinking they have the entire file.
This is a tools problem. People need to know how to use the tools in a business or let them go. You don't keep carpenter around who blames power tools not working the way he expects when he never charges the batteries.
Dropbox's response could be, "Hey we have training seminars for that".
App compatibility was also an issue with OneDrive placeholders (some apps couldn't handle placeholders). I wonder if Dropbox is going to run into similar issues with Project Infinite/
I wonder if Dropbox could just make use of this (e.g. by registering itself as an HSM backend provider to Windows somehow) rather than doing its own logic.
The cloud "wheel of reinvention" revolves to bring AFS to the masses? Or is this just showing the files using a file manager plugin, not visible as a real filesystem?
I really hope this is available to personal accounts, and not only business, as the blog post sort of implies. With the average laptop being 256GB and the Dropbox paid account being 1TB, that's badly needed.
I was thinking this as well. I have been really frustrated with selective sync because of how long the process takes to check the folders that make sense on each computer.
A reasonable question - it was my initial reaction.
However, from what I recall of how NFS worked it didn't sync anything locally - if it was remote it stayed remote. This seems to be an interesting hybrid.
NFS with selective offline caching and automatic re-syncing upon reconnetion. NFS never really did that. AFS and Coda kind of tried, I don't recall it working that well.
No. NFS requires a continuously online connection and pulls the metadata over it, so you have to wait for `ls` and `stat`. Caching in NFS is quite limited, and of course its authentication and network-traversal problems are hard work to deal with.
At first I thought "Well, Dropbox storage is 5x as expensive as hard drive space, so what's the point?". What I didn't see was the team use case: It's actually brilliant to have access to every document of all of my colleagues at all times, without saving all that stuff locally, or taking forever sync. This is really amazing.
The problem with our current shared team folders is that you need to make a deliberate effort to share something with the team. That means when you want to pull information from the shared space, you're very likely to get an outdated copy. Thanks to 1.) no storage constraints and 2.) deep OS-integration, all files can actually be always up-to-date with Dropbox Infinite. That really sounds cool.
> ..or taking forever sync. This is really amazing.
I don't think this solves the problem of taking forever to sync. Nothing has changed there. You are still bound by the same data retrieval and network latencies to get the file stored in a datacenter somewhere to you. That one is a harder problem to crack because it needs a lot of infra investment in expanding your content delivery footprint and replace SSD's with something much faster, like flash.
This solves an important aspect of the sync problem -- the ability to selectively sync individual files without having to sync the whole directory that it is part of (or even to have to know what the directory is in advance).
Dropbox has actually had Selective Sync for years. The whole part about the metadata being cached locally and a seamless experience at file system level is new though.
Dropbox storage is 5x as expensive as hard drive space, so what's the point?
Yes and no. I have about 80 GB in my dropbox. This is no problem on my two workstations or my big laptop. Unfortunately my Macbook Air has a 128 GB drive. That means I keep having to exclude and include folders to be synced to my laptop or it will run out of space. That is a pain in the ass I would love to avoid.
Microsoft had this feature built in windows 8 / 8.1 and removed it in Windows 10 cause it "confused" customers. Hope this will nudge them to bring it back.
It removed "placeholders" and added a selective Sync feature but its not the same. Placeholders experience felt more intuitive. Files that were only available online were marked so you know. I guess the problem started cause a lot of users would not be able to access files when offline and started complaining to their support service. Too bad though I loved that feature.
I disagree. While it is true that it is more expensive per GB, Dropbox has a lot of features that make it more fit for heavy lifting, such as:
* Chunked uploads. Try to modify a 1GB file in OneDrive. The complete file will be resynchronized. Dropbox just uploads the chunks that have changed.
* LAN sync. If you are sharing files with family/colleagues that are on the same network, Dropbox shares file chunks peer to peer. This usually results in much faster synchronisation than downloading from the cloud.
* Dropbox has file requests. People can upload files to your Dropbox, but what others upload is not visible.
* In my experience, the Dropbox client is a lot more stable than the OneDrive client on OS X. OneDrive crashes or sometimes cannot sync certain files.
To me the speediness, file requests, and more reliability is well worth whatever the price delta is.
(Note: I have both, but I only use OneDrive as an endpoint for Arq backups.)
To those of you who are dismissing this because AFS/NFS has had this for years, the point is that this is running over the Internet, not your local network.
My original thought was, so its a network drive huh?
But when you think about the fact that everyone has 128GB Surfaces they take everywhere remotely this is really powerful and I can image a lot easier to administer. Love how they have integrated and just added a simple icon. Well done Dropbox
I will curious about the performance. SSHFS/FUSE, expandrive /odrive renderings of your onedrive/cloud drive bucket and so on have been pretty bad in my experience.
The Dropbox Client has a lot of smarts in it when it comes to fast and reliable downloads (threading, chunking etc.), so it might probably be a lot better than many other alternatives.
297 comments
[ 5.8 ms ] story [ 281 ms ] threadLinux is also a much larger target than Mac OS or Windows ... different filesystems, desktop environments etc. etc.
After all you can bypass the problem by running a Windows VM in VirtualBox with guest additions and a shared filesystem. Is another way to map the Windows Dropbox's network filesystem to the Linux one, whatever it is.
Still, very cumbersome and probably slow. I remember not stellar performances with sharing files between host and guest OSes in that way.
Hopefully Dropbox will release the API and somebody will write a user mode filesystem to interface this new service.
You're looking at the wrong end of it. Dropbox doesn't have to deal with any filesystem, Dropbox instead needs to build a filesystem.
It doesn't have to deal with any filesystem because Linux abstracts it. The same abstraction allows it to build its own filesystem really, really easily. How easy? I wrote, mounted, and used a filesystem in less than fifteen minutes. In node.js. It's that easy.
> desktop environments
Dropbox doesn't need to support various desktop environments, just a stable API for the desktop environments (actually, just file managers) to use. Isn't this how Nautilus, Thunar, Dolphin et al. already support Dropbox's existing features?
Not saying they're doing that here, I'm sure it'll take time to implement.
The problem is when that remaining 10% takes way longer than expected.
Looking at you Google Drive.
Especially since this is only part of their Business product for now.
Sure there's selective sync, but that's an all-or nothing approach.
I like the way Google approaches this with their Google Photos service. X most recent photos are stored locally, and everything else is pulled down on the fly as you search/browse for them.
Which app are you talking about? The desktop app is merely an uploader. It doesn't really sync photos.
it randomly indexed all my wallpapers and threw them all into my timeline... i'm still not finished with the resulting cleanup.
though it might have been Google Picasa desktop app, i used both back then
That's not what we're seeing. Her phone gets to be 100% full, and deleting a photo frees up space. When we delete a photo, we get dire warnings that "this will delete the photo on all of your devices, and in the cloud too." I download them to my computer before doing this, so nothing is really lost. But it's annoying.
This is why I was amused by the breathless articles last week about how to "free space on your iPhone" -- essentially by clearing the cache by golly!
(If the 16GB is filled up 100% and things fail, that's an actual bug though. You might really have filled it up with stuff that doesn't use iCloud, leaving too little for the iCloud apps to work with)
BTW all of the above is orthogonal to "optimize for space / use lower resolution on portable devices" that others have helpfully mentioned.
Next step: migrate 100% of the OS in the Cloud?
How do you get a hold of a high level salesperson at Dropbox?
https://www.dropbox.com/developers-v1/core/docs#files-GET
Like SSHFS?
Edit:
More details here - https://news.ycombinator.com/item?id=11571640
This looks like an open source project that provides something similar for google drive: https://github.com/google/google-drive-shell-extension
On linux this could probably be even easier, just mounting a directory to a remote NFS server.
[1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...
[2] https://en.wikipedia.org/wiki/Hierarchical_storage_managemen...
Until just recently, when I found odrive[1]. Not affiliated with them in any way, just a fan - if you don't want to wait until Dropbox decides to launch this publicly, check it out. It works with Dropbox/Google Drive/OneDrive/CreativeCloud/what-have-you and does exactly what Dropbox Infinite promises to do, except that it's seasoned, working, practically bug-free (that I can tell), and, well, available now.
[1] https://www.odrive.com/
For me, it's still more than just a feature addition though - I use odrive for all my cloud storage accounts. It's all under one roof. Your mileage may vary, of course.
Dropbox's response could be, "Hey we have training seminars for that".
There's even a little Explorer-native file overlay for such files (a black clock): https://blogs.msdn.microsoft.com/oldnewthing/20030827-00/?p=...
I wonder if Dropbox could just make use of this (e.g. by registering itself as an HSM backend provider to Windows somehow) rather than doing its own logic.
edit: working at the FS level according to https://news.ycombinator.com/item?id=11571640 - good job Dropbox! Anyone know if it's also supported on Linux?
But for me it's too late.
However, from what I recall of how NFS worked it didn't sync anything locally - if it was remote it stayed remote. This seems to be an interesting hybrid.
Plus all the other basic Dropbox features
The problem with our current shared team folders is that you need to make a deliberate effort to share something with the team. That means when you want to pull information from the shared space, you're very likely to get an outdated copy. Thanks to 1.) no storage constraints and 2.) deep OS-integration, all files can actually be always up-to-date with Dropbox Infinite. That really sounds cool.
I don't think this solves the problem of taking forever to sync. Nothing has changed there. You are still bound by the same data retrieval and network latencies to get the file stored in a datacenter somewhere to you. That one is a harder problem to crack because it needs a lot of infra investment in expanding your content delivery footprint and replace SSD's with something much faster, like flash.
https://www.dropbox.com/en/help/4456
I already do this because I have to change my selective sync set quite often.
Yes and no. I have about 80 GB in my dropbox. This is no problem on my two workstations or my big laptop. Unfortunately my Macbook Air has a 128 GB drive. That means I keep having to exclude and include folders to be synced to my laptop or it will run out of space. That is a pain in the ass I would love to avoid.
* Chunked uploads. Try to modify a 1GB file in OneDrive. The complete file will be resynchronized. Dropbox just uploads the chunks that have changed.
* LAN sync. If you are sharing files with family/colleagues that are on the same network, Dropbox shares file chunks peer to peer. This usually results in much faster synchronisation than downloading from the cloud.
* Dropbox has file requests. People can upload files to your Dropbox, but what others upload is not visible.
* In my experience, the Dropbox client is a lot more stable than the OneDrive client on OS X. OneDrive crashes or sometimes cannot sync certain files.
To me the speediness, file requests, and more reliability is well worth whatever the price delta is.
(Note: I have both, but I only use OneDrive as an endpoint for Arq backups.)
But it was just an early one (1988!), many others followed. Coda, SFS, recently BackFS for FUSE, Sun's CacheFS and Red Hat's FS-Cache for NFS etc etc.
This seems vaguely astroturfy.