Love the idea of this. I've tried to use something similar (at least in function), called Cryptomator. However, Cryptomator would always choke on larger files or directories with thousands of items (e.g. my Google Photos gallery). Will be interested to see if this or another solution is what I'm looking for.
Boxcryptor is a paid solution, similar to Cryptomator. I think I gave each one a try out on my phone, but ultimately just went back to using plain Google Drive and Google Photos. It fell into the category of "what is the real tangible benefit of this to me actually?" The real cost was added complexity, decreased reliability, a worse photos experience, and possibly another subscription cost.
True, the messy bit with Cryptomator is that it's not really very easily to share files with others. I've found it more of a tool really for an individual to use. I find myself sometimes training people on it a recommending it to people and then switching them to something like Tresorit when they have problems.
I just started using Cryptomator for sensitive files on a self-hosted NextCloud (since i access it via internet). The photos thing was an issue so I just sync those unencrypted and use Cryptomator for non-media files - i suppose if I had a photo that I wouldn't want exposed i'd just move it individually over.
How do you have it setup? I had similar issues when I was trying to access my WebDAV vault directly from Cryptomator on Ubuntu. I setup Nextcloud to sync directly with my local Ubuntu install and pointed Cryptomator at the vault in the synced directory and it works great! It opens all the files in the vault almost instantly and they’re navigable as if they're local files.
Mobile is still a problem though. A gallery system would be a big help but you’re always stuck with a bandwidth limit unless you cache the vault locally which for my vault isn’t feasible and also undesirable as I don’t want the files on my device.
You'd have to find someone to permanently pin the data (nobody will be interested in random encrypted databoxes without being paid for it). And at that point you're just loosing performance compared to S3 or WebDAV.
Yes you can pay to have someone pin it--there are pinning services--but you can also just replicate it among your own nodes. It would be comparable to Dropbox plus encryption.
Another thought less well explored would be to swap space with friends such that you hold each others pins but you can't read your friends' encrypted blocks. This is kind of the premise behind filecoin, which lets you buy space (ie if you want someone to hold your blocks) and sell space (ie, mining is equal to holding someone's blocks).
I would point out that IPFS provides no encryption out of the box, you're just layering this below something that encrypts that could equally well work on Dropbox. Filecoin atleast has a integrated payment structure but modern storage providers are pretty damn cheap (I pay ~10$ for 3TB of data including ingress/egress) When I last checked by filecoin they could only barely compete with AWS storage options (which quickly get very expensive)
Transport encryption isn't something I'd mention when talking about "Encrypted File Storage", I'd assume that to either don't matter or be the default.
Dropbox and Google could easily encrypt the files locally with your own key prior to uploading them to their servers. But of course they won't do that, because then how else will they mine your data?
Also, you probably shouldn't trust them even if they did offer this anyway.
If the key for the file is the hash of the file, which they encrypt with the user’s passphrase, they can prevent anyone who didn’t originally have the file from being able to read it, including themselves.
Sure, if you don't want search or sharing or 90% of the other features of Google Drive then encrypting on the client side makes great sense. For people who demand that their computers are useful, it doesn't make any sense.
Sharing can be accomplished like encrypted file-hosting sites do: encoding a decryption key in the anchor of the URL. Search of course is harder, but it's a bit unreasonable that it's not an option at all.
I haven't looked at native OpenZFS encryption... The method I used for a while was having a Veracrypt volume locally of all my files, and zfs-sending it to a remote location. It is a large file, several gigabytes. However, any updates made locally would get synced remotely at a block level vs. a file level, so the whole encrypted file didn't need synced. Much better than rsync, for example.
I did not read the entire F'in article yet, I just stepped in while a CI/CD pipe was running at work :) BUT! this looks like a pretty accessible article about filesystems.
I also wonder if it is named UtahFS because of the NSA facility near Salt Lake City.
I'm not sure what the karma threshold is, but you might be able to open the dead comment and click vouch to make it replyable. I went ahead and did that since it seems to be a good comment. I don't know why so many of their recent comments are dead. Some I can understand, but others...
I don't know about the crypto design of OpenZFS, but I have been running Ubuntu 20.04 with root pool using OpenZFS crypto since it's release. You have to make a small hack to the installer to do it, but otherwise it seems to be working great.
They should use Peergos [1], which satisfies all their requirements. It's built on top of ipfs, encrypts metadata as well as data, and hides file sizes and directory structure too. A directory is indistinguishable from a small file. We can also use S3 compatible object storage, including allowing direct reads and writes from the browser. Disclaimer: I'm the founder.
We use a similar structure to their oram, a merkle champ, with random labels. Some operations do move blocks to new labels, but reads currently don't because of the overhead as well as wanting to support multiple concurrent readers.
ORAM is pretty costly for a feature that many people don’t actually need. Yes it is log(n) for reads and writes, but the bigger downside is that ORAM by design thrashes any cache you happen to be using. I imagine this would be particularly noticeable for filename resolution with deep directories.
Plus, ORAM provides no protection out of the box for timing based attacks.
One way to achieve encrypted remote file storage is to combine Linux's EncFS² with some other type of remote storage protocol such as CIFS.
The basic idea is to mount a CIFS volume from a server you don't trust with your unencrypted files, then you mount that directory again with EncFS to get to the unencrypted files.
The main drawbacks are that EncFS will not hide your directory structure and file sizes and also that not all of the EncFS vulnerabilities that were discovered in 2014 have been addressed with an EncFS 2.0 release.
I find it difficult to set up a good system for encrypted storage.
1. Cryptomator: it's immature and buggy, especially the 1.5 version. See comments in forum.cryptomator. The files and folders disappear, vaults crash, vaults fail to mount, etc.
Boxcryptor is the paid version and not buggy. But it's not open source.
2. EncFS. Has security issues that haven't been resolved.
3. CryFS. Too slow and immature.
4. Encrypted backup, like rclone or duplicity. These are not sync tools.
5. eCryptfs: Used for Ubuntu home encryption (even then somewhat outdated), not for cloud.
6. AWS KMS: server side encryption; amazon has the keys.
7. Gocryptfs: It's OK. Reasonably fast. Cons: command line only, and for Linux. Uses OpenSSL library which isn't all that secure.
Regarding your first paragraph, that’s always going to be a tradeoff that has to be assessed based on threat model and resource budget, since doing it differently will inherently leak sidechannel information, right?
Gocryptfs has a comparison of these projets, here [2].
Focused on Windows only, to my experience, securefs is the one that is working the best as it is not using dokany but winfsp (FUSE for Windows). With all other solutions using dokany, the copy or sync of large number of files is damn slow or hanging.
44 comments
[ 3.9 ms ] story [ 87.6 ms ] threadMobile is still a problem though. A gallery system would be a big help but you’re always stuck with a bandwidth limit unless you cache the vault locally which for my vault isn’t feasible and also undesirable as I don’t want the files on my device.
Another thought less well explored would be to swap space with friends such that you hold each others pins but you can't read your friends' encrypted blocks. This is kind of the premise behind filecoin, which lets you buy space (ie if you want someone to hold your blocks) and sell space (ie, mining is equal to holding someone's blocks).
Also, you probably shouldn't trust them even if they did offer this anyway.
I haven't looked at native OpenZFS encryption... The method I used for a while was having a Veracrypt volume locally of all my files, and zfs-sending it to a remote location. It is a large file, several gigabytes. However, any updates made locally would get synced remotely at a block level vs. a file level, so the whole encrypted file didn't need synced. Much better than rsync, for example.
I did not read the entire F'in article yet, I just stepped in while a CI/CD pipe was running at work :) BUT! this looks like a pretty accessible article about filesystems.
I also wonder if it is named UtahFS because of the NSA facility near Salt Lake City.
Yeah I wondered this too. They don't say why they chose that wording. Maybe it's a subtle nod to that datacenter
I documented the installer hack at: https://linsomniac.gitlab.io/post/2020-04-09-ubuntu-2004-enc...
Before that I was running my storage server using LUKS on the raw devices and ZFS on top of that, since 2008, and that worked well.
It will be interesting to see how this project does against its competition.
SiriKali[2] is a front end to a majority of these projects and it works on Linux, MACOS and windows.
[1] https://nuetzlich.net/gocryptfs/comparison/
[2] https://mhogomchungu.github.io/sirikali/
1. https://github.com/peergos/peergos
Plus, ORAM provides no protection out of the box for timing based attacks.
> Keep in mind that this system is not used in production at Cloudflare: it’s a proof-of-concept that I built
The main drawbacks are that EncFS will not hide your directory structure and file sizes and also that not all of the EncFS vulnerabilities that were discovered in 2014 have been addressed with an EncFS 2.0 release.
--
² or one of its similar alternatives of course
1. Cryptomator: it's immature and buggy, especially the 1.5 version. See comments in forum.cryptomator. The files and folders disappear, vaults crash, vaults fail to mount, etc.
Boxcryptor is the paid version and not buggy. But it's not open source.
2. EncFS. Has security issues that haven't been resolved.
3. CryFS. Too slow and immature.
4. Encrypted backup, like rclone or duplicity. These are not sync tools.
5. eCryptfs: Used for Ubuntu home encryption (even then somewhat outdated), not for cloud.
6. AWS KMS: server side encryption; amazon has the keys.
7. Gocryptfs: It's OK. Reasonably fast. Cons: command line only, and for Linux. Uses OpenSSL library which isn't all that secure.
It seems to me gocrytfs is the best among these.
Also it lacks authentication. The snapshots of the XTS mode are prone to certain attacks.
The second one, TIL and good points!
Gocryptfs has a comparison of these projets, here [2].
Focused on Windows only, to my experience, securefs is the one that is working the best as it is not using dokany but winfsp (FUSE for Windows). With all other solutions using dokany, the copy or sync of large number of files is damn slow or hanging.
[1] https://github.com/netheril96/securefs
[2] https://nuetzlich.net/gocryptfs/comparison/
Love the crypto fs work you're doing at Cloudflare.