Very cool. I wrote up a spec very similar to this (though I just used the hash itself for the folders, as in HA/HAS/HASH structure [there's probably a name for that scheme]), but haven't gotten around to implementing…
I hate hierarchical filesystems. A lot of my older files, sadly, are stored in "SORT/Sort Me/To be sorted/Old computer/Sort again/Miscellaneous..." and the like. My server has an mlocate index, so I'll use mlocate, and…
My HBA is an LSI 9207-8e (http://www.lsi.com/products/host-bus-adapters/pages/lsi-sas-...) and my two SAS expander chassis are NORCO DS-24E (http://www.norcotek.com/item_detail.php?categoryid=8&modelno...), which are…
Yeah, my oldest vdevs were configured with 512B block size, because that was the default and the ZFS community wasn't being particularly loud about ashift=12 being a good idea until later. As far as I know, there is no…
If you haven't seen it, you might find it interesting: http://research.google.com/archive/disk_failures.pdf
Almost all of the bitrot I see is on the oldest vdevs, which at this point probably contain mostly only old snapshots that are almost never accessed. My oldest vdevs are... 4-5 years old.
Yes, I should, and someday I will. I'm playing it fast and loose right now. :-p
My understanding is that some significant (thousands of bytes) silent corruption is inevitable when you start reaching huge capacities over long-ish periods of time; even cosmic radiation has the potential to flip a few…
Well, if you don't get a kick out of "tweaking and worrying", yes, I definitely recommend FreeNAS. Although I'm confident in my system now, it took a long time to get this way, and I could've saved hundreds of hours by…
As far as I'm concerned, the most identifiable characteristic of Debian is the packaging system, dpkg/apt. I've used FreeBSD occasionally, and that's what I always end up missing about Debian. I did consider going with…
I only have 32GB of RAM for ~73TiB under ZFS management, so I have huge SSD cache devices in my pool. The cache devices really lower ZFS' need for RAM, and it works well for me. Before the cache devices, 32GB felt very…
ZFS still protects you from bitrot when compared to ext4 over mdraid. When you get to many terabytes of data, it's almost guaranteed that you're going to lose something to bitrot. In my case, my most recent scrub…
It means I ran a scrub that completed on February 21, 2014 after almost 202 hours, and there were no errors, but there were some corrupt data blocks that were repaired from mirrors, adding up to 1.58MB of data that was…
By "something like this", do you mean ZFS? I am a HUGE fan of ZFS, and I do think that it's worth using in any situation where data integrity is a high priority. As far as ZFS on Linux, it still has its wrinkles. I use…
As you can see, I do not perform scrubs as often as I should. Until I switched to SAS a year ago, I wasn't able to complete a scrub at all. The scrub you see is one of the few I've been able to complete. I need a week…
I've got some external hard drives that I rotate in and store off-site but still nearby (down the street) for some data. I also have constant online backups running to various locations/services (Linode, AWS, CrashPlan,…
I'm in Ohio, where it apparently averages around 0.11USD per kWh, so probably closer to ~320USD here.
~330 watts.
I haven't experienced ANY drive failures. Which means I'm probably on track for one soon... When I started out, I began with a single 5-disk vdev using a SATA port multiplier enclosure to connect the drives. Over time,…
Huh, here I am using iTerm2 every day and I didn't realize it had support for this. Fantastic.
The ReGIS support is nice, but with the custom functionality it would be nice if instead of echoing an ASCII string like "HTERMFILEXFER" it used actual escape sequences. Terminology[1] does this correctly with its own…
I'd imagine this works similarly to how a slice of bread in a bag of cookies keeps the cookies chewy, as the sugar attracts and holds the water from the bread, and keeps the cookies from drying out. Something to do with…
What are you looking for in a city, besides friendly people?
I've used xpra for about a year now, and I do recommend it if it seems like it would fill a need in your workflow. For me, I'm using it primarily as a way to run Pidgin (for Jabber and AIM, primarily) on my server that…
Very cool. I wrote up a spec very similar to this (though I just used the hash itself for the folders, as in HA/HAS/HASH structure [there's probably a name for that scheme]), but haven't gotten around to implementing…
I hate hierarchical filesystems. A lot of my older files, sadly, are stored in "SORT/Sort Me/To be sorted/Old computer/Sort again/Miscellaneous..." and the like. My server has an mlocate index, so I'll use mlocate, and…
My HBA is an LSI 9207-8e (http://www.lsi.com/products/host-bus-adapters/pages/lsi-sas-...) and my two SAS expander chassis are NORCO DS-24E (http://www.norcotek.com/item_detail.php?categoryid=8&modelno...), which are…
Yeah, my oldest vdevs were configured with 512B block size, because that was the default and the ZFS community wasn't being particularly loud about ashift=12 being a good idea until later. As far as I know, there is no…
If you haven't seen it, you might find it interesting: http://research.google.com/archive/disk_failures.pdf
Almost all of the bitrot I see is on the oldest vdevs, which at this point probably contain mostly only old snapshots that are almost never accessed. My oldest vdevs are... 4-5 years old.
Yes, I should, and someday I will. I'm playing it fast and loose right now. :-p
My understanding is that some significant (thousands of bytes) silent corruption is inevitable when you start reaching huge capacities over long-ish periods of time; even cosmic radiation has the potential to flip a few…
Well, if you don't get a kick out of "tweaking and worrying", yes, I definitely recommend FreeNAS. Although I'm confident in my system now, it took a long time to get this way, and I could've saved hundreds of hours by…
As far as I'm concerned, the most identifiable characteristic of Debian is the packaging system, dpkg/apt. I've used FreeBSD occasionally, and that's what I always end up missing about Debian. I did consider going with…
I only have 32GB of RAM for ~73TiB under ZFS management, so I have huge SSD cache devices in my pool. The cache devices really lower ZFS' need for RAM, and it works well for me. Before the cache devices, 32GB felt very…
ZFS still protects you from bitrot when compared to ext4 over mdraid. When you get to many terabytes of data, it's almost guaranteed that you're going to lose something to bitrot. In my case, my most recent scrub…
It means I ran a scrub that completed on February 21, 2014 after almost 202 hours, and there were no errors, but there were some corrupt data blocks that were repaired from mirrors, adding up to 1.58MB of data that was…
By "something like this", do you mean ZFS? I am a HUGE fan of ZFS, and I do think that it's worth using in any situation where data integrity is a high priority. As far as ZFS on Linux, it still has its wrinkles. I use…
As you can see, I do not perform scrubs as often as I should. Until I switched to SAS a year ago, I wasn't able to complete a scrub at all. The scrub you see is one of the few I've been able to complete. I need a week…
I've got some external hard drives that I rotate in and store off-site but still nearby (down the street) for some data. I also have constant online backups running to various locations/services (Linode, AWS, CrashPlan,…
I've got some external hard drives that I rotate in and store off-site but still nearby (down the street) for some data. I also have constant online backups running to various locations/services (Linode, AWS, CrashPlan,…
I'm in Ohio, where it apparently averages around 0.11USD per kWh, so probably closer to ~320USD here.
~330 watts.
I haven't experienced ANY drive failures. Which means I'm probably on track for one soon... When I started out, I began with a single 5-disk vdev using a SATA port multiplier enclosure to connect the drives. Over time,…
Huh, here I am using iTerm2 every day and I didn't realize it had support for this. Fantastic.
The ReGIS support is nice, but with the custom functionality it would be nice if instead of echoing an ASCII string like "HTERMFILEXFER" it used actual escape sequences. Terminology[1] does this correctly with its own…
I'd imagine this works similarly to how a slice of bread in a bag of cookies keeps the cookies chewy, as the sugar attracts and holds the water from the bread, and keeps the cookies from drying out. Something to do with…
What are you looking for in a city, besides friendly people?
I've used xpra for about a year now, and I do recommend it if it seems like it would fill a need in your workflow. For me, I'm using it primarily as a way to run Pidgin (for Jabber and AIM, primarily) on my server that…