According to the LizardFS whitepaper [1], the chunks can be up to 64 MB. So I believe if the file is smaller, the chunk is just as large as necessary.
Looking at some chunk files on my system backs this up:
root@rem> ls -l | head
total 11702880
-rw-r----- 1 mfs mfs 33558528 Dec 12 18:17 chunk_ec_1_of_2_1_000000000000010C_00000001.mfs
-rw-r----- 1 mfs mfs 69632 Dec 12 18:12 chunk_ec_1_of_2_1_0000000000000179_00000001.mfs
-rw-r----- 1 mfs mfs 69632 Dec 12 18:11 chunk_ec_1_of_2_1_0000000000000187_00000001.mfs
-rw-r----- 1 mfs mfs 69632 Dec 12 18:26 chunk_ec_1_of_2_1_0000000000000194_00000001.mfs
-rw-r----- 1 mfs mfs 69632 Dec 12 18:06 chunk_ec_1_of_2_1_0000000000000197_00000001.mfs
-rw-r----- 1 mfs mfs 69632 Dec 12 18:36 chunk_ec_1_of_2_1_00000000000001A1_00000001.mfs
-rw-r----- 1 mfs mfs 69632 Dec 12 18:03 chunk_ec_1_of_2_1_00000000000001CE_00000001.mfs
-rw-r----- 1 mfs mfs 69632 Dec 12 18:13 chunk_ec_1_of_2_1_00000000000001D4_00000001.mfs
-rw-r----- 1 mfs mfs 135168 Dec 12 18:05 chunk_ec_1_of_2_1_0000000000000217_00000001.mfs
As for illumos/SmartOS -- maybe? The LizardFS cookbook [2] has a section for illumos, but there's nothing in it. It should probably work (it's C++), but there is probably going to have to be manual effort involved.
Depends entirely on what replication goals you have setup. My simple cluster (not EC or XOR) mandates 2 copies per chunk at a minimum. My 1-byte file was unaffected by a single chunk corruption, even on the node in which the corruption originated.
4 comments
[ 2.8 ms ] story [ 16.2 ms ] threadAnd what happens when the file is <= 64 MB?
Does it run on illumos / SmartOS?
Looking at some chunk files on my system backs this up:
As for illumos/SmartOS -- maybe? The LizardFS cookbook [2] has a section for illumos, but there's nothing in it. It should probably work (it's C++), but there is probably going to have to be manual effort involved.If the 1-byte file gets replicated across multiple disks, corrupt one of the chunks and report back.