30 comments

[ 3.0 ms ] story [ 83.3 ms ] thread
I have seen similarly weird results with flaky USB flash keys. A lot of those don't do any kind of hardware error detection.

The behavior described here can be explained by block device cache: You write a file to a device and the device doesn't report an error, but doesn't actually write the data due to some hardware error. When you read the file back, the content will come from cache in your RAM, not the device itself. Unmounting and mounting the device flushes the cache. When you then read the file again, it will come from the device and have the original content again.

what about using sync to flush changes to disk? does it flush the block device cache as well?
I think you have to play additional games with /proc/sys/vm/drop_caches then
Tried that as well, echo 3 > /proc/sys/vm/drop_caches
Some of the cheaper cards (and USB keys back in the day) sort of assume you'll use a single FAT[/32] partition and they understand the FAT and use it as poor man's TRIM information. They get super upset if you attempt anything that is not a single FAT[/32] partition. Some will actively forbid it by ignoring writes to the first sector of the disk and of the partition. Perhaps this is one of those cases?

EDIT: technically this is kosher as SD spec mandates FAT32 (just like SDXC spec mandates exFAT)

Wow, that's scary. Imagine a file system that kinda sorta passes the "looks like fat" test but works differently, causing the card to TRIM randomly all over your files :O
Frankly "active" storage is all manner of scary when you get down to it, as you have not the first clue what the firmware etc sitting between the CPU and the actual bits can get up to.
This is pretty cool, I am really liking it.
Sounds hasty. There are so many layers between you and your SD card it is hard to know sometimes which ones are broken.

Typically, if you can't use dd(1) to write to it, and the error dd(1) gives is not 'write protected media', then your card is essentially dead.

The interesting thing is that dd wouldn't even give us an error, but the write speeds were abysmal. Toward the end of the hackathon, we noticed that blocks would be corrupted here and there, while leaving other pieces of data (even entire MP3 files) intact.

I think we ended up pointing to issues with the SD card's bus as the most likely cause for the behavior we saw, but we weren't even certain about this.

    Jordan messes up and rm -rf / instead of rm -rf /dev/sdb
slapstick classic
Err, even 'rm -rf /dev/sdb' wouldn't do what they want...

Better to overwrite it completely with dd.

We used dd, gparted, parted to wipe not only the MBR and partition table but also the whole card.
I feel like there's some preface here that I'm missing.
Same here. I get that they're having trouble changing the FS on a MicroSD/writing an image on it. But what is XCalibur?
Maybe it's all some joke about how this one MicroSD card is just like excalibur? In that you can't get useful data out...
All of the data was read-only, regardless of what adapter we put it in (we tried like five different ones). So you could retrieve the data that was baked onto it, but formatting the card was a lost cause.

So getting it to format was like trying to pull Excalibur out of the stone!

the joke became that as we explained the futility of formatting the SD card to any new person, they'd instantly think "oh no, they probably haven't tried x, I bet I could do it" and try it and fail in a similar fashion to how everyone thought pulling excalibur from stone would be an easy feat, but trying it, they failed
Of course, my stupidest hackathon project makes it to HN. Great.
that night was the best 9 hours i wasted on something tbh
I had exactly the same issue (except formatting my system).

SD card was broken, and it placed itself into read-only state for data recovery. I could delete files, but those would reappear. Formating would work until restart etc. Write cache was responsible for this weird magic.

I got card replaced as part of warranty.