HN Explorer

The Great Zero Challenge (16systems.com)

28 points by tubby ↗ HN
I've heard for many years that in order to securely erase old hard drives that one must use a tool that makes multiple overwrites using random data. However, I've never seen data recovered from a hard drive that had been overwritten once with only zeros (no random data and only on pass). I've also seen heated debates in online forums over this topic. I really wish someone would take this challenge and put an end to this debate once and for all.

18 comments

[ 3.2 ms ] story [ 52.2 ms ] thread
pmjordan
Interesting. I've read the whole spiel about data being recoverable after being overwriteen many times and from many sources. I've always wondered whether it was true. I mean, I know about hysteresis loops, but given the size of the storage cells on a hard disk these days, it seemed really unlikely that they're not fully magnetised. If data recovery companies aren't going to even try, then I guess that pretty much confirms it's a myth.

I'd be intrigued whether it's possible to recover data on hard disks from 10, 15 years ago which have been treated this way. Back then, the magnetic cells were much, much bigger. What about floppies? I'm guessing the myth must have originated somewhere - although ignorance is a reasonable possibility I suppose.

Hexstream
"I'm guessing the myth must have originated somewhere - although ignorance is a reasonable possibility I suppose."

Imagination stems from trying to read from uninitialized memory, yielding an undefined value. :)

pmjordan
I don't think anyone claims it's possible to read said data using standard drive firmware, so comparing it to uninitialised memory isn't the whole story. I always figured the theory was that if the cell was magnetised twice, you could tell from the magnitude of the resulting field not only its current magnetisation but also its history. I can see how that might be possible based on magnetic properties of real materials. ( http://en.wikipedia.org/wiki/Hysteresis_loop#Magnetic_hyster... ) However, you'd have to take an analog reading, not a binary one. The idea here being that N, then S magnetisation ought to yield a weaker S magnetisation than S, S.

Strong enough magnetisation will erase that history though, and presumably make the current data more long-lived and random bit-flips rarer. I'm just wondering if the tech used back in the days wasn't sophisticated to magnetise cells strongly enough. (without affecting neighbouring cells)

Yes, I probably am thinking about this too much. I guess that serves me right for doing a physics degree at university. :)

bdr
I think Hexstream was talking about human memory
Tichy
The well-repected German magazine c't did the test a couple of years ago. They contacted three data recovery firms, and none could recover a drive that had been dd-ed with zeros once.

I wonder how to erase Flash-Drives, though.

pmjordan
That's a tricky one, as flash drives have special logic built in to avoid repeatedly overwriting the same block to prevent deterioration. On the other hand, bypassing that logic in order to read out the data in the extra blocks is not possible in software, (unless there's a backdoor) and I don't know if the logic sits on the controller chip or the actual flash chip. If it's the latter, it'll be pretty damn hard to get to.

EDIT +1 for mentioning c't

wmf
When I worked with flash chips they were quite dumb and well-standardized, so bypassing a controller should be no problem.
rw
No one is allowed to disassemble the drive! Because of that, this drive won't ever get professional-level treatment from a data recovery firm.
tel
If you can prove you're an established firm they'll let you have it for 30 days and disassemble it.
thaumaturgy
Alternatively, they're perfectly happy to sell you the drive for 60 bucks.
gojomo
The theory is that special expensive equipment could possibly do such a recovery -- so the 3-day time limit, and measly $40 prize, isn't really responsive to the question.

Further, if you were an agency with the budget and equipment to do this, would you want the world to know?

They aren't testing what they're trying to test, and even a 100x reward and 10-year time limit wouldn't prove the negative, "that recovering data from a zeroed hard drive is impossible".

A seminal paper on the possibility -- but not the reality -- of such specialized recovery is Peter Gutmann's 1996 "Secure Deletion of Data from Magnetic and Solid-State Memory" [ http://www.cs.auckland.ac.nz/%7Epgut001/pubs/secure_del.html ].

Guttman notes in an undated epilogue, however, that advances in data density and recording techniques since 1996 make any recovery from modern devices "unlikely". Still, the "Great Zero Challenge" provides very little in the form of real evidence about these questions.

pius
The terms are utter bullshit.

You may not write any data to the drive or disassemble it . . . .

The Gutmann paper referenced elsewhere in the thread concludes that overwriting the drive (something like 34 times IIRC) with zeroes is important because a dedicated analyst can measure the residual magnetism of each sector of the drive to infer the most recent "long term" binary values. Not allowing the drive to be opened makes this type of analysis kind of difficult.

xirium
It may be possible to retreive the data without openning a drive. I can think of two methods for achieving this objective. Firstly, I've seen reference to SCSI commands to retrieve "unbaked" sectors from CDROMs. Support for this functionality varies but it may be possible that some harddisks have undocumented functionality. You may wish to check the widespread implementation of DRM in harddisks for circumstantial evidence of such functionality. Alternatively, it may be possible to replace harddisk firmware which allows retrieval of magnetic traces. Again, I've seen reference to "low-level formatting" which wipes harddisk firmware. If the firmware is accessible in this manner then retreival is possible for almost all harddisks without openning them. It would also demonstrate that data recovery services are doing a shoddy job of imaging disks, running some standard recovery tools, and maybe performing some sector edits.

This test raises the bar because you have three days and writing to the disk is not allowed. That would leave you with three days to reverse engineer the existing firmware.

patrocles
Rename it the Zero-Clue Challenge.

They haven't learned a single thing from the recent uptick in challenge interest (RC4/5, DARPA, Netflix, etc.)....

imsteve
> You may not write any data to the drive or disassemble the drive.

What the heck?