It's even more impressive since the footnote says:
"This story first appeared in the Magazine of Fantasy and Science Fiction, December 1961"
Fifty-five years ago!
It reminds me of a recent hn article (if anyone has a link it would be great!) that I believe it was about how Dropbox tests it's storage system, layer by layer, and the final error graph is always a flat line at "0", since every error is caught and corrected in one of the layers below.
Will it ever reach the case where it will be impossible to really guarantee that we have the right data?
PS: I can't seem to find the article using Algolia or even Google... maybe it was not Dropbox but other storage company? I really would like to find this story but for some reason never marked it as favorite or even upvoted it! (maybe I wasn't logged in).
> we use a variant on Reed-Solomon erasure coding that is similar to Local Reconstruction Codes
> according to this model, a given block in Magic Pocket is safe with 99.9999999999999999999999999% probability!
anyone else notice that some of the transcriptions seem wrong, assuming a fixed 8-bit encoding? for instance. the "rrange" doesn't have two columns with identical bits.
This is pure buzz. Physical storage does not make any sense when light based or magnetic based storage is much faster and denser. It only makes sense if it is very permanent, ie. Etching sapphire discs [1] The position of atoms is just one classical property. Quantum properties like spin and charge are much more numerous and versatile.
True, unfortunately most scientists are nowadays practically obligated to state a potential real world use case for their research in their publications even when they are engaging in basic research. And mainstream media rather naively picks up on these statements.
And while browsing their product sheet [1], I thought this would be great for me to store my most valuable stuff. And then it hit me... if I bought the 500 or even the 2500 document disc... what would I save there?
Probably some family pictures (unfortunately it doesn't seem to support colors, understandably so), a paper I published years ago. Maybe my college thesis. Legal documents perhaps like birth certificates and such.
But then I wished I could store music. Is there any graphical representation of music that would suit this format? Or is the only reasonable option to store the actual 1's and 0's of an MP3 or something like that?
Fidelity is going to be a big issue though. I don't think the sapphire discs have enough storage density and capacity to hold a spectrogram of even mediocre sample rate. If you do the math let me know if I'm wrong. I'd be curious to listen to an mp3 af the simulated fidelity of a sapphire disc.
The resolution is too low to store even a single stereo FLAC file of a three minute song.
But this isn't really a viable digital storage system. It's a nice novelty, but it's not designed for serialisation in the way that optical ROM systems are.
It might be possible to increase the resolution with a better laser and optics, and make the disks serialisable. But I suspect you still wouldn't get the density you need for practical media storage.
Their smallest listed item (1" medallion) is 110 megapixels. Even storing just 1 bit per pixel (which is conservative) you can store almost 3 minutes of CD-quality FLAC[1]. Seeing as it's grayscale, you can almost certainly store multiple bits per pixel. At even 2bpp, you would have plenty of space for 3 minutes of flac, plus some form of BCH code.
The largest listed item is 30GP, which would probably be enough for over a day of FLAC audio.
1: 110Mbit / (32bits/sample * 44100 samples/s) ~= 78s of uncompressed audio. FLAC is roughly 50% compression ratio, so ~= 2:36 of FLAC.
[edit]
In case people were wondering: the reason this didn't pass my "sniff" test is that War and Peace is shown on an example prototype. That is ~3MB in UTF-8 and a grayscale picture of a page of text at a readable resolution is about 2 orders of magnitude larger than the binary representation, and 300MB is obviously way more than a 3 minute FLAC.
While this is very intesting, the storage mentioned does seem somewhat small to me. The article compares the area covered by the text to be a little smaller than a HIV (Human Immunodeficiency Virus). However, the genome of HIV apparently stores approximately 9.2kb (kilo basepairs, not kilobytes, so ~18KiB) of information. It also doesn't need to be kept at -196C. Are there any viable tactics for atom-based storage that could rival this density?
Genetic storage is way more resistant to bit rot than conventional computers. DNA is not stable, has a full time support system fixing it, and still has a relatively high mutation rate. But biological systems on the whole are much more resistant to individual errors.
Yes, single mutations can cause horrible diseases, but these mutations are either in the active site of proteins, or disrupt folding. A large portion of proteins are "bulk." If you look at protein simulations, only a small section of the protein is simulated, and the rest is just approximated as charged mass. Changes there are much less impactful and noticeable. As for it to be a disease, it had to be present at time of concept. If it happens during life (which is does, millions of times a day), the cell either ignores it or dies.
I am not entirely sure how the calculations are done in order to determine the information contained in the genome of HIV, but my suspicion is that the reason this is more dense is due to the genome containing more than binary bits. I'm guessing that in the genome, for every atom that could be at a certain location in the genome, that would represent a different information state. So each location corresponds to more than a binary possibility (atom A, atom B, atom C, etc.). I think this is not the current goal as our purposes with memory storage have more constraints that merely density, what might increase density significantly could potentially decrease read/write speeds or maybe even energy required to perform read/write operations.
All of this is speculation and difficult for me to say for certain since I am not intimately knowledgeable on the topic, but that is my general guess.
The storage in the article is 2D surface. DNA storage in viruses is highly compacted in 3D. Single atom storage is limited by the interatomic distance and the accessibility of the atoms (you could use a series of 2D surfaces stacked on each other to increase density). DNA storage is roughly 30 atoms/base or bit. So, probably the single atom method could ultimately be scaled further.
Well OK it spirals around, but isn't RNA/DNA really just linear? To access, you need an enzyme to walk it forwards or backwards and there are no other options that I know of. (I'm not a biologist)
No, it's not. RNA/DNA contains 4D of information. You need hundreds of proteins (also to be clear not enzymes - all enzymes are proteins, not all proteins are enzymes). Additionally (along with proteins), you need to unpack the chromatin structures - which requires huge cellular state transitions (hence the 4th dimension). So what bases are available to be replicated (read?) is dependent on the chromatin structure and their current state (euchromatin/heterochromatin), then depending on local molecular concentrations, replication is dependented on what proteins are locally available (due to the nucleosome structure). This is the very essence to why a large number of traits and diseases do not follow mendelian inheritance.
More likely, complex traits are influenced by gene regulation, which is affected by chromatin structure, but chromatin is just one form of regulation.
Complex traits are more likely just due to many interaction elements (promoters, enhancers affecting multiple genes in a nonlinear way) and the complex feedback mechanisms and redundancy mechanisms.
Chromatin state is completely abolished and recovered (in a non-deterministic way) in egg and sperm, which places some limits on which it can influence inheritence.
yeah it's 1-dimensional and processing I know about is sequential, twist is there just to make it more flexible inside the cell similar to old telephone cord (something about putting 3-meter line into few tens of micrometer sized cell)
The van der Waals radius of chlorine is 0.175 nm and they use cells with 2 by 3 atoms - two positions for zero and one and one atom wide border as separation to the neighboring cells. This suggests a size of 0.735 nm² per bit or 136 Tbit/cm² which is within a factor of two from the 78 Tbit/cm² from the article and mostly due to the additional spacing between blocks.
They stored 8,128 bits in a plane, the virus contains about 18,400 bits but in a three-dimensional structure. If we could layer this technology - again with one layer for separation - we would get a density of 15.5 billion Tbit/cm³. You could store two years worth of global IP traffic (2015) in one cubic centimeter.
There was a paper in the last few years proposing to make DNA a practical mass-storage technology, as the last and slowest level of the hierarchy. It'd need lots of error correction, and to be chunked up into small-enough strings to synthesize, and each string would need to carry address bits to match on (unless you want a content-addressable store). But IIRC the storage density and the cost trends made it at least kind of plausible.
From -269ºC to -196ºC is actually a huge improvement. Its the difference between liquid helium (expensive) to liquid nitrogen (cheap). Getting something that cold is hard, but keeping it there is just a matter of insulation. There seems to be a different kind of physics when you get that cold. Stable single atom structures, superconducting magnets, etc. It's all based around the idea that nanometer structures are now stable.
The big issue with bringing these kind of products to the general market is that we operate at a much higher temperature, the range of liquid water. And these technologies may never be able to improved to a point that they work at room temperature. Instead of moving those technologies into our range, I think could focus on moving our technologies to that range and improving/shrinking containment vessels. Self contained units with periodic maintenance/refills are viable (eg vacuum tubes). Having someone to top off a super computer's liquid nitrogen every couple of weeks is not a hard ask. We just have to make everything else work at that temperature so the entire unit can be cooled and contained, instead of just a specific section. Integrated circuit boards are already black boxes to 99.9% of the public, so sealing them in a cooled vessel is viable.
I know MRI/NMR machines already operate at that range, but they are way to big/specialized to be considered the general market. No one has an MRI machine in their medicine cabinet. I was once interning for a chemist, and asked if he ever thought we would have desktop NMR machines. He said there was no way they would ever be small enough. I then pointed at his laptop and remarked that is what people thought about computers 50 years ago.
The problem I see is what happens if the temperature doesn't stay in the required range for whatever reason like power failures, earthquakes, sloppy maintenance - you lose the data. Having such an enormous storage that is so fragile seems to have limited usefulness. If they ever get it to room temperature and have it be stabile then it might be more interesting.
We already have a demand for very large, very-high-speed storage with in-memory databases. We already fill this need with highly-volatile RAM, and back it with more-persistent magnetic storage.
From the article, this is seen as a replacement for persistent storage, for which it will need a lot more stability and temperature work.
There is an existing engineering discipline centered around keeping a lot of precious fragile data at liquid nitrogen temperatures for decades without malfunction. Say what you want about cryonics but the storage part is proven technology.
"Say what you want about cryonics but the storage part is proven technology"
I've always wanted to say that cryonics as a field of study gives me indigestion. I would also like to say that all who study cryonics are villains and fiends. There. I said it. I feel so much better, thank you.
Power failures of short/intermediate duration aren't necessarily an issue: liquid N2 makes a great thermal reservoir.
In biology there's a fairly good track record of keeping samples cold (granted, mostly at the relatively balmy -80) for many years, generally without mishap.
And anyhow, storage device failure is already considered inevitable, it's mitigated by replication.
Atomic level storage does not have to necessarily be brought to the common liquid water temperature range user. There is a lot of massive storage demand from parties that are not necessarily interested to get in touch with the physical storage medium itself. These would benefit right here and now as soon as the technology could be mass-produced.
> Instead of moving those technologies into our range, I think could focus on moving our technologies to that range and improving/shrinking containment vessels.
> I was once interning for a chemist, and asked if he ever thought we would have desktop NMR machines. He said there was no way they would ever be small enough. I then pointed at his laptop and remarked that is what people thought about computers 50 years ago.
Good-resolution NMR requires high frequencies and high magnetic fields. A 900 MHz NMR would use around 20 T--which is strong enough to pretty much require superconducting cryogenic magnets. Admittedly, a 100 MHz NMR requires only about 2-3T, which is close to what can be produced with a neodymium magnet, but you do get much, much noisier spectra.
With the larger field strengths, the need for magnetic field shielding pretty much restricts the ability to shrink it down to bench-scale machines.
(My understanding is that these machines came on the market for certain applications where the low resolution wasn't as much of a problem as other things like affordability or portability.)
I am wondering if the technique described in this Nature publication from yesterday [1] to potentially operate quantum computers at room temperature, could be used for the atomic storage as well. Does anyone know?
> Dr Otte reports read speeds of 1-2 minutes per block. He reckons he could boost those speeds drastically, to about a megabit per second. That would be a big improvement, but still thousands of times slower than modern hard drives.
Try tens of times slower. There would absolutely be applications for nonvolatile memory that reads at 1 Mbps but has this kind of insane memory density. For most applications you'd probably have to make the technology stable at room temperature though.
Thousands is correct. A magnetic hard disk has a bandwidth of well over 1 Gbps.
Long-term stability (and this would only be any good for long-term storage, at that rate!) is a pretty tall order. Even current storage methods can't really manage it.
This is because it just got published in Nature something, the earlier open arXiv version from April is here: https://arxiv.org/abs/1604.02265
Figure 3 is absolutely spectacular, and deserves to be admired by the whole world.
A comment on the inevitable promises of revolutionized data storage: Yes the areal density is fantastically high (>500Tbit / square inch as opposed to 1Tbit/in^2 in bleeding edge HDDs / NAND flash, if I crunched the numbers correctly). But it requires ultrahigh vacuum, preparation of a clean copper crystal, dosing with copper chloride and then writing/reading with a scanning tunneling microscope, maintaining liquid nitrogen temperature.
Also a modern SSD will write about 500MB in a second, while this method would write 500MB in 240 years.
I don't mean to slag it off; we should all appreciate it for being an absolutely wonderful and awe inspiring technical feat. Just don't get carried away dreaming of the applications in your laptop/server/phone.
Pretty cool, but keep in mind that it's at a massive scalability disadvantage compared to flash because of the requirement for a read/write head. Current 3D flash may only be 1.7Tbpsi, but that's with a layer 4 microns thick. Make it a millimeter thick and you're in the same areal density range.
Datacenter on the moon? At -200 degrees Celsius in the shade, the temperature on the moon is perfect. Of course, moving the data would be a problem but it would be a great backup solution!
77 comments
[ 3.2 ms ] story [ 88.4 ms ] threadIt's even more impressive since the footnote says:
"This story first appeared in the Magazine of Fantasy and Science Fiction, December 1961"
Fifty-five years ago!
It reminds me of a recent hn article (if anyone has a link it would be great!) that I believe it was about how Dropbox tests it's storage system, layer by layer, and the final error graph is always a flat line at "0", since every error is caught and corrected in one of the layers below.
Will it ever reach the case where it will be impossible to really guarantee that we have the right data?
PS: I can't seem to find the article using Algolia or even Google... maybe it was not Dropbox but other storage company? I really would like to find this story but for some reason never marked it as favorite or even upvoted it! (maybe I wasn't logged in).
I tried searching for "petabyte" but I guess I came 1 order of magnitude short...
Thanks a lot! It was driving me crazy..
[1] http://www.digitaltrends.com/cool-tech/nanoform-laser-etched...
And while browsing their product sheet [1], I thought this would be great for me to store my most valuable stuff. And then it hit me... if I bought the 500 or even the 2500 document disc... what would I save there?
Probably some family pictures (unfortunately it doesn't seem to support colors, understandably so), a paper I published years ago. Maybe my college thesis. Legal documents perhaps like birth certificates and such.
But then I wished I could store music. Is there any graphical representation of music that would suit this format? Or is the only reasonable option to store the actual 1's and 0's of an MP3 or something like that?
[1] http://www.fahrenheit2451.com/index.php/shop
Fidelity is going to be a big issue though. I don't think the sapphire discs have enough storage density and capacity to hold a spectrogram of even mediocre sample rate. If you do the math let me know if I'm wrong. I'd be curious to listen to an mp3 af the simulated fidelity of a sapphire disc.
But this isn't really a viable digital storage system. It's a nice novelty, but it's not designed for serialisation in the way that optical ROM systems are.
It might be possible to increase the resolution with a better laser and optics, and make the disks serialisable. But I suspect you still wouldn't get the density you need for practical media storage.
Their smallest listed item (1" medallion) is 110 megapixels. Even storing just 1 bit per pixel (which is conservative) you can store almost 3 minutes of CD-quality FLAC[1]. Seeing as it's grayscale, you can almost certainly store multiple bits per pixel. At even 2bpp, you would have plenty of space for 3 minutes of flac, plus some form of BCH code.
The largest listed item is 30GP, which would probably be enough for over a day of FLAC audio.
1: 110Mbit / (32bits/sample * 44100 samples/s) ~= 78s of uncompressed audio. FLAC is roughly 50% compression ratio, so ~= 2:36 of FLAC.
[edit]
In case people were wondering: the reason this didn't pass my "sniff" test is that War and Peace is shown on an example prototype. That is ~3MB in UTF-8 and a grayscale picture of a page of text at a readable resolution is about 2 orders of magnitude larger than the binary representation, and 300MB is obviously way more than a 3 minute FLAC.
Yes, single mutations can cause horrible diseases, but these mutations are either in the active site of proteins, or disrupt folding. A large portion of proteins are "bulk." If you look at protein simulations, only a small section of the protein is simulated, and the rest is just approximated as charged mass. Changes there are much less impactful and noticeable. As for it to be a disease, it had to be present at time of concept. If it happens during life (which is does, millions of times a day), the cell either ignores it or dies.
All of this is speculation and difficult for me to say for certain since I am not intimately knowledgeable on the topic, but that is my general guess.
No, it's not. RNA/DNA contains 4D of information. You need hundreds of proteins (also to be clear not enzymes - all enzymes are proteins, not all proteins are enzymes). Additionally (along with proteins), you need to unpack the chromatin structures - which requires huge cellular state transitions (hence the 4th dimension). So what bases are available to be replicated (read?) is dependent on the chromatin structure and their current state (euchromatin/heterochromatin), then depending on local molecular concentrations, replication is dependented on what proteins are locally available (due to the nucleosome structure). This is the very essence to why a large number of traits and diseases do not follow mendelian inheritance.
Complex traits are more likely just due to many interaction elements (promoters, enhancers affecting multiple genes in a nonlinear way) and the complex feedback mechanisms and redundancy mechanisms.
Chromatin state is completely abolished and recovered (in a non-deterministic way) in egg and sperm, which places some limits on which it can influence inheritence.
They stored 8,128 bits in a plane, the virus contains about 18,400 bits but in a three-dimensional structure. If we could layer this technology - again with one layer for separation - we would get a density of 15.5 billion Tbit/cm³. You could store two years worth of global IP traffic (2015) in one cubic centimeter.
And of course there's http://e-drexler.com/d/06/00/Nanosystems/toc.html chapter 12.
The big issue with bringing these kind of products to the general market is that we operate at a much higher temperature, the range of liquid water. And these technologies may never be able to improved to a point that they work at room temperature. Instead of moving those technologies into our range, I think could focus on moving our technologies to that range and improving/shrinking containment vessels. Self contained units with periodic maintenance/refills are viable (eg vacuum tubes). Having someone to top off a super computer's liquid nitrogen every couple of weeks is not a hard ask. We just have to make everything else work at that temperature so the entire unit can be cooled and contained, instead of just a specific section. Integrated circuit boards are already black boxes to 99.9% of the public, so sealing them in a cooled vessel is viable.
I know MRI/NMR machines already operate at that range, but they are way to big/specialized to be considered the general market. No one has an MRI machine in their medicine cabinet. I was once interning for a chemist, and asked if he ever thought we would have desktop NMR machines. He said there was no way they would ever be small enough. I then pointed at his laptop and remarked that is what people thought about computers 50 years ago.
From the article, this is seen as a replacement for persistent storage, for which it will need a lot more stability and temperature work.
http://www.merriam-webster.com/dictionary/track%20record
I've always wanted to say that cryonics as a field of study gives me indigestion. I would also like to say that all who study cryonics are villains and fiends. There. I said it. I feel so much better, thank you.
Seems like a good beginning to a Crichton novel.
In biology there's a fairly good track record of keeping samples cold (granted, mostly at the relatively balmy -80) for many years, generally without mishap.
And anyhow, storage device failure is already considered inevitable, it's mitigated by replication.
Redundancy, Redundancy, Redundancy.
Which could cause you to lose room-temperature storage too.
Take advantage of the density and put your data into 6 data centers instead of 3. It'll survive fine.
I think there are plenty of data centers that could learn to deal handily with liquid nitrogen temp cooling equipment.
4K and 77K are the same temperatures in absolute units.
The warmer experiment is twenty times hotter than the colder one.
For comparison in human terms, 20x hotter than room temperature is ~5700 C.
Edit: Mmm. Morning math mistakes. Thanks!
20 times room temperature that is 300K is 6000K, as hot as the shining Sun.
Smartphones with liquid nitrogen cooling? :)
Good-resolution NMR requires high frequencies and high magnetic fields. A 900 MHz NMR would use around 20 T--which is strong enough to pretty much require superconducting cryogenic magnets. Admittedly, a 100 MHz NMR requires only about 2-3T, which is close to what can be produced with a neodymium magnet, but you do get much, much noisier spectra.
With the larger field strengths, the need for magnetic field shielding pretty much restricts the ability to shrink it down to bench-scale machines.
(My understanding is that these machines came on the market for certain applications where the low resolution wasn't as much of a problem as other things like affordability or portability.)
Here's an MRI probe that's quite literally microscopic. Fig 1 says it's about 3.5µm in diameter. [1]
[1] http://arxiv.org/pdf/1603.04238v2.pdf
[1]: http://www.nature.com/ncomms/2016/160718/ncomms12232/full/nc...
Try tens of times slower. There would absolutely be applications for nonvolatile memory that reads at 1 Mbps but has this kind of insane memory density. For most applications you'd probably have to make the technology stable at room temperature though.
Long-term stability (and this would only be any good for long-term storage, at that rate!) is a pretty tall order. Even current storage methods can't really manage it.
But read speeds on a spinning HDD are typically going to be between 50-100MBps, so 400-800 Mbps.
I would say "paving the way to large-scale storage" is extremely generous for what they actually achieved, to the point of being incorrect.