HN Explorer

Ask HN: Are there any projects to do historical archiving of RF spectrum?

32 points by adamgamble ↗ HN
Something like archive.org but for rf spectrum. I'm not even sure if its legal, but I was playing with my SDR the other day and thinking of all the historical information that passes over the airwaves that is ephemeral. Being able to view this data after a major event would be amazing. Imagine listening to all the radio information as 9/11 transpired or during the coup attempt in turkey.

Obviously this couldn't be done everywhere all the time, but it would be interesting to be able to archive radio spectrum during major events.

A quick google search didn't turn up anything.

26 comments

[ 3.0 ms ] story [ 25.9 ms ] thread
[deleted]
(comment deleted)
wyldfire
No but that sounds like a good idea!
rfw1z
there are commercial solutions that do this
vitovito
From my notes on a radio archiving project I'm doing:

> Could you record the entire radio spectrum and extract stations and broadcasts later?

> In the USA, the AM radio band is 540-1710 kHz, a spread of 1170 kHZ,

> http://rtl-sdr.better-than.tv/?page_id=237 states that spectrum recordings are also a function of how many samples per second you choose to record:

> 2.8msps - 44.8mbps = 5.6 MB/sec.

> 2msps - 32mbps = 4 MB/sec.

> 1msps - 16mbps 2 MB/sec.

> http://www.myradiobase.de/perseus/ has sample files. 3.5 minutes is ~360MB. 60 seconds of 1500 kHZ is a gig. All too much. 24 hours at 1MB/min is 1.5GB, but 17MB/sec. is 1.4TB. Spectrum recordings are out.

x1798DE
Assuming most of this is dead air, don't you think this would be a highly compressible data set (even lossless)?
niftich
Highly is perhaps an overstatement, but compressible like most clear signal data, behaving similar to PCM audio.

I compressed '20090922_950_GLFS_outdoor.wav' (150 kHz - 1750 kHz) from the Perseus site [1] with some lossless compressors. In each case, I independently round-tripped the de/compression and the results were confirmed to be lossless.

The compression ratios are in line or slightly (but not overwhelmingly) better than what you'd expect [2] from audio:

  original:		 1.000
  7zip-lzma2-xz-ultra:	 0.820 (not a signal compressor)
  wavpack-normal:	 0.724
  wavpack-high:		 0.691
  wavpack-hhx6:		 0.593
I would've liked to test additional compressors. In addition to WavPack, FLAC, TAK, ALAC, OptimFROG, TTA, WMA Lossless all purport to have some level of high-resolution support, but I'm not sure how many kHz they go up to. FLAC wouldn't take the file, and the rest I don't have access to right this moment.

[1] http://www.myradiobase.de/perseus/

[2] http://wiki.hydrogenaud.io/index.php?title=Lossless_comparis...

kastnerkyle
At least in HF, this "dead air" is anything but dead (lightning, channels with lots of fading, things you only catch due to ionospheric effects, etc). Noise is basically any signal you don't care about in lower frequency bands. Also many transmissions look like wideband noise at higher frequency due to direct sequence spreadspectrum type techniques.
derptacious
This is super interesting! And if anyone else is curious about creating a project that monitors the range of RF amplitude and frequency at many data points around the world, then send me your contact info! I want to see a record of this spectrum (human and non-human made) in frequency and amplitude to see how much it is (and hopefully "has") changed in physical space. I think interesting things would come out of this data.
adamgamble
Any idea if its legal? I assume it is but I have no idea.
gravypod
Listening is 100% fine in the USA. Transmission without a ham license, and in certain bands, will net you some fines and maybe jail time.
vitovito
Any spectrum recordings would almost be guaranteed to include copyrighted content, and in the US distributing those recordings would fall afoul of the DMCA.
gravypod
That's not how RF works. The courts have heard cases like this. Look up the talk on CreepyDOL from defcon.
vesche
As vitovito and pigeons mentioned storage and collection locations are the main issues. Things like this exist, but they don't archive data and are designed for military use- Google: Wideband Recording System . They typically are mobile units that focus on fast capture rate and decently large storage (a few terabytes).

sigidwiki is worth looking at, but it's more for classification: http://www.sigidwiki.com/wiki/Signal_Identification_Guide

niftich
I find this fascinating because the context of transmissions occurring simultaneously is indeed ephemeral.

This data gets lost with traditional archiving, which is squarely considers context (including temporal simultaneity across multiple objects) to be metadata.

These sorts of efforts can be partially retrofitted/approximated with timecodes on existing archived material, which may also be a separate, worthwhile endeavor to pursue.

kastnerkyle
As vitovito mentions, wideband data in high resolution is incredibly expensive from a storage perspective. There are entities that do it, but this also has the same privacy concerns as saving every packet that comes over a wire - in fact if you record 2.4 MHz and 5 MHz you will be picking up someone's (hopefully encrypted) private data. I don't think it is illegal, but the technical aspects of this are pretty daunting.

You would be better off decoding and storing that, but that gets into v& territory pretty quickly, depending on who and what you decode (definitely don't decrypt/crack).

elbrownos
Google was deemed to have broken wiretapping laws when it scooped up unsecured wifi data while collecting images for Streetview.

http://www.bbc.co.uk/news/technology-24047235

throwaway7767
In my country back in the eighties, the TV networks broadcasting scrambled content over the airwaves sued some people who were making a profit modifying receivers to unscramble the broadcasts without entering the numeric key (it was a very weak scheme).

The court ruled that no law was broken as the signals were being sent over the air, and therefore had no expectation of privacy. And since at that time the descramblers were sold to the user, there was no problem with modifying the hardware.

This is why they switched to leasing the descrambler boxes when they upgraded their system (and chose a system for which descramblers were not easily publicly available).

It always seemed a sensible ruling to me. Though I don't think anything has changed legally, I suspect the ruling might be different if such a case were tried today. :/

gravypod
Archiving the entire spectrum can range anywhere from infeasible to impossible. First you'd need to specify the frequency ranges you want to listen to.

Then you've got a data storage problem which has been covered.

After that you've got the propagation problem. Not everything is seen from everywhere.

In any event you might want to check out http://www.reversebeacon.net/

It's a project that's got a different "simpler" archiving goal. It is just to track spots of people calling CQ in the CW bands.

barrystaes
Likely there are top-secret satellites doing exactly that for specific geographic regions. These likely wont retain all gathered data indefinitely however, and even if data is shared its tainted politically.
privong
Most of the major radio astronomy observatories maintain archives (e.g., [0]). Of course, they mostly use highly directional dishes pointed at the sky, but local radio signals can create interference[1]. So, those archives provide some probe/record of terrestrial RF spectrum. Of course, those telescopes are generally placed in radio-quiet locations.

[0] https://archive.nrao.edu/archive/advquery.jsp

[1] http://www.gb.nrao.edu/IPG/