Sobering stuff indeed. What do we know about these events as far as mitigation goes? How long would power be disrupted or devices be out of commission if a similarly strong geomagnetic storm hit? Are we talking "lights out" for 3 days or 3 months? It's hard to glean from the article how literally one should take the sentence, "If it had hit, we would still be picking up the pieces."
From what I understand, there is widespread agreement on a grid-wide disconnect policy. The idea is: when a CME of sufficient scale is detected, all switchyards and substations go open a bit before impact (CME ions travel sub-c, so we have between 6 and 10 hours notice). There is a 2-3 hour worldwide blackout, after which we close circuits again.
There are several unsolved problems, such as, how do we dump surplus generation, particularly from nuclear reactors (you can SCRAM a reactor instantly, but you're not bringing it back up anytime soon), while retaining the ability to spin back up quickly? In what order do we disconnect and reconnect? Etc
As for satellites, I'm not sure. You need long antennas to induce the currents needed to destroy things, which is why the grid is far more vulnerable than, say, cars (contrary to the "cars don't run" narrative of prepper fiction) Maybe they get through just fine, maybe we lose and have to relaunch a lot of orbital assets? Not sure on this one.
After the attack on the Metcalf station in SV, a report was released about grid vulnerability to attacks. A solar storm would be able to do even more damage than targeted attacks on a few transformers.
Maybe we can have a starlink type grid of satellites to soak up some of this load, i have no idea.
Hopefully the advance warning (not possible for flares tho) protocol mentioned in the sibling comment will reduce alot of damage but i have no idea how effective that is for big storms.
There isn't anything we can really do other than put together protocols to handle the effects of the storm, yeah? Like... we can't re-engineer our power systems to resist being taken out by this, can we? Likely wouldn't be cost effective, I imagine.
BTW, its not mentioned in the article, but are the effects temporary or permanant? I imagine the GPS errors are temporary but it also mentions that it can take out power grids... will it just be a short, forced shutdown for a few hours (rimworld players: I'm wondering if its just like the "solar flare" event).
The voltage induced in power lines would destroy lots of things connected to them and would require replacement of trillions of dollars of equipment. So it would be permanent in the sense that you couldn’t fix all that equipment by turning it off and then on again.
I heard an expert say that the whole thing could be avoided by simply adding grounding wires to power lines. They kind of already do this for lightning — a grounding rod is shorted by a special material that becomes conductive at high voltage. The high voltage is “dumped” to earth. Put enough of those on power lines and the excess voltage might never reach sensitive equipment down stream.
Better solar science is another thing that can be done. Investing in better ways of forecasting problematic solar events. We do some of that today but much is in its infancy.
There have been discussions for years about EMP-proofing the U.S. grid, which would also protect against this (to whatever degree, I’m not qualified to speculate). I seem to recall it’s surprisingly affordable; there was an article about it on HN last year as I recall.
Part of preparation should include making our system less reliant on a centralized power grid. Shorten food supply lines. Everyone who can should be growing whatever they can in whatever dirt they have. Maybe drill more wells. If the power is out, globally, for even just a few months I’m guessing a very large percentage of the population of developed nations will die.
What a great post. What is the interstellar medium? Did the 12% figure include the aspect of multiple events paving roads for one another as mentioned later in the article? It reminds me of SARS, everyone knows it can happen and when it did our lack of preparation was shocking. And if this happens, once again the people hoarding toilet paper and food rations will be vindicated.
It seems like keeping your gear in a faraday cage would be enough to protect it, along with a really nice UPS and fiber optic IO. I’ve been meaning to buy some kind of fiber optic IO hub but they don’t seem to make one that’s appropriate for electro-isolation.
You could effectively build your own fiber optic I/O hub via fiber optic Thunderbolt cable to a Thunderbolt dock. Alternatively, build one via a combination of (1) fiber ethernet, (2) fiber HDMI/DP extender(s), and (3) fiber USB extender(s).
I asked my home insurance company if all of my electronics were covered in case of damage from such an event. They said yes. Are they unaware of how big an expense this would be for all of their insurers?
My understanding is that these sorts of events are most dangerous to very long cables (miles long, like telegraph cables or power lines). Unless there's surges as transformers blow up, your electronics at home should be fairly safe.
I think that's only true for small events. Big ones destroy everything, like a global EMP. Plus there's the fires from exploding transformers and other stuff on the grid. But big ones are more rare. Maybe this 2012 near miss killshot drained our quota for the next century. Here's hoping.
Still the sun is just now entering solar cycle 25, so the period of solar quiet is coming to an end.
I think they just updated the National Electric Code so that newer homes need to have Type 1 or Type 2 surge protection devices. The guy on the radio show I was listening to was talking about how this is a "whole home" surge protection.
I'm not sure if this would cover you in an event like this though.
Insurers also sold "business interruption insurance" which claimed to cover things like business interruption due to a pandemic by SARS or similar respiratory viruses (virii?) and once Covid hit they declined to pay (some cases are being litigated right now but it's likely to take years for a verdict to be reached).
> Are they unaware of how big an expense this would be for all of their insurers?
Yes but they think the likelihood of such events happening is nil. If it did occur and they had to pay-out for all the electronics of all their customers, then they'd simply refuse to pay. If they got taken to court and they were forced to pay, then they'd declare bankrupty and it would be a government problem then.
Insurance is great for limited disasters/issues. For wide-ranging systematic issues, insurance is a scam because they simply cannot make all their customers whole. Just like banks are great as long as everyone doesn't cash out. If everyone decides to cash out, then banks start to look scammy.
I was in Afghanistan for this. A huge portion of our field radios and cryptography devices were affected by this. I want to say I remember a crystal diode having to be replaced. These are used for timing.
Interestingly, the Marines keeps a stock of non-crystal diode radios for exactly this event. We had enough radios at the time not to need this, but cool to see this event was remembered by someone other than myself.
I remember some of the talk at the time speculated that our elevation was particularly impactful. Also, all the radios that went bad were either live or outside. Ones in shelters were not affected.
Wikipedia link [0], which describes telegraph operators getting shocked. I don’t think horse wires would be a problem, since I think the induced current depends on the change in flux, which needs 100s of km to get large enough. Our high voltage power distribution network though... could make 2003 [1] look like a cookout in the park.
What kind of effects might this have on modern passenger aircraft? Specifically the digital fly-by-wire.
After seeing how poorly the US government handled the Covid response I hope I don't have to experience a major solar event such as the one described in the article.
Realistically, limited. Passenger aircraft are routinely struck by lightning, which isn't entirely the same but is relatively similar. Given warning, I don't see airlines taking the risk, but that's driven by perceived liability rather than real risk.
Civilian FBW systems (essentially all of which are digital at this point, analog would likely be more vulnerable) are multiply-redundant which provides reasonable protection against upset events. The primary threat from solar events is the induced currents, not direct radiation that is a bigger threat to transistorized digital electronics. Flying aircraft are obviously electrically isolated from the electrical grid (which is where the real damage would take place), and relatively small (compared to hundreds of km of power lines), which would limit the range of induced currents in the airframe.
The real risk to aviation would be a loss of air traffic control which would risk mid-air collisions in busy areas.
Then I'm wondering how a mobile phone could pose such a threat to an aircraft. I imagine this sort of storm would cook their instruments, or at least make them go crazy for a while, and likely induce corruption in their digital systems. As we saw with bad software in 737 max control, trouble in those systems can down craft, i think you're underestimating the risk.
Mobile phones don't. There are practical and political reasons for not allowing phone use, but meaningful risk to the safety of flight is not one of them.
Instruments are fine, what could suffer problems are RF receivers (radio, radar, GNSS) but outside of interference during the event and the potential need to reboot problematic equipment, the critical instruments (pitot-static, accelerometers and gyros) aren't particularly exposed. In terms of effects to the aircraft systems directly, all that article mentions are upset events, and those are already a design consideration.
The 737-Max story came about specifically because it was not a FBW aircraft and not designed as such with sufficiently redundant systems. The safety case was that crews could disable the system if it misbehaved, rather than it being essential for flight like a real FBW system would be.
Cool, that's good to know, thanks. What's your background on this that makes you qualified to answer with certainty? Also what are political reasons for no phones on flights?
Background in electrical engineering, presently in robotics research, and a strong interest in avionics. The concerns that commercial aircraft face are shared with basically every other self-propelled electromechanical system (planes, cars, robots, spacecraft). Insufficient care in hardening these systems has resulted in (likely) fatalities before - evidence suggests that at least some of Toyota's "unintended acceleration incidents" were caused by cosmic-ray upsets in ECUs that weren't sufficiently redundant. Note that upsets are well enough understood for SpaceX to fly using redundant aerospace processors rather than the traditional radiation-hardened processors.
The political reasons are that no one wants a plane full of people talking on their phones and further distracted from listening to the safety briefing and flight crew instructions. Airlines don't want it and passengers don't want it. People do want network access on planes, which is widely available albeit slow. The only sincere technical concerns involve legacy radio equipment (ILS and glideslope), but I'm not aware of any demonstrated interference issues. Plenty of cases of interference from someone parking a large truck or 747 in front of the ILS antenna, though.
Cool, thanks for that. I really appreciate your time on this. It's rare to find a real expert amongst all the speculators (of which for most topics I am one) here. :)
How many redundant processors do you need per one "effective" processor you can count on for a typical SpaceX launch?
The simple answer is you need at least three to identify and recover from a single failure, five for two simultaneous failures, and so on (generally assuming failures can be recovered from automatically by rebooting the failed controller). Depending on the planned exposure, you can estimate the probability of upset events and thus the likelihood of multiple failures within the failure->reboot time interval and pick the number of computers accordingly. Radiation exposure depends on altitude - low-earth orbits outside of the Van Allen belts are fairly low due to protection from the Earth's magnetic field, while trips to other planets must be more hardened (either via shielding, significantly greater redundancy, or rad-hard circuit design).
The most difficult part, historically, is ensuring no single point of failure in a redundant system. Put three computers on a single bus, and it's likely each of the three bus transceivers could cause a complete system failure (so you've tripled the failure rate). In some systems like aircraft FBW, each of the controllers has its own connection to the actuators and its own actuator. The computers are connected to each other to detect if each other have failed, but as a fallback the control surface and actuators are designed so that two good actuators can physically overpower a bad actuator, and this ensures that the mechanical coupling doesn't become the failure point.
Thanks this is really interesting. It makes sense about how to calculate how many processors you'd need based on the time and upset frequency. Really appreciate your answers! :)
37 comments
[ 2.8 ms ] story [ 89.5 ms ] threadThere are several unsolved problems, such as, how do we dump surplus generation, particularly from nuclear reactors (you can SCRAM a reactor instantly, but you're not bringing it back up anytime soon), while retaining the ability to spin back up quickly? In what order do we disconnect and reconnect? Etc
As for satellites, I'm not sure. You need long antennas to induce the currents needed to destroy things, which is why the grid is far more vulnerable than, say, cars (contrary to the "cars don't run" narrative of prepper fiction) Maybe they get through just fine, maybe we lose and have to relaunch a lot of orbital assets? Not sure on this one.
https://news.ycombinator.com/item?id=24319585
After the attack on the Metcalf station in SV, a report was released about grid vulnerability to attacks. A solar storm would be able to do even more damage than targeted attacks on a few transformers.
https://www.nationalacademies.org/news/2012/11/electric-powe...
Maybe we can have a starlink type grid of satellites to soak up some of this load, i have no idea.
Hopefully the advance warning (not possible for flares tho) protocol mentioned in the sibling comment will reduce alot of damage but i have no idea how effective that is for big storms.
There isn't anything we can really do other than put together protocols to handle the effects of the storm, yeah? Like... we can't re-engineer our power systems to resist being taken out by this, can we? Likely wouldn't be cost effective, I imagine.
BTW, its not mentioned in the article, but are the effects temporary or permanant? I imagine the GPS errors are temporary but it also mentions that it can take out power grids... will it just be a short, forced shutdown for a few hours (rimworld players: I'm wondering if its just like the "solar flare" event).
I heard an expert say that the whole thing could be avoided by simply adding grounding wires to power lines. They kind of already do this for lightning — a grounding rod is shorted by a special material that becomes conductive at high voltage. The high voltage is “dumped” to earth. Put enough of those on power lines and the excess voltage might never reach sensitive equipment down stream.
It seems like keeping your gear in a faraday cage would be enough to protect it, along with a really nice UPS and fiber optic IO. I’ve been meaning to buy some kind of fiber optic IO hub but they don’t seem to make one that’s appropriate for electro-isolation.
Do you know of some reading material for someone interested in setting up something like this?
Still the sun is just now entering solar cycle 25, so the period of solar quiet is coming to an end.
I'm not sure if this would cover you in an event like this though.
Yes but they think the likelihood of such events happening is nil. If it did occur and they had to pay-out for all the electronics of all their customers, then they'd simply refuse to pay. If they got taken to court and they were forced to pay, then they'd declare bankrupty and it would be a government problem then.
Insurance is great for limited disasters/issues. For wide-ranging systematic issues, insurance is a scam because they simply cannot make all their customers whole. Just like banks are great as long as everyone doesn't cash out. If everyone decides to cash out, then banks start to look scammy.
Here is a video with simulations of coronal mass ejections [0] and an article on this by Phil Plait ("BadAstronomer")[1].
[0] https://www.youtube.com/watch?v=cLLq6plMjU0
[1] https://www.syfy.com/syfywire/2012-solar-disaster-almost-was
Interestingly, the Marines keeps a stock of non-crystal diode radios for exactly this event. We had enough radios at the time not to need this, but cool to see this event was remembered by someone other than myself.
I remember some of the talk at the time speculated that our elevation was particularly impactful. Also, all the radios that went bad were either live or outside. Ones in shelters were not affected.
[0] https://en.m.wikipedia.org/wiki/Carrington_Event
[1] https://en.m.wikipedia.org/wiki/Northeast_blackout_of_2003
After seeing how poorly the US government handled the Covid response I hope I don't have to experience a major solar event such as the one described in the article.
Civilian FBW systems (essentially all of which are digital at this point, analog would likely be more vulnerable) are multiply-redundant which provides reasonable protection against upset events. The primary threat from solar events is the induced currents, not direct radiation that is a bigger threat to transistorized digital electronics. Flying aircraft are obviously electrically isolated from the electrical grid (which is where the real damage would take place), and relatively small (compared to hundreds of km of power lines), which would limit the range of induced currents in the airframe.
The real risk to aviation would be a loss of air traffic control which would risk mid-air collisions in busy areas.
There's more info on aviation affects here https://www.skybrary.aero/index.php/Impact_of_Space_Weather_...
Instruments are fine, what could suffer problems are RF receivers (radio, radar, GNSS) but outside of interference during the event and the potential need to reboot problematic equipment, the critical instruments (pitot-static, accelerometers and gyros) aren't particularly exposed. In terms of effects to the aircraft systems directly, all that article mentions are upset events, and those are already a design consideration.
The 737-Max story came about specifically because it was not a FBW aircraft and not designed as such with sufficiently redundant systems. The safety case was that crews could disable the system if it misbehaved, rather than it being essential for flight like a real FBW system would be.
The political reasons are that no one wants a plane full of people talking on their phones and further distracted from listening to the safety briefing and flight crew instructions. Airlines don't want it and passengers don't want it. People do want network access on planes, which is widely available albeit slow. The only sincere technical concerns involve legacy radio equipment (ILS and glideslope), but I'm not aware of any demonstrated interference issues. Plenty of cases of interference from someone parking a large truck or 747 in front of the ILS antenna, though.
How many redundant processors do you need per one "effective" processor you can count on for a typical SpaceX launch?
The most difficult part, historically, is ensuring no single point of failure in a redundant system. Put three computers on a single bus, and it's likely each of the three bus transceivers could cause a complete system failure (so you've tripled the failure rate). In some systems like aircraft FBW, each of the controllers has its own connection to the actuators and its own actuator. The computers are connected to each other to detect if each other have failed, but as a fallback the control surface and actuators are designed so that two good actuators can physically overpower a bad actuator, and this ensures that the mechanical coupling doesn't become the failure point.