Sounds more like a bug rather than a lack of memory to me ...
The flight plan did not contain an altitude for the flight, one of the sources said. While a controller entered the usual altitude for a U-2 plane - about 60,000 feet - the system began to consider all altitudes between ground level and infinity.
The conflict generated error messages and caused the system to begin cycling through restarts.
Yeah, let's hardcode a maximum altitude somewhere deep in the code and be completely surprised when, a couple of years in the future, newer, higher-flying planes/shuttles disappear from the system.
No one said anything about burying it, it can be at the top of the main configuration file if you insist. The basic problem is that one quantity was limited (memory), without dealing with another (altitude). It makes perfect sense to limit both, while you're at it. But of course most important of all, test and study the effects of going over those limits ...
Well, considering that altitudes above FL1000 have been, for the last 60-odd years, considered to be outside the FAA's jurisdiction... what is the problem exactly?
One presumes they still need to know the location of those flights to deal with them descending into the range of altitudes used by commercial aircraft.
No commercial aircraft operate at such altitudes. No, seriously: you run out of air for air-breathing engines above FL600 (the SR-71 flew at FL800 very fast indeed partly because it needed to go like a bat out of hell to keep enough airflow through its engines: and because it was able to go that fast due to there not being much air up there to produce drag), and the turbofans used on commercial airliners rapidly run out of wheeze above FL400.
The only civil aircraft that ever flew above FL500 were Concorde and the Tu-144, neither of which are in service. But they didn't fly at supersonic speed and high altitude in crowded airspace -- simply because getting an SST up to altitude and speed takes several minutes, a couple of hundred miles, and a prodigious amount of fuel. To say nothing of the sonic boom issue, which basically limited supercruise to over-ocean flight.
While there's been a lot of hot air about supersonic bizjets over the past two decades, so far nothing's escaped from the CAD package much less gone into flight testing.
So what you're asking for is for an FAA system to be retrofitted to handle exotic military requirements, future space shuttles, and a class of commercial aircraft that would not operate in that flight regime within a couple of hundred miles of any airport.
That's not what the article said. They said that since it didn't know the true altitude the computer assumed the plane could be at every altitude, and checked every single possible one.
This was made worse by a flight plan that had the plane going in and out of the area, so there were a tremendous number of permutations of possible flight paths.
The computer attempted to allocate memory to calculate every one of them and ran out of memory.
said Dan Kaminsky, co-founder of the White Ops security firm and an expert in attacks based on over-filling areas of computer memory.
That sounds more like buffer overrun than OOM. One is a Heartbleed-class vulnerability, the other is a pretty straightforward DoS situation. But of course why would we expect Reuters to be able to distinguish between the two...
Having worked on ATC, this really does sound like a software bug. Fixing such a bug on a delivered system would require an immense amount of costly testing. Adding memory is the cheap option.
The other thing to consider is that some of these systems are rather old and could easily have less than a gig of ram.
That sounds very worrying. The fact that this issue occurs shows that they didn't prepare for OOM conditions, and another issue could easily cause a different OOM condition they didn't prepare for. It's not inconceivable that there could be an issue which gobbles up an amount of memory you throw at it. It is a tragic irony that the fact that testing is costly may be the very reason this condition was not tested for.
Only in the sense of having a broken system, putting in place some procedural constraints and then praying that everyone follows the rules.
As you say, it's a software bug. Until it's been 100% characterised and tested this leaves a huge question mark hanging over the system. Adding memory is a band aid solution.
I would guess that it was some form of an numeric overflow, signed/unsigned issue or out of bound array access, because the only abnormal thing here is U2's altitude.
Atmospheric studies? Training in commercial environments? To see what happens? Because they can? Sekrit stuff?
If there's an actual military mission for any military plane that requires it to fly over or through a commercial area, wouldn't you prefer that pilots and ground operators already know how to do that?
It's not the most stealthy plane on earth, in fact it has been shot down over the Soviet Union and China in historical cases, because of being tracked by radar. Moreover it was flying over American airspace, it makes sense to register it.
A good idea when you consider that it could be mistaken for some bad guys plane, shot down and probably cause WWIII (not that long shot - the US and USSR were at a few times very close to annihilation due to glitches in their own software)
So much talk about Snowden, and then a newspaper reveals how to trigger a software bug than can shut down the entire air traffic control system for a large segment of the US. This is a genuinely damaging revelation, even if some critical details are lacking. Curious that the government is not up in arms about it.
A perfect example of why embedded or safety-critical systems should never dynamically allocate memory from a common pool.
Such systems should always preallocate fixed-size heapsfor each subsystem, each of which can then dynamically parcel out pieces of their own heaps if necessary. This helps limits the damage of any memory leaks in the system.
44 comments
[ 3.0 ms ] story [ 101 ms ] threadThe flight plan did not contain an altitude for the flight, one of the sources said. While a controller entered the usual altitude for a U-2 plane - about 60,000 feet - the system began to consider all altitudes between ground level and infinity.
The conflict generated error messages and caused the system to begin cycling through restarts.
Obviously a hardware problem...
Is there an aircraft of concern to the FAA that flies above FL1000? One or two historic flights have been that high, so raise the ceiling to 150,000'
still not infinite.
The only civil aircraft that ever flew above FL500 were Concorde and the Tu-144, neither of which are in service. But they didn't fly at supersonic speed and high altitude in crowded airspace -- simply because getting an SST up to altitude and speed takes several minutes, a couple of hundred miles, and a prodigious amount of fuel. To say nothing of the sonic boom issue, which basically limited supercruise to over-ocean flight.
While there's been a lot of hot air about supersonic bizjets over the past two decades, so far nothing's escaped from the CAD package much less gone into flight testing.
So what you're asking for is for an FAA system to be retrofitted to handle exotic military requirements, future space shuttles, and a class of commercial aircraft that would not operate in that flight regime within a couple of hundred miles of any airport.
I'm assuming they don't worry over satellites in orbit but they might have some concern if one decides to come back home.
This was made worse by a flight plan that had the plane going in and out of the area, so there were a tremendous number of permutations of possible flight paths.
The computer attempted to allocate memory to calculate every one of them and ran out of memory.
For example, Here is what the compiler for the 8-bit ATMEL Atmega 328 in an arduino does (on a platform with 32KB for program text and 2KB for RAM.)
That sounds more like buffer overrun than OOM. One is a Heartbleed-class vulnerability, the other is a pretty straightforward DoS situation. But of course why would we expect Reuters to be able to distinguish between the two...
The other thing to consider is that some of these systems are rather old and could easily have less than a gig of ram.
Only in the sense of having a broken system, putting in place some procedural constraints and then praying that everyone follows the rules.
As you say, it's a software bug. Until it's been 100% characterised and tested this leaves a huge question mark hanging over the system. Adding memory is a band aid solution.
And? Consider what this means exactly and how come the system saw the plane but didn't record the altitude.
(The AAS often appears in software engineering classes as an example of a classic "train wreck" project)
I thought such a plane would be 100% autonomous and be able to "see" and avoid other planes.
A U2 is not at all stealthy. You must be thinking of some other plane.
> if you have to register a flight plan
It's not a military operation in a foreign country, it's a routine flight in the home country.
> I thought such a plane would be 100% autonomous
The U2 was made long before such computers existed. They do have drones now.
> and be able to "see" and avoid other planes.
It wasn't the plane with the problem, it was ground control.
This must be a world record: A faulty assumption in every single sentence!
For 100% autonomous flight, I wasn't thinking no pilot, but about being able to navigate without any assistance from other radars
Is it worth asking, at all, why we're flying a military reconnaissance aircraft back-and-forth over Los Angeles?
If there's an actual military mission for any military plane that requires it to fly over or through a commercial area, wouldn't you prefer that pilots and ground operators already know how to do that?
Such systems should always preallocate fixed-size heapsfor each subsystem, each of which can then dynamically parcel out pieces of their own heaps if necessary. This helps limits the damage of any memory leaks in the system.