Ask HN: Has the race condition code of 2003 NE Blackout ever been released?

quote: GE Power is no Johnny-come-lately – their annual revenues are in excess of $ 20 billion – and their XA/21 system has over 3 million operational hours (340 years!) since it was first introduced in 1990. So, on August 14th, 2003, when 50 million people in the Northeast United States and Canada lost power, I am guessing that their QA staff in Florida went home pretty unconcerned that their XA/21 energy management and supervisory control and data acquisition (EMS/SCADA) system had any role to play in the ‘perfect storm’ that hit that day.

But by late October, after GE and their energy consultants from KEMA Inc spent weeks going through 4 million of lines of C/C++ code, they had identified the race condition that led to operators at FirstEnergy Corp’s Akron, Ohio control room being in the dark while three of the company's high voltage lines sagged into unkempt trees and tripped. Sometimes working late into the night and the early hours of the morning, the team pored over the approximately one-million lines of code that comprise the XA/21's Alarm and Event Processing Routine, written in the C and C++ programming languages. Eventually they were able to reproduce the Ohio alarm crash in GE Energy's Florida laboratory, says Mike Unum [manager of commercial solutions at GE Energy]. "It took us a considerable amount of time to go in and reconstruct the events." In the end, they had to slow down the system, injecting deliberate delays in the code while feeding alarm inputs to the program. About eight weeks after the blackout, the bug was unmasked as a particularly subtle incarnation of a common programming error called a "race condition," triggered on August 14th by a perfect storm of events and alarm conditions on the equipment being monitoring. The bug had a window of opportunity measured in milliseconds.

“There was a couple of processes that were in contention for a common data structure, and through a software coding error in one of the application processes, they were both able to get write access to a data structure at the same time," says Unum. "And that corruption lead to the alarm event application getting into an infinite loop and spinning."