"The current software generated too many nuisance warnings and resulted in poor sensor performance. Further work on software had been slowed by testing required to validate earlier fixes, the report said."
"It said Lockheed had delivered F-35 jets with 50 percent or less of the software capabilities required by its production contracts with the Pentagon."
Whenever I read about software that needs to be perfect I think of this article [1].
> But how much work the software does is not what makes it remarkable. What makes it remarkable is how well the software works. This software never crashes. It never needs to be re-booted. This software is bug-free. It is perfect, as perfect as human beings have achieved. Consider these stats : the last three versions of the program -- each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors.
If that 10 billion is accurate (I can believe 10GB of binaries, but 10 billion lines seems too much to me, even including support software), maybe the project is too ambitious.
Probably. But there are a lot of constraints and requirements on the software development process for aircraft and other critical applications. https://en.wikipedia.org/wiki/DO-178B
Well, yes, that's what I mean. But it's apparent that their planning and process was so poor that they could not appropriately schedule the entire project in keeping with the requirements.
I spent two months doing some side work for the NIH on some programming stuff, which was mediated through Lockheed. They were so disorganized and clueless than I'm amazed they are able to build a sandwich, let alone a jet fighter.
For example, the onboarding meeting I had with their HR folks was roughly 90% about retirement benefits and health care plans, neither of which I was eligible for as a temporary part-timer. This still cost NIH two hours of my time even so. And it took them weeks to get me the right codes so I could actually log my hours in their system the way I was supposed to, on a project that was supposed to last two months in total.
And in all of this, they provided not one shred of added value to the project that I could detect. Were it not for whatever bureaucratic rule required them as a middleman, I could have conceivably raised my rates 50%, cost the actual customer less, and been more productive without them.
The Joint Strike Fighter program is a prime example of the military not learning anything from its failures in the past (like the F-111). Hell, they're not even learning from the failures they've had in the history of the JSF.
If you've got 50 percent or less of the software capabilities in a system with 8 million lines of code, and it's supposed to be deployed in 18 months you're in deep shit. Yet, the program's backers blithely proclaim they're going to make their dates, ignoring the fact that the program is years overdue and 70% over budget.
Not only has an enormous amount of good money been thrown after bad, the military has staked the future of air power on this one aircraft so no one wants to admit that it should be cancelled.
From what I understand main issue is that they are trying to build "transformer". Basically they want Navy, Marines, and Airforce to use same software/chassis.
Obviously Navy, Marines, and Airforce all want different features, and requirement to the chassis renders it extremely frail (they had to cut a lot of weight for VTOL capability).
So considering that, it actually might be good idea to stop completely, fire all top airforce/navy/marine brass involved and restart as 3 separate projects from scratch, albeit using all the technology already developed.
Morale is already super low and it is hard to see bright future for something so overpriced, over budget, over engineered (in a bad way), and over politicized.
Some say this is the most politically-engineered weapons system in history (along with being the most expensive). Production for the parts of this aircraft is spread over 45 states. So you can only imagine how difficult it would be to scrap or reduce the size of this project.
The lesson from the Space Shuttle is that contractors lose big time from an unflyable abortion with a low operational tempo. If Morton Thiokol had sacrificed short-term profits to rethink the boosters, they'd have sold them 10 times faster and still be selling them today
The Air Force hijacked the Space Shuttle design process and forced it to deliver gigantic loads to polar orbit (a very tricky task). Then they abandoned the Shuttle, leaving it misdesigned for a nonexistent mission.
Morton Thiokol is a solid rocket company in the northwestern U.S. The Shuttle launched from the southwest U.S. The solid rockets, with tubes of propellant inside, had to be cut into sections to make the long journey. The joints were heavy (very bad for rockets), and a leaky joint destroyed the Challenger.
The horrible Air Force requirements made refurbishing very expensive and slow. Everything had been stripped of so much weight that it was too fragile and needed detailed inspection and repairs for each launch.
Morton Thiokol was crammed into the program to get Shuttle votes from their state's senators. The catch is that the crammin-crap-in process doomed the program. But if Morton Thiokol had threatened to cancel the program unless it was reinvented, they could have come out as a real rocket company supporting a fleet that launched every month. Honestly, the innovations SpaceX is coming out with could and should have been done by the Shuttle program, instead of the idiot business "leaders" chaining themselves to the worst possible ideas
I will never forget seeing a Lockheed Martin presentation about their bid for the X-33 contract. They had a slide titled "technical features", which looked roughly like this:
* Linear aerospike engine
* Conformal carbon-composite LH2 tanks
* Integrated metallic TPS
* Subcontractors in 38 states and 122 congressional districts
As far as I was concerned, the rest of the programme was a foregone conclusion from that point forward. Lockheed Martin of course won the bid -- the other bidders, with much simpler and more technically sound proposals that weren't driven by the need to split the project across as many districts as possible, of course lost -- and spent $1.3B without putting a single piece of hardware in the air.
The shuttle was a similarly foregone conclusion with a lot more money behind it. You're absolutely right that one of SpaceX's primary innovations, thus far, has simply been to not let politics get anywhere near the engineering process.
Indeed. The bitterest thing is that cheap access to space would have flowed forth a river of money to tech companies in every voting district. They could have sold uncancellable payloads like it was going out of style, and instead they crucified themselves on a single doomed political game.
I totally understand where you're coming from, typically with software projects canceling the whole thing is not the answer. However, military aircraft are a fundamentally different proposition.
If the military goes forward with the JSF, they are going to be stuck with a fundamentally poor platform for at least 20 to 30 years. This thing is wildly over budget, and will be wildy more expensive to operate than projected. This plane is supposed to be the backbone of US air power for decades.
If you scrap this program and start over, maybe it'll take 7-10 years to come up with 3 new aircraft for the 3 different roles. You sacrifice a decade to get something that will be far better for decades.
The military basically did exactly that when the F-111 program got off the rails in basically the same manner. It was supposed to be a fighter bomber for the Air Force and fleet air defense aircraft for the Navy but the Navy version ended up being too heavy and poor performing so they scrapped it, and ended up developing the F-14.
Honestly I would scrap the Marine Corps variant altogether. The USMC never goes where the Navy is not going to be so they could rely on close air support from the Naval variant. Sure a VSTOL aircraft could take off from a field somewhere but that capability simply doesn't provide enough benefit to offset the cost.
Isn't "going all in" how you get these things in the first place? They had to base everything on it. If they could get by with improved f15 and f16 types programs then they wouldn't get the JSF.
They are "pot committed" and can't kill the project. They made it kill proof...
> If you've got 50 percent or less of the software capabilities in a system with 8 million lines of code, and it's supposed to be deployed in 18 months you're in deep shit.
Well, at least it's not also tasked with managing health insurance accounts.
I look forward to Philip Greenspun's reaction. Each one of these planes costs more than the year-long healthcare.gov maintenance contract he freaked out about.
The software problems are bad enough. Then there's the hardware which is crippled by the marine corps's insistence on VSTOL which means that it will never be effective. The plane is all but an open joke within the air force.
The Chair Force decision makers aren't any better. The A10, one of their most successful programs ever, is/was also an open joke, it is the red-headed step-child of aircraft.
V/STOL capability was the one thing the marine corps absolutely insisted on, requiring the massive fan that makes the F-35's body so wide and high-drag. The decision to use a one-body-fits-all-applications approach means that even the non-V/STOL versions of the F-35 are fat pigs as far as military jets go. That's why these planes are so bloody slow by modern fighter standards and one reason why countries, such as Canada, that have no need for V/STOL variants of the plane are dropping out of the program.
In short, the marines screwed this plane up for everybody but themselves. If even they are not happy with the result then the F-35 project is an utter flop.
50 years ago plane looks more capable as VTOL than F-35. Modern powerful engines allow to drive the fans from the compressor shaft - as F-35 shows - and that would fix the main geometry issue with jet engine placement in XV-5A where fans were driven from jet exhaust.
We have created sufficient variety of failed VTOL plane concepts that there are certainly a number of types you've never seen. It is a recurrent trend - every engineering team thinks they have an answer which puts a novel spin on things and will fix them once and for all - and then it turns to shit.
View the 'wheel of shame', as it's known in aeronautical engineering circles:
several years ago spent a bunch of time going through the wheel, and in my view XV-5A and Bell X-22 were pretty good planes - there was nothing really wrong with them, and definitely not a "shame". Given the half-century improvements in hardware/electronics/aeronautical science i think if built today they would be much better than F-35 and V-22.
It was also a hard requirement for the British involvement in the JSF programme — with the UK meant to be contributing 10% of the original budgeted cost of development.
The USMC's requirements also kinda effed up the V-22 Osprey. Because of the space constraints for operating from amphibious carriers the rotors were undersized for the flight profile they wanted in the V-22, so they compensated by putting in more powerful engines. This caused heat problems that may have contributed to some of the fatal crashes the Osprey has had.
To a certain extent yes, but it was designed to be operated by the Air Force and Marines. Had they maybe split the variants up they could have avoided some of the design compromises that created some of the problems they had.
Still, the V-22 was a vastly more successful program than the JSF will be. At the very least the military got an aircraft that can perform missions no other aircraft can, thereby giving the services capabilities they've never had before. I don't think that will be the case for the JSF when it is all said and done.
edit: Made a mistake, Navy doesn't have V-22's yet
To be honest, the A-10 is probably the plane most of the services want, if they could make it carrier capable. It's a ridiculously impressive aircraft with probably one of the biggest kill lists in history. Along with the Spectres, designing an aircraft around a gun has almost never resulted in a disappointment.
You see lots of things like this decrying the F-35, but the real problems don't lie in features or designs, but in a very broken defense procurement process.
> V/STOL capability was the one thing the marine corps absolutely insisted on
A while ago I read a long and fascinating article tracing this back to Guadalcanal in WWII, when the Marines couldn't get the air support they needed. Apparently this history is firmly cemented in the corps collective memory and is a large part of why they insist on this.
A truly excellent article for everyone interested in fighter jets, military procurement, government contracts, national security, whether the US could win a war with China, or just really good yarns about really stupid decisions.
Yes, I think that is the one! Thanks for finding it. I was being annoyed at Facebook's lack of search, since a friend's wall was the only place I had ever seen it.
They should have just incremented on the F-16, now that is an exellent fighter - so elegant and simple. One of the F-16 designers has some great insights on the F-35: http://www.youtube.com/watch?v=UQB4W8C0rZI
This guy is advocating that all aircraft should be designed for short range dogfighting when this aircraft most likely designed for beyond visual range combat. Of course it's gonna lack the maneuverability. Try comparing it to the F22.
It's not at all clear that fighter aircraft are going to be doing much in any future conflict with e.g. China regardless of how quickly they can shoot down 30 year old technology.
Yes. By the time the F-35 works, it'll be obsolete for most of its missions because cruise missiles and drones will have utterly surpassed its capabilities.
I think modern software engineering really needs to start banking on provable languages with strong type systems. Instead of hoping and testing with Monte Carlo sims, let's prove an algorithm once as correct and move on from there.
Engineering is systematic approximations and best-guessing. The art of it is knowing when shortcuts are good enough.
Only very small parts of software can be proven, especially for things like flight control, where the parameter space is huge, and exhaustive search or manual formal proofs are impossible. Not even speaking of the fact that the spec against which one could build a proof is never error-free to begin with.
And what in your opinion is the difference between "systematic approximations and best-guessig" and "knowing when shortcuts are good enough"?
For me engineering is using proven and tested solutions every time, even if the solution is not sexy, and art is just "swinging it" - just like using js for everything when they are better more mature solutions.
Since you're not building the same thing every time, you don't have wholesale proven and tested solutions for everything. The engineering part is going about filling the gaps systematically, and figuring out what parts require closer inspection in the first place.
In my opinion, this last part is the art, figuring out what to do when you tread a new path. A good engineer will find the critical parts, but that relies on as much intuition as process. A really good engineer will find the meta-mistakes in the process.
The decision to "swing it" can be an engineering decision. If I am doing something I did before, and all the parts fit, I can swing it, and forego a tedious process. I have to balance the risk of implementation delays and bugs vs. spending time on simulation and paperwork.
It's also perfectly possible to do everything by the book and still fail, if the process is a bad fit, or you run out of time because you're all caught up in following procedure instead of taking common sense shortcuts.
So either swinging it or doing it by the book can be engineering or idiocy, depending entirely on the situation, and, in retrospect, the outcome.
More provable code is not the answer. Proving the functionality is meaningless if you can't decide on what the functionality should be. And this project has all the classic hallmarks of a requirements quagmire. The requirements are poorly specified and contradictory, and provable code and strong typing can't solve that problem.
Indeed, it makes the problem worse... the strong typing and provable code stubbornly insist that the requirements are contradictory and refuse to even compile. But managers don't want to hear that, of course, so we go back to the dumber languages that happily compile even so. This does not make the requirements any less contradictory of course... it just means that the dumber languages will let you blithely code on, and are guaranteed to do something stupid at run time instead. Progress!
A lot of things, like nonlinear control systems, can only be verified by Monte Carlo methods. Subsystem failure tolerance is difficult to handle by any existing theorem prover.
The JSF software organization uses static analyzers heavily. They are using theorem provers.
-DoD skimped on requirements and instead relied on the contractor to fill in the gaps. This led to the plane being designed without protection from lightning strikes which means it can't currently fly in bad weather
-The plane was supposed to have 70 percent parts commonality between variants. It is now 25 percent
-The plane couldn't do supersonic flight because Lockheed Martin didn't test the stealth coatings at the higher temperatures generated in supersonic flight
-The DoD let Lockheed Martin skip a lot of real world testing in favor of computer modeling
These analyses seem to overlook the impact of MADL and multi-aircraft sensor fusion. The fire on remote capability, meaning you have a fix on a target that you haven't even detected yourself, plays out huge in stuff I've seen, and it seems believable.
Unfortunately that's not an advantage unique to Blue side in this analysis. The Su's have a networked co-operative radar as well. There are a few public lectures you can watch from the designers if you like. Su in general favors continuous upgrades of existing inventory so customers don't have to wait for production of a new jet to field it either. They're quite confident in their electronics gear, even to the point of claiming purely passive detection of the F-22 at 50km.
Comparing feature bullet points is a pointless exercise. Neither of us being an actual expert on this stuff (I assume), it's most reasonable to take the pros at their word. Doubly so when they're being critical rather than cheer-leading.
When bloated pork barrel projects claim center stage with a cacaphony of acrimony, I tend to just assume that it's a facade of staged red tape, designed to act as interference and cover for wherever the real money is being spent.
You know, like when massive data centers in Utah are being built for... well... no reason at all!
I took a tour of one of the factories where the F-35 is built (or one of the components of the airframe). It is the one located in Atlanta, Georgia. The scale of the place is just mind-numbing. There was a row of C-130 in different stages of assembly and next to them the production line for the their wings.
On the F-35 side, there was this huge robot that looked like a flat toolbox. It moved the airframe part from one station to the next. Just amazing. The whole assembly line looks like something out of a movie.
I was told that the F-35 had a lot of C++ code, and that is the reason it was not meeting deadlines. It was an interesting bit of gossip to hear, but I'm not entirely sure how that would come into play. I'm also not sure if C++ has been used before in this type of application. Given how ingrained Ada, C, and Fortran are in the industry.
Ada is dead for new development by fiat of the DoD. They finally figured out that having all their real-time code written in a language almost nobody outside of DoD uses is a bad idea.
Fortran doesn't fly; that is, (almost)[1] no real-time code is written in Fortran. It is still used for simulations, but is steadily being replaced my Matlab since the latter integrates better with various engineering workflows.
C is definitely still the winner for the embedded code, but is losing to C++ as time goes on. Java is also making inroads for implementing the interfaces on operator consoles.
[1] I know of at least one program that deployed signal processing algorithms in a semi-operational state in Fortran.
72 comments
[ 2.8 ms ] story [ 133 ms ] thread"The current software generated too many nuisance warnings and resulted in poor sensor performance. Further work on software had been slowed by testing required to validate earlier fixes, the report said."
"It said Lockheed had delivered F-35 jets with 50 percent or less of the software capabilities required by its production contracts with the Pentagon."
> But how much work the software does is not what makes it remarkable. What makes it remarkable is how well the software works. This software never crashes. It never needs to be re-booted. This software is bug-free. It is perfect, as perfect as human beings have achieved. Consider these stats : the last three versions of the program -- each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors.
[1] http://www.fastcompany.com/28121/they-write-right-stuff
If that 10 billion is accurate (I can believe 10GB of binaries, but 10 billion lines seems too much to me, even including support software), maybe the project is too ambitious.
For example, the onboarding meeting I had with their HR folks was roughly 90% about retirement benefits and health care plans, neither of which I was eligible for as a temporary part-timer. This still cost NIH two hours of my time even so. And it took them weeks to get me the right codes so I could actually log my hours in their system the way I was supposed to, on a project that was supposed to last two months in total.
And in all of this, they provided not one shred of added value to the project that I could detect. Were it not for whatever bureaucratic rule required them as a middleman, I could have conceivably raised my rates 50%, cost the actual customer less, and been more productive without them.
If you've got 50 percent or less of the software capabilities in a system with 8 million lines of code, and it's supposed to be deployed in 18 months you're in deep shit. Yet, the program's backers blithely proclaim they're going to make their dates, ignoring the fact that the program is years overdue and 70% over budget.
Not only has an enormous amount of good money been thrown after bad, the military has staked the future of air power on this one aircraft so no one wants to admit that it should be cancelled.
Obviously Navy, Marines, and Airforce all want different features, and requirement to the chassis renders it extremely frail (they had to cut a lot of weight for VTOL capability).
So considering that, it actually might be good idea to stop completely, fire all top airforce/navy/marine brass involved and restart as 3 separate projects from scratch, albeit using all the technology already developed.
Morale is already super low and it is hard to see bright future for something so overpriced, over budget, over engineered (in a bad way), and over politicized.
Morton Thiokol is a solid rocket company in the northwestern U.S. The Shuttle launched from the southwest U.S. The solid rockets, with tubes of propellant inside, had to be cut into sections to make the long journey. The joints were heavy (very bad for rockets), and a leaky joint destroyed the Challenger.
The horrible Air Force requirements made refurbishing very expensive and slow. Everything had been stripped of so much weight that it was too fragile and needed detailed inspection and repairs for each launch.
Morton Thiokol was crammed into the program to get Shuttle votes from their state's senators. The catch is that the crammin-crap-in process doomed the program. But if Morton Thiokol had threatened to cancel the program unless it was reinvented, they could have come out as a real rocket company supporting a fleet that launched every month. Honestly, the innovations SpaceX is coming out with could and should have been done by the Shuttle program, instead of the idiot business "leaders" chaining themselves to the worst possible ideas
The shuttle was a similarly foregone conclusion with a lot more money behind it. You're absolutely right that one of SpaceX's primary innovations, thus far, has simply been to not let politics get anywhere near the engineering process.
The SLS, built on Shuttle parts minus the spaceplane, launches first prototype in 2017. To what end, few can say.
If the military goes forward with the JSF, they are going to be stuck with a fundamentally poor platform for at least 20 to 30 years. This thing is wildly over budget, and will be wildy more expensive to operate than projected. This plane is supposed to be the backbone of US air power for decades.
If you scrap this program and start over, maybe it'll take 7-10 years to come up with 3 new aircraft for the 3 different roles. You sacrifice a decade to get something that will be far better for decades.
The military basically did exactly that when the F-111 program got off the rails in basically the same manner. It was supposed to be a fighter bomber for the Air Force and fleet air defense aircraft for the Navy but the Navy version ended up being too heavy and poor performing so they scrapped it, and ended up developing the F-14.
Honestly I would scrap the Marine Corps variant altogether. The USMC never goes where the Navy is not going to be so they could rely on close air support from the Naval variant. Sure a VSTOL aircraft could take off from a field somewhere but that capability simply doesn't provide enough benefit to offset the cost.
They are "pot committed" and can't kill the project. They made it kill proof...
Well, at least it's not also tasked with managing health insurance accounts.
In short, the marines screwed this plane up for everybody but themselves. If even they are not happy with the result then the F-35 project is an utter flop.
http://www.youtube.com/watch?v=1XkJXSoTTb4
50 years ago plane looks more capable as VTOL than F-35. Modern powerful engines allow to drive the fans from the compressor shaft - as F-35 shows - and that would fix the main geometry issue with jet engine placement in XV-5A where fans were driven from jet exhaust.
View the 'wheel of shame', as it's known in aeronautical engineering circles:
http://vstol.org/
Still, the V-22 was a vastly more successful program than the JSF will be. At the very least the military got an aircraft that can perform missions no other aircraft can, thereby giving the services capabilities they've never had before. I don't think that will be the case for the JSF when it is all said and done.
edit: Made a mistake, Navy doesn't have V-22's yet
Edited to fix a word.
A while ago I read a long and fascinating article tracing this back to Guadalcanal in WWII, when the Marines couldn't get the air support they needed. Apparently this history is firmly cemented in the corps collective memory and is a large part of why they insist on this.
A truly excellent article for everyone interested in fighter jets, military procurement, government contracts, national security, whether the US could win a war with China, or just really good yarns about really stupid decisions.
(I closed my Facebook account a few years ago, so all I know about this is what you said.)
Only very small parts of software can be proven, especially for things like flight control, where the parameter space is huge, and exhaustive search or manual formal proofs are impossible. Not even speaking of the fact that the spec against which one could build a proof is never error-free to begin with.
In my opinion, this last part is the art, figuring out what to do when you tread a new path. A good engineer will find the critical parts, but that relies on as much intuition as process. A really good engineer will find the meta-mistakes in the process.
The decision to "swing it" can be an engineering decision. If I am doing something I did before, and all the parts fit, I can swing it, and forego a tedious process. I have to balance the risk of implementation delays and bugs vs. spending time on simulation and paperwork.
It's also perfectly possible to do everything by the book and still fail, if the process is a bad fit, or you run out of time because you're all caught up in following procedure instead of taking common sense shortcuts.
So either swinging it or doing it by the book can be engineering or idiocy, depending entirely on the situation, and, in retrospect, the outcome.
The JSF software organization uses static analyzers heavily. They are using theorem provers.
http://www.vanityfair.com/politics/2013/09/joint-strike-figh...
Some of the highlights include:
-DoD skimped on requirements and instead relied on the contractor to fill in the gaps. This led to the plane being designed without protection from lightning strikes which means it can't currently fly in bad weather
-The plane was supposed to have 70 percent parts commonality between variants. It is now 25 percent
-The plane couldn't do supersonic flight because Lockheed Martin didn't test the stealth coatings at the higher temperatures generated in supersonic flight
-The DoD let Lockheed Martin skip a lot of real world testing in favor of computer modeling
https://www.youtube.com/watch?v=aXQ2lO3ieBA
Comparing feature bullet points is a pointless exercise. Neither of us being an actual expert on this stuff (I assume), it's most reasonable to take the pros at their word. Doubly so when they're being critical rather than cheer-leading.
You know, like when massive data centers in Utah are being built for... well... no reason at all!
On the F-35 side, there was this huge robot that looked like a flat toolbox. It moved the airframe part from one station to the next. Just amazing. The whole assembly line looks like something out of a movie.
I was told that the F-35 had a lot of C++ code, and that is the reason it was not meeting deadlines. It was an interesting bit of gossip to hear, but I'm not entirely sure how that would come into play. I'm also not sure if C++ has been used before in this type of application. Given how ingrained Ada, C, and Fortran are in the industry.
Fortran doesn't fly; that is, (almost)[1] no real-time code is written in Fortran. It is still used for simulations, but is steadily being replaced my Matlab since the latter integrates better with various engineering workflows.
C is definitely still the winner for the embedded code, but is losing to C++ as time goes on. Java is also making inroads for implementing the interfaces on operator consoles.
[1] I know of at least one program that deployed signal processing algorithms in a semi-operational state in Fortran.