It makes sense that a regular pilot flying actual passengers should be familiar with all systems.
But this article is specifically about test pilots, and keeping a tester in the dark about specific implementation details of the system they are testing is a perfectly normal and good way to preserve the integrity of the testing, especially for a system that's meant to be transparent.
Yeah, keeping testers in the dark works when you're trying to observe if they can navigate a menubar, not when they're piloting a passenger airliner. Test pilots are often engineers themselves, and (at least in the early days) many test pilots worked on designing the very systems they were going airborne to test.
Test pilots should have been informed about MCAS. It's another variable to take into consideration and another system that needs to be tested. How can anyone be sure it does what it's supposed to do when nobody tried activating it? Imagine how many lives could have been saved had someone had the bright idea to put it on the checklist.
At a guess you are not a test pilot. Test pilots are - or at least are supposed to be - briefed on all of the details of the systems they are testing, including what should happen when they take the plane outside of normal operating parameters. There is no such thing as 'destructive aircraft testing', that's called a crash and is considered an anomaly.
It always amazes me how software and IT people in general assume that other industries are just as crappy in their processes.
That said, I do actually believe that this speaks in favor of Boeing to the extent that that is still possible after all that came out so far: the fact that they did not alert their own test pilots proves to a certain degree that they did not believe the change was a major one or worth mentioning. It would have been a worse signal if they had alerted their test pilots to the change but then failed to notify their customers, this is at least consistent with their narrative that they did not see the change as a major one.
> ... proves to a certain degree that they did not believe the change was a major one or worth mentioning
Yes, what Boeing is saying is that MCAS, being designed to force the new plane follow the flight envelope of the old plane
1/ wasn't an anti-stall system
2/ and therefore wasn't a "security" feature (and therefore, it was completely ok to sell upgrades to that system as expensive options)
During a recent interview, Boeing CEO repeatedly said that no specific training was necessary on MCAS because MCAS is part of the whole plane, and when you're flying the MAX you're using MCAS.
This is indeed the problem: they couldn't even consider MCAS failing -- even as a thought experiment, because if MCAS fails, the MAX isn't a 737 anymore!
It looks like they locked themselves up into an alternative reality that they couldn't escape, until planes started crashing into the ground.
> There is no such thing as 'destructive aircraft testing'
Actually, there is: https://www.youtube.com/watch?v=Ai2HmvAXcU0 "This Boeing 777 wing was tested to destruction, finally breaking at one fifty four percent of the designed limit load."
You're applying software testing logic to aircrafts! Think about it a bit. Why does hiding implementation details work with software testing? How is software tested? By trying things and catching/looking for breakage. Breaking things is cheap with software.... And when it is expensive (in terms of time/effort/resources /...) the normal approach is to mock out stuff. Would that also be acceptable for aircraft tests? Should test pilots only test simulations of the cockpit?
There's a whole world out there beyond software engineering and what works well for software does not work well elsewhere.
Excellent explicit acknowledgment that software — bits — are not the same as the real (aircraft in flight, e.g.) world — atoms. As a physician, I am routinely amused by comments on HN implying that the human body is equivalent to software, and that medical procedures are the same as steps in execution of a program on a computer. Which would be true, except for one thing: every meat-based computer — aka body — is uniquely wired, with unknown connection patterns causing identical inputs to produce wildly varying outputs. As yet, there's no app for that.
Pack the engineering elitism up, please. It doesn't help anyone at all, and just leads to flamewar.
I"m a professional tester of things, and a metrology nerd to boot. Yes. It is self bloody evident that different measurements (read: tests) require different methodologies, optimizations, set-ups, and execution procedures.
It doesn't change the fact that you still must do envelope testing to ensure your approach to failure is what your analysis says it is.
Do you take it all the way to destruction? No. However, you take it into every conceivable recoverable error state so you know not just the positive behavior of the system, but the negative ones as well.
With flying in particular, this means you have to be transparent with your test pilots. Wanna know why? They don't make test plans. They fly them. You, as the tester trying to test something should treat them as your test executor.
You are the tester. The asker of questions. Obtainer of data, verifier of assertions, and the one mofo there with the mandate (whether anyone else likes it or not) to hold up the entire show until you get your result, or they fire you, whichever comes first. The firing isn't even a big deal when you know beyond a shade of a doubt the question must be asked. That's the kind of outcome that leads to living forever.
Pilot's aren't software people! They're operators! They expect some things can fail, and that you're giving them the knowledge ahead of time to know what to do to save the aircraft.
Information doesn't magically propagate. It takes effort. It takes someone getting out of their chair and having a chat, sending an email, something.
Software testers definitely have it easy with all those fangled debuggers, and not having to gas up a plane and file flight plans and what not, but that means absolutely jack and squat here where software and hardware integrations were not tested to sufficient rigor.
But we're all just looking for the truth. There was truth that needed to be brought into the light, but wasn't. That's our problem here. We need to get down to the bottom of it, and I don't care what anyone's field of study is if they can shed some substantive light on how to keep something like this from happening again.
> keeping a tester in the dark about specific implementation details of the system they are testing is a perfectly normal and good way to preserve the integrity of the testing
Except in this case a failed test does not just result in a failed test, but instead also results in the death of the tester.
Because making the new behave like the old, while the new is not the same as the old, is itself a thing to be tested. Running the happy path is not sufficient testing.
Also, the test pilots literally have skin in the game.
If the point of an airplane is to fly, why have test pilots?
It looks like there is something very rotten recently about the culture of Boeing. I wonder if any of it is due to the HQ been moved away from the engineering base in Seattle to Chicago?
If people working in your quality managment quit because they have issues squaring current behaviour with their conscience, you have a real cultural problem.
This makes no sense, we know that they increase the MCAS action range from 0.6° to 2.5° and they didn't inform FAA of the change.
So, they must have told tests pilots about the feature, otherwise why did they change the parameters ?
It may be that they kept this information in a small circle. MCAS was supposed to keep the plane flying similar so discussions of where this idealized model was breaking were kept quiet.
It is still incredible. There was one big change in a high volume plane and that one big change should have been tested and tested again. Somehow Boeing started to believe their own cool aid.
The more I think about the MAX the scarier it gets. What if one engine fails during start? The forward mounted other engine goes full throttle pulling up more than pilots are trained to handle.
You mean that the engineers quietly analyzed the test flight data and decided to increase the parameters on their own, so that the flight data show similarity between a MAX and a NG ?
I find it so huge that it is hard to believe. Unless they took SW guys with no clue about how to do their jobs, like fresh graduate with no mentoring, or even outsourced to a SW shop ?
I would never take responsability of such a change without even talking to the pilots about it.
It is hundred timed worse than the VW engineers who cheated the homologation test.
"Never attribute to malice what can be explained by incompetence".
In VW's case, it's clear as day that it was malice and greed. In Boeings case, I was willing to write it down as incompetence at first, but as time goes by, I'm becoming more inclined to put it on the malice bin.
It all goes back to the root cause: an impossible goal was thought to be so important to the company that reality had to bend to accommodate it.
The impossible requirement was that the plane had to behave in such a way that pilots wouldn't need to know the correction system existed. If you work under that pressure for long enough, and enough of the engineers who realize it was BS move on, retire, etc, then as you approach the deadline, you rationalize and justify more and more. You've already embraced the idea that commercial pilots won't need to know about the existence this system that corrects the attitude of the plane, you've already rationalized leaving it out of the documentation and training and pretending it's not part of the control system. So why would test pilots need to know the parameters either? At the point they were at, this was a tiny leap to make.
Can we stop now blaming and smearing the pilots, who propably tried just about everything to try to control a plane that was completely out of control?
This, to me, is the most galling part in Boeing's current pr blitz.
35 comments
[ 3.2 ms ] story [ 80.5 ms ] threadBut this article is specifically about test pilots, and keeping a tester in the dark about specific implementation details of the system they are testing is a perfectly normal and good way to preserve the integrity of the testing, especially for a system that's meant to be transparent.
Test pilots should have been informed about MCAS. It's another variable to take into consideration and another system that needs to be tested. How can anyone be sure it does what it's supposed to do when nobody tried activating it? Imagine how many lives could have been saved had someone had the bright idea to put it on the checklist.
It always amazes me how software and IT people in general assume that other industries are just as crappy in their processes.
That said, I do actually believe that this speaks in favor of Boeing to the extent that that is still possible after all that came out so far: the fact that they did not alert their own test pilots proves to a certain degree that they did not believe the change was a major one or worth mentioning. It would have been a worse signal if they had alerted their test pilots to the change but then failed to notify their customers, this is at least consistent with their narrative that they did not see the change as a major one.
Yes, what Boeing is saying is that MCAS, being designed to force the new plane follow the flight envelope of the old plane
1/ wasn't an anti-stall system
2/ and therefore wasn't a "security" feature (and therefore, it was completely ok to sell upgrades to that system as expensive options)
During a recent interview, Boeing CEO repeatedly said that no specific training was necessary on MCAS because MCAS is part of the whole plane, and when you're flying the MAX you're using MCAS.
This is indeed the problem: they couldn't even consider MCAS failing -- even as a thought experiment, because if MCAS fails, the MAX isn't a 737 anymore!
It looks like they locked themselves up into an alternative reality that they couldn't escape, until planes started crashing into the ground.
A bit tongue in cheek, but it's been shown that, when MCAS fails, the 737 MAX can't really be called a plane.
Actually, there is: https://www.youtube.com/watch?v=Ai2HmvAXcU0 "This Boeing 777 wing was tested to destruction, finally breaking at one fifty four percent of the designed limit load."
There's a whole world out there beyond software engineering and what works well for software does not work well elsewhere.
I"m a professional tester of things, and a metrology nerd to boot. Yes. It is self bloody evident that different measurements (read: tests) require different methodologies, optimizations, set-ups, and execution procedures.
It doesn't change the fact that you still must do envelope testing to ensure your approach to failure is what your analysis says it is.
Do you take it all the way to destruction? No. However, you take it into every conceivable recoverable error state so you know not just the positive behavior of the system, but the negative ones as well.
With flying in particular, this means you have to be transparent with your test pilots. Wanna know why? They don't make test plans. They fly them. You, as the tester trying to test something should treat them as your test executor.
You are the tester. The asker of questions. Obtainer of data, verifier of assertions, and the one mofo there with the mandate (whether anyone else likes it or not) to hold up the entire show until you get your result, or they fire you, whichever comes first. The firing isn't even a big deal when you know beyond a shade of a doubt the question must be asked. That's the kind of outcome that leads to living forever.
Pilot's aren't software people! They're operators! They expect some things can fail, and that you're giving them the knowledge ahead of time to know what to do to save the aircraft.
Information doesn't magically propagate. It takes effort. It takes someone getting out of their chair and having a chat, sending an email, something.
Software testers definitely have it easy with all those fangled debuggers, and not having to gas up a plane and file flight plans and what not, but that means absolutely jack and squat here where software and hardware integrations were not tested to sufficient rigor.
But we're all just looking for the truth. There was truth that needed to be brought into the light, but wasn't. That's our problem here. We need to get down to the bottom of it, and I don't care what anyone's field of study is if they can shed some substantive light on how to keep something like this from happening again.
Except in this case a failed test does not just result in a failed test, but instead also results in the death of the tester.
Also, the test pilots literally have skin in the game.
If the point of an airplane is to fly, why have test pilots?
If people working in your quality managment quit because they have issues squaring current behaviour with their conscience, you have a real cultural problem.
It is still incredible. There was one big change in a high volume plane and that one big change should have been tested and tested again. Somehow Boeing started to believe their own cool aid.
The more I think about the MAX the scarier it gets. What if one engine fails during start? The forward mounted other engine goes full throttle pulling up more than pilots are trained to handle.
I find it so huge that it is hard to believe. Unless they took SW guys with no clue about how to do their jobs, like fresh graduate with no mentoring, or even outsourced to a SW shop ? I would never take responsability of such a change without even talking to the pilots about it.
It is hundred timed worse than the VW engineers who cheated the homologation test.
In VW's case, it's clear as day that it was malice and greed. In Boeings case, I was willing to write it down as incompetence at first, but as time goes by, I'm becoming more inclined to put it on the malice bin.
The danger of air pollution and global warming is that it's so hard to quantify the effects, but we already know that it's huge.
The pitchup is caused by aerodynamic forces on the engine at high angle of attack.
The impossible requirement was that the plane had to behave in such a way that pilots wouldn't need to know the correction system existed. If you work under that pressure for long enough, and enough of the engineers who realize it was BS move on, retire, etc, then as you approach the deadline, you rationalize and justify more and more. You've already embraced the idea that commercial pilots won't need to know about the existence this system that corrects the attitude of the plane, you've already rationalized leaving it out of the documentation and training and pretending it's not part of the control system. So why would test pilots need to know the parameters either? At the point they were at, this was a tiny leap to make.
This, to me, is the most galling part in Boeing's current pr blitz.