What are the numbers adjusted for inflation since then and how do they breakdown counting stockbuyback transactions, management and above corporate headquarter placeholder renumeration packages, engineers and below, and does the software perform fleetwide upgrade as Tesla does over the air?
I would hazard a guess that principal professional software engineer would make around $350-600k USD. Let's say you need $10 million USD / year to run a fundamental embedded systems team and another $20 for specifics applications on a project. Call it $30 million. Suppose the economic damage of a $2.6 B fine, $ 1 B loss in intangible goodwill, $2 B in lost sales on top of the pandemic, and killing 346 people total say $8 B. That would fund about 5k top industry developers for about 5 years, or 26k person-years of effort. Or, 244k person-years of $9 USD/hr effort. Good luck managing the code quality and project management of such a large project where there is little investment or concern for who's writing code.
TL;DR: Don't competitively incentivize short-term cost savings by strip-mining fundamental capabilities to deliver by throwing human capital to the wind as an interchangeable "fast-food worker" approach.
I used to live outside of Silicon Valley, can confirm that most companies don't pay that much. However, what you didn't say is far more interesting: that it's okay that they don't. That part is the real travesty, that someone would look at a salary and say, "They don't deserve that" rather than, "I deserve that too." The fact is that software engineering isn't about writing code. That bit is quite trivial in any engineers day. What's hard, dare I say what you're paid to do, is to solve problems. Process problems, efficiency problems, resiliency problems, security problems - you're made to answer questions, or develop systems that answer questions to some of the hardest problems corporations have - and you'd ask for anything less?
Boeing most certainly can afford to pay the price especially when human lives are at stake.
In the real world, programmers and engineers are only a relatively small cohort of the people involved in making something that can be sold. There are 10x more people handling procurement, manufacturing, testing, and a host of other activities. They can't all make 350k and it doesn't stand to reason that the design side should get a disproportionate share.
And lost tens of billions in aircraft groundings, delayed orders and reputation. The only reason the 737Max is still selling is that Airbus can't make A320s and A220s fast enough to satisfy everyone, with a bit more competition Boeing would have been in a really bad position.
Flight software is software for physical objects and is therefore bound by the laws of reality. Recently salaries have been jumping up significantly but they're still way below the low end of that range.
While I may live in a relative outlier (Norway, with its petro-boosted economy), that sounds like an exaggeration. (Deep sea business, fight for much the same talent as aerospace, senior dev, $95k base and some $35k on top of that in various compensation and bonuses. While well paid, I am probably not in the top quartile nation-wide for my education level and experience)
But yes, the OP is very much right in that companies building actual, physical products don't have the kind of money to burn on engineers that a SV unicorn does...
That income is plenty for me not to have to worry about a thing financially, though. Housing is cheap, transport cheap-ish.
Maybe in Toulouse (Airbus HQ) which is a fairly low cost of living city compared to SV or Seattle, or Paris for that matter, and would allow for a fairly comfortable life.
shouldn’t matter who does it or for how much, as long as the sw lifecycle process is followed (which would include supplier oversight). Actually writing code, btw, is the cheap/easy part- designing correct system and sw requirements, along with verification, is much more expensive.
I don't know, there's a certain point where the cost itself is a bad sign and software engineering for $9/hour is definitely below that point. If you found out the pilot of your flight was making $9 an hour, wouldn't you be concerned?
What makes you think pilots, depending on region, are making aignificabtly more than that? Heck, some are even paying to get the necessary flight hours for their type ratings...
But it wasn't a programming error, right? So why would it matter whether the code was written by an unpaid intern, a $9/h engineer or their senior software engineer? The code seemingly did exactly what it was supposed to.
My subjective opinion is that if you hire experts, not just programmers but experts at aviation programming, then possibly someone would have noticed a gap in the logic of the requirements and systems engineering. Who knows for sure? But I know plenty of contractors who write good code but oftentimes it's the architect level expert that knows what's beyond the code.
Especially when the code is directly responsible for a physical system.
MCAS was implemented to cover up the flight characteristic changes which happened when big fans were mounted further forward and higher (else they would scrape on runway). MCAS attempted to make it appear to the pilot that the plane performed like other 737s.
In my opinion the 737MAX needs a distinct type certification. This would dramatically raise costs, so it is covered up with software. FAA and regulators in other countries don't care. This management and regulator problem is enabled by all off the passengers who pay for seats on this plane.
I've said it before and I'll continue to say it despite the downvotes: Ask what plane a flight is on, and refuse the flight if it is moved to a 737 Max.
I don't care if Boeing decided "this revision" of the 737 Max is safe. I insist on only flying in airplanes that the pilots were trained to fly on, without a compatibility layer between the pilots' training and the airframe characteristics.
just an interesting tidbit: all 737 pilots were trained how to handle a runaway stabilizer.
The crime of the 737Max was that the MCAS fucking shit into the wind and introduced an unbelievably dumb stochastic failure mode (every X seconds MCAS adjusted the stabilizer), and the pilots had no idea what's doing it.
> and refuse the flight if it is moved to a 737 Max
I don't fly often, but I go further. I just don't fly Boeing unless it's a 747.
747 || Airbus || Embraer
My airline of choice is American. I can see what a particular route is flying prior to purchasing tickets. There are enough Airbus aircraft in their fleet for me to pull this off.
As someone who works in embedded systems, the least important part of my job is writing the code.
The most important is making sure the system works as intended. The code is a way to accomplish that. There are other ways to accomplish that, which frequently will include things like changes to the circuit, changes to the mechanical design, changes to specifications, operating requirements, silicon, or how the user interacts with the system in order to make sure it's working correctly.
That the code was incorrectly specced was the bug, and there it matters whether it was written by an unpaid intern, a $9 engineer, or a senior engineer who has enough system knowledge to know what the ramifications of the system interacting this way are, or allowing marketing to make it an optional fault light, or permitting only one sensor to be used as its input.
While the ultimate failure is a management one, one of the ways that management failed is they clearly didn't get the proper system-level architecture squared away.
All code does what it was supposed to, barring any bugs in the underlying system you're using.
Now, if you're reading a sensor and blindly trusting what it says, that can be a problem. And there is where it becomes important that someone with experience writes this. They understand what it means, they know the things that can go wrong.
Sending a story to India that says 'implement autopilot AOA sensor change', then that is exactly what the devs there will do.
Don't forget that the culture in India is different than here. There's no pushback. There's no asking questions. You need to be extremely explicit, and ask and ask and ask to make sure they feel comfortable asking the questions they have to you, someone they see as the one in charge.
The $9/h is obviously click baity, by itself it doesn't mean the software was malfunctioning.
But it highlights a general policy of outsourcing to the lowest bidder. This is generally a symptom of badly managed company placing spreadsheet profits above everything and seeing product development and Engineering as a cost center, which given the current Boeing situation, is quite telling.
This is a shoddy article, and conflates unrelated issues. First, it wasn’t a software issue at all that grounded the plane. Second, Boeing has categorically said that MCAS
Click-bait title/article to "whitewash" Boeing's responsibility in the disaster;
Excerpt from the article:
Boeing said the company did not rely on engineers from HCL and Cyient for the Maneuvering Characteristics Augmentation System, which has been linked to the Lion Air crash last October and the Ethiopian Airlines disaster in March. The Chicago-based planemaker also said it didn’t rely on either firm for another software issue disclosed after the crashes: a cockpit warning light that wasn’t working for most buyers.
So it was probably non-critical, routine drudgery software which was outsourced.
Where are the developers of the MCAS though? IMO they, alongside any PMs, engineers etc. that made the decisions, executed them, okayed this piece of software, and then lied about it in front of the regulator should have criminal liability.
As a reminder, we're talking about a piece of software that could control the plane's angle of attack, pointing the nose downwards (with the obvious risk that this poses), and which relies on a single physical sensor, a pitot tube, which is known for relatively frequent (for aviation) failures due to ice and other blockages, which is why there are two of them.
If the people who made those choices didn't understand the ramifications of their actions they have no place in aviation. At the very least they should be fired, but ideally they should be brought to justice. Alongside every Boeing pilot, engineer, manager, exec who knew about this, and lied about it.
What i was pointing out is that a lot of articles like this one have been planted in the media subtly hinting and throwing shade on these external companies (who are innocent) thus deflecting from Boeing's responsibility. This needs to be called out every time.
Boeing omitted MCAS from the FAA certification that they did themselves, even though it's:
a) critical - if it malfunctions, the plane can crash
b) based on something that is known to fail often, with zero redundancy
Both of those they were mandated to disclose, and they didn't, knowingly. Why is none of these bastards in jail?
> Wasn't the sensor an AoA vane instead of a pitot tube?
Yeah sorry, mistook the sensor, indeed it's the Angle of Attack sensor next to the pitot tubes. Both are prone to frequent, for aviation, failure, so the point still remains.
I mean I get the NTSB no blame approach in progressing safety practices, but there needs to exist an authority that enforces these practices.
And the worst part is that this gives a bad rep to "IT" as a whole in the eyes of laypeople, even though IT's internal view aviation is that it's full of safety standards, redundancy, reliability and maintenance is paramount, etc.
It turns out the standards were not followed, and starry eyed junior programmers' view of aviation is just smoke and mirrors.
The one thing I dislike about these articles is that it seems to paint India as a place where skilled Engineers are not from, when in reality there are a lot of talented people in India and from India. What these stories should be saying is "inadequately trained or skilled staff outsourced to for $9-an-hour". There are plenty of good engineers in India, they just cost the same amount as good engineers in other places.
"when in reality there are a lot of talented people in India and from India."
There are skilled people from India, but rarely in India I'd assume. The market for software engineers in India is not great to put it politely so the talented people will obviously go where the money and benefits are which is US, Europe etc. And yes I'm sure there are exceptions, but probably not too many or do you know anyone actually buying software engineer services from India and paying western salaries?
I’ve said it before and will say it again: as long as there are no painful legal consequences, including jail time, for the responsible managers and C-suite, such behavior will not stop.
Boeing, like ogres, has layers, except the layers are decades of bad decisions and unforced failure.
Pop quiz: name a new program that Boeing has started in the last twenty years that was successful. Acquisitions, buyouts, remnant MD programs don't count. Now, during the post-COVID era, they're somehow spending all of their[1] cash in precisely those sectors that are forecasted to never recover - ever. Congratulations. That takes work. If you want to see this poetry in motion, take a look at what happens to any company bought by Boeing. If you could weaponize this kind of value destruction we'd figure out a way to put it in a JDAM and drop it on Hainan Island.
The MAX software issues are at the very tippity tip tail end of a single one of these chains of idiocy, so I feel uncomfortable just piling all the sins on MCAS. The fact they even had MCAS should have been a gigantic red flag, and would have been in anything like a functional regulatory regime. The FAA PNW offices were unofficial Boeing locations even back in 2008.
44 comments
[ 3.9 ms ] story [ 101 ms ] threadTL;DR: Don't competitively incentivize short-term cost savings by strip-mining fundamental capabilities to deliver by throwing human capital to the wind as an interchangeable "fast-food worker" approach.
Boeing most certainly can afford to pay the price especially when human lives are at stake.
For reference, I'm a Sr aero dev pulling 150k.
But yes, the OP is very much right in that companies building actual, physical products don't have the kind of money to burn on engineers that a SV unicorn does...
That income is plenty for me not to have to worry about a thing financially, though. Housing is cheap, transport cheap-ish.
Especially when the code is directly responsible for a physical system.
In my opinion the 737MAX needs a distinct type certification. This would dramatically raise costs, so it is covered up with software. FAA and regulators in other countries don't care. This management and regulator problem is enabled by all off the passengers who pay for seats on this plane.
I don't care if Boeing decided "this revision" of the 737 Max is safe. I insist on only flying in airplanes that the pilots were trained to fly on, without a compatibility layer between the pilots' training and the airframe characteristics.
The crime of the 737Max was that the MCAS fucking shit into the wind and introduced an unbelievably dumb stochastic failure mode (every X seconds MCAS adjusted the stabilizer), and the pilots had no idea what's doing it.
I don't fly often, but I go further. I just don't fly Boeing unless it's a 747.
747 || Airbus || Embraer
My airline of choice is American. I can see what a particular route is flying prior to purchasing tickets. There are enough Airbus aircraft in their fleet for me to pull this off.
The most important is making sure the system works as intended. The code is a way to accomplish that. There are other ways to accomplish that, which frequently will include things like changes to the circuit, changes to the mechanical design, changes to specifications, operating requirements, silicon, or how the user interacts with the system in order to make sure it's working correctly.
That the code was incorrectly specced was the bug, and there it matters whether it was written by an unpaid intern, a $9 engineer, or a senior engineer who has enough system knowledge to know what the ramifications of the system interacting this way are, or allowing marketing to make it an optional fault light, or permitting only one sensor to be used as its input.
While the ultimate failure is a management one, one of the ways that management failed is they clearly didn't get the proper system-level architecture squared away.
Now, if you're reading a sensor and blindly trusting what it says, that can be a problem. And there is where it becomes important that someone with experience writes this. They understand what it means, they know the things that can go wrong.
Sending a story to India that says 'implement autopilot AOA sensor change', then that is exactly what the devs there will do.
Don't forget that the culture in India is different than here. There's no pushback. There's no asking questions. You need to be extremely explicit, and ask and ask and ask to make sure they feel comfortable asking the questions they have to you, someone they see as the one in charge.
But it highlights a general policy of outsourcing to the lowest bidder. This is generally a symptom of badly managed company placing spreadsheet profits above everything and seeing product development and Engineering as a cost center, which given the current Boeing situation, is quite telling.
Excerpt from the article:
Boeing said the company did not rely on engineers from HCL and Cyient for the Maneuvering Characteristics Augmentation System, which has been linked to the Lion Air crash last October and the Ethiopian Airlines disaster in March. The Chicago-based planemaker also said it didn’t rely on either firm for another software issue disclosed after the crashes: a cockpit warning light that wasn’t working for most buyers.
So it was probably non-critical, routine drudgery software which was outsourced.
As a reminder, we're talking about a piece of software that could control the plane's angle of attack, pointing the nose downwards (with the obvious risk that this poses), and which relies on a single physical sensor, a pitot tube, which is known for relatively frequent (for aviation) failures due to ice and other blockages, which is why there are two of them.
If the people who made those choices didn't understand the ramifications of their actions they have no place in aviation. At the very least they should be fired, but ideally they should be brought to justice. Alongside every Boeing pilot, engineer, manager, exec who knew about this, and lied about it.
What i was pointing out is that a lot of articles like this one have been planted in the media subtly hinting and throwing shade on these external companies (who are innocent) thus deflecting from Boeing's responsibility. This needs to be called out every time.
Wasn't the sensor an AoA vane instead of a pitot tube?
a) critical - if it malfunctions, the plane can crash
b) based on something that is known to fail often, with zero redundancy
Both of those they were mandated to disclose, and they didn't, knowingly. Why is none of these bastards in jail?
> Wasn't the sensor an AoA vane instead of a pitot tube?
Yeah sorry, mistook the sensor, indeed it's the Angle of Attack sensor next to the pitot tubes. Both are prone to frequent, for aviation, failure, so the point still remains.
That is the question, right?
I mean I get the NTSB no blame approach in progressing safety practices, but there needs to exist an authority that enforces these practices.
And the worst part is that this gives a bad rep to "IT" as a whole in the eyes of laypeople, even though IT's internal view aviation is that it's full of safety standards, redundancy, reliability and maintenance is paramount, etc.
It turns out the standards were not followed, and starry eyed junior programmers' view of aviation is just smoke and mirrors.
When will we reach a turning point?
There are skilled people from India, but rarely in India I'd assume. The market for software engineers in India is not great to put it politely so the talented people will obviously go where the money and benefits are which is US, Europe etc. And yes I'm sure there are exceptions, but probably not too many or do you know anyone actually buying software engineer services from India and paying western salaries?
Pop quiz: name a new program that Boeing has started in the last twenty years that was successful. Acquisitions, buyouts, remnant MD programs don't count. Now, during the post-COVID era, they're somehow spending all of their[1] cash in precisely those sectors that are forecasted to never recover - ever. Congratulations. That takes work. If you want to see this poetry in motion, take a look at what happens to any company bought by Boeing. If you could weaponize this kind of value destruction we'd figure out a way to put it in a JDAM and drop it on Hainan Island.
The MAX software issues are at the very tippity tip tail end of a single one of these chains of idiocy, so I feel uncomfortable just piling all the sins on MCAS. The fact they even had MCAS should have been a gigantic red flag, and would have been in anything like a functional regulatory regime. The FAA PNW offices were unofficial Boeing locations even back in 2008.
[1]rapidly shrinking reserves of
It doesn't make sense due to low prices for everything in their country.