It appears likely that this was a test procedure failure rather than a hardware failure. Which brings up the question, why were they doing it at 2AM? Either they were using night crew who are presumably not their A staff or they were using highly fatigued day crew.
I would assume the same. Daytime heat has to add a small complication and if I had complete control of my schedule I’d probably pick 2 or 3 am to avoid as much extra heat as possible In order to more tightly control the parameters...
But I also figure they can control around such small differences given they load supercooled cryogenic liquids into rockets during the day on a regular basis for the Falcon 9 so... yeah.
Pressure to meet schedule? And now the Boss says "don't go to test unless you're sure!" That would be the usual mixed message that amateur management gives to their team.
I'm sure the night crew welders, operators and techs are top notch. Engineers, designers and managers who depend much more on communications and collaboration? They tend to cluster.
With engineers, that may well be a fair assumption. Night shift is less desirable, therefore people with more options will be less likely to settle for it, therefore the better people are in the day shift. Unless this is like shift work where people rotate between day and night shifts - but that would be highly unusual for a company like spacex.
Just playing devil's advocate, but the logic chain is plausible.
Perhaps reclusive people have more time and energy to dedicate to their craft increasing their skill level and propensity to work night shifts? I'm just spit-ballin'.
Smart people seem to be very good at conceiving a plausible chain of logic for almost anything. Something Star Trek demonstrated well with the the Vulcans. I mean that as no snark towards yourself, it's just a subject I've been thinking about recently.
I remember from my intro to neuroscience that people who have lost all emotions due to stroke or brain injury are usually incapable of making even the most minor decisions. They are given two options for a doctor’s appointment and just can’t make up their mind.
I keep thinking about that anytime someone claims to have arrived at their strong-held opinions on purely “rational” arguments.
Haha, yeah I can rationalize almost anything and make it sound plausible. It's a talent of dubious merit.
I think the take-away here is there's no reason to assume that both shifts would have equal skill - likely there will be some differences, and it's a question of how much and why. It would be odd if it turned out there was no difference at all.
There are no shifts for engineers when it comes to testing events. From my experience they are usually all there.
Yes, during the production process you've got shifts of workers/technicians/engineers. Testing: it's all engineers plus enough of technicians they like the most (by negotiating to shuffle around the shifts where possible). It's probably the same at SpaceX.
Shame you think there even is night shift. Obviously you have no knowledge of what actually happens during test like that.
It is likely test like that was planned long in advance and peoples schedules were planned so that right people were available for the test.
If you spent a second thinking you might surmise there might be a reason. I did and noticed Space Shuttle launches happened typically on mornings which means they would be filled up overnight.
I stand by my words. At least I put my name on it. The parent comment was mindless and petty as was already pointed out by other commenters.
Telling people who are probably working their asses off to get things done that they are subpar because (because of what exactly?) I think is enough reason to be at least a little bit derisive about.
I added some additional information to point out there might be a good reason to do test at night since for example, SST missions tended to launch on mornings.
I expect that overwork of the day shift is the far more likely theory. Elon Musk expects employees to work 80-100 hours a week. I've done 100 hour weeks before and I made a lot more mistakes than when I've worked 40.
I have roughly the same experience. At the beginning I am sustained by the urge to get something done so I stop noticing warning signals and when I finally notice it is exactly because of inability to focus and because of mistakes I make.
If 'obviously' there is no night shift, and people had to adjust schedules for this, then this is an instance of "highly fatigued day crew". Because there is no night shift, none of the rocket workers are being insulted.
Also, 'petty' is a very bad description of their argument.
Also, the comment doesn't say there was no reason to do the test at this time. Obviously there was some reason. The comment is questioning whether the reason is more important than having the best engineers in their most awake state.
No way they will fly one of these to the Moon in 2022. Will see the Shuttle effect, ie. introducing engineering workarounds that will greatly increase costs.
What is the base of your theory? SpaceX has already gone through 4 prototypes in just 4 months, and the next one is being tested end of this month. By their roadmap, even with some more test failures, they could still accomplish this and more in 2 years time.
Their competitors have 10x the costs and don't get anything done on time either. This is normal for MIC contractors. They all underbid on their contracts and put their hand out for more money when the phony schedule slips.
I doubt it's 10x, but reuse of the first stage is highly likely to be saving them money. They were already undercutting companies like ULA heavily before reuse.
ULA didn't lose very expensive government satellites with no explanation. In fact, there are no notable ULA launch failures.
You get what you pay for: with ULA, your payload makes it to orbit/space every time. With SpaceX, there's a significant non-zero risk of losing your payload, and that risk discount is what lets SpaceX undercut ULA. (That risk discount is also why SpaceX has to severely undercut ULA; the economics from a customer perspective wouldn't work out if SpaceX were only a small discount from ULA prices.)
> ULA didn't lose very expensive government satellites with no explanation.
Are you talking about Zuma?
There's "no explanation" because it was a classified launch, and the indications we do have are that it was an issue with a Northrop Grumman payload adapter.
ULA's perfect score is a bit of a cheat, as the rockets they launch were inherited. The first flight of the Delta IV Heavy, for example, was a partial failure... but happened before ULA's formation.
RocketLab is in the same ballpark for a smallsat launch (when compared to the SX smallsat program, I realize there are other rideshare options that exist https://www.spacex.com/smallsat)
Unfortunately ULA's launch cost calculator doesn't show final prices anymore (https://www.rocketbuilder.com/), but they're now selling Atlas launches for under $100 million (their "ULA Value" numbers due to reduced schedule slip might be a bit generous but aren't any more egregious than how Tesla includes "potential savings" in the purchase price).
Ariane launches are in the same neighborhood as ULA, perhaps a bit cheaper although they've had occassional schedule issues due to the geopolitical situation in their remote outpost of the EU.
The reality is ISRO and RocketLab play in a totally different market. Neither can compete against dedicated small sat launches from SpaceX. That is a fact that the CEO of RocketLab and many other small launcher CEO have admitted.
And comparing those tiny rockets to Falcon 9 is a joke.
ULA cheapest Atlas with no other extra service might be 100M$ but that gives a 60M$ Falcon 9. And just btw, ULA has no commercial flights anyway, and their military flights are much more expensive. That is not close by any definition.
For the most part the only reason ULA has any flights at all, is because both NASA and DoD pick them to keep multiple providers in the market.
Arianespace is not as cheap as ULA per rocket, maybe with Ariane 6 that will change. Ariane 5 costs 200M$ and launching one only makes sense if you can stack two big Geosats.
10x was wrong, but nobody is close in any practical sense.
If you're going to make personal attacks, you should at least strive to be factually correct with your own claims.
> ULA has no commercial flights anyway, and their military flights are much more expensive. That is not close by any definition.
False. ULA has launched commercial flights in the past[1] and have more on their future manifest. It's true that their launches for the government are more expensive, but that is also the case with SpaceX's government launches: they charged about $90 million each for the most recent GPS-3 flights[2]. Since government payloads are typically uninsured, you also need to take that into account that higher premiums with SpaceX aren't reflected in those launch costs (because the risk is assumed by the gov directly)
> And comparing those tiny rockets to Falcon 9 is a joke.
Rocket Lab is winning contracts that would've previously gone to SpaceX (either directly or through a rideshare provider like Spaceflight (RIP)). They're targeting a different market but there is still some overlap. Pound for pound Rocketlab is still more expensive, but there's value in being able to choose your own orbit instead of being stuck with wherever your ridebuddies want to go. Anyone who has used Uber Pool or Lyft Line is familiar with this phenomenon.
> Ariane 5 costs 200M$ (sic)
No. An Ariane 5 costs around €100 million. Unless you got really unlucky with the exchange rate that isn't 200 million USD.
> For the most part the only reason ULA has any flights at all, is because both NASA and DoD pick them to keep multiple providers in the market.
I think that is unlikely. ULA would certainly have fewer flights if the DOD wasn't trying to ensure the viability of multiple EELV/NSSL launch vehicles, but they would definitely still have some. In particular the payloads that no other American launch vehicle is physically capable of delivering (e.g. when it comes to requirements like big fairing, and horizontal integration. There's also a number of orbits that a Falcon 9 can't hit with the sorts of heavy payloads that the IC likes to send via GTO or even direct to GEO-- it will be interesting to see how that changes now that the Heavy is certified)
> 10x was wrong
Agreed. It's hard to make a totally apples to apples comparison, but based off of recent awards[3] they are certainly within the same order of magnitude. ULA costs maybe 50% more, but it's hard to judge how the different launch providers affect the mission capabilities. e.g. if ULA is able to send something direct to GEO and get closer to the desired apogee, the bus might save a year's worth of propellant. For a constellation like SBIRS where each satellite costs just shy of $1 billion, another year of capability is nothing to sneeze at
> False. ULA has launched commercial flights in the past[1]
Lets be clear here. Before SpaceX showed up the US had about 0% of the commercial launch market. And yes, ULA had a couple of commercial launches, but incredibly few, you could probably count all the commercial launches in the post-2000 on one hand.
What you say is true, Falcon 9 is also more expensive for the DoD as well. But we are still talking about 40M plus in difference minimum.
> Rocket Lab is winning contracts that would've previously gone to SpaceX
I would disagree with that. Maybe a few rideshares (tiny, tiny market) but more likely those would launch on ISRO rocket or one from China.
> No. An Ariane 5 costs around €100 million. Unless you got really unlucky with the exchange rate that isn't 200 million USD.
Whats your evidence for that? That would price would make Ariane 5 dominate in the GTO missions and they clearly are not.
If that were their price they would not have thrown away the Ariane 5 to replace it with Ariane 6.
Btw, they officially said that Ariane 6 would be around 100M and it has less performance then Ariane 5. That would be a pretty stupid thing to do if the Ariane 5 were 100M$ as the Ariane 6 is less performant then 5.
I have done this research a few years ago, and I can't send you reference to the price, but I am 100% sure its much more then 100M$.
> In particular the payloads that no other American launch vehicle is physically capable of delivering
The DoD has already certified that they can fly all orbits that the DoD requires.
However your argument is correct for a small number of launches. Mostly those that require a larger fairing or vertical integration.
SpaceX is adding the larger fairing and a vertical integration bay, they are now already allowed to compete for all future DoD launches.
Until the last contract handout, ULA absolutely was the only provider so many of the flights on their books are from that.
But on those alone ULA could not survive. We will see how the EELV2 turns out.
I think the evidence is pretty clear that SpaceX is by far the cheapest and it seems they are making the best margin as well.
That schedule change wasn't related to a schedule slip on starship, it was a decision to switch from a small mission (by number of people) using a falcon heavy and crew dragon to a bigger mission using starship.
It might have been related to the schedule slippage on crew dragon though, considering they are just now ready to fly astronauts on crew dragon.
It’s questionable if they’ll ever consider landing a Starship on the moon. It would require multiple orbital tankers to give it enough fuel (unless it was a one way trip). And there is no possibility of ever manufacturing metholox on the moon.
October last year was the first presentation of the new design, a year before that was the presentation of the previous design, together with the "Dear Moon" mission.
I've never seen a timeline suggested for this rocket that included getting to orbit before 2020, the earliest rough timelines I can find (circa 2014) have initial test flights in 2020. That seems plausible (if tight) for orbital tests still, and if you count suborbital test flights actually happened ahead of schedule in 2019.
I don't think citing dates like that in a way that implies schedule slippage is fair.
They started on some next generation engine technology in 2012 that they hopped eventually will be used on some future spacecraft that could go to Mars. There was no plan, announcement, design or money to invest in building anything like Starship.
And they never claimed they will land on the moon by 2022. Maybe fly around the moon in 2023.
Not going to happen - there's just no way.
Here's why:
• even if they manage the first orbital flight this year, there's no guarantee they'll stick the landing as well (took quite a few tests for F-9)
• Mars missions require in-orbit refuelling, thus a tanker version of the fairing and a docking adaptor need tobe developed first
• given they manage to develop both the tanker fairing and the docking adaptor earlier next year, they'd still need to test the refuelling process
• provided they have a working Starship, a proven in-orbit refuelling capability and a reliable landing system for Mars (i.e. no carefully prepared landing pads and guided reentry), they'd still be missing any kind of payload to actually send to Mars (in-situ fuel production plant, habitat module, or anything else useful)
There's just so many firsts that they'd need to achieve that there's zero room for failure to get all that done within 24 months. And that's just not in any way realistic.
I agree with everything you said about being ready in time, but if they had a working ship I don't think they'd worry all that much about having a high-quality payload. Offer some nominal cost cubesat slots for whoever wants them and land it empty? Sure, why not.
> even if they manage the first orbital flight this year, there's no guarantee they'll stick the landing as well (took quite a few tests for F-9)
Sure, but they have 2 years to pull that off, and there is a way this happens even in one it's just a bit ambitious.
> Mars missions require in-orbit refuelling, thus a tanker version of the fairing and a docking adaptor need to be developed first
Tanker version is dead simple, just put more rings in both tank sections. In-orbit refuelling might be a bit trickier since it's reasonably novel (technically it's done with the ISS) - but the concept is really simple. SpaceX has done docking before and has complete design authority over both things docking together, I doubt the adapter part is a serious concern for them.
> and a reliable landing system for Mars
Who said anything about reliable?
> they'd still be missing any kind of payload to actually send to Mars
A cybertruck or something. The point is the PR and the capability demonstration not the payload. If they pull this off you can bet they can get significant NASA funding to do it with NASA funded cargo next time around.
The landing software is really the same, the engine controls are different but that was never the major issue.
They don't need a 'tanker version' initially any Starship can refuel any other Starship. The 'tanker version' would just make it more efficient.
How do you know that they don't have any payload? Do you have special insight into what SpaceX is doing? Or are you just assuming because they haven't developed that stuff outside in a tent it can't possibly exist.
The refueling system is fundamental to the transportation problem. Without refueling you don't have transportation, you have one time deliver and Elon knows that.
I'm sorry but that is absolutely, fundamentally and totally wrong. It goes against literally everything anybody from SpaceX including Elon has ever said.
Is it, though? I think you're stuck with the mindset that there's only one type of Starship when there isn't.
There's a distinction to be made between cargo ships, tankers, and crew transport. The first uncrewed cargo flights have zero change of returning to Earth.
There's simply no infrastructure for generating fuel and refuelling yet. So unless you want to tell me that Musk is some sort of space wizard able to conjure methane fuel into the tanks of the first cargo ships that land on Mars, this is exactly what's going to happen.
They don't have a Moon plan. They're just developing a "classic" Dragon-2 cargo capsule for potential orders that may be for the Moon or a Moon orbital station.
Yeah, gotta say that the NASASpaceflight forum does a better job of keeping on top of Starship-related things and being organized than the Reddit site.
This is really bad. I don't know how much money they have left but I cannot imagine they can continue blowing up prototipes that took weeks to build every other week like this.
These are relatively cheap prototypes. No engines, just the fuel tanks and body. Cheap stainless steel, fabricated in weeks. If this was Boeing, it'd set them back years. The whole idea is to crank out a steady stream of cheap prototypes to iterate design ideas and fixes rapidly.
At most other companies you have a whole hierarchy of executives who want their bonuses, so you have that risk aversion where they will spend tremendous time and resources covering their asses first. Failure is failure, it's not just a step to more improvement in that world.
This is not bad, this is how progress is made in this industry - These are not fully operational prototypes, they are parts fabricated purely for the purpose of testing the behavior of designs far beyond required limits, the purpose of these tests at this stage is to collect useful data not prove designs.
The Starship is made of steel. It costs only 3% of the price of the previous design. So they can burn through a lot of prototypes to improve design and manufacturing, before costs becomes an issue.
The article makes it seem like this is a good thing, I.e., there’s nothing inherently wrong with the design.
If true, it means it may be a procedural escape which can be every bit as dangerous. Some of the fallout from the CST-100 Boeing Starliner issues seem to point to process escapes as well. Makes me wonder if in the rush to get “boots on the moon” by 2024 we may be letting schedule get the best of us...
Boeings most recent mission was supposed to be a demonstration that they were ready to fly crew to the station. Both hardware and software was supposed to be effectively finalized. This is compared to Starship where no one even really knows what the final hardware or software will look like yet and they are trying out manufacturing techniques in a field. The tests being performed are experiments, to see what does and doesn't work. Comparing the two failures is completely ridiculous.
Not only was Boeing software supposed to be effectively finalized, but Boeing hardware also appears to have failed on that test. The thrusters experienced issues, and the antenna that were contractually required to provide end to end connectivity failed to do so. Moreover on the second most recent demonstration (pad abort) Boeing suffered a parachute failure.
Pretty good assessment. I couldn’t tell from your wording but I think the understanding is the thruster and comm issues were software related while the parachute issue was not?
Sorry, I guess I should have been more clear. The thruster and comms issues were also examples of hardware related issues. I can't say for certain that the comms issue couldn't be fixed in software, but it seems unlikely. The thruster issue was related to overuse and is arguably mitigated by "not doing that (via software changes and training for human pilots)", but it really shouldn't have been an issue in the first place.
The software issues were the more serious issues that grabbed more headlines. The capsule incorrectly grabbing a time value from the rocket. The control logic for whatever reason didn't understand something was wrong, but instead "proceeded as normal" as if it was further ahead in the mission. Finally there was some other error that was caught before it happened but was something related to the thruster control logic and could have caused the loss of the capsule during re-entry.
I appreciate both the thanks, and that you identified the source of the confusion. I also appreciate that you made it clear that you know what you're talking about and that I didn't really need to write such a long answer for you.
My apologies. I was remembering all of the people claiming that the facts were known before the initial investigation was done, but the initial investigation finished 3/6.
There is a second investigation currently ongoing, an "organizational root cause assessment".
No worries. And yes, I remember how much people jumped to all kinds of conclusions after the event first happened.
I think what you mean is the "organizational safety assessment" (OSA) which will occur later this year.[1] Germane to my previous comment, this will focus more on the organizational culture that may have contributed to the failure (e.g., decision making in the presence of heavy schedule pressure).
I couldn’t find any sources using that term. It might be an issue of quoting unreliable sources. Not really nitpicking in my estimation as they are different perspectives. OSAs are solely looking at workplace culture. RCA has a broader scope.
> Be kind. Don't be snarky. Have curious conversation; don't cross-examine. Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
When disagreeing, please reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3."
Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
Here are some official sources for you on there also being hardware problems
> The review team also is continuing its investigation of the intermittent space-to-ground forward link issue that impeded the flight control team’s ability to command and control the spacecraft. The team has identified the technical root cause as radiofrequency interference with the communications system. While the team has recommended specific hardware improvements already in work by the company, the full assessment and resulting recommendations will continue through March.
> The NASA source said eight or more thrusters on the service module failed at one point and that one thruster never fired at all.
> In response to a question about thruster performance, Boeing provided the following statement to Ars: "After the anomaly, many of the elements of the propulsion system were overstressed, with some thrusters exceeding the planned number of burns for a service module mission. We took a few cautionary measures to make sure the propulsion system stayed healthy for the remainder of the mission, including re-pressurizing the manifold, recovering that manifold’s thrusters. Over the course of the mission we turned off 13 thrusters and turned all but one back on after verifying their health."
This is an early manufacturing path-finding test article and test, not a test article and test designed to validate that the system is safe. The procedures, design, manufacturing techniques, and everything else are still being developed and certainly haven't been validated yet. Failures are expected.
It is a good thing that there is nothing inherently wrong with the design, just because that would mean more re-design work for SpaceX. A procedural issue (possibly this failure), a manufacturing defect (MK1 failure), a failure due to a temporary hack (SN1 failure), etc are all understandable errors and easily fixable at this point in time.
Starship isn't really being driven by the "boots on the moon by 2024" push by the government. Starship is a private venture being developed on their own schedule (and with their own money).
> Starship isn't really being driven by the "boots on the moon by 2024" push by the government.
You’re right, bad choice of phrasing on my part to boil it down to a turn-of-phrase. What I’m really poking at is how schedule pressure affects decision making. SpaceX is well known to have very ambitious timelines. I am wondering how much that pressure affects process checks like configuration control
I worry that the people who ran this test either didn't know what they were doing, or were too tired or didn't care.
The top tank was covered in ice, indicating it was nearly full, while the bottom tank had no ice. That leaves three possibilities:
* The intention is the bottom tank was pressurized with gas:. There is no way that much gas at ~4 bar is safe. It would destroy all nearby buildings if it failed. That's a serious safety failing.
* The intention is the bottom tank was filled with LN2. Whoever was running the test didn't notice the tank didn't turn white, and didn't have any kind of level gauge or flow meter?
* The intention is the tank was unpressurized. The design could never withstand the top tank filled and the bottom tank unpressurized and empty. Whoever designed and ran the test didn't understand the basics of the design.
All of those 3 possible explanations point to more than a simple oversight, and IMO suggest the team working at 3am might possibly not really understand what they were doing.
> There is no way that much gas at ~4 bar is safe. It would destroy all nearby buildings if it failed.
Wat?
This is a launchpad, the safety assumption at play here is that this vehicle (and in fact a much larger vehicle with this vehicle as a second stage) can be filled with liquid methane and liquid oxygen, explode (with the energy of a small nuclear bomb, a few kilotons), and no one will get hurt. The area around the launchpad is kept clear during testing by local law enforcement for this very reason.
Hours before the test that failed, in a different test, the same test article was pressurized with room temperature nitrogen, likely up to ~7 bar (based on previous tests for which we were given figures).
For comparison, this is the same location that the following test was performed at, all the additional structures you see have been added recently by SpaceX as ground support equipment and they are undoubtedly perfectly aware and ok with the idea that they might be damaged in the case of a failure https://www.youtube.com/watch?v=bYb3bfA6_sQ
As for the rest of your post, yes, if you assume the people running the tests were incompetent then they would be incompetent. We can't fully rule out the idea of course, but it's far from the only option. Most likely the failure was not nearly so simple as "we forgot to pressurize a tank" - particularly considering that the bottom tank was clearly pressurized as it was venting.
The most reasonable suggestion for a proximate cause I've seen, for instance, is that the pressure release valve on the lower tank iced over and leaked until the pressure was no longer enough to support the above mass and it failed. This is supported by the evidence that we know they were having valve issues related to cryogenic liquids earlier in the day, and that the bottom tank was inexplicably venting for roughly 13 straight minutes before the failure. It doesn't quite line up with Elon's suggestion that this was possibly a test configuration error.
Somewhat notable is that a test article from a recent Crew Dragon parachute test was also just destroyed due to problems with how they set up their test
What I love about SpaceX is that they truly take on the start-up approach of fast iteration, in complete opposite to most of their competitors.
This is a silly loss, as it sounds like it was a test procedure error and not a valuable experimental failure. But even then, how much is lost? The rocket was built on-site in a tent, welding some cheap steel together. They've got more test articles being built. The engine is probably salvageable. This is a month or two set-back at worst.
And just like when our software fails in a silly way, I'll bet you there will be new automatic tests, procedures, etc, so that this kind of error can't happen again. They'll learn, iterate, and continue to move fast (and break things).
With nearly all of their competitors, any kind of failure like this would be millions of dollars lost, months or years of setback. This fundamental difference is why SpaceX is slowly coming to dominate the industry.
The vehicle didn't even have any engines on it when it failed. SpaceX was planning to perform cryogenic pressure tests with simulated engine thrust, then if that was successful, mount three Raptor engines and perform static fire tests and flight tests.
> They're making progress in the industry, sure. But dominating it? In what sense is this true?
In addition to capturing most of the commercial market that makes sense for them.
They are NASA most trusted partner, for both cargo to ISS. They will be the first private company to fly humans into orbit.
They are by far the cheapest, and even the rockets that are in development can't really compete against their lineup.
They are operating the largest launch vehicle on the planet.
They are universally considered to be the global leader in rocket engine design as both their Merlin and Raptor engines break many records and did stuff people thought was not possible.
They are in prime position to capture the single most valuable space market, LEO internet.
They are clearly the dominant player in the industry, every other player has admired it. Russian are out of commercial launch, Europeans are spending 2.5 billion to develop the Ariane 6 (who already has problems finding missions), ULA is dropping both their Atlas and Delta rocket to have a chance against SpaceX, and they have a very clear strategy to focus only on military and NASA launches.
I also love how fast SpaceX iterates but it does come at a cost. If one design fails, they can quickly iterate to a different design that works. But sometimes that means they aren’t investigating the science behind the failure. Great from an operational engineering perspective but less good from a fundamental science perspective. To their credit, I don’t think the latter is their main role.
Regarding the cost, the aerospace industry needs some scrappy young start ups. However, quality pedigree is a large chunk of the cost of flight and SpaceX has shown some gaps in the past. I think a big question is whether they can keep that scrappy startup up culture while also building processes robust enough to meet space flight quality standards
If they don't investigate the science behind "why" how do you think they come up with new ideas and improvements? I'm sure they don't discard their learnings!
Say they have a check valve in their titanium system that fails. They could spend a lot of research dollars on the fundamental science of why that valve failed or just replace it with a burst disk. From an operational engineering point, they may feel like they solved the problem. But from a scientific standpoint we haven’t learned much other than “check valves are bad”.
It’s akin to the old joke about a mathematician, physicist, and engineer trying to find the volume of a cow. The mathematician tries to formulate a triple integral for the cow’s shape. The physicist drops the cow in water and measures the displacement. The engineer says, “let’s suppose the cow is a sphere...”
Point being, often in engineering they’re shooting for “good enough to get the job done” not to increase fundamental understanding
You can do both though. Replace the valve with a burst disc now, just so testing can continue. Investigate the valve in parallel with testing, and place it back in whenever you think you've solved it, if you still need it.
What I’m saying is that I don’t get the impression they do the latter for a variety of reasons. I.e., they solve the engineering problem and move forward. However, I think NASA may move forward with the research part
You can, but it is expensive. SpaceX definitely does some research, but they are also definitely not in it for researches sake. If it isn't necessary to get to mars, and it isn't going to make them money to figure out why, they're probably just going to drop it.
As I've heard that joke, it's the engineer who suggests measuring displacement and the physicist who wants to assume the cow is a sphere. The joke is that physicist's models oversimplify reality.
> What I love about SpaceX is that they truly take on the start-up approach of fast iteration, in complete opposite to most of their competitors.
I won't call their approach a "start-up" or even unique one, especially considering they made a huge component of this (Raptor) in a pretty standard way. Building in a "tent" just means they don't need anything more than it being a tent right now, not that they won't ever need more.
For some reason, rocket engineering has that reputation of being a complete waterfall process. It isn't deserved and doesn't make sense. Rapid iterative development and quick discarding of failing prototypes in the early phase is as common in space industry as it is everywhere. It just doesn't usually make headlines and isn't that visible.
Yeah, one of the things that struck me when reading about the development of the SR-71 and F-117[1] was how a lot of what we consider "start-up" culture has already existed in one form or another.
The good parts of Agile are pretty much taken verbatim from the approach taken by project Mercury in the 60s. The space industry pretty much gave birth to iterative and incremental development
Every bit of negative Elon Musk oriented news brings out an apologetic post excusing failure and painting it positively, with a pile of democratic support.
This is a failure, and failure is perfectly ok working on something like this, but what drives people to volunteer their time to spin such things as positively as possible or even signal boost ? I can’t believe it’s purely investor driven. It’s almost as if people are finding personal purpose in sacrificing their time to this cause.
Why do people cheer for sports teams? Many nerds avoid team sports, but we share some of the same psychology as the more typical.
Many people like to dream big, but most of the time life is mundane. SpaceX's efforts are the embodiment of sci-fi fantasy with a chance of becoming real. Why not cheer?
I have for years thought of SpaceX as my sports team to cheer for. I get just as excited watching their launch livestreams as many get watching football games.
Repeated exposure to the idea that anything wrong with one of Musk's projects is the end of the world could probably produce an extreme opposite response, even if they weren't interested in Musk at first.
Because people admire their courage in being so transparent about failing publicly. That's undoubtedly helped in building their audience over the last decade not because they want to see SpaceX fail, far from it but because it reminds us that they too are only human and failing is normal no matter how much others try to hide or gloss over their failures.
I turned from a Tesla / SpaceX fan to someone who's indifferent to or even enjoys failure. The cultish fans and more importantly the narcissistic CEO ruined it for me. This comment will get downvoted.
Over half of all startups fail within 5 years. In the space industry it's much much worse. Although as long as SpaceX has cash I guess they'll just keep on trucking.
My point was that hard and fast startup tactics don't guarantee success (WRT the starship project), and mostly have little to do with whether a project succeeds or fails at all.
Yeah, hardware is unforgiving like that. Made a mistake in calculations? Start over and spend a few weeks making another widget. There better not be a mistake in that, otherwise lather, rinse, repeat until you figure out how not to make mistakes.
It's funny how extreme the reactions are here. It's either the end of the world or exactly as planned. Both are nonsense.
To me, this looks like it was definitely a noteworthy fuckup, but in the big picture it doesn't seem to be a major setback -- nothing like the recent Boeing issues. That is, as long as a pattern of such events doesn't develop. SpaceX continues to push the industry forward in interesting and exciting ways.
To continue the arm-chair quarterbacking... I feel like a moderate decrease in process aversion would be a good response to this incident. Reduce the chance of similar future fuckups without slowing down too much.
From a process design/management view I'm not sure you're right. In fact I think you're probably wrong, but there is definitely room for disagreement.
To oversimplify things massively, "how careful do we want to be to avoid testing failures" is a parameter that SpaceX management gets to control. At the extreme careful end everything costs billions of dollars because you're paralyzed analyzing things and double/triple/quadruple/quintuple checking things. At the not at all careful end of things you keep building "complete rockets", pointing them at mars without any testing, and blowing them up. Obviously neither is rational.
They've settled on some parameters that are basically "don't worry that much about test failures, but keep the failures really cheap". They seem to be doing pretty well by doing that. They're experimenting with manufacturing techniques, hiring and training a workforce, building facilities. The prototypes that they are building keep failing, but that looks to be a relatively minor cost considering that the current primary goal is (according to them) to build out the manufacturing processes and make the design easy to manufacture.
Being slightly more careful would undoubtedly reduce the numbers of prototype failures, but would it actually be worth the cost of slowing other things down? Remember that the main cost of this program to SpaceX is engineering salaries, the faster it goes the cheaper it is.
So of course this test didn't go as planned, of course it would be better if it had worked, but would it have really been worth it to management to reduce the probability of this test failing? Like I said, maybe they aren't being careful enough, but I don't think we have any real evidence for that right now and I personally doubt it.
I think this is where NASA went wrong in many respects. They got so risk adverse, they built a bureaucracy of checks and oversight that moves slow. That’s one of thew reasons the industry needs fresh blood like SpaceX
I feel NASA's approach is right for manned spaceflight, but is not sensible for robotic missions - metal is cheap, design time is expensive. I feel the public appearance is not a hard - nosed calculation, but Is instead a pr requirement
I actually disagree with you. For one, consider that Starship is ultimately intended for manned spaceflight... yet SpaceX is still trying things and blowing stuff up (or imploding or...)
From my armchair perspective I think the risk adversity criticism is justified. NASA should take much more risk even in human spaceflight... in the early stages of platform development. First see if you can build the damn thing and build it repeatedly. Later, after that period, reduce risk and prove out and refine things to the human spaceflight standard. If you jump immediately to, "my God people are suppose to fly on that thing!" well, you're probably going to pay the Russians a lot of money for a fairly long time.
I do agree with you that the political management becomes a real issue. Politicians and bureaucrats are naturally disposed to a certain cravenness about such issues once they get to significant positions of power (senior execs of many companies do, too). But I think a charismatic leader that's out in front of the messaging could pull it off.
I think this is a very good point that is frequently overlooked, it would only cost 2x to have 10 rovers instead of one!
It is also absolutely embarrassing that we still haven't fully surveyed the moon for resources, dont know where all the ice is, etc. Lunar rovers should be numbering in the hundreds, maybe thousands.
Frankly I'd rather them blow up my money than spend twenty times the original budget and ten times the schedule filling the pockets of companies like Boeing.
1. Keep in mind NASA has multiple Centers and only one (Houston) is dedicated to human-rated spaceflight
2. Dirty little secret: even when NASA uses “startups” like SpaceX to launch and things go awry, the taxpayers are still on the hook for the majority of the loss because the Agency is self-insured
A JPL project manager teaching a course I took blamed it on the public's "mathematical illiteracy." He explained that there are diminishing returns on mission success rates so NASA often had the choice of paying X per mission attempt each with an 80% success rate or 2X for a single launch with 95% success rate. 80% sounds bad but the chance of two launches failing is 4%, which means a 96% chance overall success rate. If the first launch succeeds, they can usually save between 20-40% of the total budget (the remaining amount is overhead to build backups and keep scientists and engineers employed in case the first launch fails). However, since NASA is no longer doing world shattering stuff like landing on the moon, the human echo chamber amplifies each failure and drowns out the successes. Since NASA has to justify many appropriations on a per project basis, failures lead to projects getting canceled and NASA doing less actual science, even though their budget rarely shrinks - with more and more money instead allocated to inefficiencies like maintaining unused real estate/hardware and spreading out manufacturing across all districts in order to curry favor to get more projects going.
That's just for robotic/satellite missions. It's ten times worse when a manned mission ends fatally.
Yeah they said they will return to the Moon or Mars (depending on the president) since Bush 1. That is almost 40 years.
So another new moon plan. Another Shuttle derived launcher.
Not surprising that nobody is getting very exited with Artemis.
The SLS and Orion can only be described as dead-weights on NASA since they are so incredibly expensive. SLS has been in development for over 10 years at more then 2 billion per year and it will not launch until 2022 hopefully.
> NASA has its Artemis program which is doing exactly this
They've been designing Orion for a couple decades now, and the SLS is a shuttle-derived expendable launcher where what we see as key innovations are mounting 25% more SSMEs (still expendable) on the bottom of the fuel tank, extending the SRBs by a couple segments and putting the payload on top of the main tank.
Ares was interesting and we learned, with Ares I, that we shouldn't put humans on top of an SRB. There would probably be cheaper ways to find that out, but at least they tried something different.
> 80% sounds bad but the chance of two launches failing is 4%, which means a 96% chance overall success rate.
I don't disagree but I also don't think it's always as straightforward as probability theory would make us believe. One can imagine situations where a certain amount of QA leads to a certain semi-predictable amount of failures that allows for calculated risks to be taken, but one can also imagine situations where paying 2X leads to better solutions and processes and not paying 2X leads to failure on both the first and second launches.
> 80% sounds bad but the chance of two launches failing is 4%, which means a 96% chance overall success rate
I'm sorry, but this argument is also mathematically illiterate. Probability only works this way if the events are uncorrelated, but it should be pretty clear they're not.
If the first rocket has a 20% chance to fail it is likely that the second one will also fail due to similar negligence.
You're right in theory but demonstrably wrong in reality. Just look at the failure rate of launch platforms in the last 30 years. If I remember correctly, launch failures accounted for less than half of mission failures and a significant number of those weren't explosions but burn rate problems that ended up in the wrong orbit.
Rockets aren't mass manufactured and their fabrication is barely automated. Nobody except SpaceX keeps their tooling online long enough to have correlating failures except for actual design issues which are fixed in later launches. The correlated failure comes from the actual mission hardware but launch schedules are so spread out that a second mission has plenty of time to fix issues discovered in the first attempt.
If rockets, which are made one at a time and launch maybe a few times a year, had correlating failures like you said, no insurance company would cover launches.
It's unclear to me what you are basing this on. How do you know that the "how careful do we want to be to avoid testing failures" parameter was exactly where SpaceX wanted it to be in this experiment? Or was at an optimal point for SpaceX in general? If you don't know this - then your reply doesn't really mean anything, and can be be used to justify any mistake whatsoever.
I definitely don't know that the parameter is where SpaceX wants it to be, or even more importantly where SpaceX should want it to be given the information available to SpaceX. That's why there is room for disagreement.
I do have some hints that it's probably more or less where SpaceX wants it to be, some hints that they have chosen this parameter rationally, and some faith that they aren't stupid. For instance, Elon often discusses future tests failing as acceptable, they've declined to put in place more stringent QA efforts, they're still mostly building the hardware exposed to the elements, Elon has discussed that currently their approach is building a lot of starships to iterate on the manufacturing methods and to help them make an easy to manufacture design - he has signaled fairly clearly via his tweets that actually flying the "ships" is secondary, and so on.
My prior is that the parameter is as likely to be set as too high as too low. The testing failure at this stage in development is weak evidence that it is too low, the rest of the information I've just talked about is more convincing evidence that it is an appropriate value or too high, hence I think the post I responded to is "probably wrong".
An example of where I think the level of caution was probably too low (or there were other issues, but let's pretend for a second that this is the only knob management has, as I said it is an over simplification) - the recent CST 100 orbital flight test failure. That test was intended to demonstrate they were ready to fly humans, instead they suffered a series of serious failures, failing that sort of test is pretty strong evidence that they weren't careful enough. They've indicated this type of failure was unexpected and that it will cost the company slightly under half a billion dollars, which is also pretty strong evidence that they weren't careful enough. Moreover I know that they neglected to use procedures such as end-to-end software simulation that would have reduced this risk, and that they were in fact contractually required to perform. Meanwhile I don't have any real evidence that they were careful enough with regards to that test. So starting with the same prior as above, I quickly fall to the "they probably weren't careful enough side".
> Or was at an optimal point for SpaceX in general
The optimal point is definitely not a global value for all of SpaceX. Human spaceflight programs have obviously set it very high instead of very low. As they progress from experimenting with manufacturing to flying payloads it obviously gets turned up. Etc.
But what are we comparing this to? NASA and the Soviet's space program accomplished more with a fraction of the resources and had a better success rate.
I really don't understand how people try to defend Elon Musk and team on anything and everything they do. It's nice they are pushing the edge, but please, when things fail, let's acknowledge failure and shortcomings and learn from them rather than try to spin everything SpaceX or Tesla does as faultless and forward.
I assume that lots of people have financial stake in Tesla, and because of that have an inherent bias that often shows up when they comment on things. If you believe in Tesla, you may be long $TSLA, and you might tend to want to see the bright side of everything they do.
I wish these sort of meta-comments would be posted as replies to the comments they're applicable to, instead of leaving it to the reader to figure out which comments you're criticizing.
How many rockets did SpaceX blow up before they were able to launch anything in to orbit and then again how many until they were able to successfully land first stages?
After learning from all those failures they now have reusable first stage rockets and have moved the entire space industry forward.
This wasn't a Starship failure. SN3 is the second stage of what will eventually become Starship. And this wasn't even the complete second stage. It didn't include the Raptor engines, it was just the pressure tank and hull that did a RUD.
While i've happened to be wrong anytime i've questioned Musk's decisions/approach, looking at that and the previous pressure test failure and how the body crumples, i wonder whether the way the Starship body is built isn't faulty by design - like all this steel plates horizontally oriented.
Of course, fast and cheap is the only way to get to build hundreds of Starship, the Liberty ship of our time, yet the current structural design of the body seems to require re-work.
I honestly love Elon Musk and much of what he tries to do, but how are these tests still running? Are rocket scientists and engineers immune to the coronavirus?
The whole rocket industry is considered relevant to national defense, and thus "essential", so they don't have to stop if they don't want to. And Elon doesn't want to.
They didn't loose any Starship, they are still testing tank weldings with various pressure tests, which had a problem from the beginning.
Then they switched the manufacturer and the tank welding might still not be good enough. Or as announced it was just a bad test.
Such iterations are totally expected, it's the third now (SN3, serial number 3), but no engine tests yet in integration with the spectacular new Raptor engine.
i know that this sounds stupid but failure is par core for spacex. i remember a time when landing rockets were impossible and then it changed to landing orbital class rockets is very difficult to being surprised when a rocket doesnt land successfully. hopefully we will remember these failures and hopefully spacex comes through again with better designs and sees this project to success.
208 comments
[ 2.1 ms ] story [ 325 ms ] threadThankfully, they seem to be fairly easy to build, and the tests aren't burning through the really expensive bits thus far, the engines.
Edit: everybody is taking this as an attack on night crew. It was poorly worded -- I believe the latter is more likely, since Elon expects his employees to work 80-100 hours a week. https://www.businessinsider.com/elon-musk-says-80-hours-per-...
But I also figure they can control around such small differences given they load supercooled cryogenic liquids into rockets during the day on a regular basis for the Falcon 9 so... yeah.
Just playing devil's advocate, but the logic chain is plausible.
Smart people seem to be very good at conceiving a plausible chain of logic for almost anything. Something Star Trek demonstrated well with the the Vulcans. I mean that as no snark towards yourself, it's just a subject I've been thinking about recently.
I keep thinking about that anytime someone claims to have arrived at their strong-held opinions on purely “rational” arguments.
I think the take-away here is there's no reason to assume that both shifts would have equal skill - likely there will be some differences, and it's a question of how much and why. It would be odd if it turned out there was no difference at all.
Yes, during the production process you've got shifts of workers/technicians/engineers. Testing: it's all engineers plus enough of technicians they like the most (by negotiating to shuffle around the shifts where possible). It's probably the same at SpaceX.
It makes all sense to test at night and keep building during the day.
It is likely test like that was planned long in advance and peoples schedules were planned so that right people were available for the test.
If you spent a second thinking you might surmise there might be a reason. I did and noticed Space Shuttle launches happened typically on mornings which means they would be filled up overnight.
Telling people who are probably working their asses off to get things done that they are subpar because (because of what exactly?) I think is enough reason to be at least a little bit derisive about.
I added some additional information to point out there might be a good reason to do test at night since for example, SST missions tended to launch on mornings.
https://www.businessinsider.com/elon-musk-says-80-hours-per-...
If 'obviously' there is no night shift, and people had to adjust schedules for this, then this is an instance of "highly fatigued day crew". Because there is no night shift, none of the rocket workers are being insulted.
Also, 'petty' is a very bad description of their argument.
Also, the comment doesn't say there was no reason to do the test at this time. Obviously there was some reason. The comment is questioning whether the reason is more important than having the best engineers in their most awake state.
The fact that SpaceX has missed every deadline Musk has ever set?
They get some absolutely impressive and ground-breaking stuff done, but they don't do it on-time.
I think they'll get to the Moon. I don't think they'll do it on the original schedule.
SpaceX is a private company. Do you have a source for what their actual costs are for any of these projects?
Musk has claimed all kinds of things relating to costs with his other public companies, and none of it has been true.
You can afford to undercut profitable companies when you set OPM on fire.
You get what you pay for: with ULA, your payload makes it to orbit/space every time. With SpaceX, there's a significant non-zero risk of losing your payload, and that risk discount is what lets SpaceX undercut ULA. (That risk discount is also why SpaceX has to severely undercut ULA; the economics from a customer perspective wouldn't work out if SpaceX were only a small discount from ULA prices.)
Are you talking about Zuma?
There's "no explanation" because it was a classified launch, and the indications we do have are that it was an issue with a Northrop Grumman payload adapter.
ULA's perfect score is a bit of a cheat, as the rockets they launch were inherited. The first flight of the Delta IV Heavy, for example, was a partial failure... but happened before ULA's formation.
"SpaceX will meet one of Musk's deadlines" is, thus far, not a statement I'd bet on coming true.
ISRO launches are cheaper than SpaceX
RocketLab is in the same ballpark for a smallsat launch (when compared to the SX smallsat program, I realize there are other rideshare options that exist https://www.spacex.com/smallsat)
Unfortunately ULA's launch cost calculator doesn't show final prices anymore (https://www.rocketbuilder.com/), but they're now selling Atlas launches for under $100 million (their "ULA Value" numbers due to reduced schedule slip might be a bit generous but aren't any more egregious than how Tesla includes "potential savings" in the purchase price).
Ariane launches are in the same neighborhood as ULA, perhaps a bit cheaper although they've had occassional schedule issues due to the geopolitical situation in their remote outpost of the EU.
The reality is ISRO and RocketLab play in a totally different market. Neither can compete against dedicated small sat launches from SpaceX. That is a fact that the CEO of RocketLab and many other small launcher CEO have admitted.
And comparing those tiny rockets to Falcon 9 is a joke.
ULA cheapest Atlas with no other extra service might be 100M$ but that gives a 60M$ Falcon 9. And just btw, ULA has no commercial flights anyway, and their military flights are much more expensive. That is not close by any definition.
For the most part the only reason ULA has any flights at all, is because both NASA and DoD pick them to keep multiple providers in the market.
Arianespace is not as cheap as ULA per rocket, maybe with Ariane 6 that will change. Ariane 5 costs 200M$ and launching one only makes sense if you can stack two big Geosats.
10x was wrong, but nobody is close in any practical sense.
If you're going to make personal attacks, you should at least strive to be factually correct with your own claims.
> ULA has no commercial flights anyway, and their military flights are much more expensive. That is not close by any definition.
False. ULA has launched commercial flights in the past[1] and have more on their future manifest. It's true that their launches for the government are more expensive, but that is also the case with SpaceX's government launches: they charged about $90 million each for the most recent GPS-3 flights[2]. Since government payloads are typically uninsured, you also need to take that into account that higher premiums with SpaceX aren't reflected in those launch costs (because the risk is assumed by the gov directly)
> And comparing those tiny rockets to Falcon 9 is a joke.
Rocket Lab is winning contracts that would've previously gone to SpaceX (either directly or through a rideshare provider like Spaceflight (RIP)). They're targeting a different market but there is still some overlap. Pound for pound Rocketlab is still more expensive, but there's value in being able to choose your own orbit instead of being stuck with wherever your ridebuddies want to go. Anyone who has used Uber Pool or Lyft Line is familiar with this phenomenon.
> Ariane 5 costs 200M$ (sic)
No. An Ariane 5 costs around €100 million. Unless you got really unlucky with the exchange rate that isn't 200 million USD.
> For the most part the only reason ULA has any flights at all, is because both NASA and DoD pick them to keep multiple providers in the market.
I think that is unlikely. ULA would certainly have fewer flights if the DOD wasn't trying to ensure the viability of multiple EELV/NSSL launch vehicles, but they would definitely still have some. In particular the payloads that no other American launch vehicle is physically capable of delivering (e.g. when it comes to requirements like big fairing, and horizontal integration. There's also a number of orbits that a Falcon 9 can't hit with the sorts of heavy payloads that the IC likes to send via GTO or even direct to GEO-- it will be interesting to see how that changes now that the Heavy is certified)
> 10x was wrong
Agreed. It's hard to make a totally apples to apples comparison, but based off of recent awards[3] they are certainly within the same order of magnitude. ULA costs maybe 50% more, but it's hard to judge how the different launch providers affect the mission capabilities. e.g. if ULA is able to send something direct to GEO and get closer to the desired apogee, the bus might save a year's worth of propellant. For a constellation like SBIRS where each satellite costs just shy of $1 billion, another year of capability is nothing to sneeze at
[1]: A random example here https://space.skyrocket.de/doc_sdat/jupiter-2.htm
[2]: https://www.losangeles.af.mil/News/Article-Display/Article/1...
[3]: https://spacenews.com/air-force-awards-739-million-in-launch...
Lets be clear here. Before SpaceX showed up the US had about 0% of the commercial launch market. And yes, ULA had a couple of commercial launches, but incredibly few, you could probably count all the commercial launches in the post-2000 on one hand.
What you say is true, Falcon 9 is also more expensive for the DoD as well. But we are still talking about 40M plus in difference minimum.
> Rocket Lab is winning contracts that would've previously gone to SpaceX
I would disagree with that. Maybe a few rideshares (tiny, tiny market) but more likely those would launch on ISRO rocket or one from China.
> No. An Ariane 5 costs around €100 million. Unless you got really unlucky with the exchange rate that isn't 200 million USD.
Whats your evidence for that? That would price would make Ariane 5 dominate in the GTO missions and they clearly are not.
If that were their price they would not have thrown away the Ariane 5 to replace it with Ariane 6.
Btw, they officially said that Ariane 6 would be around 100M and it has less performance then Ariane 5. That would be a pretty stupid thing to do if the Ariane 5 were 100M$ as the Ariane 6 is less performant then 5.
I have done this research a few years ago, and I can't send you reference to the price, but I am 100% sure its much more then 100M$.
> In particular the payloads that no other American launch vehicle is physically capable of delivering
The DoD has already certified that they can fly all orbits that the DoD requires.
However your argument is correct for a small number of launches. Mostly those that require a larger fairing or vertical integration.
SpaceX is adding the larger fairing and a vertical integration bay, they are now already allowed to compete for all future DoD launches.
Until the last contract handout, ULA absolutely was the only provider so many of the flights on their books are from that.
But on those alone ULA could not survive. We will see how the EELV2 turns out.
I think the evidence is pretty clear that SpaceX is by far the cheapest and it seems they are making the best margin as well.
Sorry if my initial statement was rude.
There's no way they'll land on the moon in 2022.
It might have been related to the schedule slippage on crew dragon though, considering they are just now ready to fly astronauts on crew dragon.
October last year was the first presentation of the new design, a year before that was the presentation of the previous design, together with the "Dear Moon" mission.
I don't think citing dates like that in a way that implies schedule slippage is fair.
And they never claimed they will land on the moon by 2022. Maybe fly around the moon in 2023.
Musk seems to be off by 1.8 on average.
1.8 is also one Mars year in Earth years. Coincidence?
Yes.
Because I am not sure how you average out against an infinite time frame (E.g. where are the country-wide summon, snake-charger, car carrier, etc?).
Model Y was early, Model 3 was pretty close to on time, Starship is progressing well.
Will be interesting to see how Cybertruck, Semi and new Roadster stick to their timelines.
• even if they manage the first orbital flight this year, there's no guarantee they'll stick the landing as well (took quite a few tests for F-9)
• Mars missions require in-orbit refuelling, thus a tanker version of the fairing and a docking adaptor need tobe developed first
• given they manage to develop both the tanker fairing and the docking adaptor earlier next year, they'd still need to test the refuelling process
• provided they have a working Starship, a proven in-orbit refuelling capability and a reliable landing system for Mars (i.e. no carefully prepared landing pads and guided reentry), they'd still be missing any kind of payload to actually send to Mars (in-situ fuel production plant, habitat module, or anything else useful)
There's just so many firsts that they'd need to achieve that there's zero room for failure to get all that done within 24 months. And that's just not in any way realistic.
Sure, but they have 2 years to pull that off, and there is a way this happens even in one it's just a bit ambitious.
> Mars missions require in-orbit refuelling, thus a tanker version of the fairing and a docking adaptor need to be developed first
Tanker version is dead simple, just put more rings in both tank sections. In-orbit refuelling might be a bit trickier since it's reasonably novel (technically it's done with the ISS) - but the concept is really simple. SpaceX has done docking before and has complete design authority over both things docking together, I doubt the adapter part is a serious concern for them.
> and a reliable landing system for Mars
Who said anything about reliable?
> they'd still be missing any kind of payload to actually send to Mars
A cybertruck or something. The point is the PR and the capability demonstration not the payload. If they pull this off you can bet they can get significant NASA funding to do it with NASA funded cargo next time around.
They don't need a 'tanker version' initially any Starship can refuel any other Starship. The 'tanker version' would just make it more efficient.
How do you know that they don't have any payload? Do you have special insight into what SpaceX is doing? Or are you just assuming because they haven't developed that stuff outside in a tent it can't possibly exist.
I know that because Musk said so himself several times. His priority is providing the means of transportation - everything else is someone else's job.
The idea is to let the first Starships just sit on Mars and serve as part of the first colony.
There's a distinction to be made between cargo ships, tankers, and crew transport. The first uncrewed cargo flights have zero change of returning to Earth.
There's simply no infrastructure for generating fuel and refuelling yet. So unless you want to tell me that Musk is some sort of space wizard able to conjure methane fuel into the tanks of the first cargo ships that land on Mars, this is exactly what's going to happen.
Elon has already indicated that they are hoping to prove out Starship and then get NASA to approve them substituting it for those missions [1].
[1] Tweet seems to be deleted, so here's an embedded copy on NSF https://forum.nasaspaceflight.com/index.php?topic=47352.msg2...
A good example, here's their info on Starship prototypes: https://old.reddit.com/r/spacex/wiki/starships
https://forum.nasaspaceflight.com/index.php?topic=50522.0
EDIT: Wiki excluded!
https://www.youtube.com/channel/UCBNHHEoiSF8pcLgqLKVugOw/vid...
The article makes it seem like this is a good thing, I.e., there’s nothing inherently wrong with the design.
If true, it means it may be a procedural escape which can be every bit as dangerous. Some of the fallout from the CST-100 Boeing Starliner issues seem to point to process escapes as well. Makes me wonder if in the rush to get “boots on the moon” by 2024 we may be letting schedule get the best of us...
Boeing's hardware is effectively finalized, but the control software is still undergoing testing with the finalized (i.e., "frozen") hardware specs.
Not only was Boeing software supposed to be effectively finalized, but Boeing hardware also appears to have failed on that test. The thrusters experienced issues, and the antenna that were contractually required to provide end to end connectivity failed to do so. Moreover on the second most recent demonstration (pad abort) Boeing suffered a parachute failure.
The software issues were the more serious issues that grabbed more headlines. The capsule incorrectly grabbing a time value from the rocket. The control logic for whatever reason didn't understand something was wrong, but instead "proceeded as normal" as if it was further ahead in the mission. Finally there was some other error that was caught before it happened but was something related to the thruster control logic and could have caused the loss of the capsule during re-entry.
I appreciate both the thanks, and that you identified the source of the confusion. I also appreciate that you made it clear that you know what you're talking about and that I didn't really need to write such a long answer for you.
There is a second investigation currently ongoing, an "organizational root cause assessment".
I think what you mean is the "organizational safety assessment" (OSA) which will occur later this year.[1] Germane to my previous comment, this will focus more on the organizational culture that may have contributed to the failure (e.g., decision making in the presence of heavy schedule pressure).
https://blogs.nasa.gov/commercialcrew/category/boeing/
It's definitely a software problem per NASA and Boeing. But thanks for pretending that you know more than the people actually investigating.
> Be kind. Don't be snarky. Have curious conversation; don't cross-examine. Comments should get more thoughtful and substantive, not less, as a topic gets more divisive.
When disagreeing, please reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3."
Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith.
> The review team also is continuing its investigation of the intermittent space-to-ground forward link issue that impeded the flight control team’s ability to command and control the spacecraft. The team has identified the technical root cause as radiofrequency interference with the communications system. While the team has recommended specific hardware improvements already in work by the company, the full assessment and resulting recommendations will continue through March.
https://blogs.nasa.gov/commercialcrew/category/boeing/
and
> The NASA source said eight or more thrusters on the service module failed at one point and that one thruster never fired at all.
> In response to a question about thruster performance, Boeing provided the following statement to Ars: "After the anomaly, many of the elements of the propulsion system were overstressed, with some thrusters exceeding the planned number of burns for a service module mission. We took a few cautionary measures to make sure the propulsion system stayed healthy for the remainder of the mission, including re-pressurizing the manifold, recovering that manifold’s thrusters. Over the course of the mission we turned off 13 thrusters and turned all but one back on after verifying their health."
https://arstechnica.com/science/2020/01/nasa-and-boeing-are-...
Some days ago I read that they are currently trying to reduce material thickness to reduce weight. Maybe that contributed to the RUD.
It is a good thing that there is nothing inherently wrong with the design, just because that would mean more re-design work for SpaceX. A procedural issue (possibly this failure), a manufacturing defect (MK1 failure), a failure due to a temporary hack (SN1 failure), etc are all understandable errors and easily fixable at this point in time.
Starship isn't really being driven by the "boots on the moon by 2024" push by the government. Starship is a private venture being developed on their own schedule (and with their own money).
You’re right, bad choice of phrasing on my part to boil it down to a turn-of-phrase. What I’m really poking at is how schedule pressure affects decision making. SpaceX is well known to have very ambitious timelines. I am wondering how much that pressure affects process checks like configuration control
The top tank was covered in ice, indicating it was nearly full, while the bottom tank had no ice. That leaves three possibilities:
* The intention is the bottom tank was pressurized with gas:. There is no way that much gas at ~4 bar is safe. It would destroy all nearby buildings if it failed. That's a serious safety failing.
* The intention is the bottom tank was filled with LN2. Whoever was running the test didn't notice the tank didn't turn white, and didn't have any kind of level gauge or flow meter?
* The intention is the tank was unpressurized. The design could never withstand the top tank filled and the bottom tank unpressurized and empty. Whoever designed and ran the test didn't understand the basics of the design.
All of those 3 possible explanations point to more than a simple oversight, and IMO suggest the team working at 3am might possibly not really understand what they were doing.
Wat?
This is a launchpad, the safety assumption at play here is that this vehicle (and in fact a much larger vehicle with this vehicle as a second stage) can be filled with liquid methane and liquid oxygen, explode (with the energy of a small nuclear bomb, a few kilotons), and no one will get hurt. The area around the launchpad is kept clear during testing by local law enforcement for this very reason.
Hours before the test that failed, in a different test, the same test article was pressurized with room temperature nitrogen, likely up to ~7 bar (based on previous tests for which we were given figures).
For comparison, this is the same location that the following test was performed at, all the additional structures you see have been added recently by SpaceX as ground support equipment and they are undoubtedly perfectly aware and ok with the idea that they might be damaged in the case of a failure https://www.youtube.com/watch?v=bYb3bfA6_sQ
As for the rest of your post, yes, if you assume the people running the tests were incompetent then they would be incompetent. We can't fully rule out the idea of course, but it's far from the only option. Most likely the failure was not nearly so simple as "we forgot to pressurize a tank" - particularly considering that the bottom tank was clearly pressurized as it was venting.
The most reasonable suggestion for a proximate cause I've seen, for instance, is that the pressure release valve on the lower tank iced over and leaked until the pressure was no longer enough to support the above mass and it failed. This is supported by the evidence that we know they were having valve issues related to cryogenic liquids earlier in the day, and that the bottom tank was inexplicably venting for roughly 13 straight minutes before the failure. It doesn't quite line up with Elon's suggestion that this was possibly a test configuration error.
https://spacenews.com/spacex-reports-problem-in-crew-dragon-...
This is a silly loss, as it sounds like it was a test procedure error and not a valuable experimental failure. But even then, how much is lost? The rocket was built on-site in a tent, welding some cheap steel together. They've got more test articles being built. The engine is probably salvageable. This is a month or two set-back at worst.
And just like when our software fails in a silly way, I'll bet you there will be new automatic tests, procedures, etc, so that this kind of error can't happen again. They'll learn, iterate, and continue to move fast (and break things).
With nearly all of their competitors, any kind of failure like this would be millions of dollars lost, months or years of setback. This fundamental difference is why SpaceX is slowly coming to dominate the industry.
What has convinced you that this is optimal, or even "good", in either software or space technology?
>This fundamental difference is why SpaceX is slowly coming to dominate the industry.
They're making progress in the industry, sure. But dominating it? In what sense is this true?
In addition to capturing most of the commercial market that makes sense for them.
They are NASA most trusted partner, for both cargo to ISS. They will be the first private company to fly humans into orbit.
They are by far the cheapest, and even the rockets that are in development can't really compete against their lineup.
They are operating the largest launch vehicle on the planet.
They are universally considered to be the global leader in rocket engine design as both their Merlin and Raptor engines break many records and did stuff people thought was not possible.
They are in prime position to capture the single most valuable space market, LEO internet.
They are clearly the dominant player in the industry, every other player has admired it. Russian are out of commercial launch, Europeans are spending 2.5 billion to develop the Ariane 6 (who already has problems finding missions), ULA is dropping both their Atlas and Delta rocket to have a chance against SpaceX, and they have a very clear strategy to focus only on military and NASA launches.
Regarding the cost, the aerospace industry needs some scrappy young start ups. However, quality pedigree is a large chunk of the cost of flight and SpaceX has shown some gaps in the past. I think a big question is whether they can keep that scrappy startup up culture while also building processes robust enough to meet space flight quality standards
Say they have a check valve in their titanium system that fails. They could spend a lot of research dollars on the fundamental science of why that valve failed or just replace it with a burst disk. From an operational engineering point, they may feel like they solved the problem. But from a scientific standpoint we haven’t learned much other than “check valves are bad”.
It’s akin to the old joke about a mathematician, physicist, and engineer trying to find the volume of a cow. The mathematician tries to formulate a triple integral for the cow’s shape. The physicist drops the cow in water and measures the displacement. The engineer says, “let’s suppose the cow is a sphere...”
Point being, often in engineering they’re shooting for “good enough to get the job done” not to increase fundamental understanding
https://en.m.wikipedia.org/wiki/Spherical_cow
I won't call their approach a "start-up" or even unique one, especially considering they made a huge component of this (Raptor) in a pretty standard way. Building in a "tent" just means they don't need anything more than it being a tent right now, not that they won't ever need more.
For some reason, rocket engineering has that reputation of being a complete waterfall process. It isn't deserved and doesn't make sense. Rapid iterative development and quick discarding of failing prototypes in the early phase is as common in space industry as it is everywhere. It just doesn't usually make headlines and isn't that visible.
[1] https://www.amazon.com/Skunk-Works-Personal-Memoir-Lockheed/...
The industrial revolution made iterative design possible because it made the time and effort of producing something far cheaper.
This is a failure, and failure is perfectly ok working on something like this, but what drives people to volunteer their time to spin such things as positively as possible or even signal boost ? I can’t believe it’s purely investor driven. It’s almost as if people are finding personal purpose in sacrificing their time to this cause.
Many people like to dream big, but most of the time life is mundane. SpaceX's efforts are the embodiment of sci-fi fantasy with a chance of becoming real. Why not cheer?
Let people enjoy things.
To me, this looks like it was definitely a noteworthy fuckup, but in the big picture it doesn't seem to be a major setback -- nothing like the recent Boeing issues. That is, as long as a pattern of such events doesn't develop. SpaceX continues to push the industry forward in interesting and exciting ways.
To continue the arm-chair quarterbacking... I feel like a moderate decrease in process aversion would be a good response to this incident. Reduce the chance of similar future fuckups without slowing down too much.
To oversimplify things massively, "how careful do we want to be to avoid testing failures" is a parameter that SpaceX management gets to control. At the extreme careful end everything costs billions of dollars because you're paralyzed analyzing things and double/triple/quadruple/quintuple checking things. At the not at all careful end of things you keep building "complete rockets", pointing them at mars without any testing, and blowing them up. Obviously neither is rational.
They've settled on some parameters that are basically "don't worry that much about test failures, but keep the failures really cheap". They seem to be doing pretty well by doing that. They're experimenting with manufacturing techniques, hiring and training a workforce, building facilities. The prototypes that they are building keep failing, but that looks to be a relatively minor cost considering that the current primary goal is (according to them) to build out the manufacturing processes and make the design easy to manufacture.
Being slightly more careful would undoubtedly reduce the numbers of prototype failures, but would it actually be worth the cost of slowing other things down? Remember that the main cost of this program to SpaceX is engineering salaries, the faster it goes the cheaper it is.
So of course this test didn't go as planned, of course it would be better if it had worked, but would it have really been worth it to management to reduce the probability of this test failing? Like I said, maybe they aren't being careful enough, but I don't think we have any real evidence for that right now and I personally doubt it.
1. The risk is different when there are human lives on the line.
2. NASA is publicly funded and if they want to keep receiving funding, they can’t be blowing up the public’s money.
From my armchair perspective I think the risk adversity criticism is justified. NASA should take much more risk even in human spaceflight... in the early stages of platform development. First see if you can build the damn thing and build it repeatedly. Later, after that period, reduce risk and prove out and refine things to the human spaceflight standard. If you jump immediately to, "my God people are suppose to fly on that thing!" well, you're probably going to pay the Russians a lot of money for a fairly long time.
I do agree with you that the political management becomes a real issue. Politicians and bureaucrats are naturally disposed to a certain cravenness about such issues once they get to significant positions of power (senior execs of many companies do, too). But I think a charismatic leader that's out in front of the messaging could pull it off.
It is also absolutely embarrassing that we still haven't fully surveyed the moon for resources, dont know where all the ice is, etc. Lunar rovers should be numbering in the hundreds, maybe thousands.
They literally couldn't do it, and the admired that they basically can't develop it internally without it costing 10s of billions.
So that's why the used the Commercial Crew program and that worked out for them.
The SLS is a good example, 10+ years of building at a cost far above 20+ billion until the first launch.
Obviously, there's not much tolerance when human lives on the line. But you don't have to test with humans.
2. Dirty little secret: even when NASA uses “startups” like SpaceX to launch and things go awry, the taxpayers are still on the hook for the majority of the loss because the Agency is self-insured
That's just for robotic/satellite missions. It's ten times worse when a manned mission ends fatally.
NASA has its Artemis program which is doing exactly this, returning to the Moon with a goal of a longer-term lunar base.
And they just held a period of application (it closed a few days ago) for the next generation of Artemis astronauts.
Artemis : https://www.nasa.gov/specials/artemis/
So another new moon plan. Another Shuttle derived launcher.
Not surprising that nobody is getting very exited with Artemis.
The SLS and Orion can only be described as dead-weights on NASA since they are so incredibly expensive. SLS has been in development for over 10 years at more then 2 billion per year and it will not launch until 2022 hopefully.
They've been designing Orion for a couple decades now, and the SLS is a shuttle-derived expendable launcher where what we see as key innovations are mounting 25% more SSMEs (still expendable) on the bottom of the fuel tank, extending the SRBs by a couple segments and putting the payload on top of the main tank.
Ares was interesting and we learned, with Ares I, that we shouldn't put humans on top of an SRB. There would probably be cheaper ways to find that out, but at least they tried something different.
I don't disagree but I also don't think it's always as straightforward as probability theory would make us believe. One can imagine situations where a certain amount of QA leads to a certain semi-predictable amount of failures that allows for calculated risks to be taken, but one can also imagine situations where paying 2X leads to better solutions and processes and not paying 2X leads to failure on both the first and second launches.
I'm sorry, but this argument is also mathematically illiterate. Probability only works this way if the events are uncorrelated, but it should be pretty clear they're not.
If the first rocket has a 20% chance to fail it is likely that the second one will also fail due to similar negligence.
Rockets aren't mass manufactured and their fabrication is barely automated. Nobody except SpaceX keeps their tooling online long enough to have correlating failures except for actual design issues which are fixed in later launches. The correlated failure comes from the actual mission hardware but launch schedules are so spread out that a second mission has plenty of time to fix issues discovered in the first attempt.
If rockets, which are made one at a time and launch maybe a few times a year, had correlating failures like you said, no insurance company would cover launches.
* Fail Fast. * Red. Green. Refactor.
I do have some hints that it's probably more or less where SpaceX wants it to be, some hints that they have chosen this parameter rationally, and some faith that they aren't stupid. For instance, Elon often discusses future tests failing as acceptable, they've declined to put in place more stringent QA efforts, they're still mostly building the hardware exposed to the elements, Elon has discussed that currently their approach is building a lot of starships to iterate on the manufacturing methods and to help them make an easy to manufacture design - he has signaled fairly clearly via his tweets that actually flying the "ships" is secondary, and so on.
My prior is that the parameter is as likely to be set as too high as too low. The testing failure at this stage in development is weak evidence that it is too low, the rest of the information I've just talked about is more convincing evidence that it is an appropriate value or too high, hence I think the post I responded to is "probably wrong".
An example of where I think the level of caution was probably too low (or there were other issues, but let's pretend for a second that this is the only knob management has, as I said it is an over simplification) - the recent CST 100 orbital flight test failure. That test was intended to demonstrate they were ready to fly humans, instead they suffered a series of serious failures, failing that sort of test is pretty strong evidence that they weren't careful enough. They've indicated this type of failure was unexpected and that it will cost the company slightly under half a billion dollars, which is also pretty strong evidence that they weren't careful enough. Moreover I know that they neglected to use procedures such as end-to-end software simulation that would have reduced this risk, and that they were in fact contractually required to perform. Meanwhile I don't have any real evidence that they were careful enough with regards to that test. So starting with the same prior as above, I quickly fall to the "they probably weren't careful enough side".
> Or was at an optimal point for SpaceX in general
The optimal point is definitely not a global value for all of SpaceX. Human spaceflight programs have obviously set it very high instead of very low. As they progress from experimenting with manufacturing to flying payloads it obviously gets turned up. Etc.
After learning from all those failures they now have reusable first stage rockets and have moved the entire space industry forward.
Same thing happening here.
https://www.spacex.com/missions
My question is "What is their net income and their current cash assets? What does their runway look like?"
Even in the best case scenario SN3 was planned to after this test make a static fire and then a hope to about 150m.
SN4 is already being assembled and will go threw the same tests.
Not their overall financial situation, but looks like investor appetite has been pretty strong.
https://youtu.be/kkqgkccWKYI?t=130
While i've happened to be wrong anytime i've questioned Musk's decisions/approach, looking at that and the previous pressure test failure and how the body crumples, i wonder whether the way the Starship body is built isn't faulty by design - like all this steel plates horizontally oriented.
Of course, fast and cheap is the only way to get to build hundreds of Starship, the Liberty ship of our time, yet the current structural design of the body seems to require re-work.
Von Braun's V-2 team built 700 rockets before hitting a target.
They didn't loose any Starship, they are still testing tank weldings with various pressure tests, which had a problem from the beginning. Then they switched the manufacturer and the tank welding might still not be good enough. Or as announced it was just a bad test.
Such iterations are totally expected, it's the third now (SN3, serial number 3), but no engine tests yet in integration with the spectacular new Raptor engine.