I've been slowly refining a pitch deck over the past couple years and the feedback from reviews and test pitches has strongly reinforced for me just how important it is for slides to be short and laid out precisely.
You want the most important information in the right places, communicated with as few words as possible, using the most accurate words possible.
You want the key takeaways to be the things that people are most likely to remember from each slide.
You want to minimize distractions and try not to pollute slides with a bunch of vaguely related stuff. A crowded slide risks communicating nothing.
It's a real dramatic change compared to how I am used to using powerpoint for technical audiences or when I had to make presentations during school.
This article (as it makes clear) owes it's analysis at least largely to what Tufte has written about the Challenger disaster (1986) and Columbia Disaster (2003). He wrote about the Columbia one more fully in the second edition of The Cognitive Style of Powerpoint.
Given that the link in the article to his report on his website is now broken, people might be interested in teh few page grabs that he has included in the "comments" on his site here[0].
See also the article that he has re-posted under the "comments" section on his page on the matter[1].
The full report (2003 edition, low-res) is available on ResearchGate. It appears to be a lawful copy, uploaded by the author himself. Fascinating reading, indeed.
The problem with sending up another shuttle is they hadn't actually fixed the underlying foam problem. If there was another foam strike on the rescue shuttle, now they have two craft in orbit with crew on that can't re-enter
My dad headed up the redesign effort on the Lockheed Martin side to remove the foam PAL ramps (where the chunk of foam that broke off and hit the orbiter came from) from the external tank, as part of return-to-flight after the Columbia disaster. At the time he was the last one left at the company from when they had previously investigated removing those ramps from the design. He told me how he went from basically working on this project off in a corner on his own, to suddenly having millions of dollars in funding and flying all over for wind tunnel tests when it became clear to NASA that return-to-flight couldn't happen without removing the ramps.
I don't think his name has ever come up in all the histories of this—some Lockheed policy about not letting their employees be publicly credited in papers—but he's got an array of internal awards from this time around his desk at home (he's now retired). I've always been proud of him for this.
As an engineer, that slide looks completely reasonable to me. Its purpose was to communicate technical info, which it did adequately. Keep in mind that the subject matter is highly technical, given that we're literally talking about the Space Shuttle, and more than a high school level of reading comprehension is heavily implied. If the NASA personnel weren't competent enough to review technical data without a pithy summary, that's on them.
The slide shown here is completely horrendous, even by 2003 standard. I've worked in a field as technical as this one for a long time, and this slide would not pass a review, even with people familiar with the content.
While it does not wash the responsibility of the executives, the engineers have also the responsibility to be clear in their communication
I wish this was satire. The slide is full of spelling mistakes and all kinds of horrible communication. Even when I know it documents an issue that killed people I still struggle to read it. Only the bottom part hints at anything relevant.
Another example of "guy can't spell" => "he's retarded". If you can't get everything right, you probably got lots of things wrong.
It also shows why software engineers are superior engineers. The history of software engineering has some single-digit kills, at most 20 in total. Meanwhile, shuttle engineers are supposed to be the best in the aerospace business and they've lost some 13 or so? All aerospace would be put in the thousands.
An old saying is "Any fool can build a bridge. It takes an engineer to build a bridge that barely stands". But these are fools who built bridges that didn't stand.
One day, I will teach them real engineering: It is Rails backend with React frontend. Zero kills. Life above all.
> One day, I will teach them real engineering: It is Rails backend with React frontend. Zero kills. Life above all.
Buddy, if you think that is "real engineering", you've got a lot of things to learn.
Even in software engineering, rails and react aren't really engineering. Real engineering includes calculations -- data structures, algorithms, data structures, and user interfaces designed to work with people who are deaf or blind. You wanna tell me that rails is performant? Have you done calculations? You wanna tell me react's algorithms are well-defined in memory and time? Have you ensured that your "software" is usable by people with disabilities?
This was an interesting article but it doesn't really provide solutions. I watched a few tech talks teaching a new API. Most slides were split, left side bullet poitns, right side either code or an image. As I was watching I was thinking "isn't this supposed to be almost the worst style"? but I was also thinking "I can't think of any way to do this better". It's an API. It requires examples. And it requires something describing what to concentrate on, what the example or image is showing.
I've been the plenty of great talks with just images, no words. But they fit the type of talk. I'm not sure an API talk would be better without bullet points. If you know of some to reference, please post links.
I found it surprising that the slide in the article uses Calibri, a typeface that wasn’t publicly available at the time. The original discussion confirms that the slide in the article is a recreation of the original one:
> The slide in the article has the same text, but is a recreation of the original (The Calibri typeface used wasn't part of PowerPoint until 2007).
> The original slide can be seen in the full report linked in the article:
I don't see how this has anything to do with PowerPoint. There wasn't clear communication; the medium was completely incidental to that. They could have been writing on a chalkboard and had a communication failure, does that mean that chalkboards should be blamed in that case?
PowerPoint gets used because it requires less effort from the audience. They sit back and zone out like couch potatoes. Scrap the PowerPoint and throw the technical report at the managers. Any of them who complain or otherwise don't read it are incompetent and should be fired on the spot.
I think it's a lot harder to have this particular type of communications failure if you're writing on a chalkboard. Imagine trying to write out that whole slide, it would take forever. If you really did have to present that information on a chalkboard, you'd be significantly more likely to write something along the lines of:
"We checked the test data: possible to damage tiles significantly"
"Foam that hit wing was way bigger than the tests"
Obviously you can miscommunicate via any medium, but I think the author's point here (which I agree with) is some mediums lend themselves to specific types of miscommunication.
This isn't a failure of PowerPoint. I work for NASA and we still use it all the time, and I'll assure anyone that the communication errors are rife regardless of what medium we're working in. The issue is differences in the way that in-the-weeds engineers and managers interpret technical information, which is alluded to in the article but the author still focuses on the bullets and the PowerPoint, as if rewriting similar facts in a technical paper would change everything.
My own colleagues fall victim to this all the time (luckily I do not work in any capacity where someone's life is directly on the line as a result of my work.) Recently, a colleague won an award for helping managers make a decision about a mission parameter, but he was confused because they chose a parameter value he didn't like. His problem is that, like many engineers, he thought that providing the technical context he discovered that led him to his conclusion was as effective as presenting his conclusion. It never is; if you want to be heard by managers, and really understood even by your colleagues, you have to say things up front that come across as overly simple, controversial, and poorly-founded, and then you can reveal your analyses as people question you.
I've seen this over and over again, and I'm starting to think it's a personality trait. Engineers are gossiping among themselves, saying "X will never work". They get to the meeting with the managers and present "30 different analyses showing X is marginally less effective than Y and Z" instead of just throwing up a slide that says "X IS STUPID AND WE SHOULDN'T DO IT." Luckily for me, I'm not a very good engineer, so when I'm along for the ride I generally translate well into Managerese.
- bad communication possible in any medium
- pptx in NASA even today!
- issue is managers/SMEs communication differences
- issues with technical papers
- long
- boring
- vs word, excel, pdf...
(Next slide please)
Manager/SME Differences
- context vs conclusion
- tell a compelling story
- but give away the ending FIRST
- inherent personality differences
- motivations/incentives/mindsets
(Next slide)
Learning from disasters
- medium guides message and messenger
- blame tool - binary choice?
- presentation aide vs distributed technical artifact
I feel that. I looked at the powerpoint from the post and I cringed because it looks just like the kind of thing I could have written, with half finished thoughts all over the place masquerading as something a decider could use...
I think the problem is that most people, especially non-engineers, are over-selling and over-promising all the time. Being honest about risks, issues, and short-comings makes a project or product look bad in comparison.
The most feasible way to get X done is saying "X is a great option, the risks are managable, and it's fairly quick". Then, it will unexpectedly take a bit longer, plus some unforeseen trade-offs will need to be made.
Agree and was going to say the same thing. Messages need to be created for a specific audience.
When I'm sending an email to non-tech mgrs that has a bunch of tech details like that slide, I typically separate more detailed stuff from the conceptual message:
Summary:
System performance is not good enough for go live.
A 2008 episode of the PBS NOVA program covers the Columbia disaster. It does not get into the focus of the article posted here, but it does well covering how poorly the situation was handled (along with other things like the broad history of the Shuttle program).
Everyone should note that the slides in this blog post are fabricated, and the author of this post misrepresents Tufte and the conclusions of the review board—subtly pointing people in a direction that's the exact opposite of the real takeaways from the investigation. It's almost insidious.
The first time I came across this blog post a few years ago, I found it enraging, and I still do. As I said I said in my personal notes at the time, "The choice to use this fabricated material for this article and the false claims in it—claims not actually confirmed by the sources that it cites—amounts to what would be considered serious misconduct elsewhere." But there's no real accountability here, of course, because at the end of the day it's just some jerk with a blog serving up distorted pop science insights to aspiring entrepreneurs.
No, that's not how the physics works. The foam is moving at the same velocity as the shuttle when it breaks off, and had a short time to accelerate(decelerate) before hitting the shuttle.
> Why, given that the foam strike had occurred at a force massively out of test conditions had NASA proceeded with re-entry?
What was the alternative?
Columbia could not have made it to ISS.
Columbia could not have repaired the damage in orbit.
Columbia could not have lasted, after two weeks in space, long enough to launch a rescue mission.
I know the "In Flight Options Assessment" said they could launch at an accelerated pace but the assessment assumes that it's ok to launch another vehicle with the same problem, no fix, and no completed analysis of the cause.
Yeah, they suspected the external tank bipod foam, but WHY did the foam come off? Was it a fluke? Had some unknown factor not present in previous external tank bipod foam applications but now present in all external tank bipod foam applications manifested?
>Two major assumptions, apart from the already stated assumption that the damage had to be visible, have to be recognized – the first is that there were no problems during the preparation and rollout of Atlantis, and the second is the question of whether NASA and the government would have deemed it acceptable to launch Atlantis with exposure to the same events that had damaged Columbia. At this point, at least two of the last three flights (STS-112 and STS-107) had bipod ramp foam problems, and the flight in-between these two, STS-113, was a night launch without adequate imaging of the External Tank during ascent.
This is from a pre-flight safety report for STS-113
>“More than 100 External Tanks have flown with only 3 documented instances of significant foam loss on a bipod ramp”
STS-1 through STS-111, April 1981 - June 2002: three "significant" bipod foam losses
STS-112, October 2002: significant foam loss
STS-113, November 2002: night time, but they saw 112 and went "oh shit" and wrote a report
STS-107, January 2003: yet another, fatal, significant foam loss
If two of the last three flights had foam problems and the one that didn't only didn't because you couldn't see if it did, and over 100 of the preceding flights only had three, you don't risk four more lives.
For once I will defend PowerPoint: 1/ the slide is atrocious, even by 2003 standard 2/ you don't decide on the life or death based on a crappy slide you don't understand.
The slide is so bad it might be intentional. The intent would be to be reassuring enough that no one will point at Boeing and say "Boeing technical failure strands astronauts in space", while providing just enough information that in the event (at the time of writing the slide considered very unlikely) things go bad, they can say that nothing was withheld.
48 comments
[ 4.7 ms ] story [ 71.8 ms ] threadYou want the most important information in the right places, communicated with as few words as possible, using the most accurate words possible.
You want the key takeaways to be the things that people are most likely to remember from each slide.
You want to minimize distractions and try not to pollute slides with a bunch of vaguely related stuff. A crowded slide risks communicating nothing.
It's a real dramatic change compared to how I am used to using powerpoint for technical audiences or when I had to make presentations during school.
Given that the link in the article to his report on his website is now broken, people might be interested in teh few page grabs that he has included in the "comments" on his site here[0].
See also the article that he has re-posted under the "comments" section on his page on the matter[1].
[0]: https://www.edwardtufte.com/notebook/new-edition-of-the-cogn... [1]: https://www.edwardtufte.com/notebook/the-columbia-evidence/
The full report (2003 edition, low-res) is available on ResearchGate. It appears to be a lawful copy, uploaded by the author himself. Fascinating reading, indeed.
https://www.researchgate.net/publication/208575160_The_Cogni...
If they did a spacewalk and found the damage, what were their options?
I don't think his name has ever come up in all the histories of this—some Lockheed policy about not letting their employees be publicly credited in papers—but he's got an array of internal awards from this time around his desk at home (he's now retired). I've always been proud of him for this.
To folks out there: do the important work, not the glamorous work, and you'll not only sleep well, but you might actually matter as well.
While it does not wash the responsibility of the executives, the engineers have also the responsibility to be clear in their communication
It also shows why software engineers are superior engineers. The history of software engineering has some single-digit kills, at most 20 in total. Meanwhile, shuttle engineers are supposed to be the best in the aerospace business and they've lost some 13 or so? All aerospace would be put in the thousands.
An old saying is "Any fool can build a bridge. It takes an engineer to build a bridge that barely stands". But these are fools who built bridges that didn't stand.
One day, I will teach them real engineering: It is Rails backend with React frontend. Zero kills. Life above all.
BTW: Here's is the original content with Tufte review https://www.edwardtufte.com/wp-content/uploads/bboard/images...
Buddy, if you think that is "real engineering", you've got a lot of things to learn.
Even in software engineering, rails and react aren't really engineering. Real engineering includes calculations -- data structures, algorithms, data structures, and user interfaces designed to work with people who are deaf or blind. You wanna tell me that rails is performant? Have you done calculations? You wanna tell me react's algorithms are well-defined in memory and time? Have you ensured that your "software" is usable by people with disabilities?
I've been the plenty of great talks with just images, no words. But they fit the type of talk. I'm not sure an API talk would be better without bullet points. If you know of some to reference, please post links.
I found it surprising that the slide in the article uses Calibri, a typeface that wasn’t publicly available at the time. The original discussion confirms that the slide in the article is a recreation of the original one:
> The slide in the article has the same text, but is a recreation of the original (The Calibri typeface used wasn't part of PowerPoint until 2007).
> The original slide can be seen in the full report linked in the article:
> https://www.edwardtufte.com
PowerPoint gets used because it requires less effort from the audience. They sit back and zone out like couch potatoes. Scrap the PowerPoint and throw the technical report at the managers. Any of them who complain or otherwise don't read it are incompetent and should be fired on the spot.
"We checked the test data: possible to damage tiles significantly" "Foam that hit wing was way bigger than the tests"
Obviously you can miscommunicate via any medium, but I think the author's point here (which I agree with) is some mediums lend themselves to specific types of miscommunication.
Some previous discussions:
2022: https://news.ycombinator.com/item?id=30615114
2019: https://news.ycombinator.com/item?id=19668161
My own colleagues fall victim to this all the time (luckily I do not work in any capacity where someone's life is directly on the line as a result of my work.) Recently, a colleague won an award for helping managers make a decision about a mission parameter, but he was confused because they chose a parameter value he didn't like. His problem is that, like many engineers, he thought that providing the technical context he discovered that led him to his conclusion was as effective as presenting his conclusion. It never is; if you want to be heard by managers, and really understood even by your colleagues, you have to say things up front that come across as overly simple, controversial, and poorly-founded, and then you can reveal your analyses as people question you.
I've seen this over and over again, and I'm starting to think it's a personality trait. Engineers are gossiping among themselves, saying "X will never work". They get to the meeting with the managers and present "30 different analyses showing X is marginally less effective than Y and Z" instead of just throwing up a slide that says "X IS STUPID AND WE SHOULDN'T DO IT." Luckily for me, I'm not a very good engineer, so when I'm along for the ride I generally translate well into Managerese.
Manager/SME Differences
(Next slide)Learning from disasters
(Next slide)Questions?
The most feasible way to get X done is saying "X is a great option, the risks are managable, and it's fairly quick". Then, it will unexpectedly take a bit longer, plus some unforeseen trade-offs will need to be made.
Agree and was going to say the same thing. Messages need to be created for a specific audience.
When I'm sending an email to non-tech mgrs that has a bunch of tech details like that slide, I typically separate more detailed stuff from the conceptual message:
Summary:
System performance is not good enough for go live.
Working on a few possible solutions.
Details (for those interested):
System x is connected to ...blah blah blah...
https://www.youtube.com/watch?v=6t48bc2dyzo
The first time I came across this blog post a few years ago, I found it enraging, and I still do. As I said I said in my personal notes at the time, "The choice to use this fabricated material for this article and the false claims in it—claims not actually confirmed by the sources that it cites—amounts to what would be considered serious misconduct elsewhere." But there's no real accountability here, of course, because at the end of the day it's just some jerk with a blog serving up distorted pop science insights to aspiring entrepreneurs.
https://justin.searls.co/posts/sprinkling-self-doubt-on-chat...
No, that's not how the physics works. The foam is moving at the same velocity as the shuttle when it breaks off, and had a short time to accelerate(decelerate) before hitting the shuttle.
He has a legendary hatred of PowerPoint.
https://www.edwardtufte.com/product/the-cognitive-style-of-p...
Death by PowerPoint: the slide that killed seven people (2019) - https://news.ycombinator.com/item?id=30615114 - March 2022 (197 comments)
Death by PowerPoint: The slide that killed seven people - https://news.ycombinator.com/item?id=19668161 - April 2019 (127 comments)
What was the alternative?
Columbia could not have made it to ISS.
Columbia could not have repaired the damage in orbit.
Columbia could not have lasted, after two weeks in space, long enough to launch a rescue mission.
I know the "In Flight Options Assessment" said they could launch at an accelerated pace but the assessment assumes that it's ok to launch another vehicle with the same problem, no fix, and no completed analysis of the cause.
Yeah, they suspected the external tank bipod foam, but WHY did the foam come off? Was it a fluke? Had some unknown factor not present in previous external tank bipod foam applications but now present in all external tank bipod foam applications manifested?
>Two major assumptions, apart from the already stated assumption that the damage had to be visible, have to be recognized – the first is that there were no problems during the preparation and rollout of Atlantis, and the second is the question of whether NASA and the government would have deemed it acceptable to launch Atlantis with exposure to the same events that had damaged Columbia. At this point, at least two of the last three flights (STS-112 and STS-107) had bipod ramp foam problems, and the flight in-between these two, STS-113, was a night launch without adequate imaging of the External Tank during ascent.
https://govinfo.library.unt.edu/caib/news/report/pdf/vol2/pa... (page 397)
That's not a valid assumption.
This is from a pre-flight safety report for STS-113
>“More than 100 External Tanks have flown with only 3 documented instances of significant foam loss on a bipod ramp”
STS-1 through STS-111, April 1981 - June 2002: three "significant" bipod foam losses
STS-112, October 2002: significant foam loss
STS-113, November 2002: night time, but they saw 112 and went "oh shit" and wrote a report
STS-107, January 2003: yet another, fatal, significant foam loss
If two of the last three flights had foam problems and the one that didn't only didn't because you couldn't see if it did, and over 100 of the preceding flights only had three, you don't risk four more lives.
You start designing a memorial at Arlington.
> Think about your message. Don’t let that message be lost
That was the important bit of the article. Regardless the medium, ensuring you deliver the message you want should always be the driving point.
Also: Yes to less slides and more Position Papers (but keep em brief please :-D)
https://norvig.com/Gettysburg/
This problem seems mostly solved now.