The fact that the NHTSA formula gives a score of 5.4 doesn't mean that NHTSA recognizes the car as superior to those with a technical score of 5.3. To put it differently, if a class gives a grade of A,B,C,D,or F, having a numeric score of 99 versus 98 is somewhat irrelevant if the ultimate grade is A in both cases.
If they really did want to make a distinction, they'd introduce a new scoring metric that does allow for those differences
If you were in school and had a bet with your roommate about final grades, you wouldn't call off the bet as a draw if you both got A's but you got a higher numeric score than he.
If you had a bet with your roommate about final grades, you would look at the transcript at the end of the semester. And that transcript most likely says A for both of you, so it would be a draw.
I'd jump on the web and see the grades posted by the professor on the course website actually. I always got numeric grades at the end of the semester unless I waited long enough that the course website was nuked, or unless that professor was lazy.
You could also just go ask the professor at their office for your grade, but using their site is easier.
For a bet like that, it doesn't matter if the score difference results from random chance and not any underlying difference in skill or ability. The NHTSA is in the business of assessing car safety though, and it makes little sense for tnem to rank cars within a category if they don't think the difference is actually meaningful.
But having a numeric score of 59 versus 9 is relevant, despite them both being F. Likewise, 89 versus 90. NHTSA can do whatever they want; I can say it is just a silly metric that is about as good as gaming reviews, as another comment points out. A 5.4 score is still relevant; why would they throw away precision?
The NHTSA probably doesn't think that a safety score of 5.3 versus 5.4 has any sort of meaning. If the set of accidents under which you would survive unscathed in a car with score 5.4 but would be injured in a car with score 5.3 has measure 0 (or negligible), then there is no real difference in terms of safety (even if the scores are different)
OTOH, the whole rating system is about taking real tests and progressively throwing away levels of precision. Like combining front and rear and side impact into a single score.
The Tesla blog post makes it abundantly clear that the 5.4 is the report the manufacturer receives (not the issued rating which is capped at 5). Why does it stop at 5 if you can still die in those cars?
I don't think it does make that clear. It says "safety levels better than 5 stars are captured in the overall Vehicle Safety Score (VSS) provided to manufacturers" but the VSS value for the Tesla was 0.43 according to the blog post. So where did the 5.4 come from? Extrapolation?
You're right - they didn't make it abundantly clear that the VSS report was 0.43. Regardless, the conversion they made isn't insane when you look at the other VS scores. Plus - watch the videos of other 5 star rated vehicle versus the Model S. It's like the comparison between a 1980s car and one from 2013.
All I meant was that if 5 stars isn't perfect why is that the scale they're using? Obviously the range of what a 5 star rating represents could mean the difference between life and death in a serious collision.
It says as much in the official Tesla press release:
NHTSA does not publish a star rating above 5, however safety levels better than 5 stars are captured in the overall Vehicle Safety Score (VSS) provided to manufacturers, where the Model S achieved a new combined record of 5.4 stars.
You can't blame Tesla for taking that number and running with it, given that (1) modeling car safety from two seaters to SUVs on a range from 1 to 5 stars is ridiculous and (2) the star systems suffer from the same problems as contemporary game and movie reviews: 99% of all evaluated products land between 3 and 5 stars, where 3 then becomes the new horrible, compressing the already low entropy of a star value further.
Nothing wrong with the 5 star rating system as far as it goes... the marketing 'usefulness' of a measure is not the be all and end all, in that regards it is very different from video game and movie reviews.
This makes me wonder, do they adjust the rating system over time, and if so, are they doing it frequently enough? You can buy five star cars today, and I assume you could buy 5-star cars 15 years ago, but 5 star cars from 15 years ago wouldn't get rated as highly today, right?
Maybe they need to be more aggressive with their adjustment so that the minimum expected really is 1 star.
The VSS is not a star score though. It looks like Tesla got a 0.43 VSS. Tesla was the one who converted that to a 5.4, by making up a methodology, and their wording on that is definitely a bit unclear.
P-values aren't probabilities you should multiply or add.
At best you're computing odds according to the null hypothesis (probably not what you indented). At worst it's as meaningless as adding meters to seconds.
> (2) the star systems suffer from the same problems as contemporary game and movie reviews: 99% of all evaluated products land between 3 and 5 stars, where 3 then becomes the new horrible, compressing the already low entropy of a star value further.
That's not a bad thing. We wanted safer cars. We started measuring safety, and publishing the measurements. As a result, safety improved. Working As Intended.
The threshold for each star is increased each year, based on the available safety features and safety performance of cars. A car that received 5 stars under the 2000 rating system would receive only 2 stars today because it lacks some of the now-required safety features.
3 is the new horrible simply because that is the lowest that a commercially viable car can go--Chevy and Subaru have released sub-3 cars in the past and suffered for it with poor sales. They and others learned that consumers wouldn't buy cars that were rated below-average safety-wise and have vastly increased R&D spending on safety features. That's a good thing for consumers.
For a truly excellent product, the distribution should approximate a power law graph:
XXXXXXXXXXXXXXXX
XXXXXX
XX
X
X
If you see something like these:
XXXXXXX
X
X
X
X
X
X
X
X
XXXXX
XXXXXXXXXXXXXXXX
XXXXXX
XXX
X
XXXXXXXX
Then ask yourself if the sample size makes this plausible, in which case the star rating isn't worth much, because there isn't enough data. Ask yourself if it's possible that someone has been gaming the system for or against.
The uppercase gamma-shaped graph is either someone gaming the system or it's a power law graph hidden by aliasing. (In which case, the thing is of godlike quality.)
We should do away with star ratings, and instead just present an aggregate "rating." Genuine customer ratings are valuable, but are almost immediately gamed by cheaters. If I were Apple or Amazon, I would build stealth captchas and "honeypot" traps for ratings manipulation, and figure those into the rating. Be sure to separate detection from consequences by a random time interval of at least a few days. This makes the "development cost" for the cheaters much higher.
The problem with star ratings is obviously what PG refers to as "schlepping". Is anyone tackling this already?
EDIT: I would surmise that AirBNB is somewhat tackling this problem already.
I take it from the article, nothing but a statement from the safety agency asking "car manufactures" (not even mentioning Tesla) to not misrepresent their data to the public.
I wonder how many other cars got more than five stars by Tesla's methodology? From the graph, it looks like some have very close ratings.
And clearly, the goal here is to stop every manufacturer from just coming up with their own rules for interpreting the score that give them a different rating from the one that the NHTSA gives. That's definitely worthwhile.
"The agency has dominion over any part of a vehicle sold in the United States, from the headlamps to the owner’s manual, but can’t regulate the speech of a corporation or its employees."
True, but the agency gives legal cover. When there is a question in court as to whether the safety systems performed "as advertised", the NHTSA is free to say that Tesla didn't follow the advertising guidelines, and therefore cannot comment on the performance of the safety system.
Drug companies can also advertise drugs outside the FDA guidelines, and would win a court challenge on 1st Amendment grounds. None do it because it would expose them to unbounded liability.
>Drug companies can also advertise drugs outside the FDA guidelines, and would win a court challenge on 1st Amendment grounds. None do it because it would expose them to unbounded liability.
No, no, no. This is utterly and confirmably false. It is illegal under the Federal Food, Drug and Cosmetic Act to advertise a prescription drug for a use that is not FDA approved, and it is within the FDA's mandate to recommend investigation to the Department of Justice in such circumstances. The FTC is authorized to oversee advertising for over-the-counter drugs under the Wheeler-Lea Act. As a recent example, Pfizer agreed to settle with the DoJ earlier this summer for half a billion to end an investigation brought under these very circumstances[1].
" Caronia argues that he was
convicted for his speech -- for promoting an FDA-approved
drug for off-label use -- in violation of his right of free
speech under the First Amendment. We agree. Accordingly,
-3- we vacate the judgment of conviction and remand the case to the district court. "
It is uncertain whether marketing off label is protected by the 1st amendment. The courts don't all agree.
The one case you cite is very fresh precedent in a single Circuit, and it's highly controversial. No other Federal Circuit has applied First Amendment standards to FDA misbranding regulations before. Even within the Second Circuit (three states), is it still illegal under law to misbrand a drug in marketing materials. The decision in Cariona hinges on the question of whether "misbranding" and "promotion of off-label use" are different narrowly defined legal scenarios. The Second Circuit was concerned that, in the latter scenario, doctors may be legally deterred from promoting off-label use, which is a common practice in medicine and research. You should not take away from this case that drug marketing is now a First Amendment free-for-all. Yes, I am awfully certain.
This is an interesting case, however its important to point out that the legal debate around this is ongoing. This ruling only applies to the 2nd Circuit at this point. It's looking like a Supreme Court case will be necessary to ultimately decided this issue, given the murkiness around it.
Until then, however, off-label drug marketing is not something companies get away with without a substantial legal battle (often not worth it), and the FDA has so far collected billions of dollars in settlements and fines through successful off-label drug marketing suits.
I've been out of the industry for a while but I spent 5 years working on regulatory projects in the MD&D sector.
We never advertised off-label. Whether it's legal or not, I do recall you'd get slapped by the FDA for lack of controls. We had many cases where doctors would use a product off-label and it'd come back to us in the form of a complaint or defect (product comes back as defective, we investigate and find out the doctor was using the product for an off-label procedure it wasn't designed for and would fail, hence the 'defect'). At which point we'd have to proactively train doctors to not do this and other corrective action measures would have to be put in place. If upon investigating we find this would be a good use of the product, we'd have to go through another round of FDA fun (sometimes clinicals and how much testing depended on some factors, definitely rounds of internal testing and implement new QA tests and procedures to ensure the product can work for the new use and have proper fault tolerances as well as new labeling and training, etc).
Now, do companies try to skirt around this? Sure. Part of my job at the time was to make sure we didn't. FDA Warning letters are no joke in this industry, it can literally put you out of business.
This is an interesting 1st Amendment discussion (relating to the article...I am not touching the separate FDA/drug issue).
The most analogous case that comes to mind, is the recent California small claims case against Honda.
Facts: According to the EPA (who independently tests vehicles fuel efficiency) the Prius gets an average of 51 MPG. The Plaintiff claimed she only got an average of approximately 30 MPG. The Plaintiff won the lawsuit against Honda for fraudulent misrepresentation winning $10K the small claims limit in the jurisdiction. However, one of the most interesting arguments (and legitimate) was that while Honda does advertise the 51 MPG, that Honda could not be fraudulently misrepresenting the MPG, because Honda had no control over the testing or findings handled by the EPA, and going further Honda does not have a right (1st amendment) to contradict the "EPA sticker". Ergo according to the argument Honda should not be liable, rather the EPA - nevertheless the Court found Honda liable for fraudulent misrepresentation.
Assuming this law is on point and controlling, for example, if the Tesla were to under perform in a real crash and Tesla was sued, Tesla may be liable despite having no control over the NHTSA test or findings. Going further, if the dicta from the Honda case is also on point, if Tesla's internal crash tests contradict NHTSA's ratings, Tesla would not have a 1st Amendment right to contradict the NHTSA findings publicly.
Note: That SCOTUS makes a distinction between "commercial speech" and other forms of speech, and the government has a lot more power to regulate commercial speech than say political speech.
Nice catch...the suit was against Honda. I must have subconsciously invoked Prius as a matter of bias (not in a negative capacity, simply as the dominate hybrid in my mind subjectively)
It was easy to see that coming once the media story took off the way it did. But this just generates more coverage of Tesla's results.
I doubt that anyone at Tesla is feeling the least bit chastised. Musk is certainly not the type of person who is afraid to generate a little controversy.
HN isn't just about software/engineering, it's also about entrepreneurship.
Tesla is a startup with cool technology. They're taking on a market where startups never succeed and are navigating significant challenges regarding new technology, consumer bias, competitors, regulatory issues, business model, press relations, and more. Some of us find it fascinating and enlightening to watch them take this journey.
The Tesla Motors post that heralded this news (https://news.ycombinator.com/item?id=6240862) was on the front page for nearly a full day and garned 794 upvotes. HN readers are clearly intrigued by TM
I really like Tesla and want them to succeed, but they've got some weird ideas about good PR. The cost calculator that factored in the cost of pump time at your billable rate; the over-the-top posture against the NY Times reviewer who discovered that the battery heater could draw enough power to affect range; and now this, where they made up their own methodology to convert VSS into stars and found themselves to be the best. (I'm not even considering the weird stuff like Elon Musk offering publicly to consult on Boeing's batteries, although I think that points to the source.) Aren't the "best car ever made" reviews enough?
I think that may have been less about winning the case, and more about getting the word out that the 'test' Top Gear did was edited in a way that maximized humor vs. being 100% accurate.
Sure, and that's probably a fair point. They should have said exactly that. Instead they implied the show maliciously falsified a review and then spent months (years?) unsuccessfully litigating them.
Right, and Tesla lost because Top Gear argued (successfully) in court that what they aired was not a review but a comedy show. And being that the review was not a review, it could not be a malicious review. That's very thin ice to be standing on.
I really think you are mistaken. Do you have a source that says that's why the case was dismissed?
The Top Gear review was essentially accurate, at least IMHO.
Tesla argued in court that it was unfair to mention the car's disappointing range on the test track because most people won't be driving on a test track. A strange argument, especially for a sports car. Note that they did not argue that the range number cited was wrong -- just that it might confuse potential buyers.
The judge ruled that: "no reasonable person watching the programme could understand that the performance on the track is capable of direct comparison with the performance, including the range, on the road"
I thought the problem was Top Gear showed them pushing the Tesla, as if it had run out of juice, when in fact it still had battery and could have driven under its own power?
part of the issue is how quick people are to judge, combined with the particular historical perception of electric cars. Tesla has to be very aggressive about the image of its vehicles, otherwise the prevailing prejudices about electric cars in general will override the image of its own cars regardless of how awesome they may actually be
Their vehicle safety score was 0.42, the closest one was 0.50. That's a big difference. The lowest with a 5 star rating was 0.66 and the average was 0.60. That's totally something they should be allowed to brag about, but they should have just used those numbers or clearly explained their methodology. I think they were just trying to show off that number in a familiar way, and I can't blame them too much for that.
> the over-the-top posture against the NY Times reviewer who discovered that the battery heater could draw enough power to affect range
Actually he claimed to have the heater keeping the cabin at near-freezing temperatures, and the cabin temperature graph captured from the car strongly disputed that. Going over-the-top at the guy was all they could do after he had made so many different false claims. He was probably right with some of the claims, and the proof of him circling around in the parking lot wasn't as open-shut as initially presented, but when you are hit by that many misrepresentations at once you have to fight back fast.
You mean the reporter who drove around in circles trying to kill the car off and lied about the heater settings he used, also in an attempt to kill the car? I think you missed the whole week there where the actual drive data proved he was a fraud.
I don't like that reviewer, because he's evidently a very sloppy reviewer with a pretty poor understanding of physics and mechanics involved. However, I no longer think he was gaming the system deliberately. Assume incompetence before malice.
He may very well have been incompetent, but at the very least he was dishonest too. He put a serious bias on what he "reported" in his article. It may have been a malicious attempt to kill the car for a better story, or it may have just been an attempt to cover up his own incompetence, but either way, it was a pretty bad article and deserved to be trashed.
It just seems like the NHTSA is just reiterating that there's no official higher than 5 star rating to protect themselves, rather than scolding Tesla. Tesla did say that a rating higher than 5 stars does not exist, and that the extra high rating should be taken with a grain of salt seemed implied to me, but I guess they could have been clearer. The blog post was very in depth, so it's not like they were misleading people with their wording.
Yeah. This is a non-story; it makes a good headline but no one has been slapped. Tesla just noted that they received some papers which noted that for the internal purposes of the ranking, it got 5.4 stars. Then the NHTSA made a clarification that the official ranking actually doesn't go past 5 stars.
It's good for Tesla shareholders that Forbes decided to run with this story, though. Free advertising. There is no way anyone will ever perceive a fight about whether the car should have 5 or 5.4 stars as negative.
Tesla is going to bend and stretch the truth as far as they can whenever they can.
Tesla is more than an electric car. It has enough momentum now to be a measurable threat to almost 100 years of oil-driven businesses from supplier to manufacturers to dealers to the oil companies themselves. This creates no shortage of enemies who do not want adapt or change at all.
It's going to be a rough road ahead.
Tesla is going to apply the same tactics that its enemies do, the good and the bad. It's how war actually works. So don't be surprised when it happens.
Does it really hurt consumers if carmakers are bragging about doing well on tests?
I'd argue that it might: NHTSA wants to encourage automakers to build safer cars. One way to do that is to provide them with detailed information on the results ("scores") of its tests, so that automakers can benchmark their engineering against each other and tune individual safety features as needed. Another way is to aggregate those results for each model and publish a consumer-friendly figure that automakers are motivated to be able to advertise as a "stamp of approval".
What they don't want is automakers getting into NHTSA test "hot-rodding" games, focusing their efforts on topping the individual tests by some small relative figure in order to claim things like "safest car ever". In an environment like that, NHTSA has to either stop publishing detailed results, or keep the mechanics of their testing secret in an attempt to prevent "benchmark gaming" of the kind we see all the time in technology. That sounds like an NHTSA that is competing with automakers rather than working with them, and I suspect that is how consumers could be ultimately hurt.
73 comments
[ 2.4 ms ] story [ 62.9 ms ] thread> NHTSA does not rate vehicles beyond 5 stars and does not rank or order vehicles within the star rating categories.
So the following statement is flat-out untrue:
> SETS NEW NHTSA VEHICLE SAFETY SCORE RECORD
If NHTSA does not rate vehicles beyond 5 stars, the maximum safety score is 5 and thus 5.4 is irrelevant
I think if the headline was "sets new vehicle safety score record", dropping the NHTSA acronym, then it would be fine.
If they really did want to make a distinction, they'd introduce a new scoring metric that does allow for those differences
You could also just go ask the professor at their office for your grade, but using their site is easier.
It might not move fast, but you need military-grade hardware or construction equipment to make a dent.
NHTSA does not publish a star rating above 5, however safety levels better than 5 stars are captured in the overall Vehicle Safety Score (VSS) provided to manufacturers, where the Model S achieved a new combined record of 5.4 stars.
You can't blame Tesla for taking that number and running with it, given that (1) modeling car safety from two seaters to SUVs on a range from 1 to 5 stars is ridiculous and (2) the star systems suffer from the same problems as contemporary game and movie reviews: 99% of all evaluated products land between 3 and 5 stars, where 3 then becomes the new horrible, compressing the already low entropy of a star value further.
Maybe they need to be more aggressive with their adjustment so that the minimum expected really is 1 star.
sum(p(accident type) * p(estimated of surviving corresponding safety test))
At best you're computing odds according to the null hypothesis (probably not what you indented). At worst it's as meaningless as adding meters to seconds.
That's not a bad thing. We wanted safer cars. We started measuring safety, and publishing the measurements. As a result, safety improved. Working As Intended.
3 is the new horrible simply because that is the lowest that a commercially viable car can go--Chevy and Subaru have released sub-3 cars in the past and suffered for it with poor sales. They and others learned that consumers wouldn't buy cars that were rated below-average safety-wise and have vastly increased R&D spending on safety features. That's a good thing for consumers.
It better not be a real average if everyone is always above average!
You mean to say poorly rated, not below average, since it's not an average.
For a truly excellent product, the distribution should approximate a power law graph:
If you see something like these: Then ask yourself if the sample size makes this plausible, in which case the star rating isn't worth much, because there isn't enough data. Ask yourself if it's possible that someone has been gaming the system for or against.The uppercase gamma-shaped graph is either someone gaming the system or it's a power law graph hidden by aliasing. (In which case, the thing is of godlike quality.)
We should do away with star ratings, and instead just present an aggregate "rating." Genuine customer ratings are valuable, but are almost immediately gamed by cheaters. If I were Apple or Amazon, I would build stealth captchas and "honeypot" traps for ratings manipulation, and figure those into the rating. Be sure to separate detection from consequences by a random time interval of at least a few days. This makes the "development cost" for the cheaters much higher.
The problem with star ratings is obviously what PG refers to as "schlepping". Is anyone tackling this already?
EDIT: I would surmise that AirBNB is somewhat tackling this problem already.
And clearly, the goal here is to stop every manufacturer from just coming up with their own rules for interpreting the score that give them a different rating from the one that the NHTSA gives. That's definitely worthwhile.
True, but the agency gives legal cover. When there is a question in court as to whether the safety systems performed "as advertised", the NHTSA is free to say that Tesla didn't follow the advertising guidelines, and therefore cannot comment on the performance of the safety system.
Drug companies can also advertise drugs outside the FDA guidelines, and would win a court challenge on 1st Amendment grounds. None do it because it would expose them to unbounded liability.
No, no, no. This is utterly and confirmably false. It is illegal under the Federal Food, Drug and Cosmetic Act to advertise a prescription drug for a use that is not FDA approved, and it is within the FDA's mandate to recommend investigation to the Department of Justice in such circumstances. The FTC is authorized to oversee advertising for over-the-counter drugs under the Wheeler-Lea Act. As a recent example, Pfizer agreed to settle with the DoJ earlier this summer for half a billion to end an investigation brought under these very circumstances[1].
[1] http://www.newstatesman.com/business/business/2013/07/pfizer...
http://www.reedsmith.com/files/uploads/DrugDeviceLawBlog/Car...
" Caronia argues that he was convicted for his speech -- for promoting an FDA-approved drug for off-label use -- in violation of his right of free speech under the First Amendment. We agree. Accordingly, -3- we vacate the judgment of conviction and remand the case to the district court. "
It is uncertain whether marketing off label is protected by the 1st amendment. The courts don't all agree.
The one case you cite is very fresh precedent in a single Circuit, and it's highly controversial. No other Federal Circuit has applied First Amendment standards to FDA misbranding regulations before. Even within the Second Circuit (three states), is it still illegal under law to misbrand a drug in marketing materials. The decision in Cariona hinges on the question of whether "misbranding" and "promotion of off-label use" are different narrowly defined legal scenarios. The Second Circuit was concerned that, in the latter scenario, doctors may be legally deterred from promoting off-label use, which is a common practice in medicine and research. You should not take away from this case that drug marketing is now a First Amendment free-for-all. Yes, I am awfully certain.
Until then, however, off-label drug marketing is not something companies get away with without a substantial legal battle (often not worth it), and the FDA has so far collected billions of dollars in settlements and fines through successful off-label drug marketing suits.
We never advertised off-label. Whether it's legal or not, I do recall you'd get slapped by the FDA for lack of controls. We had many cases where doctors would use a product off-label and it'd come back to us in the form of a complaint or defect (product comes back as defective, we investigate and find out the doctor was using the product for an off-label procedure it wasn't designed for and would fail, hence the 'defect'). At which point we'd have to proactively train doctors to not do this and other corrective action measures would have to be put in place. If upon investigating we find this would be a good use of the product, we'd have to go through another round of FDA fun (sometimes clinicals and how much testing depended on some factors, definitely rounds of internal testing and implement new QA tests and procedures to ensure the product can work for the new use and have proper fault tolerances as well as new labeling and training, etc).
Now, do companies try to skirt around this? Sure. Part of my job at the time was to make sure we didn't. FDA Warning letters are no joke in this industry, it can literally put you out of business.
The most analogous case that comes to mind, is the recent California small claims case against Honda.
Facts: According to the EPA (who independently tests vehicles fuel efficiency) the Prius gets an average of 51 MPG. The Plaintiff claimed she only got an average of approximately 30 MPG. The Plaintiff won the lawsuit against Honda for fraudulent misrepresentation winning $10K the small claims limit in the jurisdiction. However, one of the most interesting arguments (and legitimate) was that while Honda does advertise the 51 MPG, that Honda could not be fraudulently misrepresenting the MPG, because Honda had no control over the testing or findings handled by the EPA, and going further Honda does not have a right (1st amendment) to contradict the "EPA sticker". Ergo according to the argument Honda should not be liable, rather the EPA - nevertheless the Court found Honda liable for fraudulent misrepresentation.
Assuming this law is on point and controlling, for example, if the Tesla were to under perform in a real crash and Tesla was sued, Tesla may be liable despite having no control over the NHTSA test or findings. Going further, if the dicta from the Honda case is also on point, if Tesla's internal crash tests contradict NHTSA's ratings, Tesla would not have a 1st Amendment right to contradict the NHTSA findings publicly.
Note: That SCOTUS makes a distinction between "commercial speech" and other forms of speech, and the government has a lot more power to regulate commercial speech than say political speech.
I doubt that anyone at Tesla is feeling the least bit chastised. Musk is certainly not the type of person who is afraid to generate a little controversy.
Tesla is a startup with cool technology. They're taking on a market where startups never succeed and are navigating significant challenges regarding new technology, consumer bias, competitors, regulatory issues, business model, press relations, and more. Some of us find it fascinating and enlightening to watch them take this journey.
Edit: Reworded.
The Top Gear review was essentially accurate, at least IMHO.
Tesla argued in court that it was unfair to mention the car's disappointing range on the test track because most people won't be driving on a test track. A strange argument, especially for a sports car. Note that they did not argue that the range number cited was wrong -- just that it might confuse potential buyers.
The judge ruled that: "no reasonable person watching the programme could understand that the performance on the track is capable of direct comparison with the performance, including the range, on the road"
Actually he claimed to have the heater keeping the cabin at near-freezing temperatures, and the cabin temperature graph captured from the car strongly disputed that. Going over-the-top at the guy was all they could do after he had made so many different false claims. He was probably right with some of the claims, and the proof of him circling around in the parking lot wasn't as open-shut as initially presented, but when you are hit by that many misrepresentations at once you have to fight back fast.
Entities have rights over how they are used in promotions, i.e., the right of publicity / personality rights http://en.wikipedia.org/wiki/Personality_rights
For example, Consumer Reports No Commercial Use Policy - http://www.consumerreports.org/cro/about-us/no-commercial-us...
It's good for Tesla shareholders that Forbes decided to run with this story, though. Free advertising. There is no way anyone will ever perceive a fight about whether the car should have 5 or 5.4 stars as negative.
Tesla is more than an electric car. It has enough momentum now to be a measurable threat to almost 100 years of oil-driven businesses from supplier to manufacturers to dealers to the oil companies themselves. This creates no shortage of enemies who do not want adapt or change at all.
It's going to be a rough road ahead.
Tesla is going to apply the same tactics that its enemies do, the good and the bad. It's how war actually works. So don't be surprised when it happens.
I'd argue that it might: NHTSA wants to encourage automakers to build safer cars. One way to do that is to provide them with detailed information on the results ("scores") of its tests, so that automakers can benchmark their engineering against each other and tune individual safety features as needed. Another way is to aggregate those results for each model and publish a consumer-friendly figure that automakers are motivated to be able to advertise as a "stamp of approval".
What they don't want is automakers getting into NHTSA test "hot-rodding" games, focusing their efforts on topping the individual tests by some small relative figure in order to claim things like "safest car ever". In an environment like that, NHTSA has to either stop publishing detailed results, or keep the mechanics of their testing secret in an attempt to prevent "benchmark gaming" of the kind we see all the time in technology. That sounds like an NHTSA that is competing with automakers rather than working with them, and I suspect that is how consumers could be ultimately hurt.