53 comments

[ 3.9 ms ] story [ 17.0 ms ] thread
I know some people will disagree, but I think the solution should be government regulation. Anything that connects wirelessly should have a standard security protocol. The way they make these devices now are laughably insecure. Until there's an actual law requiring manufacturer's to go the extra mile for security, they're going to continue to be lazy about it.
On the whole I agree with the author, but they have glossed over some things I feel should be addressed:

> The computer manufacturer Lenovo, for instance, used to sell its computers with a program called “Superfish” preinstalled.

Most "serious" users reformat the machine and re-install the OS to get rid of the OEM included crapware. Not something the average user does, sure, but you're still allowed to re-install the OS on a commodity x86 system like a laptop. It's a matter of inconvenience, not impossibility.

> This system means that a car company can’t stop me from painting my car a shocking shade of pink or from getting the oil changed at whatever repair shop I choose. I can even try to modify or fix my car myself.

You cannot do this with a Tesla. Go look on the internet about people waiting months for Tesla to repair their cars, or people who have bought totaled Teslas and fixed them, only to be denied re-certification by Tesla.

I think we will see other electric car manufacturers moving in the direction of a more closed ecosystem than what we have now with ICE cars. Reason being that electric cars have a unique drive system (even more than ICE cars) and usually a proprietary HVDC battery (>60V) and battery management system. The manufacturer risks increased liability and bad PR if they let people modify these cars and then they end up in flames.

> Samsung cuts deals with lots of software providers which want to take my data for their own use.

Same deal as with Lenovo, but slightly complicated by the fact that ARM is not as standardized at a platform level as x86 (yet, I think this is slowly changing).

Pick a phone that comes without OEM crapware like Google Pixel, or a phone with good 3rd party ROM support like LineageOS.

> What is important is that we recognize and reject what these companies are trying to do, buy accordingly, vigorously exercise our rights to use, repair and modify our smart property, and support efforts to strengthen those rights.

Agree 100%, but this requires consumers to research products before they buy the cheapest one. We can encourage good behaviour by manufacturers, but it will mean educating our friends and family and collectively voting with our wallets.

> It's a matter of incovenice

It's a matter of the moral bankruptcy of supporting a company that does this, and voting with your wallet that this kind of behavior ought to continue.

The manyfacturerer risks increased liability and bad PR if they let people modify these cars and then they end up in flames.

Car modding has gone on for around a century(!) and I can't remember a single instance where someone modified a car, was injured or killed, then blamed the manufacturer, and the latter was found at fault (instead of the modder or driver.) It's just the overreaching paternalism that seems all too common with companies today.

Electric cars are closing the ecosystem simply because the companies can exert more control (= profit) that way; the "safety" justifications are really to divert attention away from that.

Of course, Tesla can't stop you trying to paint your car...

> Car modding has gone on for around a century(!)

Perhaps I'm mistaken, but my impression is that people weren't as litigious in previous decades.

> and I can't remember a single instance where someone modified a car, was injured or killed, then blamed the manufacturer, and the latter was found at fault (instead of the modder or driver.)

Sure, but for the past century we've had cars which run on flammable liquid and are powered by thousands of explosions per minute.

So, it's not entirely news worthy when a car which burns stuff catches fire. For a car which runs on electricity it's not immediately obvious to people that there are still components inside plenty capable of producing fire when damaged or used incorrectly.

I'm not sure which I'd least like to be near a burning tank of gasoline or a burning lithium ion battery.....
Frankly, the battery. Gasoline fumes aren't nearly as toxic, and neither is likely to explode. In either case you have to run away ASAP.
> you're still allowed to re-install the OS on a commodity x86 system like a laptop

Remember when Microsoft tried to change that with Secure Boot? If it hadn't been for the public outcry in the tech columns, they probably would've proceeded with it.

> Remember when Microsoft tried to change that with Secure Boot? If it hadn't been for the public outcry in the tech columns, they probably would've proceeded with it.

Yes, but Secure Boot doesn't/shouldn't prevent you from re-installing Windows, since Microsoft's Secure Boot keys are installed anyway.

I believe in all but a few cases OEMs have made Secure Boot an optional flag in UEFI, so you can choose to disable it if you want to install another OS such as Linux.

However I am aware of several non-x86 platforms which use Secure Boot to prevent you from installing another OS (e.g. Surface 2)

>> The computer manufacturer Lenovo, for instance, used to sell its computers with a program called “Superfish” preinstalled.

> Most "serious" users reformat the machine and re-install the OS to get rid of the OEM included crapware. [...] It's a matter of inconvenience, not impossibility.

And this is why said Lenovo included the crapware and spyware in non-volatile memory that is executed by Windows boot automatically by design, so your reinstallation doesn't work as intended unless you change the OS you're installing.

> And this is why said Lenovo included the crapware and spyware in non-volatile memory that is executed by Windows boot automatically by design, so your reinstallation doesn't work as intended

Could you please provide a citation for this claim?

I've never heard of Lenovo bundling parts of Superfish into UEFI to be installed again if Windows is reinstalled.

I suspect Tesla will face significant legal issues with the way they lock down their platform in the near future. The wealthy folks who can afford the S and X can afford to only deal with the manufacturer, and are likely to buy a new car when the warranty expires. But with the Model 3, a lot more people are going to take issue with Tesla's rules.
> Most "serious" users reformat the machine and re-install the OS to get rid of the OEM included crapware.

And then you get all the fun of running around the web for random drivers, navigating the OEM's and third parties websites to find the multi hundred GB driver package that reinstalls the crapware. That's if the OEM site will let you download the drivers of course, I've run into a few where it wasn't possible.

Wasn't the Roomba sharing maps thing already debunked?
Yes. From http://lifehacker.com/tell-your-roomba-to-stop-sharing-a-map...:

> To clarify, iRobot has not formed any plans to sell data. iRobot is committed to the absolute privacy of our customer-related data, including data collected by our connected products. No data is sold to third-parties. No data will be shared with third-parties without the informed consent of our customers. If a customer had already signed up/opted in, iRobot will delete the data from our servers if a customer requests it. This is retroactive.

> Clean Map Reports are not shared with third parties. If a Roomba owner does not want to share data with a third party such as Amazon (for example, to enable voice control from Amazon Alexa), the owner can simply disable the skill in the Amazon Alexa app.

Some technologies empower, others disempower and if iot devices and things like self driving cars are not rolled out with proper systems of user control and privacy they will disempower people.

For many in the industry this is not a concern, a mere technology issue, an inevitability, an opportunity for profit.

Fortunately there is growing skepticism and awareness of the intrusiveness. And self congratulatory messages on technology forums belittling user ignorance - oh they are dumb so we will take control - as if everyone should be a technology expert will be seen for the betrayal of users it is.

Self preservation comes naturally to humans and given most technology folks are happy, even proud to work with deeply intrusive spyware companies, they are not going to be perceived as harmless as before. There is going to be a backlash against the so called 'nerds' involved in this one way transfer of power.

The problem is software complexity. You can't really own anything you don't understand, and you can't be free in a world you don't understand. And we don't understand software.

Imagine a naive young king who doesn't understand how to do anything for himself. He will easily be controlled by those around him that do understand the world.

Of course, nobody before ever understood everything about the world. But different people understood different aspects in depth, and education gave literacy so people could share expertise.

Software is so complex that nobody really understands it. Nobody understands how a car works, nobody understands how a phone works, because they are both controlled by hopelessly complex software. Governments can't regulate emissions because the software changes the emissions controls based on arbitrary unknown factors. Security problems are just a small subset of the surprising ways complex software behaves -- surprising even to the software engineers who build it.

And the key is that this is all new complexity that didn't exist before. Physics and biology were developed to understand pre-existing complexity, so we always made forward progress. But we are moving backwards now because we are introducing complexity faster than we are understanding it -- much faster.

Therefore this is actually worse than the naive king mentioned above. In that example, it's just an asymmetry of information, and that can be resolved through education. Software is so complex education can't hope to keep up.

and much of that complexity, at least on personal computers, gets added in the name of "user friendliness"...
also in the name of saving money/time (by using large libraries/frameworks and such)
And it does! That's what's maddening about it. The complexity always seems to justify itself, but the end results are probably not where we want to be.
>> But we are moving backwards now because we are introducing complexity faster than we are understanding it -- much faster.

That's a brilliant insight - thank you

> The problem is software complexity. You can't really own anything you don't understand, and you can't be free in a world you don't understand. And we don't understand software.

What this means is that for all practical purposes AI is already here. AI, or rather the fear of AI, is about software making decisions that impact us outside of our knowledge or control.

Prior to software this was done by bureaucracy. We can see software (on the slope to AI) as disempowering in the same way.

+1. The AI apocalypse looks more like a Kafka-esque bureaucracy and not so much like I, Robot with machines shooting at us.
Is it truly Kafka-esque when it provides us with adorable kitten gifs? At worst Brave New World-esque, at best actually good. Kitty!
This is one of the issues with sci-fi movies for the less metaphorically minded. Or, well, anyone. I find books make the ideas much more portable, which is half the fun of sci-fi in the first place. Things in real life are never exactly what they're like in the pictures.
> And we don't understand software.

People are perfectly capable of understanding very large programs even from reverse engineering.

The issue is that things change so quickly that generating such knowledge is economically useless.

I have this discussion with my CTO all the time:

"I'd like you to go do <X>".

"No, please give that to <junior person>."

"But he's junior. I want someone senior to do it."

"The half-life on the knowledge to do <X> is weeks to months. Senior people will be no better at <X> than a junior person, and our junior people are quite capable precisely because we trust them with things that other places would consider a "senior" task. The first couple of weeks are going to be spent on Google, Slack, IRC, mailing lists, forums, and anything else figuring out all the ways that thing fails and how to debug it. After a week or two, they won't be junior at it, and we'll make them give a talk about it if we deploy it."

Sometimes I still have to step in, and that's fine. But I avoid learning software things with short half-lives as much as possible, nowadays.

Of course, that makes me an out-of-touch, crusty, old fart to my juniors.

But, they are smart enough to acknowledge that my code always seems to be so much more reliable than theirs. And that's fine, too.

"The issue is that things change so quickly that generating such knowledge is economically useless."

That's a good point, and certainly part of the problem, but it does not detract from my point.

The bottom line is that we don't understand the software in the world around us and that it would be hopeless to try. Whether that's because of static complexity or dynamic complexity seems like a separate discussion.

Changing software on the scale that humans do is kind of like changing the laws of physics every year. Great to know that we understand some past snapshot of the software stack that we built, but not relevant to living in today's world.

A prime example of all this are AI black boxes like neural nets.
Nobody understands how a car works, nobody understands how a phone works, because they are both controlled by hopelessly complex software.

At least for cars and other home appliances, you can still buy older models which don't have software at all and are thus relatively understandable and repairable (and such machines often sell for surprisingly high prices even when well-used, for this and a few other reasons), but then you give up convenience and efficiency and safety and other things that could be considered progress.

Of course, a lot of people will think you're a redneck or similar if you pursue that way of life, but it's certainly one way to stay away from the "complexity explosion".

> Software is so complex that nobody really understands it.

Unless this statement is trying to imply something deeper than what it states, I would disagree.

I work with embedded systems and connected devices. To me it's not a problem of complexity, but a problem of not having a strong standard of practices on both the device side and the facility side.

Have sensitive data on your network? Separate it from the rest of the network, or don't put devices on the network that don't meet your security needs.

"don't put devices on the network that don't meet your security needs"

Does it not disturb you that the mere presence of a device on your network could compromise everything else? Why is that?

I argue it's because of complexity. We don't really know how these devices behave outside of very controlled circumstances.

And the vulnerability could be anywhere in the stack. I remember this bug from a while back:

https://lcamtuf.blogspot.com/2014/10/psa-dont-run-strings-on...

The gnu strings utility was vulnerable to untrusted input! Who would possibly imagine that would ever be the case!

Honestly, that bug just made me give up. Software cannot be reasoned about any longer. I used to believe that solid components strung carefully together could add up to something understandable. But no, we are beyond that.

I'm not talking about crazy James Bond hackers that somehow infected your compiler or something. I mean that, by accident, a basic utility does something crazy.

This is not math or science or engineering any more. It's wizardry, witchcraft, and alchemy.

(I'm exaggerating a bit, but it really is discouraging to me.)

I agree, and this is why "safe" languages like Rust, OCaml, Erlang, etc. are important. It's not possible to guarantee safety under all circumstances, but as software becomes more complex, anything we can do ensure some degree of safety is pretty crucial.
You might find Normal Accidents by Perrow interesting. He describes accidents caused by systems (e.g. nuclear power plants) becoming so complex that they are incomprehensible to humans.

"I used to believe that solid components strung carefully together could add up to something understandable. But no, we are beyond that."

I think that's true, but it turns out that we don't have any solid components. None.

If you put two nuclear power plants on the same grid, it is pretty hard to imagine how a meltdown of one plant would trigger meltdowns elsewhere on the grid (because the grid carries electrical energy and is incapable of carrying high-speed neutrons). But with software, you don't have to imagine such failures, they happen all the time (because the internet can carry any data, including more software). So I still maintain that software systems are more complex. And if they aren't more complex today, they will be soon, because the complexity is growing without any obvious bound.

But for the sake of argument, let's say they are of comparable complexity. If you show a layperson a nuclear power plant, and say "who do you think should run this: you, or a team of nuclear engineers?" they would probably answer "a team of nuclear engineers, please". Show the same person a television, and they will feel like they should be able to operate it. But it's actually an internet-enabled TV running sophisticated software that is on the same home network as your internet-enabled security camera system, and it's a very unsettling situation. In other words, now software makes everything -- toasters, TVs, phones, cars -- into incomprehensible systems.

I disagree. Not understanding how my car works is not a problem for ownership, so long as I can decide where to delegate all the maintenance etc that requires it to work. When I can choose my mechanic, I own it. When I have to take it to a dealership, because of all the DRM'd electronics inside, I no longer do.
"I disagree...When I can choose my mechanic, I own it."

I think we are in agreement here. You don't personally need know everything about everything you own. But someone needs to know enough about it that you can use their expertise or learn from them if you care to.

The myth is that it's about who knows -- the right laws (or the right consumer pressure) will open up the right information, and freedom will supposedly follow. That's a minor factor, sure, but I believe it's more of a myth.

The real thing holding us back is that nobody understands these complex software stacks, we are just building them bigger and bigger and understanding them less and less.

The problem is that hardware manufacturers want to keep collecting income after they have sold the product. Imagine if when you bought a car and paid it off, you still had to pay a fee to the manufacturer for every mile you drove. And ditto anyone you sold you car to. That is what is basically doing on today. Rent-seeking anyone?
Authored by a law professor.

It is Apple who pioneered this concept of the device that the user really does not own or control - the "smartphone". The computer that remains tethered to the manfacturer continually feeding back user data.

Manufacturer makes replacing the OS difficult. Manufacturer gets root. Manufacturer gets right to modify OS remotely. And no surprise, manuacturer gets plenty of data about users. Manufacturer did not ask for these privileges. They just took them. It was not always this way.

But users accepted it.

And so other companies followed Apple's lead. I find the company's official statements about how they protect privacy to be offensive to common sense.

Want to protect privacy? Stop forcibly collecting any and all data. (Not just "less than so-and-so".)

If users really believe that sending data to Apple will help Apple create the "best experience" then let them opt-in to collection and send it voluntarily.

The users own the device. If they wish untether from Apple's "ecosystem", then that is their right.

There exists at least one user who does not want a company to collect their data, put it on a networked computer and then purport to "protect" it. They want the company to stop collecting their data. Why? Because they have weighed the risks and decided it is not worth it.

IMO the "smartphone" is part of the IoT and was the first "thing". Because it was a thing where software was used to take away some of the control from the owner (control that owners had previously); owners only get partial control.

Going forward, if one is opposed to internet connected things one does not fully control, the solution is not to select the things that collect the least data or that offer the most control. ("I trust Company X because they collect less data than the other companies.") The solution is to choose things that are not internet connected.

Apple collects very little user data. Developers have complained about the limits Apple puts on collecting data and Apple News partners have been quite frustrated. Even the people who originally developed Siri were frustrated by not having access to more user data.

There are plenty of companies like Facebook and Google that have no regard for user privacy but I wouldn't include Apple in that bucket.

it is Apple who has pioneered this concept of the device the user really does not own or control - the "smartphone".

Users controlled the Super-NES? Users controlled the Sony Playstation? Users controlled Blackberries with their perma-connection back to RIM servers? Symbian S60 users got a choice of the OS? Internet connections come with choices of user-controlled routers? Early internet connections with AOL-Compuserve-software was user-controlled and didn't talk back to one company?

Devices where the user has any control at all are the rarity of device makes/models/types, even if they are a majority of sales due to the PC-compatible hardware.

> The solution is to choose things that are not internet connected.

A few ultra privacy-conscious people will do this, but they'll be statistically irrelevant.

I believe the more likely answer we'll see is whistleblowers. Either internal employees of tech firms or hackers. It's a lot of fun to tear down IoT devices and RE firmware. It's getting harder to get any type of reasonable coverage across the plethora of devices available, but every tech firm knows if they're up to shifty things, it's going to end up as a twitter storm at some point in the future.

> It is Apple who pioneered this concept of the device that the user really does not own or control - the "smartphone".

Oh bullshit! Microsoft and other companies shipped user-hostile EULAs for years before smartphones existed (let alone the iPhone). These shrink-wrap licenses used this concept for years, and there were probably others before them.

I agree. Copy+pasting an old comment of mine from this old thread https://news.ycombinator.com/item?id=12683924

> A general solution to IoT security would be for all IoT devices to only communicate to your personally owned home gateway, which would run open-source drivers for each device to provide the networking/external communication functionality. The IoT device could even be assigned its own isolated network link to the router (i.e. sandboxed).

This is how my home works. My automation devices use Insteon, which is local only communication, and then my computer has an adapter for it that I wrote the software for.
This is a great solution to the ownership problem, but now each individual is responsible for patching and updating their 25 devices? Doesn't solve the security problem.
The home gateway could download the patches and send them to the devices. Its firewall will prevent them to phone home or somewhere else no matter what into those patches.
Just wait for full on AI, then you'll see some real digital serf action.
This is in part why supporting the FSF and other bodies that seek to ensure that choice and transparency are promoted. The only real way to compel companies to comply is to support viable alternatives to the dominance of a vendor controlled market.