> The most likely way people will get killed by robots is by taking their job, forcing them into poverty and eventually letting them die on the street.
Serves 'em right for not being able to provide value to capital. /s
Interesting that the article focuses on the unintentional killing of people by robots. That is unfortunate and worth writing about, but society is good at proportional reaction to things like that. The more disasters there are, the more we'll take it seriously, and the more lawsuits & regulation that will follow.
See: self-driving cars.
The more challenging topic is military/government use of robots in war or citizen control scenarios. Areas like that seem very relevant if you look forward a few decades given the pace of progress of Boston Robotics and AI.
See: Terminator, the movie. Time travel aside... :P
>The more challenging topic is military/government use of robots in war or citizen control scenarios.
Absolutely not challenging. They will come and will be used. Nothing you can do about it unless you DRM everything. And then of course the government will still have them. Topic closed.
>Interesting that the article focuses on the unintentional killing of people by robots
I doubt that there was ever killing. The first two examples are people trying to do maintenance on plugged machines. If you try to fix moving steamroller by standing in front of the wheel - don't be surprised at the consequences.
The south africa one - it depends - if the turret just misfired in this general direction - is no different than a mine or shell exploding. If it scanned the soldiers, recognized them as targets and then shoot - then it is killing.
> the robot simply determined that “the most efficient way to eliminate the threat was to push the worker into an adjacent machine.”
This seems inflammatory and highly misleading. I could be wrong, but I highly doubt it: the robot didn't "determine" anything. It had assembly instructions to follow, and it followed them. The worker got in the way and was struck.
Schneier falls into this same error, saying Tesla Autopilot sometimes "ignor[es] red lights, stop signs, and crosswalks." Tesla isn't perfect, but I'm pretty confident there's no code in Autopilot that says, "yeah, I see that crosswalk, but Elon's in a hurry, so let's goooo!" -- okay, maybe in Elon mode. But not in general. So no, Autopilot doesn't "ignore" those items: it misses them.
Assigning intention to robots at this point is pretty much equivalent to assigning intention to a clock because the hands go around. We have a long way to go before we can/should assign intentionality to "robots" this way. Of course, at the rate things are going, "a long way" could happen in the next few years. But it's not happening now. I'll happily consider counter-examples.
Absolutely, but you're missing the most inflammatory part of all! The robot had to "eliminate the threat" -- when this was about a human who got in the way.
What's the meaningful distinction between the robot ignoring or missing relevant information? We're not in a court of law trying to suss out intent or lay blame on a specific entity. The essay is about robots killing people, and supposing this will continue without government regulation. The people will be killed either way...
The irony is that we're in this problem in the first place because people are releasing problematic tech before it is understood in the first place. Humanity at large shouldn't be part of the experimentation process, and the essay is arguing that we need tighter regulations around that.
The essay suggests "Any regulation of industrial robots stems from existing industrial regulation, which has been evolving for many decades" & that regulations need to be clear about other robots as well. Interestingly enough, if we applied existing regulations to Tesla's Autopilot as we would to a driver who racked up as many fatalities and crashes (736 crashes and 17 deaths in the U.S. since 2019 [0]), they'd be behind bars long ago, let alone never able to drive again.
Intentionality is a central concept in AI philosophy - from debates
between Dennet, Churchland and Searle going back to the 1980s.
Aat this stage I agree that we are dealing with "blind machines", with
no more "intention" than a steam engine.
However, the central argument in Schneier's piece seems to be that
robots inherit the intentionality of their creators.
If I create a machine that kills someone, my "intentionality" is fair
game to be dissected in court, and ultimately I may be found guilty as
if I had killed with my own hands.
That is the kind of high bar I think society expects, and is certainly
one that should scare the crap out of robotics developers (aside: I
have been one) to ensure the highest standards of engineering
discipline and industrial ethics.
> Idiots walk into machinery all the time. Put up signs, do safety training, yell "Stop!" and still Darwin takes his share.
It's still on leadership if they choose to hire idiots who get hurt. The solution is: don't hire idiots.
If you look historically at manufacturing and the origins of signage and safety training, you'll find them mandated by the government (i.e. OSHA, etc.) because those things weren't always voluntarily provided by the employer. Upton Sinclair opened our eyes to this 100 years ago.
What you're missing here is the numbers game. The size of the operation vs. the tolerance for worker injuries has a massive impact on this stuff.
When you run a shop with five experienced people, if each of them has a 1-in-a-million chance of getting seriously injured on any given day, each year you have a 1-in-548 chance of a serious injury. Any sane person would call this risk acceptable.
If you run a shop with 5000 people of equal skill level to those five above, you're talking a couple of deaths or permanent disability injuries per year, just from the numbers. Any rational person would say hey, that's the same risk, why are you mad? But society isn't rational and if any legal entity no matter how large kills or maims a couple of workers a day, there's an outcry and executive heads will roll. So yeah, they try to do better. At this point maybe 'hire smarter people' works?
Now imagine you're running a multinational company with 500,000 people. If you operate like that original shop you'll see a couple of hundred deaths in your first year and then get sued into oblivion or lynched, and remember those were skilled experts. There aren't 500,000 skilled experts available for hire, and most of these jobs are ahem less skilled anyway, so you're hiring people with much higher rates of personal injury. To operate at this level you need to do stuff that seems dumb to the uninitiated, like relying on layers of subcontracting companies for everything while implementing byzantine and ever-shifting safety regulations that each subcontractor must commit to following. Sure, it costs an order of magnitude more, but it's worth it for the severability.
Heavy machinery (in production) is designed that you cannot just walk into them and will stop, if you do. It is a bit different approach from software engeneering.
Well, I probably have to add, that my point of view comes from old school german engeneering (through my dad). His approach to building something vs. mine as a software dev are quite different ..
And the accidents that I know of, are where the security mechanism have been tampered with. A proper designed and certified german machine is hard to get injured by if followed protocol (and the reasons are past accidents and suing the manufactor).
Wow, this is absurdly callous IMO. How dumb does one have to be to deserve death? Why do we have lines on the road when discerning drivers could just avoid other cars without them? I think the idea that industrial deaths are inevitable/NBD/the fault of the victim is more outlandish than it’s implied to be in your comment
Basically the same as when your company produced a pesticide that caused cancer and killed someone. There was no intent in the molecule, but there was at the very least negligence on the side of the producer; if they knew about any negative side effects but swept it under the rug, then I suppose it could qualify as manslaughter. In short, nothing really new here.
> If I create a machine that kills someone, my "intentionality" is fair game to be dissected in court, and ultimately I may be found guilty as if I had killed with my own hands.
The weapons industry (and its lobby organization, the NRA) would like to have a word with you...
> Intentionality is a central concept in AI philosophy
Sure (and thanks for pointing this out) but this isn’t a philosophical piece, it was written for the Atlantic, and he’s being careless with his language.
I agree with your first comment, but disagree on the second one. It is weird to say that the robot wanted to eliminate a threat, most likely it was simply not programmed to avoid obstacles given the environemnt it was in. He ascribes an intention that was not there. To be fair, it is a quote from a book and not the words chosen directly by the author, but he could have clarified.
The second one seems fair: he is describing the result of what the Tesla Autopilot is doing. I do not think that he means to say that the Autopilot is understanding that there is a red light but then it makes a consciuous decision to itgnore it. He is just saying that the robot is malfunctioning, as it should be clear from context:
"Today’s AI-related robot deaths are no different from the robot accidents of the past. Those industrial robots malfunctioned, and human operators trying to assist were killed in unexpected ways. [..] Malfunctioning Teslas on Autopilot have deviated from their advertised capabilities by misreading road markings, suddenly veering into other cars or trees, crashing into well-marked service vehicles, or ignoring red lights, stop signs, and crosswalks".
I do not think there is a more neutral way of describing the events.
> most likely it was simply not programmed to avoid obstacles given the environment it was in.
Even more likely, it’s a manufacturing robot, with no vision at all except maybe for the pieces it is manipulating. It almost certainly has a line on the floor, or a railing even, to keep people out of its way.
In short, it has zero obstacle avoidance. I’d be interested in counterexamples.
> The second one seems fair: he is describing the result of what the Tesla Autopilot is doing. I do not think that he means to say that the Autopilot is understanding that there is a red light but then it makes a consciuous decision to itgnore it. He is just saying that the robot is malfunctioning, as it should be clear from context:
"Today’s AI-related robot deaths are no different from the robot accidents of the past. Those industrial robots malfunctioned, and human operators trying to assist were killed in unexpected ways. [..] Malfunctioning Teslas on Autopilot have deviated from their advertised capabilities by misreading road markings, suddenly veering into other cars or trees, crashing into well-marked service vehicles, or ignoring red lights, stop signs, and crosswalks".
He describes Teslas as “no different” from non-intentional machines of the past, and then describes them with “intentional” language.
>I do not think there is a more neutral way of describing the events.
How about, “Industrial accidents happen through lack of human foresight and carelessness. The same is currently true of ‘AI’ products like Autopilot. But the time will come, and probably soon, where AI alignment and intentionality will be part of the conversation. We should prepare for that starting now.”
Not as catchy as “Robot rampage!” But more accurate and, as you say, neutral.
What if this author doesn’t care about intentionality? AFAICT you’re assuming they do because they use the active verb “ignore” for a machine, but that says little to me. We use language like this all the time for non-intentional systems: “the plant reached skyward”, “the economy soared”, etc etc etc.
I really really don’t think this is an article about convincing robots to be nice to us (AKA alignment); I think it’s an article about how we need stricter regulations for software that controls physical machines. Doesn’t that sound like a good idea?
The difference is that no one (very few people) would ever consider a plant to be sentient. So the metaphor works as intended. Since many (most?) people are willing to entertain the notion that AI can be intentional, the metaphor is ambiguous and fails.
I’m not arguing against the underlying point, I’m just disagreeing with how it’s presented.
To put it another way, we might as well call a steam train a robot who will murder any human who tries to obstruct its desire to get to the next station, instead of jumping the track and going around.
That quote leapt out at me too. I think the reason it leapt out was because the rest of the article was so thoroughly objective, well reasoned, and rooted in historical precedence. Given the context in which it appeared, I’m willing to give it the benefit of the doubt as slightly expedient wording that was shorthand for the more nuanced understanding he clearly possesses.
Yeah this is a great example of what’s wrong with all our robots: their perception is garbage.
Reasoning is actually pretty good now. An AI can compose a sonnet on electron degeneracy pressure no-problem but ask it where the red ball is - and really inspect its localization of that red ball and it’s way off. Orders of magnitude of imprecision.
A modern AI can reason through the trolley problem and millions of variants. But it can’t see the person on the track with very high confidence, or be sure that the trolley is going to hit the person or not.
You're right about AI capabilities being much better in the "ungrounded" realm (like writing sonnets), but I would revise the statement a little to be more accurate.
AI systems are quite good at perception in a traditional sense (e.g. determining a red ball is in the sensor receptive field), but once actual cognition about how the red ball relates to the world around it becomes important, current approaches are limited.
I would argue that the first wave of perception is solved much more completely than the ability to reason about complex interactions of objects grounded in the physical world.
I think it matters how accurate something should be before we consider it "good". For many applications, a 99% success rate is unusably bad. A self driving car that recognizes 99% of red lights or stop signs is probably a criminal liability.
A red ball detector that operates at 60hz with a 99.9% accuracy will make an error on average once every 11.5 seconds. I think the issue is not so much the interactions of objects and the world but rather we're just wildly off-base regarding the base performance needed.
If a human driver isn't paying attention to the road, fails to see a stop sign, and drives past it causing an accident, isn't it a reasonable use of English to say they "ignored" the sign?
Ignoring can mean failing to notice something, as well as noticing but disregarding it. Words can often have multiple similar complementary meanings. I don't think the usage here is incorrect, or even misleading.
Any sort of ticket from the traffic police will say something like "failed to obey traffic control devices," which is more of a statement of fact. "Ignore" definitely has shades of nuance.
No it can’t. Every definition of ignore you will find has intention in it, words mean things.
You don’t think it’s important to draw a distinction between whether a robot was deliberately programmed to do something malicious as opposed to a bug which caused an oversight? Come on.
> to intentionally not listen or give attention to:
> to give no attention to something or someone:
Two definitions - one with intention, one without
> words mean things.
Yes, but they often mean multiple things. Very few dictionary entries only have a single definition for a word. You can't use "words mean things" to claim that "this word means only what I say it means, regardless of the other accepted uses it already has".
> You don’t think it’s important to draw a distinction between whether a robot was deliberately programmed to do something malicious as opposed to a bug which caused an oversight?
I do think there are times when it is very important to do that.
I just don't think that "ignore" is the right word to make that distinction, and isn't a distinction the authors were trying to make at that particular time, because if they were they would probably have used another word.
...at least, that's how I read it. It seemed clear to me. ¯\_(ツ)_/¯
I would say if the human driver was distracted and drove past a stop sign, he did not "ignore" the stop sign. He simply "missed" or did not see the stop sign.
To ignore something, you have to be aware of what you are actively ignoring. Even in the link you provided for the "second" definition, look at the examples paired with it.
> > the robot simply determined that “the most efficient way to eliminate the threat was to push the worker into an adjacent machine.”
> This seems inflammatory and highly misleading. I could be wrong, but I highly doubt it: the robot didn't "determine" anything. It had assembly instructions to follow, and it followed them. The worker got in the way and was struck.
I was prepared to argue with you, but upon reading the article, I agree that the description you quote makes a huge category error.
I think it's believable that we'll soon get AIs that are advanced enough they can do agent modeling and that some subgoal-maximizer accident could happen where the robots determines "the most efficient way to accomplish <objective> is to take out that human so he doesn't turn me off", in which case the article's poetic description would be correct.
Except the quoted paragraph is talking about an industrial accident in 1981, so the whole "eliminate the threat" presentation is obviously absurd.
I absolutely agree with the comment from "Winter" at the bottom of the post:
> The examples are more about Machines killing people than Robots killing people...
The watershed will be when robots are build to kill people and then actually do kill people the intended way, but not necessarily the intended targets.
A mine is a machine, not a robot. At least the more simple mechanical mines. The distinction for robots in general is some kind of automated intelligence where it is actively selective of its targets. Some modern missiles and drones would be far better examples.
Modern mines have several kinds of sensors (pressure, accelerometer, magnetic, timer) and only activate when there's a pre-programmed combination of events (to prevent easy de-mining). They can often also be detonated or disabled remotely.
Also sounds like he's blaming the tool, instead of the operator of the tool. I really expected that number to be higher when you account for:
Have all the operators had proper training to even work on this machinery?
We'lre all the safety features built into the machine from the OEM being utilized? Alot of the additional safety features are disabled to make it easier to run. Also no training is needed to have someone press the run button.
We're there auxillary safeties installed? (Light curtains, safety scanners with properly set warning and red zones).
A lot of times machine operators have no control over the work environment or equipment integration and are told to just get the job done with what they have.
As far as I know, the closest thing currently being developed that fits the sci-fi model of 'killer robot' is an autonomous airborne drone programmed to fly to a particular set of GPS coordinates and then kill every human it can recognize with its built-in AI-powered target-recognition system within 1000 meters of those GPS coordinates. This fits into the 'fire and forget' category of munitions - and rumors are that something like this has already been used in Libya. Note this kind of system isn't really subject to jamming as it's not being controlled by a human operator. See:
As far as the effort to regulate LLMs, it seems more about the desire to keep them from generating 'bad ideas' or teaching people how to do bad things (which they can already learn how to do in any library or on the internet already, incidentally). Unaligned LLMs are not likely to go around killing people, although they could be very offensive.
But, beyond mashing people's emotional buttons, how exactly do realistic terminator robots differ, morally, from many of the robot-free (but lethal) military technologies which were widely used a century-ish ago? - mass bombing of heavily populated areas, artillery barrages, minefields, etc.? Germany alone produced almost 2 million of their S-mines for WWII.
With autonomous public robots, it becomes fair to question their programming. For instance I was derided on this venue for unreasonably introducing the 'trolley car' philosophical puzzle unnecessarily into the mix.
But already programmers have intentionally written code to react when facing this dilemma. We just don't know what code they put in there.
Example: a truck stops in front of the autodriving car, too quickly for it to stop in time. It knows this; it understands its vehicle handling to many decimal places.
So what should it do? Continue futilely braking to a likely-fatal accident when it impacts the truck? Or swerve to avoid?
Swerving may involves other people now: people standing by the side of the road, or emergency vehicles and personnel at work. Even the people in cars behind, auto-driving or not they are part of this. We hit the truck; they hit us. Or we swerve; they hit the truck.
You see? Kill my passenger, or potentially more by saving my passenger.
It's not simple, and not obvious (thus the longevity of the philosophical puzzle). But now robots are making the decision.
We can make laws to force the programmers' hands. Then it's no longer the intentionality of the programmer we have to examine. It's the rules and regulations we put in place, the intentionality of the regulator, of the standards committee. Ultimately of voters.
My humble opinion is, autodriving cars should never leave the roadway. See, the people on the road have voluntarily put themselves 'in the game'. Kids playing by the side of the road are not part of this; they should under no circumstances ever be involved if possible.
Anyway, the trolley car problem. It sucks. It's here.
>Example: a truck stops in front of the autodriving car, too quickly for it to stop in time.
That is physically extremely rare - trucks have lower deceleration and higher breaking distance than high performance passenger cars. Also why TF is not the autodriving car keeping distance allowing it to break in time. The focus should be on not getting into shitty situations - and because AI attention is unlimited, we can do all kinds of sensors and also mesh the cards, couldn't be bored and is vigilant 24/7 it should be theoretically possible.
The specific example is beside the point. There are any number of events that can happen outside of a driver's control, human or computer, that could cause this or a similar predicament, e.g. the truck could have just changed lanes abruptly without signaling
> So what should it do? Continue futilely braking to a likely-fatal accident when it impacts the truck? Or swerve to avoid?
Just speculating, but I'd wager we end up with a suboptimal solution to particular instances like this in favor of a general policy that is a) predictable, and b) minimizes liability. IOW, the car will be programmed to brake rather than swerve, because swerving opens many unpredictable cans of worms, as you've noted. Braking may be the worse answer here, but as a general matter, I think you're going to have an easier time in court arguing that the car did its best to brake to avoid a collision vs. arguing that the fact that it swerved and, say, hit a schoolbus or went over a cliff.
The trolley problem is idealistic and in reality there won't be almost never a either ... or.
In practice, these days most likely a neural net decides what is the best maneuver to avoid serious harm to the people in its car.
As alternative the net could try to enforce rules that "global harm" should be minimized.
And then both options as in the trolley problem are most likely not equal, in terms of probability.
> My humble opinion is, autodriving cars should never leave the
roadway. See, the people on the road have voluntarily put
themselves 'in the game'.
I think this is as close to the "correct" answer for the Trolleycar
problem as one can get. Even when your passenger is the Nobel winning
cancer scientist and the hapless bystanders are "worthless criminals"
(or whatever arbitrary utility standard we suppose society might
choose).
Restricting the fallout to those who choose to use autonomous devices
while totally protecting "externalities" also aligns with market
forces, if you believe in those.
The autonomous car absolutely should kill the passengers, even if a
better utilitarian outcome seems available but involving others who
did not choose to play that game.
And herein lies the current deep fault in all technologies (but
particularly digital ones): The convenience of the few (or even the
majority) is becoming an imposition on the welfare of others who do not
choose or want to play that game. It is, in these terms, quite simply
"tyranny".
Example: a truck stops in front of the autodriving car, too quickly for it to stop in time.
Not sure if it matters for your example, but if that happens to a human, it's his fault for not keeping a safe following distance. It could also be the truck's driver fault if the sudden stop is not justified, but in the case of the robot, it should be programmed to keep the distance.
(As of now) You’re being downvoted by people who don’t leave enough following room on the highway. This is however the correct answer. Collisions need to be avoided by assuming others will do stupid things. Because they will.
However, I wouldn't be surprised of some technologist has been hyping self-driving technology as a way to achieve a magic utopia by increasing efficiency by getting rid of those kinds of safeguards (e.g.) safe following distance), because machines are perfect won't make mistakes.
After all, once we get rid of those pesky humans, shouldn't cars be able to barrel down the highway at 100mph with the following distance of stop-and-go traffic, because of the magic of internet and algorithms?
I agree, but if I understood correctly your previous reasoning, the passengers of the robotic vehicle put themselves 'in the game', not only more than roadside bystanders, but also more than others in vehicles that are driven respecting the rules.
I used to drive to work every day, very busy highway, and faced an annoying dilemma. If I kept a safe distance, the driver immediately behind me would pass me inserting himself in the gap, forcing me to brake and leaving more distance, inviting a new vehicle to pass me. Or I could tailgate and risk being part of the next multiple colision accident. After seen a couple of those, no thanks.
This is the kind of human idiocy that robots were suppossed to avoid. But if they're going to mimic it, what's the point? We will be competing with them and sooner than later, we will see how big companies behind them will outmanouver us, also in courts. So not as optimistic as you about regulation.
It's impossible to have a completely safe following distance. Some off-road hazard can inject itself onto the road without notice. All vehicles would be creeping along the freeway if they wanted perfect safety from such events.
So we can only trust in probability and do the best we can. Robot or human, same issue.
"Safe" is a simplification, but some practices are obviously wrong, like less than the length of two vehicles at 120 kmH.
Some off-road hazard can inject itself onto the road without notice. All vehicles would be creeping along the freeway if they wanted perfect safety from such events.
That's a tricky reasoning line. Safe distance is not intended for those cases. The point is that, given a colision, it won't be propagated.
About congestion, it doesn't seem a problem if you can drive fast enough to make an issue of keeping the distance. Bottlenecks come mostly from exits.
Expecting an autonomous robot to be omniscient and omnibenevolent is BS from cool-sounding Sci-Fi stories and ivory-tower philosophers.
The roads are currently full of human drivers, and ~zero of their driver-training programs include anything resembling trolley problems.
A reasonable programming goal and standard for autonomous robots would be: "Behave like a human of decently-above-average skill, training, and safety-consciousness".
...any case where the lawyer knows there could be Deep Pockets in play. Quite true. And there is a wide gulf between "X would be a reasonable engineering and regulatory standard", and "X is the current standard".
OTOH...
> Example: a truck stops in front of the autodriving car, too quickly for it to stop in time. It knows this; it understands its vehicle handling to many decimal places ... a likely-fatal accident ...
This is not a "well-understood traffic situation", and it may not be physically possible without a crazy assumption or few. The autodriving car has brakes, a crumple zone in front, air bags, etc. Cars generally have much shorter braking distances than trucks, and the autodriving car presumably has quick reactions. Was the autodriving car blindly approaching the truck at already-lethal (even if the truck had not stopped) speeds? Is the truck loaded with high-sensitivity explosives, that a fairly minor collision will set off?
Citation? That auto-driving cars exist. That is not in doubt. They auto-brake for road obstruction; that can be observed in any number of youtube videos. Such experiments are conducted by highway safety boards; Tesla famously failed some of them.
It would be more helpful to post comments that illuminate the issue or introduce new ideas. "You so stupid" is not one of those posts. Try to do better.
In my opinion robots are the up-and-coming problem that will exacerbate existing problems that need to be addressed first. I would like to see his opinion on the existing problem of drive-by-wire vessels or at the risk of sounding like hyperbole what I call Internet-Connected-Death-Machines or ICDM's. Not just automobiles but also aircraft, trains and sea vessels.
Internet connected vehicles plus control of steering, acceleration and braking via the vessels computer is already an existing problem. Some researchers got into trouble some time ago for taking over remote control of a vehicle on a live highway. There are also videos of people mysteriously accelerating and driving into buildings whilst the driver is trying and failing to stop the vehicle. I'm not linking the ring videos of Anne Heche but those can not be unseen.
Before we can secure robots and prevent them from taking people out I think we first have to secure their predecessors of which they are based on. If a trusted partner or random skiddie can execute malicious undocumented codes or add their own malicious code then the game was over before it started in my cranky opinion. Cars and their remote networks get released and patched just like existing internet services and have a long lag time in between patches giving malevolent parties plenty of time to erase drivers. Also missing from these automobiles is a truly-tamper-and-fire-proof "little black box" that records every instruction executed by the car for the last 72 hours, including the undocumented or hidden instructions.
I grew up riding mechanical, carburetor, dumb motorcycles, and every mishap was due to my youth, bad environment, judgement, what have you. However, I now ride a motorcycle with a 6-axis accelerometer, inclinometer, ABS braking, fuel-mapped ignition, and all controlled by the onboard ECU. I can say it has been a disruptive experience. I can disable things when I want to do stupid stuff - burnouts, wheelies, etc. - or leave them on so they can save my tired ass in the rain or a tight turn I've entered too fast. I have felt the cornering ABS kick in once. The throttle by wire is ultra responsive and programmable depending on the mode you are in, and not like the mechanical throttle of my old bikes. The fuel mapping is intelligent and changes under different temperatures and demands. I'd say it has saved rather than killed me. But, I must say, I talk to my moto, Ginger, and thank her for taking care of me on every trip. It makes me feel better ;)
2021 KTM Duke 890R. Decat with dual-titanium Arrow race exhausts, Power Commander, Rottweiler race air intake. Replaced Michelin PC2s with Michelin P5's..Love it!
As for me, I waited until I was sure I wouldn't kill myself to buy a motorcycle, which for me meant "last year, at age 52." Sure, I still gotta worry about everyone else, but that's a risk I've been taking on a bicycle for years already.
Mine is a Royal-Enfield Classic 350, which is perfect for 85% of what I'd want to do -- errands in town, etc -- but is obviously limited to surface streets and slower back roads. It tops out at 70, notionally, but it's not super happy above 55(mph).
We'd dropped to one car during the pandemic, because my wife has traditionally taken the bus to work. I'd have the car during the day for the odd daytime errand. Then she got a better job that required driving to work, and we really really didn't want to buy another car - and I had always wanted a motorcycle... She wasn't super excited, but the price tag helped a lot ($4800US).
It's lovely, and I get comments on it constantly, but after a year I'm definitely starting to think about Next Bike. Top contender is a Bonneville T120.
Schneier is asking for an dedicated, FAA-like Robot/AI governmental regulatory agency.
I do not believe we need one.
Farm implements have been killing us for far longer, and in far larger numbers. (approx. 20 deaths per 100K, out of 2.1mm in agro in 2020. [1]) Yet, we do not have a "tractor overturn death prevention agency (TODPA)" There is already ROPS 1928.51.
In the USA, re-evaluating the OSHA requirements would reduce the number of at-work robot accidents.
THIS. But add in all the accidents with non-"robot" industrial machinery, and mine accidents, and people killed by livestock, and etc. The problem is millennia old. Yes, the regulations do need to be savvy about the more-complex ways that robots and AI's can go wrong. But that is also true of industrial accidents at a large chemical works. If Tank 372 is full, and Valve 8601b has a faulty sensor, and ... and ... and BOOM.
I think Bruce has the right perspective, but I also can’t help but wonder about whether calls to slow the pace of AI research are loud warnings about the barn door being open long after the horse has left.
For example: the Chinese have created a massive surveillance network of CCTVs and drones feeding data into the AI-augmented social credit score system. These are the AI technologies of theirs that we know about. I wouldn’t be surprised if they have lethal drones controlled by AI as a military project. We’ve seen time and time again that the Chinese Communist Party are perfectly happy subverting anything and everything in the service of upholding their own power. It’s a no-brainer to assume that AI ethics are just one more line item in the laundry list of the things they’d subvert.
I wonder about the world’s capabilities to counteract such a ruthlessly amoral threat without a deep understanding of the technologies they use, and how to counteract them.
> The 1905 Grover Shoe Factory disaster led to regulations governing the safe operation of steam boilers
That seems awfully late?! Germany already had the "Society for the Supervision and Insurance of Steam Boilers" since 1866, and according to https://en.wikipedia.org/wiki/Technischer_%C3%9Cberwachungsv..., "from 1871, membership in such an association exempted them from inspection by a state inspector" - so at that point, there was already a system of state inspection. And all this emulated a system that was already established in the UK, so the concept is even older. But maybe it's the American attitude to regulation that delayed similar regulations for so long in the US?
There is simply not a culture of quality in software, and trying to graft one on to software controlling robots does not get to the root of the problem.
To take just one example, software makers are uniquely able to shield themselves from liability via EULA. I can sell you software for millions of dollars, it can say “reliable, helps route 911 calls” on the tin, but it I make you click an EULA as everyone does you can’t sue me even for the most basic negligent defect in the software.
Indeed, we tend not to even use words like “defect” when discussing software. We use words with more forgiving associations like “bug.”
We do not have any widely used metric of software quality. Even hypothesizing such a measure feels a little absurd in todays climate.
It’s basically still the Wild West in software. Even as the systems controlled by software become more critical and the those critical systems become more widely deployed.
I think some people attempted to get us out of this Wild West during the 20th century, with people like Margaret Hamilton trying to turn software engineering into "real" engineering. I don't really know why we seem to have reverted back to this stage nowadays, I suspect the boom/burst cycles of tech, especially 2001, might not have helped.
It's interesting how public discourse has those blind spots, with people talking about autonomous machines killing people as if it's some danger looming on the horizon. They are already here and have been around for decades, we just call them mines (some of which attempt to filter out unintended targets), missiles (anti-ship missiles were collaboratively and selectively targeting ships since the 80s), and torpedoes. Anti-radiation loitering munitions are effectively autonomous killer drones, and Harpy first flew in 1989.
Wider recognition of the fact would do good to the quality of the discourse. What exactly are we scared of, considering that we had decades of experience with these systems? How much truth is there to those fears? How much of it is shallow media driven emotion vs genuine risk?
> From 1992 to 2017, workplace robots were responsible for 41 recorded deaths in the United States—and that’s likely an underestimate, especially when you consider knock-on effects from automation, such as job loss.
That sentence right there makes me angry. Not at Schneier specifically, but at this blatant lie that somehow automation is responsible for even a fraction of job loss, unemployment, and the deaths that ensued. Wake the fuck up, people: it’s not robots, it’s capitalism.
I was born in 1982. Many people my age still remember being promised that machines would lift the burden of work from us, that we’d all enjoy 3-day work weeks and so on. A huge amount of automation did happen, but the rest, not so much. There are many reasons for this, but a big one is that fewer and fewer people concentrated a bigger and bigger share of the profits. Now when we automate something, it’s not, as we were promised, to reduce the physical or mental burden of workers. It’s to reduce the costs associated with wages.
The only way Schneier’s sentence makes sense is if we can’t change our economic system, but for some reason we can stop or limit automation. But even that isn’t going to work: capital owners are going to continue automating stuff as long as doing so increases their profits.
(Edit: I guess that kind of criticism is not exactly welcome in a startup forum.)
My expectation: It's about the history, present and future capabilities of "robots"/"automation processes" specifically designed to neutralize enemy targets; one of the optimization being, of course: to kill as many humans as possible.
Alas, it's about the unintended dangers of ever sophisticated and fully automated tool use. Is this satire?
Yeah, something heavy on wheels can roll you over, crush you and kill you.
I'm not trying to dismiss any safety regulation but isn't the big elephant in the room one of the main driver of all technology: military use?
Maybe most people gave up on that and try to focus on things in the civilian sphere that they can potentially impact. A "healthy" compartmentalization of sorts.
But for me the contrast is so stark that the examples mentioned in the article are unintentionally comical and forced, even if the underlying questions valid and transferable to military use.
> Large corporations have a tendency to develop computer technologies to self-servingly shift the burdens of their own shortcomings onto society at large, or to claim that safety regulations protecting society impose an unjust cost on corporations themselves, or that security baselines stifle innovation.
You can say that again. Also the first part ("technologies to self-servingly shift the burdens ... onto society at large") is frequently the very innovation they're talking about in the last part.
I’m starting to see the whole “robots killing people” is really just going to be the flagrant deflection of responsibility and blame by the people who deploy them. Just think, smart humanoid “intelligence” will make a wonderful scapegoat.
Wired for War came out in 2009 and if you are new to this topic I would recommend giving it a read. It focuses on offensive military uses rather than civilian "accidents" discussed in this article. I do think there are crossover ethical dilemmas at play with both.
106 comments
[ 5.1 ms ] story [ 237 ms ] threadServes 'em right for not being able to provide value to capital. /s
See: self-driving cars.
The more challenging topic is military/government use of robots in war or citizen control scenarios. Areas like that seem very relevant if you look forward a few decades given the pace of progress of Boston Robotics and AI.
See: Terminator, the movie. Time travel aside... :P
Absolutely not challenging. They will come and will be used. Nothing you can do about it unless you DRM everything. And then of course the government will still have them. Topic closed.
>Interesting that the article focuses on the unintentional killing of people by robots
I doubt that there was ever killing. The first two examples are people trying to do maintenance on plugged machines. If you try to fix moving steamroller by standing in front of the wheel - don't be surprised at the consequences.
The south africa one - it depends - if the turret just misfired in this general direction - is no different than a mine or shell exploding. If it scanned the soldiers, recognized them as targets and then shoot - then it is killing.
This seems inflammatory and highly misleading. I could be wrong, but I highly doubt it: the robot didn't "determine" anything. It had assembly instructions to follow, and it followed them. The worker got in the way and was struck.
Schneier falls into this same error, saying Tesla Autopilot sometimes "ignor[es] red lights, stop signs, and crosswalks." Tesla isn't perfect, but I'm pretty confident there's no code in Autopilot that says, "yeah, I see that crosswalk, but Elon's in a hurry, so let's goooo!" -- okay, maybe in Elon mode. But not in general. So no, Autopilot doesn't "ignore" those items: it misses them.
Assigning intention to robots at this point is pretty much equivalent to assigning intention to a clock because the hands go around. We have a long way to go before we can/should assign intentionality to "robots" this way. Of course, at the rate things are going, "a long way" could happen in the next few years. But it's not happening now. I'll happily consider counter-examples.
The essay suggests "Any regulation of industrial robots stems from existing industrial regulation, which has been evolving for many decades" & that regulations need to be clear about other robots as well. Interestingly enough, if we applied existing regulations to Tesla's Autopilot as we would to a driver who racked up as many fatalities and crashes (736 crashes and 17 deaths in the U.S. since 2019 [0]), they'd be behind bars long ago, let alone never able to drive again.
0 - https://www.caranddriver.com/news/a44185487/report-tesla-aut...
Aat this stage I agree that we are dealing with "blind machines", with no more "intention" than a steam engine.
However, the central argument in Schneier's piece seems to be that robots inherit the intentionality of their creators.
If I create a machine that kills someone, my "intentionality" is fair game to be dissected in court, and ultimately I may be found guilty as if I had killed with my own hands.
That is the kind of high bar I think society expects, and is certainly one that should scare the crap out of robotics developers (aside: I have been one) to ensure the highest standards of engineering discipline and industrial ethics.
It's definitely fair to consider the 'intentionality' of the idiot too.
It's still on leadership if they choose to hire idiots who get hurt. The solution is: don't hire idiots.
If you look historically at manufacturing and the origins of signage and safety training, you'll find them mandated by the government (i.e. OSHA, etc.) because those things weren't always voluntarily provided by the employer. Upton Sinclair opened our eyes to this 100 years ago.
When you run a shop with five experienced people, if each of them has a 1-in-a-million chance of getting seriously injured on any given day, each year you have a 1-in-548 chance of a serious injury. Any sane person would call this risk acceptable.
If you run a shop with 5000 people of equal skill level to those five above, you're talking a couple of deaths or permanent disability injuries per year, just from the numbers. Any rational person would say hey, that's the same risk, why are you mad? But society isn't rational and if any legal entity no matter how large kills or maims a couple of workers a day, there's an outcry and executive heads will roll. So yeah, they try to do better. At this point maybe 'hire smarter people' works?
Now imagine you're running a multinational company with 500,000 people. If you operate like that original shop you'll see a couple of hundred deaths in your first year and then get sued into oblivion or lynched, and remember those were skilled experts. There aren't 500,000 skilled experts available for hire, and most of these jobs are ahem less skilled anyway, so you're hiring people with much higher rates of personal injury. To operate at this level you need to do stuff that seems dumb to the uninitiated, like relying on layers of subcontracting companies for everything while implementing byzantine and ever-shifting safety regulations that each subcontractor must commit to following. Sure, it costs an order of magnitude more, but it's worth it for the severability.
And the accidents that I know of, are where the security mechanism have been tampered with. A proper designed and certified german machine is hard to get injured by if followed protocol (and the reasons are past accidents and suing the manufactor).
The weapons industry (and its lobby organization, the NRA) would like to have a word with you...
So not all that weak, really.
Sure (and thanks for pointing this out) but this isn’t a philosophical piece, it was written for the Atlantic, and he’s being careless with his language.
The second one seems fair: he is describing the result of what the Tesla Autopilot is doing. I do not think that he means to say that the Autopilot is understanding that there is a red light but then it makes a consciuous decision to itgnore it. He is just saying that the robot is malfunctioning, as it should be clear from context:
I do not think there is a more neutral way of describing the events.Even more likely, it’s a manufacturing robot, with no vision at all except maybe for the pieces it is manipulating. It almost certainly has a line on the floor, or a railing even, to keep people out of its way.
In short, it has zero obstacle avoidance. I’d be interested in counterexamples.
> The second one seems fair: he is describing the result of what the Tesla Autopilot is doing. I do not think that he means to say that the Autopilot is understanding that there is a red light but then it makes a consciuous decision to itgnore it. He is just saying that the robot is malfunctioning, as it should be clear from context: "Today’s AI-related robot deaths are no different from the robot accidents of the past. Those industrial robots malfunctioned, and human operators trying to assist were killed in unexpected ways. [..] Malfunctioning Teslas on Autopilot have deviated from their advertised capabilities by misreading road markings, suddenly veering into other cars or trees, crashing into well-marked service vehicles, or ignoring red lights, stop signs, and crosswalks".
He describes Teslas as “no different” from non-intentional machines of the past, and then describes them with “intentional” language.
>I do not think there is a more neutral way of describing the events.
How about, “Industrial accidents happen through lack of human foresight and carelessness. The same is currently true of ‘AI’ products like Autopilot. But the time will come, and probably soon, where AI alignment and intentionality will be part of the conversation. We should prepare for that starting now.”
Not as catchy as “Robot rampage!” But more accurate and, as you say, neutral.
I really really don’t think this is an article about convincing robots to be nice to us (AKA alignment); I think it’s an article about how we need stricter regulations for software that controls physical machines. Doesn’t that sound like a good idea?
I’m not arguing against the underlying point, I’m just disagreeing with how it’s presented.
Reasoning is actually pretty good now. An AI can compose a sonnet on electron degeneracy pressure no-problem but ask it where the red ball is - and really inspect its localization of that red ball and it’s way off. Orders of magnitude of imprecision.
A modern AI can reason through the trolley problem and millions of variants. But it can’t see the person on the track with very high confidence, or be sure that the trolley is going to hit the person or not.
AI systems are quite good at perception in a traditional sense (e.g. determining a red ball is in the sensor receptive field), but once actual cognition about how the red ball relates to the world around it becomes important, current approaches are limited.
I would argue that the first wave of perception is solved much more completely than the ability to reason about complex interactions of objects grounded in the physical world.
Disclaimer: I am a robotics research scientist
A red ball detector that operates at 60hz with a 99.9% accuracy will make an error on average once every 11.5 seconds. I think the issue is not so much the interactions of objects and the world but rather we're just wildly off-base regarding the base performance needed.
Ignoring can mean failing to notice something, as well as noticing but disregarding it. Words can often have multiple similar complementary meanings. I don't think the usage here is incorrect, or even misleading.
You don’t think it’s important to draw a distinction between whether a robot was deliberately programmed to do something malicious as opposed to a bug which caused an oversight? Come on.
https://dictionary.cambridge.org/us/dictionary/english/ignor...
> to intentionally not listen or give attention to:
> to give no attention to something or someone:
Two definitions - one with intention, one without
> words mean things.
Yes, but they often mean multiple things. Very few dictionary entries only have a single definition for a word. You can't use "words mean things" to claim that "this word means only what I say it means, regardless of the other accepted uses it already has".
> You don’t think it’s important to draw a distinction between whether a robot was deliberately programmed to do something malicious as opposed to a bug which caused an oversight?
I do think there are times when it is very important to do that.
I just don't think that "ignore" is the right word to make that distinction, and isn't a distinction the authors were trying to make at that particular time, because if they were they would probably have used another word.
...at least, that's how I read it. It seemed clear to me. ¯\_(ツ)_/¯
https://dictionary.cambridge.org/us/dictionary/english/ignor...
> Definition: to give no attention to something or someone:
> Example 1: They ignored our warnings.
> Example 2: The mayor ignored the hecklers and went on with her speech.
> This seems inflammatory and highly misleading. I could be wrong, but I highly doubt it: the robot didn't "determine" anything. It had assembly instructions to follow, and it followed them. The worker got in the way and was struck.
I was prepared to argue with you, but upon reading the article, I agree that the description you quote makes a huge category error.
I think it's believable that we'll soon get AIs that are advanced enough they can do agent modeling and that some subgoal-maximizer accident could happen where the robots determines "the most efficient way to accomplish <objective> is to take out that human so he doesn't turn me off", in which case the article's poetic description would be correct.
Except the quoted paragraph is talking about an industrial accident in 1981, so the whole "eliminate the threat" presentation is obviously absurd.
> The examples are more about Machines killing people than Robots killing people... The watershed will be when robots are build to kill people and then actually do kill people the intended way, but not necessarily the intended targets.
I presume the GP is talking about the case when the robot just goes "Nah, I'm going to ignore your target and kill these other people over here".
Also sounds like he's blaming the tool, instead of the operator of the tool. I really expected that number to be higher when you account for:
Have all the operators had proper training to even work on this machinery?
We'lre all the safety features built into the machine from the OEM being utilized? Alot of the additional safety features are disabled to make it easier to run. Also no training is needed to have someone press the run button.
We're there auxillary safeties installed? (Light curtains, safety scanners with properly set warning and red zones).
A lot of times machine operators have no control over the work environment or equipment integration and are told to just get the job done with what they have.
https://thebulletin.org/2021/05/was-a-flying-killer-robot-us...
As far as the effort to regulate LLMs, it seems more about the desire to keep them from generating 'bad ideas' or teaching people how to do bad things (which they can already learn how to do in any library or on the internet already, incidentally). Unaligned LLMs are not likely to go around killing people, although they could be very offensive.
Military organizations globally are now working at breakneck speed to create *literal* “terminators.”
I know cause I make them.
Replicator is skynet v2…we tried it and weren’t ready with MDC2 in the teens ABMS etc are the follow on attempts.
Every “unmanned” “cooperative” etc… system are the terminators
As Bertrand Russell said : “until there is no more war, all technologies will be used for war”
But, beyond mashing people's emotional buttons, how exactly do realistic terminator robots differ, morally, from many of the robot-free (but lethal) military technologies which were widely used a century-ish ago? - mass bombing of heavily populated areas, artillery barrages, minefields, etc.? Germany alone produced almost 2 million of their S-mines for WWII.
Don’t worry, the pentagon is totally incapable of breakneck speed.
But already programmers have intentionally written code to react when facing this dilemma. We just don't know what code they put in there.
Example: a truck stops in front of the autodriving car, too quickly for it to stop in time. It knows this; it understands its vehicle handling to many decimal places.
So what should it do? Continue futilely braking to a likely-fatal accident when it impacts the truck? Or swerve to avoid?
Swerving may involves other people now: people standing by the side of the road, or emergency vehicles and personnel at work. Even the people in cars behind, auto-driving or not they are part of this. We hit the truck; they hit us. Or we swerve; they hit the truck.
You see? Kill my passenger, or potentially more by saving my passenger.
It's not simple, and not obvious (thus the longevity of the philosophical puzzle). But now robots are making the decision.
We can make laws to force the programmers' hands. Then it's no longer the intentionality of the programmer we have to examine. It's the rules and regulations we put in place, the intentionality of the regulator, of the standards committee. Ultimately of voters.
My humble opinion is, autodriving cars should never leave the roadway. See, the people on the road have voluntarily put themselves 'in the game'. Kids playing by the side of the road are not part of this; they should under no circumstances ever be involved if possible.
Anyway, the trolley car problem. It sucks. It's here.
That is physically extremely rare - trucks have lower deceleration and higher breaking distance than high performance passenger cars. Also why TF is not the autodriving car keeping distance allowing it to break in time. The focus should be on not getting into shitty situations - and because AI attention is unlimited, we can do all kinds of sensors and also mesh the cards, couldn't be bored and is vigilant 24/7 it should be theoretically possible.
Just speculating, but I'd wager we end up with a suboptimal solution to particular instances like this in favor of a general policy that is a) predictable, and b) minimizes liability. IOW, the car will be programmed to brake rather than swerve, because swerving opens many unpredictable cans of worms, as you've noted. Braking may be the worse answer here, but as a general matter, I think you're going to have an easier time in court arguing that the car did its best to brake to avoid a collision vs. arguing that the fact that it swerved and, say, hit a schoolbus or went over a cliff.
In practice, these days most likely a neural net decides what is the best maneuver to avoid serious harm to the people in its car. As alternative the net could try to enforce rules that "global harm" should be minimized.
And then both options as in the trolley problem are most likely not equal, in terms of probability.
I think this is as close to the "correct" answer for the Trolleycar problem as one can get. Even when your passenger is the Nobel winning cancer scientist and the hapless bystanders are "worthless criminals" (or whatever arbitrary utility standard we suppose society might choose).
Restricting the fallout to those who choose to use autonomous devices while totally protecting "externalities" also aligns with market forces, if you believe in those.
The autonomous car absolutely should kill the passengers, even if a better utilitarian outcome seems available but involving others who did not choose to play that game.
And herein lies the current deep fault in all technologies (but particularly digital ones): The convenience of the few (or even the majority) is becoming an imposition on the welfare of others who do not choose or want to play that game. It is, in these terms, quite simply "tyranny".
Not sure if it matters for your example, but if that happens to a human, it's his fault for not keeping a safe following distance. It could also be the truck's driver fault if the sudden stop is not justified, but in the case of the robot, it should be programmed to keep the distance.
After all, once we get rid of those pesky humans, shouldn't cars be able to barrel down the highway at 100mph with the following distance of stop-and-go traffic, because of the magic of internet and algorithms?
Regardless of fault there's the issue of programming are response, which was the point here.
I used to drive to work every day, very busy highway, and faced an annoying dilemma. If I kept a safe distance, the driver immediately behind me would pass me inserting himself in the gap, forcing me to brake and leaving more distance, inviting a new vehicle to pass me. Or I could tailgate and risk being part of the next multiple colision accident. After seen a couple of those, no thanks.
This is the kind of human idiocy that robots were suppossed to avoid. But if they're going to mimic it, what's the point? We will be competing with them and sooner than later, we will see how big companies behind them will outmanouver us, also in courts. So not as optimistic as you about regulation.
So we can only trust in probability and do the best we can. Robot or human, same issue.
Some off-road hazard can inject itself onto the road without notice. All vehicles would be creeping along the freeway if they wanted perfect safety from such events.
That's a tricky reasoning line. Safe distance is not intended for those cases. The point is that, given a colision, it won't be propagated.
About congestion, it doesn't seem a problem if you can drive fast enough to make an issue of keeping the distance. Bottlenecks come mostly from exits.
The roads are currently full of human drivers, and ~zero of their driver-training programs include anything resembling trolley problems.
A reasonable programming goal and standard for autonomous robots would be: "Behave like a human of decently-above-average skill, training, and safety-consciousness".
And, this argument doesn't address at all what to do about this well-understood traffic situation and the programming we will accept.
...any case where the lawyer knows there could be Deep Pockets in play. Quite true. And there is a wide gulf between "X would be a reasonable engineering and regulatory standard", and "X is the current standard".
OTOH...
> Example: a truck stops in front of the autodriving car, too quickly for it to stop in time. It knows this; it understands its vehicle handling to many decimal places ... a likely-fatal accident ...
This is not a "well-understood traffic situation", and it may not be physically possible without a crazy assumption or few. The autodriving car has brakes, a crumple zone in front, air bags, etc. Cars generally have much shorter braking distances than trucks, and the autodriving car presumably has quick reactions. Was the autodriving car blindly approaching the truck at already-lethal (even if the truck had not stopped) speeds? Is the truck loaded with high-sensitivity explosives, that a fairly minor collision will set off?
Citation needed.
> was derided on this venue for unreasonably introducing the 'trolley car' philosophical
Appropriately.
It would be more helpful to post comments that illuminate the issue or introduce new ideas. "You so stupid" is not one of those posts. Try to do better.
The unfounded assumption that I was challenging was the existence of trolley problem solvers being deliberately written. Nobody is doing that.
Are people (and kids) crossing the street at a marking "in the game"?
Internet connected vehicles plus control of steering, acceleration and braking via the vessels computer is already an existing problem. Some researchers got into trouble some time ago for taking over remote control of a vehicle on a live highway. There are also videos of people mysteriously accelerating and driving into buildings whilst the driver is trying and failing to stop the vehicle. I'm not linking the ring videos of Anne Heche but those can not be unseen.
Before we can secure robots and prevent them from taking people out I think we first have to secure their predecessors of which they are based on. If a trusted partner or random skiddie can execute malicious undocumented codes or add their own malicious code then the game was over before it started in my cranky opinion. Cars and their remote networks get released and patched just like existing internet services and have a long lag time in between patches giving malevolent parties plenty of time to erase drivers. Also missing from these automobiles is a truly-tamper-and-fire-proof "little black box" that records every instruction executed by the car for the last 72 hours, including the undocumented or hidden instructions.
And you?
As for me, I waited until I was sure I wouldn't kill myself to buy a motorcycle, which for me meant "last year, at age 52." Sure, I still gotta worry about everyone else, but that's a risk I've been taking on a bicycle for years already.
Mine is a Royal-Enfield Classic 350, which is perfect for 85% of what I'd want to do -- errands in town, etc -- but is obviously limited to surface streets and slower back roads. It tops out at 70, notionally, but it's not super happy above 55(mph).
We'd dropped to one car during the pandemic, because my wife has traditionally taken the bus to work. I'd have the car during the day for the odd daytime errand. Then she got a better job that required driving to work, and we really really didn't want to buy another car - and I had always wanted a motorcycle... She wasn't super excited, but the price tag helped a lot ($4800US).
It's lovely, and I get comments on it constantly, but after a year I'm definitely starting to think about Next Bike. Top contender is a Bonneville T120.
I do not believe we need one.
Farm implements have been killing us for far longer, and in far larger numbers. (approx. 20 deaths per 100K, out of 2.1mm in agro in 2020. [1]) Yet, we do not have a "tractor overturn death prevention agency (TODPA)" There is already ROPS 1928.51.
In the USA, re-evaluating the OSHA requirements would reduce the number of at-work robot accidents.
[1] https://www.cdc.gov/niosh/topics/aginjury/default.html caveat emptor lots of lumped in deaths
How about "Safety Alliance for Farm Equipment Turnover Risk Elimination and Awareness Division (SAFE-TREAD)"?
:D
For example: the Chinese have created a massive surveillance network of CCTVs and drones feeding data into the AI-augmented social credit score system. These are the AI technologies of theirs that we know about. I wouldn’t be surprised if they have lethal drones controlled by AI as a military project. We’ve seen time and time again that the Chinese Communist Party are perfectly happy subverting anything and everything in the service of upholding their own power. It’s a no-brainer to assume that AI ethics are just one more line item in the laundry list of the things they’d subvert.
I wonder about the world’s capabilities to counteract such a ruthlessly amoral threat without a deep understanding of the technologies they use, and how to counteract them.
That seems awfully late?! Germany already had the "Society for the Supervision and Insurance of Steam Boilers" since 1866, and according to https://en.wikipedia.org/wiki/Technischer_%C3%9Cberwachungsv..., "from 1871, membership in such an association exempted them from inspection by a state inspector" - so at that point, there was already a system of state inspection. And all this emulated a system that was already established in the UK, so the concept is even older. But maybe it's the American attitude to regulation that delayed similar regulations for so long in the US?
To take just one example, software makers are uniquely able to shield themselves from liability via EULA. I can sell you software for millions of dollars, it can say “reliable, helps route 911 calls” on the tin, but it I make you click an EULA as everyone does you can’t sue me even for the most basic negligent defect in the software.
Indeed, we tend not to even use words like “defect” when discussing software. We use words with more forgiving associations like “bug.”
We do not have any widely used metric of software quality. Even hypothesizing such a measure feels a little absurd in todays climate.
It’s basically still the Wild West in software. Even as the systems controlled by software become more critical and the those critical systems become more widely deployed.
Wider recognition of the fact would do good to the quality of the discourse. What exactly are we scared of, considering that we had decades of experience with these systems? How much truth is there to those fears? How much of it is shallow media driven emotion vs genuine risk?
That sentence right there makes me angry. Not at Schneier specifically, but at this blatant lie that somehow automation is responsible for even a fraction of job loss, unemployment, and the deaths that ensued. Wake the fuck up, people: it’s not robots, it’s capitalism.
I was born in 1982. Many people my age still remember being promised that machines would lift the burden of work from us, that we’d all enjoy 3-day work weeks and so on. A huge amount of automation did happen, but the rest, not so much. There are many reasons for this, but a big one is that fewer and fewer people concentrated a bigger and bigger share of the profits. Now when we automate something, it’s not, as we were promised, to reduce the physical or mental burden of workers. It’s to reduce the costs associated with wages.
The only way Schneier’s sentence makes sense is if we can’t change our economic system, but for some reason we can stop or limit automation. But even that isn’t going to work: capital owners are going to continue automating stuff as long as doing so increases their profits.
(Edit: I guess that kind of criticism is not exactly welcome in a startup forum.)
It's a welling-up from mysterious depths and only the closer tip is "rational", if at all.
And no, the machine does not access these depths (which is kind of the whole point of machines) and has no "intentions".
Alas, it's about the unintended dangers of ever sophisticated and fully automated tool use. Is this satire?
Yeah, something heavy on wheels can roll you over, crush you and kill you.
I'm not trying to dismiss any safety regulation but isn't the big elephant in the room one of the main driver of all technology: military use?
Maybe most people gave up on that and try to focus on things in the civilian sphere that they can potentially impact. A "healthy" compartmentalization of sorts.
But for me the contrast is so stark that the examples mentioned in the article are unintentionally comical and forced, even if the underlying questions valid and transferable to military use.
You can say that again. Also the first part ("technologies to self-servingly shift the burdens ... onto society at large") is frequently the very innovation they're talking about in the last part.
https://en.wikipedia.org/wiki/Wired_for_War