> The brain is solving a very different problem from most of our neural nets. You’ve got roughly 100 trillion synapses.
Hinton doesn't mention microtubules at all. Do any of the neural net or other connected neuron models incorporate the quantum vibration effects inside the neuron's microtubules?
Not a neurophysiologist. Can you point me to any controlled studies verifying this microtubule quantum effects? If not, a principle based description of the theory and hypothesized effects would be helpful too. Microtubule physiology characterization would also be good. Completely not up to date on neurophysiology so pretty much any references would help. Thanks in advance!
Edit: Nevermind on microtubule resources. Plenty on the physiology references for those.
The quantum vibrations might be crackpottery, but the microtubules might still be information carriers[1]. I'm still curious if any neural net models incorporate these paths.
Nets used for practical machine learning today are too abstract to incorporate them in any meaningful way.
When you reduce the whole "neuron" to a weighted sum wrapped by some nonlinear function, in discrete timesteps, you've killed so much structure in the modelling it doesn't matter if the original information transfer path was supposed to be by membrane depolarization or by modifying cytoskeleton strands.
Arguably if there is a cytoskeleton mechanism for signalling or storage it's captured in some ways in changing the weights and evaluating the net, but not if it had any unique function beyond the abstraction which was already in place from the conventional view.
Studying other structures than the synapses and membranes is prudent, but Hameroff must be somewhere on the line between crackpot and intellectual poseur.
Use your time on some real science instead.
Stuart Hammeroff gave a talk here at Caltech about them last year.....which surprised me a bit. It is still considered far afield from mainstream neuroscience, however, quantum biology is something many are seriously thinking about.
There are definitely a few non-crackpots interested in quantum microtubules, but it seems very far from mainstream.
It's conceivable that quantum effects could matter in some way on very small scales, and that doesn't really have anything to do with quantum mysticism.
Statement 1:I’ve always been worried about potential misuses in lethal autonomous weapons. I think there should be something like a Geneva Convention banning them, like there is for chemical weapons.
Statement 2: You should regulate them (AI) based on how they perform. You run the experiments to see if the thing’s biased, or if it is likely to kill fewer people than a person.
The intention of utilizing AI in weapons systems is to reduce human error in their application. So is it unethical to implement an "AI" in a weapons system that kills fewer people than a person would?
Seperately, I am curious why Hinton is not talking more about state representation and prediction, as almost uniformly across DL/RL researchers this is agreed upon as the next step forward. Hinton certainly is working with Richard Sutton on his RL programs at U Alberta. Sutton believes that the next step in AI systems is state representation and prediction and gave a quick talk on it recently [1].
That feels like a question of decision making vs execution. I think most people would prefer we don’t let machines choose who to kill. We are probably ok with AI guided projectiles towards selected targets.
The intention of utilizing AI in weapons systems is to reduce human error in their application. So is it unethical
Reducing human error is one possible benefit of AI weapon systems but there are numerous drawbacks. AI weapons could be programmed to violate laws and commit atrocities on an incomprehensible scale. They could conceivably cause the extinction of human beings or even all life on the planet. They don't even need to be sentient, let alone super intelligent, to accomplish this. If they can be mass produced and armoured well enough to resist most small arms fire, they could easily overwhelm any human armies.
The scariest part of the Terminator films is not Skynet, it's the terminators themselves and they're far more plausible than a godlike super-AI.
Statement 2 is not in regards to weapons systems. It is referencing things like self-driving cars.
You can see that if you include the next sentence.
"ou should regulate them based on how they perform. You run the experiments to see if the thing’s biased, or if it is likely to kill fewer people than a person. With self-driving cars, I think people kind of accept that now."
>With self-driving cars, I think people kind of accept that now. That even if you don’t quite know how a self-driving car does it all, if it has a lot fewer accidents than a person-driven car then it’s a good thing.
Unless there is a condition in the environment that causes a systemic failure involving large number of self-driving cars at the same time. This happens all the time with other types of software. People shouldn't assume that machine-learned models are magically different and ignore all common-sense rules we apply to safety-critical applications. In your daily job, would you be willing to bet your entire car on a black box piece of code that seems reliable, because it worked most of the time in the past?
Also, recall that flash crash of the stock market. Imagine something like that, but with lots of cars on a highway. If you don't understand "how a self-driving car does it", you are unlikely to install appropriate safeguards.
PS: Please don't insult your intelligence by replying that we don't understand how people drive either, so it makes any opaque self-driving software okay.
Just look at the airline industry, planes are flown by software 90% of the time. Do you also worry about a global blackout of autopilot software? Seems like a silly thing to get stuck on. Our banking and electrical grid systems are far more prone to failure than anything built in SV.
This is a common conflation of intelligence evaluation criteria which is discussed in detail in Jose Hernandez-Orallo's 2017 book Measure of all minds.
It can be simultaneously true that we cannot describe in a causally deterministic way how a system works while also determining with some confidence that the system is performing as desired. This is true of all human systems.
So if you don't know how people drive, but have enough confidence to let them drive based on performance in evaluation scenarios, then similarly we should be able to let machines drive assuming they perform equally as well as humans over the same contexts without knowing how they drive either.
1) good self driving cars aren’t machine learning models. They leverage machine learning for certain tasks like object classification, but it’s not some black box neural net that learned how to get the car from A to B.
2) you’re describing how a system can crash. Driverless cars don’t share state, besides, I guess, vast scale astronomical and weather conditions. It’s not likely something would cause all cars to crash because of a collective hiccup. There may be a set of circumstances that consistently cause individual cars to crash, but that’s not really the same thing. And we can understand how these events are processed, so unlike people, we can adapt for future repeats of this scenario.
Malicious actions are a concern, but they’re at about the same level of concern as alternative acts of terrorism. Unintentional errors in the things you list don’t feel like they’re huge threats. I would Imagine incidents with at most tens of cars and a relatively low risk at that. Not good, but not a scalable threat.
Some car companies are more conscious about this than others. Tesla has a refreshingly anti-fragile approach, where Waymo is heavily relying on data captured through massive amounts of infrastructure.
systemic failure involving large number of self-driving cars at the same time
This happens to human drivers every year. Snow, freezing rain, fog; these adverse weather conditions can lead to large numbers of collisions at the same time. Expecting self driving cars to never have a problem during rare environmental corner cases is to expect them to be vastly superior to human drivers all the time. If you're waiting for that day before you consent to self-driving cars sharing the road with you, you'll be waiting a long time.
Well, self-driving cars can currently handle none of the aforementioned conditions. That's why they are being tested where those conditions don't exist, and that's why we're about 20 years from widespread adoption. It rains and snows in much of the rest of the world.
It rains and snows in much of the rest of the world.
Yes, I live in the rest of the world and my family has been affected by fatal collisions brought on by adverse weather.
I remain skeptical that self driving cars will be able to handle all of these things. I do think they'll be okay for mundane driving where the biggest risk is driver distractions, however.
I would like to see cars and maybe even roads made smarter in order to make driving safer. If all the cars on the road were part of a wireless network and they could signal to one another when they're braking or turning, then other vehicles could respond more quickly.
Heck, these systems could even be included in phones and pedestrian crossings so that cars could be signalled instantly when a pedestrian steps into the road in order to cross.
I used to think that this kind of system must be the future, but now I can only think of the greatly increased attack surface if my self-controlled car is not only taking directions from visual input and other sensors, but also all sorts of other unknown devices.
The system needs to be sane. The network should not be able to force your car to drive into a wall. Like any safety critical system, it should fail-safe. That means if the data it receives over the network would be unsafe to follow, given information from the other sensors, then it should discount that information as erroneous. This is something humans do all the time when, for example, reading a stranger's intentions.
But people are just too creative when it comes to exploiting systems. What if I force someone’s car to halt just under a bridge and then accidentally drop another car on top of it. Oops, a terrible ‘accident’! I realise this is a bit ridiculous but some other people are way more creative than I in this way.
What if I point a gun at someone and shoot them? As usual, the only thing that keeps you alive is that there are no motivated individuals that want to kill you. If there are, then they don't need you to drive a self-driving car.
You're ignoring the possibility of a mass-exploit, meaning the kind of cheap exploit that affects multiple vehicles at the same time, possibly killing a bunch of people, destroying infrastructure, or stopping the flow of traffic. Your typical adversarial "single pixel" change that makes your network believe unreasonable things, but in the real world. The existence of such exploits has already been demonstrated.
I think there is a potential for them to exceed human performance in all driving conditions. Machines can perceive and react much faster than humans, and once the perceptual and control problems are solved, all of them will become uniformly "good" at driving in difficult conditions.
BUT (and there's always a "but"), the problem is exponentially hard, and it can't be solved with today's technology.
I agree with you that widespread adoption will require modification of roadways, and potentially also segregation of autonomous and human-controlled traffic streams.
>these adverse weather conditions can lead to large numbers of collisions at the same time.
Cars sliding out of control on a snowy road is not a systemic failure.
If you have an entire fleet of self-driving cars using the same (or similar) technology, the exact same condition could cause all of them to misbehave in the exact the same way. This could happen simultaneously with a large number of cars. Or could be something that propagates in a cascading manner. (A car breaks, turns and spins out of control. The next car detects that, breaks, turns and spins out of control, etc.)
If you have no clear idea how your driving software works, how are you going to make sure this doesn't happen?
People have common sense, variability and adaptive learning to prevent things like that. Moreover, our traffic system, our cars and our driver education have been co-evolving prioritizing safety for the last 100 years. (Heck, some of it started with horses and carriages.) The complexity and size of the system have been gradually scaled up for just as long, and it was designed and refined to be used by humans.
It's astounding how many of AI enthusiasts fail to see these issues and differences. Traffic system isn't a board game.
> If you have no clear idea how your driving software works, how are you going to make sure this doesn't happen?
A lot of people don't know how typing "www.google.com" into their browser works but DNS "just works" for them. There is software in nearly every part of our lives that could systemically go wrong in catastrophic ways - we do our best to plan and prevent them.
> People have common sense, variability and adaptive learning to prevent things like that. Moreover, our traffic system, our cars and our driver education have been co-evolving for safety for the last 100 years. (Heck, some of it started with horses and carriages.) The complexity and size of the system have been gradually scaled up for just as long, and it's designed and refined to be used by humans.
Isn't that incredible? The complexity goes up: we adapt. As self-driving cars are introduced we adapt and evolve and incorporate these changes in technology, the driving "environment", security, safety measures, and laws and regulations.
> Traffic system isn't a board game.
I didn't see any efforts at self-driving vehicles just say "ahhhh! this driving stuff is so easy!" Of course it comes with huge difficulties, an enormous amount of complexity, a baffling amount of data, a ton of edge cases, and now a wealth of tribal knowledge on the subject.
When considering these things think of the systemic benefits as well. There was a story on the Google self-driving cars in Mountain View not taking a turn when the passengers of the car expected it would. Turns out that the IR sensors on the vehicle saw that a jogger behind a wall of hedges was about to enter the intersection to cross while jogging. Systemically implementing that change across a fleet of cars would then save untold numbers of injuries or lives.
> Turns out that the IR sensors on the vehicle saw that a jogger behind a wall of hedges was about to enter the intersection to cross while jogging.
Some cars today already have IR sensors (night vision systems) installed in them, and include pedestrian detection monitors as well. Cars don't have to be full on self-driving to have a large increase in safety and accident prevention.
To guard against the risk of a bug/design flaw affecting all vehicles simultaneously, I would include an independent "dumb" override that was too dumb to drive but could stop all vehicles as soon as any of several TBD conditions occurred, regardless of what the primary "smart" system was telling the vehicle to do. It wouldn't be "smart" AI, but just something listening to both the AI's smart sensors and it's own, independent, simple sensors, and if ANY of them reported a listed condition, it would override the smart system and stop traffic.
Human intervention would be required to enable the vehicles to move again.
After some real-world experience, I would want to give the emergency stop system the ability to release the hold on one layer of vehicles at a time, allowing the smart AI to drive again but at a speed limited by the "dumb" system to nothing faster than human walking speed, enabling them to clear the area. If they cleared without further incident, the next layer would be released until the whole stoppage either cleared itself or was restopped because something was still wrong.
Self driving cars do not represent a correlated risk. The situation you describe is completely fanciful. There is no plausible circumstance in which a large number of cars would simultaneously crash.
These kinds of crashes generally have the some root cause: everybody is traveling too fast for conditions, and when one thing goes wrong, the end result is that you get a chain reaction of everyone accreting into a single crash. If self-driving cars are capable of making the same misjudgements, then these kinds of crashes absolutely can happen.
Sure, but you can't criticize SDC's by saying that they have the same failure modes as humans. The comment I was referring to was suggesting that they'd have some sort of mass failure mode, in which some substantial percentage of all operating SDC's in an area would crash simultaneously. That is clearly not going to happen.
Yes outside of a few limited areas (mostly controlled access highways) and benign weather conditions we're going to be waiting a long time for level 4+ self-driving cars.
> would you be willing to bet your entire car on a black box piece of code that seems reliable, because it worked most of the time in the past
For the average non-technical person, this is pretty much their relationship with all technology.
I don't think it's exclusive to tech either. I know lots of people that have absolutely no idea how the mechanical side of a car works, but they trust their lives to one at 100km/hr because it has worked generally pretty well in the past.
> I know lots of people that have absolutely no idea how the mechanical side of a car works, but they trust their lives to one at 100km/hr because it has worked generally pretty well in the past.
Car safety has made great progress in the past 10 or so years. People don't have to know how they work under the hood as long as designers and engineers already built safety into them and these designs have been heavily tested by third parties (e.g. Euro NCAP, IIHS, etc)
Self driving cars will have to prove their safety, especially in extreme weather and non paved roads.
Definitely! Everyone here agrees that they need to prove themselves effective, and there is more work to do there. I think Geoff's point that the parent poster disagrees with is that it isn't necessary to completely understand every single aspect of the model's weights in order to be able to empirically validate it as safe.
Every year something like 30,000 people are killed in car crashes in the US. Say the introduction of self-driving cars cuts that number in half and also say that of those 15,000 deaths, 2/3 of them are due to software or system errors, that's still progress, isn't it?
I think your concern here is less about flaws in the AI training and more about scale issues when you have a million cars with the same brain and thus the same flaw. I share your concern. Human brains have lots of flaws too, but at least every human driver has a different brain, so some fraction of mistakes cancel out.
But another way to look at this is the fact that "cloning" the same AI brain across multiple cars is equivalent to a single AI brain controlling many cars. And that does have human analogies where you have a single person who has a dramatic amount of power. Any flaws in that human's reasoning can be disastrous.
Trump having access to the nuclear launch codes comes to mind.
In practice, the way we mitigate this is by surrounding that human with advisors and other brains to try to help shore up deficiencies. We could do something similar with AI. Have a couple of differently-trained models and compare their outputs. If they disagree wildly, then try to gracefully shut down the system.
The vast majority of failures in automobile usage today are caused by human conditions that will never affect a machine. Alcohol, drugs, fatigue, social distractions, aggressive driving.
Google something like "top causes of automobile accidents". No list I found had exogenous causes (rain, snow, etc) in the top 5. For machine driving to top that isn't going to be that hard given that we are our own worst enemy on the road.
This is still a biased argument. Once it scales up, you'll be able to google "top causes of self-driving car accidents." They will have their own flaws and quirks that we don't anticipate yet, and we don't have any good reason to believe the death toll will be inherently less with computers than with people. Plus, there will be exogenous causes of accidents that are far down on the list for humans, but for AI it might represent a much higher risk and this changes the nature of that list entirely.
There just aren't reliable statistics on what the safety of this tech will be at this point in time. All we have is extrapolations with lots of bias and little independent review. Maybe it will eventually be worth it, but that experiment will come with a great many human deaths and we can't simply brush aside the ethics of that. I personally think it won't prove to be safer than humans for many decades. This is far more of a complex problem than a lot of the developers think, and I believe there's a case of engineering arrogance going on here as well.
On the surface it seems like we have something very close to a viable product, but the remaining little details and issues are important and represent an enormous amount of work, where we are going to see diminishing returns and enormous difficulty making a system that is flexible and robust enough to handle all the required situations more reliably than the average driver on its own for thousands of miles, scaled up to millions of units around the world. They will be good enough for niche applications, but the dream of a general self-driving infrastructure is pretty far off.
This is a straw man too. We don’t flip a switch and suddenly all cars are driverless, and then we discover that they can’t handle all situations and everyone dies. We slowly scale it out to environments where it is provably safe. We will not see “top causes of self driving car accidents” because known conditions will merit additional development or restriction on usage. What kind of thing could conceivably make that list? Something like snow doesn’t fit, because they won’t run in the snow unless they’re safe to run in the snow.
You don’t need to be better than humans for all situations. You need to be safe. Humans are generally well above that threshold, except for when they’re drunk, high, tired, distracted, or otherwise stupid. It doesn’t really make sense to compare deaths per mile for cars vs attentive humans in stable conditions. Both can honestly be reduced to the point at which they’re practically 0 unless it’s the victim’s fault. We do need to be attentive to edge cases that cause cars not to be safe, but we can also observe the overall pattern and draw conclusions on how likely unknown edge cases are to be encountered.
So, this reminds me of a nuclear meltdown vs. the constant pollution form a coal power plant nearby a city. Even with one of those single, ugly, massive failures they could still very well cause fewer deaths.
You could divide this into two kinds of failures: one local to a physical area (such as a traffic light that somehow always looks green to the self-driving cameras) and global ones (such as virus, or a bug like a GPS time rollover causing all cars to do something crazy simultaneously.)
Local ones seem unlikely to cause casualties larger than events that fool large numbers of human drivers, like sinkholes or malfunctioning traffic lights or black ice on a freeway. Some of these will probably happen, and the systems will be improved, and they'll happen less and less often.
Global ones could be very large, and there are several instances of global, cascading failures in other systems like telephone or data networks. There are known ways to prevent many classes of failures, but getting to 100.000000% is hard.
Also there's a lack of compelling evidence that self-driving cars have a lot fewer accidents than people do, and this argument is always worded in a deceptive way to kind of quickly shoehorn that assumption in and hope nobody notices. Even measuring this metric reliably is hard, and it's not helped at all by the fact that most of these cars in the real world rely on a human operator as a backup. How do we separate their responsibility? And where's the robust, independent review?
Any system that requires a human operator to passively observe a boring process for hours at a time, yet snap to attention immediately and without warning to take control and prevent human deaths is simply insane. That's the diametric opposite of how a safe system should be designed. If the human is required for safe operation, they should just be driving at all times to maintain vigilance. I don't see people talk about that much. The assumption seems to be that AI will quickly get better and the human back-up will not be needed--that the human is kind of a temporary workaround. Well I've seen plenty of engineering projects where the temporary fix becomes a long-term pain in the ass because the difficulty of the real solution was massively underestimated and under-budgeted. We might get stuck with these terribly designed AI-human hybrid systems for many decades as AI keeps getting pushed into the horizon and we slowly realize that this is one of those hard problems that doesn't progress the same as most other tech does.
There are pure driverless systems out there, but they don't cope well with many of the incongruities and fringe situations which are an unfortunate reality on the open road in most of our country. I think they will be confined to niche uses for a long, long time.
Good point. The answer is probably that nobody really cares about safety, they just want to have their cool toys right now. If people really cared about safety, they 'd be advocating for the only certain way to reduce car accidents: fewer cars (or in other words, more public transport, more trains, etc). Except that wouldn't make sense because the whole point is to sell more cars. Self driving "safety" is just the latest marketing gimmick.
Despite your warning, I think it's important to say that we don't understand exactly how the brain works. Yet, we still allow people to drive. To know whether we should trust someone to drive, we created a test (driver's license test).
I don't see why this would be any different for deciding whether computers should be allowed to drive - we can just make sure self-driving cars pass a road test. We could even do better than driver's license tests and have a human driver at the wheel during the test just in case.
we don't just let people drive because they can pass a road test. There's probably 12 year old's that can pass a road test with enough training.
We let people drive because we know that they are fully capable agents able to make decisions even in case of failure. In other words, they are responsible, reliable, fully cognisant individuals, far beyond the technical ability of operating a machine.
A self-driving car is like a parrot trained to speak. Not only is it not an agent aware of its larger surrounding and potential implications of getting into unknown situations. It also has no real understanding of what driving is at all, it's even stupider than a pet. And because its software is usually deployed to thousands if not millions of devices at the same time, there is little to no variance to protect from extreme failure.
> We let people drive because we know that they are fully capable agents able to make decisions even in case of failure. In other words, they are responsible, reliable, fully cognisant individuals, far beyond the technical ability of operating a machine.
I wish I could share your optimism in humanity, but what you just claimed sounds far from being true universally.
Take your typical driver and throw in a failure situation they have never trained before (because, who has time to train them for every possible, or evens few likely failures), and it is not likely to end well.
>Take your typical driver and throw in a failure situation they have never trained before (because, who has time to train them for every possible, or evens few likely failures
Humans have a base line of common sense reasoning and the ability to create coherent models of the world. A human driver isn't (rare exceptions aside) going to drive up an impossible street or try to dodge a street sign mistaken for a kangoroo in West Germany.
There's nice examples of how an ML algorithm can be tricked into all sorts of pathological behaviour by choosing adversarial images, there's no baseline of reason to fall back on.
Imagine a model that has one of these fatal edge cases baked in and a thousand cars suddenly mistake a flash of light for a u-turn. Sure, one or two human drivers out of a ten thousand might drive that irrationally on occasion, but compared to a machine we can usually cope with this.
And I don't even want to imagine what happens if this turns malicious. Imagine a bad actor on a bridge above a highway with some artifact that messes automated cars recognition up and the developers forgot to put some hard fail-safe in for that piece of road.
It's worth noting the downsides to this agency too, though: Agency is what makes the stereotypical male 20yo driver speed and drive dangerously to show off to his friends, which leads to a very large number of accidents.
It's unclear what accidents this agency avoids though. I can imagine a few situations where it helps (avoiding a terrorist attack by speeding away from the scene?). But in most cases the default operation for a self-driving car will be to stop as quickly and safely as possible in unusual circumstances, and it's unclear how often this is the wrong decision.
> PS: Please don't insult your intelligence by replying that we don't understand how people drive either, so it makes any opaque self-driving software okay.
I won't insult my intelligence; I'll ask why you trust people even though you don't know how they work and you know that at a high rate they intentionally impair themselves before driving?
Recall that time when a human trader lost a billion dollars?
Remember when the economy tanked because everyone in finance used Black-Scholes model that seemed reliable, because it worked most of the time in the past?
> Unless there is a condition in the environment that causes a systemic failure involving large number of self-driving cars at the same time.
This happens all the time already: there is a condition in the environment (eg when it snows in Seattle) there is a systematic failure involving large numbers of people driven cars at the same time.
People just don’t have experience (erm, “training”) with the weird unusual environmental condition and fail to deal with it. Not surprising that computers would have similar problems.
> PS: Please don't insult your intelligence by replying that we don't understand how people drive either, so it makes any opaque self-driving software okay.
But we actually do know how well people drive. We have data on failure rates for sure. We don’t know how they drive, that is completely opaque.
"You should regulate them based on how they perform. You run the experiments to see if the thing’s biased, or if it is likely to kill fewer people than a person. With self-driving cars, I think people kind of accept that now"
I don't. In my opinion, we are all getting carried away with ourselves.
Google or not, I challenge anyone who makes a statement including the phrase "...more like brains", to define that statement accurately. In response, you will likely hear all sorts of things that sound like we know what we are doing and are very clever, but.
If we can't even ask the questions properly, I doubt we will get deliberately valuable answers. Sure, we may luck out, with AI researchers stumbling across random combinations (layered AI) that do produce useful results hap-hazardly, but I would never rely on it.
Eg. I rely on software running in an aircraft, because:
a) it works, by design, not trial and error.
b) it is verified.
c) we know how it works.
unless you have (c) you cannot achieve (b), with any degree of certainty. Experimenting until confident is not good enough for me for anything important.
Brains fail at times. Computers should not make mistakes.
Even if you knew how a brain works, and even if its possible to replicate it, please don't make computers like this. Software fails enough on its own without any help, even when we supposedly understand it.
With this AI stuff, even if we knew where we are going, we don't know how to get there. And even if we did get there (wherever that is), in the long run, AI is not going to be used for the good.
As always with humanity, we will find ways to use AI to extract value out of other people. This is what drives business.
No thanks, but it is interesting for fluff. A bit too voodoo for me.
> Brains fail at times. Computers should not make mistakes.
I don't think this is a reasonable position to take. We are going to have to accept that if we want computers to do the things brains can do, they will also make mistakes like brains do sometimes. It's not reasonable to insist that the error rate is exactly zero. Besides, if you consider the computer as a complete system including the hardware and human-computer interface, it's clear that no complete system can ever be error-free, even if the CPU executes its instructions faultlessly.
As long as it's as good or better than a human, it's good enough to be useful. Even with "fallible" AI techniques, we should be able to make systems that are much more reliable than humans.
"...if we want computers to do the things brains can do..."
That's my point though. I don't want computers to do what my brain does, mainly because I have no idea what my brain does, nor does anyone. Sure, some people think we do, but we don't.
It also seems silly to try to engineer something that we know may fail (by design), on the probability that it may haphazardly produce better results (even if this is most of the time). That may be useful for things like search results, where the outcome is not critically important.
Please excuse the example, but I would be happier passing away in an aircraft accident knowing it was because although the system was designed not to fail, we did our best in terms of understanding the systems in place. That would be my definition of "sh&t happens".
If I passed away because the aircraft "AI" took an untested path of execution (there are too many to verify, unlike most regular "dumb" software), I would be less happy.
Maybe there is a balance, Eg verifiable software for the important stuff, AI fluffery for the less so. But in that case, we should not get carried away (eg. self driving cars). Remember, planes could fly themselves completely for a long time, and we still have 2 pilots for a reason.
OK, that's a valid position to take, and I'm sure some people agree, but I and many other people do want computers to do what our brains do. And it is pretty likely to happen.
> I would be happier passing away in an aircraft accident knowing it was [not AI]
If I was in an aircraft accident caused by human pilot error while pilot AIs with 1/100 the error rate of humans were blocked from deployment because they don't meet your "zero errors" requirement, I would not be happy.
"...I and many other people do want computers to do what our brains do" (emphasis, mine).
I hear you, but my point is "what do our brains do?"
In the case of driving, if the answer is "drive a car", then that is a bit like "brexit means brexit" :) The definition of the thing is the outcome of the thing.
Instead, if you try to can decompose the question into a more useful form, one that can drive a specification of software, then progress can be made. However, all that has done, in that case, is a description of the outcome, not the process. (And incomplete, due to the complexity).
It is at this point where my problem with all this AI stuff starts :)
The lack of understanding of the process, rather than the outcome is why I call it voodoo, and anyone talking different is trying to impress, in my opinion.
Generally our brain is composed of a bunch of different subsystems and networks that accomplish varying tasks or objectives, and communicate the results with each other. There's a lot of data sharing going on, as well as reinforcement of known successes thanks to one subsystem known as the reward center.
While we don't understand how our brain works to the bits and bytes, our knowledge and understanding of how our brain functions is Growing. A fuller understanding of how our neurotransmitters work on a chemical/biological level, and an effective way to model it, will likely result in interesting advances in the field of AI
I chat with my neuroscience friend a lot, and the overlap I see between how he describes how the brain works, and general computer science concepts is surprising.
How many passengers died in the USA last year on Part 121 flights due to aircraft accidents caused by human pilot error? When do we expect to develop pilot AIs with 1/100 the error rate of humans? How will those pilot AIs cope with unexpected environment conditions or mechanical failures which aren't programmed in advance?
Obviously this is a hypothetical example and I cannot predict the future. If you want to talk seriously then take self-driving cars, where humans kill over a million people per year and we are very likely to have AIs that would kill less than that, but still a nonzero amount, within 20 years.
In my view it would be unethical to stop the deployment of such AIs on the grounds that we don't "understand" them well enough or can't predict their behavior in every hypothetical circumstance, because we can still test them well enough to know with statistical certainty that they will save many, many lives.
Let's first see a prototype of such an AI for level 4+ autonomous driving on the majority of roads worldwide. Then we can quantify the potential benefits and have a meaningful discussion about the ethics of deployment. Until then it's premature and pointless since we don't know what the actual parameters will be.
Estimates about availability have consistently been overly optimistic. This may be a case where the first 99% of the work takes 99% of the effort, and the last 1% takes the other 99% of the effort.
Absolutely. Want a computer that works like a brain? Obviously we need to know how a brain works first. We've certainly made huge strides compared to what we used to know, but as far as I'm aware we are still staggeringly far from a working model. Maybe we've increased our knowledge 2000% since the turn of the twentieth century or whatever, which sounds great and really is great. But to look at that from the other side--what if we're only 0.02% of the way to an adequate, functioning model of the human brain? We shouldn't be getting a big head just yet.
These are the kind of problems which have profound amounts of complexity lurking in every unexpected pitfall. We are going to discover that it's a lot more of a tangle than we thought before it becomes simple again. I think most of us here know a thing or two about that sinking feeling when you're attacking a problem and discover that what seemed like a minor pothole is actually a wormhole leading into an entire universe full of waist-high bullshit with submerged landmines, but this problem will be in another class entirely. Figuring out the brain is a species-level problem and it's going to be VERY long and arduous.
I also resent the article's assertion that "if an AI is a safer driver than a human, who cares if it's a black box?" Fallacious argument number one around self-driving cars. Hear it all the time, almost always from laymen. It's a nice way to sneak in the assumption that AI will be better than average human drivers any time soon. The way that metric has been measured has been very deceptive so far, and no AI has come close to matching human safety and robustness. Not in real life applications in the field at scale. Not even close. There's been shenanigans like discounting all driver interrupts, which by nature disguises any flaws of the AI and offloads that responsibility on a person as a sort of PR heat-sink.
We don't give ourselves enough credit for how well we handle driving. Reality is chock full of exceptions, quirks, and legacy systems that aren't going to go away for many decades, and compared to the progress we've made on AI so far it's going to be many, many times more work for us in the future to deal with those niggling details. The devil's in there, he really is.
He did, he specified the synapses are like weights. I'll extend that a bit and say the collection of ion channels on the neuron's membrane, which regulate the surface potential, will act as the classifier. They form a non-linear classifier (voltage gradiant dissipates with r^2) but still a classifier.
> GH: No, there's not going to be an AI winter, because it drives your cellphone. In the old AI winters, AI wasn't actually part of your everyday life. Now it is.
Oh good; although, that predicament about publishing AI papers seems pretty bad.
There's something interesting about AI winters: they begin with researchers claiming incremental science (say, heuristic tree search) as Whoa Artificial Intelligence and end up with some pretty complex technology totally taken for granted (car routing apps). In this cycle, we're in love with Computer Vision -- at the other end of the tunnel we might be stuck with omnipresent facial recognition and nada-de-nada of Jeff Hawkins and Ray Kurzweil's dreams.
We place too much emphasis on learning and not enough on designing AI network topology. No matter how hard you try you’re not going to get a chimpanzee to write novels. They have different brain architectures.
We do have some kind of shared understanding when it comes to humans though, especially within a culture and even more so within a limited context like driving a car.
We don't generally expect someone to freak out and start treating a car in sunlight as a house because they've seen a lot of houses in sunlight lately.
The difference between a human and ML is that a human has a clue, demanding perfect introspection is missing the point.
This borders on dishonesty if you ask me, he should know better.
>> People can’t explain how they work, for most of the things they do. When you
hire somebody, the decision is based on all sorts of things you can quantify,
and then all sorts of gut feelings. People have no idea how they do that. If
you ask them to explain their decision, you are forcing them to make up a
story.
Manager: Why did you delete the production database in the middle of a
work-day, no less?
DBA: My decision was based on all sorts of things you can quantify, and then
all sorts of gut feeling. I have no idea how I do that. Do not ask me to
explain my decision, or you are forcing me to make up a story.
Manager: But, you crashed our client's entire business for a whole day!
98 comments
[ 1.7 ms ] story [ 161 ms ] threadHas Professor Hinton ever made a known appearance on HN?
Hinton doesn't mention microtubules at all. Do any of the neural net or other connected neuron models incorporate the quantum vibration effects inside the neuron's microtubules?
Edit: Nevermind on microtubule resources. Plenty on the physiology references for those.
[1]: https://onlinelibrary.wiley.com/doi/pdf/10.1111/jnc.12621
Arguably if there is a cytoskeleton mechanism for signalling or storage it's captured in some ways in changing the weights and evaluating the net, but not if it had any unique function beyond the abstraction which was already in place from the conventional view.
Studying other structures than the synapses and membranes is prudent, but Hameroff must be somewhere on the line between crackpot and intellectual poseur. Use your time on some real science instead.
Here's one option: http://www.gatsby.ucl.ac.uk/~lmate/biblio/dayanabbott.pdf
It's conceivable that quantum effects could matter in some way on very small scales, and that doesn't really have anything to do with quantum mysticism.
Statement 1:I’ve always been worried about potential misuses in lethal autonomous weapons. I think there should be something like a Geneva Convention banning them, like there is for chemical weapons.
Statement 2: You should regulate them (AI) based on how they perform. You run the experiments to see if the thing’s biased, or if it is likely to kill fewer people than a person.
The intention of utilizing AI in weapons systems is to reduce human error in their application. So is it unethical to implement an "AI" in a weapons system that kills fewer people than a person would?
Seperately, I am curious why Hinton is not talking more about state representation and prediction, as almost uniformly across DL/RL researchers this is agreed upon as the next step forward. Hinton certainly is working with Richard Sutton on his RL programs at U Alberta. Sutton believes that the next step in AI systems is state representation and prediction and gave a quick talk on it recently [1].
[1] https://www.youtube.com/watch?v=6-Uiq8-wKrg
Reducing human error is one possible benefit of AI weapon systems but there are numerous drawbacks. AI weapons could be programmed to violate laws and commit atrocities on an incomprehensible scale. They could conceivably cause the extinction of human beings or even all life on the planet. They don't even need to be sentient, let alone super intelligent, to accomplish this. If they can be mass produced and armoured well enough to resist most small arms fire, they could easily overwhelm any human armies.
The scariest part of the Terminator films is not Skynet, it's the terminators themselves and they're far more plausible than a godlike super-AI.
"ou should regulate them based on how they perform. You run the experiments to see if the thing’s biased, or if it is likely to kill fewer people than a person. With self-driving cars, I think people kind of accept that now."
If the logic can be applied to one system it should be able to be applied to all systems.
I wouldn't say this is necessary the sole intention. AI will be a force multiplier as well.
Unless there is a condition in the environment that causes a systemic failure involving large number of self-driving cars at the same time. This happens all the time with other types of software. People shouldn't assume that machine-learned models are magically different and ignore all common-sense rules we apply to safety-critical applications. In your daily job, would you be willing to bet your entire car on a black box piece of code that seems reliable, because it worked most of the time in the past?
Also, recall that flash crash of the stock market. Imagine something like that, but with lots of cars on a highway. If you don't understand "how a self-driving car does it", you are unlikely to install appropriate safeguards.
PS: Please don't insult your intelligence by replying that we don't understand how people drive either, so it makes any opaque self-driving software okay.
It can be simultaneously true that we cannot describe in a causally deterministic way how a system works while also determining with some confidence that the system is performing as desired. This is true of all human systems.
So if you don't know how people drive, but have enough confidence to let them drive based on performance in evaluation scenarios, then similarly we should be able to let machines drive assuming they perform equally as well as humans over the same contexts without knowing how they drive either.
2) you’re describing how a system can crash. Driverless cars don’t share state, besides, I guess, vast scale astronomical and weather conditions. It’s not likely something would cause all cars to crash because of a collective hiccup. There may be a set of circumstances that consistently cause individual cars to crash, but that’s not really the same thing. And we can understand how these events are processed, so unlike people, we can adapt for future repeats of this scenario.
Errors or malicious attacks upstream could lead to disruptive or even dangerous behavior by a lot of cars at the same time.
This happens to human drivers every year. Snow, freezing rain, fog; these adverse weather conditions can lead to large numbers of collisions at the same time. Expecting self driving cars to never have a problem during rare environmental corner cases is to expect them to be vastly superior to human drivers all the time. If you're waiting for that day before you consent to self-driving cars sharing the road with you, you'll be waiting a long time.
Well, self-driving cars can currently handle none of the aforementioned conditions. That's why they are being tested where those conditions don't exist, and that's why we're about 20 years from widespread adoption. It rains and snows in much of the rest of the world.
Yes, I live in the rest of the world and my family has been affected by fatal collisions brought on by adverse weather. I remain skeptical that self driving cars will be able to handle all of these things. I do think they'll be okay for mundane driving where the biggest risk is driver distractions, however.
I would like to see cars and maybe even roads made smarter in order to make driving safer. If all the cars on the road were part of a wireless network and they could signal to one another when they're braking or turning, then other vehicles could respond more quickly.
Heck, these systems could even be included in phones and pedestrian crossings so that cars could be signalled instantly when a pedestrian steps into the road in order to cross.
BUT (and there's always a "but"), the problem is exponentially hard, and it can't be solved with today's technology.
I agree with you that widespread adoption will require modification of roadways, and potentially also segregation of autonomous and human-controlled traffic streams.
Cars sliding out of control on a snowy road is not a systemic failure.
If you have an entire fleet of self-driving cars using the same (or similar) technology, the exact same condition could cause all of them to misbehave in the exact the same way. This could happen simultaneously with a large number of cars. Or could be something that propagates in a cascading manner. (A car breaks, turns and spins out of control. The next car detects that, breaks, turns and spins out of control, etc.)
If you have no clear idea how your driving software works, how are you going to make sure this doesn't happen?
People have common sense, variability and adaptive learning to prevent things like that. Moreover, our traffic system, our cars and our driver education have been co-evolving prioritizing safety for the last 100 years. (Heck, some of it started with horses and carriages.) The complexity and size of the system have been gradually scaled up for just as long, and it was designed and refined to be used by humans.
It's astounding how many of AI enthusiasts fail to see these issues and differences. Traffic system isn't a board game.
A lot of people don't know how typing "www.google.com" into their browser works but DNS "just works" for them. There is software in nearly every part of our lives that could systemically go wrong in catastrophic ways - we do our best to plan and prevent them.
> People have common sense, variability and adaptive learning to prevent things like that. Moreover, our traffic system, our cars and our driver education have been co-evolving for safety for the last 100 years. (Heck, some of it started with horses and carriages.) The complexity and size of the system have been gradually scaled up for just as long, and it's designed and refined to be used by humans.
Isn't that incredible? The complexity goes up: we adapt. As self-driving cars are introduced we adapt and evolve and incorporate these changes in technology, the driving "environment", security, safety measures, and laws and regulations.
> Traffic system isn't a board game.
I didn't see any efforts at self-driving vehicles just say "ahhhh! this driving stuff is so easy!" Of course it comes with huge difficulties, an enormous amount of complexity, a baffling amount of data, a ton of edge cases, and now a wealth of tribal knowledge on the subject.
When considering these things think of the systemic benefits as well. There was a story on the Google self-driving cars in Mountain View not taking a turn when the passengers of the car expected it would. Turns out that the IR sensors on the vehicle saw that a jogger behind a wall of hedges was about to enter the intersection to cross while jogging. Systemically implementing that change across a fleet of cars would then save untold numbers of injuries or lives.
We adapt.
Some cars today already have IR sensors (night vision systems) installed in them, and include pedestrian detection monitors as well. Cars don't have to be full on self-driving to have a large increase in safety and accident prevention.
On the other hand, noone was ever run over by a mistyped URL.
Human intervention would be required to enable the vehicles to move again.
After some real-world experience, I would want to give the emergency stop system the ability to release the hold on one layer of vehicles at a time, allowing the smart AI to drive again but at a speed limited by the "dumb" system to nothing faster than human walking speed, enabling them to clear the area. If they cleared without further incident, the next layer would be released until the whole stoppage either cleared itself or was restopped because something was still wrong.
These kinds of crashes generally have the some root cause: everybody is traveling too fast for conditions, and when one thing goes wrong, the end result is that you get a chain reaction of everyone accreting into a single crash. If self-driving cars are capable of making the same misjudgements, then these kinds of crashes absolutely can happen.
For the average non-technical person, this is pretty much their relationship with all technology.
I don't think it's exclusive to tech either. I know lots of people that have absolutely no idea how the mechanical side of a car works, but they trust their lives to one at 100km/hr because it has worked generally pretty well in the past.
Car safety has made great progress in the past 10 or so years. People don't have to know how they work under the hood as long as designers and engineers already built safety into them and these designs have been heavily tested by third parties (e.g. Euro NCAP, IIHS, etc)
Self driving cars will have to prove their safety, especially in extreme weather and non paved roads.
But another way to look at this is the fact that "cloning" the same AI brain across multiple cars is equivalent to a single AI brain controlling many cars. And that does have human analogies where you have a single person who has a dramatic amount of power. Any flaws in that human's reasoning can be disastrous.
Trump having access to the nuclear launch codes comes to mind.
In practice, the way we mitigate this is by surrounding that human with advisors and other brains to try to help shore up deficiencies. We could do something similar with AI. Have a couple of differently-trained models and compare their outputs. If they disagree wildly, then try to gracefully shut down the system.
Google something like "top causes of automobile accidents". No list I found had exogenous causes (rain, snow, etc) in the top 5. For machine driving to top that isn't going to be that hard given that we are our own worst enemy on the road.
There just aren't reliable statistics on what the safety of this tech will be at this point in time. All we have is extrapolations with lots of bias and little independent review. Maybe it will eventually be worth it, but that experiment will come with a great many human deaths and we can't simply brush aside the ethics of that. I personally think it won't prove to be safer than humans for many decades. This is far more of a complex problem than a lot of the developers think, and I believe there's a case of engineering arrogance going on here as well.
On the surface it seems like we have something very close to a viable product, but the remaining little details and issues are important and represent an enormous amount of work, where we are going to see diminishing returns and enormous difficulty making a system that is flexible and robust enough to handle all the required situations more reliably than the average driver on its own for thousands of miles, scaled up to millions of units around the world. They will be good enough for niche applications, but the dream of a general self-driving infrastructure is pretty far off.
You don’t need to be better than humans for all situations. You need to be safe. Humans are generally well above that threshold, except for when they’re drunk, high, tired, distracted, or otherwise stupid. It doesn’t really make sense to compare deaths per mile for cars vs attentive humans in stable conditions. Both can honestly be reduced to the point at which they’re practically 0 unless it’s the victim’s fault. We do need to be attentive to edge cases that cause cars not to be safe, but we can also observe the overall pattern and draw conclusions on how likely unknown edge cases are to be encountered.
Local ones seem unlikely to cause casualties larger than events that fool large numbers of human drivers, like sinkholes or malfunctioning traffic lights or black ice on a freeway. Some of these will probably happen, and the systems will be improved, and they'll happen less and less often.
Global ones could be very large, and there are several instances of global, cascading failures in other systems like telephone or data networks. There are known ways to prevent many classes of failures, but getting to 100.000000% is hard.
Any system that requires a human operator to passively observe a boring process for hours at a time, yet snap to attention immediately and without warning to take control and prevent human deaths is simply insane. That's the diametric opposite of how a safe system should be designed. If the human is required for safe operation, they should just be driving at all times to maintain vigilance. I don't see people talk about that much. The assumption seems to be that AI will quickly get better and the human back-up will not be needed--that the human is kind of a temporary workaround. Well I've seen plenty of engineering projects where the temporary fix becomes a long-term pain in the ass because the difficulty of the real solution was massively underestimated and under-budgeted. We might get stuck with these terribly designed AI-human hybrid systems for many decades as AI keeps getting pushed into the horizon and we slowly realize that this is one of those hard problems that doesn't progress the same as most other tech does.
There are pure driverless systems out there, but they don't cope well with many of the incongruities and fringe situations which are an unfortunate reality on the open road in most of our country. I think they will be confined to niche uses for a long, long time.
I don't see why this would be any different for deciding whether computers should be allowed to drive - we can just make sure self-driving cars pass a road test. We could even do better than driver's license tests and have a human driver at the wheel during the test just in case.
We let people drive because we know that they are fully capable agents able to make decisions even in case of failure. In other words, they are responsible, reliable, fully cognisant individuals, far beyond the technical ability of operating a machine.
A self-driving car is like a parrot trained to speak. Not only is it not an agent aware of its larger surrounding and potential implications of getting into unknown situations. It also has no real understanding of what driving is at all, it's even stupider than a pet. And because its software is usually deployed to thousands if not millions of devices at the same time, there is little to no variance to protect from extreme failure.
I wish I could share your optimism in humanity, but what you just claimed sounds far from being true universally.
Take your typical driver and throw in a failure situation they have never trained before (because, who has time to train them for every possible, or evens few likely failures), and it is not likely to end well.
Humans have a base line of common sense reasoning and the ability to create coherent models of the world. A human driver isn't (rare exceptions aside) going to drive up an impossible street or try to dodge a street sign mistaken for a kangoroo in West Germany.
There's nice examples of how an ML algorithm can be tricked into all sorts of pathological behaviour by choosing adversarial images, there's no baseline of reason to fall back on.
Imagine a model that has one of these fatal edge cases baked in and a thousand cars suddenly mistake a flash of light for a u-turn. Sure, one or two human drivers out of a ten thousand might drive that irrationally on occasion, but compared to a machine we can usually cope with this.
And I don't even want to imagine what happens if this turns malicious. Imagine a bad actor on a bridge above a highway with some artifact that messes automated cars recognition up and the developers forgot to put some hard fail-safe in for that piece of road.
It's worth noting the downsides to this agency too, though: Agency is what makes the stereotypical male 20yo driver speed and drive dangerously to show off to his friends, which leads to a very large number of accidents.
It's unclear what accidents this agency avoids though. I can imagine a few situations where it helps (avoiding a terrorist attack by speeding away from the scene?). But in most cases the default operation for a self-driving car will be to stop as quickly and safely as possible in unusual circumstances, and it's unclear how often this is the wrong decision.
I won't insult my intelligence; I'll ask why you trust people even though you don't know how they work and you know that at a high rate they intentionally impair themselves before driving?
Recall that time when a human trader lost a billion dollars?
https://traderhq.com/traders-cost-company-billions/
Remember when the economy tanked because everyone in finance used Black-Scholes model that seemed reliable, because it worked most of the time in the past?
This happens all the time already: there is a condition in the environment (eg when it snows in Seattle) there is a systematic failure involving large numbers of people driven cars at the same time.
People just don’t have experience (erm, “training”) with the weird unusual environmental condition and fail to deal with it. Not surprising that computers would have similar problems.
> PS: Please don't insult your intelligence by replying that we don't understand how people drive either, so it makes any opaque self-driving software okay.
But we actually do know how well people drive. We have data on failure rates for sure. We don’t know how they drive, that is completely opaque.
I don't. In my opinion, we are all getting carried away with ourselves.
Google or not, I challenge anyone who makes a statement including the phrase "...more like brains", to define that statement accurately. In response, you will likely hear all sorts of things that sound like we know what we are doing and are very clever, but.
If we can't even ask the questions properly, I doubt we will get deliberately valuable answers. Sure, we may luck out, with AI researchers stumbling across random combinations (layered AI) that do produce useful results hap-hazardly, but I would never rely on it.
Eg. I rely on software running in an aircraft, because:
a) it works, by design, not trial and error. b) it is verified. c) we know how it works.
unless you have (c) you cannot achieve (b), with any degree of certainty. Experimenting until confident is not good enough for me for anything important.
Brains fail at times. Computers should not make mistakes.
Even if you knew how a brain works, and even if its possible to replicate it, please don't make computers like this. Software fails enough on its own without any help, even when we supposedly understand it.
With this AI stuff, even if we knew where we are going, we don't know how to get there. And even if we did get there (wherever that is), in the long run, AI is not going to be used for the good.
As always with humanity, we will find ways to use AI to extract value out of other people. This is what drives business.
No thanks, but it is interesting for fluff. A bit too voodoo for me.
I don't think this is a reasonable position to take. We are going to have to accept that if we want computers to do the things brains can do, they will also make mistakes like brains do sometimes. It's not reasonable to insist that the error rate is exactly zero. Besides, if you consider the computer as a complete system including the hardware and human-computer interface, it's clear that no complete system can ever be error-free, even if the CPU executes its instructions faultlessly.
As long as it's as good or better than a human, it's good enough to be useful. Even with "fallible" AI techniques, we should be able to make systems that are much more reliable than humans.
That's my point though. I don't want computers to do what my brain does, mainly because I have no idea what my brain does, nor does anyone. Sure, some people think we do, but we don't.
It also seems silly to try to engineer something that we know may fail (by design), on the probability that it may haphazardly produce better results (even if this is most of the time). That may be useful for things like search results, where the outcome is not critically important.
Please excuse the example, but I would be happier passing away in an aircraft accident knowing it was because although the system was designed not to fail, we did our best in terms of understanding the systems in place. That would be my definition of "sh&t happens".
If I passed away because the aircraft "AI" took an untested path of execution (there are too many to verify, unlike most regular "dumb" software), I would be less happy.
Maybe there is a balance, Eg verifiable software for the important stuff, AI fluffery for the less so. But in that case, we should not get carried away (eg. self driving cars). Remember, planes could fly themselves completely for a long time, and we still have 2 pilots for a reason.
Computers are a human's tool, not replacement.
OK, that's a valid position to take, and I'm sure some people agree, but I and many other people do want computers to do what our brains do. And it is pretty likely to happen.
> I would be happier passing away in an aircraft accident knowing it was [not AI]
If I was in an aircraft accident caused by human pilot error while pilot AIs with 1/100 the error rate of humans were blocked from deployment because they don't meet your "zero errors" requirement, I would not be happy.
I hear you, but my point is "what do our brains do?"
In the case of driving, if the answer is "drive a car", then that is a bit like "brexit means brexit" :) The definition of the thing is the outcome of the thing.
Instead, if you try to can decompose the question into a more useful form, one that can drive a specification of software, then progress can be made. However, all that has done, in that case, is a description of the outcome, not the process. (And incomplete, due to the complexity).
It is at this point where my problem with all this AI stuff starts :)
The lack of understanding of the process, rather than the outcome is why I call it voodoo, and anyone talking different is trying to impress, in my opinion.
While we don't understand how our brain works to the bits and bytes, our knowledge and understanding of how our brain functions is Growing. A fuller understanding of how our neurotransmitters work on a chemical/biological level, and an effective way to model it, will likely result in interesting advances in the field of AI
I chat with my neuroscience friend a lot, and the overlap I see between how he describes how the brain works, and general computer science concepts is surprising.
In my view it would be unethical to stop the deployment of such AIs on the grounds that we don't "understand" them well enough or can't predict their behavior in every hypothetical circumstance, because we can still test them well enough to know with statistical certainty that they will save many, many lives.
Estimates about availability have consistently been overly optimistic. This may be a case where the first 99% of the work takes 99% of the effort, and the last 1% takes the other 99% of the effort.
These are the kind of problems which have profound amounts of complexity lurking in every unexpected pitfall. We are going to discover that it's a lot more of a tangle than we thought before it becomes simple again. I think most of us here know a thing or two about that sinking feeling when you're attacking a problem and discover that what seemed like a minor pothole is actually a wormhole leading into an entire universe full of waist-high bullshit with submerged landmines, but this problem will be in another class entirely. Figuring out the brain is a species-level problem and it's going to be VERY long and arduous.
I also resent the article's assertion that "if an AI is a safer driver than a human, who cares if it's a black box?" Fallacious argument number one around self-driving cars. Hear it all the time, almost always from laymen. It's a nice way to sneak in the assumption that AI will be better than average human drivers any time soon. The way that metric has been measured has been very deceptive so far, and no AI has come close to matching human safety and robustness. Not in real life applications in the field at scale. Not even close. There's been shenanigans like discounting all driver interrupts, which by nature disguises any flaws of the AI and offloads that responsibility on a person as a sort of PR heat-sink.
We don't give ourselves enough credit for how well we handle driving. Reality is chock full of exceptions, quirks, and legacy systems that aren't going to go away for many decades, and compared to the progress we've made on AI so far it's going to be many, many times more work for us in the future to deal with those niggling details. The devil's in there, he really is.
He did, he specified the synapses are like weights. I'll extend that a bit and say the collection of ion channels on the neuron's membrane, which regulate the surface potential, will act as the classifier. They form a non-linear classifier (voltage gradiant dissipates with r^2) but still a classifier.
Oh good; although, that predicament about publishing AI papers seems pretty bad.
We don't generally expect someone to freak out and start treating a car in sunlight as a house because they've seen a lot of houses in sunlight lately.
The difference between a human and ML is that a human has a clue, demanding perfect introspection is missing the point.
This borders on dishonesty if you ask me, he should know better.
Manager: Why did you delete the production database in the middle of a work-day, no less?
DBA: My decision was based on all sorts of things you can quantify, and then all sorts of gut feeling. I have no idea how I do that. Do not ask me to explain my decision, or you are forcing me to make up a story.
Manager: But, you crashed our client's entire business for a whole day!
DBA: I can't explain how I work.
(Somehow, I don't see that flying.)
I guess pointing that out while explaining yourself is a bad idea though.