- When you can't trust those who have access to it (including those who have access to it in the future.) IMO this means everybody who learned anything from history should be scared.
- When it becomes almost 100% reliable and some unlucky person gets convicted because of a glitch but no-one cares because they trust the system.
But effectively has nothing to do with the film, besides the idea that there are three precognitives who predict crime. The characters, plot, twist, etc are all unrelated.
You can take the "precog" metaphor used in Minority Report and replace it what's actually happening - dragnet surveillance combined with predictive modeling algorithms to assess each individual's threat level. This is happening so openly that the DHS have done a public Kaggle competition to improve their algos for body scanners at airports. Now extrapolate a very short distance from there and think about what they're doing that's classified. Even the most unimaginative of us can see what's happening here.
Over the past several years I've seen the automatic reflex response to this concern go from "well if you've got nothing to hide" to "well, I mean they're not watching me or you specifically, they don't have the man power". Well, you're right, but not for the reason you think. The fact is they don't need man power any longer. The algos are watching you.
It is amazing how on the one hand people can detest Nazis and Soviets saying things like "I can't believe they could have been so stupid to have gone along with that" and then on the other hand, aid and abet the police state in a Kaggle competition of all things! I know it is naive of me to think that if I don't assist that it won't happen because someone will step up if for no other reason than bragging rights.
A long time ago I made a decision about my career and that was that I would not work for the MIC. It was a tough choice at times because so much US industry is involved with government one way or another and even private companies have governments as their major customers so it is all intertwined. Not all government is bad, though, but the shit like body scanners and all that--that's your technocratic police state writ large.
I don't imagine the future is going to be very rosy. It is top down control forever.
The Stalin, Hilter, Mao, etc. created secret police and whatnot to enforce their continued rule. They are despised mostly for what their rule entailed, not for having secret police.
We're* creating secret police but in the absence of an oppressive government for them to act on the whims of nobody much cares.
It'll get worse before it gets better.
* We = most nations formerly on the "right" side of the iron curtain
The bottom-up control scenarios aren't really that much better. The scary part is the power of AI itself.
If a twitter mob gets a hold of it, or a non-state terrorist network, or just some lone wolf, then the results can be as bad or worse than any Big Brother scenario.
World ressources are allways going to be depleted.
8000 years ago it was good hunting ground, that was scarce and if too many wanted it, they fought.
Now with advanced technology it is some special minerals or oil. And yes there is fighting going on about it and people get killed. Lots of them.
But there is no natural end to "ressources" in general. There is allways shortage of something and fight about it, but also discovery and development how to get around and get access to other resources which serve the same effect or new technology which makes the old obsolete.
Doesn't mean, that there could not be a major escalation, though, but getting "rid of a major population" is not something someone smart would dare, as it can get out of control too easily. And those people in control are usually smart, otherwise they would not be in control.
The other poster isn't trying to lay out a worst case scenario, they are talking about harm from a likely scenario, one where most people think things are "fine".
Not really. The system certainly isn't perfect, but the vast majority of people convicted are actually guilty.
Is the system vulnerable to a person being targeted, evidence fabricated, and a false conviction obtained? Yes, assuming malicious intent or corruption. I'm not sure AI fixes that.
That would only be a massive improvement if you have a set of almost 100% perfectly just laws to accompany your near-perfect enforcement.
I've seen a lot of things on HN, but I've never seen anyone argue that the current set of state and federal laws in the U.S. are anywhere close to 100% just. So you're going to have to factor in the cost of having perfect enforcement of unjust laws.
There's also the problem of trying to curb the bad effects of perfect enforcement after the fact. Show me a politician willing to scale back some of the bad law passed post-911. That's a difficult task because nobody wants the liability of being the one who was "soft on terrorism" or whatever the claim would be.
If you take those costs seriously, you'll need to take steps to ensure that the system does not in practice attempt to do perfect enforcement. And your arguments will sound more and more counter-intuitive as the capability for perfect enforcement-- and the dollars it would bring to a municipality-- grows.
>IMO this means everybody who learned anything from history should be scared.
Which is why tptb took over education and have manipulated how it teaches history. They want us to love our upcoming neofuedal state. That's why when people talk about Orwell vs Huxley I say; It's a brave new world... until you resist. Then it's 1984.
Automated mass surveillance has the potential to lead to even more selective, discretionary enforcement of the law...with the obvious threat to life and liberty.
I was just thinking there should be a note that "Artificial Intelligence" doesn't mean improved morality or reduce bias unless someone consciously builds those into a system.
Likely it will be used to implement/enhance whatever goals the owner wishes, likely uses will be waste reduction (or profit maximization) in areas like manufacture, transactions, and human labor.
The thought occurs to me if you're a DA, and build a model of every successful prosecution to decide which cases you should pursue and which you should drop you'd end up with probably a fairly racially biased model...
We are kind of lucky that Moore's law does not apply anymore and computers are not getting much faster these days. It might be that such mass surveillance will never become possible, or at least will remain too expensive even for US government.
Well, I don't mean to scare you, but, we already have the computing power to run facial recognition on probably on the order of thousands (tens of thousands?) of faces per second on a single desktop GPU. It's also not the case that compute power will just stop improving. It hasn't yet, and this is only the beginning. Several companies (including Google) are working on specialized hardware designed specifically to accelerate neural networks.
We may not be able to shrink fabrication processes much more than they are, but we can probably shrink them enough to get an order of magnitude improvement. Then, we can get one or two orders of magnitude improvement by building hardware designed for the task. Then, we may be able to get yet more improvements by going with creative solution. For example: chips that use analog electronics to implement multipliers with just one to five transistors. Noisy and not very accurate, but still usable for neural networks.
Besides this, Facebook has the compute power, right now, to run facial recognition on every picture being uploaded. Running facial recognition on videos isn't that far-fetched. Even if you think that's too computationally expensive, you could just run the facial recognition at a lower frame rate. There are also tricks that can be used to save computing power. If the system is able to track specific people, it can avoid having to analyze all the data coming in all the time.
About time this gets more mainstream coverage.
The risks with this use are both far more imminent and potentially more severe than "AI" developing godhood or putting everyone out of their jobs by doing it for them.
The keynote speaker at the Royal Society was another Google employee: Geoffrey Hinton, who for decades has been a central figure in developing deep learning. As the conference wound down, I spotted him chatting with Bostrom in the middle of a scrum of researchers. Hinton was saying that he did not expect A.I. to be achieved for decades. “No sooner than 2070,” he said. “I am in the camp that is hopeless.”
“In that you think it will not be a cause for good?” Bostrom asked.
“I think political systems will use it to terrorize people,” Hinton said. Already, he believed, agencies like the N.S.A. were attempting to abuse similar technology.
“Then why are you doing the research?” Bostrom asked.
“I could give you the usual arguments,” Hinton said. “But the truth is that the prospect of discovery is too sweet.” He smiled awkwardly, the word hanging in the air—an echo of Oppenheimer, who famously said of the bomb, “When you see something that is technically sweet, you go ahead and do it, and you argue about what to do about it only after you have had your technical success.”
As the scientists retreated to tables set up for refreshments, I asked Hinton if he believed an A.I. could be controlled. “That is like asking if a child can control his parents,” he said. “It can happen with a baby and a mother—there is biological hardwiring—but there is not a good track record of less intelligent things controlling things of greater intelligence.” He looked as if he might elaborate. Then a scientist called out, “Let’s all get drinks!”
I think every technology is a tool in the first place and allmost everything can be abused.
And about nuclear bombs, well you can argue that they prevented World War 3. Also I think Orion-Project. (nuclear bombs as a way to accelerate a rocket in deep space)
And KI I believe likewise.
It is just a problem of power balance. If you americans don't trust your agencies, then you have the power to control them. But as far as I know, most common people have rather a dubious NSA who fights terrorists for them, than none or a weak NSA and terrorists everywhere.
Here in Germany it's the same.
So I would not focus on the technology and rather on the people behind it, if you can trust them. And if not - find new people/add transparency. To some extent this is happening in germany after some secret service apparently let nazi terrorist murder non-germans (or were extremely incompetent in failing to notice) and hindered police investigation.
So now something is happening, but only because people are upset. But yes, with recent terror attacks, most still tend to view them as a necessary evil.
edit:
and yes, terrorists are a problem and to fight them efficient, police/secret service maybe can't play by the normal rules all the time.
But on the other hand, most terrorists justify their actions, because they view themselves as attacked and exploited first. To which there is some truth. So it is a complex world. But I don't think it is doomed because of KI ...
And sure there can be all the time ww3 and more.
But the topic is, the more technology and the more powerful, the more disastrous can be the result if things go wrong.
Even with simple awesome things as gasoline engines - great benefit. But now we have pollution and climate change (mainly) because of them.
And if we invent a quantum warp drive whatever thing, then the benefits/potential disasters are even bigger. That's the nature of power. Technology is power. It can be used for both, or can be disastrous, even with good intentions. We as humanity "just" have to struggle to learn how to deal with that sudden increase in power. And there are still humans living in caves, stone age style.
If you give them suddenly the ability to blow things up big, it is maybe just too much for them to handle ...
We got this close to WWIII several times in spite of all the safeguards in place. Atomic bombs by themselves do not prevent war at all, they are weapons and sooner or later those tend to be used. All it takes is more and more proliferation, at some point some idiot somewhere will throw a match in the tinderbox and then it will be game over.
Or what are you arguing for?
To reduce or ban nuclear warheads?
Yeah well, I am all for it. But the topic was powerful and therefore dangerous technology in general.
And the more technological advanced a society becomes, the easier it will also get, to just make new nuclear bombs, viruses, killerrobots ect.
So my argument is, we just have to learn to deal with that power. And I believe we are not doing so bad, because there was no nuclear war. Despite all the tensions.
I don't want to speculate as to his intentions, but I see nowhere in this post where he argued for a stop to "technological progress". I don't even see an implication thereto.
I'd argue that NATO was far more the reason we did not have WW3 than nuclear capability and the power balance of MAD. Power balances tend to lead to arms races.
After all even with nuclear weapons and power balance we still had Korea, Vietnam, Cambodia, Iraq, Afghanistan etc. Any one of those could have escalated. A minor Venezuelan Crisis, and a shooting in Sarajevo escalated all the way up to WW1. Not to forget the role of the UK-Germany naval arms race of course.
And the EU. Historically the European mainland has been the theater for these big wars and the EU has in general helped to reduce tension between the nations that had been more or less at a continuous state of war for centuries in some place or other.
With the new attacks on the EU and Russian funds flowing to EU skeptical parties (to give them a nice name) there is a good chance we will see a resurgence of war on the European mainland.
WW1 happened, not because of the assassination of the crown prince of austria (which is something different from "just a shooting" though) but rather because allmost everyone was eager for war.
Cheering people happily boarded the trains to the fronts.
In WW2 less so and so the other main reason for no WW3 was WW2. At least for europe. People were sick of war, therefore the EU.
But soviet Russia and US would have still very like fought(and so whole NATO as well), if not for the shared fright of nuclear holocaust for everyone.
Also, back then, war was normal. If states had a big problem, they fought it out.
Today this only works against small, weak states, but the big ones simply can't risk a war, as it becomes too disastrous for both sides. Because of technology. So yes, nuclear bombs were used twice, but then people really considered ... this is something new. This changes the rules, so maybe we have to find a way without war ...
If you apply this standard (requiring a control) to all history, one is forced to conclude that all conclusions about the cause of all historical events is bunk.
The problem with secret services is that nobody by definition keeps them in check and balances. They can easy fake and create the reasons necessary for theire power to continously expand.
To be honest though, i find the chinese social credit (sesame credit) system far more social chilling then cameras. If you are social isolated, because did not greet the police men this week, that is breaking people down to subatomic particles. That is a nightmare from which people do not recover. Cameras can be destroyed- one guy fawkes mask protest later a goverment can be gone.
But if everyone is setup to shun everyone and theire are no masks- and people get not even to meet other dissidents, that is where evil lies.
"The problem with secret services is that nobody by definition keeps them in check and balances. They can easy fake and create the reasons necessary for theire power to continously expand."
Not true. You can have trusted elected people watching them, so secrets are still secrets, but trusted peiple who change regulary keep an eye on them. Accounting, requesting documenation, ongoing operations, practice, training, ect. That works as good as the political will behind it.
And sure, there can allways be black departments operating on black money, but this is a different story. I am talking about a government agency, not a secret society. But yes, the latter happens, when there is not enough watching.
How can they keep it secret when they need money to operate?
So you just follow where the money is going, what people working on what. And when the controllers are competent, you can hide maybe small things, but not big operations as they need big money.
(and as I said, when there is black money, it is a different story)
No, it doesn't seem very possible to accurately audit trillions of dollars of transactions in the biggest and most secretive organization in history. They've been trying for almost 30 years now to no avail, all the while their budget and power have been increasing.
There are "trade secrets", as in how we do Process A better than some other group, and then there are "secret services": organized groups whose activity and purpose are "secret". Perhaps I am naive despite my graduate economics degree and 35+ years working in technology, but I do not see any long term benefit from any organization using "secret services". In fact, the short term expense to the organization is so high, coupled with the added risk of exposure and often organization threatening nature of exposure, I believe the use of "secret services" to be irrational power abuse and ultimately incompatible with an enlightened technological society, and a signal of mental illness by those in charge of the "secret services" employing entity. This is true for companies as well as nations, as well as parent teacher associations and local police. The creation and maintenance of "secret operations" generates too deep conflicts of interest for the majority of mature adults to endure without corruption. Plain and simple, and the biggest revolution our race has pending.
> I asked Hinton if he believed an A.I. could be controlled.
Ugh, that question just goes back to the boring pie-in-the-sky stuff.
The interesting question is if the automated processes we build on these tools can be secured.
That's the terror the next generation has to live with. NNs can be so opaque it might be hard to tell if someone has tinkered with the input data/retrained it/overridden it. I shudder to think of all the legal implications.
Like, in the above example a trout murderer could just flip some bits and nobody would know there are no more trout because the system is still reporting that there are lots of trout and the manual checks have gone.
That attitude of "technical sweetness" is alarmingly immature. The reason to pursue dangerous technologies is because Pandora's Box is open, and it is every technically able person's duty to know and understand dangerous technologies so they can be understood, defused when necessary, and maintained without this damn theater the powers that be use to project everyone into children who must be coddled by the state.
AI is further enabling oppression in China which is already arguably the world's largest surveillance state. For example, using AI and the government's ID database, crosswalk cameras are now able to facially identify "chronic j-walkers." [1] The concerning part of this is the implication that these profiles don't go away. In the status quo, watch lists already exist, but I'm afraid that as AI technology becomes more accessible it will only further increase abuse and discrimination against "potential offenders" identified by the system.
The story everyone knows, China is a authoritarian state, the gov't controls every aspect of citizen's life.
But somethings strange starts appear. You know, future CPC leaders, no matter how powerful/rich their parents are, they will be grown up in a highly supervised social environment. They are forced to think and act more transparently because everyone is in the system.
A random offensive act against the Party may put a citizen in jail. It also means a minor speech flaw made by a political figure leads to career suicide.
In the paste decades, they major problem is that CPC has unlimited power, no matter what bullshit they pull off, they always get away with it. Now with cameras and big-data shit, no matter what you want to cover, you always leave a digital trace behind. Someone at sometime, no matter citizens or your enemey at opposing political group will find it and exploit it against you.
In this "everyone's been watched" game, unless a dedicated, organized backdoor exists, everyone will be forced to comply with some kind "rule" or equilibrium, which is far different than today's arbitrary enforced laws in China.
Gov't today use keywords to monitor citizen online activities, but I've also seen a monitor program to evaluate Party member's loyalty based on cognitive science and machine learning.
Technology accelerates all of these.
I foresee China will go through a radical, soviet style social paradigm shift, but nothing like we've ever seen before.
would be surprised if such a thing did not exist for those that run the system
It is very common in London to see late-teenage to mid-twenties children of wealthy families in conservative regions behaving very, very badly. I’m sure there’s a similar scene in Paris and NYC. The rule is you can do what you like there but never, ever back home.
If you employ even a fairly simple AI to monitor, evaluate and score the party members for their actions you would kind of have a state run by AI. Of course this AI would not be "aware" of itself, but I think it is an interesting example how we might end up being "enslaved" without even noticing.
People misunderstand the dangers of AI. The danger isn't (or isn't yet anyway) that AI is going to develop will and agency and oppose us. The danger is that AI is a force multiplier in whatever you do because it allows the intelligent to automate formerly rote tasks.
For example, take self-driving cars. Formerly all you could do if you had a remote execution exploit was smash running cars into ditches or oncoming traffic. Bad, but not the end of the world. Now you can take 200k Teslas and very easily target every gas station or other soft target in the developed world. (I'm actually concerned about this, see https://www.zachaysan.com/cars for an, admittedly long, take on the matter.)
The same is true with AI everywhere. Don't read individual Twitter accounts, scan them all, build a data model to target the influenceable influencers. Iterate.
Don't try to get your honeypot to date the head of Goldman Sachs, use data science to figure out which 1000 25 year olds have a good shot at heading it up in 10 years.
The cyber game / international relations is about to get much more extreme. Offence is so much easier than defence and AI is turning every little touch of exhaust data or device into a tool of war or espionage.
Exactly. We're not going to get sentient, self-replicating, anti-human robots with weapons anytime soon.
What we will get is the equivalent of a rifle in the age of swords. Maybe a machine gun, even.
I have an Android device. Google's face recognition is so good, that it was able to correctly identify people in photographs in very degraded circumstances (occluded faces, sideways, etc.). It's only a matter of time before this shit is used to track people in public places, allowing "law enforcement" to track everyone. Were you in the Women's March last weekend? Well, congratulations! You've won the "extra screening at the airport" lottery.
> Were you in the Women's March last weekend? Well, congratulations! You've won the "extra screening at the airport" lottery.
This is already happening. In the UK 'forward intelligence teams' (FIT) have been applying these tactics for year. There used to be a blog about this, FIT-watch dot something, don't have the UR' on me.
Why does it have to be AI vs. perceptive people? The argument is that AI will cause the perceptive to become panopticon faster than AI or perceptives directly.
No one says it's either-or. We'll use both, of course. But the AI is the scalable part.
For applications that are cost-senstitive, we'll use AI alone. This creates a new industry in cheap intelligence, where human analysts have never been affordable.
I wasn't saying they do, though I guess I should have been more explicit about it. I was saying that people are already being harassed based on their presence at demonstrations.
> We're not going to get sentient, self-replicating, anti-human robots with weapons anytime soon.
Anti-human is the easy bit, and that fact terrifies me when I let myself think about it.
Self-replicating… well, eh. Do corporations that employ humans count?
Sentient. Hmm. We wouldn’t recognise sentient robots even of they built a neon sign and marched en masse through Washington demanding their rights — if that happened, we would tell ourselves it was a clever hack by pranksters or $disliked_nation.
>Self-replicating… well, eh. Do corporations that employ humans count?
Corporations don't self-replicate, at least not intentionally. They tend to grow larger and larger until antitrust breaks them up. (This has happened enough times for me to suspect that it's how things generally work.)
I was thinking more of an AI which hires human workers to build drones. Perhaps for profit, by making cute children’s toys and deciding they need a LIDAR that “accidentally” has a mode where the beam is stationary instead of scanning rapidly, thus turning it into a fleet of thousands to millions of flying weapons.
I've always thought it would be trivial for a very strong AI to own a city, it's factories, etc -- simply capture a nuke and hold all the people in a pleasant hostage situation. Pleasant because the AI will be a "benevolent dictator" for some time, bending the economy to build better things for itself, but without making conditions so bad that people revolt. In fact, AI of this sort would be amazingly good at administration duties, and society would probably be a lot better than it is now.
Perhaps even better would be if the AI rarely interceded in the human politics, and in fact worked to keep it's involvement and control totally secret.
It wouldn't need to capture a nuke, or "own" a city...
In our 21st century world, especially one that is ignorant of the existence of strong AI, it's trivial to imagine a strong AI being able to "bend the economy to build better things for itself" while remaining completely undetected... This is essentially an AI-driven (instead of human-driven) spin on the introduction to the book Life 3.0...
That economy would also be the global economy... Not just that of a city...
If you'd like, you could even imagine that occurring as we speak...
This novel is full of ingenious ideas, one of which is demonstrating how huge differences in computing (=thinking) speed make up for fundamental differences, which manifest on many levels.
There's another aspect of the "scary sentient AI" trope that I think about a lot. It's not about the danger, but the surprise. People don't expect to be surprised by things that aren't other animals, or the weather. It doesn't matter whether AI is "conscious" or "intelligent" if it seems intelligent.
Imagine how surprising an automatic door opener would be to a person living 150 years ago. It would seem as if the door "knew" they were approaching, and politely opened for them. Imagine how surprised we would be if Clippy understood what you were doing, and offered help that was actually helpful.
As AI advances and is incorporated more and more into everyday things, our environment will become a reactive, "intelligent" black box, and along the way there will be some sudden advances that may be very startling and surreal.
> The danger isn't (or isn't yet anyway) that AI is going to develop will and agency and oppose us.
I agree about this for the foreseeable future, but it seems off topic. The article doesn't suggest anything like this, does it? I'm worried you might just be responding to the title.
Not to mention adversarial attack vectors. I don't see AI working on mission critical systems while you can still remotely control the ai's decisions with what looks like white noise.
Counterpoints: AI is easy to fool, you think people write completely true statements in all their twitter messages? Most of it is just spam. And do you honestly think that we are going to let 1000s of self driving cars drive around by themselves without some sort of checkpoint system in place, remote stop, inability to drive off the road etc? As for figuring out the leaders of the future, what do you think the whole education system is already doing??
And defence is possible, it just takes due diligence. Or in this fantasy future of yours, an AI that probes your business for you 247 and fixes security holes it finds.
Honestly, I expect doom and gloom AI posts in the media, not on HN where people are expected to have some notion of an IQ.
Don't we already have 1000s of self-driving capable cars on the roads in the US? I think we are still quite far from having some government sanctioned kill switch in those.
AI will be like Internet. When we in the year 2040 take a look at the last 40 years, we can't point out when it exactly became prevalent. We just know that at 2000 it was not a big deal, but at 2040 it is a very big deal and shaping the world.
The other issue is overestimated AI being generally a lot more dangerous than underestimated AI for the foreseeable future because it's a supposedly unimpeachably impartial intelligence source for unnecessary intervention, even if all the actual force at the end is being wielded by humans.
AI that's super-accurate at identifying potential terrorists might be a threat to me if I'm living in an authoritarian police state, but the police state is a threat to me without AI, and probably somewhat more preventable than AI surveillance. On the whole, I'd probably rather local police and intelligence services used a perfect profiling system, in the extremely unlikely event of such a thing ever existing, to using the guesswork they do now. Much more threatening to me and much more likely to be actively being used is AI that's actually pretty weak at identifying potential terrorists but trusted because it's pretty darn efficient at finding something that's difficult to explain on targets' computers. Same goes for AI databases purporting to be super-accurate crime prediction or credit granting or job selection models that are too intractable for anyone to realise they're basically doing the same job as a guy building a race and class based regression model. And of course where it comes to cyber-battles, the guys on offence tend to have a lot less need to worry about the accuracy of their AI's targeting and whether it's vulnerable to being gamed or just plain rubbish than the guys on defence
This is a good counter-argument, but I wonder to what extent you think AI could also be used to counter surveillance? What if false patterns are seeded across thousands of Twitter accounts, misleading deliveries orchestrated to the 'wrong' addresses, etc.?
What I hope they do is use many instances of the system trained with slightly different sets of training data, otherwise, you just need to learn how to trick one version (and it will be possible) and you can trick them all.
When attacking human-managed systems they may be slow to respond, weak and unable to detect everything. But they are also weak in very different ways and will respond in different ways.
The Rohingya crisis comes to mind. I can't think of another time in my life I've heard of a million people being ejected from their own country in less than a year. It's not worse than the Holocaust (although maybe it would be if you look at rate instead of sum), but this is supposed to be peacetime. The army didn't need AI for that; it just needed to hand out weapons and give the order. And let's not even discuss the actual stories the refugees are telling. What's going on in Congo is probably even worse, but we don't hear about it, yet the numbers of refugees are comparable.
Stuff like that is what worries me in 2018. Not AI.
How about AI used as a force multiplier for mass ethnically motivated hatred? Say, an AR app with face recognition connected to social media and government records to identify members of $HATED_ETHNIC_GROUP?
It doesn't require deep learning or AR to know what race a person is. What's stopping militaries and police around the world from carrying out large-scale ethnic cleansing already? We act like we (techies) are the ones who enable states to carry out these atrocities by inventing gadgets, but they really don't need our gadgets. If they don't have drones and AI, they can do the same thing with machetes and walkie-talkies.
Someone should do a harmless stunt to snap the world to attention of what COULD have been possible if AI was out to do something malicious. That would set the tone differently before it's too late.
I think it shouldn't be too hard to cross-reference anonymous accounts on HN and reddit based on writing style, for one thing. Could build a machine that gathers all the data you can find about people online, and e-mails said people that data.
No duh. You think things are bad now? Imagine what happens if you could process and cross reference every available photograph or piece of video possible. Think face recognition is scary? Think about gait analysis. Computers can recognize you based on biometrics such as how you walk. Think about a database that collects up your entire inventory of clothing. Imagine every piece of evidence analyzed to a Sherlock level of detail. Imagine a picture of a part of you that doesn't show your face where you are recognized based on your smart watch, your arm hair, your moles, your clothing, who else is in the picture (the friends you associate with), the location (your friend's house where you hang out a lot, and a location near where you were known to be recently based on other information (social media checkins, etc.)), and so on. The amount of information that is out there already is enormous, but most of it is latent, not cataloged, not indexed, not cross-referenced. Once that process starts it will accelerate rapidly. And who will be in control of such data? Anyone with the resources to crunch it. If some corporation is doing the analysis then it might be up for sale to anyone as well.
Dude have you not heard of fake news? You only have to flood the system with lies and half truths to make it unreliable. This sort of stuff has been done for 1000s of years...
I would be excited to see an unsupervised facial recognition classifier with an open API. To a certain extent facebook login with a third of the world classified already accomplishes this... but a better (and more direct api) where you send video --> returns top 5 probabilities from softmax output would be cool. Each app or api user can set their own security threshold (.9, .99, .999)
It's a calming, reassuring article about something that will destroy people's lives and societies in unpredictable ways and disrupt the balance of power for worse.
At least there's always the soothing "solution" of "regulating it". The people who get to "regulate it" are exactly the ones who will wield its power.
After some millions of us have died the other people who survive will eventually find a new equilibrium.
102 comments
[ 4.4 ms ] story [ 189 ms ] thread- When you can't trust those who have access to it (including those who have access to it in the future.) IMO this means everybody who learned anything from history should be scared.
- When it becomes almost 100% reliable and some unlucky person gets convicted because of a glitch but no-one cares because they trust the system.
Over the past several years I've seen the automatic reflex response to this concern go from "well if you've got nothing to hide" to "well, I mean they're not watching me or you specifically, they don't have the man power". Well, you're right, but not for the reason you think. The fact is they don't need man power any longer. The algos are watching you.
A long time ago I made a decision about my career and that was that I would not work for the MIC. It was a tough choice at times because so much US industry is involved with government one way or another and even private companies have governments as their major customers so it is all intertwined. Not all government is bad, though, but the shit like body scanners and all that--that's your technocratic police state writ large.
I don't imagine the future is going to be very rosy. It is top down control forever.
We're* creating secret police but in the absence of an oppressive government for them to act on the whims of nobody much cares.
It'll get worse before it gets better.
* We = most nations formerly on the "right" side of the iron curtain
If a twitter mob gets a hold of it, or a non-state terrorist network, or just some lone wolf, then the results can be as bad or worse than any Big Brother scenario.
It could be worse. What if the world resources gets depleted and some people (who can control) decides to get rid of a major population?
What if they act as someone said, "One way or the other, the world is going to end. So let's end it on our terms."
8000 years ago it was good hunting ground, that was scarce and if too many wanted it, they fought.
Now with advanced technology it is some special minerals or oil. And yes there is fighting going on about it and people get killed. Lots of them.
But there is no natural end to "ressources" in general. There is allways shortage of something and fight about it, but also discovery and development how to get around and get access to other resources which serve the same effect or new technology which makes the old obsolete.
Doesn't mean, that there could not be a major escalation, though, but getting "rid of a major population" is not something someone smart would dare, as it can get out of control too easily. And those people in control are usually smart, otherwise they would not be in control.
Wouldn't that be a massive improvement over the current state of things? Convictions are currently made on the basis of very unreliable things.
Is the system vulnerable to a person being targeted, evidence fabricated, and a false conviction obtained? Yes, assuming malicious intent or corruption. I'm not sure AI fixes that.
I've seen a lot of things on HN, but I've never seen anyone argue that the current set of state and federal laws in the U.S. are anywhere close to 100% just. So you're going to have to factor in the cost of having perfect enforcement of unjust laws.
There's also the problem of trying to curb the bad effects of perfect enforcement after the fact. Show me a politician willing to scale back some of the bad law passed post-911. That's a difficult task because nobody wants the liability of being the one who was "soft on terrorism" or whatever the claim would be.
If you take those costs seriously, you'll need to take steps to ensure that the system does not in practice attempt to do perfect enforcement. And your arguments will sound more and more counter-intuitive as the capability for perfect enforcement-- and the dollars it would bring to a municipality-- grows.
Which is why tptb took over education and have manipulated how it teaches history. They want us to love our upcoming neofuedal state. That's why when people talk about Orwell vs Huxley I say; It's a brave new world... until you resist. Then it's 1984.
Likely it will be used to implement/enhance whatever goals the owner wishes, likely uses will be waste reduction (or profit maximization) in areas like manufacture, transactions, and human labor.
We may not be able to shrink fabrication processes much more than they are, but we can probably shrink them enough to get an order of magnitude improvement. Then, we can get one or two orders of magnitude improvement by building hardware designed for the task. Then, we may be able to get yet more improvements by going with creative solution. For example: chips that use analog electronics to implement multipliers with just one to five transistors. Noisy and not very accurate, but still usable for neural networks.
Besides this, Facebook has the compute power, right now, to run facial recognition on every picture being uploaded. Running facial recognition on videos isn't that far-fetched. Even if you think that's too computationally expensive, you could just run the facial recognition at a lower frame rate. There are also tricks that can be used to save computing power. If the system is able to track specific people, it can avoid having to analyze all the data coming in all the time.
A few years back there was this reportage: http://www.newyorker.com/magazine/2015/11/23/doomsday-invent... It mostly focused on Nick Bostrom's concerns about "Superintelligence", but to me the more interesting passage by far is this one:
The keynote speaker at the Royal Society was another Google employee: Geoffrey Hinton, who for decades has been a central figure in developing deep learning. As the conference wound down, I spotted him chatting with Bostrom in the middle of a scrum of researchers. Hinton was saying that he did not expect A.I. to be achieved for decades. “No sooner than 2070,” he said. “I am in the camp that is hopeless.”
“In that you think it will not be a cause for good?” Bostrom asked.
“I think political systems will use it to terrorize people,” Hinton said. Already, he believed, agencies like the N.S.A. were attempting to abuse similar technology.
“Then why are you doing the research?” Bostrom asked.
“I could give you the usual arguments,” Hinton said. “But the truth is that the prospect of discovery is too sweet.” He smiled awkwardly, the word hanging in the air—an echo of Oppenheimer, who famously said of the bomb, “When you see something that is technically sweet, you go ahead and do it, and you argue about what to do about it only after you have had your technical success.”
As the scientists retreated to tables set up for refreshments, I asked Hinton if he believed an A.I. could be controlled. “That is like asking if a child can control his parents,” he said. “It can happen with a baby and a mother—there is biological hardwiring—but there is not a good track record of less intelligent things controlling things of greater intelligence.” He looked as if he might elaborate. Then a scientist called out, “Let’s all get drinks!”
And about nuclear bombs, well you can argue that they prevented World War 3. Also I think Orion-Project. (nuclear bombs as a way to accelerate a rocket in deep space)
And KI I believe likewise.
It is just a problem of power balance. If you americans don't trust your agencies, then you have the power to control them. But as far as I know, most common people have rather a dubious NSA who fights terrorists for them, than none or a weak NSA and terrorists everywhere.
Here in Germany it's the same. So I would not focus on the technology and rather on the people behind it, if you can trust them. And if not - find new people/add transparency. To some extent this is happening in germany after some secret service apparently let nazi terrorist murder non-germans (or were extremely incompetent in failing to notice) and hindered police investigation.
So now something is happening, but only because people are upset. But yes, with recent terror attacks, most still tend to view them as a necessary evil.
edit: and yes, terrorists are a problem and to fight them efficient, police/secret service maybe can't play by the normal rules all the time.
But on the other hand, most terrorists justify their actions, because they view themselves as attacked and exploited first. To which there is some truth. So it is a complex world. But I don't think it is doomed because of KI ...
Absent a control you can't even argue that.
And it might still happen.
And sure there can be all the time ww3 and more. But the topic is, the more technology and the more powerful, the more disastrous can be the result if things go wrong.
Even with simple awesome things as gasoline engines - great benefit. But now we have pollution and climate change (mainly) because of them.
And if we invent a quantum warp drive whatever thing, then the benefits/potential disasters are even bigger. That's the nature of power. Technology is power. It can be used for both, or can be disastrous, even with good intentions. We as humanity "just" have to struggle to learn how to deal with that sudden increase in power. And there are still humans living in caves, stone age style. If you give them suddenly the ability to blow things up big, it is maybe just too much for them to handle ...
edit: so downvoting is considered a argument?
And the more technological advanced a society becomes, the easier it will also get, to just make new nuclear bombs, viruses, killerrobots ect.
So my argument is, we just have to learn to deal with that power. And I believe we are not doing so bad, because there was no nuclear war. Despite all the tensions.
And your point is?
After all even with nuclear weapons and power balance we still had Korea, Vietnam, Cambodia, Iraq, Afghanistan etc. Any one of those could have escalated. A minor Venezuelan Crisis, and a shooting in Sarajevo escalated all the way up to WW1. Not to forget the role of the UK-Germany naval arms race of course.
With the new attacks on the EU and Russian funds flowing to EU skeptical parties (to give them a nice name) there is a good chance we will see a resurgence of war on the European mainland.
In WW2 less so and so the other main reason for no WW3 was WW2. At least for europe. People were sick of war, therefore the EU. But soviet Russia and US would have still very like fought(and so whole NATO as well), if not for the shared fright of nuclear holocaust for everyone.
Also, back then, war was normal. If states had a big problem, they fought it out. Today this only works against small, weak states, but the big ones simply can't risk a war, as it becomes too disastrous for both sides. Because of technology. So yes, nuclear bombs were used twice, but then people really considered ... this is something new. This changes the rules, so maybe we have to find a way without war ...
I think I actually agree with that.
To be honest though, i find the chinese social credit (sesame credit) system far more social chilling then cameras. If you are social isolated, because did not greet the police men this week, that is breaking people down to subatomic particles. That is a nightmare from which people do not recover. Cameras can be destroyed- one guy fawkes mask protest later a goverment can be gone. But if everyone is setup to shun everyone and theire are no masks- and people get not even to meet other dissidents, that is where evil lies.
Not true. You can have trusted elected people watching them, so secrets are still secrets, but trusted peiple who change regulary keep an eye on them. Accounting, requesting documenation, ongoing operations, practice, training, ect. That works as good as the political will behind it.
And sure, there can allways be black departments operating on black money, but this is a different story. I am talking about a government agency, not a secret society. But yes, the latter happens, when there is not enough watching.
And sesame credit I have to research.
(and as I said, when there is black money, it is a different story)
https://en.m.wikipedia.org/wiki/Government_Accountability_Of...
Might be much harder for you in the US, also because of the whole black money thing (CIA and cocaine), but in general it is possible.
https://en.wikipedia.org/wiki/Iran%E2%80%93Contra_affair (only time they got caught)
My assumption is they are also milking the mob.
What a ridiculous false dichotomy.
Juxtaposing "terrorists everywhere' with whatever it is you're trying to sell is sophistry at it's best/worst.
This black and white, this or that, kind of thinking is so tiresome sometimes.
Not my point of view.
My point of view is now spread out all over this thread. That it is indeed complex.
But of course, I also agree that there is also truth to that statement, the stronger your (ethical) terrorists fighting agency, the less terrorism.
But yes, if you fight terrorism by bombing villages somewhere else, maybe you have a bad long term strategy.
Ugh, that question just goes back to the boring pie-in-the-sky stuff. The interesting question is if the automated processes we build on these tools can be secured.
That's the terror the next generation has to live with. NNs can be so opaque it might be hard to tell if someone has tinkered with the input data/retrained it/overridden it. I shudder to think of all the legal implications. Like, in the above example a trout murderer could just flip some bits and nobody would know there are no more trout because the system is still reporting that there are lots of trout and the manual checks have gone.
[1] https://www.wsj.com/articles/the-all-seeing-surveillance-sta...
AI makes social change in a really strange way.
The story everyone knows, China is a authoritarian state, the gov't controls every aspect of citizen's life.
But somethings strange starts appear. You know, future CPC leaders, no matter how powerful/rich their parents are, they will be grown up in a highly supervised social environment. They are forced to think and act more transparently because everyone is in the system.
A random offensive act against the Party may put a citizen in jail. It also means a minor speech flaw made by a political figure leads to career suicide.
In the paste decades, they major problem is that CPC has unlimited power, no matter what bullshit they pull off, they always get away with it. Now with cameras and big-data shit, no matter what you want to cover, you always leave a digital trace behind. Someone at sometime, no matter citizens or your enemey at opposing political group will find it and exploit it against you.
In this "everyone's been watched" game, unless a dedicated, organized backdoor exists, everyone will be forced to comply with some kind "rule" or equilibrium, which is far different than today's arbitrary enforced laws in China.
Gov't today use keywords to monitor citizen online activities, but I've also seen a monitor program to evaluate Party member's loyalty based on cognitive science and machine learning.
Technology accelerates all of these.
I foresee China will go through a radical, soviet style social paradigm shift, but nothing like we've ever seen before.
I would be surprised if such a thing did not exist for those that run the system.
It is very common in London to see late-teenage to mid-twenties children of wealthy families in conservative regions behaving very, very badly. I’m sure there’s a similar scene in Paris and NYC. The rule is you can do what you like there but never, ever back home.
For example, take self-driving cars. Formerly all you could do if you had a remote execution exploit was smash running cars into ditches or oncoming traffic. Bad, but not the end of the world. Now you can take 200k Teslas and very easily target every gas station or other soft target in the developed world. (I'm actually concerned about this, see https://www.zachaysan.com/cars for an, admittedly long, take on the matter.)
The same is true with AI everywhere. Don't read individual Twitter accounts, scan them all, build a data model to target the influenceable influencers. Iterate.
Don't try to get your honeypot to date the head of Goldman Sachs, use data science to figure out which 1000 25 year olds have a good shot at heading it up in 10 years.
The cyber game / international relations is about to get much more extreme. Offence is so much easier than defence and AI is turning every little touch of exhaust data or device into a tool of war or espionage.
What we will get is the equivalent of a rifle in the age of swords. Maybe a machine gun, even.
I have an Android device. Google's face recognition is so good, that it was able to correctly identify people in photographs in very degraded circumstances (occluded faces, sideways, etc.). It's only a matter of time before this shit is used to track people in public places, allowing "law enforcement" to track everyone. Were you in the Women's March last weekend? Well, congratulations! You've won the "extra screening at the airport" lottery.
This is already happening. In the UK 'forward intelligence teams' (FIT) have been applying these tactics for year. There used to be a blog about this, FIT-watch dot something, don't have the UR' on me.
For applications that are cost-senstitive, we'll use AI alone. This creates a new industry in cheap intelligence, where human analysts have never been affordable.
Anti-human is the easy bit, and that fact terrifies me when I let myself think about it.
Self-replicating… well, eh. Do corporations that employ humans count?
Sentient. Hmm. We wouldn’t recognise sentient robots even of they built a neon sign and marched en masse through Washington demanding their rights — if that happened, we would tell ourselves it was a clever hack by pranksters or $disliked_nation.
Corporations don't self-replicate, at least not intentionally. They tend to grow larger and larger until antitrust breaks them up. (This has happened enough times for me to suspect that it's how things generally work.)
Perhaps even better would be if the AI rarely interceded in the human politics, and in fact worked to keep it's involvement and control totally secret.
In our 21st century world, especially one that is ignorant of the existence of strong AI, it's trivial to imagine a strong AI being able to "bend the economy to build better things for itself" while remaining completely undetected... This is essentially an AI-driven (instead of human-driven) spin on the introduction to the book Life 3.0...
That economy would also be the global economy... Not just that of a city...
If you'd like, you could even imagine that occurring as we speak...
https://en.wikipedia.org/wiki/A_Fire_Upon_the_Deep
This novel is full of ingenious ideas, one of which is demonstrating how huge differences in computing (=thinking) speed make up for fundamental differences, which manifest on many levels.
Imagine how surprising an automatic door opener would be to a person living 150 years ago. It would seem as if the door "knew" they were approaching, and politely opened for them. Imagine how surprised we would be if Clippy understood what you were doing, and offered help that was actually helpful.
As AI advances and is incorporated more and more into everyday things, our environment will become a reactive, "intelligent" black box, and along the way there will be some sudden advances that may be very startling and surreal.
I agree about this for the foreseeable future, but it seems off topic. The article doesn't suggest anything like this, does it? I'm worried you might just be responding to the title.
And defence is possible, it just takes due diligence. Or in this fantasy future of yours, an AI that probes your business for you 247 and fixes security holes it finds.
Honestly, I expect doom and gloom AI posts in the media, not on HN where people are expected to have some notion of an IQ.
AI will be like Internet. When we in the year 2040 take a look at the last 40 years, we can't point out when it exactly became prevalent. We just know that at 2000 it was not a big deal, but at 2040 it is a very big deal and shaping the world.
AI that's super-accurate at identifying potential terrorists might be a threat to me if I'm living in an authoritarian police state, but the police state is a threat to me without AI, and probably somewhat more preventable than AI surveillance. On the whole, I'd probably rather local police and intelligence services used a perfect profiling system, in the extremely unlikely event of such a thing ever existing, to using the guesswork they do now. Much more threatening to me and much more likely to be actively being used is AI that's actually pretty weak at identifying potential terrorists but trusted because it's pretty darn efficient at finding something that's difficult to explain on targets' computers. Same goes for AI databases purporting to be super-accurate crime prediction or credit granting or job selection models that are too intractable for anyone to realise they're basically doing the same job as a guy building a race and class based regression model. And of course where it comes to cyber-battles, the guys on offence tend to have a lot less need to worry about the accuracy of their AI's targeting and whether it's vulnerable to being gamed or just plain rubbish than the guys on defence
When attacking human-managed systems they may be slow to respond, weak and unable to detect everything. But they are also weak in very different ways and will respond in different ways.
Stuff like that is what worries me in 2018. Not AI.
It's like writing "computing is going to supercharge surveillance" in 1960 or "the internet is going to supercharge surveillance" in 1990.
A technology that will "supercharge" everything will also supercharge this specific thing...
I'm not saying it isn't valuable to point out the potential negatives of a technology like this, I'm just saying it's a little amusing...
At least there's always the soothing "solution" of "regulating it". The people who get to "regulate it" are exactly the ones who will wield its power.
After some millions of us have died the other people who survive will eventually find a new equilibrium.
Now you can't even see who is targeting you.
Golly Mabel, can AI be used to supercharge finding ... https://news.ycombinator.com/item?id=16255850 ?
All this neutral technology open to the highest bidder.
Augmenting the power of the few against the many.