A self-sustaining civilization of robots. Humans are gone, but generations and generations of robots evolve and, eventually, incorporating organic compounds, they become cyborgs.
However, the robots were doomed by their limited intelligence and their fervor to produce additional robots. The robots eventually depleted all available resources, thus making the planet uninhabitable for themselves.
Until they learned to explore the cosmos and do this on planet after planet leading to the Grey Goo[1]. Or the flipside, in that they create a Big Bang to reverse it like the multivac did in Asimov's The Last Question when they finally have sufficient information to process it.
If you would assume technology to another biology- then a error is a mutation, thus robots would divide into predators and hunted animals, resulting in a new wildlife.
What I am worried about is that in future AI swarm military robots will enable something that wasn't seen yet in history. Now to make dictatorship state you need support of military/rebels/people behind it, you can't make it on your own. Without humans in equation who will stop some oligarch with enough money to take over some nation in for example Africa or Middle East? You wouldn't need hundreds of thousands/millions people supporting you anymore.
There is no need to create a new robot army if you can blackmail the people who control the existing army.
For me, the most interesting revelation in the recent Wikileaks dumps is the surprising willingness of prominent individuals to share their most private thoughts via gmail.
Larry Page (or his unscrupulous AI successor) may already have enough blackmail material on hand to seize the levers of power, if he so chooses.
In that case, Google's new motto could be, "Absolute power corrupts absolutely".
Coincidentally, I'm re-reading Lord of the Rings at the moment. Not sure if a work of fiction is the best source of moral lessons, but the whole point of the book is that the Ring of Power would corrupt even the most incorruptible. And somehow, this doesn't surprise me.
Why is a moral lesson in a story a moral lesson? I can invent any moral lesson and put it in a story. That a supposed moral lesson is featured in a work of fiction does not lend credence to it.
So you acknowledge morals can be told through stories. And they are. Like, you know, in the Bible or fables. You just don't believe they have "credence". Which I never asked about. Good debate.
You seem to be extremely confused. Of course moral lessons can be taught through stories. That a moral lesson is taught through story does not make the moral taught true in any sense. Which makes bringing up the One Ring and Tolkien extremely strange.
I am not confused at all. Perhaps you're projecting your own confusion onto me. You simply have an unbelievably naive viewpoint with no coherent argument.
My point is that exploring and conveying "morals" is actually one of the primary functions of literature. I can't tell if you agree or disagree with this. But if you believe that something being "fiction" means it's not worth paying attention to or that it cannot offer valid moral instruction, well, you are a tremendous fool.
Whether you agree or disagree with the moral or ethical argument being made is a completely separate point.
> That a moral lesson is taught through story does not make the moral taught true in any sense.
I never said that a moral lesson appearing in a story makes it "true". That's a stupid thing to even suggest. You are, however, suggesting that any moral lesson in any narrative that isn't "true" (would you mind sharing your definition of true so we can determine how we can come to that conclusion?) cannot contain a valid moral. Which is absurd.
Checkout a book called "Kill Decision". It's not as far ahead as "dictator with robot army", since it's attempting to be close to what's possible /today/.
I thought Daemon and Freedom were better, (but you have to read them both! It's more one book than two.) but Kill Decision does have some interesting points to it.
They were definitely better. I'm decently sure that Suarez had a few great ideas, and a wrote his first book using the best one, and was then on the hook for more books.
Edit: Oh! If you liked Daemon, check out Nexus. Same "problem", different core tech solution.
Great read...how about media ? They are playing larger and larger role in how our elections are run. One big media comapny could have indirect affect on how people will feel and behave including governments, armies and nations.
Obviously only evil Trump supporters who hate democracy would distrust the media, though. I mean, haven't you read the news? (Also, one of the complaints in this talk is that Facebook and Twitter don't use their power to control how people feel and behave, so the author likely thinks of the media using this power as a good thing.)
> They are playing larger and larger role in how our elections are run.
What?! How many news outlets endorsed Trump? How many fired people because they're making less money? Unless Macedonian Facebook spammers count as "media" these days, "the media" has less money, less reach, and less influence every year.
This is genius in that it starts by seeming to be a portentous talk about the evil other and turns the tables to show that the enemy may be us: How do we let ourselves be robots? Do we really like social control? Why do we expect the technology that got us into these problems to get us out of it.
I don't think we are really in a terrible place. In so many ways the world is better than it was. But we are kind of blind to the problems we are creating. Read this.
There's no "we" about it. Broadly, there are people who like having social control, people who like feeling that others are under control, and people who don't like control.
To be technical, that'd be the second group if it was phrased as "people who like feeling that others are in control", using the word "in" instead of "under".
I believe there is another group: People who like control, when everyone involved understands and agrees with it.
It can be very hard, and it involves a consistent questioning of authority. Including the authority of ourselves, and the authority we hold over others.
We can have modern medicine, but we have to trust the people researching it.
We can have modern robots, but we have to trust the people building them.
Fear is a great way to prevent people from questioning. But if we learn to question our fear itself, it can be useful to have been afraid.
I don't think we're blind to the problems being created. It's more that there are no obvious solutions beyond stasis, which is no real solution as it just fixes us with today's set of problems instead of tomorrow's.
I like Maciej's talks, they're always entertaining, but at the end of them I always feel not quite satisfied. He is good at presenting well known problems in amusing ways but rarely identifies solutions beyond the hopelessly vague (like "we should all think about stuff more").
He is also utterly resistant to giving large companies any credit at all. Google and Facebook have moved the needle on strong cryptography and anti-surveillance tools more than any other groups, but in these talks they're always the bad guys because they make money through advertising instead of credit cards (which are legally linked to your real name and address: far far worse for privacy than ads).
A big part of this talk boils down to, the US Government can order American companies to undo any protections they themselves create. But this is hardly worth commenting on. It is a political problem and the only solution is for political candidates to appear that manage to appeal to large numbers of voters whilst simultaneously being strong on civil rights and shutting down surveillance systems. Neither Clinton nor Trump seem very likely to do that, but nor did any other candidates. The issue matters to people but it matters a lot less than other issues.
Raising awareness is important - and urgent - work. These problems are not "well known" outside of the HN/tech bubble.
> in amusing ways
Also important, as it is much more memorable than the exhaustive clinical delineation offered by (e.g.) Pro Publica
> but rarely identifies solutions beyond the hopelessly vague
If you don't have a solution to what you see as a critical problem, do you wllow in despair? Or do you raise awareness of the problem in the hope that, individually or collectively, others will devise solutions?
There are two point I would dispute. First, I do give large tech companies (and the NSA!) credit for being internally heterogeneous, and doing good things along with bad.
Second, it would be ridiculous to give Facebook or Google credit for making anti-surveillance tools. That would be like praising tobacco companies for inventing a better cigarette filter.
My beef is that you sort of place the finger of blame on big companies for the concentration of data in the cloud, even though that appears to be a natural evolution and what users want. It's not like Google or Facebook engaged in an evil master plan to force users to give them lots of data. Users willingly did this because they didn't want to manage that data themselves, and if Google/Facebook hadn't offered them that service they'd simply have gone to another company that did.
Given that this is the way technology has evolved independent of any one firm, is it really fair to compare them to tobacco companies?
I yield to no HN commenter in sympathy for the Google security team, as my comment history will surely indicate, but it's possible to take that sympathy too far. Yes, Google's security team has done important things for the privacy of all Internet users. However, for its own users, it has also created the largest subpoenable body of private information in the history of the world.
Google's security team has its hands full just shipping a browser that doesn't unlock the computers of all its users for anyone who can write a heap overflow exploit (which is why you should probably prefer Chrome over other browsers). They are not in fact moving the needle on the problem of protecting their own data from hostile governments.
My take on the reason for the US's endless warfare is that it is simply a way to transfer billions of dollars of tax money every year to arms manufacturers and their political allies.
Hopefully it's something that can actually be used to make decisions and not some pandering fluff designed to appease the norms of their own thoughtless and cynical in-group.
On the other hand, maybe it's actually ok to not pretend to have all the answers.
Well, I want to understand why people would actively engage in and promote something so awful and also, on the surface, so illogical and wasteful.
This explanation makes sense at least to me. You cant improve things without first understanding them.
There will be a time in the near future where a CEO of a large company similar to Google will have the ability to control a billion robots, Google has over a billion android phones that it could subvert if it wanted to. Such level of power would effectively allow such a company or CEO to take over the world.
At the end though, such a CEO might think of him or herself as being totally altruistic and saving the world from its idiotic leadership. US is already headed towards tyranny. The political climate could be just rife in 30-40 years for that sort of thing to be justified by a well-meaning person.
He'll only do that if the political environment becomes an active impediment to building a city on Mars, since taking over the world is a fairly major distraction.
"Of all tyrannies, a tyranny sincerely exercised for the good of its victims may be the most oppressive. It would be better to live under robber barons than under omnipotent moral busybodies. The robber baron's cruelty may sometimes sleep, his cupidity may at some point be satiated; but those who torment us for our own good will torment us without end for they do so with the approval of their own conscience."
wikileaks publishes leaked documents. they're propaganda in the sense that they present information under an ethos meant to serve an ideological purpose, but that's a meaning so broad that cnn and npr also easily fall under it. if you take issue with the veracity of what they've published, make a case, but otherwise, calling something 'propaganda' and thinking that alone is somehow a knock to its value or truth is meaningless.
i'm totally willing to believe that russia obtained the emails and gave them to wikileaks -- it's far from proven, at least on the public side of discourse, but it makes sense and doesn't seem impossible or bullshitty to me. but that doesn't mean that the emails aren't real. it's foolish to cover your ears and ignore things just because someone you don't like said it.
wikileaks has also published much, much more than the dnc/podesta emails, from a very wide variety of sources.
>i'm totally willing to believe that russia obtained the emails and gave them to wikileaks -- it's far from proven, at least on the public side of discourse, but it makes sense and doesn't seem impossible or bullshitty to me
Yet we've seen zero evidence for it. Blaming Russia is convenient for DNC since it shifts the dialogue from the emails to "evil empire" that's supposedly behind it. First, they attempted to claim that emails have been tampered with and then when DKIM signatures proved that hypothesis wrong, they shifted their tactics.
Funniest part of that whole election was the claim by Dems that Trump has some kind of a secret server that communicates with Russians. Laughable. It's all just a misdirection.
there were also, iirc, at least six other private security companies that came to the same conclusion, who weren't hired by the dnc. additionally, every US intelligence agency backed up the claim.
again, not a proven statement, but there's a substantial amount of evidence we're aware of.
>First, they attempted to claim that emails have been tampered with and then when DKIM signatures proved that hypothesis wrong, they shifted their tactics.
do you have a source for the first part? afaik it was just twitter randos floating the idea
i'm not a partisan in this fight. i didn't vote, i despised both candidates, but i also think both sides can be a little blind in their eagerness to paint the other as disingenuous. i'm very interested in this subject and it's one i've followed closely, and i'm reasonably sure russian state actors were behind the dnc and podesta hacks, but i also think it's worth keeping an open mind as a general principle and not to be blinded by ideological alliances.
This is a pretty brilliant talk, frightening and funny at the same time; in the end I take refuge in the fact that the robotic future will likely not involve humans as no sentient technology can be as stupid as we are and will likely eliminate us at the first opportunity.
Wait... You're saying that you're hopeful that something we make can fix the mess from us fucking up making things because it'll be less fucked up?
I'd find some agreement with you that logically we're the dumbest possible intelligence, because the trend is up and we're at the beginning, except we're not at the beginning, people like the Sumerians were. Except they're not the beginning, either...
I suspect the answer (perhaps unfortunately) will be 'anyone with the money to afford the technology'. I can see an actual industry appearing around robotics and AI controlled weapons systems, and a fair few companies willing to sell to special interests outside of the police or military.
So I suspect we'll get a situation where everyone from the police and military to political groups, companies (mostly larger ones), richer/more tech obsessed civilians and criminals have access to military grade robots. Like a more expensive version of firearms in the US.
Consider Pokémon Go, which when it was initially released required full access to your Gmail account. To play America's most popular game, you practically had to give it power of attorney.
To nitpick this totally inconsequential point: you could use a Pokémon Trainer Club account instead of a Google account if you wanted to. Most people still chose the convenience of logging into their existing account instead of creating a new one, though.
When you request account access, there's a scope parameter to specify which aspects you want. They had inadvertently set that to "all" instead of "user info". Those oauth tokens would have been keys to the kingdom.
Why do we even have armed forces and the police? There are a lot of different reasons, but the big one is to defend property. And, who owns most of the property? And, who will own most of the robots? Fewer and fewer people as time goes on.
The other day, I heard a guy joke (in very bad taste), that when there are no more jobs left, there will always be prostitution and slavery. It's a sick thought, but how far off is it?
Free market is definitely not helping here at the moment. To get Star Trek (sans transporters and warp drive), what we need to do is to make all our existing tech interoperable. Which is exactly the thing businesses are icentivized against. Hence the utter bullshit mobile ecosystem is, not to even start on IoT...
An extensively researched talk. I hope people like
Boredom and ennui. What are you going to do when
Most labour is done by robots, and you sit at home
Collecting a monthly cheque from the government.
I know this is provocative, but many occupations
Are simply busywork.
I do think robot armies can be hacked, so try not to
Centralize everything.
Hiking, reading, writing, walking, talking, singing, loving, painting, cooking, eating, competing...
Everything that people say "I wish I did more..." on their deathbeds.
General Grievous. But actually Count Dooku. But actually senator Palpatine. Who has recently been elected Supreme Chancellor. But actually Darth Sidious.
171 comments
[ 2.9 ms ] story [ 215 ms ] threadProblem eliminated and no other unintended consequences as far as I can see.
[1] https://en.wikipedia.org/wiki/Grey_goo
Otherwise it's going to be kind of a dice roll--what are the values of the first team to achieve the governance-eclipsing technology.
So like all history until now.
The winners dictate morality. Nothing will change.
For me, the most interesting revelation in the recent Wikileaks dumps is the surprising willingness of prominent individuals to share their most private thoughts via gmail.
Larry Page (or his unscrupulous AI successor) may already have enough blackmail material on hand to seize the levers of power, if he so chooses.
In that case, Google's new motto could be, "Absolute power corrupts absolutely".
To me this is the most plausible direction.
Personality characteristics, like "The Dark Triad/Tetrad," do a better job of explaining tendencies for misbehavior.
https://en.wikipedia.org/wiki/Dark_triad
My point is that exploring and conveying "morals" is actually one of the primary functions of literature. I can't tell if you agree or disagree with this. But if you believe that something being "fiction" means it's not worth paying attention to or that it cannot offer valid moral instruction, well, you are a tremendous fool.
Whether you agree or disagree with the moral or ethical argument being made is a completely separate point.
> That a moral lesson is taught through story does not make the moral taught true in any sense.
I never said that a moral lesson appearing in a story makes it "true". That's a stupid thing to even suggest. You are, however, suggesting that any moral lesson in any narrative that isn't "true" (would you mind sharing your definition of true so we can determine how we can come to that conclusion?) cannot contain a valid moral. Which is absurd.
Edit: Oh! If you liked Daemon, check out Nexus. Same "problem", different core tech solution.
Turn out Youtube didn't manage to stop him.
J. R. "Bob" Dobbs?
If so, pardon my intrusion, and abundant slack to you.
What?! How many news outlets endorsed Trump? How many fired people because they're making less money? Unless Macedonian Facebook spammers count as "media" these days, "the media" has less money, less reach, and less influence every year.
I don't think we are really in a terrible place. In so many ways the world is better than it was. But we are kind of blind to the problems we are creating. Read this.
It can be very hard, and it involves a consistent questioning of authority. Including the authority of ourselves, and the authority we hold over others.
We can have modern medicine, but we have to trust the people researching it. We can have modern robots, but we have to trust the people building them.
Fear is a great way to prevent people from questioning. But if we learn to question our fear itself, it can be useful to have been afraid.
I like Maciej's talks, they're always entertaining, but at the end of them I always feel not quite satisfied. He is good at presenting well known problems in amusing ways but rarely identifies solutions beyond the hopelessly vague (like "we should all think about stuff more").
He is also utterly resistant to giving large companies any credit at all. Google and Facebook have moved the needle on strong cryptography and anti-surveillance tools more than any other groups, but in these talks they're always the bad guys because they make money through advertising instead of credit cards (which are legally linked to your real name and address: far far worse for privacy than ads).
A big part of this talk boils down to, the US Government can order American companies to undo any protections they themselves create. But this is hardly worth commenting on. It is a political problem and the only solution is for political candidates to appear that manage to appeal to large numbers of voters whilst simultaneously being strong on civil rights and shutting down surveillance systems. Neither Clinton nor Trump seem very likely to do that, but nor did any other candidates. The issue matters to people but it matters a lot less than other issues.
Raising awareness is important - and urgent - work. These problems are not "well known" outside of the HN/tech bubble.
> in amusing ways
Also important, as it is much more memorable than the exhaustive clinical delineation offered by (e.g.) Pro Publica
> but rarely identifies solutions beyond the hopelessly vague
If you don't have a solution to what you see as a critical problem, do you wllow in despair? Or do you raise awareness of the problem in the hope that, individually or collectively, others will devise solutions?
There are two point I would dispute. First, I do give large tech companies (and the NSA!) credit for being internally heterogeneous, and doing good things along with bad.
Second, it would be ridiculous to give Facebook or Google credit for making anti-surveillance tools. That would be like praising tobacco companies for inventing a better cigarette filter.
Given that this is the way technology has evolved independent of any one firm, is it really fair to compare them to tobacco companies?
Google's security team has its hands full just shipping a browser that doesn't unlock the computers of all its users for anyone who can write a heap overflow exploit (which is why you should probably prefer Chrome over other browsers). They are not in fact moving the needle on the problem of protecting their own data from hostile governments.
On the other hand, maybe it's actually ok to not pretend to have all the answers.
https://en.m.wikipedia.org/wiki/The_purpose_of_a_system_is_w...
After all, that is what we incessantly task the government with doing. "100% employment", "create jobs", "create jobs"!
― C.S. Lewis
Fascism reeeeeeeeeeeee!!
[I was going for a chuckle here]
https://www.youtube.com/watch?v=pMYYx_im5QI
http://www.npr.org/sections/parallels/2016/08/19/490515026/i...
Edit. Another source (of many):
http://edition.cnn.com/2016/10/13/politics/russia-us-electio...
The amount of anti-russian nonsense coming from DNC/Clintons and liberal is off the charts.
And please point out when WikiLeaks was wrong. Go ahead, I'll wait.
i'm totally willing to believe that russia obtained the emails and gave them to wikileaks -- it's far from proven, at least on the public side of discourse, but it makes sense and doesn't seem impossible or bullshitty to me. but that doesn't mean that the emails aren't real. it's foolish to cover your ears and ignore things just because someone you don't like said it.
wikileaks has also published much, much more than the dnc/podesta emails, from a very wide variety of sources.
Yet we've seen zero evidence for it. Blaming Russia is convenient for DNC since it shifts the dialogue from the emails to "evil empire" that's supposedly behind it. First, they attempted to claim that emails have been tampered with and then when DKIM signatures proved that hypothesis wrong, they shifted their tactics.
Funniest part of that whole election was the claim by Dems that Trump has some kind of a secret server that communicates with Russians. Laughable. It's all just a misdirection.
https://www.crowdstrike.com/blog/bears-midst-intrusion-democ...
there were also, iirc, at least six other private security companies that came to the same conclusion, who weren't hired by the dnc. additionally, every US intelligence agency backed up the claim.
again, not a proven statement, but there's a substantial amount of evidence we're aware of.
>First, they attempted to claim that emails have been tampered with and then when DKIM signatures proved that hypothesis wrong, they shifted their tactics.
do you have a source for the first part? afaik it was just twitter randos floating the idea
i'm not a partisan in this fight. i didn't vote, i despised both candidates, but i also think both sides can be a little blind in their eagerness to paint the other as disingenuous. i'm very interested in this subject and it's one i've followed closely, and i'm reasonably sure russian state actors were behind the dnc and podesta hacks, but i also think it's worth keeping an open mind as a general principle and not to be blinded by ideological alliances.
Root passwords will be handed down eldest son to eldest son.
I'd find some agreement with you that logically we're the dumbest possible intelligence, because the trend is up and we're at the beginning, except we're not at the beginning, people like the Sumerians were. Except they're not the beginning, either...
Collectively, yes. But individually, not so much, considering how hard it is to replicate human abilities in AI.
Seldom Smith, from The Monkey-Wrench Gang by Edward Abbey.
-- Agent Kay (Men in Black)
So I suspect we'll get a situation where everyone from the police and military to political groups, companies (mostly larger ones), richer/more tech obsessed civilians and criminals have access to military grade robots. Like a more expensive version of firearms in the US.
To nitpick this totally inconsequential point: you could use a Pokémon Trainer Club account instead of a Google account if you wanted to. Most people still chose the convenience of logging into their existing account instead of creating a new one, though.
We would be amused into slavery.
The other day, I heard a guy joke (in very bad taste), that when there are no more jobs left, there will always be prostitution and slavery. It's a sick thought, but how far off is it?
PID 1.
Why are people wasting time on this stuff? The "free market"?
better keep your programmers happy
I know this is provocative, but many occupations Are simply busywork.
I do think robot armies can be hacked, so try not to Centralize everything.
Ender Wiggin