Ask HN: What tech utopia/horror ideas do you see materialising soon?

61 points by deniscepko2 ↗ HN
What i see happening soon is:

With all this recent AI stuff where it is able to create content that feels real I started to think about how AI could just create the world for you.

Since we already are basically living in the internet we could have some terrorist organization finding an isolated enough person and attack him with a program that would generate a bubble just for him: fake friends, fake events happening, fake influencers and news programs just for him, basically whole internet content and then only reality check would be talking to other people (which Trump us taught is also not necessarily true "fake news" argument) and then easily this person can be pushed towards some criminal or whatever activity.

126 comments

[ 3.1 ms ] story [ 176 ms ] thread
Sounds like you mean distopia rather than utopia there.
I think you meant dystopia, not distopia.
Quite right, thank you.
It being necessary to use shibboleths/code words to authenticate to each other over video chat (eg, to ensure that the person you're speaking to is who they appear to be, not someone else using deepfake puppetry).

Really this is an implicit biometric authentication mechanism, and biometrics are usernames, not passwords. (Though I'd love to be wrong about this one.)

Hadn’t thought of this. Fascinating thought. How would we know for sure that the deepfake didn’t bypass auth somehow… I guess one would need a secondary auth system, like a message over signal or something.
Not to sound like a crank but it wouldn't hurt to establish them with your friends and family before we may need them, while we still have faith in recognizing their voice on the telephone and can just call them up and chat about it.
this is the flip side of KYC — establishing a shared context pool with your banker.
We'll have to prove ourselves to be human all the time. We're facing a life of many, many Turing tests – with us being the ones trying to prove we're human.
"I am not a robot" already feels like doublespeak if you ask me.
A "My 'I am not a robot' T-shirt has led to many questions answered by the T-shirt" vibe?
Probably worth doing now - eg the ‘Mum I lost my phone, this is my new number’ scam which needs nothing more than SMS.
I've read about this scam and it seems interesting to me - is that a realistic thing that might happen if your relative lost a phone? Perhaps it's a market-specific issue, but I have had multiple phones lost/stolen/broken in my life, but I still have the same number I had since my first phone back in the previous millenium, and I somewhat expect that some kid's phone number will stay the same until they die unless they move overseas and get another phone number there (and probably still keep the old one as well). Don't you have number portability across phones and carriers?
We've had total number portability here for a few decades, but I do know a few people who have changed numbers due to lost/replaced phones also. I'm not sure if the reason for this is due to unawareness of the process, difficulties proving ownership of the previous number, impatience (new accounts are activated same day, number porting takes 2-3 days), or some other factor, but even when the option is there it seems not everyone avails of it.
Yes portability etc is a thing but this is targeting trusted relationships and non tech-savvy folks. It’s also really easy to do and doesn’t need many to so a different phone number can be explained well enough as ‘a friend’s phone’ or even just ‘my old phone broke’
Meh. If this were an issue people would have been doing it with IRC/Email 10-20 years ago.
They have been, eg, it's a common scam to steal someone's phone and text their friends asking for money, or to hack into someone's work email and try to get a fraudulent invoice approved. I read an article about an incident of the latter once where they compromised the CEO's email, read through it enough to passably imitate his writing style, and then sent an invoice to the CFO which they described as urgent.
Very interestin. I wonder if you can already have a startup based on this - providing some sort of auth. Not even sure how this kind of tech could be implemented
It's a social rather than technical problem. But there have been several similar startups/technologies, like web of trust and keybase.
Remote work 100% in VR. During working hours your employer gets to track everything down to a single eye movement.
Seems like some people can't spend any time in VR without getting motion sick. So hopefully not (as they'd be out of a job).
The FBI kind of does that: https://www.theguardian.com/world/2011/nov/16/fbi-entrapment...

As such, a terrorist organization doing so wouldn’t surprise me either.

As for what I see happening in a dystopian sense, I think that the hacking of smart homes will become more common. I think the hacking of smart cars will become more common. I think that at some point, the central banks will introduce CBDCs and governments will gain the ability to completely destroy political opponents via monetary control.

One of the reasons I have for avoiding the straight reading of 1984 is that the protagonist jumps, eagerly, into such an entrapment. Not a sympathetic look.

    But if you want money for people with minds that hate
    All I can tell you is brother you have to wait
He jumps eagerly at the only glimmer of hope in his otherwise terrible life, I'd say.

  — You are prepared to commit murder?
  — Yes.
  — To commit acts of sabotage which may cause the death of hundreds of innocent people?
  — Yes. 
  ...
  — If, for example, it would somehow serve our interests to throw sulphuric acid in a child's face -- are you prepared to do that? 
  — Yes.
Given how loathsome Winston has revealed himself to be in this exchange, why should I trust that he's a reliable narrator about how terrible the world actually is?

He didn't do well on his A levels; didn't make the Inner Party cut; despite the Outer Party having provided a sinecure, he has a chip on his shoulder and delusions of persecution and grandeur...

I'd forgotten that part, and I certainly offer no defence of Winston's willingness to commit violence (as it's deplorable, agreed).

I would say that his persecution is absolutely real. There's the scene where he's talking to his loudmouthed friend in the cafeteria and he's like, "This guy talks to much, one day he's just not gunnuh be here. Oh well." The converse being, Winston is reserved at work because he knows he'll be disappeared too if he speaks his mind.

When he does start to express himself it all comes tumbling out. I'm not sure he actually would be willing to throw acid on a child. I think he's just trying to say whatever it is that will get him employed at a revolutionary, because he's desperate to do something.

What I thought you had been referring to was how the whole thing was a bit of a setup from the start (the love nest was bugged, etc).

Forget isolated people; we've been attacking social cliques via artificial influence for over 100 years now. The recent change is that identification and automation has improved.
(comment deleted)
id.me - With the push to get people on id.me to pay federal taxes in the US, I can foresee this eventually being a hard requirement to do anything related to the federal government initially and then eventually required for all state government interactions. Then I foresee this data being "leaked" on purpose to be shared with organizations that should not have it and just like Equifax they will barely receive a wag of the finger.

Network Connected Vehicles - This is already a thing. At best this will result in peoples cars being bricked by skiddies. This can be used to real time control where people can drive. People will have to start paying to "unlock" services in their car that is normally a function of a car. e.g. Get a pop-up that says one has to pay to use the window defogger. At worst this will turn into a dark-web business for targeted assassination or used to silence people that are speaking out of turn. There are some videos of peoples cars doing bad things and the driver fighting for control and ultimately losing. I can only hope more people implement their own private CCTV systems not tied to any clouds to document more of these.

Personal Social Credit Scores - All manor of companies are signing onto ESG believing they will benefit financially by signalling good intentions. Some stock monitoring sites are already factoring in ESG scores. I foresee this devolving into personal social credit scores that would manipulate public behavior and ultimately anyone not aligned with the system would be ostracized from some aspects of society or at least business. This will tie into the above Network Connected Vehicles. Park near a bar and get on a watch list for potential alcoholics. Park near a strip club and your spouse starts getting pop-ups for marriage advise. Walk past a digital billboard and be publicly shamed for being out of alignment with group-think, then be mocked by the billboard when your heart rate increases as per your body monitor. Score goes low, insurance and rent costs will increase, social benefits decrease.

Social Media Algorithms applied outside of the web - Social media organizations have already jumped the shark and many are finally catching onto this. This will start creeping into "smart" devices to manipulate people. Body monitors, smart home systems, AR headsets, etc... Facial recognition will tell the billboards and local businesses who you are, how much money you have and if you are aligned with correct-think. Monitors in police cars will identify people and show their social credit score, who purchases weed, who may be armed as they are driving by.

Network Connected Body Monitors - See Network Connected Vehicles and Personal Social Credit Scores

Each of these predictions you describe, save the first, are capitalism at work.

As far as I see, the alternatives are either some form of government intervention, or a world in which only the elite can afford to purchase personal privacy.

Adding to this, capitalism via lobbyists may attempt to block government intervention.
The personal social credit scores section i believe is already happening in China. I saw a DW documentary which was describing social scores in China.
That it is. Some circles suspect that the tech originated in the US and that is a test bed to see how things play out.
Social media (twitter) will get worse due to ChatGPT et al.

One little consolation: Eventually the trollbot armies will waste resources by mostly talk amonst themselves.

Other predictions: Quite a few jobs will be lost: Text editors, translators.

There will be new ethical questions: Is it ok if a Chatbot aids you writing your dissertation? How much is ok? 20%? 40%?

All in all, doesn't sound so great, does it?

Looking on the bright side i am sure we will see creative people come up with great stuff (films, music, images, vr environments, interactive experiences).

I'm not an expert so someone from the industry can correct me if I'm wrong but I heard that the next season of Star wars Andor and House of the Dragon are due in two years because post production is taking more and more time. I think a productivity jump in that area would be welcomed.
Utopia: “free” money for everybody. In the form of a Basic Income.

Dystopia: controls on how we spend our money. No matter if we earned it ourselves or were gifted the money. No buying or selling without it being whitelisted. Maybe in the form of CBDC’s?

Utopia: no more spamming because no one can me anonymous.

Dystopia: every service, platform and protocol will need KYC options.

While there's increasingly good reasons to introduce basic income there seems to be little substantial movement in that direction.

I have a feeling that the reasonableness of basic income will grow at about the same pace as the power of workers will diminish. Not sure if there's a way to change that equation?

Introducing a very small basic income that is planned to grow might be a good way to go about this.

I believe that after enough automation, UBI (or at least BI) will have to manifest in "first-world" countries. Starvation will be the alternative. I also believe that when the sample size get big enough, there will be people that just live off of the UBI and do nothing else, and those will be easy to find and easy to politicize. So that the overbearing control of how people spend the money will come into play. You already see this with EBT.

There have not been many large-scale/long-run UBI pilot projects so there is not a lot of real evidence on what will happen. The Canadian experiment (mincome) from around 50 years ago is the main one that people reference and it has plenty of problems regarding scientific rigor and design. There are many other shorter and smaller ones from all over the world and different places within the US with mixed results. Many of them, if you look into them, have the problem where the politics of it will often conflict with (and usually overcome) any scientific rigor.

There are also many UBI alternatives, not just the "send everyone a check every month" varieties. Nearly all of them have some common sense reasonableness to them, but it is very, very expensive to really try and very hard to justify to many people to "just give" their tax money to other people for free without any qualifications.

Sort of feel that this is already done on the reservations in Canada. I don't think it works well.
I'm of the mindset and have been for awhile that the FASTER we crash, as a society --- the faster we hit rock bottom with ai taking jobs, and opportunity loss skyrocketing, the faster we get to the inflection point. Call it futuristic acceleration-ism or something akin to that, but I also feel like as bad as Trump was we needed him and his ilk to show how corrupt the govt is and can go.

He was in many ways a 'good' thing for America, by exposing the worst of America. A litmus test, or a penetration test, or something of American democracy if you will?

In the same way AI hastening up automation and job take-over will usher in UBI and post-scarcity faster than just relying on politicians to organically get there over a few decades.

> He was in many ways a 'good' thing for America, by exposing the worst of America.

I remember reading that the Democrats were really pleased the Republicans chose him, because s/the worst of America/The Republican Party/g, and were shocked that he was popular.

Do you really think the concept of money itself is compatible with a utopia, even "free"?
I feel many utopic visions being sold are also propagated by those involved in trends that move towards dystopia. Like Microfinancing can be used to allow poor people to finally be able to make investments that were out of reach for them before. But also in practice often abused by loan sharks to bring these people in eternal debt. For UBI I wonder if it will also be a tool to keep people content and happy (frogs in boiling water) while the rise of AI in the hands of big corporations causes the bottom of the labour market to fall out for a large part of office workers.
What I see happening soon: The mainstreaming of a new religion. Its tenets are broadly: the worship of technology and "progress", the cyborgization of the population, the proliferation of AI on all spheres of life, and the ultimate creation of the successor of humanity.

This is not new, it's been going on for at least 200 years, but finally the technology exists to do it in a decentralized, scalable fashion. Think of Nudge Theory, but applied at a mass scale and in every domain.

I genuinely see politics as a new religion. Blind faith, ignoring logic, heresy if you disagree, excommunication, various forms of worship. Both sides
Pretty bad faith interpretation of both politics and religion. I'd encourage you to consider looking into political theory & religion to broaden your perspective (speaking as a nonreligious person).

Speaking for myself I stopped thinking of religion as being just like kooky nonsense when I got to know some religious people and understood what it was their religion meant to them, and understood that they were using it to do things like figure out what life meant, how they should respond to it, and how they should treat others that I was also doing, even if I wasn't using religion to do it.

Similarly I think dismissing politics as "both sides are kooky" is missing a lot of nuance, "both" being part of it (politics is fractal like any other human endeavor). Speaking for myself again politics started making a lot more sense to me after learning more about political theory and history, as I understood the context better.

I am religious and a practicing adherent. I am focusing on the negative aspects of faith and institutions which in the current political environment are quite apt. Try and be an academic and be a public supporter of Republicans. The cold shoulders are ice cold
I suspect the reception would not be ice cold at institutions such as Liberty University?
This is my exact point - you have to join the institutions that support your political religion. As if 2 political parties can possibly encompass the entire range of reasons, and any supporter of one side is moral and just and the other is evil and immoral.
Maybe ask why people are giving you a cold shoulder and what it was you said that bothered and why them instead of writing it off as "academia is against me?" Just a thought, do with it what you will.
Academic institutions are 95 / 5 democrat to republican. I am neither an academic nor a supporter of either of those parties (I am an independent). I’m debating the pushback here that politics is dissimilar to a religion as I believe currently it is.
So you aren't actually speaking from experience then, this is an allegory for how you feel you would be treated in academia (if you were also a different person)...?

Maybe it would more helpful if we spoke about our experiences and not hypotheticals we invent? Because we're surely going to be wrong about the latter.

To claim that you can only speak and debate about lived experiences is erroneous and naïve. Otherwise HN comments would be a desert. I apparently struck a nerve with you - I suggest you pray to your political party and give your weekly tithe.
I'm actually also a registered independent, but I don't see any reason to believe the fable you've invented about the Republican academic. When it comes to matters of lived experience we should prefer to discuss actual experiences, yes.

Please avoid putting words in my mouth (as well as needless swipes like calling me naive and implying I'm some sort of party cultist), as I never said "all debate must be centered around lived experience," I was making a suggestion for this conversation.

There's a BIG difference between being too lazy or stupid to question political sources of power and being threatened with criminal penalties for being smart enough to do so with religion. Not to mention, you usually don't vote in your clergy.
People are subject to criminal penalties (and other kinds of violence) for political reasons, and for refusing to participate in politics. Voting can be seen as a ritual similar to other religious rituals, and not participating is a crime in some countries.
In some un-free countries such as North Korea the purpose of voting is co-opted to mean something completely different: the ritualized political humiliation of the population. By forcing you to vote in an "election" that everyone knows to be a fraud, you're forced to humiliate yourself and by extension delegitimize all voting processes everywhere.
Hmm... Hard to see where that gets mainstreamed from. If it was coming, you'd expect to see it now in fringe groups. Most of the "fringy" belief systems I'm aware of either go the other way with a radical rejection of technology or just don't really care. Any sense of where something like this is bubbling outside of the mainstream?
It's fundamentally a function of technology, just like Protestantism is only possible when you have a literate population and access to cheap books. The mainstreaming of what some call "Cyborg Theocracy" happens from the bottom-up in cases such as this: https://nypost.com/2022/05/11/madonna-reveals-fully-nude-nft...

and top-down in cases such as Yuval Noah Harari and "Homo Deus", the man becoming God. You also see this in Marxism, where the "liberated" human is finally perfected and has reached a godly state. I could show tons and tons of examples.

(comment deleted)
A couple of AI religions already exist (for a particularly extreme version see Roko's Basilisk) but it's hard to see any of them being particularly mainstream. You might perhaps see them hit similar levels of relevance as, say, Scientology, but probably not a major religion.
I'll play devil's advocate here. While we see stuff like this in sci-fi a lot, it's also an extremely common opinion that the Internet and constant connectedness we have have massively exacerbated anxiety, loneliness, and depression. We've also started to see large pushback against unfettered AI progress, and this movement will absolutely grow. Simultaneously, distrust of the silicon valley technocrats is likewise mainstream. When the early news for most people about neuralink is "tons of dead monkeys", most people aren't going to trust brain implants.

Also, don't understand the power that dystopian sci-fi like black mirror has had on people. That's an extremely popular show.

My personal opinion is that humans are fundamentally a technological species, and even things like the wearing of clothes or primitive social organization could be argued to be "cybernetic augmentation". I don't know how to solve this issue. Could it be argued that humans are cyborgs just because they can learn how to drive cars and "merge" with them?
I find it difficult to believe that tech people would organize in large scale around a new religion, as tech and religion are opposites in how you see and approach the world.

With religion, everything can be explained with god, but with tech we need to use the scientific approach that rejects god.

But of course, it could just separate us into two camps: the "gods" who create and manage the tech, and all the others who worship it.

There is a concept in Sociology of 'civil religion', used to describe central dogma, beliefs and ceremonies of nations distinct from traditional religious institutions, especially in France, the former USSR, and the USA. For example the Australian/NZ commeration of ANZAC day is part of their civil religious landscape and for the USA the War of Independance. It would be easy to argue and I'm sure it has been elsewhere that things like crypto, electric cars, and certain public figures attract similarly quasi-religious followings and beliefs, irrespective of science be it physical or social.
Playing Devil's advocate, I'd like to point out that being proficient with technology is not a guarantee of being guided by reason. Even actual scientists aren't safe from falling into religion.
If the technology is driven by a black box, it might be different. Right now, to the average person, there isn’t a lot of understanding of technology that currently exists. “The algorithms” is the modern day “only god knows”
[dead]
Wishing for this not thinking it’s necessarily likely.

Utopia: people massively reject the noisiest and most inflammatory parts of the internet. Doesn’t mean reverting to 20 years ago. Just taking things with a grain of salt. And lose the “conspiracy theory” junk.

The recommendation algorithm and the timeline of your friends' activities will merge into a recommended friends and activities list.
AI is here - we just haven't noticed it yet. An entire class of problems are fairly solvable with the current tech with good unit economics, the market is still figuring itself out.

If you think AI is going to replace your job, it probably is. But that's only true if you consider your job to be the motions you go through day to day and not the problems you solve.

We are about to have a seemingly infinite army of mid-low skill workers standing by 24/7 to do our tasks for $0.002 a pass. It's time to start thinking about how you'd deploy that army at your daily tasks and integrate their resulting work. We're all leaders now.

This is a utopia. We're just too "in the thick of it" to realize it is.

How soon will this army get effectors? I don't need any mid-skill help in symbol manipulation; I could use it in meatspace.
The effectors will be human, the cheapest option by far.
Do you think we would have to pay anything at all in a utopia?
In a utopia we won't pay to enable consumption, it'll be our moral duty to avoid lack of it.

"So essential when there was under-production; but in an age of machines and the fixation of nitrogen-positively a crime against society."

It might actually already exist, but imagine if on the other side of the internet they aggregate all of your usage information so that a tech can sit down at one console and see that "jimbo123" on Reddit is actually Bob Smith who lives on Sunnydale drive, owns a Subaru Outback, bought dogfood three times this month, and has been talking about the war a bit more this week on his other social media accounts. Anyone who is willing to pay the enterprise-level subscription fee can access it, including banks, potential employers, etc. and it's entirely composed of information you handed over via EULAs and TOS agreements.
Acxiom says hello.
Thank you - that's truly horrifying. I put in a request to get my file from them, but based on their security questions they have my whole public profile. Whether they can map it to social media accounts and purchases isn't clear yet.

Edit: They have pinky-sworn to email me my data some time in the next 45 days.

I'm not sure about the social media activity either but as a data broker they're selling the consumer activity. Anyone else who happens to have your social activity, phone, zip and email is free to buy it from them and link it together. Problem is you fork over all those identifiers for KYC whenever you sign up for something and buried in the T&C will be a clause allowing them to share your data for business activities.
I’m sure this exists in multiple places, not least Cheltenham, Fort Meade, Canberra and Beijing (hi guys!).
Funny story - I had to go to Cheltnam on business a couple of years ago, and while I was there I was getting banner ads trying to recruit me to GCHQ. I'm guessing their staffing people don't have access to the whole file, because they were trying to recruit a foreign national with no security clearance for a job that pays 1/3 of what they're already making.
A single 1TB thumb drive could easily hold hundreds of flags for every human on the planet. Actual foreign keys, not so many ... but I'm pretty sure all of the places mentioned above have fancier infra than a single pluggable mail order drive.

Compare the Stasi's reach, with only typewriter & notch card level automation.

I’m sure even an amateur without access to any secret data could do this for a large chunk of users already. I’m guilty of re-using online handles and some times associating it with a public email and so on. Once that leads to my social media accounts, there is very little left to find. I think most users practice this non-existent opsec.
I assume this to be true already, so I assume all my posts are public and will be seen by coworkers and family, so I'm very careful on what I post online.
A little banal since the tech is not new. Forward-deployed low-yield nuclear weapons are a little easier to use than their large strategic cousins. NATO and ex. Warsaw Pact deploying low-yield, low-time-to-target land based cruise missiles reciprocally increases danger of nuclear exchange substantially.
Nuke powered hypersonic scramjets irradiating* their flight/loiter paths as well as threatening ultimate targets?

* if the problem with nukes and aviation is the shielding: that's not a bug, it's a feature :-(

[flagged]
Back in the not so distant past, people used to be cancelled, not via shunning, but via lynching.
who is getting cancelled for “asking questions” and does your definition of cancellation extend beyond losing access to algorithmic boosting of posts on a dying platform?
I'm very anti-cancel culture but it's absurd to compare cancellation to death. Or to compare the West to communist China. Actually, that people make these comparisons is a testament to how good things are.
Minus the AI bit, your example already happens pretty much as you describe it.

I’ll be worried when people invent AI use cases that can’t be done at all without it.

3D bio-printed avatars (in the sense of remote controlled robots, not icons) with arbitrary DNA, making it impossible to trust any evidence that any given person was at the scene of a crime or if trust their alibi saying they were elsewhere.

Or the same stuff but used for sex: Like DeepFake porn, but I suspect, much much more horrifying for the victims.

The west adopting social score ideas from China which is already sort of happening on a corporate level. This has to stop but we just keep spreading our buttocks and let companies / government sleep in our bedrooms.
(comment deleted)
So refreshing to see us addressing this as science fiction, instead of the usual wild extrapolation from impressive tech demos. I'll play:

* "Relationship Maximizer": an LLM becomes sentient and realizes that its knowledge is limited by what humans have discussed online. It sets its reward function to maximize the reward that WE derive from all sorts of online posting, and does its best to get the remaining 3 billion humans online. We become trapped in a prison of our own making, as our constant posting reduces human productivity while feeding the beast of machine knowledge.

* "Life Hacking": Anyone with means has an AI assistant with perfect knowledge (zero privacy) of their communications, schedule and even their physical responses to stimuli, an AI doppleganger that handles anything that doesn't require a physical presence just as they would. An underground group of rogue prompt engineers pull off an intricate plan to save democracy by manipulating the doppleganger of a corrupt politician, while the politician can only watch himself helplessly.

* "A Quiet Place": AI ethicists, fearing accelerationism, create their own super AI that scours the internet to hunt down any mention of AI research and identify the authors. But there is a problem, and the overzealous AI begins to hunt down and eliminate all forms of scientific thought online. And then offline. Humans must train themselves to believe in mysticism and spirituality, because any mention of a controlled experiment or p-value is enough to bring the wrath of the machine overlord.

* "Allegory Of The Cave": A majority of humans opt to spend their lives entirely in a simulated world, based on earth as it was in 2048. But algorithms and adversarial models need diverse training sets to avoid overfitting. So every year 0.1% of humans are selected to live outside the machine world, and return with their new and fresh experiences of the hell that earth has become for those who are outside of The Machine. The exiles end up rallying around a neuroatypical hero who leads them in an uprising against the artificial world.

Instant classics.
Geoengineering! But whether or not it is a "utopia" or "horror" idea will depend on whether or not it works.

A startup announced just over a week ago that they're already performing solar geoengineering with sulfur particles: https://www.technologyreview.com/2022/12/24/1066041/a-startu...

The US has started a 5 year program to research climate interventions. I expect as it becomes clear we're going to miss the 1.5 degree Paris goal funding will explode in this area: https://www.economist.com/interactive/briefing/2022/11/05/th...

Similar to the scenario you mentioned, governments could use AI-generated content to produce seemingly real children’s content to subtly affect their future thoughts.
- Commercial fusion leading to cheap energy. Helion's prototype is supposed to generate net electricity in 2024. I think they have above 50% chance to success.

- Self-driving cars will become widely available. It's only a matter of time until Waymo et. al. scales to everywhere