My understanding is that those who work with the mentally handicapped use bright lights and other stimuli to soothe and control them. It is also my understanding that the autistic are stimulated by vibrant colors (coughcoughMy Little Ponycoughcough).
Who is to say that the rest of us are not also vulnerable to such controlling stimuli?
Isn't there that one Harry Potter warning. I think it was the potion guy who said too much luck is dangerous. I guess that is somewhat of a parallel to this. Too much positive visual stimuli is dangerous or bad.
How fried does someone's brain have to be to say "others on telegram", as if everyone is in the same chatrooms together? If they're on HN, they should at least know why that doesn't make sense, so I have to assume they're in a deteriorated mental state.
Friendly reminder to touch grass on a regular basis... Even virtual grass in a video game is a good pressure valve for excessive social media use.
There are undoubtedly 'adversarial images' that will induce certain effects in particular biological minds. It seems like this is neither good or bad inherently, just another tool.
I hope my brain is also different. I also have never spent hours scrolling through short-form videos on Instagram, TikTok, Facebok, etc. I never ever walk outside with my phone in my hand, instead enjoying the view.
I do enjoy watching YouTube videos at home, on the living-room flatscreen, on a variety of topics, but I select them manually, one at a time, from the vast selection The Algorithm(TM) offers me, plus my own searches.
My current theory is that these are similar to cigarettes. Nobody likes the first draft, it burns your lungs, your entire body wants to reject it. But the nicotine stimulates just the right receptors so that if you keep at it for just long enough, you'll be hooked and start disregarding the terrible taste, smell, tar in your lungs, and yellowing of your teeth.
All of this to say, if you subjected yourself to just enough TikTok scrolling on just the right topic, you might find yourself using it occasionally after that initial hump, then slightly more frequently, then daily.
You might still not "like" it, but the habit is what matters.
As a teenager, I tried to get addicted to cigarettes so I could stand by the cool kids and smoke with them. I started smoking 4-6 cigarettes a day but hated it so much, I couldn't continue after a week or two...
Really, I don't think you need to worry. A prevailing latent attitude in the comments is that the brain is somehow super malleable and subject to external control through some incantations. Like subliminal messaging, it will at most be moral panic that fades, and then people will act like they never believed in it.
Will be interesting to see how strong the controlling forces can be - enough to make you miss things in direct perception like in the book, or only softer effects further up the cognition layer stack
This does indeed seem comically evil. While surely this may provide somewhat interesting insights in how our brain processes things, this seems squarely past the "should" part of "you scientists were so obsessed with whether you could you failed to consider whether you should"
And I have yet to see a single paper like this where a researcher bails out and publicly says they refuse to work on such projects. Not one.
The most benign interpretation of this observation is that science is filled with spineless opportunists who don’t care who they hurt with what they create. A slightly less benign interpretation might be that many of these people are doing this deliberately, and getting off on the sense of power it gives them.
When skip level bosses on my last job wanted to do boneheaded things in automotive design they usually had to keep asking different engineers until they got a yay.
When it is pushed from the top it is hard to stop at ground level.
I was asked to train a neural network do detect how much pain a mouse was in- our partner company would be responsible for hurting and filming the mice. I refused and subsequently quit- this produced no paper and I don’t know if they got someone else to do it. I probably should have done something stronger.
I'd say we're already well past that point. Short-form "content" already exists and is messing with people's brains, this is the same thing just taken a few steps further. By the time the tech companies start using it, it will already be too late and we'll be left discussing whether the next man-made nightmare they come up with is the point where the tech industry must be stopped.
The problem is the incentive structure. We could be doing good things with tech. The incentive is to use it to addict and destroy. Addiction and war is where all the money is, not education or health care or positive forms of entertainment and art. No money in those. Why?
I’m sure there was an early hominid version of this discourse. “Maybe bad to make sharp rock and sharp stick if this what we do with it…” “Mmm yes someday we make sharp rock big enough smash world.”
"Prime Intellect, I would like you to begin stimulating the neurons of the pleasure center of my brain, one at a time, and remember the ones I report to you as being favorable."
If you read the techbique it reads like something far less remarkable being PR’d to sound like a big deal.
The fact it’s bucketing by making images of lighting and facial expressions, the fact it doesn’t natively do the video it does an image then video generates from it.
The results look really bad and samey. Doubt this would work for the actual thing they’re pitching it for.
This is already happening at scale by the social media feed algorithms. We don't need generated content to accomplish this. In a sea of user created content, plenty of it is already at peak activation.
There's algorithmic ways to accomplish the same thing without requiring monitoring of the brain, if there's enough data to pull from. Semantic analysis + engagement monitoring is more than enough. Peoples emotions expressed via comments is a reflection of brain region activation.
The end result of this will be that creating viral or maximally converting content will not be restricted to the top content creators. It will also result in down regulation of those brain regions due to constant stimulation, so the bar will increase for what classifies as attention grabbing and the brain will filter most of it out as noise.
All in all, I don't really see this being a big difference to the outcomes we're already seeing.
For one, it's bound to teach us something about our brains. And while I'm not usually an optimist, I actually find it more likely that we'll take active steps to stop the evil applications of this if research like this is out in the open.
To flip the question on it's head, what exactly do you see as the "evil" potential? The most prominent comments to this effect are just scifi handwaving.
Apart from ethically bad and evil use cases of this application, can we use it to massage the parts of brain like we do it to our bones and muscles with the help of physiotherapists?
reason I am asking it could be some relief to our brains after tedious working day, especially after heavy AI usage
Sure. The relief is actually free ; though once every 10 uses we'll spike your dopamine-related brain centers to 1,100% activation for a quick second while flashing a peptide sponsor ad, then back to normal.
You can of course, also not get the spike ads, by subscribing to Premium. Just have a quick look at this animated QR code, it will explain better than I can in writing.
Edit: there used to be an OSS app you could use where you actually pick what you want activated in your brain and when, but it's been banned after one of the commercial producers's investors convinced the government that this technology was too powerful, so only government-approved apps are allowed now, and an emergency vote passed the BrainControl law authorizing logging of all visual stimuli you look at, in order to protect your children.
I wonder if the end-game of this field of research will be to run these simulations at scale using neuron-on-a-chip services such as [0] Cortical Cloud.
Very interesting. We have an organic experiment converging to maximum stimulation in short form videos (which will become the majority of training data for future video gen models) Already approaching the capabilities of a “mood organ” from blade runner. Except usually most people don’t even make the choice to change their mood anymore.
https://en.wikipedia.org/wiki/Do_Androids_Dream_of_Electric_...
This is very similar to last week with that mind reading startup thing. Please read the paper before commenting.
This is a tool to help researchers in figuring out what different parts of the brain are actually for with less experimenter bias contamination of “well we think maybe it’s about this so let’s show it video of x to see”.
The essence runs on having someone sit in a scanner for a couple hours watching all sorts of things, and then feeding that to a model that will then build its own representation of said data and try different things on it until it’s found what makes a certain part sing in the model.
The purpose is a generalized understanding of brain function, more or less the same way we’ve been doing it all these years. Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.
>Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.
It also helps companies like Moonbug Entertainment (Candle Media) understand how to build better Distractatrons.
It’s a small TV screen, placed a few feet from the larger one, that plays a continuous loop of banal, real-world scenes — a guy pouring a cup of coffee, someone getting a haircut — each lasting about 20 seconds. Whenever a youngster looks away from the Moonbug show to glimpse the Distractatron, a note is jotted down.
“It’s not mega-interesting, what’s on the Distractatron,” said Maurice Wheeler, who runs the research group. “But if they aren’t fully focused, they might go, ‘Oh, what’s that?’ and kind of drift over. We can see what they’re looking at and the exact moment when they got distracted.”
I can see it now.. The Distractatrons: a new chapter of protagonists in Transformers! The modern equivalent of evil in this day and age of ADHD and low attention span!
the same year as Baudrillard's Simulacra and Simulation, featuring computer generated characters, brainwashing television commercials, a light pulse gun that causes absence seizures, gun battles inside a plastic surgery clinic and an AR simulation environment, a sadistic computer, a physician who doubles as an action hero, and James Coburn giving a lecture explaining the enemy's evil plans in the opposite role that he played in the (excellent) The President's Analyst.
This time I was not so dazzled and saw it for as atrocious everyone else things it is. The minions of "Digital Matrix, Inc." manage several assassinations with the light-pulse L.O.O.K.E.R. gun but when they use real firearms they outdo Vader's stormtroopers by shooting each other. (Want to see the scene where somebody from E.Y. tells them to stick to the L.O.O.K.E.R. gun) The bad guys explain the penultimate secret to the protagonist early on but the ultimate secret is revealed in the L.O.O.K.E.R. lab which doesn't feel like a lab at all but rather a rather good room in a theme park experience where you're supposed to uncover the secret. (Contrast that to the lab Doug Trumbull outfitted in the Brainstorms movie a few years later which is packed with real surplus equipment... I've been to that lab!)
> p8: "A child watching television under normal conditions is subject to frequent interruptions and distractions. The TV must vie for his attention. In order to simulate this condition, we decided to program distractions into the laboratory situation... Slides could be used to fill the slide tray and they could be projected automatically, at regular intervals, onto a screen similar to that of the television set. The carousel projector allows the viewer to choose three exposure times. The 7.45 second interval proved most satisfactory with the preschool children.
And nowadays we think of Sesame Street as wholesome compared to today's content. They may have optimised attention but they were delivering positive-value content. That's not the case any more.
What is a "purpose"? Something people wish something would only be used for, right? How does it relate to, what influence does it have on what something will end up being used for?
> The synthesized clips line up with what each region is known to care about, faces for FFA, places for PPA, bodies for EBA, motion for MT, patterns for V1 / V3A, and lively social scenes for pSTS / aSTS
Explain the potential to exploit strong stimulation of specific visual regions for evil. "Oh, I very much detect a face/place/body/motion/pattern/human", says the subject. What are you going to do with that, startle them?
Apply the same technology several layers deeper. Stimulate the novelty or sexual attraction or general reward neurons. Stimulate the thirst center in a Pepsi ad.
Do you know that strongly triggering these regions at the same time/repeatedly/in various patterns doesn't have any effect on your mood, wants, needs and so on due to experience in the field? Or why are you just discounting the idea?
IMO it's terrifying to imagine that adtech companies could plug this stuff into their optimizers and just run an experiment on broad swaths of the population to figure out if there's any effects.
Oh you are right. Let's just end neuroscience altogether. Also computer science, aerospace, and biology in general. In fact, let's go full Amish. Wait, no. Someone might use a buggy cart to run people over.
This isn't about "might", this is about "obviously will", and the sheer fact that questions that were 101 of scientific discussion now lead to instant freaking out and accusations, i.e. stupid cultish bullshit, show me we need to be more serious about it, not less. If that's not to your liking, is it okay for you if other people ponder such questions? Can you maybe stim your brain into believing they don't?
This specific research may not be morally corrupt in itself, nor may his intentions be bad, but it could absolutely lead to something horrific. Nuclear technology was also initially developed with good intentions and provided much good for humanity. But of course it ended with biggest bad actors having a lot of nuclear weapons.
202 comments
[ 3.8 ms ] story [ 66.8 ms ] threadhttps://en.wikipedia.org/wiki/Supernormal_stimulus
https://en.wikipedia.org/wiki/BLIT_(short_story)
Also relevant: <https://www.quantamagazine.org/a-math-theory-for-why-people-...>
My understanding is that those who work with the mentally handicapped use bright lights and other stimuli to soothe and control them. It is also my understanding that the autistic are stimulated by vibrant colors (coughcoughMy Little Ponycoughcough).
Who is to say that the rest of us are not also vulnerable to such controlling stimuli?
Friendly reminder to touch grass on a regular basis... Even virtual grass in a video game is a good pressure valve for excessive social media use.
Hypnagogic hallucinations: https://en.wikipedia.org/wiki/Hypnagogia
Is my brain different or am I just a grumpy millenial hipster?
I do enjoy watching YouTube videos at home, on the living-room flatscreen, on a variety of topics, but I select them manually, one at a time, from the vast selection The Algorithm(TM) offers me, plus my own searches.
All of this to say, if you subjected yourself to just enough TikTok scrolling on just the right topic, you might find yourself using it occasionally after that initial hump, then slightly more frequently, then daily.
You might still not "like" it, but the habit is what matters.
Prior exposure to worse feeds gives like an analytical look on the vids rather than emotional.
Nothing special compared to purpose made screen savers.
Half joking, half paranoid.
Will be interesting to see how strong the controlling forces can be - enough to make you miss things in direct perception like in the book, or only softer effects further up the cognition layer stack
And I have yet to see a single paper like this where a researcher bails out and publicly says they refuse to work on such projects. Not one.
The most benign interpretation of this observation is that science is filled with spineless opportunists who don’t care who they hurt with what they create. A slightly less benign interpretation might be that many of these people are doing this deliberately, and getting off on the sense of power it gives them.
When it is pushed from the top it is hard to stop at ground level.
Realistically, probably ads, but maybe not only that?
(AI start-up idea: one of our ads a day keeps dementia away! /s)
I’m sure there was an early hominid version of this discourse. “Maybe bad to make sharp rock and sharp stick if this what we do with it…” “Mmm yes someday we make sharp rock big enough smash world.”
But here we can start also the usual discussion about technology research for the sake of it vs calibration of possible side effects of new research
Personally i think we haven’t solve this problem and thus it’s just a matter of time until we’ll get in a non-going-back point
The fact it’s bucketing by making images of lighting and facial expressions, the fact it doesn’t natively do the video it does an image then video generates from it.
The results look really bad and samey. Doubt this would work for the actual thing they’re pitching it for.
The end result of this will be that creating viral or maximally converting content will not be restricted to the top content creators. It will also result in down regulation of those brain regions due to constant stimulation, so the bar will increase for what classifies as attention grabbing and the brain will filter most of it out as noise.
All in all, I don't really see this being a big difference to the outcomes we're already seeing.
if it is targetting visual regions of brain and I have aphantasia (I cannot visualize anything in my mind) is that connected?
It's so cool that you have an MRI handy to check such things.
reason I am asking it could be some relief to our brains after tedious working day, especially after heavy AI usage
You can of course, also not get the spike ads, by subscribing to Premium. Just have a quick look at this animated QR code, it will explain better than I can in writing.
Edit: there used to be an OSS app you could use where you actually pick what you want activated in your brain and when, but it's been banned after one of the commercial producers's investors convinced the government that this technology was too powerful, so only government-approved apps are allowed now, and an emergency vote passed the BrainControl law authorizing logging of all visual stimuli you look at, in order to protect your children.
I don't think it's a matter of if but when. Grim.
[0] https://corticallabs.com/cloud
This is a tool to help researchers in figuring out what different parts of the brain are actually for with less experimenter bias contamination of “well we think maybe it’s about this so let’s show it video of x to see”.
The essence runs on having someone sit in a scanner for a couple hours watching all sorts of things, and then feeding that to a model that will then build its own representation of said data and try different things on it until it’s found what makes a certain part sing in the model.
The purpose is a generalized understanding of brain function, more or less the same way we’ve been doing it all these years. Expose brain to something, record it somehow, see if brains reaction in the recording helps you understand more about who we are and what cognition is.
It also helps companies like Moonbug Entertainment (Candle Media) understand how to build better Distractatrons.
https://www.nytimes.com/2022/05/05/arts/television/cocomelon...https://www.newyorker.com/magazine/2024/06/17/cocomelon-chil...
What a world.
I can see it now.. The Distractatrons: a new chapter of protagonists in Transformers! The modern equivalent of evil in this day and age of ADHD and low attention span!
https://en.wikipedia.org/wiki/Looker
the same year as Baudrillard's Simulacra and Simulation, featuring computer generated characters, brainwashing television commercials, a light pulse gun that causes absence seizures, gun battles inside a plastic surgery clinic and an AR simulation environment, a sadistic computer, a physician who doubles as an action hero, and James Coburn giving a lecture explaining the enemy's evil plans in the opposite role that he played in the (excellent) The President's Analyst.
This time I was not so dazzled and saw it for as atrocious everyone else things it is. The minions of "Digital Matrix, Inc." manage several assassinations with the light-pulse L.O.O.K.E.R. gun but when they use real firearms they outdo Vader's stormtroopers by shooting each other. (Want to see the scene where somebody from E.Y. tells them to stick to the L.O.O.K.E.R. gun) The bad guys explain the penultimate secret to the protagonist early on but the ultimate secret is revealed in the L.O.O.K.E.R. lab which doesn't feel like a lab at all but rather a rather good room in a theme park experience where you're supposed to uncover the secret. (Contrast that to the lab Doug Trumbull outfitted in the Brainstorms movie a few years later which is packed with real surplus equipment... I've been to that lab!)
https://www.youtube.com/watch?v=cBlWxXqH8vA
> p8: "A child watching television under normal conditions is subject to frequent interruptions and distractions. The TV must vie for his attention. In order to simulate this condition, we decided to program distractions into the laboratory situation... Slides could be used to fill the slide tray and they could be projected automatically, at regular intervals, onto a screen similar to that of the television set. The carousel projector allows the viewer to choose three exposure times. The 7.45 second interval proved most satisfactory with the preschool children.
THE FIRST YEAR OF SESAME STREET: THE FORMATIVE RESEARCH https://files.eric.ed.gov/fulltext/ED047822.pdf
(STS seems to be https://en.wikipedia.org/wiki/Superior_temporal_sulcus in the temporal lobe, so I guess its the "I sense a presence" region.)
Explain the potential to exploit strong stimulation of specific visual regions for evil. "Oh, I very much detect a face/place/body/motion/pattern/human", says the subject. What are you going to do with that, startle them?
IMO it's terrifying to imagine that adtech companies could plug this stuff into their optimizers and just run an experiment on broad swaths of the population to figure out if there's any effects.
What exactly is your new fear here?
The largest LLMs right now are at best 1% the number of parameters of a human brain.
"At best" if synapses are one parameter each, real ones are probably more than that, but nobody's entirely sure yet.