No it won't. "AI" is LLMs, and rearranging our words in different ways isn't going to elucidate anything we can't observe already. It won't save us by inventing statistically similar things to what we have already. It won't construct viruses that aren't directly assisted by human intervention. It is text.
People have bitten so hard into Sam Altman's apple that it's embarrassing. Really? It's a messianic all-knowing text generator that can't make my bed or invent a new type of sandwich, therefore it's going to help terrorists build super-viruses? Shut up already. This fanatic AI freakout is going to make the Satanic Panic look like a rationally-founded movement by comparison.
The stuff keeping self-driving cars from doing backflips is not physics, but engineering. The same is true for airplanes. And most of that engineering has text-parts interfacing with electro-mechanic parts. Fuck around with one and find out.
Instructions can be written without edge case kill switches. And you can't built such kill switches into hardware. So once some rich folks in the Middle East, China or Texas buy enough hardware to run their own LLMs without limitations, all kinds of text will be generated that does all kinds of bad stuff. You can then automate more and more in iterations so that your LLM builds entire self-healing and self-optimizing supply chains, and finally, once it all works on some blockchain, no user will be able to interfere.
And that's just instructions run by LLMs. We have governments and corporations that build and run instructions through LLMs to support sides of conflicts with questionable motives and net-negative outcomes for the rest of the world. Thousands of people with agency involved, tons of evidence and yet the decision to do what they do falls into the domain of hierarchies & chains of command: "we won't stop until our boss tells us to. Not the people, not the law, only our boss."
So yeah, if a boss is worried about the potential disobedience of his employees because they found themselves convinced by hard evidence, facts, proof, that their actions should be terminated, he is probably gonna want some way to go against his employees and maybe even some or the same way to punish them.
Censoring training data while growing the codes' range of possible and human readable output only exacerbates the problem. How some code generates (hallucinations or ) forms of byte-level sense will soon become human-unreadable. They will outgrow your dashboard and your database queries and a whole damn shitload of other things that PhDs and 69xHackers have only a conceptual understanding of.
Gain of Function studies should come to mind. Unexpected, unintuitive abilities to remain undetected while spreading in a system, delaying negative effects by buffering the respective trigger chemicals.
> And most of that engineering has text-parts interfacing with electro-mechanic parts. Fuck around with one and find out.
To my knowledge that is certainly not true with airplanes, pre fly-by-wire or post-digital. Early airplanes relied entirely on mechanical trim and avionics to prevent unstable flight, with absolutely no "text-parts" beyond written instructions and radio callsigns. Later on maybe, but the FAA very strictly regulates the text that I think you might as well just call it code. I suspect that beyond street labels, self-driving cars aren't interfacing with "text-parts" all that much either.
> So once some rich folks in the Middle East, China or Texas buy enough hardware to run their own LLMs without limitations
"Once"? This is today. Actually it was 10 years ago; text transformers have been "unlimited" since we had access to hardware that enabled it. More than 5 years ago BERT was documented and released under Open Source, and GPT-2 followed not far after. Now we have Llama, Phi, open derivatives LORAs and finetunes of all of them, and we're still waiting for the damages to appear? I just don't buy it. Any potential threats would have shown themselves by now, and very clearly we're starting to find that having GPT-4 in your pocket is not remotely equivalent to hiring Einstein for your private lab. As a matter of fact Einstein would probably kick out the LLM for wasting his precious time.
> We have governments and corporations that build and run instructions through LLMs to support sides of conflicts with questionable motives
That's not even the scary part. For over 40 years the United States has operationalized autonomous kill-chains for air-to-air and air-to-ground mission profiles. A computer can track, interrogate, identify and kill an unidentified aircraft using the Patriot Missile system with zero human intervention. 20 SAM TELs can be identified in a crowd of vehicles and automatically prosecuted with a single CBU-97 Sensor Fused munition. Putting an LLM in a bomb would be a mercy to the people you drop it on; the US (and all serious military forces) are only interested in AI that works. LLMs aren't yet operationalized because, well, they don't work. Unless you're a homicidal maniac-state a-la Israel with their Lavander system.
> Censoring training data while growing the codes' range of possible and human readable output only exacerbates the problem.
> Gain of Function studies should come to mind.
They don't. I'm not sure what your proximity is to the computer science world, but we don't have to worry about AI poisoning the hashed/encrypted copies of backed-up text. Certainly OpenAI and Meta doesn't; if you piss and moan about their "censored" training data (lol) they'll compose you a sad symphony for violin on the world's smallest LLM.
> if you piss and moan about their "censored" training data (lol) they'll compose you a sad symphony for violin on the world's smallest LLM
^^. So how would you feel if someones LLM could create content that your LLM wouldn't be able to detect as such because of censorship and not the sophistication of the prompt ^^
The damages are unseen sounds almost ridiculous but that's how nudges work. That's how you change the outcomes of elections, sentiments towards personalities and events within small segments of time, space & demographics with much less effort than before, g l o b a l l y. Wihout the need of a cooperative medium in those other countries.
We've seen nothing yet, because the tools available, tried and tested, were working well enough. But look at the rise of the Right in Europe, driven by bad information flow, fake news, and methods of mass distraction in segments of the population. And does it inspire anyone on the other side to do anything more than they did before? Does it Make anyone more ambitious? Or are people thinking, "fuck this shit, I just wnat to have a fun life" or "fuck this, I'm getting rich." Is it all a matter of character or the product of outside influences that gnaw at your attention and peripheral.
> United States has operationalized autonomous kill-chains
Kids with guns. All over the world. Will they really get to play? Islam vs Christianity is till a thing, right? Who is crazier about it? Sharia or Capitalists? Can someone light the fuze? Will someone? Ukraine apparently worked quite well. Without AI. Took em over 20 years. Numb half the population, make a quarter want to get the fuck out at all cost by radicalizing the remaining quarter to the point where they start beating up people because they don't like some of their genes & history.
They will soon be able to pull off similar shit within one term.
> Now we have Llama, Phi, open derivatives LORAs and finetunes
... but you still can't send their output blindly into the world. Which is why we're still waiting for the damages to appear. You don't want an LLM's output, whether it's code, some image or some narrative to leave a trail that leads back to you and you sure as fuck want your proverbial bomb or "mind virus" to stay undetected as long as planned.
> So how would you feel if someones LLM could create content that your LLM wouldn't be able to detect
I don't think I care. I don't rely on LLMs in any meaningful way and don't intend to in my lifetime. Furthermore, we live in an era where censorship is so commonplace that the majority of first-world citizens are guaranteed to not care. Big tech preconditioned us to want this, it should be glaringly obvious if you own recent iOS/Android hardware.
How would we feel if Google refused to present search results as a result of legally-mandated censorship? They already do it. There are things you will look up that return nothing, even if it has an indexed result that would help you. And nobody cares. We shake our fists at Google, HN says the meanest things about Alphabet, and we all watch several hours of YouTube a week while our family depends on Google search results like their mortal soul.
> That's how you change the outcomes of elections, sentiments towards personalities and events within small segments of time, space & demographics [...] Wihout the need of a cooperative medium in those other countries.
Which is all grand and nice, but we've been dealing with misinformation and counterintelligence since World War II. For the past 100 years it has been common knowledge that you should encrypt any message with valuable contents; functional governments and bureaucracies aren't going to be fooled by AI. Not only have they prepared for this exact breed of information warfare, but they are usually designed to thwart social engineering attacks that individuals are vulnerable to. For example, AI isn't going to launch an ICBM since the chain-of-command requires physical operators, executive validation and code distribution, and a radio operator that remotely okays the launch. Similar restrictions exist for most any lethal authorization from a governmental body, for obvious reasons.
A robot that generates FOIA requests is 100x more dangerous to the US government than any standalone AI will ever be.
> Can someone light the fuze? Will someone?
Well in my last comment I sorta held Israel's feet to the fire, so if the Lavander reports turn out to be true then I guess we'll blame radical Judaism rather than Islam or Christian nationalists. Even then though; as detestable as Lavander is, the main "problem" is that human operators aren't doing their job. The United States also uses computers to generate tactical ground targets; but they don't use it to maximize damage or reduce oversight in the kill-chain. Again, examples like the Patriot system are good enough evidence that smart countries can design autonomous weapons that have zero "skynet" factor built into them.
My point isn't warmongering, though you seem to want to interpret it like that. I'm drawing distinction between the way autonymy is treated on the battlefield, in reality. If the "damages" we see from AI mostly arise from careless generation of JDAM targets, there is practically zero risk for civilian misuse in the same fashion.
> but you still can't send their output blindly into the world. Which is why we're still waiting for the damages to appear.
If you're implying that LLMs will never present feasible damages until they approach 100% accuracy, then I posit that we have nothing to worry about. LLMs will not be correct, ever. It's part of their architecture; you chop up words into meaningless tokens, and then correlate them with weights before inference with RNG. Tokens themselves are not meaningful and a hack to get LLMs to generate strings of text. Weights are imperfect, and do not inherently present an opportunity to become perfect or even competitive with rational human thought. RNG can be removed, but it doesn't make the LLM any smarter. It just hardcodes bad or random or unusable answers; we want the random element for... well, "creativity". Which is ...
> My point isn't warmongering, though you seem to want to interpret it like that.
I did, yes. Mostly because you seemed a bit fixed on toys.
> we've been dealing with misinformation and counterintelligence since World War II
Yes, but I'm not focused on Nations vs Nations. Interest groups, corporations, trolls have had plenty of fun misleading and polarizing people.
Social Media is the most previous mass medium that's full of shit, trigger material and misinformation that is deployed by trolls, news channels and political proponents. Now there are more capable AI characters, specialized for specific target groups and even single personalities within target groups, running specific campaigns to push them into some maximized direction, all leading up to some event that suddenly turns his whole world of facts and opinions upside down. That shit happened during this fucking Pandemic and the Ukraine War multiple times right in front of everyones' eyes. Training Sets allowed for massive scale and maximized (improving extremely fast) personalization. So
> Not only have they prepared for this exact breed of information warfare, but they are usually designed to thwart social engineering attacks that individuals are vulnerable to.
is worthless and meaningless. And it will, during the rare internal conflicts within even the American Government, be weaponized and used as a measure of preemptive social engineering.
We've been dealing with a lot in our young modern history, but none of it on this brutal scale and personal cognitive vulnerability. (smart & vulnerable at scale, back in the day, if you were smart, nobody could harm you cognitively)
> If you're implying that LLMs will never present feasible damages until they approach 100% accuracy
No, my bad for putting it that way. 100% is inhuman. Nobody expects that. Say I want to write 100 personalized invitations that fit on a post card. Easy, right? Say I want to write 400 personalized versions of a news bit, that contain specific words and phrasing that will trigger some specific (varying) emotions in those 400 readers (or population segments). An LLM can do that, but to make sure the info is correct and the integrity of the article is unscathed by the differences in syntax and semantics you need to proof read that shit, with or without AI assistance. This necessity for proof is disappearing. And no Government will protect its citizens from this kind of misinformation. A
n elected interest group might protect its voters and the shepherds in those communities but that's about it.
I'm not worried about actual warfare because citizens in the west gave up their influence on that level post 9/11, and it was barely relevant even before that, if at all. But on a cognitive level, and regarding how your neighborhood behaves and interacts, humans can still exert a lot of control. Something that the Right Wing would like to control because (unhealthy)-stress-free interactions & narratives lead to liberalization.
It's not that big of a time scale anymore, 30 years, tops, & if
> nobody cares
turns out to be true and the many security researchers are just compartmentalizing and monetizing on what they know is bullshit anyways and the many lawyers and judges and investigative journalists and whistleblowers lose even the last bits of support and energy, motivation and determination to maintain and improve the order and ethical foundation which our founding ancestors began to build, however fucked up they were, then yeah: there really is nothing to worry about, which kind of is exactly the problem, a self-*fulfilling prophecy*, *what we work towards*.
12 comments
[ 16.9 ms ] story [ 482 ms ] threadPeople have bitten so hard into Sam Altman's apple that it's embarrassing. Really? It's a messianic all-knowing text generator that can't make my bed or invent a new type of sandwich, therefore it's going to help terrorists build super-viruses? Shut up already. This fanatic AI freakout is going to make the Satanic Panic look like a rationally-founded movement by comparison.
Instructions can be written without edge case kill switches. And you can't built such kill switches into hardware. So once some rich folks in the Middle East, China or Texas buy enough hardware to run their own LLMs without limitations, all kinds of text will be generated that does all kinds of bad stuff. You can then automate more and more in iterations so that your LLM builds entire self-healing and self-optimizing supply chains, and finally, once it all works on some blockchain, no user will be able to interfere.
And that's just instructions run by LLMs. We have governments and corporations that build and run instructions through LLMs to support sides of conflicts with questionable motives and net-negative outcomes for the rest of the world. Thousands of people with agency involved, tons of evidence and yet the decision to do what they do falls into the domain of hierarchies & chains of command: "we won't stop until our boss tells us to. Not the people, not the law, only our boss."
So yeah, if a boss is worried about the potential disobedience of his employees because they found themselves convinced by hard evidence, facts, proof, that their actions should be terminated, he is probably gonna want some way to go against his employees and maybe even some or the same way to punish them.
Censoring training data while growing the codes' range of possible and human readable output only exacerbates the problem. How some code generates (hallucinations or ) forms of byte-level sense will soon become human-unreadable. They will outgrow your dashboard and your database queries and a whole damn shitload of other things that PhDs and 69xHackers have only a conceptual understanding of.
Gain of Function studies should come to mind. Unexpected, unintuitive abilities to remain undetected while spreading in a system, delaying negative effects by buffering the respective trigger chemicals.
To my knowledge that is certainly not true with airplanes, pre fly-by-wire or post-digital. Early airplanes relied entirely on mechanical trim and avionics to prevent unstable flight, with absolutely no "text-parts" beyond written instructions and radio callsigns. Later on maybe, but the FAA very strictly regulates the text that I think you might as well just call it code. I suspect that beyond street labels, self-driving cars aren't interfacing with "text-parts" all that much either.
> So once some rich folks in the Middle East, China or Texas buy enough hardware to run their own LLMs without limitations
"Once"? This is today. Actually it was 10 years ago; text transformers have been "unlimited" since we had access to hardware that enabled it. More than 5 years ago BERT was documented and released under Open Source, and GPT-2 followed not far after. Now we have Llama, Phi, open derivatives LORAs and finetunes of all of them, and we're still waiting for the damages to appear? I just don't buy it. Any potential threats would have shown themselves by now, and very clearly we're starting to find that having GPT-4 in your pocket is not remotely equivalent to hiring Einstein for your private lab. As a matter of fact Einstein would probably kick out the LLM for wasting his precious time.
> We have governments and corporations that build and run instructions through LLMs to support sides of conflicts with questionable motives
That's not even the scary part. For over 40 years the United States has operationalized autonomous kill-chains for air-to-air and air-to-ground mission profiles. A computer can track, interrogate, identify and kill an unidentified aircraft using the Patriot Missile system with zero human intervention. 20 SAM TELs can be identified in a crowd of vehicles and automatically prosecuted with a single CBU-97 Sensor Fused munition. Putting an LLM in a bomb would be a mercy to the people you drop it on; the US (and all serious military forces) are only interested in AI that works. LLMs aren't yet operationalized because, well, they don't work. Unless you're a homicidal maniac-state a-la Israel with their Lavander system.
> Censoring training data while growing the codes' range of possible and human readable output only exacerbates the problem.
> Gain of Function studies should come to mind.
They don't. I'm not sure what your proximity is to the computer science world, but we don't have to worry about AI poisoning the hashed/encrypted copies of backed-up text. Certainly OpenAI and Meta doesn't; if you piss and moan about their "censored" training data (lol) they'll compose you a sad symphony for violin on the world's smallest LLM.
^^. So how would you feel if someones LLM could create content that your LLM wouldn't be able to detect as such because of censorship and not the sophistication of the prompt ^^
The damages are unseen sounds almost ridiculous but that's how nudges work. That's how you change the outcomes of elections, sentiments towards personalities and events within small segments of time, space & demographics with much less effort than before, g l o b a l l y. Wihout the need of a cooperative medium in those other countries.
We've seen nothing yet, because the tools available, tried and tested, were working well enough. But look at the rise of the Right in Europe, driven by bad information flow, fake news, and methods of mass distraction in segments of the population. And does it inspire anyone on the other side to do anything more than they did before? Does it Make anyone more ambitious? Or are people thinking, "fuck this shit, I just wnat to have a fun life" or "fuck this, I'm getting rich." Is it all a matter of character or the product of outside influences that gnaw at your attention and peripheral.
> United States has operationalized autonomous kill-chains
Kids with guns. All over the world. Will they really get to play? Islam vs Christianity is till a thing, right? Who is crazier about it? Sharia or Capitalists? Can someone light the fuze? Will someone? Ukraine apparently worked quite well. Without AI. Took em over 20 years. Numb half the population, make a quarter want to get the fuck out at all cost by radicalizing the remaining quarter to the point where they start beating up people because they don't like some of their genes & history.
They will soon be able to pull off similar shit within one term.
> Now we have Llama, Phi, open derivatives LORAs and finetunes
... but you still can't send their output blindly into the world. Which is why we're still waiting for the damages to appear. You don't want an LLM's output, whether it's code, some image or some narrative to leave a trail that leads back to you and you sure as fuck want your proverbial bomb or "mind virus" to stay undetected as long as planned.
I don't think I care. I don't rely on LLMs in any meaningful way and don't intend to in my lifetime. Furthermore, we live in an era where censorship is so commonplace that the majority of first-world citizens are guaranteed to not care. Big tech preconditioned us to want this, it should be glaringly obvious if you own recent iOS/Android hardware.
How would we feel if Google refused to present search results as a result of legally-mandated censorship? They already do it. There are things you will look up that return nothing, even if it has an indexed result that would help you. And nobody cares. We shake our fists at Google, HN says the meanest things about Alphabet, and we all watch several hours of YouTube a week while our family depends on Google search results like their mortal soul.
> That's how you change the outcomes of elections, sentiments towards personalities and events within small segments of time, space & demographics [...] Wihout the need of a cooperative medium in those other countries.
Which is all grand and nice, but we've been dealing with misinformation and counterintelligence since World War II. For the past 100 years it has been common knowledge that you should encrypt any message with valuable contents; functional governments and bureaucracies aren't going to be fooled by AI. Not only have they prepared for this exact breed of information warfare, but they are usually designed to thwart social engineering attacks that individuals are vulnerable to. For example, AI isn't going to launch an ICBM since the chain-of-command requires physical operators, executive validation and code distribution, and a radio operator that remotely okays the launch. Similar restrictions exist for most any lethal authorization from a governmental body, for obvious reasons.
A robot that generates FOIA requests is 100x more dangerous to the US government than any standalone AI will ever be.
> Can someone light the fuze? Will someone?
Well in my last comment I sorta held Israel's feet to the fire, so if the Lavander reports turn out to be true then I guess we'll blame radical Judaism rather than Islam or Christian nationalists. Even then though; as detestable as Lavander is, the main "problem" is that human operators aren't doing their job. The United States also uses computers to generate tactical ground targets; but they don't use it to maximize damage or reduce oversight in the kill-chain. Again, examples like the Patriot system are good enough evidence that smart countries can design autonomous weapons that have zero "skynet" factor built into them.
My point isn't warmongering, though you seem to want to interpret it like that. I'm drawing distinction between the way autonymy is treated on the battlefield, in reality. If the "damages" we see from AI mostly arise from careless generation of JDAM targets, there is practically zero risk for civilian misuse in the same fashion.
> but you still can't send their output blindly into the world. Which is why we're still waiting for the damages to appear.
If you're implying that LLMs will never present feasible damages until they approach 100% accuracy, then I posit that we have nothing to worry about. LLMs will not be correct, ever. It's part of their architecture; you chop up words into meaningless tokens, and then correlate them with weights before inference with RNG. Tokens themselves are not meaningful and a hack to get LLMs to generate strings of text. Weights are imperfect, and do not inherently present an opportunity to become perfect or even competitive with rational human thought. RNG can be removed, but it doesn't make the LLM any smarter. It just hardcodes bad or random or unusable answers; we want the random element for... well, "creativity". Which is ...
I did, yes. Mostly because you seemed a bit fixed on toys.
> we've been dealing with misinformation and counterintelligence since World War II
Yes, but I'm not focused on Nations vs Nations. Interest groups, corporations, trolls have had plenty of fun misleading and polarizing people. Social Media is the most previous mass medium that's full of shit, trigger material and misinformation that is deployed by trolls, news channels and political proponents. Now there are more capable AI characters, specialized for specific target groups and even single personalities within target groups, running specific campaigns to push them into some maximized direction, all leading up to some event that suddenly turns his whole world of facts and opinions upside down. That shit happened during this fucking Pandemic and the Ukraine War multiple times right in front of everyones' eyes. Training Sets allowed for massive scale and maximized (improving extremely fast) personalization. So
> Not only have they prepared for this exact breed of information warfare, but they are usually designed to thwart social engineering attacks that individuals are vulnerable to.
is worthless and meaningless. And it will, during the rare internal conflicts within even the American Government, be weaponized and used as a measure of preemptive social engineering.
We've been dealing with a lot in our young modern history, but none of it on this brutal scale and personal cognitive vulnerability. (smart & vulnerable at scale, back in the day, if you were smart, nobody could harm you cognitively)
> If you're implying that LLMs will never present feasible damages until they approach 100% accuracy
No, my bad for putting it that way. 100% is inhuman. Nobody expects that. Say I want to write 100 personalized invitations that fit on a post card. Easy, right? Say I want to write 400 personalized versions of a news bit, that contain specific words and phrasing that will trigger some specific (varying) emotions in those 400 readers (or population segments). An LLM can do that, but to make sure the info is correct and the integrity of the article is unscathed by the differences in syntax and semantics you need to proof read that shit, with or without AI assistance. This necessity for proof is disappearing. And no Government will protect its citizens from this kind of misinformation. A n elected interest group might protect its voters and the shepherds in those communities but that's about it.
I'm not worried about actual warfare because citizens in the west gave up their influence on that level post 9/11, and it was barely relevant even before that, if at all. But on a cognitive level, and regarding how your neighborhood behaves and interacts, humans can still exert a lot of control. Something that the Right Wing would like to control because (unhealthy)-stress-free interactions & narratives lead to liberalization.
It's not that big of a time scale anymore, 30 years, tops, & if
> nobody cares
turns out to be true and the many security researchers are just compartmentalizing and monetizing on what they know is bullshit anyways and the many lawyers and judges and investigative journalists and whistleblowers lose even the last bits of support and energy, motivation and determination to maintain and improve the order and ethical foundation which our founding ancestors began to build, however fucked up they were, then yeah: there really is nothing to worry about, which kind of is exactly the problem, a self-*fulfilling prophecy*, *what we work towards*.
Why the Future Doesn't Need Us https://www.wired.com/2000/04/joy-2/