“That surveillance capitalism is a logic in action and not a technology is a vital point because surveillance capitalists want us to think that their practices are inevitable expressions of the technologies they employ. For example, in 2009 the public first became aware that Google maintains our search histories indefinitely: data that are available as raw-material supplies are also available to intelligence and law-enforcement agencies. When questioned about these practices, the corporation’s former CEO Eric Schmidt mused, ‘The reality is that search engines including Google do retain this information for some time.’
In truth, search engines do not retain, but surveillance capitalism does. Schmidt’s statement is a classic of misdirection that bewilders the public by conflating commercial imperatives and technological necessity.”
Shoshanna Zuboff, “The Age of Surveillance Capitalism”
> In papers filed with the court on November 20, the DOJ told the court that Windows 95 would function without Internet Explorer 3.0, and that Microsoft should be required to offer computer manufacturers (OEMs) the option of installing Windows, free of Internet Explorer 3.0 in
“all” respects. In response, Microsoft explained that the latest versions of Windows 95 would not work under such a scenario
I read "The level of social opprobrium directed at LLM use and at people attempting to pass off generated slop as work is immense" and immediately checked for the author's Bluesky link. This person is living in a bubble. Most people in tech are now using LLMs at work and there is no social opprobrium.
The rest of the analysis seems to take it granted that we have hit a wall and LLMs will improve no further. OK. We'll see how that prediction plays out.
> Most people in tech are now using LLMs at work and there is no social opprobrium discouraging them.
I don't know if "most people" is true, yet. I do think that the opprobrium has been silenced, and in corporate environments that has been systematic and top-down. Tokenmaxxing, leaderboards, monitoring: this is all part of making sure developers do not spend time considering the ethics, because the conclusions have been rendered moot.
We need to start talking about anti-AI psychosis. That people actually think their delusions fueled by these echo chambers are real is almost as worrying as the AI booster bubbles.
I'm gonna go ahead and make the prediction that this eventually morphs into a kind of religion, like flat earthers.
Nothing that AI will ever be valuable, everything AI does is steal, nothing it creates is new, everything that the big labs publishes is a scam or hype. Oh and "it's just doing X", where you can substitute "X" with some kind of surface level description of how the output layer in an LLM works.
These tenets are already pretty much formed, all it needs is to be formalized by some kind of name or manifesto similar to Q-anon, and we've arrived at a durable state of permanent denial.
If the majority of society is simply wrong as a matter of fact, should we not talk about it?
For instance, should we not talk about MMR vaccine skeptics or climate change denial in societies where they are prevalent? When the majority of a society believes something that is wrong, that is the time we should talk about it the most.
The first part of the article is a thoughtful framework. The conclusion that LLMs are not inevitable because they aren't useful or that their utility can be substituted by humans is an extraordinarily flimsy claim.
It helps if the author does not confuse what LLMs actually do with the general misunderstanding that "LLMs do work". They do not "do work" and have never "done work", and this misunderstanding is all over tech.
What do LLMs to? You know this: they are sentence completion engines, that through this process of sentence completion can be used to create software than then "does work". This indirection is critical, it is "how this works" and is the true power of LLMs that is STILL not being recognized. The power of LLMs is their communications capacity, their ability to complete sentences, which is so subtle the entire world of software engineers and artificial intelligence everybody has not identified that simple "variable expansion" as the true power of LLMs. LLMs do not do work, they enable the creation of software that then does work.
With a technology so subtle, want to know what is inevitable? A bifurcation of the population into those that believe in magic and those that understand it well enough to make magic nobody considered possible. Why did I say "magic"? Because people are giving up understanding, and that makes what we do magic.
One thing I've seen with LLMs that I've not seen with any other emergent technology is the level of backlash against it. I think we're seeing a point of polarization in society where there will be two truly opposing sides that are against each other and no reconciliation will be possible between them (pro and anti-AI), and that it has a good chance of fundamentally destabilizing society.
I haven't seen much splitting into two groups of people who are for and against. It's more that individuals can see upsides - does cool stuff, and downsides - something like a rival lifeform that will be smarter than us.
I keep seeing it and it's really bipolar. Boosters think it's the best thing ever and downplays all the issues while antis think it's completely useless and a disaster environmentally and morally.
Good luck saying LLMs are useful but still a net negative in the filter bubble this author is in. No, you also need to think it has zero utility or you're the enemy.
One datapoint on the same people seeing pro and anti is Nick Bostrom (Swedish philosopher & writer) who was one of the instigators of AI doom with his 2014 book:
I've seen a lot of people not wanting to learn to use a computer ("I prefer the old way") or not wanting to have a mobile phone ("I don't want to be reachable all the time") or a smartphone with touchscreen ("This feels like a too close interaction between the human and the device").
Well yeah, that's because it's political. The theat of job (and more importantly income) losses as well as twitter accelerationists, AI art slop spammers and anxiety trolls have really fueled the fire to the degree that people hate this stuff.
Not to mention how creepy Sam Altman and Dario Amodei are.
"The first point is the question of whether an LLM can do work not markedly inferior to that of a human doing the same task or not. This one is debatable. An LLM absolutely cannot produce work to the highest human standards, or even a good human standard at the moment, and it's unlikely that it ever will."
This could have been fine maybe two years ago, but saying that in the era when LLMs are solving Millenium Prize Problems is incredibly obnoxious. All those opinionated "AI articles" feel like ragebait at this point.
It's so tiresome. I'm going to stop getting into these threads altogether soon. There's nothing to discuss with the ones who treat it like a religious belief. I keep thinking that they'll just try some of the tools and see reason, but you can't rationalize with the irrational.
I wish people who felt this way would just go use the tools they hate so much before passing judgement. I think many of them used it once, in a much worse form, many years ago and not since. Judging these things by the standards of 3 months ago is unfair, let alone the standards of 3 years ago.
We now have professional mathematicians that are on the record carelessly dismissing major math questions being decided. And I'm not talking about having a subtle discussion about what proofs mean.
So it's not really about trying it out, I think. It's something more emotional.
Last I heard they had only data mined their own (human) users, submitted a pile of slop for (human) mathematicians to verify, and then withdrew much of it when it proved not to be a solution.
The author's absurd fantasy of how an LLM has to be used in order to fit their imagined adversary's idea of inevitable is absolute drivel. This article doesn't even build a straw man. It's just a pile of straw and manure.
The fact that the author is using PowerBI as the example makes me think they are completely oblivious of the actual impact and utility LLMs have in other fields such as programmering. Claiming it's useless and that anything it outputs is "markedly inferior to that of a human doing the same task" is an absurd basis to use for their argument.
It’s also just false. The good LLMs already produce output far better than the average programmer in most domains.
People like to talk about code as craft and art but the vast majority of the code that was written has always been mediocre at best. LLMs already beat it.
A focused experienced skilled programmer can do better but the market niche for that kind of code is actually small. Most commercial code just has to be good enough to accomplish a business need.
If you think of LLMs as statistically averaging machines and think of human programmer skill as a bell curve, it helps understand the different experiences with LLMs of different people. Those on the left hand side of the skill curve will see an uplift, those on the right hand side will experience degradation.
The more pertinent question in my mind is.. why does it still matter? Your customers don’t care about elegant code, they care about working products. There is some relation to good code and good customer experience e.g in terms of performance, but in general your customers won’t feel the impact from poor code. That assumes both pieces of code are functional though, and you do need to babysit LLMs to make sure it actually remains functional.
> If you think of LLMs as statistically averaging machines and think of human programmer skill as a bell curve ...
The problem with this view is that in the scale of LLM progress, this is essentially accurate understanding of archaic models. Modern models are not trained to program using human generated code. They are trained in agentic harnesses where they are able to access real computing environments, and they are rewarded for completing tasks within that environment. This is in contrast the being rewarded for the most likely next token, which used to create essentially an average of human output like you say. Modern training frameworks are able to comfortably produce models that perform well above the center of the bell curve.
On the last point: your customers absolutely feel the impact of your poor code. I’ve worked on very well written codebases and very poorly written ones. The well written ones have had far fewer outages and complaints. When you have multi million line codebases, being “functional” (I understood you to mean this as “working correctly” rather than the technical term) is a spectrum.
> The more pertinent question in my mind is.. why does it still matter? Your customers don’t care about elegant code, they care about working products.
No, they don't care about nice code, but they do care about code that actually works. In my experience, there is a correlation between code that has been thought out correctly and code that hasn't, in terms of security, safety, compliance, data-safety, performance, etc. And so far, all LLM-produced code I've seen is severely lacking.
> That assumes both pieces of code are functional though, and you do need to babysit LLMs to make sure it actually remains functional.
> A focused experienced skilled programmer can do better but the market niche for that kind of code is actually small.
At this point I think the only reason someone would do better without an LLM is an unwillingness to use it effectively. Even if you’re going to write all code by hand and only use an agent for validation and debugging, that’s a significant boost.
> It’s also just false. The good LLMs already produce output far better than the average programmer in most domains.
I hear this being repeated, but every time I've asked a LLM to come up with code without proper supervision, the result only superficially looked correct, but didn't stand up to scrutiny.
I do feel that a lot of pessimists I hear keep banging away about hallucinations and untrustworthy outputs. It makes me think they’re willfully ignorant of the progress since they tried ChatGPT 3 several years ago and were underwhelmed.
I’m happy to engage in discourse on the impact and risks to environment, society, jobs, mental health, etc. There’s plenty of room for thoughtful exploration there.
The one thing I have no patience for is that LLMs produce nothing of value or have no technical worth.
It's hard for me to take environmental risks regarding AI very seriously as well (and this is coming from someone who chooses to not drive or own a car, in large part for environmental reasons).
The recent efficiency gains in AI have been unparalleled in the history of computing as far as I know. We often point at how a single computer used to occupy an entire room, and over the course of decades of progress, are now wildly surpassed by computers that fit in our pocket. Inference of frontier language models that used to occupy entire data centers earlier this year (Claude Opus 4.6) are now comfortably surpassed in capability by models that can run on a single high end consumer grade gaming PC. I predict data center construction will collapse not just due to pushback from people who hate them, but also lack of demand from people who actually do use AI. People who like AI tend to like running their AI at home, and now they can.
The most important risks in AI right now are the social ones you mentioned. How does the economy evolve? How do we take advantage of this to maximize joy and love? Perhaps most importantly: how do we avoid (incentivize against) falling into a techno-dictatorship?
People who understand computers and those who do not.
I have to keep reminding myself though, many of those pessimists are people “getting paid to not understand” as the saying goes.
It took 30 seconds with that Davinci model to predict where we are today, where we’ll be next year, and the eventual destruction if not outright extinction of humanity as we know it. Maybe the future is so grim many are simply in the denial stage still.
Share examples outside of tech. The simple: we're winning / their losing without context sounds cheeky but doesn't help anyone understand your perspective.
So what has an "LLM" truly improved in your life outside of simplistic automations and/code generation/textual work? It's definitely not improved support. The only support interaction that I've noted to be somewhat better was one (of a few) with Amazon where the bot was able to refund a purchase without talking to a real person. But even then it was a case wherein a human would have probably done it just as quick. Is there a win there? Sure: for Amazon but it didn't "delight" me.
How would you say LLMs are impacting the markets? Do you think they're useful to the economy thus far? Or are you looking at the spend of a small fraction of companies that could cause a broader recession when the cyclical funding dries up? If you're going to go to open weight models - well those don't appear out of thin air.
Again, your statement seems absurd to me. I think most people can agree there are useful modalities with LLMs. But it's always at a cost and I, like the author of the article, don't think that LLMs are as universally useful as many "optimists" believe. But to say that the pessimists can't actually rationalize their thoughts, when you provide none of your own, is truly foot in mouth garbage.
I think that they will impact markets by reducing the amount of labor needed. I do not think that they will increase output or new products. Many companies believe that if they just had more manpower that they could generate new ideas in the market. But I don’t think that is true. This just leaves reduction in labor.
And just to be clear, I think LLMs will be a net negative on society.
I hate LLMs to the bone and am a master at "people not using the technology can change their behavior to neutralize the impact of the technology". Coupled with an 150+ IQ, I can tell you this technology won't go away. Cynically and pragmatically I use and will use it to the maximum extent of enslaving and eliminating the drone of drones who worship it. Go ahead, replace your mediocrity and lack of critical thinking with something that surely won't spit you out in a ditch a few years from now!
Every day you all just find your little articles and blogs on here to get mad at and then write the same exact response to it.
If you really are on the winning/right/good side of this debate, then just be in it, you don't need to sweat it all so much. We have all heard what you or anyone else are going to say on the matter, trust me.
Why care at this point? Why even give the conversation any air by going and responding with the same lines over and over? Is it really just too much to y'all still that some guy has the wrong opinion on the internet?
These people are weighing in with their opinions in the discussion. I think the pessimists weigh in because if they don't then people will confuse the optimists as being the majority opinion. It's important to have representation of the counter arguments so people don't go along their merry life thinking "everyone thinks this way". It sounds like you are "the pot calling the kettle black" right now, due to the fact that you're whining about the same thing, just on the inverse side. I understand your frustration, but they feel the same thing on the other side of the argument
I think it's a form of insecurity. They've forfeited their entire craft to a corporate black box and want reassurance that they're on the good and righteous path.
Its also not true though, especially recently. The remembering and constructing is especially superior to humans recently. So many technical issues in my personal life and professional lifr have been sorted by ai that has been problems that i havent been able to sort myself. The better you specify and constrain the problem the better the solution you get
Human intellectual effort is the tightrope walker in Thus Spoke Zarathustra; AI is the jester. LLMs can already solve programming and mathematical problems better and faster than humans. It's only a matter of time before other intellectual disciplines fall. We would be profoundly stupid not to embrace this change.
Empathy and big-picture thinking over myopic specialization will be the new moat.
This was published today, I had to check the date to be sure. I used to be an applied researcher working in big tech and for the things I wanted to build if I had to pick between LLMs and hand picking a team of 100 from the best programmers / researchers in the world I would pick the LLMs. Of course the combination would be even better. I've had LLMs immediately understand concepts that took me months to effectively communicate to a colleague and there are some concepts I was unable to communicate to any other human at all. Then the LLMs could build what I needed in less time it would take me to set up a meeting about it.
> Any actor that's at all serious in their field is, sooner or later, going to hit a point where LLMs cease being useful to them and actively hinder their further progress, and we absolutely do not see the massively overwhelming advantages that LLMs would need to have in order to become inevitable.
I can only conclude from this that I am not at all serious in my field. That's okay. I'm too busy enjoying myself.
> The technology can, in volume of adequate output or in quality of output, overwhelmingly outclass what alternative technologies can do. A moderate improvement isn't really enough to force inevitability: it has to be a massive difference.
> Now we come to the question of whether or not the difference that LLMs bring to the table is sufficiently overwhelming to make their adoption inevitable, and on this I think the answer is a clear no. As established above, LLMs just can't do top-flight work at all: at the heights of basically any discipline, we're still very much relying on unaugmented human work.
The clear flaw here is that cost isn't taken into account. A new technology needs to be better than the status quo at a given cost/price point.
Examples: mass produced textiles, photography, IKEA. Ostensibly not the highest quality, but the cheapest at a price point.
If you don’t use AI to scan for security bugs yourself, someone else will likely be using AI to find them first. So in that way, at least, LLMs could be considered inevitable.
What a waste of time. Arguing over the definition of “inevitable” like a child trying to get out of something. Almost nothing aside from perhaps heat death of the universe is literally inevitable. However, in the world we live in, AI is practically inevitable outside of an asteroid strike ending all life on earth. It’s almost as certain as the sun rising in the east.
67 comments
[ 0.23 ms ] story [ 24.6 ms ] threadShoshanna Zuboff, “The Age of Surveillance Capitalism”
https://news.microsoft.com/source/1997/12/23/dojs-request-fo...
The rest of the analysis seems to take it granted that we have hit a wall and LLMs will improve no further. OK. We'll see how that prediction plays out.
I don't know if "most people" is true, yet. I do think that the opprobrium has been silenced, and in corporate environments that has been systematic and top-down. Tokenmaxxing, leaderboards, monitoring: this is all part of making sure developers do not spend time considering the ethics, because the conclusions have been rendered moot.
That needs a citation, and context. Globally? The USA? The Valley? Where is this number from?
Nothing that AI will ever be valuable, everything AI does is steal, nothing it creates is new, everything that the big labs publishes is a scam or hype. Oh and "it's just doing X", where you can substitute "X" with some kind of surface level description of how the output layer in an LLM works.
These tenets are already pretty much formed, all it needs is to be formalized by some kind of name or manifesto similar to Q-anon, and we've arrived at a durable state of permanent denial.
Maybe an Orange Catholic Bible.
A majority of society already hates AI. I'm sure insulting them will really help your cause!
For instance, should we not talk about MMR vaccine skeptics or climate change denial in societies where they are prevalent? When the majority of a society believes something that is wrong, that is the time we should talk about it the most.
What do LLMs to? You know this: they are sentence completion engines, that through this process of sentence completion can be used to create software than then "does work". This indirection is critical, it is "how this works" and is the true power of LLMs that is STILL not being recognized. The power of LLMs is their communications capacity, their ability to complete sentences, which is so subtle the entire world of software engineers and artificial intelligence everybody has not identified that simple "variable expansion" as the true power of LLMs. LLMs do not do work, they enable the creation of software that then does work.
With a technology so subtle, want to know what is inevitable? A bifurcation of the population into those that believe in magic and those that understand it well enough to make magic nobody considered possible. Why did I say "magic"? Because people are giving up understanding, and that makes what we do magic.
I haven't seen much splitting into two groups of people who are for and against. It's more that individuals can see upsides - does cool stuff, and downsides - something like a rival lifeform that will be smarter than us.
Good luck saying LLMs are useful but still a net negative in the filter bubble this author is in. No, you also need to think it has zero utility or you're the enemy.
Superintelligence 2014 https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang...
has now brought out another book on the positives:
Deep Utopia 2024 https://ndpr.nd.edu/reviews/deep-utopia-life-and-meaning-in-...
I was just watching an interview with him:
Nick Bostrom Says We Are Clueless About What's Coming https://youtu.be/J9WBZ8BTLLg
Yudkowsky wrote about AI since the 1990s, his first realization of the AI danger was circa 2001, his "Sequences" were 2006-2009.
Bostrom also wrote much earlier about AI existential risk, see e.g. this paper of him from 2002:
https://www.jetpress.org/volume9/risks.html
Not to mention how creepy Sam Altman and Dario Amodei are.
Blockchain and other cryptocurrency related technologies?
This could have been fine maybe two years ago, but saying that in the era when LLMs are solving Millenium Prize Problems is incredibly obnoxious. All those opinionated "AI articles" feel like ragebait at this point.
I wish people who felt this way would just go use the tools they hate so much before passing judgement. I think many of them used it once, in a much worse form, many years ago and not since. Judging these things by the standards of 3 months ago is unfair, let alone the standards of 3 years ago.
So it's not really about trying it out, I think. It's something more emotional.
People like to talk about code as craft and art but the vast majority of the code that was written has always been mediocre at best. LLMs already beat it.
A focused experienced skilled programmer can do better but the market niche for that kind of code is actually small. Most commercial code just has to be good enough to accomplish a business need.
The more pertinent question in my mind is.. why does it still matter? Your customers don’t care about elegant code, they care about working products. There is some relation to good code and good customer experience e.g in terms of performance, but in general your customers won’t feel the impact from poor code. That assumes both pieces of code are functional though, and you do need to babysit LLMs to make sure it actually remains functional.
The problem with this view is that in the scale of LLM progress, this is essentially accurate understanding of archaic models. Modern models are not trained to program using human generated code. They are trained in agentic harnesses where they are able to access real computing environments, and they are rewarded for completing tasks within that environment. This is in contrast the being rewarded for the most likely next token, which used to create essentially an average of human output like you say. Modern training frameworks are able to comfortably produce models that perform well above the center of the bell curve.
No, they don't care about nice code, but they do care about code that actually works. In my experience, there is a correlation between code that has been thought out correctly and code that hasn't, in terms of security, safety, compliance, data-safety, performance, etc. And so far, all LLM-produced code I've seen is severely lacking.
> That assumes both pieces of code are functional though, and you do need to babysit LLMs to make sure it actually remains functional.
So, I guess we agree on that :)
At this point I think the only reason someone would do better without an LLM is an unwillingness to use it effectively. Even if you’re going to write all code by hand and only use an agent for validation and debugging, that’s a significant boost.
I hear this being repeated, but every time I've asked a LLM to come up with code without proper supervision, the result only superficially looked correct, but didn't stand up to scrutiny.
I wonder why our experiences differ.
The optimists have been more correct in predicting the capabilities and usefulness of LLMs.
The pessimists seem to have a hard time rationalizing what they thought with what’s actually happening.
I’m happy to engage in discourse on the impact and risks to environment, society, jobs, mental health, etc. There’s plenty of room for thoughtful exploration there.
The one thing I have no patience for is that LLMs produce nothing of value or have no technical worth.
The recent efficiency gains in AI have been unparalleled in the history of computing as far as I know. We often point at how a single computer used to occupy an entire room, and over the course of decades of progress, are now wildly surpassed by computers that fit in our pocket. Inference of frontier language models that used to occupy entire data centers earlier this year (Claude Opus 4.6) are now comfortably surpassed in capability by models that can run on a single high end consumer grade gaming PC. I predict data center construction will collapse not just due to pushback from people who hate them, but also lack of demand from people who actually do use AI. People who like AI tend to like running their AI at home, and now they can.
The most important risks in AI right now are the social ones you mentioned. How does the economy evolve? How do we take advantage of this to maximize joy and love? Perhaps most importantly: how do we avoid (incentivize against) falling into a techno-dictatorship?
I have to keep reminding myself though, many of those pessimists are people “getting paid to not understand” as the saying goes.
It took 30 seconds with that Davinci model to predict where we are today, where we’ll be next year, and the eventual destruction if not outright extinction of humanity as we know it. Maybe the future is so grim many are simply in the denial stage still.
So what has an "LLM" truly improved in your life outside of simplistic automations and/code generation/textual work? It's definitely not improved support. The only support interaction that I've noted to be somewhat better was one (of a few) with Amazon where the bot was able to refund a purchase without talking to a real person. But even then it was a case wherein a human would have probably done it just as quick. Is there a win there? Sure: for Amazon but it didn't "delight" me.
How would you say LLMs are impacting the markets? Do you think they're useful to the economy thus far? Or are you looking at the spend of a small fraction of companies that could cause a broader recession when the cyclical funding dries up? If you're going to go to open weight models - well those don't appear out of thin air.
Again, your statement seems absurd to me. I think most people can agree there are useful modalities with LLMs. But it's always at a cost and I, like the author of the article, don't think that LLMs are as universally useful as many "optimists" believe. But to say that the pessimists can't actually rationalize their thoughts, when you provide none of your own, is truly foot in mouth garbage.
And just to be clear, I think LLMs will be a net negative on society.
And then there are LLM realists ( Read realpolitik: https://en.wikipedia.org/wiki/Henry_Kissinger ).
I hate LLMs to the bone and am a master at "people not using the technology can change their behavior to neutralize the impact of the technology". Coupled with an 150+ IQ, I can tell you this technology won't go away. Cynically and pragmatically I use and will use it to the maximum extent of enslaving and eliminating the drone of drones who worship it. Go ahead, replace your mediocrity and lack of critical thinking with something that surely won't spit you out in a ditch a few years from now!
If you really are on the winning/right/good side of this debate, then just be in it, you don't need to sweat it all so much. We have all heard what you or anyone else are going to say on the matter, trust me.
Why care at this point? Why even give the conversation any air by going and responding with the same lines over and over? Is it really just too much to y'all still that some guy has the wrong opinion on the internet?
Empathy and big-picture thinking over myopic specialization will be the new moat.
I can only conclude from this that I am not at all serious in my field. That's okay. I'm too busy enjoying myself.
> Now we come to the question of whether or not the difference that LLMs bring to the table is sufficiently overwhelming to make their adoption inevitable, and on this I think the answer is a clear no. As established above, LLMs just can't do top-flight work at all: at the heights of basically any discipline, we're still very much relying on unaugmented human work.
The clear flaw here is that cost isn't taken into account. A new technology needs to be better than the status quo at a given cost/price point.
Examples: mass produced textiles, photography, IKEA. Ostensibly not the highest quality, but the cheapest at a price point.