ChatGPT didn't end the knowledge economy, because it doesn't possess reliability, niche knowledge, and experience which is hard to put into words. Reliability, niche knowledge, and experience (pick two) are what get people get high-paying, stable, knowledge-based jobs, like legacy system engineer, sysadmin, and <thing> advisor.
Surface-level knowledge (which ChatGPT is good at) was already accessible by anyone who can do basic research, albeit slower. It wasn't a gated skill, so I can imagine some people achieved decent jobs from it, but most managers probably either did it themselves or assigned it to someone with a different job.
You are 100% correct about AI today, however, I feel people are way too fast at making a final assessment of what AI can and cannot do as if it was set in stone. People seem so sure that the problems AI are having now are just intrinsic and just won't be overcome.
But think about what computers were like when they first hit the market. What about the first year of public internet? Or the first generation of cellphones. These technologies also had major issues and shortcomings that would, if not improved, severely limit their usefulness.
Nobody knows what AI will be like in 5 to 10 years with the burst of investment and focus that this past year has ensured.
> But think about what computers were like when they first hit the market. What about the first year of public internet? Or the first generation of cellphones. These technologies also had major issues and shortcomings that would, if not improved, severely limit their usefulness.
The question is whether AI's limitations are more like too I/O limited, too slow, not enough storage, or more like fundamental things like GIGO and (virtually) all software having bugs and people having to handle that.
You aren't wrong, but we can't deny that internet changed the world, changed industries, and changed society. It's not all positive, but it's become a fundamental building block of mankind at this point.
> You are 100% correct about AI today, however, I feel people are way too fast at making a final assessment of what AI can and cannot do as if it was set in stone.
OpenAI is valued at $100b or something and given the hype I would have thought that it would have solved basic knowledge economy things like spreadsheet analysis, something that millions of well-compensated people are currently employed to do.
Why is there no "AI Excel" that can take a 1GB CSV and answer questions that are one step more complex than Pivot Tables?
As a business user, I don't care about AI that pretends to 'talk' like a human and 'draws' pictures. I need it to be able to do the work of a securities analyst.
You can't actually use startup investment numbers as a metric anymore because just like the stock market it's just become a avenue for financial gambling with little regard for the underlying industry.
I've thought a lot about exactly these limits to AI. The reliability problem will be solved sooner or later. Same for tacit skills that somebody trains into AI (manage people, do laundry). But if your job is specifically about being human ("nostalgic jobs", judge, athlete, surrogate mother...), or some niche tightly guarded ability (how to breed some special spice, how many oil barrels ship thru Suez Channel), you are safe - until some unforseeable rearrangement of the economy, at least.
> until some unforseeable rearrangement of the economy
I mean, telling the majority of the population that hey, you don't have secret knowledge, now you are mostly worthless is a completely foreseeable rearrangement of the economy, at least in the sense the rearrangement is coming. The question of how the new arrangement will look is unpredictable and likely spans anywhere from UBI to guillotines.
What counts as "niche knowledge"? Less than 5% of the workforce knowing it? 3%?
Software engineers collectively are 2.54% of the total USA workforce; the only time I've used YACC was at university, and given what it does I have to assume that (despite the large number of languages out there) YACC knowledge is relatively niche… and it's something ChatGPT "knows".
It might not be reliable enough (yay, I still get to be paid for my existing skills!), but I think it absolutely does have niche knowledge.
I'm talking more niche: topics where the amount of people in the world who know range from no more than 10,000 to under a dozen. Topics like, how to navigate a legacy system or bureaucracy (which may not be 100% public), dense academic subjects (especially recent ones), edge-cases in libraries and operating systems (ChatGPT is likely to miss anything unintuitive unless it's well-known). Maybe some topics with a wider audience are also protected from ChatGPT, but those definitely are.
You can (or at least could before ChatGPT) definitely get a job just by knowing yacc, aria, Kubernetes, etc. which are certainly niche subjects. But you get a very high-paying job knowing how to write highly-optimized code for a specific type of hardware, how to implement a cutting-edge algorithm in <insert language>, how to integrate a badly-documented library filled with edge cases, or how to satisfy legal/government requirements. These are jobs which rely primarily on knowledge, and a tiny bit of putting pieces of knowledge together.
If I could add to this, in recent hiring I have dealt with people using chatgpt in written communication and in interviews.
I interview remotely. The long pauses that get longer as the questions go from superficial to "you need to have done this for a while to know" are frustrating and obvious but also a great indicator if what is going on.
I always ask "what is happening during the long pauses". No one yet has said " I'm thinking". That's probably because I have always said going in "if you need time to think, or don't know, we can just talk about your thought process; noone knows everything and we all learn as we go". So a non-threatening out exists.
Only happened a few times.
Last time I had a senior dev on the call typing in the questions to chatgpt as we went and he was getting the same answers as the candidate with points in the same order. Some of these were wrong in very specific ways.
Back when I was marking assignments in university that was by far the most reliable way to catch cheating: two assignments with the same quirky error in the same spot. It didn’t even have to be a shared code/logic error; a misspelled comment or variable name was enough to mentally pop up a red flag and go back to find the other assignment with the same error in it.
On some of the assignments we gave out we would provide a set of, say, 10 initial test cases that the students could use to verify their work with the expectation that they would write more of their own. As a marking team we would also generate our own more detailed test suite; partly to make our job easier, partly to try to identify trends of places where students might not be understanding the course material. That ended up being another interesting way to identify plagiarism: if only 2 or 3 assignments failed a given unit test we would go and look to see if the implementations on those assignments was identical…
This is our experience as well and is an immediate rejection if the person is not honest about it. If they say "i need to check this quickly on a search engine or chatgpt" that's fine. But if they make it sound like it is their own answers, that's dishonesty and you don't want to work with people you can't trust when you are given the choice.
> We live in a knowledge economy. What you know—and your ability to bring it to bear in any given circumstance—is what creates economic value for you.
And for the first time the thought occurred to me that AI might in fact make actual artists even more in demand. To be sure it may well end up only being wealthy patrons but we may come to prize a thing demonstrably created by human as more "genuine" and therefore more valuable.
AI should be a force multiplier. That's why I don't see it as a replacement for human workers, except for those that do the very basics. But then they should be empowered to do more.
That’s what recorded media effectively did to live performance. While far more people consume records/streaming, concerts and plays still command a premium.
The author talks of "summarizing" but I think there's a deeper concept at play: compression.
Human thoughts are compressed into speech. Many people probably speak only a tiny fraction of their thoughts. Writing is compressed speach, most people write only a fraction of what they say (unless you're a professional content producer/author/developer/etc). Entire worlds of thought are regularly compressed into writing. Poetry is often further compressed. It's trying to fit vast complexity into as few words as possible.
Think how many words have been written trying to accurately decompress the full meanings in Hamlet, the Odyssey, or Beowulf.
Tweets are compressed writing. Memes are compressed tweets.
LLM can compress huge amounts of text into a few words. This is pretty remarkable, but it is only ever as good as the input. It's never creating, it's simply compressing concepts it is trained on. You can unlock parts of that compressed data with prompts.
If my friend and I both shared the same LLM, I could send them a few words to use as a prompt, knowing that it will "expand" into paragraphs or even chapters of meaning already pre-compressed inside the LLM.
I think this is possibly a new thing, imagine like HugeLol, Reddit, X.com, but instead of tweets and memes it's LLM prompts. We're able now more than ever to transmit complex concepts to each other with the smallest possible bits.
I've seen this a bit already on some of the LocalLlama online groups. They'll post prompts to each other to produce a "personality" they can interact with. I suspect this will be more and more used to compress and send data to each other.
A group of coworkers is commuting on the same train. To pass the time, they tell jokes. Over time, they told the same joke so many times they come up with a system to make things more efficient. One guy goes: "5!" Everybody roars with laughter. Another guy goes: "23!" Others are in tears. Third guy says: "15." No reaction. He repeats: "15!" Still nothing. He goes, "Come on guys 15 is a pretty funny joke!" "Well it is, you just don't know how to tell it!"
> but instead of tweets and memes it's LLM prompts. We're able now more than ever to transmit complex concepts to each other with the smallest possible bits.
This sounds like passing a URL-shortened link instead of the contents. That only works efficiently if both already know the contents. To (actually) communicate from one who knows to one who doesn't, sending a prompt is like a download link that the receiver will have to get and read.
Also communicating involves drawing attention to particular aspects, would the prompt also cave out the right amount/shape of what you had in mind? An example would be explaining a point to someone who's not quite getting it, referring them to a full treatise on the topic may not be of help. AI's could get interactive though with the system learning what the person understands or doesn't and presenting analogies based on what they do know.
>To (actually) communicate from one who knows to one who doesn't, sending a prompt is like a download link that the receiver will have to get and read.
And when they download it, it may turn it to be different than what you uploaded and sent a link to.
>Think how many words have been written trying to accurately decompress the full meanings in Hamlet, the Odyssey, or Beowulf.
There's simply no way that more than a tiny fraction of the various meanings imputed by later literary analyses were actually intended by the authors. This is not a desirable property for a compression scheme.
> once I made that connection, I started to see summarizing everywhere
One of the most powerful (and dangerous) aspects of dogma is the tendency of its followers to promote it to a universal pattern.
I, for one, am horrified at the prospect of a future where any kind of non-managerial labour is viewed as "summarising" and automated out of our collective skillset. GPT output may often be equivalent to human writing/thinking as a commodity, but human writing & thinking is not a commodity in its essence.
To me this is not the end of knowledge economy. This is a metastasis of the same capitalist disease that attacked the traditional crafts sector more than 100 years ago, attempting to replace it with a mix of industrially exploited labour in the Global North, colonial/slave labour in the Global South, and eventually mechanisation + automation. This brought about fantastic levels of productivity and wealth, along with insane amounts of pollution, the climate crisis and growing inequality. In sectors such as fashion the market is flooded with low-quality goods with a lifetime of a few months, which has led to astronomical amounts of waste.
The difference with AI is, now the Western creative middle class is affected, and due to the shadowy nature of the industry, it is not yet completely clear who is getting exploited (though we are starting to find out[1]). The good thing is, traditional crafts have not disappeared, in fact, their products are increasingly more prized and appreciated. I firmly believe generative AI's onslaught can also be withstood, and a better world is still possible - one where artisan labour, attention and connectedness prevail over whatever hellish future generative AI would create.
(side note: IMO high-quality code is much, much more than a StackOverflow summary)
Here here. All of this I think is slowly blowing up the Toffler-esque fantasy that many people in the "information worker" class tier had, that the "information age" was unique in creating a set of workers and industries immune to the ugliness and exploitation of industrial capitalism.
For periods of time we've been paid well to do craftsman-like work, without (on the whole) industrial automation and intense labour discipline.
Capitalism finds a way to route around that kind of blockage. And this is what Sundar and Musk and crew are up to now. And why there's such intense investment in "AI."
consumers most of the time dont care about craftmanship, even if the end product is far superior & lasitng than the cheaper shoddier alternative, and when they do its mostly for status. As european i remember how discounts had to remarket themselves hardly and show consumers they arent anymore the places where "poors" shop
“New technology is about to change the landscape completely.” Sure, it will. I’m not so sure that making an effort directed at being a “model manager” is the best way to prepare yourself for the possible changes.
In my opinion, education (for all ages) waste a lot of time chasing “digital literacy”, with few results to show for it. I think most people will have a greater return on investment from simply writing, practicing “math”, and organising their personal work and knowledge. These skills are surprisingly hard to get good at, and will probably keep you in demand as long as human labor is.
Edit: … however, this is an interesting thought-experiment. It just reads like career development advice.
Does no one notice that ChatGPT sucks at summarizing? Am I the only person in the whole fucking world who looks at LLMs without rose glasses? Does no one else test these things before declaring how great they are?
Good luck outsourcing anything interesting to ChatGPT, but I think all you are getting is mediocrity minus minus.
It's one thing to want flying cars, jet packs, hoverboards, etc. It's another thing to pretend we have them when we don't. Sober up, boys.
Advances in knowledge and technology will always come from people who understand the nitty gritty details very well and recognize patterns between abstract ideas.
I don't think current generation AIs are very good at this. A nearest neighbor search returns similar concepts but misses the very highest level abstractions. A NN search on a piece of text won't return text that talks about completely different ideas but uses similar sentence structure and argument patterns
“Ok, chatgpt summarise all emails sent in the company and determine who is duplicating work”
“Ok, summarise the minutes of every meeting”
“Hell stop there, record every meeting in the company, summarise the discussion and determine who is working on what project. Is there duplication”
“Ok now we record every meeting, and chatGPT can put people working on related projects inntouchbwithbeach other, tell me why we need a layer of management”
We are about to enter a world of zero secrets and zero privacy.
Everything we do is either tracked or inferred or easily guessable (hmm what does a forty something man think about each day)
And the issue is not how do I stop people finding out my stuff but since everyone will have access to my stuff how do we prevent that being exploited?
AI improves faster than economies adapt. A singularity?
Present LLMs are glorified search. As they approach real-time training, Google will go away, finally disrupted. (Google's 20% time was meant to avoid this very thing, but you need a whole organization behind an idea to disrupt, however small: e.g. IBM's PC developed in startup-like separate business unit.)
Isn't this the standard automation trajectory, of lower level work being done by machines, leaving humans to do the higher level work? Summarizing is a step before generalizing/inferring/what-if/creating. Even for knowledge-workers, wasn't the spreadsheet a similar automation?
Historically, automation creates more work than it replaces; but why should that continue? One reason is that automation is a commodity, so competition shifts to non-automated functions. Human demand continues.
Of course, this doesn't address the non-economic concerns of the Luddites, that the human investment in skills is lost, and the derived sense of human value and dignity. Unfortunately in the long-term: whatever your labour, AI is coming for you.
automation will increase worker productivity making smaller cohesive teams more efficient into producing results, entrepreneurship will be a lor more accessible, leading to a creation of specialized "skilled" jobs.
I think a few of the main points are good but part of the worldview here is pretentious and classist to an enraging degree.
Managers aren't in the position they are in because they have special skills that non-managers don't. They are often in those positions because of social class, i.e. their rich parents paid for a better college or were role models for management tracks or connections for landing executive roles etc.
Or they were promoted because they had very effective technical skills.
But putting aside all of that, the idea that AI seems like it should make most if not all of us into managers is something I have been thinking about and trying to accelerate for my life as much as possible.
The closest I have come to making this a practical reality so far is the 'aider' programming tool. I have also started on my own agent framework. It seems like the ability to put these things in a loop over a period of time with direct feedback such as executing scripts they are writing for you is where we are headed. We can already do that of course but I mean that the effectiveness will likely continue to increase as the models and agent systems are refined.
I think there is huge potential for more specialized models that can run locally and continuously without racking up OpenAI bills. The theory is that if the models don't need to know how to do literally everything, they can be smaller and still effective enough at narrow tasks.
To make this convenient we need an easy way to share more specific models and ideally a way to automatically discover and load them on the fly. So my goal is to build the WordPress of agent frameworks at least as far as the ability to very easily install plugins and agents.
Somewhere between the hype and the doom I think there’s a much simpler answer. We have always needed to use language to interface with computers. In early days, we spent more time learning to talk the language of a computer. As computers became more powerful, we made their language more like ours. This is just the next “higher level”.
In the near future, I think programming language will be natural language and LLMs will be a translator to lower level code. Why should we have an LLM program Python, when it could probably just write low level instructions only meant to be tested and verified but not read? Translation is what LLMs are good at, and summarization is fundamentally a translation task of verbose text into key information. Code is a translation of our ideas into machine language.
The reasoning aspects are not the strength of an LLM. Without detailed instructions to translate, they are not good at writing code.
First of all, natural language is actually not good at precisely and concisely specifying logic, computations, behavior, etc. That's why things like math and propositional logic were invented. You can get around that and attempt to specify things extremely precisely in natural language, but it ends up producing gigantic specifications and legalese and so on, which are often harder to read and certainly harder to write accurately then at least the ideal conception of a language designed for expressing computability and logic and behavior. It would end up being more work trying to hedge in the edge cases and ambiguities of natural language then it would be to just communicate in a language designed without them. So while, yes, perhaps more advanced computer programs will allow us to write in much more high level programming languages that resemble pseudocode more, there will probably never be a situation in which someone can just communicate using average natural language and get a program that accurately reflects what they wanted. Whether we go the legalese route or the high level pseudocode route, there will always have to be some sort of special language for doing this that will require a change in how we conceptualize things and in how we communicate them to the computer that has to be learned. It's the classic "detailed enough specification" problem: whenever people talk about how in the future product managers will be able to just directly tell the computer what they want and the computer will be able to produce it, the problem with that is that you need to be able to produce a specification precise enough to accurately communicate all of the computations and behaviors you want in a fair amount of detail, which is normally what programmers do when they take the specifications given them by managers and turn them into code — there's an interpretive and additive process there and if you move it up into the specification, someone's still going to need to do that.
Secondly, llms are not actually simple expanders like a compiler or a decompression algorithm, they are much more complicated and more difficult to predict and less deterministic, and they also have no actual conceptual understanding of anything or reasoning processes governed by self conscious principles, and therefore far less accurate and trustworthy in what they do, so asking them to produce vast amounts of code is a bad idea. You will still have to manually carefully check the logic and behavior of the code yourself, instead of just writing the logic and behavior yourself, and we all know that reading someone else's unfamiliar code is much more difficult than writing code to do something yourself, so it will probably be even more error prone. Having llms produce lower level code to skip out on the compiler just magnifies this checking problem, and also introduces other problems like the fact that you specified things at a certain level of abstraction comment and now it has to go down far more levels of abstraction, which means there are many more chances for its non-deterministic and unpredictable algorithm to mess things up.
Now, I get the sense that you specified that we would test the behavior of the output that llms would produce via essentially tested driven development in order to stave off this problem, but unfortunately I don't think that's the right way to go about things. If you go that route, then you have to remember to test all code paths and all possible scenarios and edge cases, and all the behaviors you want, which is difficult to do, whereas if you write or check the code itself yourself, then you can more easily tell, at least in general, a priori whether the logic is correct. You can figure out if something has been implemented correctly by just looking at the kernel from which behavior is generated inside of having to look at the branching wave front of generated behavior, which has a much larger surface area. This isn't always true, of co...
I’m a little suspicious about the authors characterization of using a wide variety LLM workflows as “management” - it misses things like training and supporting your supervises, and generally contributing to a healthy organizational ecosystem. I think a better analogy would be “specification writing”.
This is an interesting reflection, and I'm glad to have read it.
A few things came to mind:
The view of programming as "summarizing what's on StackOverflow" is really alien
to me. I suspect this is indicative of a particular approach to programming, and
perhaps to working in general, which I don't share. The author's view seems to
be that knowledge exists "out there" and the role of the "knowledge worker" is
to accumulate, internalize, and reshape it into products derived by
summarization. Compare this with another view on "knowledge work" taken from
https://en.wikipedia.org/wiki/Knowledge_worker
> Nonaka described knowledge as the fuel for innovation, but was concerned that
> many managers failed to understand how knowledge could be leveraged. Companies
> are more like living organisms than machines, he argued, and most viewed
> knowledge as a static input to the corporate machine. Nonaka advocated a view
> of knowledge as renewable and changing, and that knowledge workers were the
> agents for that change. Knowledge-creating companies, he believed, should be
> focused primarily on the task of innovation.
High value knowledge work involves creating and transforming knowledge, not just
compressing or reconfiguring it.
To expand on this in abstract terms: knowledge work is fundamentally cognitive
and it gets its higher order purpose and potential from the application of
reason; i.e., it is concerned with rational cognition. Rational cognition is
mainly about synthesizing new, higher order concepts that direct and evolve
given concepts into more general and potent structures. This work involves
re-cognition as a necessary component, but if it were only recognitive -- as
it would be if were only concerned with recollection and summarization -- it
would not have the creative dynamic which it does.
To expand in more specific terms: programming work involves problem solving, but
it is not mainly about reassembling existing solutions to solve known problems.
The most valuable aspects of this work come from problem discovery, root
cause analysis, and solution invention. (Programming work that consists in
StackOverflow copy pasta is probably best viewed as the production of tech debt
:) This is not to say resources like StackOverflow arent useful, they definitely
are!)
I suspect it says more about the author's career aspirations and the reigning
interests of the political-economic system that they envision a future where
everyone is a manager. First, only if you have "manager brain" can you look at
what's happening in tech and see a future where everyone is a manager as a
positive development, when compared with a future where everyone is a
researcher, artists, arisen, inventor, etc. Second, the managerialization of
work is actually describing an idealized view of the present situation, and if
it is looming in the future it is only as an intensification of the current
dynamics. The rise of [the Professional Managerial Class was heralded in the
70s][1], and most tech workers are in the PMC:
> Who are these Americans working in the upper echelons of the knowledge
> economy, exactly? ... the Professional Managerial Class. The PMC, as they are
> now often called, came into existence in the late nineteenth and early
> twentieth centuries. They were not the old petty bourgeoisie of small-business
> proprietors and independent farmers, but a new class whose expertise was
> required to make an industrial economy function: engineers, scientists,
> teachers, doctors, social workers, functionaries, bureaucrats, and other
> professionals and managers who had the know-how to create and control the
> levers of the modern capitalist world [0].
The managerialization of everything does seem very likely, because that is
basically what our curr...
There is a more reasonable interpretation to "summarizing what's on StackOverflow" than the pure face value
There are many "Solved problems" which you're disincentivized to try to solve personally (sayauth or encryption) since implementing a widely accepted solution tends to have aditional credibility and you're not likely to have any additional value to add
That's what I think of when I see that phrase, similar to my current work, I'm creating basically a glorified CRUD for a small business to try and reduce their manual paperwork, none of the programming is particularly technical, and whatever platonic ideal of the system I had for it to be smart has been positively removed by the people that will use the system, since they prefer it to be completely manual because they're supposed to make the decisions
The Business logic isn't particularly complicated, and anyone competent could have done the same by taking a random accepeted SO answer and adapting the code
I think the fundamental premise of this article is wrong, because I don't think the kind of thing chat GPT does is the same kind of thing that knowledge workers do. It isn't just a difference in reliability and talent in knowing what knowledge is relevant and understanding of niche knowledge, although those are also important factors that he doesn't weigh highly enough, it's also that there is a fundamental difference in kind between what's being done, making his comparison essentially a category error. Even if we assume that chat GPT doesn't hallucinate, it is an information retrieval system alone with some simple synthesis capabilities, whereas knowledge work is not just having the knowledge but having a full conceptual understaning of it and solving problems with creative application of thay knowledge and conceptual understanding — it's not just about being able to regurgitate a couple paragraphs of synthesized text or take a couple snippets from Stack Overflow and put them together, it's about actually understanding the meaniny and concepts of things and the principles behind them, and having a good understanding of the methods of reasoning and problem solving that you can self critically apply to your own thought processes in a self-correcting manner (which is something the structure of chatGPT precludes), and being able to creatively apply the knowledge you have, through the semantic lens of the things I just listed, in order to creatively solve a specific problem within a specific context.
It concerns me when people insist that chat GPT is doing the same thing humans are, or even the same thing knowledge workers are doing, because it indicates to me that a lot of people aren't aware of conceptual understanding and reasoning and critical thinking and problem solving and applying those things to your own thinking through metacognition in order to self-improve and self-correct, and are instead just acting by rote and regurgitating things essentially from memory with a little bit of synthesis, when they are capable of so much more and absolutely should be doing so much more. So perhaps the thing we've learned is that chat GPT has taken over the realm of some things only humans used to be able to do, but even though it fundamentally is not structured in a way to allow it to do the more advanced things humans do, so many people were just doing those basic things that it took over, that they can't tell the difference between a human and a large language model, and I think that's really kind of sad and upsetting.
"whereas knowledge work is not just having the knowledge but having a full conceptual understaning of it and solving problems [...] with creative application of thay knowledge and conceptual understanding"
I think the point the author is making is that models (services) like ChatGPT are doing the former in your statement and it's up to the human user to do the latter. Application and allocation here are synonyms in this context.
I use ChatGPT multiple times every day now, mostly to sift through knowledge and ask questions to make sure I understand that retrieved knowledge and also identify sources so that I can verify that knowledge if the truth isn't obvious.
I mean, you can say whatever you want, but I personally find it tremendously useful productivity enhancement tool right now - otherwise I wouldn't waste my time with it.
First, a quick note: good God the number of typos in my messages is becoming unacceptable lol.
Anyway, to your point: that's a fair enough interpretation of what he might be saying. So that's a fair point and maybe I shouldn't have bailed on the article so early.
On a related note, I do fundamentally question the real helpfulness of using chat GPT to find and summarize knowledge for you, as opposed to the perceived helpfulness. Every time I've attempted to use it, or for instance someone has attempted to use it to summarize essays I've written, it has turned out data that is subtly wrong in a multitude of ways, often ones that are not individually super obvious, but which subtly pervert the meaning or import of whatever knowledge was being summarized and can very quickly add up to getting the wrong impression of something. So you might be receiving data that you think is helpful and useful, but is actually far less helpful than you think it is. And yes, you can double check things yourself, but that depends on knowing what to double check, which means that you either double check absolutely everything — and then it's equivalent to doing the research yourself! — or you only double check the things that jump out at you, in which case you're probably missing a whole lot. And that may have turned out fine so far, but that seems like a ticking time bomb to me. Even with asking for sources, that's a dangerous game to play, because it isn't actually giving you the actual sources of data used to generate the answers, because it has no memory or knowledge of that: links to the sources used were probably not embedded in the sources themselves, and so are probably not associated with them in the training data. So it's really just generating what it thinks a plausible answer to the question of sources might be, with no necessary relation to what it actually used.
I often compare talking to chat GPT to talking to some kind of slick narcissist, who has only a passing knowledge of a huge range of subjects, gained through osmosis and half remembered details, but who isn't even really committed to giving you accurate knowledge, and is more committed to just saying whatever sounds right and is likely to make you happy. Or in this case, is likely to make the reward function go up.
Not that I think there couldn't be a computer system that does what chat GPT does in theory, I just don't think this approach is correct for generating a reliable knowledge summarizing machine.
Tech developers keep getting things wrong. Now we have self driving cars that won't obay authorities telling them to stop, one of the most foundational driving skills.
I do property maintenance and there is a lot of similar difficulty. It will take a lot of development before robots can do cleaning, dusting, and make beds well and efficiently. But the larger task is negotiating what exactly should be done in available time and how exactly, such as using low phosphate soap with soft cloth or bleach soap with tough sponge.
What I see happening is there is a huge amount of subtle know how being lost as people retire without training replacements, let alone AI replacements.
> You won’t be judged on how much you know, but instead on how well you can allocate and manage the resources to get work done.
Why would someone waste time judging me at all if such a mass levelling off of useful skill happens? It seems to suggest that in future there will only be ultra-managers and those yet to take the two-day manager course.
It's very wishful thinking to hope all former work achievements as markers for future success go by the wayside. I agree that no one really cares what constitutes "work", only "doneness". And that's why when someone wants a hole in the wall they are not going to judge you by how well you hold a drill in your hand for 5 seconds if better markers exist.
It's still the Knowledge Economy, knowledge that already propels an economy as much as it does because it can't be monopolized like minerals etc. I'd only expect to see more of this heaving force. Humans will always take credit for everything they can (even for not taking credit), so it will still be called "his knowledge" the moment it touches him. The stigma will disappear soon, just like it always does with tech.
77 comments
[ 3.9 ms ] story [ 154 ms ] threadSurface-level knowledge (which ChatGPT is good at) was already accessible by anyone who can do basic research, albeit slower. It wasn't a gated skill, so I can imagine some people achieved decent jobs from it, but most managers probably either did it themselves or assigned it to someone with a different job.
But think about what computers were like when they first hit the market. What about the first year of public internet? Or the first generation of cellphones. These technologies also had major issues and shortcomings that would, if not improved, severely limit their usefulness.
Nobody knows what AI will be like in 5 to 10 years with the burst of investment and focus that this past year has ensured.
The question is whether AI's limitations are more like too I/O limited, too slow, not enough storage, or more like fundamental things like GIGO and (virtually) all software having bugs and people having to handle that.
We don't know what and how LLM will impact the future.
A backup communication system in case the Soviets took out the phone system? What's non-positive about that?
OpenAI is valued at $100b or something and given the hype I would have thought that it would have solved basic knowledge economy things like spreadsheet analysis, something that millions of well-compensated people are currently employed to do.
Why is there no "AI Excel" that can take a 1GB CSV and answer questions that are one step more complex than Pivot Tables?
As a business user, I don't care about AI that pretends to 'talk' like a human and 'draws' pictures. I need it to be able to do the work of a securities analyst.
Not sure what file size limit is but chatgpt can do this.
I mean, telling the majority of the population that hey, you don't have secret knowledge, now you are mostly worthless is a completely foreseeable rearrangement of the economy, at least in the sense the rearrangement is coming. The question of how the new arrangement will look is unpredictable and likely spans anywhere from UBI to guillotines.
Software engineers collectively are 2.54% of the total USA workforce; the only time I've used YACC was at university, and given what it does I have to assume that (despite the large number of languages out there) YACC knowledge is relatively niche… and it's something ChatGPT "knows".
It might not be reliable enough (yay, I still get to be paid for my existing skills!), but I think it absolutely does have niche knowledge.
You can (or at least could before ChatGPT) definitely get a job just by knowing yacc, aria, Kubernetes, etc. which are certainly niche subjects. But you get a very high-paying job knowing how to write highly-optimized code for a specific type of hardware, how to implement a cutting-edge algorithm in <insert language>, how to integrate a badly-documented library filled with edge cases, or how to satisfy legal/government requirements. These are jobs which rely primarily on knowledge, and a tiny bit of putting pieces of knowledge together.
I interview remotely. The long pauses that get longer as the questions go from superficial to "you need to have done this for a while to know" are frustrating and obvious but also a great indicator if what is going on.
I always ask "what is happening during the long pauses". No one yet has said " I'm thinking". That's probably because I have always said going in "if you need time to think, or don't know, we can just talk about your thought process; noone knows everything and we all learn as we go". So a non-threatening out exists.
Only happened a few times.
Last time I had a senior dev on the call typing in the questions to chatgpt as we went and he was getting the same answers as the candidate with points in the same order. Some of these were wrong in very specific ways.
On some of the assignments we gave out we would provide a set of, say, 10 initial test cases that the students could use to verify their work with the expectation that they would write more of their own. As a marking team we would also generate our own more detailed test suite; partly to make our job easier, partly to try to identify trends of places where students might not be understanding the course material. That ended up being another interesting way to identify plagiarism: if only 2 or 3 assignments failed a given unit test we would go and look to see if the implementations on those assignments was identical…
And for the first time the thought occurred to me that AI might in fact make actual artists even more in demand. To be sure it may well end up only being wealthy patrons but we may come to prize a thing demonstrably created by human as more "genuine" and therefore more valuable.
Human thoughts are compressed into speech. Many people probably speak only a tiny fraction of their thoughts. Writing is compressed speach, most people write only a fraction of what they say (unless you're a professional content producer/author/developer/etc). Entire worlds of thought are regularly compressed into writing. Poetry is often further compressed. It's trying to fit vast complexity into as few words as possible.
Think how many words have been written trying to accurately decompress the full meanings in Hamlet, the Odyssey, or Beowulf.
Tweets are compressed writing. Memes are compressed tweets.
LLM can compress huge amounts of text into a few words. This is pretty remarkable, but it is only ever as good as the input. It's never creating, it's simply compressing concepts it is trained on. You can unlock parts of that compressed data with prompts.
If my friend and I both shared the same LLM, I could send them a few words to use as a prompt, knowing that it will "expand" into paragraphs or even chapters of meaning already pre-compressed inside the LLM.
I think this is possibly a new thing, imagine like HugeLol, Reddit, X.com, but instead of tweets and memes it's LLM prompts. We're able now more than ever to transmit complex concepts to each other with the smallest possible bits.
I've seen this a bit already on some of the LocalLlama online groups. They'll post prompts to each other to produce a "personality" they can interact with. I suspect this will be more and more used to compress and send data to each other.
A group of coworkers is commuting on the same train. To pass the time, they tell jokes. Over time, they told the same joke so many times they come up with a system to make things more efficient. One guy goes: "5!" Everybody roars with laughter. Another guy goes: "23!" Others are in tears. Third guy says: "15." No reaction. He repeats: "15!" Still nothing. He goes, "Come on guys 15 is a pretty funny joke!" "Well it is, you just don't know how to tell it!"
This sounds like passing a URL-shortened link instead of the contents. That only works efficiently if both already know the contents. To (actually) communicate from one who knows to one who doesn't, sending a prompt is like a download link that the receiver will have to get and read.
Also communicating involves drawing attention to particular aspects, would the prompt also cave out the right amount/shape of what you had in mind? An example would be explaining a point to someone who's not quite getting it, referring them to a full treatise on the topic may not be of help. AI's could get interactive though with the system learning what the person understands or doesn't and presenting analogies based on what they do know.
And when they download it, it may turn it to be different than what you uploaded and sent a link to.
There's simply no way that more than a tiny fraction of the various meanings imputed by later literary analyses were actually intended by the authors. This is not a desirable property for a compression scheme.
Keep your fountain pen. You’re still going to need it.
One of the most powerful (and dangerous) aspects of dogma is the tendency of its followers to promote it to a universal pattern.
I, for one, am horrified at the prospect of a future where any kind of non-managerial labour is viewed as "summarising" and automated out of our collective skillset. GPT output may often be equivalent to human writing/thinking as a commodity, but human writing & thinking is not a commodity in its essence.
To me this is not the end of knowledge economy. This is a metastasis of the same capitalist disease that attacked the traditional crafts sector more than 100 years ago, attempting to replace it with a mix of industrially exploited labour in the Global North, colonial/slave labour in the Global South, and eventually mechanisation + automation. This brought about fantastic levels of productivity and wealth, along with insane amounts of pollution, the climate crisis and growing inequality. In sectors such as fashion the market is flooded with low-quality goods with a lifetime of a few months, which has led to astronomical amounts of waste.
The difference with AI is, now the Western creative middle class is affected, and due to the shadowy nature of the industry, it is not yet completely clear who is getting exploited (though we are starting to find out[1]). The good thing is, traditional crafts have not disappeared, in fact, their products are increasingly more prized and appreciated. I firmly believe generative AI's onslaught can also be withstood, and a better world is still possible - one where artisan labour, attention and connectedness prevail over whatever hellish future generative AI would create.
(side note: IMO high-quality code is much, much more than a StackOverflow summary)
[1] https://time.com/6247678/openai-chatgpt-kenya-workers/
For periods of time we've been paid well to do craftsman-like work, without (on the whole) industrial automation and intense labour discipline.
Capitalism finds a way to route around that kind of blockage. And this is what Sundar and Musk and crew are up to now. And why there's such intense investment in "AI."
In my opinion, education (for all ages) waste a lot of time chasing “digital literacy”, with few results to show for it. I think most people will have a greater return on investment from simply writing, practicing “math”, and organising their personal work and knowledge. These skills are surprisingly hard to get good at, and will probably keep you in demand as long as human labor is.
Edit: … however, this is an interesting thought-experiment. It just reads like career development advice.
Good luck outsourcing anything interesting to ChatGPT, but I think all you are getting is mediocrity minus minus.
It's one thing to want flying cars, jet packs, hoverboards, etc. It's another thing to pretend we have them when we don't. Sober up, boys.
I don't think current generation AIs are very good at this. A nearest neighbor search returns similar concepts but misses the very highest level abstractions. A NN search on a piece of text won't return text that talks about completely different ideas but uses similar sentence structure and argument patterns
“Ok, chatgpt summarise all emails sent in the company and determine who is duplicating work”
“Ok, summarise the minutes of every meeting”
“Hell stop there, record every meeting in the company, summarise the discussion and determine who is working on what project. Is there duplication”
“Ok now we record every meeting, and chatGPT can put people working on related projects inntouchbwithbeach other, tell me why we need a layer of management”
Thus creating toxic positivity.
We are about to enter a world of zero secrets and zero privacy. Everything we do is either tracked or inferred or easily guessable (hmm what does a forty something man think about each day)
And the issue is not how do I stop people finding out my stuff but since everyone will have access to my stuff how do we prevent that being exploited?
Present LLMs are glorified search. As they approach real-time training, Google will go away, finally disrupted. (Google's 20% time was meant to avoid this very thing, but you need a whole organization behind an idea to disrupt, however small: e.g. IBM's PC developed in startup-like separate business unit.)
Historically, automation creates more work than it replaces; but why should that continue? One reason is that automation is a commodity, so competition shifts to non-automated functions. Human demand continues.
Of course, this doesn't address the non-economic concerns of the Luddites, that the human investment in skills is lost, and the derived sense of human value and dignity. Unfortunately in the long-term: whatever your labour, AI is coming for you.
Managers aren't in the position they are in because they have special skills that non-managers don't. They are often in those positions because of social class, i.e. their rich parents paid for a better college or were role models for management tracks or connections for landing executive roles etc.
Or they were promoted because they had very effective technical skills.
But putting aside all of that, the idea that AI seems like it should make most if not all of us into managers is something I have been thinking about and trying to accelerate for my life as much as possible.
The closest I have come to making this a practical reality so far is the 'aider' programming tool. I have also started on my own agent framework. It seems like the ability to put these things in a loop over a period of time with direct feedback such as executing scripts they are writing for you is where we are headed. We can already do that of course but I mean that the effectiveness will likely continue to increase as the models and agent systems are refined.
I think there is huge potential for more specialized models that can run locally and continuously without racking up OpenAI bills. The theory is that if the models don't need to know how to do literally everything, they can be smaller and still effective enough at narrow tasks.
To make this convenient we need an easy way to share more specific models and ideally a way to automatically discover and load them on the fly. So my goal is to build the WordPress of agent frameworks at least as far as the ability to very easily install plugins and agents.
In the near future, I think programming language will be natural language and LLMs will be a translator to lower level code. Why should we have an LLM program Python, when it could probably just write low level instructions only meant to be tested and verified but not read? Translation is what LLMs are good at, and summarization is fundamentally a translation task of verbose text into key information. Code is a translation of our ideas into machine language.
The reasoning aspects are not the strength of an LLM. Without detailed instructions to translate, they are not good at writing code.
First of all, natural language is actually not good at precisely and concisely specifying logic, computations, behavior, etc. That's why things like math and propositional logic were invented. You can get around that and attempt to specify things extremely precisely in natural language, but it ends up producing gigantic specifications and legalese and so on, which are often harder to read and certainly harder to write accurately then at least the ideal conception of a language designed for expressing computability and logic and behavior. It would end up being more work trying to hedge in the edge cases and ambiguities of natural language then it would be to just communicate in a language designed without them. So while, yes, perhaps more advanced computer programs will allow us to write in much more high level programming languages that resemble pseudocode more, there will probably never be a situation in which someone can just communicate using average natural language and get a program that accurately reflects what they wanted. Whether we go the legalese route or the high level pseudocode route, there will always have to be some sort of special language for doing this that will require a change in how we conceptualize things and in how we communicate them to the computer that has to be learned. It's the classic "detailed enough specification" problem: whenever people talk about how in the future product managers will be able to just directly tell the computer what they want and the computer will be able to produce it, the problem with that is that you need to be able to produce a specification precise enough to accurately communicate all of the computations and behaviors you want in a fair amount of detail, which is normally what programmers do when they take the specifications given them by managers and turn them into code — there's an interpretive and additive process there and if you move it up into the specification, someone's still going to need to do that.
Secondly, llms are not actually simple expanders like a compiler or a decompression algorithm, they are much more complicated and more difficult to predict and less deterministic, and they also have no actual conceptual understanding of anything or reasoning processes governed by self conscious principles, and therefore far less accurate and trustworthy in what they do, so asking them to produce vast amounts of code is a bad idea. You will still have to manually carefully check the logic and behavior of the code yourself, instead of just writing the logic and behavior yourself, and we all know that reading someone else's unfamiliar code is much more difficult than writing code to do something yourself, so it will probably be even more error prone. Having llms produce lower level code to skip out on the compiler just magnifies this checking problem, and also introduces other problems like the fact that you specified things at a certain level of abstraction comment and now it has to go down far more levels of abstraction, which means there are many more chances for its non-deterministic and unpredictable algorithm to mess things up.
Now, I get the sense that you specified that we would test the behavior of the output that llms would produce via essentially tested driven development in order to stave off this problem, but unfortunately I don't think that's the right way to go about things. If you go that route, then you have to remember to test all code paths and all possible scenarios and edge cases, and all the behaviors you want, which is difficult to do, whereas if you write or check the code itself yourself, then you can more easily tell, at least in general, a priori whether the logic is correct. You can figure out if something has been implemented correctly by just looking at the kernel from which behavior is generated inside of having to look at the branching wave front of generated behavior, which has a much larger surface area. This isn't always true, of co...
A few things came to mind:
The view of programming as "summarizing what's on StackOverflow" is really alien to me. I suspect this is indicative of a particular approach to programming, and perhaps to working in general, which I don't share. The author's view seems to be that knowledge exists "out there" and the role of the "knowledge worker" is to accumulate, internalize, and reshape it into products derived by summarization. Compare this with another view on "knowledge work" taken from https://en.wikipedia.org/wiki/Knowledge_worker
> Nonaka described knowledge as the fuel for innovation, but was concerned that > many managers failed to understand how knowledge could be leveraged. Companies > are more like living organisms than machines, he argued, and most viewed > knowledge as a static input to the corporate machine. Nonaka advocated a view > of knowledge as renewable and changing, and that knowledge workers were the > agents for that change. Knowledge-creating companies, he believed, should be > focused primarily on the task of innovation.
High value knowledge work involves creating and transforming knowledge, not just compressing or reconfiguring it.
To expand on this in abstract terms: knowledge work is fundamentally cognitive and it gets its higher order purpose and potential from the application of reason; i.e., it is concerned with rational cognition. Rational cognition is mainly about synthesizing new, higher order concepts that direct and evolve given concepts into more general and potent structures. This work involves re-cognition as a necessary component, but if it were only recognitive -- as it would be if were only concerned with recollection and summarization -- it would not have the creative dynamic which it does.
To expand in more specific terms: programming work involves problem solving, but it is not mainly about reassembling existing solutions to solve known problems. The most valuable aspects of this work come from problem discovery, root cause analysis, and solution invention. (Programming work that consists in StackOverflow copy pasta is probably best viewed as the production of tech debt :) This is not to say resources like StackOverflow arent useful, they definitely are!)
I suspect it says more about the author's career aspirations and the reigning interests of the political-economic system that they envision a future where everyone is a manager. First, only if you have "manager brain" can you look at what's happening in tech and see a future where everyone is a manager as a positive development, when compared with a future where everyone is a researcher, artists, arisen, inventor, etc. Second, the managerialization of work is actually describing an idealized view of the present situation, and if it is looming in the future it is only as an intensification of the current dynamics. The rise of [the Professional Managerial Class was heralded in the 70s][1], and most tech workers are in the PMC:
> Who are these Americans working in the upper echelons of the knowledge > economy, exactly? ... the Professional Managerial Class. The PMC, as they are > now often called, came into existence in the late nineteenth and early > twentieth centuries. They were not the old petty bourgeoisie of small-business > proprietors and independent farmers, but a new class whose expertise was > required to make an industrial economy function: engineers, scientists, > teachers, doctors, social workers, functionaries, bureaucrats, and other > professionals and managers who had the know-how to create and control the > levers of the modern capitalist world [0].
The managerialization of everything does seem very likely, because that is basically what our curr...
There are many "Solved problems" which you're disincentivized to try to solve personally (sayauth or encryption) since implementing a widely accepted solution tends to have aditional credibility and you're not likely to have any additional value to add
That's what I think of when I see that phrase, similar to my current work, I'm creating basically a glorified CRUD for a small business to try and reduce their manual paperwork, none of the programming is particularly technical, and whatever platonic ideal of the system I had for it to be smart has been positively removed by the people that will use the system, since they prefer it to be completely manual because they're supposed to make the decisions
The Business logic isn't particularly complicated, and anyone competent could have done the same by taking a random accepeted SO answer and adapting the code
I think the point the author is making is that models (services) like ChatGPT are doing the former in your statement and it's up to the human user to do the latter. Application and allocation here are synonyms in this context.
I use ChatGPT multiple times every day now, mostly to sift through knowledge and ask questions to make sure I understand that retrieved knowledge and also identify sources so that I can verify that knowledge if the truth isn't obvious.
I mean, you can say whatever you want, but I personally find it tremendously useful productivity enhancement tool right now - otherwise I wouldn't waste my time with it.
Anyway, to your point: that's a fair enough interpretation of what he might be saying. So that's a fair point and maybe I shouldn't have bailed on the article so early.
On a related note, I do fundamentally question the real helpfulness of using chat GPT to find and summarize knowledge for you, as opposed to the perceived helpfulness. Every time I've attempted to use it, or for instance someone has attempted to use it to summarize essays I've written, it has turned out data that is subtly wrong in a multitude of ways, often ones that are not individually super obvious, but which subtly pervert the meaning or import of whatever knowledge was being summarized and can very quickly add up to getting the wrong impression of something. So you might be receiving data that you think is helpful and useful, but is actually far less helpful than you think it is. And yes, you can double check things yourself, but that depends on knowing what to double check, which means that you either double check absolutely everything — and then it's equivalent to doing the research yourself! — or you only double check the things that jump out at you, in which case you're probably missing a whole lot. And that may have turned out fine so far, but that seems like a ticking time bomb to me. Even with asking for sources, that's a dangerous game to play, because it isn't actually giving you the actual sources of data used to generate the answers, because it has no memory or knowledge of that: links to the sources used were probably not embedded in the sources themselves, and so are probably not associated with them in the training data. So it's really just generating what it thinks a plausible answer to the question of sources might be, with no necessary relation to what it actually used.
I often compare talking to chat GPT to talking to some kind of slick narcissist, who has only a passing knowledge of a huge range of subjects, gained through osmosis and half remembered details, but who isn't even really committed to giving you accurate knowledge, and is more committed to just saying whatever sounds right and is likely to make you happy. Or in this case, is likely to make the reward function go up.
Not that I think there couldn't be a computer system that does what chat GPT does in theory, I just don't think this approach is correct for generating a reliable knowledge summarizing machine.
I do property maintenance and there is a lot of similar difficulty. It will take a lot of development before robots can do cleaning, dusting, and make beds well and efficiently. But the larger task is negotiating what exactly should be done in available time and how exactly, such as using low phosphate soap with soft cloth or bleach soap with tough sponge.
What I see happening is there is a huge amount of subtle know how being lost as people retire without training replacements, let alone AI replacements.
Why would someone waste time judging me at all if such a mass levelling off of useful skill happens? It seems to suggest that in future there will only be ultra-managers and those yet to take the two-day manager course.
It's very wishful thinking to hope all former work achievements as markers for future success go by the wayside. I agree that no one really cares what constitutes "work", only "doneness". And that's why when someone wants a hole in the wall they are not going to judge you by how well you hold a drill in your hand for 5 seconds if better markers exist.
It's still the Knowledge Economy, knowledge that already propels an economy as much as it does because it can't be monopolized like minerals etc. I'd only expect to see more of this heaving force. Humans will always take credit for everything they can (even for not taking credit), so it will still be called "his knowledge" the moment it touches him. The stigma will disappear soon, just like it always does with tech.