I agree with the overall idea, but we don’t need to limit the AI to human collaborators.
I think the future lies in the Copilot approach of giving the Agent relevant context by building smarter prompt generation pipelines. The AI are really good when given enough context and they are bad at recalling information from their own training corpus. When you combine multiple AI agents together with good context and tools like in the LangChain approach these AI agents suddenly become really good at full automation.
Search is a tool that provides more context to the agent in order to answer a human task. By combining many agents together we can solve complex tasks.
Cooperation with people is what I thought of when I heard of the forthcoming Business Chat feature for Microsoft Teams <https://news.ycombinator.com/item?id=35182827>. From what I have read, it is going to produce summaries of conversations. That's not quite what I'd imagined, which is something like this:
----
A: ... and that is why I think we should go with option 1.
B: No, the points you mentioned support my case for option 2.
C: Nothing you guys have said changes my mind about option 3 being best.
D: Business Chat, what do you think?
BC: Based on this discussion, and my research, option 1 seems more realistic but option 2 would be more profitable if possible. My reasons are ...
C: Business Chat and you guys all don't understand point N, which is the main reason why option 3 is best.
B: Higher profit is exactly why I think option 2 is the way to go.
A: No, our rival is going to hit the market next month. We need to get something out there ASAP. Option 1 can do this.
D: You've all given me things to think about. Thank you for coming. Business Chat, email me a summary of the meeting, and set up a followup meeting for Tuesday 3pm.
----
That is, AI used as a colleague/assistant, not necessarily subordinate but not seen as omniscient, either; another viewpoint to consider. When will the above be feasible? This year? Five years?
In a way you could use these AI systems similar to kintsugi. Or in the same way a good wood sculptor uses the flow of the wood rather than working against it.
Using AI to provide a basic frame work on which folks can iterate, manipulate or redact is not such a dystopian view of these things.
Absolutely. If reading any of the computer technology pessimists of the last 50 years have taught me anything is that they are usually right to some degree.
I don’t understand what we have to gain from a system that generates code that needs human verification. It’s like Tesla’s autopilot all over again. Either it can drive itself or it can’t — if it can’t absolute, 100% reliably manage itself, why are we even pursuing it? I bet a more clever person than I could actually mathematically prove that neural network based models could never provide 100% reliable results.
I just watched an interview with Max Tegmark from 2 years ago where he talks about "Intelligble Intelligence" and that he feels that's the way forwards, it's an awesome concept. He said what he fears the most is huge black box AI, which shows human like intelligence by throwing more hardware at it, and we keep doing that until we maybe get an AGI.
But fundamentally some things you said in your comment aren't actually true.
Any of the "grasping context" stuff you're seeing is a combination of a few simple programming techniques to retain samples of what was previously typed by the user and similar techniques to give the illusion of continuity.
All of these different models being released ranging from stable diffusion to chatgpt are essentially just the classic Chinese room. There's nothing happening internally. There's no subjective experience of being ChatGPT. The model isn't updated as it interacts with you.
While the results to queries are interesting and sometimes helpful, it's still fundamentally a system just processing input and giving some kind of generated output.
These models aren't live. They're a snapshot that captures a point in time of the node weights when someone decided "good enough for now". A generous analogy might be for the patient with brain damage that can only remember things for 10 seconds. Think the movie 50 First Dates.
There's no actually learning about the world happening until the next update. Every new request you make to chatgpt is as brand new to the model as the last request.
Yes, apparently if you throw millions or billions of input parameters to a more or less standard gradient descent model you get some interesting and almost convincing results. But there's still the basic issues that these systems don't learn based on interactions.
This is mostly why I think the hype around these systems is hype right now. When an AI system starts asking questions back and the learning from them, that's the revolution.
In the meantime if probably less than 5 years, this is just where a lot of nervous investors are trying to dump money to cover the downturn.
All of that sounds true to me, and yet it doesn't seem to distract from the comment you're replying to in any way.
It can grasp context and be very useful for a lot of things even if it doesn't learn continuously, isn't alive, can be described as a Chinese Room, etc.
The problem is that human cognition itself could be described as a Chinese room. We don't understand what understanding is, or what thinking is, what consciousness is, etc.
So we can't adequately defend our own thought process as being any "better" than an AI except at a functional level, and AI's are quickly catching up to and even exceeding human functions, so the moat humans have drawn to defend themselves as exceptional beings is ever shrinking.
The idea that updating weights is particularly necessary is questionable. Memory augmented llms are computationally universal. https://arxiv.org/abs/2301.04589
and make no mistake, in context learning can be used in a meaningful way to resolve deficiencies. gpt-3 getting 98.5% accuracy (on even very large numbers) on addition arithmetic by describing the algorithm of addition to be performed on 2 numbers. https://arxiv.org/abs/2211.09066
as for asking questions, llms can do that just fine. anyone who's used bing will tell you as much.
You don't know what subjective experience cGPT may or may not be having. just like i don't actually know for you or you for me. you've simply chosen to assume such.
"You don't know what subjective experience cGPT may or may not be having. just like i don't actually know for you or you for me. you've simply chosen to assume such."
This point is underappreciated by most people.
We are close to the stage when conversations with AI's will be indistinguishable from conversations with humans, and then philosophical quibbles about whether such beings are intelligent or conscious will be irrelevant to most people, as the proof will be in the pudding.
We're already past that stage in some limited ways, with some people and some AI's, but these limits will likely be overcome soon, and only some luddite holdouts will refuse to believe they're dealing with "real" people/beings.
> There's no subjective experience of being ChatGPT.
Probably not, although of course you can’t prove that. But how do you think this is relevant to the uses that people are putting it to?
When people talk about grasping context, they’re talking mainly about how relevant the responses are to the prompt. In that respect, these models are a significant improvement on their predecessors, and that’s what makes them useful. We don’t need AI to be conscious to be useful, in fact it’s probably better if its not, since conscious AI opens up a whole can of worms which we may not be ready to deal with.
No, because current AI is most likely not conscious.
They can of worms I was alluding to is that it’s easy to justify using non-conscious machines purely as tools. But if they’re conscious, suddenly you have a new species of enslaved beings instead.
>But just how good are the new AIs at approximating human consciousness? Consider the bar that recently advertised a drink special with the following terms: “Buy one beer for the price of two and receive a second beer absolutely free!” To any human, this is obviously a joke. The classic “buy one, get one” special is reformulated to cancel itself out. It is an expression of cynicism that will be appreciated as comic honesty, all to boost sales. Would a chatbot pick up on any of this?
ChatGPT flips into bullshit mode and interprets it as a straightforward offer; GPT4 grasps the humor and identifies it as a marketing strategy, and proffers several reasons why such a strategy might be effective (icebreaker for customers, encourages social media engagement, promoting a "quirky" laid-back atmosphere). It's fun to watch AI skepticism get rebutted in real time. Frightening too. We really have no idea what's about to hit us, do we...
It used to be that classical software was useful for verification, but not so good at generating ideas.
With LLMs we have the opposite: machines are now good at idea generation and exploration, but their ideas need to be verified by a human (or classical logic-based software).
There are plenty of different ways this is happening.
1) already happened with Galactica - the model included python scripts as part of it's "working memory" as part of a <work> token, which could be used in a container
2) via copilot, and copilot 365, as someone stated elsewhwre
3) GPT-> Wolfram alpha as a plugin, which will be out very soon
Galactica is the only one that is already (technically) available
That raises an interesting point. On the one hand, novelty is in the eye of the beholder, if you will. What has been done before might still be new to many. On the other, randomness is built in and language has a remarkable capacity for uniqueness (see statistics about novel queries on Google). So it's possible and likely for LLMs to generate novel sentences. Whether those are valuable is a harder thing to think about.
Given that even an army of monkey typists can generate a Shakespeare, why wouldn't an army of llms? Whether we call that "generating" or "creating" might be only philosophical.
An army of monkeys can only generate Shakespeare if you have enough manpower to sit around monitoring the masses of output, which makes it effectively not possible in any meaningful sense.
Some combination of users interactions with the endless output might create a decent enough opportunity. For example, Spotify discover combined with auto generated music (of a certain level) maybe is a possibility for this in that genre?
LLMs have randomness, but they are not random. LLMs probably won't generate new words, for example, so they could not have written shakespeare based on contemporary writings.
Since the LLMs are probabalistic is it possible for the inference engines to highlight lower probability content. That might at least make them more "human" rather than being confident and inaccurate.
Since LLMs generate words one by one, you can only determine whether the next word has a high probability. Some ideas occurred to me earlier, like applying prompt engineering to evaluate if the model is confident about the contents to be generated.
Like "Please reply T if you are confident about the text that you are going to generate, and F if you are not".
But if this could be done by architecting the neural networks, they would perform much better.
we do understand that they generate words with probabilities one by one though. They are transformers, the AI bit takes the entire prompt as input and returns a range of probabilities for the next token (token == word, it doesnt understand or see letters). A "dumb" algorithm then selects from these probabilities randomly using the probabilities returned by the ai based on the "temprature" setting (low temprature favors high probability words & high temprature selects from a whole lot of words but leads into really weird territory fast)
It's also worth remembering that this is not just a markov chain as many people seem to think, it doesn't simply remember what words come next in its training set, because statistically if you take a random 10 consecutive words from a piece of text, chances are its never been written before. (also the trained model is much smaller than the size of the dataset, so to simply remember everything it would have to be the worlds best compression algorithm by orders of magnitude) Thats why we need an AI here, to learn the general rules of language so it can respond to chains of words it has never seen before.
The sense that we "dont understand how they work" is that we dont know what the "rules of language" that it has learned are.
"we do understand that they generate words with probabilities one by one though"
This is no more helpful in understanding AI's than is knowing that human brains operate according to the laws of physics is helpful in understanding the human mind.
Yes, there are important mysteries that need to be solved through research [1] but there are still some limits we can deduce from how they are built [2].
My guess is that a low-probability token corresponds as much with creativity (it's saying something in an unusual way) as truth, since it's at the word level. Words aren't normally true or false, sentences are. Do LLM's represent their confidence in what they're saying anywhere? I don't think anyone knows. Another mystery.
But another problem is that confidence is a character attribute, not a writer attribute. If you ask an LLM to imitate Richard Feynman it's going to write pretty confidently about physics, despite not knowing as much as him about physics.
This is the equivalent of giving your RPG character high intelligence on the character sheet. Doesn't make you smart!
To make an LLM express confidence consistent with its actual knowledge, it would need to have good self-knowledge and actually use it. So far this doesn't happen automatically. Instead, OpenAI uses reinforcement learning based on what the people at OpenAI think the LLM can do. So that's why it sometimes refuses to answer for some kinds of questions.
That has the most effect on the default character, the "helpful AI assistant". Any other characters you ask for will likely have poorer self-knowledge.
I've read that, mysteriously, more training does make bigger LLM's better calibrated, but the reinforcement learning makes it worse again.
One other thought I had, was did anyone fact-check the training corpus. It is probable it contained factual errors, which would mean that the trained model would propagate errors some how.
It follows the "yes, and" rule of improv. This is cooperative if you understand what it's doing. If you want to believe it has any fixed motivations other than that, you fool yourself.
I fully understand what it's doing: it exploits the human brain being wired to see patterns where there is none. It vomits up words in such patterns humans have a high tendency to trust them. Of course , there is zero actual information content in it. It's a zeroday on human cognition, a terrible, a terrible danger to society but there's no stopping it. https://futureoflife.org/open-letter/pause-giant-ai-experime...
Good ideas can sometimes be generated via play, without proper justification for them. There is chance involved, but if you test them and they hold up then they become meaningful.
They are good at telling people what they want to hear. There are lots of people like that too, but yeah they generally dont achieve much on their own.
Right. If you don’t dig deeper, it will give you only generic answers, and even fully incorporate the intention of your questions, strengthening your biases.
I was trying to get ChatGPT to generate a table for all the products I'm trying to put together for Stripe for this conference I'm helping out with, and with all the permutations there's 24 products at different prices.
I fed it the options for all the columns and it couldn't do it! So in the end I resorted to telling it to give me a JS function to generate a CSV, and it did it perfectly. I think I'll end up telling it to give me code that gets me the thing I want (when it comes to data), rather than get the formatted data directly. It sometimes makes weird mistakes, and it's less reproducible.
I'm amazed at how much nay-saying there is on HN about LLMs. This community has bought into so many fads, like crypto, yet when presented with what's clearly going to be the most disruptive technology of your life, you act like it's no big deal. Is this just nervous denial? Are you suddenly incapable of linear (not exponential) extrapolation? Imagine what ChatGPT, Midjourney, or whatever Apple and Google are cooking up will be capable of if they only improve linearly, at say 1% per year?
People ought to be preparing themselves for this as if it were a COVID-like black swan event, except much, much worse. And do not bank on Sam Altman and his fellow tech bro billionaires holding the reins of these job-annihilating machines delivering with their promise of UBI.
I get why you're being down voted but your overall point is valid imo. We didn't get any good answers about what happens if this thing keeps getting better.
"People will find new jobs/things to do" doesn't really cut it. It's not really the tech titans fault, the governments really need to start stepping in.
There are AI boosters making exaggerated claims in their enthusiasm and therefore people take the other side in reaction to them because they’re wrong. The skeptics then sometimes write blanket dismissals in opposition, and that’s wrong too, and you get a loop. That’s just how it goes on the Internet sometimes.
I totally get the motivation and it’s why I started blogging again. To get more interesting conversations, I think we need curiosity while keeping in mind how little we know and how bad we are at predicting the future.
I've just read these models are getting to the point they run out of training data. They pretty much scanned most of the internet already. I wonder if we will reach a disappointing situation in 3-4 years where we're at "peak A.I" abilities.
Well disappointing for some, for others it means they will get to keep having a job with income :)
The current problem isn’t a lack of data. Meaningful data can be augmented with GPTs, just iterate over sparsely represented subjects and enrich them with context-aware samples …generated with GPT.
We are actively leveraging GPT4s brainstorming-ability to generate datasets used for finetuning downstream. The fine tuned models downstream can then be used to augment new training data for more complex root language models, just formalize your quality assurance with common sense DSLs. We’ve reached an upward spiral with quite the exponential feel to it, where it will lead, nobody knows.
That’s only going to get you so far. Sparsely represented subjects have have an actual reality you’re trying to model. So while yes, you can generate synthetic data to interpolate between the points that have already been sampled, whether those interpolated points have anything to do with the underlying reality is a matter of chance. If you really try to drill down the existing tools into a specialized problem area, it becomes very clear that they lack a sufficiently informed model of the subject to be useful. That’s why I’ve found that ChatGPT is great at the beginning of a project and increasingly irrelevant as you approach the core technical challenges of the project.
Agree - we're taking a hybrid approach - meaning the gen AI is 'prebuilding' the automation and then the user reviews and possibly corrects it before fully executing - check it out here: bardeen.ai/ai
73 comments
[ 3.0 ms ] story [ 143 ms ] threadI think the future lies in the Copilot approach of giving the Agent relevant context by building smarter prompt generation pipelines. The AI are really good when given enough context and they are bad at recalling information from their own training corpus. When you combine multiple AI agents together with good context and tools like in the LangChain approach these AI agents suddenly become really good at full automation.
Which imo is the path to AGI.
Search is a tool that provides more context to the agent in order to answer a human task. By combining many agents together we can solve complex tasks.
----
A: ... and that is why I think we should go with option 1.
B: No, the points you mentioned support my case for option 2.
C: Nothing you guys have said changes my mind about option 3 being best.
D: Business Chat, what do you think?
BC: Based on this discussion, and my research, option 1 seems more realistic but option 2 would be more profitable if possible. My reasons are ...
C: Business Chat and you guys all don't understand point N, which is the main reason why option 3 is best.
B: Higher profit is exactly why I think option 2 is the way to go.
A: No, our rival is going to hit the market next month. We need to get something out there ASAP. Option 1 can do this.
D: You've all given me things to think about. Thank you for coming. Business Chat, email me a summary of the meeting, and set up a followup meeting for Tuesday 3pm.
----
That is, AI used as a colleague/assistant, not necessarily subordinate but not seen as omniscient, either; another viewpoint to consider. When will the above be feasible? This year? Five years?
Using AI to provide a basic frame work on which folks can iterate, manipulate or redact is not such a dystopian view of these things.
I just watched an interview with Max Tegmark from 2 years ago where he talks about "Intelligble Intelligence" and that he feels that's the way forwards, it's an awesome concept. He said what he fears the most is huge black box AI, which shows human like intelligence by throwing more hardware at it, and we keep doing that until we maybe get an AGI.
Well...here we are...
Any of the "grasping context" stuff you're seeing is a combination of a few simple programming techniques to retain samples of what was previously typed by the user and similar techniques to give the illusion of continuity.
All of these different models being released ranging from stable diffusion to chatgpt are essentially just the classic Chinese room. There's nothing happening internally. There's no subjective experience of being ChatGPT. The model isn't updated as it interacts with you.
While the results to queries are interesting and sometimes helpful, it's still fundamentally a system just processing input and giving some kind of generated output.
These models aren't live. They're a snapshot that captures a point in time of the node weights when someone decided "good enough for now". A generous analogy might be for the patient with brain damage that can only remember things for 10 seconds. Think the movie 50 First Dates.
There's no actually learning about the world happening until the next update. Every new request you make to chatgpt is as brand new to the model as the last request.
Yes, apparently if you throw millions or billions of input parameters to a more or less standard gradient descent model you get some interesting and almost convincing results. But there's still the basic issues that these systems don't learn based on interactions.
This is mostly why I think the hype around these systems is hype right now. When an AI system starts asking questions back and the learning from them, that's the revolution.
In the meantime if probably less than 5 years, this is just where a lot of nervous investors are trying to dump money to cover the downturn.
It can grasp context and be very useful for a lot of things even if it doesn't learn continuously, isn't alive, can be described as a Chinese Room, etc.
So we can't adequately defend our own thought process as being any "better" than an AI except at a functional level, and AI's are quickly catching up to and even exceeding human functions, so the moat humans have drawn to defend themselves as exceptional beings is ever shrinking.
and make no mistake, in context learning can be used in a meaningful way to resolve deficiencies. gpt-3 getting 98.5% accuracy (on even very large numbers) on addition arithmetic by describing the algorithm of addition to be performed on 2 numbers. https://arxiv.org/abs/2211.09066
In context learning is just overpowered. It's essentially implicit fine-tuning https://arxiv.org/abs/2212.10559v2
as for asking questions, llms can do that just fine. anyone who's used bing will tell you as much.
You don't know what subjective experience cGPT may or may not be having. just like i don't actually know for you or you for me. you've simply chosen to assume such.
This point is underappreciated by most people.
We are close to the stage when conversations with AI's will be indistinguishable from conversations with humans, and then philosophical quibbles about whether such beings are intelligent or conscious will be irrelevant to most people, as the proof will be in the pudding.
We're already past that stage in some limited ways, with some people and some AI's, but these limits will likely be overcome soon, and only some luddite holdouts will refuse to believe they're dealing with "real" people/beings.
Probably not, although of course you can’t prove that. But how do you think this is relevant to the uses that people are putting it to?
When people talk about grasping context, they’re talking mainly about how relevant the responses are to the prompt. In that respect, these models are a significant improvement on their predecessors, and that’s what makes them useful. We don’t need AI to be conscious to be useful, in fact it’s probably better if its not, since conscious AI opens up a whole can of worms which we may not be ready to deal with.
The can is at least partially open already, and the worms are crawling out.
They can of worms I was alluding to is that it’s easy to justify using non-conscious machines purely as tools. But if they’re conscious, suddenly you have a new species of enslaved beings instead.
https://www.project-syndicate.org/commentary/ai-chatbots-nai...
ChatGPT flips into bullshit mode and interprets it as a straightforward offer; GPT4 grasps the humor and identifies it as a marketing strategy, and proffers several reasons why such a strategy might be effective (icebreaker for customers, encourages social media engagement, promoting a "quirky" laid-back atmosphere). It's fun to watch AI skepticism get rebutted in real time. Frightening too. We really have no idea what's about to hit us, do we...
Seeing this happen to HN is... sad.
With LLMs we have the opposite: machines are now good at idea generation and exploration, but their ideas need to be verified by a human (or classical logic-based software).
Galactica is the only one that is already (technically) available
Like "Please reply T if you are confident about the text that you are going to generate, and F if you are not".
But if this could be done by architecting the neural networks, they would perform much better.
We don't fully understand how they work or what their limits are.
It's also worth remembering that this is not just a markov chain as many people seem to think, it doesn't simply remember what words come next in its training set, because statistically if you take a random 10 consecutive words from a piece of text, chances are its never been written before. (also the trained model is much smaller than the size of the dataset, so to simply remember everything it would have to be the worlds best compression algorithm by orders of magnitude) Thats why we need an AI here, to learn the general rules of language so it can respond to chains of words it has never seen before.
The sense that we "dont understand how they work" is that we dont know what the "rules of language" that it has learned are.
This is no more helpful in understanding AI's than is knowing that human brains operate according to the laws of physics is helpful in understanding the human mind.
[1] https://skybrian.substack.com/p/dont-settle-for-a-superficia...
[2] https://skybrian.substack.com/p/ai-chats-are-turn-based-game...
But another problem is that confidence is a character attribute, not a writer attribute. If you ask an LLM to imitate Richard Feynman it's going to write pretty confidently about physics, despite not knowing as much as him about physics.
This is the equivalent of giving your RPG character high intelligence on the character sheet. Doesn't make you smart!
To make an LLM express confidence consistent with its actual knowledge, it would need to have good self-knowledge and actually use it. So far this doesn't happen automatically. Instead, OpenAI uses reinforcement learning based on what the people at OpenAI think the LLM can do. So that's why it sometimes refuses to answer for some kinds of questions.
That has the most effect on the default character, the "helpful AI assistant". Any other characters you ask for will likely have poorer self-knowledge.
I've read that, mysteriously, more training does make bigger LLM's better calibrated, but the reinforcement learning makes it worse again.
fooling
FTFY
I fed it the options for all the columns and it couldn't do it! So in the end I resorted to telling it to give me a JS function to generate a CSV, and it did it perfectly. I think I'll end up telling it to give me code that gets me the thing I want (when it comes to data), rather than get the formatted data directly. It sometimes makes weird mistakes, and it's less reproducible.
People ought to be preparing themselves for this as if it were a COVID-like black swan event, except much, much worse. And do not bank on Sam Altman and his fellow tech bro billionaires holding the reins of these job-annihilating machines delivering with their promise of UBI.
"People will find new jobs/things to do" doesn't really cut it. It's not really the tech titans fault, the governments really need to start stepping in.
I totally get the motivation and it’s why I started blogging again. To get more interesting conversations, I think we need curiosity while keeping in mind how little we know and how bad we are at predicting the future.
We are actively leveraging GPT4s brainstorming-ability to generate datasets used for finetuning downstream. The fine tuned models downstream can then be used to augment new training data for more complex root language models, just formalize your quality assurance with common sense DSLs. We’ve reached an upward spiral with quite the exponential feel to it, where it will lead, nobody knows.