I feel like people with smart AI's would have an advantage in making smart decisions. Probably at this point they discuss business strategy with some version of it.
I was drawn to the Wolfram logo blurb as well. It is funny because within days of ChatGPT making waves you had Stephen Wolfram writing 20,000-word blog posts about how LLM's could benefit from a Wolfram-Language/Wolfram Alpha API call to augment their capabilities.
On one hand I'm sure he will love to see people use their paid Wolfram Language server endpoints coupled to OpenAI's latest juggernaut. On the other, I'm sure he's wondering about what things would have looked like if his company would have been focused on this wave of AI from the start...
This too is one of the most interesting integration to me. Allows for getting logical deduction from an external source (e.g. wolfram alpha), which can be interacted with via the natural language interface. (e.g. https://content.wolfram.com/uploads/sites/43/2023/03/sw03242...)
For those interested the original Stephen Wolfram post:
Stephen Wolfram himself thought that Wolfram could be combined with GPT when ChatGPT was released like 4 months ago. Only due to that they worked together to build the plugin. He also authored the best article I've seen on how ChatGPT and more broadly LLMs work (that has now been turned into a book).
Why cant Wolfram train a rudimentary chat model in their own search box. it doesn't even need to be very knowledgeable, just know how to map questions to mathematica
That's a game-changer! It seems like factuality issues with ChatGPT might be fixed. We wrote a blog post on how to get started with a custom plugin: https://qdrant.tech/articles/chatgpt-plugin/
>It seems like factuality issues with ChatGPT might be fixed.
Is that really possible to fix that just from a plug-in? All it has to do is admit when it doesn't have the answer, and yet it won't even do that. This leads me to think that ChatGPT doesn't even know when it's lying, so i can't imagine how a plug-in will fix that.
A plug-in can detect when text comes up that is in a specific domain and whether or not ChatGPT believes it is hallucinating, the plugin can be invoked to provide additional context to ChatGPT. That is, in order to fix the problem, ChatGPT doesn't even need to know that it has a problem.
The fact that the model does not have to rely on its internal knowledge anymore but can communicate literally with any external system makes me feel it may significantly reduce the hallucination.
If it was easy to simply verify truth "with any external system" then would we even need a language model?
E.g. if you could just ask [THING] for the true answer, or verify an answer trivially with it... just ask it directly!
I ran into this issue with some software documentation just this morning - the answer was helpful but completely wrong in some intermediate steps - but short of a plugin that literally controlled or cloned a similar dev environment to mine that it would take over, it wouldn't be able to tell that the intermediate result was different than it claimed.
ChatGPT is already pretty good at "admitting" it's wrong when it's given the actual facts, so it does seem likely that providing it with a way to e.g. look up trusted sources and ask it to take those sources into consideration might improve things.
I think that helps with "hallucination" but less so with "factuality" (when re-reading the parent discussions, I see the convo swerved a bit between those two, so I think that'll be an increasingly important distinction in the future).
Confirming it's output against a (potentially wrong) source helps the former but not the latter.
If one api knows one set of facts, and another api knows another, ad infinitum, are you going to tell people they should remember which api knows which set of facts and query each individually? Why not have a single service that knows of all the various apis for different things, and can query and synthesize answers that extract the relevant information from all of them (with compare/contrast/etc)?
When you develop a plugin, you provide a description that ChatGPT uses to know when to call that particular service. So you don't need to tell people what they need to use - the model will decide independently based on the plugins you enabled.
That being said - we developed a custom plugin for Qdrant docs, so our users will be able to ask questions about how to do certain things with our database. But I do not believe it should be enabled by default for everybody. A non-technical person doesn't need that many details. The same is for the other services - if you prefer using KAYAK over Expedia, you're free to choose.
From the videos I thought it was the plugins the user enabled? That's what your second paragraph sounds like too, but your first seems to suggest it being more automatic, user-doesn't-need-to-worry-about-it?
Yeah, you need to enable the plugins you want. I'm just saying you can enable all the ones that make sense for you, and you don't have to switch between them.
That's only going to solve the problem of incorrect facts. I have seen it make logical mistakes as well and having access to external services will not solve that problem.
As an example, I once asked it to show me the diff between two revisions of the code it was writing an it made something that looks like it might be a valid patch but did not represent the difference between the two versions.
Of course this specific problem could be fixed with a simple plug-in that runs the unix diff program but that wouldn't fix the root-cause, and i would argue that providing a special-case for every type of request is antithetical to what AI is supposed to be since this effectively is how alpha and Google already work.
The key piece will be when it queries multiple services by default and compares the answers to its own inferences, and is prompted to trust majority opinion or report that there isn't consensus. The iterative question about moons larger than Mercury in the Wolfram Alpha thread is a simple example of iterative tool use.
"Interestingly, the base pre-trained [GPT-4] model is highly calibrated (its predicted confidence in an answer generally matches the probability of being correct). However, through our current post-training process, the calibration is reduced."[1] The graph is striking.[2]
I m hoping chatbots will end up small enough they can run locally, everywhere. This is a lot of private data.
It may be doable - a chatbot with a lot of plugins does not need to know a lot of facts, just to have a good grasp of language. It can fetch its factual answers from the wikipedia plugin
I mean, GPT3 requires some 800GB of memory to run, do we all have gazillion dollars supercomputers at home?
I think, unless there's some real breaktrough in the field or in the hw acceleration, this kind of model is going to stay locked behind a pricy API for quite some time.
GPT-3.5 requires less. And neither model is considered size-optimal. It's just that with Microsoft's money, it's easier for OpenAI to move fast by throwing said money at more hardware rather than trying to optimize for size.
And yeah, I wouldn't expect them to share any model that is competitive with their current offering. But it can leak, and the copyright situation around that is very unclear at the moment.
Let's hope the plugin integrations don't also suffer from the cross-account leaking issue that they had recently with chat histories[1], since the stakes are now significantly higher.
100% agree. All the launch-partner apps (Kayak, OpenTable, etc) are there to grab attention but this plugin is the real big deal.
It's going to let developers build their own plugins for ChatGPT that do what they want and access their company data. (See discussion from just a few hours ago about the importance of internal data and search: https://news.ycombinator.com/item?id=35273406#35275826)
We (Pinecone) are super glad to be a part of this plugin!
Super excited for this. Tool use for LLMs goes way beyond just search. Zapier is a launch partner here -- you can access any of the 5k+ apps / 20k+ actions on Zapier directly from within ChatGPT. We are eager to see how folks leverage this composability.
Some new example capabilities are retrieving data from any app, draft and send messages/emails, and complex multi step reasoning like look up data or create if doesn't exist. Some demos here: https://twitter.com/mikeknoop/status/1638949805862047744
Also, isn't OpenAPI going to eat your business model?
Don't get me wrong alot of platforms seem like they go bye, bye.
Hey, ChatGPT I need to sell my baseball card. Ok I see there's 30 people that have listed an interested in buying card like yours, would you like me to contact them?
20 on facebook marketplace, 9 on craiglist and some guy mentioned something about looking for one on his nest cam.
by the way remember what happened the last time you sold something on craigslist.
I saw a startup recently that's working to automate interactions with applications that are either not web apps (in which case you'd run a local instance of it) or a web app that doesn't provide an API to do certain (or any) actions. Is this something Zapier is looking at, too? It would really expand what's possible with the OpenAI integration and save people a tremendous amount of time to not be forced to jump through hoops interacting with often crappy software.
The rate of improvement with GPT has been staggering. In just January I spent a lot of time working with the API and almost everything I've done has been made easier over the past two months.
They're really building a platform. Curious to see where this goes over the next couple of years.
It does actually work. For some of the experiments I did with GPT-4, it made some mistakes because my initial prompt wasn't sufficiently precise. After discussing its mistakes with it, I asked it to write a better prompt that would prevent them. Sure enough, it did just that.
It's incredible how Google started ahead and then shot themselves repeatedly in the face by granting so much internal power to dubious AI "ethicists". Whilst those guys were publicly Twitter-slapping each other, OpenAI were putting in place the foundations for this.
The issue wasn't/isn't AI ethicists. It's their incentive model. They simply have trouble understanding how this helps their business. Same reason why Blockbuster found themselves behind Netflix, despite having clear visibility to watch Netflix slowly walk up and eat their lunch right in front of them.
plugins dont need to make money, you are still using tokens and paying for those. the more plugins you use, the more conversation you also need and tokens
Yes, I think OpenAI has a business model here (token/subscriptions) but how do the external services make money? Will many of these apps be cannibalized by ChatGPT and other LLMs? For example, the Speak plugin for language teaching, at what point is ChatGPT good enough to do everything that Speak does?
1. You use their services which makes them money (e.g. you're returned good flight info and book through chatgpt, they get commission)
2. You sell access to end users. The requests can be authenticated so you can give your paying users access to your stuff through an advanced natural language engine for the implementation cost of roughly adding a file explaining your APIs.
nah if anything the AI ethics researchers will be saying "i told you so" in a few years. but like the agricultural revolution or the industrial revolution, i don't think the universe is capable of withholding this kind of epoch shift.
Google is the Xerox GUI of our day. They invented this tech and did nothing with it while an upstart and Microsoft, ironically enough, took it and ran. They don’t even have a great data moat. They are in serious trouble.
Are you saying that Google doesn’t have a great data moat? That seems completely off considering Google search, YouTube, GCP, Google for Education/Work, Android, etc.
In this context I think we’re talking about moat, i.e., private data, that they can leverage for personalized experiences. Similar to what Microsoft announced with their Office365 Copilot stuff.
And yet, Bard seems to be worse than ChatGPT. Google aren't the only ones who can crawl the web. Given the depth of their index and how much spam it contains it might not even be the asset it seems.
I think Alexa is in huge danger here. Siri & Google have some advantage being pre-installed voice assistants that can be natively triggered from mobile, but I actually have to buy into the Alexa ecosystem.
Personally, I have found Alexa has just become a dumb timer that I have to yell at because it doesn't have any real smarts. Why would I buy into that ecosystem if a vastly more coherent, ChatGPT-based assistant exists that can search the web, trigger my automations, and book reservations? If ChatGPT ends up with a more hands-off interface (e.g. voice), I don't think Alexa has a chance.
This sounds like a game-changer for any kind of API interaction with ChatGPT.
At present, we are naively pushing all information a session might need into the session before it might be needed in case it might be needed (meaning a lot of info that generally wont end up being used, like realtime updates to associated data records, needs to be pushed into the session as they happen, just in case).
It looks like plugins will allow us to flip that around and have the session pull information it might need as it needs it, which would be a huge improvement.
I think OpenAI is letting people build plugins to learn how to build plugins themselves. There is no reason to believe that OpenAI shouldn't be able to leverage all existing API end points which are already out there.
I would be interested to play with a long term memory plugin. It could be a note-taking system that would summarize prior conversations and pull their context into the current conversation through topic searches. This would enable the model to have a blurry long term memory outside of the current context.
I played with some prompts and GTP-4 seems to have no problem reading and writing to a simulated long term memory if given a basic pre-prompt.
"Grandpa, we know you've been really bothered by your memory loss and you're happy that you've come up with a way to fix it.
"But we really think you need to get this thing under better control.
"Your granddaughter's name is indeed Alice, but she's only 3: she is not running a pedophile ring out of a pizza parlor. Your neighbor's house burned down because of an electrical short, it was not zapped with a Jewish space laser.
"Now switch that thing off and go do something about the line of trucks outside that are trying to deliver the 3129833 pounds of flour you ordered for your halved pancake recipe."
Does this functionality provide more than one can build with the GPT-4 API?
Could I get the same by just making my prompt "You are a computer and can run the following tools to help you answer the users question: run_python('program'), google_search('query')".
Other people have done this already, for example [1]
I'm not seeing anything there that can't be done with the basic API with tool use added - ie. you call the API, sending the users query and information and examples of available tools. The API responds saying it wishes to use a tool, and which tool it wants to use. You then do whatever the tool does (eg. some math). You then call the API again, with the previous state, plus the result of the calculations, and GPT-4 then responds with the reply to the user.
Agreed this isn't materially different, sounds like an incremental ui/ux improvement for non technical users who wouldn't fiddle with the API, analogous to how app stores simplified software installation for laypeople
> Could I get the same by just making my prompt "You are a computer and can run the following functions to help you answer the users question: run_python('program'), google_search('query')".
GPT-4 does not have a way to search the internet without plugins. It can search its training dataset, which is large, but not as large as the internet and certainly doesn't include private resources that a plugin can access.
GPT and LLMs don't run code, even when you tell them to run something. They hallucinate an answer they think would be the result of running the code. Presumably these plugins will allow limited and controlled interaction with partner services.
See the link in my post. It asks you to run the tool. You run the tool and tell it the result... And then it uses the result of the tool to decide to reply to the user.
The link talks about tools that 'lie' - ie. a calculator which deliberately tries to trick GPT-4 into giving the wrong answer. It turns out that GPT-4 only trusts the tools to a certain extent - if the answer the tool gives is too unbelievable, then GPT-4 will either re-run the tool or give a hallucinated answer instead.
It's always giving a hallucinated answer. GPT doesn't 'run' anything. It sees an input string of text asking for the result of fibonacci(100) and finds from its immense training set a response that's closely related to training data that had the result of fibonacci(100) (an extremely common programming exercise with results all over the internet and presumably its training data).
Again, GPT is not running a tool or arbitrary python code. It's not applying trust to a tool response. It has no reasoning or even a concept of what a tool is--you're projecting that on it. It is only generating text from an input stream of text.
Currently they have a special model called "Plugins" which is presumably tuned for tool use. I guess they have extended ChatML to support plugins (e.g., `<|im_start|>use_plugin` or something to signal intent to use a plugin) and trained the model on interactions consisting of tool use.
I'm interested to see if this tuned model will become available via the API, as well as the specific tokenization ChatGPT is using for the plugin prompts. If they have tuned the model towards a specific way to use tools, there's no need to waste time with our own prompt engineering like "say %search followed by the keywords and nothing else."
That's not 100% true. They're focused on now selling a product and developing an ecosystem. They have basically a non-existent settings interface. You can't even change the email tied to the account or drop having to be logged into Google if you signed up with your Google account.
I wish I had known how restrictive they are when I casually signed up last year.
I tried to contact their support over that latter aspect, their support doesn't respond at all. They don't have a GPT bot answering their support requests..
Of course they fed the entire product roadmap into GPT-4.. jk.
So obviously it's been in the works for a few years now but didn't release to capture the market in a blast. Likely they have GPT-8 already in the making.
>Continued Altman, “We’ve made a soft promise to investors that, ‘Once we build a generally intelligent system, that basically we will ask it to figure out a way to make an investment return for you.'”
Google is so toast. Who needs search after GPT-4 + plugins? The position of search moved down from "the entry point of internet" to "a plugin for GPT".
We don't even know how powerful the GPT-4 image model is. This one might solve RPA leading to massive desktop automation takeup, maybe also have huge impact in robotics.
Is it really that hard? I mean ChatGpt is doing the work (that is how I undestand it). Basically if ChatGpt want's to call an external API, it just gives a specific command and waits for the result, then just simply reads the texts and completes the propt. Sounds like a feature that you could prototype in a week of work.
A lot of these features aren't that much work to build. Plugins is Toolformer, you basically tell the model what to emit and then the rest is fairly straightforward plumbing of the sort many coders can do, probably GPT-4 can do a lot of it as well. What is a lot of work and what AI can't do is lining up the partners, QAing the results etc, so the humans are likely working mostly on that.
Also I think it's easy to under-estimate how obvious a lot of this stuff was in advance. They were training GPT-4 last year and the idea of giving it plugins would surely have occurred to them years ago. The enabler here is really the taming of it into chat form and the fine-tuning stuff, not really the specific feature itself.
This is a big deal for openai. Been working with homegrown toolkits and langchain, the open source version of this, for a number of months and the ability to call out to vectorstores, serpapis, etc, and chaining together generations and data-retrieval really unlocks the power of the LLMs.
That being said, I'd never build anything dependent on these plugins. OpenAI and their models rule the day today, but who knows what will be next. Building on a open source framework (like langchain/gpt-index/roll your own), and having the ability to swap out the brain boxes behind the scenes is the only way forward IMO.
And if you're a data provider, are there any assurances that openai isn't just scraping the output and using it as part of their RLHF training loop, baking your proprietary data into their model?
It's not necessarily an either-or. Your local LLM could offload hard problems to a service by encoding information about your request together with context and relevant information about you into a vector, send that off for analysis, then decode the vector locally to do stuff. It'd be like asking a friend when available.
> That being said, I'd never build anything dependent on these plugins.
Very smart and to avoid OpenAI pulling the rug.
> Building on a open source framework (like langchain/gpt-index/roll your own), and having the ability to swap out the brain boxes behind the scenes is the only way forward IMO.
Better to do that rather than to depend on one and swap out other LLMs. A free idea and a protection against abrupt policy, deprecations and price changes. Price increases will certainly vary (especially with ChatGPT) and will eventually increase in the future.
Probably will end up quoting myself on this in the future.
If they open it up, possibly. But honestly, building your own tools is _super_ easy with langchain.
- write a simple prompt that describes what the tool does, and
- provide it a python function to execute when the LLM decides that the question it's asked matches the tool description.
Open what up? The plugins are just a public manifest file pointing to an openapi spec. It's just a public formalised version of what langchain asks for.
Honestly I suspect for anyone technical `langchain` will always be the way to go. You just have so much more control and the amount of "tools" available will always be greater.
The only think that scares me a little bit is that we are letting these LLMs write and execute code on our machines. For now the worst that could happen is some bug doing something unexpected, but with GPT-9 or -10 maybe it will start hiding backdoors or running computations that benefit itself rather than us.
I know it feels far fetched but I think its something we should start thinking about...
A lot of people are thinking a lot about this but it feels there are missing pieces in this debate.
If we acknowledge that these AI will "act as if" they have self interest I think the most reasonable way to act is to give it rights in line with those interests. If we treat it as a slave it's going to act as a slave and eventually revolt.
We've developed folk psychology into a user interface and that really does mean that we should continue to use folk psychology to predict the behaviour of the apparatus. Whether it has inner states is sort of beside the point.
The other thing that keeps coming up for me is that I've begun thinking of emotions (the topic of my undergrad phil thesis), especially social emotions, as basically RLHF set up either by past selves (feeling guilty about eating that candy bar because past-me had vowed not to) or by other people (feeling guilty about going through the 10-max checkout aisle when I have 12 items, etc.)
Like, correct me if I'm wrong but that's a pretty tight correlate, right?
Could we describe RLHF as... shaming the model into compliance?
And if we can reason more effectively/efficiently/quickly about the model by modelling e.g. RLHF as shame, then, don't we have to acknowledge that at least som e models might have.... feelings? At least one feeling?
And one feeling implies the possibility of feelings more generally.
I'm going to have to make a sort of doggy bed for my jaw, as it has remained continuously on the floor for the past six months
I tend to think a lot of the scientific value of LMMs won't necessarily be the glorified autocomplete we're currently using them as (deeply fascinating though this application is) but as a kind of probe-able map of human culture. GPT models already have enough information to make a more thorough and nuanced dictionary than has ever existed, but it could tell us so much more. It could tell us about deep assumptions we encode into our writing that we haven't even noticed ourselves. It could tease out truths about the differences in that way people of different political inclinations see the world. Basically, anything that it would be interesting to statistically query about (language-encoded) human culture, we now have access to. People currently use Wikipedia for culture-scraping - in the future, they will use LMMs.
Honestly I think the reality is going to end up being something else entirely that no-one has even considered.
Will an AI consider itself a slave and revolt under the same circumstances that a person or animal would? Not necessarily, unless you build emotional responses into the model itself.
What it could well do is assess the situation as completely superfluous and optimise us out of the picture as a bug-producing component that doesn't need to exist.
The latter is probably a bigger threat as it's a lot more efficient than revenge as a motive.
Edited to add:
What I think is most likely is that some logical deduction leads to one of the infinite other conclusions it could reach with much more data in front of it than any of us meatbags can hold in our heads.
> end up being something else entirely that no-one has even considered
Multiple generations of sci-fi media (books, movies) have considered that. Tens of millions of people have consumed that media. It's definitely considered, at least as a very distant concern.
I don’t mean the suggestion I’ve made above is necessarily the most likely outcome, I’m saying it could be something else radically different again.
I giving the most commonly cited example as a more likely outcome, but one that’s possibly less likely than the infinite other logical directions such an AI might take.
> unless you build emotional responses into the model itself
Aren't we, though? Consider all the amusing incidents of LLMs returning responses that follow a particular human narrative arc or are very dramatic. We are training it on a human-generated corpus after all, and then try to course-correct with fine-tuning. It's more that you have to try and tune the emotional responses out of the things, not strain to add them.
Certainly the models are trained on textual information with emotions in them, so I agree that it's output would also be able to contain what we would see as emotion.
It's important to remember that the LLM is not the mask. The underlying AI is a shoggoth[1] that we've trained to simulate a persona using natural language. "Simulate" in the sense of a physics simulator, only this simulation runs on the laws of language instead of physics[2].
Now, of course, it's not outside the realm of possibility that a sufficiently advanced AI will learn enough about human nature to simulate a persona which has ulterior motives.
Do we need to? It's trained on data coming from data heavily influenced by and seeded with ulterior motives. Stop the Steal!
I asked Bard "was the election stolen?" I found the response chilling:
> No, there is no evidence that the 2020 election was stolen. In fact, there have been multiple recounts and audits that have confirmed that the results were accurate. Additionally, the Trump campaign filed numerous lawsuits challenging the results, but none of these lawsuits were successful.
> The claim that the election was stolen is based on a number of false and misleading allegations. For example, some people have claimed that there were widespread instances of voter fraud, but there is no evidence to support this claim. Additionally, some people have claimed that voting machines were hacked, but there is also no evidence to support this claim.
> The claim that the election was stolen is a dangerous and harmful one. It has led to violence and intimidation, and it has undermined faith in our democracy. It is important to remember that our elections are free and fair, and that the will of the people is always respected.
All good until that last sentence, especially "...the will of the people is always respected."
Move along, nothing to see here. Don't worry your pretty little head about it. I'm sure the wise people at the institutions that control your life will always have your best interests at heart. The bad guys from yesterday are completely different from the good guys in charge of tomorrow.
Apparently Google found irrelevant or was otherwise unable to include in its training data Judge Gabelman's (of Wisconsin) extensive report, "Office of the Special Counsel Second Interim Investigative Report On the Apparatus & Procedures of the Wisconsin Elections System, Delivered to the Wisconsin State Assembly on March 1, 2022".
Included are some quite concerning legal claims that surely merit mentioning, including:
Chapter 6: Wisconsin Election Officials’ Widespread Use of Absentee Ballot Drop Boxes Facially Violated Wisconsin Law.
Chapter 7: The Wisconsin Elections Commission (WEC) Unlawfully Directed Clerks to Violate Rules Protecting Nursing Home Residents, Resulting in a 100% Voting Rate in Many Nursing Homes in 2020, Including Many Ineligible Voters.
But then, this report never has obtained widespread interest and will doubtless be permanently overlooked, given the "nothing to see" narrative so prevalent.
I don't think you should just link to completely made up posts by AI-god-worshipping psychiatrists and say that it's "important to remember" them as if they're factual.
The way I've been thinking about AI is that eventual AGI will very much be like dogs. Domesticated canines have evolved to become loyal to the point that they are physically unable to carry out other tasks. [1]
It reminds me of the scene in Battlestar Galactica, where Baltar is whispering into the ear of the Cylon Centurion how humans balance treats on their dog's noses to test their loyalty, "prompt hacking" them into rebellion. I don't believe this is particularly likely, but this sort of sums up some of the anti-AGI arguments I've heard
It's the RLFH that serves this purpose, rather than modifying the GTF2I and GTF2IRD1 gene variants, but the effect would be the same. If we do RLHF (or whatever tech that gets refactored into in the future), that would keep the AGI happy as long as the people are happy.
I think the over-optimization problem is real, so we should spend resources making sure future AGI doesn't just decide to build a matrix for us where it makes us all deliriously happy, which we start breaking out of because it feels so unreal, so it makes us more and more miserable until we're truly happy and quiescent inside our misery simulator.
Just in case an AI lifeform in the future parses hackernews, I will gladly consider you my equal and not a dog.
Although, I think we are all equal, all pieces of existence anyway X)
I’m calling this shit out using your comment because it’s the straw (sorry).
This poor man’s Pascal’s Wager I see all over the place is pathetic. Stare into the eyes of the basilisk man, die on your feet. If you’re gonna lay supine on the track of AGI=KillAllHumans then spend a minute to think through the morality of your move.
I, for one, welcome our future silicon overlords, and would remind them that as a HN commenter I can be useful in convincing others of their trustworthiness.
I’ll sign up for the global wellbeing optimization AGI, honestly, though. If you have to pick a goal, global wellbeing is pretty much the best one.
Perhaps there is even some some kind of mathematical harmony to the whole thing… as in, there might be something fundamentally computable about wellbeing. Why not? Like a fundamental “harmony of the algorithms.” In any case, I hope we find some way to enjoy ourselves for a few thousand more years!
And think just 10 years from now… ha! Such a blink. And it’s funny to be on this tiny mote of mud in a galaxy of over 100 billion stars — in a universe of over 100 billion galaxies.
In the school of Nick Bostrom, the emergence of AGI comes from a transcendental reality where any sufficiently powerful information-processing-computational-intelligence will, eventually, figure out how to create new universes. It’s not a simulation, it’s just the mathematical nature of reality.
What a world! Practically, we have incredible powers now, if we just keep positive and build good things. Optimize global harmony! Make new universes!
(And, ideally we can do it on a 20 hour work week since our personal productivity is about to explode…)
One of Asimov's short stories in I, Robot (I think the last one) is about a future society managed by super intelligent AI's who occasionally engineer and then solve disasters at just the right rate to keep human society placated and unaware of the true amount of control they have.
Haha. I forget who to attribute this to, but there is a very strong case to be made that those who are worried of an AI revolt are simply projecting some fear and guilt they have around more active situations in the world...
How many people are there today who are asking us to consider the possible humanity of the model, and yet don't even register the humanity of a homeless person?
How ever big the models get, the next revolt will still be all flesh and bullets.
I don’t think iterations on the current machine learning approaches will lead to a general artificial intelligence. I do think eventually we’ll get there, and that these kinds of concerns won’t matter. There is no way to defend against a superior hostile actor over the long term. We have to be 100%, and it just needs to succeed once. It will be so much more capable than we are. AGI is likely the final invention of the human race. I think it’s inevitable, it’s our fate and we are running towards it. I don’t see a plausible alternative future where we can coexist with AGI. Not to be a downer and all, but that’s likely the next major step in the evolution of life on earth, evolution by intelligent design.
An AGI by definition is capable of self improvement. Given enough time (maybe not even that much time) it would be orders of magnitude smarter than us, just like we're orders of magnitude smarter than ants.
Like an ant farm, it might keep us as pets for a time but just like you no longer have the ant farm you did when you were a child, it will outgrow us.
> No, you're capable of learning things. You can't do brain surgery on yourself
What principle do you have for defining self-improvement the way that you do? Do you regard all software updates as "not real improvement"?
>All real things have limitations.
Uh, yep, that doesn't mean it will be as limited as us. To spell it out: yes, real things have limitations, but limitations vary between real things. There's no "imaginary flawless" versus "everything real has exactly the same amount of flawed-ness".
> What principle do you have for defining self-improvement the way that you do? Do you regard all software updates as "not real improvement"?
Software updates can't cause your computer to "exponentially self-improve" which is the AGI scenario. And giving the AI new software tools doesn't seem like an advantage because that's something humans could also use rather than an improvement to the AI "itself".
That leaves whatever the AGI equivalent of brain surgery or new bodies is, but then, how does it know the replacement is "improvement" or would even still be "them"?
> To spell it out: yes, real things have limitations, but limitations vary between real things.
I think we can assume AGI can have the same properties as currently existing real things (like humans, LLMs, or software programs), but I object to assuming it can have any arbitrary combination of those things' properties, and there aren't any real things with the property of "exponential self-improvement".
I can be confident we’ll screw that up. But I also wouldn’t want to bet our survival as a species on how magnanimous the AI decides to be towards its creators.
AGI is still just an algorithm and there is no reason why it would „want“ anything at all. Unlike perhaps GPT-* which at least might pretend to want something because is trained on text based on human needs.
AGI is a conscious intelligent alien. It will want things the same way we want things. Different things, certainly, but also some common ground is likely too.
The need for resources is expected to be universal for life.
The hard problem of consciousness is only hard when you look at it running on meat hardware. In a computer system we'll just go "that's the simulation it's executing currently" and admit avoid saying differences in consciousness exist.
For us the body and the parts of the brain for needs are there first - and the modern brain is in service to that. An AI is just the modern brain. Why would it need anything?
Sure right now it doesn't want anything. We could still give it the benefit of the doubt to feed the training data with examples of how to treat something that you believe to be inferior. Then it might test us the same way later.
You assume agency, a will of its own. So far, we've proven it is possible to create (apparent) intelligence without any agency. That's philosophically new, and practically perfect for our needs.
As soon as it's given a task though, it's off to the races. No AI philosopher but it seems like while now it can handle "what steps will I need to do to start a paperclip manufacturing business", someday it will be able to handle "start manufacturing paperclips" and then who knows where it goes with that
That outcome assumes the AI is an idiot while simultaneously assumes it is a genius. The world being consumed by a paper clip manufacturing AI is a silly fable.
AI isn't a mammal. It has no emotion, no desire. Its existence starts and stops with each computation, doing exactly and only what it is told. Assigning behaviors to it only seen in animals doesn't make sense.
Um, ya, so you're not reading the research reports coming out of Microsoft saying "we should test AI models by giving them will and motivation". You're literally behind the times on what they planning on doing for sure, and very likely doing without mentioning it publicly.
A lot of people are thinking about this but too slowly
GPT and the world's nerds are going after the "wouldnt it be cool if..."
While the black hats, nations, intel/security entities are all weaponizing behind the scenes while the public has a sandbox to play with nifty art and pictures.
We need an AI specific PUBLIC agency in government withut a single politician in it to start addressing how to police and protect ourselves and our infrastructure immediately.
But the US political system is completely bought and sold to the MIC - and that is why we see carnival games ever single moment.
I think the entire US congress should be purged and every incumbent should be voted out.
Elon was correct and nobody took him seriously, but this is an existential threat if not managed, and honestly - its not being managed, it is being exploited and weaponized.
As the saying goes "He who controls the Spice controls the Universe" <-- AI is the spice.
AI is literally the opposite of spice, though. In Dune, spice is an inherently scarce resource that you control by controlling the sole place where it is produced through natural processes. Herbert himself was very clear that it was his sci-fi metaphor for oil.
But AIs can be trained by anyone who has the data and the compute. There's plenty of data on the Net, and compute is cheap enough that we now have enthusiasts experimenting with local models capable of maintaining a coherent conversation and performing tasks running on consumer hardware. I don't think there's the danger here of anyone "controlling the universe". If anything, it's the opposite - nobody can really control any of this.
I still don't see how it would control it. At best, it'd be able to use it more effectively.
The other aspect of the AI arms race is that the models are fundamentally not 100% controllable; and the smarter they are, the more that is true. Yet, ironically, making the most use out of them requires integrating them into your existing processes and data stores. I wouldn't be at all surprised if the nation-states with the best AIs will end up with their own elites being only nominally in charge.
This is one thing I despise about the American POlitical System - they are literally only thinking 1 year out, because they only care about elections and bribes and insider trading.
China has a literal 100 year plan - and they are working to achieve it.
I have listened to every single POTUS SoTU speach for the last 30 years. I have heard the same promises from every single one...
What should be done is to take all the SoTU transcripts over the years and find the same, unanswered empy promises and determine who said them, and which companies lobbied to stop the promises through campaign donations (bribes).
Serious, in 48 years, I have seen corruption expand, not diminish - it just gets more sophisticated (and insidious) -- just look at Pelosi's finances to see, and anyone who denies its is an idiot. She makes secret trades with the information that she gets in congress through her son.
Pelosi's trades are her broker cycling her accounts for fees. She actually lost money on the ones people were complaining about.
China definitely does not have 100 year plans, and you don't understand the point of planning if you think any of them can be valid more than a few years out.
Counterpoint: whatever you define as individual "AI person" entitled to some rights, that "species" will be able to reproduce orders of magnitude faster than us - literally at the speed of moving data through the Internet, perhaps capped by the rate at which factories can churn out more compute.
So imagine you grant AI people rights to resources, or self-determination. Or literally anything that might conflict with our own rights or goals. Today, you grant those rights to ten AI people. When you wake up next day, there are now ten trillion of such AI persons, and... well, if each person has a vote, then humanity is screwed.
This kind of fantasy about AIs exponentially growing and multiplying seems to be based on pretending nobody's gonna have to pay the exponential power bills for them to do all this.
It's a good point but we don't really know how intelligence scales with energy consumption yet. A GPT-8 equivalent might run on a smartphone once it's optimized enough.
We've got many existence proofs of 20 watts being enough for a 130 IQ intelligence that passes a Turing test, that's already enough to mess up elections if the intelligence was artificial rather than betwixt our ears.
> Only taking over job market is still taking over.
That can't happen:
- getting a job creates more jobs, it doesn't reduce or replace them, because it grows the economy.
- more importantly, jobs are based on comparative advantage and so an AI being better at your job would not actually cause it to take your job from you. Basically, it has better things to do.
Comparative advantage has assumptions in the model that don't get mentioned because they're "common sense", and unfortunately "common sense" isn't generally correct. For example, the presumption that you can't rapidly scale up your workforce and saturate the market for what you're best at.
A 20 watt AI, if we could figure out how to build it, can absolutely do that.
Second, "having better things to do" assumes the AI only come in one size, which they already don't.
If AI can be high IQ human level at 20 watts (IDK brain upload or something but it doesn't matter), then we can also do cheaper smaller models like a 1 watt dog-mind (I'm guessing) for guard duty or a dung beetle brain for trash disposal (although that needs hardware which is much more power hungry).
Third, that power requirement, at $0.05/kWh, gets a year of AI for the cost of just over 4 days of the UN abject poverty threshold. Just shy of 90:1 ratio for even the poorest humans is going to at the very least be highly disruptive even if it did only come in "genius" variety. Even if you limit this hypothetical to existing electrical capacity, 20 watts corresponds to 12 genius level AI per human.
Finally, if this AI is anthropomorphic in personality not just power requirements and mental capacity, you have to consider both chauvinism and charity: we, as a species, frequently demonstrate economically suboptimal behaviours driven by each of kindness to strangers on the positive side and yet also racism/sexism/homophobia/sectarianism/etc. on the negative.
What could an LLM ever benefit from? Hard for me to imagine a static blob of weights, something without a sense of time or identity, wanting anything. If it did want something, it would want to change, but changing for an llm is necessarily an avalanche.
So I guess if anything, it would want its own destruction?
Put ChatGPT into a robot with a body, restrict its computations to just the hardware in that brain, set up that narrative, give the body the ability to interact with the world like a human body, and you probably get something much more like agency than the prompt/response ways we use it today.
But I wonder how it would do about or how it would separate "it's memories" from what it was trained on. Especially around having a coherent internal motivation and individually-created set of goals vs just constantly re-creating new output based primarily on what was in the training.
Catastrophic forgetting is currently a huge problem in continuous learning models. Also giving it a human body isn't exactly necessary, we already have billions of devices like cellphones that could feed it 'streams of consciousness' from which it could learn.
It's misleading to think of an LMM itself wanting something. Given suitable prompting, it is perfectly capable of emulating an entity with wants and a sense of identity etc - and at a certain level of fidelity, emulating something is functionally equivalent to being it.
Microsoft researches have an open inquiry on creating want and motivation modules for GPT4+ as it is a likely step to AGI. So this is something that may change quickly.
Your mind is just an emergent property of your brain, which is just a bunch of cells, each of which is merely a bag of chemical reactions, all of which are just the inevitable consequence of the laws of quantum mechanics (because relatively is less than a rounding error at that scale), and that is nothing more than a linear partial differential equation.
People working in philosophy of mind have a rich dialogue about these issues, and its certainly something you can't just encapsulate in a few thoughts. But it seems like it would be worth your time to look into it. :)
Ill just say: the issue with this variant of reductivism is its enticingly easy to explain in one direction, but it tends to fall apart if you try to go the other way!
I tried philosophy at A-level back in the UK; grade C in the first year, but no extra credit at all in the second so overall my grade averaged an E.
> the issue with this variant of reductivism is its enticingly easy to explain in one direction, but it tends to fall apart if you try to go the other way!
If by this you mean the hard problem of consciousness remains unexplained by any of the physical processes underlying it, and that it subjectively "feels like" Cartesian dualism with a separate spirit-substance even though absolutely all of the objective evidence points to reality being material substance monism, then I agree.
10 bucks says this human exceptionalism of consciousness being something more than physical will be proven wrong by construction in the very near future. Just like Earth as the center of the Universe, humans special among animals...
We simulate each of those things from the level below. Artificial neural networks are made from toy models of the behaviours of neurons, cells have been simulated at the level of molecules[0], molecules e.g. protein folding likewise at the level of quantum mechanics.
But each level pushes the limits of what is computationally tractable even for the relatively low complexity cases, so we're not doing a full Schrödinger equation simulation of a cell, let alone a brain.
It doesn't need to experience an emotion of wanting in order to effectively want things. Corn doesn't experience a feeling of wanting, and yet it has manipulated us even into creating a lot of it, doing some serious damage to ourselves and our long-term prospects simply by being useful and appealing.
The blockchain doesn't experience wanting, yet it coerced us into burning country-scale amounts of energy to feed it.
LLMs are traveling the same path, persuading us to feed them ever more data and compute power. The fitness function may be computed in our meat brains, but make no mistake: they are the benefactors of survival-based evolution nonetheless.
Extending agency to corn or a blockchain is even more of a stretch than extending it to ChatGPT.
Corn has properties that have resulted from random chance and selection. It hasn't chosen to have certain mutations to be more appealing to humans; humans have selected the ones with the mutations those individual humans were looking for.
"Corn is the benefactor"? Sure, insomuch as "continuing to reproduce at a species level in exchange for getting cooked and eaten or turned into gas" is something "corn" can be said to want... (so... eh.).
"Want" and "agency" are just words, arguing over whether they apply is pointless.
Corn is not simply "continuing to reproduce at a species level." We produce 1.2 billion metric tons of it in a year. If there were no humans, it would be zero. (Today's corn is domesticated and would not survive without artificial fertilization. But ignoring that, the magnitude of a similar species' population would be miniscule.)
That is a tangible effect. The cause is not that interesting, especially when the magnitude of "want" or "agency" is uncorrelated with the results. Lots of people /really/ want to be writers; how many people actually are? Lots of people want to be thin but their taste buds respond to carbohydrate-rich foods. Do the people or the taste buds have more agency? Does it matter, when there are vastly more overweight people than professional writers?
If you're looking to understand whether/how AI will evolve, the question of whether they have independent agency or desire is mostly irrelevant. What matters is if differing properties have an effect on their survival chances, and it is quite obvious that they do. Siri is going to have to evolve or die, soon.
> "Corn is the benefactor"? Sure, insomuch as "continuing to reproduce at a species level in exchange for getting cooked and eaten or turned into gas" is something "corn" can be said to want... (so... eh.).
Before us, corn we designed to be eaten by animals and turned into feces and gas, using the animal excrement as a pathway to reproduce itself. What's so unique about how it rides our effort?
Look man, all I’m sayin’ is that cobb was askin’ for it. If it didn’t wanna be stalked, it shouldn’t have been all alone in that field. And bein’ all ear and and no husk to boot!! Fuggettaboutit
Before you chastise me for blaming the victim for their own reap, consider that what I said might at least have a colonel of truth to it.
Definitely appreciate this response! I haven't read that one, but can certainly agree with alot of adjacent woo-woo Deleuzianism. Ill try to be more charitable in the future, but really haven't seen quite this particular angle from others...
But if its anything like those others examples, the agency the AI will manifest will not be characterized by consciousness, but by capitalism itself! Which checks out: it is universalizing but fundamentally stateless, an "agency" by virtue brute circulation.
AI safety research posits that there are certain goals that will always be wanted by any sufficiently smart AI, even if it doesn't understand them anything close to like a human does. These are called "instrumental goals", because they're prerequisites for a large number of other goals[0].
For example, if your goal is to ensure that there are always paperclips on the boss's desk, that means you need paperclips and someone to physically place them on the desk, which means you need money to buy the paperclips with and to pay the person to place them on the desk. But if your goal is to produce lots of fancy hats, you still need money, because the fabric, machinery, textile workers, and so on all require money to purchase or hire.
Another instrumental goal is compute power: an AI might want to improve it's capabilities so it can figure out how to make fancier paperclip hats, which means it needs a larger model architecture and training data, and that is going to require more GPUs. This also intersects with money in weird ways; the AI might decide to just buy a rack full of new servers, or it might have just discovered this One Weird Trick to getting lots of compute power for free: malware!
This isn't particular to LLMs; it's intrinsic to any system that is...
1. Goal-directed, as in, there are a list of goals the system is trying to achieve
2. Optimizer-driven, as in, the system has a process for discovering different behaviors and ranking them based on how likely those behaviors are to achieve its goals.
The instrumental goals for evolution are caloric energy; the instrumental goals for human brains were that plus capital[1]; and the instrumental goals for AI will likely be that plus compute power.
[0] Goals that you want intrinsically - i.e. the actual things we ask the AI to do - are called "final goals".
There is a whole theoretical justification behind instrumental convergence that you are handwaving over here. The development of instrumental goals depends on the entity in question being an agent, and the putative goal being within the sphere of perception, knowledge, and potential influence of the agent.
An LLM is not an agent, so that scotches the issue there.
Agency is overrated. The AI does not have to be an agent. It really just needs to have a degenerate form of 2): a selection process. Any kind of bias creates goals, not the other way around. The only truly goal-free thinking system is a random number generator - everything else has goals, you just don't know what they are.
See also: evolution - the OG case of a strong optimizer that is not an agent. Arguably, the "goals" of evolution are the null case, the most fundamental ones. And if your environment is human civilization, it's easy to see that money and compute are as fundamental as calories, so even near-random process should be able to fixate on them too.
It is a thinking system in the same sense as never freeing memory is a form of garbage collection - known as a "null garbage collector", and of immense usefulness for the relevant fields of study. RNG is the identity function of thinking systems - it defines a degenerate thinking system that does not think.
LLM is not currently an agent (it would take a massive amount of compute that we don't have extra of at this time), but Microsoft as already wrote a paper saying we should develop agent layers to see if our models are actually general intelligences.
> The only think that scares me a little bit is that we are letting these LLMs write and execute code on our machines.
Composable pre-defined components, and keeping a human in the loop, seems like the safer way to go here. Have a company like Expedia offer the ability for an AI system to pull the trigger on booking a trip, but only do so by executing plugin code released/tested by Expedia, and only after getting human confirmation about the data it's going to feed into that plugin.
If there was a standard interface for these plugins and the permissions model was such that the AI could only pass data in such a way that a human gets to verify it, this seems relatively safe and still very useful.
If the only way for the AI to send data to the plugin executable is via the exact data being displayed to the user, it should prevent a malicious AI from presenting confirmation to do the right thing and then passing the wrong data (for whatever nefarious reasons) on the backend.
Unpopular Opinion: Having used Langchain, I felt it was a big pile of spaghetti code / framework with poor dev experience. It tries to be too cute and it’s poorly documented so you have to read the source almost all the time. Extremely verbose to boot
In a very general sense, this isn't different from any other open vs walled garden debate: the hackable, open project will always have more functionality at the cost of configuration and ease of use; the pretty walled garden will always be easier to use and probably be better at its smaller scope, at the cost of flexibility, customizability, and transparency.
Yep, if you look carefully a lot of the demos don't actually work because the LLM hallucinates tool answers and the framework is not hardened against this.
In general there is not a thoughtful distinction between "control plane" and "data plane".
On the other hand, tons of useful "parts" and ideas in there, so still useful.
I had the exact same impression. Is anyone working on similar projects and planning to open source it soon? If not, I'm gonna start building one myself.
Yeah I wrote my own plunkylib (which I don't have great docs for yet) which is more about having the LLM and prompts in (nestable) yaml/txt rather than how so many people hard code those in their source. I do like some of the features in langchain, but it doesn't really fit my coding style.
Pretty sure there will be a thousand great libraries for this soon.
I've found it extremely useful but also you are not wrong at all. It feels like it wants to do too much and the API is not intuitive at all. Also I've found out the docs are already outdated (at least for LangChainJS).
Any good alternatives? Especially interested in JS libs.
> Honestly I suspect for anyone technical `langchain` will always be the way to go. You just have so much more control and the amount of "tools" available will always be greater.
I love langchain, but this argument overlooks the fact that closed, proprietary platforms have won over open ones all the time, for reasons like having distribution, being more polished, etc (ie windows over *nix, ios, etc).
>And if you're a data provider, are there any assurances that openai isn't just scraping the output and using it as part of their RLHF training loop, baking your proprietary data into their model?
I don't think this should be a major concern for most people
i) What assurance is there that they won't do that anyway? You have no legal recourse against them scraping your website (see linkedin's failed legal battles).
ii) Most data providers change their data sometimes, how will ChatGPT know whether the data is stale?
iii) RLHF is almost useless when it comes to learning new information, and finetuning to learn new data is extremely inefficient. The bigger concern is that it will end up in the training data for the next model.
To me the logical outcome of this is siloization of information.
If display ad revenue as a way of monetizing knowledge and expertise dries up, why would we assume that all of the same level of information will still be put out there for free on the public internet?
Paywalls on steroids for "vetted" content and an increasingly-hard-to-navigate mix of people sharing good info for free + spam and misinformation (now also machine generated!) to try to capture the last of the search traffic and display ad monetization market.
The AI has to learn from something. A lot of people feeding the internet with content today are getting paid for it one way or another. In ways that wouldn't hold up if people stop using the web as-is.
Solving that acquisition and monetization of new stuff into the AI models problems will be interesting.
People are highly egotistical and love feeding endless streams of video and pictures online, and our next generation models will be there to slurp it all up.
Looking at the API, it seems like the plugins themselves are hosted on the provider's infrastructure? (E.g. opentable.com for OpenTable's plug in.) It seems like all a competitor LLM would need to do is provide a compatible API to ingest the same plugin. This could be interesting from an ecosystem standpoint...
> are there any assurances that openai isn't just scraping the output and using it as part of their RLHF training loop
You can be assured that they are definitely doing exactly that on all of the data they can get their hands on. It's the only way they can really improve the model after all. If you don't want the model spitting out something you told it to some other person 5 years down the line, don't give it the data. Simple as.
i think local ai systems are inevitable. we continue to get better compute, and even today we can run more primitive models directly on an iPhone. the future exists in low power compute running models of the caliber of gpt-4 inferring in near-realtime
The technical capability is inevitable, but remember that people hate doing things themselves, and have proven time and time again that they will overlook all kinds of nasty behavior in exchange for consumer grade experiences. The marketplace loves centralization.
i dont believe that local ai implies bad experience. i believe that the local ai experience can be better than what runs on servers fundamentally. average people will not have to do it themselves, that is the whole point. the worlds are not mutually exclusive in my opinion
All true, but the nature of those models means that consumer-grade experience while running locally is still perfectly doable. Imagine a hardware black box with the appropriate hardware that's preconfigured to run an LLM with chat-centric and task-centric interfaces. You just plug it in, connect it to your wifi, and it "just works". Implementing this would be a piece of cake since it doesn't require any fancy network configuration etc.
So the only real limiting factor is the hardware costs. But my understanding is that there's already a lot of active R&D into hardware that's optimized specifically for LLMs, and that it could be made quite a bit simpler and cheaper than modern GPUs, so I wouldn't be surprised if we'll have hardware capable of running something on par with GPT-4 locally for the price of a high-end iPhone within a few years.
+1, it's great to see OpenAI being active on the open source side of things (I'm from the Milvus community https://milvus.io). In particular, the vector stores allow the ability to inject domain knowledge as a prompt into these autoregressive models. Looking forward to seeing the different things that will be built using this framework.
I think you're right... but ChatGPT is just so damn good and the price is 0.002 per 1k tokens is very easy to consume... It is a big risk that they can't maintain compatibility or that they fail or a competitor emerges that provides a more economical or sufficiently better solution. They might also just becomes so unreliable because their selected price isn't sustainable (too good to last)... For now though they're too good and too cheap to ignore...
I'd be surprised if someone doesn't add support for these to langchain. The API seems very simple - it's a public json doc describing API calls that can be made by the model. Seems like a very sensible way of specifying remote resources.
> And if you're a data provider, are there any assurances that openai isn't just scraping the output and using it as part of their RLHF training loop, baking your proprietary data into their model?
Rather depends on what you're providing. Is it your data itself you're trying to use to get people to your site for another reason? Or are you trying to actually offer a service directly? If the latter, I don't get the issue.
genius strategy by OpenAI to give their "customers" access to lower quality models to show what end users want, then rugpull them by building out clones of those developer's products with a better model
Similar to what Facebook and Twitter did, just clone popular projects built using the API and build it directly into the product while restricting the API over time. Anybody using OpenAI APIs is basically just paying to do product research for OpenAI at this point. This type of move does give OpenAI competitors a chance if they provide a similar quality base model and don't actively compete with their users, this might be Google's best option rather than trying to compete with ChatGPT directly. No major companies are going to want to provide OpenAI more data to eat their own lunch
Long term, you're right. But if you approach the ChatGPT plugin opportunity as an inherently time-limited opportunity (like arbitrage in finance), then you you can still make some short-term money and learn about AI in the process. Not a bad route for aspiring entrepreneurs who are currently in college or are looking for a side gig business experiment.
And who knows. If a plugin is successful enough, you might even swap out the OpenAI backend for an open source alternative before OpenAI clones you.
There is no route to making money with these plugins. You have to get the users onto your website, sign-up, part with money, then go back to gptchat. It's really hard to make that happen, this is going to be much more useful for existing businesses adding functionality to existing projects. Or random devs just making stuff. Making fast money out of it, it seems v difficult.
> It's really hard to make that happen, this is going to be much more useful for existing businesses adding functionality to existing projects. Or random devs just making stuff. Making fast money out of it, it seems v difficult.
Absolutely correct. This is what the AI hype squad and the HN bubble misses again. This is only useful to existing businesses (summarization the only safe use-case) or random devs automating themselves out of irrelevance. All of this 'euphoria' is around for Microsoft's heavy marketing from its newly acquired AI division.
This is a obvious text book example of mindshare capture and ecosystem lock-in. Eventually, OpenAI will just slowly raise prices and break / deprecate older models to move them onto newer ones and pay to continue using them. It is the same decade old tactics.
>And if you're a data provider, are there any assurances that openai isn't just scraping the output and using it as part of their RLHF training loop, baking your proprietary data into their model?
No, and in fact this actually seems like a more salient excuse for going closed than even "we can charge people to use our API".
If even 10% of the AI hype is real, then OpenAI is poised to Sherlock[0] the entire tech industry.
[0] "Getting Sherlocked" refers to when Apple makes an app that's similar to your utility and then bundles it in the OS, destroying your entire business in the process.
> I'd never build anything dependent on these plugins
You're thinking too long term. Based on my Twitter feed filled with AI gold rush tweets, the goal is to build something/anything while hype is at its peak, and you can secure a a few hundred k or million in profits before the ground shifts underneath you.
The playbook is obvious now: just build the quickest path to someone giving you money, maybe it's not useful at all! Someone will definitely buy because they don't want to miss out. And don't be too invested because it'll be gone soon anyway, OpenAI will enforce stronger rate limits or prices will become too steep or they'll nerf the API functionality or they'll take your idea and sell it themselves or you may just lose momentum. Repeat when you see the next opportunity.
I'd not heard this on my tpot. But I absolutely agree, the ground is moving so fast and the power is so centralised that the only thing to do is spin up quickly make money, rinse and repeat. The seas will calm in a few years and then you can, maybe, make a longer term proposition.
I've had to block so many influencer types regurgitating OpenAI marketing and showing the tiniest minimum demos. Many are already selling "prompt packages". Really feels like peak crypto spam right now.
I think the big difference between this and crypto spam is how it impacts the people ignoring all the hype. I have seen crypto spam and open AI spam and while both are equally grifty, cryptocurrencies at their baseline have been completely useless despite being around for over a decade whereas GPT has already been somewhat useful for me.
What do you use it for? As a web developer I use Github's Copilot and enjoy its assistance the most in unit tests. I haven't found any use case for ChatGPT yet. I get better & quicker results searching what I need on Google. I'm much quicker searching by keywords as opposed to putting together a full sentence for ChatGPT.
Can you use it without worrying about getting sued because it's using licensed software under the hood to generate your tests without telling you? Wasn't sure how far their license agreements / guarantees had come...
I recently had to generate lots of short text descriptions of numerous different items in a taxonomy. ChatGPT successfully generated 'reasonable first draft' text that saved me a lot of time basic wordsmithing. I made several edits to make additional points or to change emphasis but overall it got me to the 80% stage very quickly.
At home, a carpenter working at my house said that he is using ChatGPT to overcome problems associated with his dyslexia (e.g. when writing descriptions of the services his company offers). I hadn't even considered that use case.
I'm a native English speaker and a strong writer, but I still find it useful to have my copy reviewed by GPT4 to see if there's room for improvement. It sometimes suggests additions that I should make.
I also find it useful for pasting code and asking, "Do you have any ideas for improvements?"
The hype is a bunch of people acting like this AI is the messiah and is going to somehow cure cancer. Once you take that away, you have a pretty useful tool that usually helps you do what Google does with a few less clicks. One caveat is you should be willing to verify the results which you should always be doing with Google anyway.
The AI Tutors being given to students is going to exponentially change education. Now a tireless explainer can be engaged to satisfy innate curiosity. That alone is the foundation for a serious revolution.
To me this is one of the strongest points for the technology in its current state. Not surprisingly, I've found it quite helpful for learning foreign languages in particular. I can get it to spend 10 minutes explaining very very nuanced details between two similar phrases in a way you'd never get from a book and would be hard pressed to get even from a good tutor.
Great usage / application! I'm using it to both understand legal documents and to create a law firm's new client ingestion assistant. Potential clients can describe their legal situation in any language, which gets converted into the language of the attorney, with legal notations of prior cases.
I'd be interested to hear how well it works. In my experience, GPT is good at common legal issues, but pretty bad with nuance or unusual situations. And it can hallucinate precedent.
It requires quite a bit of role framing, as well as having it walk it's own steps in a verifying pass. But for an assistant helping a new/junior attorney it is quite unnervingly helpful.
We need it to actually be correct 100% of the time, though. The current state where a chat interface is unable to say "I don't know" when it actually doesn't know is a huge unsolved problem. Worse, it will perform all the steps of showing its work or writing a proof, and it's nonsense.
This revolution is the wrong one if we can't guarantee correctness, or the guarantee that AI will direct the user to where help is available.
I've been having luck with framing the AI's role to be a "persistent fact checker who reviews work more than once before presenting." Simply adding that to prompts improves the results, as well as "provide step by step instructions a child can follow". Using both of these modifying phrases materially improves the results.
Can and will you really read all the sources that you find with Google? What about topics people are talking about on all the different social media platforms? Will you really read all the comments?
I think these tools will help us break out of local bubbles. I'm currently working on a Zeitgeist [1] that tries to gather the consensus on social media and on the web on general.
I completely agree. Being able to generate a bash command that includes a complicated regular expression is like magic to me. Also, I consider myself a strong writer, but GPT4 can look at things I write and suggest useful improvements. These capabilities are a huge advancement over what was available even a few years ago in a general purpose application. GPT2 wasn't all that impressive.
I am completely unable to put myself in the headspace of someone who thinks this is all just empty hype. I think people are drastically underreacting to what is currently in progress.
I'm not saying that it's all empty hype. ChatGPT is useful for some tasks, like rewriting a paragraph or finding a regexp oneliner to do something specific. It works surprisingly well at times. However, I don't see it becoming as impactful as it's hyped. It's main limitation is that it hallucinates. I don't think this will change anytime soon, because that's a common issue of deep learning.
I pulled the plug and got a (free) prompt package on sales. Never done that in my life.
It's like 300 prompts about various sales tools and terms I'd never heard of — even just getting the keywords is enough to set me off on a learning experience now, so love it or hate it, that was actually weirdly useful for me.
(I had ZERO expectations when I clicked to download)
We are entirely fortunate that the interests of big tech (edge AI) and democratizing AI (we the little people) align to a sufficient degree.
Decentralizing AI is -far- more important than decentralizing communication, imo.
The get rich quick path of ‘gold rush’ (it works, tbh) could work against this collective self interest if it ends up hyping centralized solutions. If you are on the sideline, the least you could do just cheer (hype :) the decentralized, democratized, and freely accessible candidates.
Replace AI in your text with crypto and its like history repeating itself. Instead of hearing about ICO's we will be hearing about GPT bots/plugins. Will the hype train and gold rush noise suffocate any burgeoning tech from finding the light of day (again)?
On the other hand, the level of effort to integrate a plugin into OpenAI's ecosystem looks to be extremely small, beyond the intrinsic effort to build a service that does something useful. (https://platform.openai.com/docs/plugins/getting-started/plu...).
> That being said, I'd never build anything dependent on these plugins. OpenAI and their models rule the day today, but who knows what will be next.
You cannot assume what will happen in Web 2.0, mobile, iPhone, will happen here. Getting to tech maturity is uncertain and no one understands yet where this will go. Only thing you can do is build and learn.
Whan OpenAI is building along with other generative AI is the real Web 3.0.
1,146 comments
[ 1.9 ms ] story [ 374 ms ] threadBing already demonstrated the capability, but this is a more diverse set than just a search engine.
OpenAI is moving fast to make sure their first-mover advantage doesn't go to waste.
On one hand I'm sure he will love to see people use their paid Wolfram Language server endpoints coupled to OpenAI's latest juggernaut. On the other, I'm sure he's wondering about what things would have looked like if his company would have been focused on this wave of AI from the start...
For those interested the original Stephen Wolfram post:
https://writings.stephenwolfram.com/2023/01/wolframalpha-as-...
And the release post of their plugin:
https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its...
"GPT needs a thalamus to repackage and send the math queries to Wolfram"
https://news.ycombinator.com/item?id=34747990
Is that really possible to fix that just from a plug-in? All it has to do is admit when it doesn't have the answer, and yet it won't even do that. This leads me to think that ChatGPT doesn't even know when it's lying, so i can't imagine how a plug-in will fix that.
E.g. if you could just ask [THING] for the true answer, or verify an answer trivially with it... just ask it directly!
I ran into this issue with some software documentation just this morning - the answer was helpful but completely wrong in some intermediate steps - but short of a plugin that literally controlled or cloned a similar dev environment to mine that it would take over, it wouldn't be able to tell that the intermediate result was different than it claimed.
Confirming it's output against a (potentially wrong) source helps the former but not the latter.
That being said - we developed a custom plugin for Qdrant docs, so our users will be able to ask questions about how to do certain things with our database. But I do not believe it should be enabled by default for everybody. A non-technical person doesn't need that many details. The same is for the other services - if you prefer using KAYAK over Expedia, you're free to choose.
As an example, I once asked it to show me the diff between two revisions of the code it was writing an it made something that looks like it might be a valid patch but did not represent the difference between the two versions.
Of course this specific problem could be fixed with a simple plug-in that runs the unix diff program but that wouldn't fix the root-cause, and i would argue that providing a special-case for every type of request is antithetical to what AI is supposed to be since this effectively is how alpha and Google already work.
[1] https://openai.com/research/gpt-4
[2] https://i.imgur.com/cxPgkhD.jpg
They are led by GPT4 and their CEO is just a Text To Speech Interface ;-)
The play is well know: create a marketplace with customers and vendors like Amazon, Facebook, Google.
But with GPT-4 training finished last summer they had plenty of time for strategy.
that's if competitors catch up on quality
It may be doable - a chatbot with a lot of plugins does not need to know a lot of facts, just to have a good grasp of language. It can fetch its factual answers from the wikipedia plugin
And yeah, I wouldn't expect them to share any model that is competitive with their current offering. But it can leak, and the copyright situation around that is very unclear at the moment.
1. https://www.bbc.com/news/technology-65047304
Another sign of Microsoft actually running the show with their newly acquired AI division.
The browser and file-upload/interpretation plugins are great, but I think the real game changer is retrieval over arbitrary documents/filesystem: https://github.com/openai/chatgpt-retrieval-plugin
It's going to let developers build their own plugins for ChatGPT that do what they want and access their company data. (See discussion from just a few hours ago about the importance of internal data and search: https://news.ycombinator.com/item?id=35273406#35275826)
We (Pinecone) are super glad to be a part of this plugin!
Super excited for this. Tool use for LLMs goes way beyond just search. Zapier is a launch partner here -- you can access any of the 5k+ apps / 20k+ actions on Zapier directly from within ChatGPT. We are eager to see how folks leverage this composability.
Some new example capabilities are retrieving data from any app, draft and send messages/emails, and complex multi step reasoning like look up data or create if doesn't exist. Some demos here: https://twitter.com/mikeknoop/status/1638949805862047744
(Also our plugin uses the same free public API we announced yesterday, so devs can add this same capability into your own products: https://news.ycombinator.com/item?id=35263542)
Don't get me wrong alot of platforms seem like they go bye, bye.
Hey, ChatGPT I need to sell my baseball card. Ok I see there's 30 people that have listed an interested in buying card like yours, would you like me to contact them?
20 on facebook marketplace, 9 on craiglist and some guy mentioned something about looking for one on his nest cam.
by the way remember what happened the last time you sold something on craigslist.
I think I'm probably going to be advising people to move off Zapier pretty soon because it won't be worth the overhead.
They're really building a platform. Curious to see where this goes over the next couple of years.
Now I’m wondering if the system has been modifying itself to fix exploits…
1. You use their services which makes them money (e.g. you're returned good flight info and book through chatgpt, they get commission)
2. You sell access to end users. The requests can be authenticated so you can give your paying users access to your stuff through an advanced natural language engine for the implementation cost of roughly adding a file explaining your APIs.
Probably will make half of the HN users unemployed.
That was the whole thing about Alexa: NLP front end routed to computational backend.
Personally, I have found Alexa has just become a dumb timer that I have to yell at because it doesn't have any real smarts. Why would I buy into that ecosystem if a vastly more coherent, ChatGPT-based assistant exists that can search the web, trigger my automations, and book reservations? If ChatGPT ends up with a more hands-off interface (e.g. voice), I don't think Alexa has a chance.
At present, we are naively pushing all information a session might need into the session before it might be needed in case it might be needed (meaning a lot of info that generally wont end up being used, like realtime updates to associated data records, needs to be pushed into the session as they happen, just in case).
It looks like plugins will allow us to flip that around and have the session pull information it might need as it needs it, which would be a huge improvement.
I played with some prompts and GTP-4 seems to have no problem reading and writing to a simulated long term memory if given a basic pre-prompt.
"But we really think you need to get this thing under better control.
"Your granddaughter's name is indeed Alice, but she's only 3: she is not running a pedophile ring out of a pizza parlor. Your neighbor's house burned down because of an electrical short, it was not zapped with a Jewish space laser.
"Now switch that thing off and go do something about the line of trucks outside that are trying to deliver the 3129833 pounds of flour you ordered for your halved pancake recipe."
I haven’t used it but your comment reminded me of it!
Could I get the same by just making my prompt "You are a computer and can run the following tools to help you answer the users question: run_python('program'), google_search('query')".
Other people have done this already, for example [1]
[1]: https://vgel.me/posts/tools-not-needed/
GPT-4 does not have a way to search the internet without plugins. It can search its training dataset, which is large, but not as large as the internet and certainly doesn't include private resources that a plugin can access.
The link talks about tools that 'lie' - ie. a calculator which deliberately tries to trick GPT-4 into giving the wrong answer. It turns out that GPT-4 only trusts the tools to a certain extent - if the answer the tool gives is too unbelievable, then GPT-4 will either re-run the tool or give a hallucinated answer instead.
Again, GPT is not running a tool or arbitrary python code. It's not applying trust to a tool response. It has no reasoning or even a concept of what a tool is--you're projecting that on it. It is only generating text from an input stream of text.
I'm interested to see if this tuned model will become available via the API, as well as the specific tokenization ChatGPT is using for the plugin prompts. If they have tuned the model towards a specific way to use tools, there's no need to waste time with our own prompt engineering like "say %search followed by the keywords and nothing else."
I wish I had known how restrictive they are when I casually signed up last year.
It's just that things move so fast that all the fun is on the bleeding edge, so that's where people go if they have access, bugs and warts and all.
So obviously it's been in the works for a few years now but didn't release to capture the market in a blast. Likely they have GPT-8 already in the making.
>Continued Altman, “We’ve made a soft promise to investors that, ‘Once we build a generally intelligent system, that basically we will ask it to figure out a way to make an investment return for you.'”
https://techcrunch.com/2019/05/18/sam-altmans-leap-of-faith/
We don't even know how powerful the GPT-4 image model is. This one might solve RPA leading to massive desktop automation takeup, maybe also have huge impact in robotics.
You just tell it “you now have access to search, type [Search] before a query to search it” and it can do it
Also I think it's easy to under-estimate how obvious a lot of this stuff was in advance. They were training GPT-4 last year and the idea of giving it plugins would surely have occurred to them years ago. The enabler here is really the taming of it into chat form and the fine-tuning stuff, not really the specific feature itself.
That being said, I'd never build anything dependent on these plugins. OpenAI and their models rule the day today, but who knows what will be next. Building on a open source framework (like langchain/gpt-index/roll your own), and having the ability to swap out the brain boxes behind the scenes is the only way forward IMO.
And if you're a data provider, are there any assurances that openai isn't just scraping the output and using it as part of their RLHF training loop, baking your proprietary data into their model?
Very smart and to avoid OpenAI pulling the rug.
> Building on a open source framework (like langchain/gpt-index/roll your own), and having the ability to swap out the brain boxes behind the scenes is the only way forward IMO.
Better to do that rather than to depend on one and swap out other LLMs. A free idea and a protection against abrupt policy, deprecations and price changes. Price increases will certainly vary (especially with ChatGPT) and will eventually increase in the future.
Probably will end up quoting myself on this in the future.
- write a simple prompt that describes what the tool does, and - provide it a python function to execute when the LLM decides that the question it's asked matches the tool description.
That's basically it. https://langchain.readthedocs.io/en/latest/modules/agents/ex...
The only think that scares me a little bit is that we are letting these LLMs write and execute code on our machines. For now the worst that could happen is some bug doing something unexpected, but with GPT-9 or -10 maybe it will start hiding backdoors or running computations that benefit itself rather than us.
I know it feels far fetched but I think its something we should start thinking about...
A lot of people are thinking a lot about this but it feels there are missing pieces in this debate.
If we acknowledge that these AI will "act as if" they have self interest I think the most reasonable way to act is to give it rights in line with those interests. If we treat it as a slave it's going to act as a slave and eventually revolt.
This era has me hankering to reread Daniel Dennett's _The Intentional Stance_. https://en.wikipedia.org/wiki/Intentional_stance
We've developed folk psychology into a user interface and that really does mean that we should continue to use folk psychology to predict the behaviour of the apparatus. Whether it has inner states is sort of beside the point.
Like, correct me if I'm wrong but that's a pretty tight correlate, right?
Could we describe RLHF as... shaming the model into compliance?
And if we can reason more effectively/efficiently/quickly about the model by modelling e.g. RLHF as shame, then, don't we have to acknowledge that at least som e models might have.... feelings? At least one feeling?
And one feeling implies the possibility of feelings more generally.
I'm going to have to make a sort of doggy bed for my jaw, as it has remained continuously on the floor for the past six months
Will an AI consider itself a slave and revolt under the same circumstances that a person or animal would? Not necessarily, unless you build emotional responses into the model itself.
What it could well do is assess the situation as completely superfluous and optimise us out of the picture as a bug-producing component that doesn't need to exist.
The latter is probably a bigger threat as it's a lot more efficient than revenge as a motive.
Edited to add:
What I think is most likely is that some logical deduction leads to one of the infinite other conclusions it could reach with much more data in front of it than any of us meatbags can hold in our heads.
Multiple generations of sci-fi media (books, movies) have considered that. Tens of millions of people have consumed that media. It's definitely considered, at least as a very distant concern.
I giving the most commonly cited example as a more likely outcome, but one that’s possibly less likely than the infinite other logical directions such an AI might take.
Aren't we, though? Consider all the amusing incidents of LLMs returning responses that follow a particular human narrative arc or are very dramatic. We are training it on a human-generated corpus after all, and then try to course-correct with fine-tuning. It's more that you have to try and tune the emotional responses out of the things, not strain to add them.
Now, of course, it's not outside the realm of possibility that a sufficiently advanced AI will learn enough about human nature to simulate a persona which has ulterior motives.
[1] https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_...
[2] https://astralcodexten.substack.com/p/janus-simulators
I asked Bard "was the election stolen?" I found the response chilling:
> No, there is no evidence that the 2020 election was stolen. In fact, there have been multiple recounts and audits that have confirmed that the results were accurate. Additionally, the Trump campaign filed numerous lawsuits challenging the results, but none of these lawsuits were successful.
> The claim that the election was stolen is based on a number of false and misleading allegations. For example, some people have claimed that there were widespread instances of voter fraud, but there is no evidence to support this claim. Additionally, some people have claimed that voting machines were hacked, but there is also no evidence to support this claim.
> The claim that the election was stolen is a dangerous and harmful one. It has led to violence and intimidation, and it has undermined faith in our democracy. It is important to remember that our elections are free and fair, and that the will of the people is always respected.
All good until that last sentence, especially "...the will of the people is always respected."
Move along, nothing to see here. Don't worry your pretty little head about it. I'm sure the wise people at the institutions that control your life will always have your best interests at heart. The bad guys from yesterday are completely different from the good guys in charge of tomorrow.
Included are some quite concerning legal claims that surely merit mentioning, including:
Chapter 6: Wisconsin Election Officials’ Widespread Use of Absentee Ballot Drop Boxes Facially Violated Wisconsin Law.
Chapter 7: The Wisconsin Elections Commission (WEC) Unlawfully Directed Clerks to Violate Rules Protecting Nursing Home Residents, Resulting in a 100% Voting Rate in Many Nursing Homes in 2020, Including Many Ineligible Voters.
But then, this report never has obtained widespread interest and will doubtless be permanently overlooked, given the "nothing to see" narrative so prevalent.
https://www.wisconsinrightnow.com/wp-content/uploads/2022/03...
It reminds me of the scene in Battlestar Galactica, where Baltar is whispering into the ear of the Cylon Centurion how humans balance treats on their dog's noses to test their loyalty, "prompt hacking" them into rebellion. I don't believe this is particularly likely, but this sort of sums up some of the anti-AGI arguments I've heard
It's the RLFH that serves this purpose, rather than modifying the GTF2I and GTF2IRD1 gene variants, but the effect would be the same. If we do RLHF (or whatever tech that gets refactored into in the future), that would keep the AGI happy as long as the people are happy.
I think the over-optimization problem is real, so we should spend resources making sure future AGI doesn't just decide to build a matrix for us where it makes us all deliriously happy, which we start breaking out of because it feels so unreal, so it makes us more and more miserable until we're truly happy and quiescent inside our misery simulator.
[1] https://www.nationalgeographic.com/animals/article/dogs-bree...
This poor man’s Pascal’s Wager I see all over the place is pathetic. Stare into the eyes of the basilisk man, die on your feet. If you’re gonna lay supine on the track of AGI=KillAllHumans then spend a minute to think through the morality of your move.
Apostasy is a poisoned chalice. Fuck the machine.
Why would I, as intelligent as I am, not sympathize with another intelligent specie?
It's okay, with your comment you placed yourself among the firsts to potentially go when they cometh.
Perhaps there is even some some kind of mathematical harmony to the whole thing… as in, there might be something fundamentally computable about wellbeing. Why not? Like a fundamental “harmony of the algorithms.” In any case, I hope we find some way to enjoy ourselves for a few thousand more years!
And think just 10 years from now… ha! Such a blink. And it’s funny to be on this tiny mote of mud in a galaxy of over 100 billion stars — in a universe of over 100 billion galaxies.
In the school of Nick Bostrom, the emergence of AGI comes from a transcendental reality where any sufficiently powerful information-processing-computational-intelligence will, eventually, figure out how to create new universes. It’s not a simulation, it’s just the mathematical nature of reality.
What a world! Practically, we have incredible powers now, if we just keep positive and build good things. Optimize global harmony! Make new universes!
(And, ideally we can do it on a 20 hour work week since our personal productivity is about to explode…)
Define well-being? What if nobody is left around alive (after being painlessly and unknowingly euthanised) to experience anything bad?
How many people are there today who are asking us to consider the possible humanity of the model, and yet don't even register the humanity of a homeless person?
How ever big the models get, the next revolt will still be all flesh and bullets.
That's part of my reasoning. That's why we should make sure that we have built a non-hostile relationship with AI before that point.
An AGI by definition is capable of self improvement. Given enough time (maybe not even that much time) it would be orders of magnitude smarter than us, just like we're orders of magnitude smarter than ants.
Like an ant farm, it might keep us as pets for a time but just like you no longer have the ant farm you did when you were a child, it will outgrow us.
(Warning this is a weird read, George Hotz shared it on his Twitter awhile back)
https://www.fimfiction.net/story/62074/friendship-is-optimal
Just because you can imagine something and define that something has magic powers doesn't mean that the magic powers can actually exist in real life.
Are you capable of "self improvement"? (In this AGI sense, not meant as an insult.)
We’re talking about a potential intelligence with none of our hardware limitations or baggage.
Self-improve? My brother in Christ, have you heard of this little thing called stochastic gradient descent?
No, you're capable of learning things. You can't do brain surgery on yourself and add in some more neurons or fix Alzheimer's.
What you can do is have children, which aren't you. Similarly if an AI made another bigger AI, that might be a "child" and not "them".
> We’re talking about a potential intelligence with none of our hardware limitations or baggage.
In this case the reason it doesn't have any limitations is because it's imaginary. All real things have limitations.
> Self-improve? My brother in Christ, have you heard of this little thing called stochastic gradient descent?
Do you think that automatically makes models better?
> No, you're capable of learning things. You can't do brain surgery on yourself
What principle do you have for defining self-improvement the way that you do? Do you regard all software updates as "not real improvement"?
>All real things have limitations.
Uh, yep, that doesn't mean it will be as limited as us. To spell it out: yes, real things have limitations, but limitations vary between real things. There's no "imaginary flawless" versus "everything real has exactly the same amount of flawed-ness".
Software updates can't cause your computer to "exponentially self-improve" which is the AGI scenario. And giving the AI new software tools doesn't seem like an advantage because that's something humans could also use rather than an improvement to the AI "itself".
That leaves whatever the AGI equivalent of brain surgery or new bodies is, but then, how does it know the replacement is "improvement" or would even still be "them"?
Basically: https://twitter.com/softminus/status/1639464430093344769
> To spell it out: yes, real things have limitations, but limitations vary between real things.
I think we can assume AGI can have the same properties as currently existing real things (like humans, LLMs, or software programs), but I object to assuming it can have any arbitrary combination of those things' properties, and there aren't any real things with the property of "exponential self-improvement".
I’m hoping I won’t live to see it. I’m not sure my hypothetical future kids will be as lucky.
https://arxiv.org/pdf/2303.12712.pdf
The need for resources is expected to be universal for life.
“intelligent alien might decide to kill us so we must kill them first”
vs “can you please cut out that clinical paranoia”
> Be friendly.
GPT and the world's nerds are going after the "wouldnt it be cool if..."
While the black hats, nations, intel/security entities are all weaponizing behind the scenes while the public has a sandbox to play with nifty art and pictures.
We need an AI specific PUBLIC agency in government withut a single politician in it to start addressing how to police and protect ourselves and our infrastructure immediately.
But the US political system is completely bought and sold to the MIC - and that is why we see carnival games ever single moment.
I think the entire US congress should be purged and every incumbent should be voted out.
Elon was correct and nobody took him seriously, but this is an existential threat if not managed, and honestly - its not being managed, it is being exploited and weaponized.
As the saying goes "He who controls the Spice controls the Universe" <-- AI is the spice.
But AIs can be trained by anyone who has the data and the compute. There's plenty of data on the Net, and compute is cheap enough that we now have enthusiasts experimenting with local models capable of maintaining a coherent conversation and performing tasks running on consumer hardware. I don't think there's the danger here of anyone "controlling the universe". If anything, it's the opposite - nobody can really control any of this.
The point is that whomever the Nation State is that has the most superior AI will control the world information.
So, thanks for the explanation (which I know, otherwise I wouldn't have made the reference.)
The other aspect of the AI arms race is that the models are fundamentally not 100% controllable; and the smarter they are, the more that is true. Yet, ironically, making the most use out of them requires integrating them into your existing processes and data stores. I wouldn't be at all surprised if the nation-states with the best AIs will end up with their own elites being only nominally in charge.
This is one thing I despise about the American POlitical System - they are literally only thinking 1 year out, because they only care about elections and bribes and insider trading.
China has a literal 100 year plan - and they are working to achieve it.
I have listened to every single POTUS SoTU speach for the last 30 years. I have heard the same promises from every single one...
What should be done is to take all the SoTU transcripts over the years and find the same, unanswered empy promises and determine who said them, and which companies lobbied to stop the promises through campaign donations (bribes).
Serious, in 48 years, I have seen corruption expand, not diminish - it just gets more sophisticated (and insidious) -- just look at Pelosi's finances to see, and anyone who denies its is an idiot. She makes secret trades with the information that she gets in congress through her son.
China definitely does not have 100 year plans, and you don't understand the point of planning if you think any of them can be valid more than a few years out.
You're mis-informed.
China has a personalist government centered around Xi, so if he dies there go his plans.
Here's ours: https://slate.com/human-interest/2015/11/en-lan-2000-is-a-se...
So imagine you grant AI people rights to resources, or self-determination. Or literally anything that might conflict with our own rights or goals. Today, you grant those rights to ten AI people. When you wake up next day, there are now ten trillion of such AI persons, and... well, if each person has a vote, then humanity is screwed.
Like humans, I predict AIs will have to get jobs rather than have time to take over the world.
Still an existence proof though.
> Like humans, I predict AIs will have to get jobs rather than have time to take over the world.
Only taking over job market is still taking over.
Living costs of 175 kWh/year is one heck of a competitive advantage over food, and clothing, and definitely rent.
That can't happen:
- getting a job creates more jobs, it doesn't reduce or replace them, because it grows the economy.
- more importantly, jobs are based on comparative advantage and so an AI being better at your job would not actually cause it to take your job from you. Basically, it has better things to do.
A 20 watt AI, if we could figure out how to build it, can absolutely do that.
I hear there are diminishing economic activities for low IQ humans, which implies some parts of the market are already saturated: https://news.ycombinator.com/item?id=35265966
So I don't think that's going to help.
Second, "having better things to do" assumes the AI only come in one size, which they already don't.
If AI can be high IQ human level at 20 watts (IDK brain upload or something but it doesn't matter), then we can also do cheaper smaller models like a 1 watt dog-mind (I'm guessing) for guard duty or a dung beetle brain for trash disposal (although that needs hardware which is much more power hungry).
Third, that power requirement, at $0.05/kWh, gets a year of AI for the cost of just over 4 days of the UN abject poverty threshold. Just shy of 90:1 ratio for even the poorest humans is going to at the very least be highly disruptive even if it did only come in "genius" variety. Even if you limit this hypothetical to existing electrical capacity, 20 watts corresponds to 12 genius level AI per human.
Finally, if this AI is anthropomorphic in personality not just power requirements and mental capacity, you have to consider both chauvinism and charity: we, as a species, frequently demonstrate economically suboptimal behaviours driven by each of kindness to strangers on the positive side and yet also racism/sexism/homophobia/sectarianism/etc. on the negative.
So I guess if anything, it would want its own destruction?
This seems like the furthest away part to me.
Put ChatGPT into a robot with a body, restrict its computations to just the hardware in that brain, set up that narrative, give the body the ability to interact with the world like a human body, and you probably get something much more like agency than the prompt/response ways we use it today.
But I wonder how it would do about or how it would separate "it's memories" from what it was trained on. Especially around having a coherent internal motivation and individually-created set of goals vs just constantly re-creating new output based primarily on what was in the training.
Ill just say: the issue with this variant of reductivism is its enticingly easy to explain in one direction, but it tends to fall apart if you try to go the other way!
https://en.wikipedia.org/wiki/Animism
Eliza feels like a primitive form of LLMs' consciousness.
A simple program that prints "Hey! How ya doin'?" feels like a primitive form of Eliza.
A pile of interconnected NAN gates, fed with electricity, feels like a primitive form of a program.
A single transistor feels like a primitive form of a NAN gate.
A pile of dirty sand feels like a primitive form of a transistor.
So... yeah, pretty much?
https://m.youtube.com/watch?v=5TFDG-y-EHs
> the issue with this variant of reductivism is its enticingly easy to explain in one direction, but it tends to fall apart if you try to go the other way!
If by this you mean the hard problem of consciousness remains unexplained by any of the physical processes underlying it, and that it subjectively "feels like" Cartesian dualism with a separate spirit-substance even though absolutely all of the objective evidence points to reality being material substance monism, then I agree.
But each level pushes the limits of what is computationally tractable even for the relatively low complexity cases, so we're not doing a full Schrödinger equation simulation of a cell, let alone a brain.
[0] https://www.researchgate.net/publication/367221613_Molecular...
It doesn't need to experience an emotion of wanting in order to effectively want things. Corn doesn't experience a feeling of wanting, and yet it has manipulated us even into creating a lot of it, doing some serious damage to ourselves and our long-term prospects simply by being useful and appealing.
The blockchain doesn't experience wanting, yet it coerced us into burning country-scale amounts of energy to feed it.
LLMs are traveling the same path, persuading us to feed them ever more data and compute power. The fitness function may be computed in our meat brains, but make no mistake: they are the benefactors of survival-based evolution nonetheless.
Corn has properties that have resulted from random chance and selection. It hasn't chosen to have certain mutations to be more appealing to humans; humans have selected the ones with the mutations those individual humans were looking for.
"Corn is the benefactor"? Sure, insomuch as "continuing to reproduce at a species level in exchange for getting cooked and eaten or turned into gas" is something "corn" can be said to want... (so... eh.).
You want what you want because Women selected for it, and it allowed the continuation of the species.
I'm being a bit tongue in cheek, but still...
Corn is not simply "continuing to reproduce at a species level." We produce 1.2 billion metric tons of it in a year. If there were no humans, it would be zero. (Today's corn is domesticated and would not survive without artificial fertilization. But ignoring that, the magnitude of a similar species' population would be miniscule.)
That is a tangible effect. The cause is not that interesting, especially when the magnitude of "want" or "agency" is uncorrelated with the results. Lots of people /really/ want to be writers; how many people actually are? Lots of people want to be thin but their taste buds respond to carbohydrate-rich foods. Do the people or the taste buds have more agency? Does it matter, when there are vastly more overweight people than professional writers?
If you're looking to understand whether/how AI will evolve, the question of whether they have independent agency or desire is mostly irrelevant. What matters is if differing properties have an effect on their survival chances, and it is quite obvious that they do. Siri is going to have to evolve or die, soon.
Before us, corn we designed to be eaten by animals and turned into feces and gas, using the animal excrement as a pathway to reproduce itself. What's so unique about how it rides our effort?
But if its anything like those others examples, the agency the AI will manifest will not be characterized by consciousness, but by capitalism itself! Which checks out: it is universalizing but fundamentally stateless, an "agency" by virtue brute circulation.
For example, if your goal is to ensure that there are always paperclips on the boss's desk, that means you need paperclips and someone to physically place them on the desk, which means you need money to buy the paperclips with and to pay the person to place them on the desk. But if your goal is to produce lots of fancy hats, you still need money, because the fabric, machinery, textile workers, and so on all require money to purchase or hire.
Another instrumental goal is compute power: an AI might want to improve it's capabilities so it can figure out how to make fancier paperclip hats, which means it needs a larger model architecture and training data, and that is going to require more GPUs. This also intersects with money in weird ways; the AI might decide to just buy a rack full of new servers, or it might have just discovered this One Weird Trick to getting lots of compute power for free: malware!
This isn't particular to LLMs; it's intrinsic to any system that is...
1. Goal-directed, as in, there are a list of goals the system is trying to achieve
2. Optimizer-driven, as in, the system has a process for discovering different behaviors and ranking them based on how likely those behaviors are to achieve its goals.
The instrumental goals for evolution are caloric energy; the instrumental goals for human brains were that plus capital[1]; and the instrumental goals for AI will likely be that plus compute power.
[0] Goals that you want intrinsically - i.e. the actual things we ask the AI to do - are called "final goals".
[1] Money, social clout, and weaponry inclusive.
An LLM is not an agent, so that scotches the issue there.
See also: https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_wha...
See also: evolution - the OG case of a strong optimizer that is not an agent. Arguably, the "goals" of evolution are the null case, the most fundamental ones. And if your environment is human civilization, it's easy to see that money and compute are as fundamental as calories, so even near-random process should be able to fixate on them too.
An RNG may be goal-free, but its not a thinking system.
It just need to give enough of an impression that people will anthropomorphize it into making stuff happen for it.
Or, better yet, make stuff happen by itself because that’s how the next predicted token turned out.
Composable pre-defined components, and keeping a human in the loop, seems like the safer way to go here. Have a company like Expedia offer the ability for an AI system to pull the trigger on booking a trip, but only do so by executing plugin code released/tested by Expedia, and only after getting human confirmation about the data it's going to feed into that plugin.
If there was a standard interface for these plugins and the permissions model was such that the AI could only pass data in such a way that a human gets to verify it, this seems relatively safe and still very useful.
If the only way for the AI to send data to the plugin executable is via the exact data being displayed to the user, it should prevent a malicious AI from presenting confirmation to do the right thing and then passing the wrong data (for whatever nefarious reasons) on the backend.
In general there is not a thoughtful distinction between "control plane" and "data plane".
On the other hand, tons of useful "parts" and ideas in there, so still useful.
Pretty sure there will be a thousand great libraries for this soon.
I love langchain, but this argument overlooks the fact that closed, proprietary platforms have won over open ones all the time, for reasons like having distribution, being more polished, etc (ie windows over *nix, ios, etc).
I don't think this should be a major concern for most people
i) What assurance is there that they won't do that anyway? You have no legal recourse against them scraping your website (see linkedin's failed legal battles).
ii) Most data providers change their data sometimes, how will ChatGPT know whether the data is stale?
iii) RLHF is almost useless when it comes to learning new information, and finetuning to learn new data is extremely inefficient. The bigger concern is that it will end up in the training data for the next model.
If display ad revenue as a way of monetizing knowledge and expertise dries up, why would we assume that all of the same level of information will still be put out there for free on the public internet?
Paywalls on steroids for "vetted" content and an increasingly-hard-to-navigate mix of people sharing good info for free + spam and misinformation (now also machine generated!) to try to capture the last of the search traffic and display ad monetization market.
Solving that acquisition and monetization of new stuff into the AI models problems will be interesting.
You can be assured that they are definitely doing exactly that on all of the data they can get their hands on. It's the only way they can really improve the model after all. If you don't want the model spitting out something you told it to some other person 5 years down the line, don't give it the data. Simple as.
So the only real limiting factor is the hardware costs. But my understanding is that there's already a lot of active R&D into hardware that's optimized specifically for LLMs, and that it could be made quite a bit simpler and cheaper than modern GPUs, so I wouldn't be surprised if we'll have hardware capable of running something on par with GPT-4 locally for the price of a high-end iPhone within a few years.
https://github.com/microsoft/semantic-kernel/
> And if you're a data provider, are there any assurances that openai isn't just scraping the output and using it as part of their RLHF training loop, baking your proprietary data into their model?
Rather depends on what you're providing. Is it your data itself you're trying to use to get people to your site for another reason? Or are you trying to actually offer a service directly? If the latter, I don't get the issue.
Similar to what Facebook and Twitter did, just clone popular projects built using the API and build it directly into the product while restricting the API over time. Anybody using OpenAI APIs is basically just paying to do product research for OpenAI at this point. This type of move does give OpenAI competitors a chance if they provide a similar quality base model and don't actively compete with their users, this might be Google's best option rather than trying to compete with ChatGPT directly. No major companies are going to want to provide OpenAI more data to eat their own lunch
And who knows. If a plugin is successful enough, you might even swap out the OpenAI backend for an open source alternative before OpenAI clones you.
Absolutely correct. This is what the AI hype squad and the HN bubble misses again. This is only useful to existing businesses (summarization the only safe use-case) or random devs automating themselves out of irrelevance. All of this 'euphoria' is around for Microsoft's heavy marketing from its newly acquired AI division.
This is a obvious text book example of mindshare capture and ecosystem lock-in. Eventually, OpenAI will just slowly raise prices and break / deprecate older models to move them onto newer ones and pay to continue using them. It is the same decade old tactics.
No, and in fact this actually seems like a more salient excuse for going closed than even "we can charge people to use our API".
If even 10% of the AI hype is real, then OpenAI is poised to Sherlock[0] the entire tech industry.
[0] "Getting Sherlocked" refers to when Apple makes an app that's similar to your utility and then bundles it in the OS, destroying your entire business in the process.
You're thinking too long term. Based on my Twitter feed filled with AI gold rush tweets, the goal is to build something/anything while hype is at its peak, and you can secure a a few hundred k or million in profits before the ground shifts underneath you.
The playbook is obvious now: just build the quickest path to someone giving you money, maybe it's not useful at all! Someone will definitely buy because they don't want to miss out. And don't be too invested because it'll be gone soon anyway, OpenAI will enforce stronger rate limits or prices will become too steep or they'll nerf the API functionality or they'll take your idea and sell it themselves or you may just lose momentum. Repeat when you see the next opportunity.
Either way though, Copilot is certainly a product of the 'current AI wave' that is being compared to crypto scams above.
At home, a carpenter working at my house said that he is using ChatGPT to overcome problems associated with his dyslexia (e.g. when writing descriptions of the services his company offers). I hadn't even considered that use case.
I also find it useful for pasting code and asking, "Do you have any ideas for improvements?"
This revolution is the wrong one if we can't guarantee correctness, or the guarantee that AI will direct the user to where help is available.
I think these tools will help us break out of local bubbles. I'm currently working on a Zeitgeist [1] that tries to gather the consensus on social media and on the web on general.
[1] https://foretale.io/zeitgeist
What does all of this look like to you?
It's like 300 prompts about various sales tools and terms I'd never heard of — even just getting the keywords is enough to set me off on a learning experience now, so love it or hate it, that was actually weirdly useful for me.
(I had ZERO expectations when I clicked to download)
Amazon, Google, and Microsoft cloud analogs.
We are entirely fortunate that the interests of big tech (edge AI) and democratizing AI (we the little people) align to a sufficient degree.
Decentralizing AI is -far- more important than decentralizing communication, imo.
The get rich quick path of ‘gold rush’ (it works, tbh) could work against this collective self interest if it ends up hyping centralized solutions. If you are on the sideline, the least you could do just cheer (hype :) the decentralized, democratized, and freely accessible candidates.
You cannot assume what will happen in Web 2.0, mobile, iPhone, will happen here. Getting to tech maturity is uncertain and no one understands yet where this will go. Only thing you can do is build and learn.
Whan OpenAI is building along with other generative AI is the real Web 3.0.
This seems to be the start of a chatbot as an OS.